This CL improves the performance of below functions in helping with conversion
between utf8/utf16 with libutils:
- utf8_to_utf16_length
- utf8_to_utf16
- utf16_to_utf8_length
- utf16_to_utf
The basic idea is to keep the loop as tight as possible for the most
common cases, e.g. in UTF16-->UTF8 case, the most common case is
when the character is < 0x80 (ASCII), next is when it's < 0x0800 (
most Latin), and so on.
This version of implementation reduces the number of instructions
needed for every incoming utf-8 bytes in the original implementation
where:
1) calculating how many bytes needed given a leading UTF-8 byte
in utf8_codepoint_len(), it's a very clever way but involves
multiple instructions to calculate regardless
2) and an intermediate conversion to utf32, and then to utf16
utf8_to_utf32_codepoint()
The end result is about ~1.5x throughput improvement.
Benchmark results on redfin (64bit) before the change:
utf8_to_utf16_length: bytes_per_second=307.556M/s
utf8_to_utf16: bytes_per_second=246.664M/s
utf16_to_utf8_length: bytes_per_second=482.241M/s
utf16_to_utf8: bytes_per_second=351.376M/s
After the change:
utf8_to_utf16_length: bytes_per_second=544.022M/s
utf8_to_utf16: bytes_per_second=471.135M/s
utf16_to_utf8_length: bytes_per_second=685.381M/s
utf16_to_utf8: bytes_per_second=580.004M/s
Ideas for future improvement could include alignment handling and loop
unrolling to increase throughput more.
This CL also fixes issues below:
1. utf16_to_utf8_length() should return 0 when the source string has
length of 0, the original code returns -1 as below:
ssize_t utf16_to_utf8_length(const char16_t *src, size_t src_len)
{
if (src == nullptr || src_len == 0) {
return -1;
}
...
2. utf8_to_utf16() should check whether input string is valid.
Change-Id: I546138a7a8050681a524eabce9864219fc44f48e
It could never have gotten much coverage.
Bug: 288741501
Test: libutils_fuzz_vector (2,000,000 iterations)
(~60k-100k iterations/s)
Change-Id: I6f442642b5a3246dd08784f735db5aad5fd4d398
The default initalization for mNullValue uses 0 which is in the case of
a std::string TValue will invoke the contructor with undefined behavior
parameter.
Using an empty uniform initialization {} addresses the problem.
Test: Already tested in lrucache_test.cpp
Bug: 257127748
Change-Id: I37420ce8a16c99f3014538a0208d7e113870b1c7
`stderror` is used in this file and it is defined in `string.h/cstring`, so
the header file needs to be directly included.
Test: build
Bug: 285204695
Change-Id: Idf34126626ad0e9bb397df3d5da50439bf18381d
The RacingDestructors test was occasionally timing out, by non-huge
amounts. Halve the number of iterations for long-running tests here.
Bug: 284964396
Test: Treehugger
Change-Id: If639ce98fbeb783431f07006dece4cd69f9f8b50
The ABI dumps add visibility for API changes in a commonly used library.
Test: development/vndk/tools/header-checker/utils/create_reference_dumps.py \
-libs libutils -products aosp_arm64 \
-ref-dump-dir system/core/libutils/abi-dumps
Test: m libutils.vendor
Bug: 227282691
Change-Id: I498c62853562a8fc6702bdd622603b6f7d516274
This CL added additional tests for converting between utf16 and utf8,
specifically tests that:
1. check utf16_to_utf8_length() returns 0 if input is an
empty UTF16 string
2. check utf16_to_utf8_length() returns 1 if input is a
single ASCII character UTF16 string
3. check utf16_to_utf8_length() returns 3 if input is a
single UTF-16 character between U+0800 - U+FFFF
4. check utf16_to_utf8_length() returns 4 if input has
a surrogate pair
5. check unpaired UTF-16 surrogate is handled correctly
(skipped)
6. check utf16_to_utf8_length(0 handles invalid surrogate
case correctly, by skipping the first but handling the
rest correctly
7. check a normal string with a mix of 1/2/3/4-byte UTF8
characters is correctly converted by utf16_to_utf8()
8. check conversion from invalid utf8 sequence with invalid
leading byte and/or invalid trailing byte(s) should still
work and not crash
Change-Id: If68e514af0e84ddebf5900b2e140e76ba4f44553
This includes memory header for shared_ptr and unique_ptr templates.
Fixes the following errors:
RefBase.h:803:1: error: ‘shared_ptr’ does not name a type
RefBase.h:810:1: error: ‘unique_ptr’ does not name a type
Change-Id: I6a7a67333c9ef05250c5a3c6199d7fac288f946b
This fixes the following error:
core/libutils/String8.cpp:316:67: error: incomplete type 'std::numeric_limits<long long unsigned int>' used in nested name specifier
316 | if (static_cast<size_t>(n) > std::numeric_limits<size_t>::max() - 1 ||
| ^~~
Change-Id: I80cdd2442e2798c37e066e3fdeee7dd5ac34b669
Done here:
- path saved to a useable location on host
- path always printed (for visibility for new users)
- open(.., 666) - That's '01204' not '0644' oops
Future considerations:
- make defines constexpr instead of ifdef
- copy malloc hook design to avoid needing to recompile code
- make libutilscallstack default on host
- run libutils tests, not just test compile debug mode
- code incorrectly prints ref 'doesn't exist'
seems it's gotten out of sync, but still good enough
to use stacktraces
Bug: 244325464
Test: manual
Change-Id: I732e5b8aec3cd946ef3559a2a814caf693846cc0
I've seen these fixed before, so we compile the debug modes
now.
Bug: 244325464
Test: 'm libutils_test_compile'
Change-Id: I4271909e81893ad448bc46b8a3a567a84c40f8a3
Unify all our "noinline" variants to the current most common one, not
least because the new [[noinline]] syntax is fussier about where it goes.
Test: treehugger
Change-Id: Icfcb75c9d687f0f05c19f66ee778fd8962519436
The CallStack unwind of a different thread was broken since it
wasn't properly setting the tid value.
Fix this problem and add new unit tests to verify the behavior.
Bug: 246405269
Test: New unit tests pass.
Test: Ran unit tests for 1000 operations to verify not flaky.
Change-Id: I00342e6cdcdb4bcb68f29734dadee6c987c98040
The function objects work equally well without them, and the base
classes were wrong for both types:
* HashForEntry: returns size_t but declared to return hash_t
(uint32_t)
* EqualityForHashedEntries: returns bool and takes two parameters but
declared to return hash_t and take one parameter
std::unary_function was deprecated in C++11 and removed in C++17.
Upstream libc++ now removes the type for new-enough C++ dialects.
Bug: http://b/175635923
Test: treehugger
Change-Id: I2ff15c5da6a4e4f71df08c243f8af2f11d8d2b0d
Surprised this isn't breaking anything, so wanted to
make sure it worked.
Bug: 232557259
Test: libutils_test
Change-Id: Iaec47d644c02dc190e397c6f84dcfab4cc76f566
Before, we only did this in sp<> constructors, but we can always make
this check during the initial incStrong.
Since prebuilts call into the existing report race function declared
in StrongPointer.h, we still call this function from RefBase.cpp.
Bug: 232557259
Test: libutils_test
Change-Id: I4080b1869b83ecf655fc9c182b6de768a6358adf
The diff_flags make the ABI checker ignore the weak symbol difference
caused by PGO.
Ignore-AOSP-First: This patch relaxes the ABI check for T only.
b/232982219 tracks the long-term solution.
Test: make libutils.vendor
Bug: 230076879
Change-Id: I8ec2c0f5a540263b4e8a0a4f9cf26f9c297593b5
Replaces libbacktrace in CallStack. There is one small behavioral
change, the BuildId data is added to the unwinds.
Bug: 120606663
Test: All unit tests pass.
Test: Run the fuzzer for over an hour without any crashes.
Change-Id: Ic8a4247c515ce0d3cdc4d2cc15167d1948b15fa5
Let's turn a bug into a feature... Since this code is built with intsan,
anyone who caused overflow here will have had an abort, so we know
no-one actually needs the BAD_INDEX return that was presumably the
original author's intent. So let's just mandate that, since it's a lot
harder to ignore an abort than it is to ignore an error return.
Bug: http://b/179044558
Test: treehugger
Change-Id: I08f1018f9da1e09de885699138b7543d55bb2a36
(cherry picked from commit a5f2e4d421)
Merged-In: I08f1018f9da1e09de885699138b7543d55bb2a36
Let's turn a bug into a feature... Since this code is built with intsan,
anyone who caused overflow here will have had an abort, so we know
no-one actually needs the BAD_INDEX return that was presumably the
original author's intent. So let's just mandate that, since it's a lot
harder to ignore an abort than it is to ignore an error return.
Bug: http://b/179044558
Test: treehugger
Change-Id: I08f1018f9da1e09de885699138b7543d55bb2a36
Mark this as 29 for easy inclusion in APEXes when debugging. Make
sure we don't introduce calls to things here which would prevent
this from being used on old platforms.
Fixes: 228561718
Test: build
Change-Id: I2574455dbfe681117e4afcf6eef3546be51393fa
The callback can be called by the destructor of LruCache, so it needs to be destructed last.
Bug: 227635615
Change-Id: I7d965a2000c1ec32d9c9e88d25ab0c1ba3e9c739
