Adding the new option log_allocator_stats_on_signal that will call
mallopt(M_LOG_STATS, 0), when a signal is sent to a process.
This call does not happen in the signal handler, but a variable
is set to trigger the mallopt call. The next time any allocation
call occurs, the mallopt call will be made.
Also, includes new unit tests for the new config option.
Test: All unit tests pass.
Test: Enabling the new feature and stop/start and sending the signal to
Test: a process and observing the log contains the memory stats.
Test: Did the above for a scudo and jemalloc based device.
Change-Id: Idcf6fca74036efb065b9c4cc037aaf9bcf0f139e
This will, hopefully, reduce the number of flaky runs of this test.
Add skipping xml files for the notice file parser.
Bug: 280572235
Test: atest malloc_debug_system_tests
Change-Id: I6fb76287f55d0cff5b695dce09cc2b7a69b62874
Using the long option names might not fit in the malloc debug option
property since properties have a 92 character limit.
This patch creates new aliases for the original options.
Bug: 264504531
Test: set new options pass
Test: Config unit tests pass.
Change-Id: Id985720f36a2bf0da7b35ff444c2c80eb1fb4363
We swap the 2nd and 3rd arguments to the CallocEntry constructor
to match the order in the cpp file, and match the C calloc convention.
We also fix an invocation of this constructor.
Change-Id: Iebe16d82a74459e5e957c1d9e2cc1aebb15150d0
Test: TreeHugger
With timestamps, we are able to tell the details of allocator
performance such as the average time for malloc() in different size
class, the potential contention time by examing the overlap between
operations, .etc.
Not all malloc et al. operations are recorded with timestamp. Only
operations relates to memory usage change will have them.
Test: All unit tests pass.
Change-Id: I5c2016246a6f10b221387001bb44778969bb26ae
There are many cases where there are no more allocations
when a dump of the record allocs might occur. Move the entries
being written to file in the interrupt handler.
Update the unit tests for this new functionality and add a new
unit test that verifies no allocations occur while the entries
are being written.
Fix the unit tests so that the record allocs files get deleted
properly.
Test: All unit tests pass.
Test: Ran 1000 iterations of unit tests to verify no flakes.
Change-Id: I20941596c1dda5a761522050dc155b06f3f3e735
Add backtrace_size for only backtracing a single size.
Add backtrace_min_size to set the minimum value of size to backtrace.
Add backtrace_max_size to set the maximum value of size to backtrace.
Documented the new options.
Test: New unit test pass.
Change-Id: I1a773737910cd4bc2af9546547b3a2740bbcb22b
Modify the tests that require a single filename, to use a filename
that has the pid as part of the name. This allows multiple different
versions of the test to run on the same machine, and allows
each test to be run at the same time.
Test: Ran unit tests on device.
Test: Ran the unit tests 100 times, no failures.
Change-Id: Ia38483049e7b66bd3da824bcd484c03e46f85280
The new object incorporates all Android specific knowledge into
a single place and makes everything simpler.
Fixed a bug where if backtrace_full was enabled, the AddBacktrace
function would always set the size to the maximum number
of frames instead of the actual number of frames.
Added a new smoke system tests for backtrace_full.
Modified the smoke test to do a malloc/free, so it's really
a smoke test.
Bug: 232575330
Test: Unit tests pass on device.
Test: Verify the full backtrace actually produces valid backtraces.
Test: Run bionic-unit-tests with backtrace_full enabled.
Test: Run bionic-benchmarks --benchmark_filter=stdlib_malloc_free_decay1/512
Change-Id: I23128a73a8691007e1c7f69e0c99bb4dcd713db8
The new option is named check_unreachable_on_signal. It is meant
to duplicate dumpsys meminfo --unreachable <PID> for non-java
processes. When enabled, a user can send a signal to a process
which will trigger the unreachable check on the next allocation
call.
Added new unit tests.
Test: New unit tests pass.
Test: Enabled for the entire system, then dumped on the netd
Test: process and also system_server.
Change-Id: I73561b408a947a11ce21a211b065d59fcc39097b
Modify libfdtrack to use the normal Unwinder object. In addition,
update the libfdtrack so that it doesn't record frames in
libfdtrack.so rather than skipping frames it thinks will be in
the library.
Modify the malloc debug code to use the normal Unwinder object.
Bug: 120606663
Test: All unit tests pass.
Change-Id: I3c9612dd10e62389e6219e68045ee87f7b2625f5
strerror is nice, but usually I don't care about the text, I care about
the uppercase enum
Bug: N/A
Test: ./tests/run-on-host.sh glibc (existing failures -> b/201305529)
Test: atest bionic-unit-tests-static
Test: atest malloc_debug_unit_tests
Change-Id: I407bd9f4dfa918fff66a0da7df8d7239f789c7b8
I accidentally made the tests run MAX_RETRIES times instead of
running once when passing, and at most MAX_RETRIES when the
test fails. Also, add a bit of randomness to the usleep to try and
avoid tests syncing up on failures.
Bug: 193898572
Test: Ran unit tests and verified that a pass doesn't result in another run.
Test: Ran three copies of the unit tests at the same time to verify that
Test: there isn't a flaky test failure.
Change-Id: I8b8d3cd05ca7d1e87ce34bf10aeef84f6989fdab
Refactor the code a bit to allow retrying if a log message is missing.
This is because there is a possibility of a log message getting dropped.
If that happens, merely rerun up to three times and pass if the missing
message is present.
Also fix a race condition that can occur if the LogReader threads are
being terminated but happen to be allocating memory while they are
in the signal handler. This situation causes aborts in the memory
allocator or a deadlock. Before this change, the verify_leak*
tests would fail in less than twenty iterations. After, I could run
for hundreds of iterations.
Bug: 193898572
Test: Ran unit tests in a loop.
Change-Id: I0fb5beab6041dcf3e3fd19f6748bb20bb81a2506
These tests can be flaky if something is spamming the log and
expiring messages. Previously, the log wasn't read until the
forked process was complete. Now, two threads are spawned right
after the process forks to read the log while running. This
should avoid any problems if the log is being spammed while the
test runs.
In addition, fixed some potential flakiness in the test that might
occur if the test incorrectly gets stale log data.
Bug: 193898572
Test: All unit tests pass on cuttlefish (both 32 bit and 64 bit).
Test: Forced an abort and verified that the crash log reader sees
Test: the message.
Test: Verified that the log reader for the main and crash log properly
Test: read data.
Test: Verified the test passes while swamping the log by running
Test: the log unit tests while running two copies of the system
Test: tests.
Change-Id: I68bd92a8c483eac0146ada87dd4201dda0902c49
This is a follow-on to commit 5358cc40ab.
I forgot that there were two places the undefined behavior could occur.
Make the literal unsigned in the other place.
Test: TreeHugger
Bug: 189541929
Change-Id: Iaef81507ca61e802d277246bf5995157c93d86ce
Technically, shifting a signed value beyond its maximum range is
undefined behavior in C++. A bare integer literal is signed, and
defaults to 'int'. On platforms where 'int' is 32-bits, we
shift outside this range with 1 << 31.
We make our literal an unsigned integer to avoid this.
Test: TreeHugger
Bug: 189541929
Change-Id: Iade1bcd3f86d025dd6e10c26622d10c26e2c8295
When the main thread is exiting, the code deleted the g_debug global
pointer and destroys the disable pthread key. Unfortunately, if
malloc debug was enabled in a way that requires a header for the pointer,
any frees that occur after the main thread is torn down result in calls
to the underlying allocator with bad pointers.
To avoid this, don't delete the g_debug pointer and don't destroy the
disable pthread key.
Added a new system test that allocates a lot of pointers and frees them
after letting the main thread finish.
Also, fix one test that can fail sporadically due to a lack of unwinding
information on arm32.
Bug: 189541929
Test: Passes new system tests.
Change-Id: I1cfe868987a8f0dc880a5b65de6709f44a5f1988
When malloc debug is enabled, using libbacktrace to unwind can
result in a deadlock. This happens when an unwind of a thread
is occuring which triggers a signal to be sent to that thread. If
that thread is interrupted while a malloc debug function is
executing and owns a lock, that thread is then stuck in the signal
handler. Then the original unwinding thread attempts to do an
allocation and gets stuck waiting for the same malloc debug lock.
This is not a complete deadlock since the unwinder has timeouts,
but it results in truncated unwinds that take at least five
seconds to complete.
Only the backtrace signals needs to be blocked because it is the only
known signal that will result in a thread being paused in a signal
handler.
Also, added a named signal in the reserved signal list for the
special bionic backtrace signal.
Bug: 150833265
Test: New unit tests pass with fix, fail without fix.
Change-Id: If3e41f092ebd40ce62a59ef51d636a91bc31ed80
malloc debug and malloc hooks have been broken for a long time
and no one noticed. So add them to be run by default on bionic
changes since that provides the most coverage.
Change the malloc debug and malloc hooks tests to support isolated
runs.
Changed the name of the malloc hooks unit tests to system tests
because they weren't really unit tests.
Changed the verify leak malloc debug tests to print out extra
information so it is possible to figure out what sized allocation
failed.
Test: Ran tests.
Change-Id: Idea4c864f1d62598148ee78d7c9397e45234b1ca
When using a FILE object for some malloc debug functions, calling
fprintf will trigger an allocation to be put in the object. The problem
is that these allocations were not allocated by the malloc debug
wrapper and they get freed during the fclose as if they are malloc
debug allocation. In most cases, the code will detect the bad pointer
and leak the memory, but it might also cause a crash.
The fix is to avoid using fprintf so that no allocations are made
in the object that survive and need to be freed in the fclose call.
Change the MallocXmlElem.h to use a file decsriptor not a FILE object.
Add new unit and system tests to detect this case.
Bug: 143742907
Test: Ran unit and system tests.
Test: Ran bionic unit tests.
Change-Id: I524392de822a29483aa5be8f14c680e70033eba2
Instead of having platform directories directly include the
private header, create a platform header directory and export it.
Bug: 130763340
Test: Builds.
Change-Id: Ie0f092b3fe077a3de8b90266c0b28bfbc20d0dfa
Merged-In: Ie0f092b3fe077a3de8b90266c0b28bfbc20d0dfa
(cherry picked from commit 8f582ef2f8)
I wrote a new unit test that would fail on the old version of the
code.
On a walleye big cpu, this costs about 40ns-50ns (going from ~430ns to ~480ns).
I think this is an acceptable performance degradation.
Bug: 131867816
Test: New unit tests pass.
Change-Id: I4c0f4373fb0694bf29c3824dbb1224a8a17e211e
Remove this global variable and change the setting of it to non-zero
to a call to android_mallopt.
In addition, change the initialize function to use pass a bool* instead of
int*.
Bug: 130028357
Test: Ran malloc_debug/malloc_hooks/perfetto tests.
Change-Id: I20d382bdeaaf38aac6b9dcabea5b3dfab3c945f6
Merged-In: I20d382bdeaaf38aac6b9dcabea5b3dfab3c945f6
(cherry picked from commit 5225b342f0)
Add a new option verbose for malloc debug that is not enabled by default.
This disables all of the info log messages. It turns out these log
messages can add a measurable amount of time and can change the boot up.
Bug: 129239269
Test: Adjusted unit tests pass.
Test: Verified no messages unless verbose option used.
Change-Id: I805cb7c8ecb44de88119574e59d784877cacc383
Introduce an M_SET_ALLOCATION_LIMIT enumerator for android_mallopt(),
which can be used to set an upper bound on the total size of all
allocations made using the memory allocation APIs.
This is useful for programs such as audioextractor and mediaserver
which need to set such a limit as a security mitigation. Currently
these programs are using setrlimit(RLIMIT_AS) which isn't exactly
what these programs want to control. RLIMIT_AS is also problematic
under sanitizers which allocate large amounts of address space as
shadow memory, and is especially problematic under shadow call stack,
which requires 16MB of address space per thread.
Add new unit tests for bionic.
Add new unit tests for malloc debug that verify that when the limit
is enabled, malloc debug still functions for nearly every allocation
function.
Bug: 118642754
Test: Ran bionic-unit-tests/bionic-unit-tests-static.
Test: Ran malloc debug tests and perfetto integration tests.
Change-Id: I735403c4d2c87f00fb2cdef81d00af0af446b2bb
malloc_info needs to be per native allocator, but the code treated it
like a global function that doesn't depend on the native memory allocator.
Update malloc debug to dump the actual pointers that it has been tracking.
Test: bionic-unit-tests pass.
Test: malloc debug tests pass.
Test: malloc hook tests pass.
Change-Id: I3b0d4d748489dd84c16d16933479dc8b8d79013e
Merged-In: I3b0d4d748489dd84c16d16933479dc8b8d79013e
(cherry picked from commit a3656a98b1)
Jemalloc does not verify that the size parameter is a multiple of
alignment. Fix this since it only went into P.
Fix the unit tests, and fix malloc debug/malloc hooks to handle this
new restrictive behavior.
Bug: 126944692
Test: Ran bionic unit tests.
Test: Ran bionic unit tests with malloc hooks enabled (no new tests fail).
Test: Ran bionic unit tests with malloc debug enabled (no new tests fail).
Test: Ran malloc debug unit tests.
Change-Id: I4d50785928815679c781ca729f998454d76b9192
When free_track option is enabled and malloc debug detects error in
VerifyFreedPointer flow, if freed pointer's usable_size is more than
g_debug->config().fill_on_free_bytes(), the error log is not correct.
The max. bytes printed to error message should be the max bytes to
cmp, not usable size.
Bug: 124420174
Test: build pass and test pass
Change-Id: I41f35ab3330e49e0a6ad276d405bf4f6c3f0ea92
This new option causes an abort after malloc debug detects an error.
This allows vendors to get process coredumps to analyze memory for
corruption.
Bug: 123009873
Test: New test cases added for unit tests and config tests.
Change-Id: I6b480af7f747d6a82f61e8bf3df204a5f7ba017f
There is a hang when enabling leak_track since the dumping of the
leak data can wind up doing an allocation.
Add new system unit test to make sure this doesn't happen again.
Bug: 111146059
Test: Test program that leaks does not hang forever.
Test: Unit tests pass.
Change-Id: Icf99be58ba5db98ee124a471b957a086045f5870
Bump the version from v1.1 to v1.2 and add a build fingerprint line.
Update the heap dump documentation to match the new format and reflect
what made it in P and what made it in Q.
Update the unit tests for this change.
Add -O0 to unit test code to make it easier to debug.
Add an external function that can be used by the framework code
so that there is only one way to dump the heap.
Bug: 110095681
Test: Ran unit tests.
Test: Did a dump of a real process and verified fingerprint.
Test: Did a dump of a process without malloc debug enabled.
Change-Id: I769a476cbeaf4c85c5d75bd6d6385f0e3add948c
Merged-In: I769a476cbeaf4c85c5d75bd6d6385f0e3add948c
(cherry picked from commit c84a2a2601)
This adds a new option backtrace_full, when it is set, then it will use
libunwindstack.
Modify the dump to file data to dump the extra information from libunwindstack.
Along with the new dump file format, change the version to v1.1.
Updated document for new format of file data.
Add unit tests for the new functionality.
Bug: 74361929
Test: Ran unit tests.
Change-Id: I40fff795f5346bba7b9d7fde2e04f269ff4eb7f1
Changes
- Refactor the code so that only guards require creating a special header
for every pointer allocated.
- Store only a single copy of every backtrace. This saves memory so that
turning on the backtrace option doesn't result in 10X memory usage.
- Added new option track_allocs that only verifies pointers are valid for
free/malloc_usable_size/realloc.
- Remove suffix from test names.
- Add the TRACK_ALLOCS options to all guard options.
- Add new option verify_pointers that is a lightweight way to verify
pointers that are passed to allocation routines.
- Do auto-formatting of the code.
- Updated documentation for all of these changes.
Bug: 74361929
Test: Ran unit tests.
Test: Ran libmemunreachable unit tests.
Test: Ran an app with backtrace enabled.
Change-Id: I3246c48ae4f9811f64622d90d0a9b4d9d818702c
Strictly not needed in the WNOHANG case, but it's probably best to have
every waitpid wrapped for future copy & pasters.
Bug: https://issuetracker.google.com/69525592
Test: ran tests
Change-Id: I013b0a52d2753e3d32638e9b84c79af7327fb405
While this was never an inline, this function alone has caused most of
the bug reports related to _FILE_OFFSET_BITS=64. Providing an inline
for it should allow a lot more code to build with _FILE_OFFSET_BITS=64
when targeting pre-L.
Test: make checkbuild
Test: built trivial cc_binary for LP32 against API 14 with
_FILE_OFFSET_BITS=64 set
Bug: lots
Change-Id: I8479d34af4da358c11423bee43d45b59e9d4143e
