Add a hook that's called upon file descriptor creation to libc, and a
library that uses it to capture backtraces for file descriptor creation,
to make it easier to hunt down file descriptor leaks.
Currently, this doesn't capture all of the ways of creating a file
descriptor, but completeness isn't required for this to be useful as
long as leaked file descriptors are created with a function that is
tracked. The primary unhandled case is binder, which receives file
descriptors as a payload in a not-trivially-parsable byte blob, but
there's a chance that the leak we're currently trying to track down
isn't of a file descriptor received over binder, so leave that for
later.
Bug: http://b/140703823
Test: manual
Change-Id: I308a14c2e234cdba4207157b634ab6b8bc539dd9
(cherry picked from commit b7eccd4b15)
mte_supported() lets code efficiently detect the presence of MTE, and
ScopedDisableMTE lets code disable MTE RAII-style in a particular region
of code.
Bug: 135772972
Change-Id: I628a054b50d79f67f39f35d44232b7a2ae166afb
On Android, fcntl is always implemented by fcntl64(2). This means that
an LP32 binary can `fcntl(F_SETLK, struct flock)` (because fcntl64(2)
passes through to the 32-bit fcntl(2) to handle F_SETLK), and it can
also `fcntl(F_SETLK64, struct flock64)`. What it can't do before this
patch is set _FILE_OFFSET_BITS=64 and then `fcntl(F_SETLK, struct
flock)` where that `struct flock` is actually implicitly `struct
flock64`.
Move the kernel uapi structs out of the way, define them ourselves based
on __LP64__ and _FILE_OFFSET_BITS, and fix up the relevant F_ constants.
(Also add a .clang-format to turn off clang-format in libc/include/.)
Bug: N/A
Test: treehugger (and strace!)
Change-Id: Iccd6c83d9133e1efcf93a7b49a6ae0f1bbd3d58b
The alloc after fork is a test that should pass, but jemalloc
doesn't right now. Leave the test disabled until the native
allocator can pass this.
Test: Ran the test 1000 times on glibc to verify it passes.
Test: On device, verified it does not run.
Change-Id: I482af4db2fee81c947ac081c7a6f25a2aff80350
Test (a) that we can load the library, but also (b) that readelf thinks
it contains the relocation encoding we were expecting. Do this for all
four of RELR, ANDROID_RELR, relocation packer, and the original ELF
relocation encoding.
Bug: http://b/147452927
Test: treehugger
Change-Id: I5fab72f99d46991c1b206a1c15c76e185b7148b3
Until now we've only supported RELR with our own OS-private-use
constants. Add support for the official numbers (while maintaining
support for the historical numbers).
Add tests to ensure we continue to support both indefinitely.
We can't yet flip the build system over to using the official constants
because the old GNU binutils objcopy we still use in most cases (for the
mini-debug section) only supports the historical constants.
Bug: http://b/147452927
Test: treehugger
Change-Id: If214fce7fade4316115947e90b78ab40864b61f2
This patch introduces tagged pointers to bionic. We add a static tag to
all pointers on arm64 compatible platforms (needs requisite
top-byte-ignore hardware feature and relevant kernel patches).
We dynamically detect TBI-compatible devices (a device with the TBI feature and
kernel support) at process start time, and insert an implementation-dependent
tag into the top byte of the pointer for all heap allocations. We then check
that the tag has not been truncated when deallocating the memory.
If an application incorrectly writes to the top byte of the pointer, we
terminate the process at time of detection. This will allow MTE-incompatible
applications to be caught early.
Bug: 135754954
Bug: 147147490
Test: cd bionic && atest .
Change-Id: Ie424325ba1e3c4443040ac265aeaa28d9e405d28
setprogname() does a basename, but we were initializing __progname
directly. Stop doing that, and add some tests.
Test: treehugger
Change-Id: I06f306ade4161b2f0c7e314a3b1b30c9420117b7
Use O_PATH like musl to let the kernel do the hard work, rather than the
traditional BSD manual scheme.
Also add the most obvious missing tests from reading the man page, plus
a non-obvious test for deleted files.
Bug: http://b/131435126
Test: treehugger
Change-Id: Ie8a8986fea55f045952a81afee377ce8288a49d5
This reverts commit 43d5f9d4dd.
Bug: 135754954
Bug: 147147490
Exempt-From-Owner-Approval: clean revert
Reason for revert: Breaks ART gtest, see:
https://ci.chromium.org/p/art/builders/ci/angler-armv8-non-gen-cc/561
The crash happens on mprotect of a page, the test crashes with ENOMEM.
Change-Id: I52eea1abbfaf8d8e2226f92d30aa55aba3810528
Bug: 122332847
Bug: 130734182
Test: mmm bionic/tests; same files installed before & after
Test: m cts; same files in cts before & after
Test: `objdump -x <files> | grep NEEDED` is identical
Test: `objdump -x <files> | grep RUNPATH` difference make sense
(additional lib64 entries from soong, minimal reorder)
Test: treehugger
Change-Id: I6eea4c3f9c0040efeba64e2b47c7b573767cd386
This patch introduces tagged pointers to bionic. We add a static tag to
all pointers on arm64 compatible platforms (needs requisite
top-byte-ignore hardware feature and relevant kernel patches).
We dynamically detect TBI-compatible devices (a device with the TBI feature and
kernel support) at process start time, and insert an implementation-dependent
tag into the top byte of the pointer for all heap allocations. We then check
that the tag has not been truncated when deallocating the memory.
If an application incorrectly writes to the top byte of the pointer, we
terminate the process at time of detection. This will allow MTE-incompatible
applications to be caught early.
Bug: 135754954
Bug: 147147490
Test: cd bionic && atest .
Change-Id: I6e5b809fc81f55dd517f845eaf20f3c0ebd4d86e
This patch refactors heapprofd_malloc to make it easier to reuse the
reserved signal for multiple purposes. We define a new generic signal
handler for profilers, which dispatches to more specific logic based on
the signal's payload (si_value).
The profiler signal handler is installed during libc preinit, after
malloc initialization (so races against synchronous heapprofd
initialization need not be considered). In terms of code organization, I
copied the existing approach with a loosely referenced function in
bionic_globals.h. Do tell if you'd rather a different approach here.
The profileability of a process is quite tied to the malloc
files/interfaces in bionic - in particular, it's set through
android_mallopt. I do not change that, but instead introduce a new
android_mallopt option to be able to query profileability of the
process (which is now used by the new profiler signal handler). As part
of that, gZygoteChildProfileable is moved from heapprofd_malloc to
common (alongside gZygoteChild).
I've removed the masking and reraising of the heapprofd signal when
racing against malloc_limit init. We're ok with taking a simpler
approach and dropping the heapprofd signal in such an unlikely race.
Note: this requires a corresponding change in heapprofd to use sigqueue()
instead of kill(), as the latter leaves the si_value uninitialized(?) on
the receiving side.
Bug: 144281346
Change-Id: I93bb2e82cff5870e5ca499cf86439860aca9dfa5
Bug: 146576216
Test: Ran the test on jemalloc and glibc.
Test: Ran the test on scudo and verified that failed without the align
Test: change.
Change-Id: I31a96f8672c6bce2423210300288a13df1eda42a
Before, the helper method for this test had two local variables:
char buf[128];
volatile char* p;
Then the test wrote zeros into the buffer and one past the buffer end.
This relied on the fact that the compiler constructed the stack frame
with 'p' first and then the buffer (and also optimized away the 'size'
variable).
However, some compiler options (namely -ftrivial-auto-var-init=pattern)
result in the stack frame being reordered so that 'p' is actually after
buf, and the test cannot pass.
Fixes: 132780819
Test: bionic-unit-tests-static (w/ w/o flag)
Change-Id: Icc87c02add211c2afb7c96ae22701ec27990364c
Historically we've made a few mistakes where they haven't matched the
right number. And most non-Googlers are much more familiar with the
numbers, so it seems to make sense to rely more on them. Especially in
header files, which we actually expect real people to have to read from
time to time.
Test: treehugger
Change-Id: I0d4a97454ee108de1d32f21df285315c5488d886
The executable flag might not be present for the executable segment
if the code is being handled by native bridge implementation. Old
heuristics for this case was to assume first segment is executable.
This doesn't stand any more. Instead, look for segment that contains
the function test is going to call.
Test: bionic-unit-tests --gtest_filter=dlext.ns_anonymous
Change-Id: I960bebba86e225d739b73f6d093e145fff17b4a8
'kill(pid, sig)' sends signal to arbitrary thread within a process, thus
test was occasionally sending signal to the killing thread, not to the
waiting thread. Use 'tgkill(pid, tid, sig)' instead.
Test: bionic-unit-tests --gtest_filter=signal.sigwait64_SIGRTMIN
Change-Id: I80377295e2362cb87eb4fb4de2489c51c520ea77
With previous sleep time, if unlucky, newly started thread can send
signal before main thread starts sigwait.
Test: bionic-unit-tests --gtest_filter=signal.sigwait64_SIGRTMIN
Change-Id: I63d7825a695988e388903c7951e2435cb69773a1
The code was checking PTRACE_GETREGSET output even in case of error.
This seems not correct. Though ptrace can still write some output in
case of EINVAL, implementation should still check updated iov_len before
reading it.
Change the code to avoid checking output in case of error at all.
Test: bionic-unit-tests --gtest_filter=sys_ptrace.watchpoint_stress
Change-Id: I7b1ca18ac64f81055ff89f56b453aff0ce8e1057
dl#exec_linker* tests are failing on devices with emulated
architecture due to hardcoded path to linker.
Test: bionic-unit-tests --gtest_filter=dl.exec_linker*
Bug: b/141914915
Change-Id: Id6d8d3ee7114e70b07e44034aa62dce0a3e0760e
Signed-off-by: Dmytro Chystiakov <dmytro.chystiakov@intel.com>
In configs like ASAN, we can't use _chk functions. This CL builds off of
previous work to allow us to still emit diagnostics in conditions like
these.
Wasn't 100% sure what a good test story would look like here. Opinions
appreciated.
Bug: 141267932
Test: checkbuild on internal-master. TreeHugger for x86_64.
Change-Id: I65da9ecc9903d51a09f740e38ab413b9beaeed88
For reasons explained in the code comment, go back to roughly our old
code. The "new" tests are just the old tests resurrected.
This also passes the current toybox xargs tests, which were the
motivation for going back on our earlier decision.
Test: bionic and toybox tests
Change-Id: I33cbcc04107efe81fdbc8166dc9ae844e471173e
Right now, when we read a system property, we first (assuming we've
already looked up the property's prop_info) read the property's serial
number; if we find that the low bit (the dirty bit) in the serial
number is set, we futex-wait for that serial number to become
non-dirty. By doing so, we spare readers from seeing partially-updated
property values if they race with the property service's non-atomic
memcpy to the property value slot. (The futex-wait here isn't
essential to the algorithm: spinning while dirty would suffice,
although it'd be somewhat less efficient.)
The problem with this approach is that readers can wait on the
property service process, potentially causing delays due to scheduling
variance. Property reads are not guaranteed to complete in finite time
right now.
This change makes property reads wait-free and ensures that they
complete in finite time in all cases. In the new approach, we prevent
value tearing by backing up each property we're about to modify and
directing readers to the backup copy if they try to read a property
with the dirty bit set.
(The wait freedom is limited to the case of readers racing against
*one* property update. A writer can still delay readers by rapidly
updating a property --- but after this change, readers can't hang due
to PID 1 scheduling delays.)
I considered adding explicit atomic access to short property values,
but between binary compatibility with the existing property database
and the need to carefully handle transitions of property values
between "short" (compatible with atomics) and "long" (incompatible
with atomics) length domains, I figured the complexity wasn't worth it
and that making property reads wait-free would be adequate.
Test: boots
Bug: 143561649
Change-Id: Ifd3108aedba5a4b157b66af6ca0a4ed084bd5982
This reverts commit d7e11b8853.
Reason for revert: Breaks aosp_x86_64-eng. Will look into it and
unbreak when it's not almost midnight. :)
Change-Id: I21f76efe4d19c70d0b14630e441376d359a45b49
When using a FILE object for some malloc debug functions, calling
fprintf will trigger an allocation to be put in the object. The problem
is that these allocations were not allocated by the malloc debug
wrapper and they get freed during the fclose as if they are malloc
debug allocation. In most cases, the code will detect the bad pointer
and leak the memory, but it might also cause a crash.
The fix is to avoid using fprintf so that no allocations are made
in the object that survive and need to be freed in the fclose call.
Change the MallocXmlElem.h to use a file decsriptor not a FILE object.
Add new unit and system tests to detect this case.
Bug: 143742907
Test: Ran unit and system tests.
Test: Ran bionic unit tests.
Change-Id: I524392de822a29483aa5be8f14c680e70033eba2
This change makes it easier to diagnose mistakes in linker
configuration that result in a library being accidentally loaded in
multiple namespaces without its dependencies available everywhere.
Test: manually tested the error message
Test: bionic-unit-tests
Change-Id: I03a20507f8fc902c2445a7fbbf59767ffffd5ebf
In configs like ASAN, we can't use _chk functions. This CL builds off of
previous work to allow us to still emit diagnostics in conditions like
these.
Wasn't 100% sure what a good test story would look like here. Opinions
appreciated.
Bug: 141267932
Test: checkbuild on internal-master
Change-Id: I8d4f77d7b086a8128a18a0a0389243d7fa05b00f
Upstream keeps rearranging the deckchairs for these, so let's just
switch to the [roughly] one-liners rather than track that...
Test: treehugger
Change-Id: If655cf7a7f316657de44d41fadd43a8c55ee6f23
In order for an ifunc resolver to detect the presence of certain CPU features,
access to getauxval(AT_HWCAP) or getauxval(AT_HWCAP2) may be required. In order
for getauxval() to work, it needs to access the pointer to the auxiliary vector
stored by the linker in the libc shared globals data structure. Accessing the
shared globals requires libc to call the __libc_shared_globals() function
exported by the linker. However, in order to call this function, libc must
be fully relocated, which is not guaranteed to be the case at the point when
ifunc resolvers are called.
glibc solves this problem by passing the values of getauxval(AT_HWCAP)
(and getauxval(AT_HWCAP2) on aarch64) as arguments to the ifunc resolver.
Since this seems to be not only the most straightforward way to solve the
problem but also improves our compatibility with glibc, we adopt their
calling convention.
This change is ABI compatible with old resolvers because the arguments are
passed in registers, so the old resolvers will simply ignore the new arguments.
Bug: 135772972
Change-Id: Ie65bd6e7067f0c878df3d348c815fda61dc12de2
This patch decreases created threads to 50 (instead of 90)
on devices with 2 cores CPU. It is a second decrease as this
test case started to fail on same devices after kernel uprev
Bug: b/142210680
Test: Run CtsBionic module on 2 core device with command
"run cts -m CtsBionicTestCases
Change-Id: I4f73363b342b38c92d64aaf213071f899e5b52ac
Signed-off-by: Dmytro Chystiakov <dmytro.chystiakov@intel.com>
I plan to add tests to this test binary that take a long time to run.
The first is a malloc test that demonstrates that jemalloc appears
to be leaking memory slowly over time.
Test: Ran the bionic-stress-tests on host and verified it runs forever,
Test: but doesn't display RSS/VSS data.
Test: Ran the bionic-stress-tests on a taimen and verified it runs forever,
Test: and does display RSS/VSS data.
Change-Id: Ic612e4181ffea898d4d83af097939cd517a180ee
Scudo creates a large map on 64 bit which can cause a slow down trying
to get the usage stats for all maps. Since the test only really cares
about a small subset of maps, only get the usage stats for those maps.
Test: Ran unit tests on scudo and jemalloc based systems.
Change-Id: Iba3cff1487ca304083aac323a3971b9f939f5c11
As the first pass, the malloc_info for scudo doesn't return anything.
This feature is hardly ever used, so this seems fine for the first
revision.
Bug: 137795072
Test: Ran unit tests on a taimen using scudo and a taimen using jemalloc.
Change-Id: I675386ae17ba5505c27bd9da12c60242d795464a
There are places in frameworks and art code that directly included
private bionic header files. Move these files to the new platform
include files.
This change also moves the __get_tls.h header file to tls.h and includes
the tls defines header so that there is a single header that platform
code can use to get __get_tls and the defines.
Also, simplify the visibility rules for platform includes.
Bug: 141560639
Test: Builds and bionic unit tests pass.
Change-Id: I9e5e9c33fe8a85260f69823468bc9d340ab7a1f9
Merged-In: I9e5e9c33fe8a85260f69823468bc9d340ab7a1f9
(cherry picked from commit 44631c919a)
A kernel update broke three clock_getres tests, so disable
them while the kernel is fixed.
Bug: 141515847
Test: All tests pass on cuttlefish.
Change-Id: I7db789b2b8ba0bc5f8b13bb06e85711031735925
This was necessary because we weren't doing -Werror=fortify-source
everywhere. As of I2715ea411ef067f801534dab4d306fef5431f290, we should
be doing this.
Bug: 131861088
Test: mma
Change-Id: I00d499938ae24c3870760470576d6be603fbecd1
Clang recently grew its own diagnostics for memcpy and such. These are
generally higher-quality than what we can do with diagnose_if, since
clang is happy to include e.g., sizes of things per-callsite. Move to
those instead where applicable.
Bug: 131861088, 123644155
Test: blueline internal-master checkbuild; treehugger
Change-Id: I701f5a8b247ba2948ca47fdc60ff5198b564c03e
Instead of having platform directories directly include the
private header, create a platform header directory and export it.
Bug: 130763340
Test: Builds.
Change-Id: Ie0f092b3fe077a3de8b90266c0b28bfbc20d0dfa
Merged-In: Ie0f092b3fe077a3de8b90266c0b28bfbc20d0dfa
(cherry picked from commit 8f582ef2f8)
Add a test for the new flag and add a test for the EFD_CLOEXEC flag.
Test: New unit tests pass on glibc and target.
Change-Id: Ib7a6ea4aadbd67ba8a523b6114a49fb8d6a43f12
This avoids issues where an allocation could create a map while trying
to check data from the maps and running malloc_iterate.
Bug: 137795072
Test: Runs in unit tests.
Change-Id: If8509845d86dd9d002aeac3aa9278fbcf026af17
Previously, Bionic's dl_phdr_info only included the first four
dl_iterate_phdr fields. Several other libc's have these additional fields:
unsigned long long dlpi_adds -- incremented when a library is loaded
unsigned long long dlpi_subs -- incremented when a library is unloaded
size_t dlpi_tls_modid -- TLS module ID
void* dlpi_tls_data -- pointer to current thread's TLS block or NULL
These extra fields are also exposed by glibc, musl, and FreeBSD. The
unwinder in libgcc.a, linked into shipping Android DSOs, has a
PC->eh_frame cache that activates if dl_phdr_info has the dlpi_adds and
dlpi_subs fields (indicated at run-time by a sufficiently-large size
argument to the callback).
Bug: https://github.com/android-ndk/ndk/issues/1062
Test: bionic unit tests
Change-Id: I6f0bab548cf8c828af2ddab9eb01c5c6d70cd81f
The test starts failing in the upcoming lld update. The root cause is
that no relro section is specified by the linker script, and the orphan
placement algorithm decides to place them after .text which is a
misaligned place.
Test: bionic-unit-tests
Bug: 139089152
Bug: 131328001
Change-Id: I2140d8a9fd7c712f1d31a541a3b4a10788385e98
The call to the load hook needs to be moved before the call to link_image()
because the latter calls ifunc resolvers which might access global
variables. This fixes a bunch of ifunc tests.
The dlfcn.segment_gap test is currently failing. One problem is that the name
of the .bss.end_of_gap section changes as a result of global instrumentation.
Add some wildcards in so that we match both names. The other problem seems
to be the same as b/139089152.
It turns out that we need to untag pointers in a few more places. Since we have
quite a few of these now it seems worth creating a function for it.
Test: bionic-unit-tests
Change-Id: I44e2b0904faacdda7cc0c5e844ffc09de01dea2d
Host includes are updated now, so we always must include
bits/stdatomic.h
Bug: 137876753
Bug: 138701943
Test: not yet
Change-Id: Ib2ce34ae5dace8f9a114fa8c04f864cd7f41a5d9
If one of our threads exits and a new process shows up with the same
tid, we would previously loop until that process exits, leading to
flakiness in tests.
Also fix use of uninitialized memory that happens when threads_count !=
kThreadCount.
Bug: http://b/139315254
Test: /data/nativetest/bionic-unit-tests/bionic-unit-tests --gtest_filter="pthread_leak.detach" on cuttlefish
Change-Id: I2aad7f3f105aeba87c3929bfce53ef08e1484b2d
These are old enough now that the latest devices will have kernels that
support them.
Also add basic doc comments to <sys/mman.h>.
Test: treehugger
Change-Id: I1b5ff5db0b6270f5c374287cac1d6a751a0259f5
