Rename LinkerMemoryAllocator -> BionicAllocator
Rename LinkerSmallObjectAllocator -> BionicSmallObjectAllocator
libc and the linker need to share an instance of the allocator for
allocating and freeing dynamic ELF TLS memory (DTVs and segments). The
linker also continues to use this allocator.
Bug: http://b/78026329
Test: /data/nativetest/bionic-unit-tests-static
Test: /data/nativetest64/bionic-unit-tests-static
Test: /data/nativetest/linker-unit-tests/linker-unit-tests32
Test: /data/nativetest64/linker-unit-tests/linker-unit-tests64
Change-Id: I2da037006ddf8041a75f3eba2071a8fcdcc223ce
If a signal handler is blocking all of their signals, we should
probably respect that and not silently unblock bionic's reserved
signals for them. Otherwise, user code can deadlock, run out of stack,
etc. through no fault of their own, if one of the reserved signals
comes in while they've pivoted onto their signal stack.
Bug: http://b/122939726
Test: treehugger
Change-Id: I6425a3e7413edc16157b35dffe632e1ab1d76618
On user builds, heapprofd should only be allowed to profile apps that
are either debuggable, or profileable (according to the manifest). This
change exposes extra zygote-specific knowledge to bionic, and makes the
dedicated signal handler check for the special case of being in a zygote child.
With this & the corresponding framework change, we should now be
handling the 4 combinations of:
{java, native} x {profile_at_runtime, profile_at_startup}.
See internal go/heapprofd-java-trigger for further context.
Test: on-device unit tests (shared & static) on blueline-userdebug.
Test: flashed blueline-userdebug, confirmed that java profiling activates from startup and at runtime.
Bug: 120409382
Change-Id: Ic251afeca4324dc650ac1d4f46976b526eae692a
(cherry picked from commit 998792e2b6)
Merged-In: Ic251afeca4324dc650ac1d4f46976b526eae692a
This relocation is used for static TLS's initial-exec (IE) accesses.
A TLS symbol's value is its offset from the start of the ELF module's
TLS segment. It doesn't make sense to add the load_bias to this value,
so skip the call to soinfo::resolve_symbol_address.
Allow TLS relocations to refer to an unresolved weak symbol. In that case,
sym will be non-zero, but lsi will be nullptr. The dynamic linker resolves
the TPREL relocation to 0, making &missing_weak_symbol equal the thread
pointer.
Recognize Gold-style relocations to STB_LOCAL TLS symbols/sections and
issue an error.
Remove the "case R_AARCH64_TLS_TPREL64", because the R_GENERIC_TLS_TPREL
case handles it.
Remove the no-op R_AARCH64_TLSDESC handler. It's better to issue an error.
dlopen_library_with_ELF_TLS now fails with a consistent error about an
unimplemented dynamic TLS relocation.
Bug: http://b/78026329
Test: bionic unit tests (elftls tests are added in a later CL)
Change-Id: Ia08e1b5c8098117e12143d3b4ebb4dfaa5ca46ec
If the alignment of a TLS segment in a shared object is invalid, return
an error through dlerror() rather than aborting the process.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: I60e589ddd8ca897f485d55af089f08bd3ff5b1fa
This implementation simply iterates over each static TLS module and
copies its initialization image into a new thread's static TLS block.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: Ib7edb665271a07010bc68e306feb5df422f2f9e6
Replace reserve_tcb with reserve_exe_segment_and_tcb, which lays out both
the TCB and the executable's TLS segment, accounting for the difference in
layout between variant 1 and variant 2 targets.
The function isn't actually called with a non-null TlsSegment* yet.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: Ibd6238577423a7d0451f36da7e64912046959796
* changes:
Add a __bionic_get_tls_segment function
Factor out ScopedRWLock into its own header
Build the linker with -D_USING_LIBCXX
Provide a stub aeabi.read_tp on other archs
Remove TLS_SLOT_TSAN(8)
The function searches for a TLS segment in a ElfXX_Phdr table.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: I221b13420d1a2da33fc2174b7dd256589f6ecfdb
For ELF TLS "local-exec" accesses, the static linker assumes that an
executable's TLS segment is located at a statically-known offset from the
thread pointer (i.e. "variant 1" for ARM and "variant 2" for x86).
Because these layouts are incompatible, Bionic generally needs to allocate
its TLS slots differently between different architectures.
To allow per-architecture TLS slots:
- Replace the TLS_SLOT_xxx enumerators with macros. New ARM slots are
generally negative, while new x86 slots are generally positive.
- Define a bionic_tcb struct that provides two things:
- a void* raw_slots_storage[BIONIC_TLS_SLOTS] field
- an inline accessor function: void*& tls_slot(size_t tpindex);
For ELF TLS, it's necessary to allocate a temporary TCB (i.e. TLS slots),
because the runtime linker doesn't know how large the static TLS area is
until after it has loaded all of the initial solibs.
To accommodate Golang, it's necessary to allocate the pthread keys at a
fixed, small, positive offset from the thread pointer.
This CL moves the pthread keys into bionic_tls, then allocates a single
mapping per thread that looks like so:
- stack guard
- stack [omitted for main thread and with pthread_attr_setstack]
- static TLS:
- bionic_tcb [exec TLS will either precede or succeed the TCB]
- bionic_tls [prefixed by the pthread keys]
- [solib TLS segments will be placed here]
- guard page
As before, if the new mapping includes a stack, the pthread_internal_t
is allocated on it.
At startup, Bionic allocates a temporary bionic_tcb object on the stack,
then allocates a temporary bionic_tls object using mmap. This mmap is
delayed because the linker can't currently call async_safe_fatal() before
relocating itself.
Later, Bionic allocates a stack-less thread mapping for the main thread,
and copies slots from the temporary TCB to the new TCB.
(See *::copy_from_bootstrap methods.)
Bug: http://b/78026329
Test: bionic unit tests
Test: verify that a Golang app still works
Test: verify that a Golang app crashes if bionic_{tls,tcb} are swapped
Merged-In: I6543063752f4ec8ef6dc9c7f2a06ce2a18fc5af3
Change-Id: I6543063752f4ec8ef6dc9c7f2a06ce2a18fc5af3
(cherry picked from commit 1e660b70da)
This change makes it easier to move the location of the pthread keys
(e.g. into the bionic_tls struct).
Bug: http://b/78026329
Test: bionic unit tests
Test: disassembly of libc.so doesn't change
Merged-In: Ib75d9dab8726de96856af91ec3daa2c5cdbc2178
Change-Id: Ib75d9dab8726de96856af91ec3daa2c5cdbc2178
(cherry picked from commit ecad24fad9)
Track movement of ICU .dat file into the runtime module
This reverts commit b9e7b699a2.
This is a reapply of commit 68c6bd8b43
Bug: 120853401
Bug: 119293618
Test: build / boot
Change-Id: I69e0dde2756da64ac0b17f1726145dccacf5961f
PR_SET_VMA takes mmap_sem, which can cause contention and reduce
performance any time many threads are created at the same time,
like app startup.
Test: camera launch performance
Bug: 122471935
Change-Id: If7fa7ad99654c01d503f694976fd92bfd30d2afd
Having a constexpr constructor should guarantee that the static
`globals` variable in __libc_shared_globals is initialized statically
(as opposed to dynamically), which is important because
__libc_shared_globals is called very early (before the linker has
relocated itself). With the constructor, though, the fields can safely
have in-line default initializers.
Bug: none
Test: bionic unit tests
Change-Id: Icde821557369625734a4d85d7ff55428bad5c247
We could cause illegal memory access due to not checking the scanddir
return value.
Test: run linux-bionic dex2oat to build boot.oat
Bug: 119332362
Change-Id: Ic5824e0373b9b99d8f892b4fcf22e742c6f792da
* Add explicit to conversion constructors/operators
Bug: 28341362
Test: make with WITH_TIDY=1 DEFAULT_GLOBAL_TIDY_CHECKS=-*,google-explicit-constructor
Change-Id: Id1ad0327c1b8c6f094bcbb3ae599bc1f716b3f2f
This change is intended to allow native-bridge to use independent
TLS memory for host and guest environments, while still sharing a
thread-local errno between the two.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: I838cd321e159add60760bc12a8aa7e9ddc960c33
Merged-In: I838cd321e159add60760bc12a8aa7e9ddc960c33
(cherry picked from commit a9c7c55462)
Instead of passing the address of a KernelArgumentBlock to libc.so for
initialization, use __loader_shared_globals() to initialize globals.
Most of the work happened in the previous CLs. This CL switches a few
KernelArgumentBlock::getauxval calls to [__bionic_]getauxval and stops
routing the KernelArgumentBlock address through the libc init functions.
Bug: none
Test: bionic unit tests
Change-Id: I96c7b02c21d55c454558b7a5a9243c682782f2dd
Merged-In: I96c7b02c21d55c454558b7a5a9243c682782f2dd
(cherry picked from commit 746ad15912)
Make getauxval() work in .preinit_array. It still won't be usable for
ifuncs unless we can guarantee that the __loader_shared_globals relocation
is resolved before the ifunc calls [__bionic_]getauxval.
Define __bionic_getauxval for use in replacing calls to
KernelArgumentBlock::getauxval, which doesn't (and sometimes isn't allowed
to) access TLS variables like errno.
Bug: http://b/25751302
Test: bionic unit tests
Change-Id: I461feeaed7f43cfa2a2b6c34147194f0df82b516
Merged-In: I461feeaed7f43cfa2a2b6c34147194f0df82b516
(cherry picked from commit bdab4a2b97)
Split __libc_init_main_thread into __libc_init_main_thread_early and
__libc_init_main_thread_late. The early function is called very early in
the startup of the dynamic linker and static executables. It initializes
the global auxv pointer and enough TLS memory to do system calls, access
errno, and run -fstack-protector code (but with a zero cookie because the
code for generating a cookie is complex).
After the linker is relocated, __libc_init_main_thread_late finishes
thread initialization.
Bug: none
Test: bionic unit tests
Change-Id: I6fcd8d7587a380f8bd649c817b40a3a6cc1d2ee0
Merged-In: I6fcd8d7587a380f8bd649c817b40a3a6cc1d2ee0
(cherry picked from commit 39bc44bb0e)
Because we also want to profile Java applications, which have longer
names, the character limit of comm is a problem. To avoid complexity, it
is preferable to apply the same logic for finding running processes
(which includes Java apps), and determining whether to profile a process
from startup.
Test: m
Test: flash sailfish
Test: setprop heapprofd.enable 1
setprop heapprofd.enable.ls 1
ls
/system/bin/ls
/system/bin/ls /
Bug: 120175590
Change-Id: Id0859d4a333efcb05883e611ea6a31a51468f82c
* changes:
Use shared globals to init __progname + environ
Move the abort message to libc_shared_globals
Expose libc_shared_globals to libc.so with symbol
Initialize the __progname and environ global variables using
libc_shared_globals rather than KernelArgumentBlock.
Also: suppose the linker is invoked on an executable:
linker prog [args...]
The first argument passed to main() and constructor functions is "prog"
rather than "linker". For consistency, this CL changes the BSD
__progname global from "linker" to "prog".
Bug: none
Test: bionic unit tests
Change-Id: I376d76953c9436706dbc53911ef6585c1acc1c31
__libc_shared_globals() is available in dynamic modules as soon as
relocation has finished (i.e. after ifuncs run). Before ifuncs have run,
the android_set_abort_message() function already doesn't work because it
calls public APIs via the PLT. (If this matters, we can use a static
bool variable to enable android_set_abort_message after libc
initialization).
__libc_shared_globals() is hidden, so it's available in the linker
immediately (i.e. before relocation). TLS memory (e.g. errno) currently
isn't accessible until after relocation, but a later patch fixes that.
Bug: none
Test: bionic unit tests
Change-Id: Ied4433758ed2da9ee404c6158e319cf502d05a53
Previously, the address of the global variable was communicated from the
dynamic linker to libc.so using a field of KernelArgumentBlock, which is
communicated using the TLS_SLOT_BIONIC_PREINIT slot.
As long as this function isn't called during relocations (i.e. while
executing an ifunc), it always return a non-NULL value. If it's called
before its PLT entry is relocated, I expect a crash.
I removed the __libc_init_shared_globals function. It's currently empty,
and I don't think there's one point in libc's initialization where
shared globals should be initialized.
Bug: http://b/25751302
Test: bionic unit tests
Change-Id: I614d25e7ef5e0d2ccc40d5c821dee10f1ec61c2e
Other minor changes:
* document assignment that relies on _Atomic assignments to use
atomic_store.
* consistently use atomic_store when assigning to atomics.
* remove incorrect comment.
Test: m
Test: flash & boot sailfish
Change-Id: I4789c08f7ac28a2de8d6925d03af354514bfd9d7
__sanitize_environment_variables is only called when getauxval(AT_SECURE)
is true.
Instead of scanning __libc_auxv, reuse getauxval. If the entry is missing,
getauxval will set errno to ENOENT.
Reduce the number of times that __libc_sysinfo and __libc_auxv are
initialized. (Previously, __libc_sysinfo was initialized 3 times for the
linker's copy). The two variables are initialized in these places:
- __libc_init_main_thread for libc.a (including the linker copy)
- __libc_preinit_impl for libc.so
- __linker_init: the linker's copy of __libc_sysinfo is still initialized
twice, because __libc_init_main_thread runs after relocation. A later
CL consolidates the linker's two initializations.
Bug: none
Test: bionic unit tests
Change-Id: I196f4c9011b0d803ee85c07afb415fcb146f4d65
remove android_open_proxy and use dns_open_proxy instead of it.
dns_open_proxy is in libnetd_client and
it does the same thing as android_open_proxy except return value.
It returns fd directly now.
Test: build, dns works fine
Change-Id: I984743fb50b23eeb9a7d24e9fc347832acfe2afe
This lets us do two things:
1) Make setjmp and longjmp compatible with shadow call stack.
To avoid leaking the shadow call stack address into memory, only the
lower log2(SCS_SIZE) bits of x18 are stored to jmp_buf. This requires
allocating an additional guard page so that we're guaranteed to be
able to allocate a sufficiently aligned SCS.
2) SCS overflow detection. Overflows now result in a SIGSEGV instead
of corrupting the allocation that comes after it.
Change-Id: I04d6634f96162bf625684672a87fba8b402b7fd1
Test: bionic-unit-tests
Also move this and android_get_device_api_level into <android/api-level.h>
so that they're always available.
This involves cleaning up <sys/cdefs.h> slightly.
Bug: N/A
Test: builds
Change-Id: I25435c55f3549cd0d827a7581bee75ea8228028b
