Symbol lookup is O(L) where L is the number of libraries to search (e.g.
in the global and local lookup groups). Factor out the per-DSO work into
soinfo_do_lookup_impl, and optimize for the situation where all the DSOs
are using DT_GNU_HASH (rather than SysV hashes).
To load a set of libraries, the loader first constructs an auxiliary list
of libraries (SymbolLookupList, containing SymbolLookupLib objects). The
SymbolLookupList is reused for each DSO in a load group. (-Bsymbolic is
accommodated by modifying the SymbolLookupLib at the front of the list.)
To search for a symbol, soinfo_do_lookup_impl has a small loop that first
scans a vector of GNU bloom filters looking for a possible match.
There was a slight improvement from templatizing soinfo_do_lookup_impl
and skipping the does-this-DSO-lack-GNU-hash check.
Rewrite the relocation processing loop to be faster. There are specialized
functions that handle the expected relocation types in normal relocation
sections and in PLT relocation sections.
This CL can reduce the initial link time of large programs by around
40-50% (e.g. audioserver, cameraserver, etc). On the linker relocation
benchmark (64-bit walleye), it reduces the time from 131.6ms to 71.9ms.
Bug: http://b/143577578 (incidentally fixed by this CL)
Test: bionic-unit-tests
Change-Id: If40a42fb6ff566570f7280b71d58f7fa290b9343
This supports the soong commit which causes most platform binaries to stop
statically linking against the unwinder implementation. The soong commit
message has more motivation for this change.
ARM32 uses LLVM libunwind, while all other platforms use libgcc as the
unwinder implementation. This matches the current choices of unwinders on
the various architectures, but means that apps which were directly linking
against the libc.so unwinder symbols on ARM32 are now using LLVM libunwind
instead of libgcc.
Set libc_headers sdk_version to 1 so that libunwind_llvm can depend on it,
and stop statically linking libunwind into libc_malloc_debug.
Bug: 144430859
Change-Id: I52c7f7893d93f500383aeb0b76086c3b6f1935a5
These are no longer necessary now that LLVM no longer emits references to
this symbol on Android.
Bug: 144430859
Change-Id: I6c43338f755ce5a79e2df36bd3f2006a748fab27
A later linker CL defines a function that needs to forward a printf format
and argument list to TRACE(), but there is no version of the TRACE macro
that works with a va_list. The CL also needs to check the verbosity level,
using a slow path if tracing is enabled, so define LINKER_VERBOSITY_xxx
macros.
Bug: none
Test: bionic unit tests
Change-Id: I578d36a12dc40f9a651956b4b09adc1a7c644e24
Validate the list of defined versions explicitly, during library
prelinking, rather than implicitly as part of constructing the
VersionTracker in soinfo::link_image.
Doing the validation upfront allows removing the symbol lookup failure
code paths, which only happen on a library with invalid version
information.
Helps on the walleye 64-bit linker relocation benchmark (146.2ms ->
131.6ms)
Bug: none
Test: bionic unit tests
Change-Id: Id17508aba3af2863909f0526897c4277419322b7
Previously, during a find_libraries call that loaded a library, a
library was prelinked once for each DT_NEEDED reference to the library.
This CL has a negligible effect on the linker relocation benchmark
(146.9ms -> 146.2ms).
Bug: none
Test: bionic unit tests
Change-Id: I385f312b8acf8d35aa0af9722131fe367b5edd9b
The soinfo instances of linker and vdso have been added to g_default_namespace
before init_default_namespace() is called. So init_default_namespace() don't
have to add them a second time.
Test: manual
Change-Id: I29b3da782b1e9445509f45a7698561fc3e19e9a1
Historically we've made a few mistakes where they haven't matched the
right number. And most non-Googlers are much more familiar with the
numbers, so it seems to make sense to rely more on them. Especially in
header files, which we actually expect real people to have to read from
time to time.
Test: treehugger
Change-Id: I0d4a97454ee108de1d32f21df285315c5488d886
DL_WARN message when failed to find generated linker config makes some
of the ART tests fail. Lowering log level as ART test does not have
linkerconfig generated for the test.
Bug: 146386369
Test: Cuttlefish boot succeeded without any error
Test: run_build_test_target.py art-linux-bionic-x64-zipapex passed
Change-Id: I4f876c3ac5c30d32d51346d4cd16b5205da8f1bf
New linker configuration from /linkerconfig is not suitbale for emulated
architectures. But as of now, native_bridge linkers pick it up as well
and thus fail to find the libraries for emulated architectures.
This is a (temporary) fix so native_bridge linker still picks up
configuration from old location.
Bug: 138920271
Test: native_bridge linker works
Change-Id: I0abbd3e95f9e6830385b0f19db0688e6183030b9
Current linker configuration is only enabled from fully treblelized
devices. This change will allow linker to first check generated linker
configuration even for non-treblelized devices and recovery.
Bug: 139638519
Test: Tested from cuttlefish
Change-Id: I655b1ab807cd8db5696d07fd2bdd00ce0558901d
Due to some special environment, linker config should not be located
under /dev partition. It would be better to relocate linker config under
new root dir /linkerconfig.
Bug: 144966380
Test: m -j && tested from cuttlefish
Change-Id: Icda1d2ef34b42159c6ebce58b03211cc13f08121
The string is still "lib" or "lib64" with native bridge. It doesn't need
to be configured in the Android.bp file anymore, so move it to a header
file. This change will ensure that ${LIB} expands to the same thing in
both ld.config.txt and DT_RUN_PATH.
Bug: http://b/145197367
Test: manual
Change-Id: Iab87f3156f2984dd3a20e4ccda423892c8b58763
Merged-In: Iab87f3156f2984dd3a20e4ccda423892c8b58763
With -O0, on arm64, Clang uses memset for this declaration:
bionic_tcb temp_tcb = {};
arm64 doesn't currently have an ifunc for memset, but if it did, then this
line would crash when the linker is compiled with -O0. It looks like other
architectures would only use a memset call if the bionic_tcb struct were
larger.
Avoid memset by using a custom memclr function that the compiler optimizes
into something efficient.
Also add __attribute__((uninitialized)) to ensure that
-ftrivial-auto-var-init does not generate a call to memset. See this
change[1] in build/soong.
[1] If085ec53c619e2cebc86ca23f7039298160d99ae
Test: build linker with -O0, linker[64] --help works
Bug: none
Change-Id: I0df8065a362646de4fa021cae63a7d68ca3966b6
This change makes it easier to diagnose mistakes in linker
configuration that result in a library being accidentally loaded in
multiple namespaces without its dependencies available everywhere.
Test: manually tested the error message
Test: bionic-unit-tests
Change-Id: I03a20507f8fc902c2445a7fbbf59767ffffd5ebf
Using ifuncs allows the linker to select faster versions of libc functions
like strcmp, making linking faster.
The linker continues to first initialize TLS, then call the ifunc
resolvers. There are small amounts of code in Bionic that need to avoid
calling functions selected using ifuncs (generally string.h APIs). I've
tried to compile those pieces with -ffreestanding. Maybe it's unnecessary,
but maybe it could help avoid compiler-inserted memset calls, and maybe
it will be useful later on.
The ifuncs are called in a special early pass using special
__rel[a]_iplt_start / __rel[a]_iplt_end symbols. The linker will encounter
the ifuncs again as R_*_IRELATIVE dynamic relocations, so they're skipped
on the second pass.
Break linker_main.cpp into its own liblinker_main library so it can be
compiled with -ffreestanding.
On walleye, this change fixes a recent 2.3% linker64 start-up time
regression (156.6ms -> 160.2ms), but it also helps the 32-bit time by
about 1.9% on the same benchmark. I'm measuring the run-time using a
synthetic benchmark based on loading libandroid_servers.so.
Test: bionic unit tests, manual benchmarking
Bug: none
Merged-In: Ieb9446c2df13a66fc0d377596756becad0af6995
Change-Id: Ieb9446c2df13a66fc0d377596756becad0af6995
(cherry picked from commit 772bcbb0c2)
Define a "linker_bin_template" cc_defaults module that a native bridge
implementation can inherit to define a guest linker.
Break the debuggerd_init call off into separate
linker_debuggerd_{android,stub}.cpp files to allow opting in/out of the
debuggerd integration without needing to change how linker_main.cpp is
compiled. (This is necessary for a later commit that moves
linker_main.cpp into a new static library.)
Test: bionic unit tests
Bug: none
Merged-In: I7c5d79281bce1e69817b266dd91d43ea40f78522
Change-Id: I7c5d79281bce1e69817b266dd91d43ea40f78522
(cherry picked from commit 5adf402ee9)
This reverts commit 61a97e9505.
Reason for revert: Breaks ART run-tests (b/143458513).
Test: Run ART tests on device in a chroot environment
Bug: 143458513
Bug: 139638519
Change-Id: Ib047a24d6e82e38ebdaafeab294b8be44b74bd9c
In order for an ifunc resolver to detect the presence of certain CPU features,
access to getauxval(AT_HWCAP) or getauxval(AT_HWCAP2) may be required. In order
for getauxval() to work, it needs to access the pointer to the auxiliary vector
stored by the linker in the libc shared globals data structure. Accessing the
shared globals requires libc to call the __libc_shared_globals() function
exported by the linker. However, in order to call this function, libc must
be fully relocated, which is not guaranteed to be the case at the point when
ifunc resolvers are called.
glibc solves this problem by passing the values of getauxval(AT_HWCAP)
(and getauxval(AT_HWCAP2) on aarch64) as arguments to the ifunc resolver.
Since this seems to be not only the most straightforward way to solve the
problem but also improves our compatibility with glibc, we adopt their
calling convention.
This change is ABI compatible with old resolvers because the arguments are
passed in registers, so the old resolvers will simply ignore the new arguments.
Bug: 135772972
Change-Id: Ie65bd6e7067f0c878df3d348c815fda61dc12de2
With VNDK APEX, the path for VNDK libs has been changed
from /system/lib/vndk-VER to /apex/com.android.vndk.vVER/lib
In most cases, vndk version can be substituted when generating
ld.config.txt by linkerconfig.
But, ld.config.txt files in APEX packages still rely on runtime substitution.
Specifically, com.android.media.swcodec is using VNDK_VER variable.
Moreover, it cannot migrate to a new location since it should run on
older version of system due to APEX requirements.
For backward compatibility, instead of changing the value of old
variable, a new variable is added.
- VNDK_VER : "-" prefixed vndk version, used for older path
(e.g. /system/lib/vndk-sp-29)
- VNDK_APEX_VER : "v" prefixed vndk versions.
(e.g. /apex/com.android.vndk.v29/lib)
Test: add a vendor binary to /apex/com.android.media.swcodec/bin
which opens /system/lib/vndk-sp{VNDK_VER}/hw/android.hidl.memory@1.0-impl.so
via android_load_sphal_library()
Bug: 142912195
Change-Id: I3dfb3c1068cff00d5b63e92d51da6c4af00d264e
COUNT_PAGES tries to count the pages dirtied by relocations, but this
implementation is broken because it's merging rel->r_offset values from
multiple DSOs. The functionality is hard to use, because it requires
rebuilding the linker, and it's not obvious to me that it should belong
in the linker. If we do want it, we should make it work without rebuilding
the linker.
Similar information can currently be collected by parsing the result of
`readelf -r` on a binary (or a set of binaries).
Bug: none
Test: m linker libc com.android.runtime ; adb sync ; run something
Change-Id: I760fb6ea4ea3d1927eb5145cdf4ca133851d69b4
Linker config generator now covers ld.config.txt for Legacy and
VNDK-Lite devices, so linker can use those instead of existing ones
under /system/etc
Bug: 139638519
Test: m -j passed
Change-Id: I90f14727148cbf9629b90dc4fd78362bed8ea4e4
This relocation is labeled as a static relocation in the ARM ELF ABI and
shouldn't appear in position-independent code.
It currently calculates the value to relocate incorrectly:
"sym_addr - rel->r_offset" should be "sym_addr - reloc"
I don't know of any other dynamic linker that handles this relocation.
Test: bionic unit tests
Bug: http://b/19197129
Change-Id: Ia0c0018c82fe98d5edb54ee6f5c9f402b1fa3076
Combine:
- R_AARCH64_ABS64
- R_ARM_ABS32
- R_X86_64_64
- R_386_32
They do mostly the same thing as R_GENERIC_GLOB_DAT. They always have an
addend, though, and R_GENERIC_GLOB_DAT currently only has an addend on
RELA targets.
Test: bionic unit tests
Bug: none
Change-Id: Ibe964c3b28705086aecb6e7d80c90998aad3c0a4
Specifically, remove:
- R_AARCH64_ABS32
- R_AARCH64_ABS16
- R_AARCH64_PREL64
- R_AARCH64_PREL32
- R_AARCH64_PREL16
These relocations never currently appear in dynamic ELF files, and the
linker didn't handle them correctly. The AArch64 ELF ABI document
classifies them as "static relocations", which dynamic linkers don't need
to handle. (The document also classifies R_AARCH64_ABS64 as static,
though, and that relocation is common in DSOs. Perhaps static linkers
can't use R_AARCH64_GLOB_DAT to relocate data outside the GOT.)
Previously, for {ABS,PREL}{32,16}, Bionic always failed with an
out-of-range error. e.g. For {ABS,PREL}16, the value had to satisfy two
conditions:
- be at least (Elf64_Addr)INT16_MIN, i.e. 0xffff_ffff_ffff_8000
- be at most (Elf64_Addr)UINT16_MAX, i.e. 0xffff
The PREL relocations should have used sym_addr + addend - reloc, not
sym_addr + addend - rel->r_offset.
Bug: http://b/19197129
Test: bionic unit tests
Change-Id: I791da8ac471b3fb108cf77405c222f6e4bd34ae4
The linker shouldn't throw exceptions, but because it links with
libc++_static.a, there are code paths that could throw an exception. On
those code paths, the unwinder needs to lookup EH information for the
linker binary, and the linker had two inconsistent ways of doing this:
* dl_iterate_phdr (for libgcc): dlfcn.cpp defined a linker-internal
version of this API that forwarded to __loader_dl_iterate_phdr
* __gnu_Unwind_Find_exidx (for arm32 libgcc): linker_exidx_static.c was
an old, broken copy of exidx_static.c that used
__exidx_start/__exidx_end symbols. (The file should have used the
addresses of the symbols rather than their contents.)
The linker's data structures might be in an inconsistent state at a point
where exceptions are thrown, so it seems better to limit its unwinder to
just the linker binary's EH info. Rather than forward the dl* EH APIs,
link in the static-binary versions from libc_unwind_static.a. That library
is already part of libc_nomalloc.a, but include it directly with
whole_static_libs so that __gnu_Unwind_Find_exidx is defined when we're
using libgcc on arm32.
Try to link in libunwind_llvm.a into the arm32 linker binary so we're
using the same unwinder as normal arm32 binaries. I'm not sure the library
will appear in the right order, but maybe it doesn't matter given LLD's
unconventional archive linking semantics.
Test: bionic unit tests
Test: "readelf --dyn-syms linker" reports no UNDEF symbols
Test: "readelf -r linker" reports only relative relocations
Bug: none
Change-Id: I5982ec830ba0f15d066536de24f6cd7e9503498b
Merged-In: I5982ec830ba0f15d066536de24f6cd7e9503498b
With VNDK APEX, the path for VNDK libs has been changed
from /system/lib/vndk-VER to /apex/com.android.vndk.vVER/lib
Previously, VNDK_VER is replaced with prefix(e.g. "-29"). We could
still prepend prefix("v") to the vndk version, but this change uses a
raw vndk version as the value of VNKD_VER.
Bug: 141451661
Test: m && boot (tested with cuttlefish)
Change-Id: Ibf4cf5e29b7f28e733d4b3bc15171f4359e1d2f2
The bionic libs are now restricted to be in the runtime APEX and the
platform (for bootstrapping). It can still be referenced from other
APEXes but can't be included there.
Bug: 139870423
Test: m
Change-Id: I7f99eef27ccf75844ca5c9a7ea866496841b738f
The run-time of dl_iterate_phdr can be noticeable for exception-handling.
e.g. I tested extest on the GitHub issue by:
- disabling the dlpi_subs/dlpi_adds-based caching in libgcc
- adding a weaker "load base hint" caching in libgcc that doesn't need
the dlpi_subs/dlpi_adds fields, but still has to iterate over every
module to validate a cache hit
extest throws 10000 exceptions, and I saw a regression from ~1550ms
runtime on Q to about ~1950ms on master. This CL reduces the regression to
about ~1700ms.
Bug: https://github.com/android/ndk/issues/1062
Test: bionic unit tests
Change-Id: I099e97e1a20f5b2aa6737789e49d965170eb85a8
There are places in frameworks and art code that directly included
private bionic header files. Move these files to the new platform
include files.
This change also moves the __get_tls.h header file to tls.h and includes
the tls defines header so that there is a single header that platform
code can use to get __get_tls and the defines.
Also, simplify the visibility rules for platform includes.
Bug: 141560639
Test: Builds and bionic unit tests pass.
Change-Id: I9e5e9c33fe8a85260f69823468bc9d340ab7a1f9
Merged-In: I9e5e9c33fe8a85260f69823468bc9d340ab7a1f9
(cherry picked from commit 44631c919a)
Use generated linker config by default, but with some back up plan
(sys.linker.use_generated_config property). Linker config generator
still does not support non-treblelized devices and vndk-lite, so these
cases should be handled later.
Bug: 138920271
Test: m -j && atest passed
Test: Tested from cuttlefish
Change-Id: I39e9d089a82f9409eccdcaa4fb26660caf3f5779
Main change is to log errors directly where they occur, to correlate with
other dlopen/dlsym logs.
Test: Build & boot with and without LinkerLogger::flags_ initialised to kLogDlopen
Change-Id: If36f52914dc97cedd95dc9375c291640c6891728
This reverts commit 39699ea095.
Reason for revert: This breaks swcodec apex. (It still uses VNDK_VER)
Bug: 140014331
Change-Id: I3a4c5b68fdd5e42fdf93c4c71f6c3be839ea8a28
Previously, Bionic's dl_phdr_info only included the first four
dl_iterate_phdr fields. Several other libc's have these additional fields:
unsigned long long dlpi_adds -- incremented when a library is loaded
unsigned long long dlpi_subs -- incremented when a library is unloaded
size_t dlpi_tls_modid -- TLS module ID
void* dlpi_tls_data -- pointer to current thread's TLS block or NULL
These extra fields are also exposed by glibc, musl, and FreeBSD. The
unwinder in libgcc.a, linked into shipping Android DSOs, has a
PC->eh_frame cache that activates if dl_phdr_info has the dlpi_adds and
dlpi_subs fields (indicated at run-time by a sufficiently-large size
argument to the callback).
Bug: https://github.com/android-ndk/ndk/issues/1062
Test: bionic unit tests
Change-Id: I6f0bab548cf8c828af2ddab9eb01c5c6d70cd81f
The call to the load hook needs to be moved before the call to link_image()
because the latter calls ifunc resolvers which might access global
variables. This fixes a bunch of ifunc tests.
The dlfcn.segment_gap test is currently failing. One problem is that the name
of the .bss.end_of_gap section changes as a result of global instrumentation.
Add some wildcards in so that we match both names. The other problem seems
to be the same as b/139089152.
It turns out that we need to untag pointers in a few more places. Since we have
quite a few of these now it seems worth creating a function for it.
Test: bionic-unit-tests
Change-Id: I44e2b0904faacdda7cc0c5e844ffc09de01dea2d
The linker currently sets VMA name ".bss" for bss sections in DSOs
loaded by the linker. With this change, the linker now also sets VMA
name for bss sections in the linker itself and the main executable, so
that they don't get left out in various accounting.
Test: Run 'dd' and check its /proc/<pid>/maps.
Change-Id: I62d9996ab256f46e2d82cac581c17fa94836a228
This reverts commit 66b5e3ca19.
Reason for revert: Causes installd_dexopt_test to fail. Presumably there is an existing issue which this change is turning into a hard fail, so that needs to be fixed before this can be rolled forwards. See b/139092341.
Bug: 139092341
Bug: 137788030
Change-Id: I1beccddff64c86541bc003b9b009f4046ec9cbf1
Anonymous namespace is not created separately. When a regular namespace
is created with ANDROID_NAMESPACE_TYPE_ALSO_USED_AS_ANONYMOUS, that
namespace is used as the anonymous namespace.
Bug: 130388701
Test: CtsBionicTestCases
Change-Id: Ie449a59f303487a7a9ff6fff98e6148e9f968fd2
Make fdsan fatal by default on native processes, and Java processes that
target R or later.
Bug: http://b/137788030
Test: manual
Change-Id: I5900efc606fc0d47b69223574ad7dca89afe4eda
Linker config now generates ld.config.txt under /dev/linkerconfig from
init. To use this, this change updates linker to pick up
/dev/linkerconfig/ld.config.txt if system property
(sys.linker.use_generated_config) is true.
Bug: 138920271
Test: m -j && tested from cuttlefish
Change-Id: I0f527741ace2d512b45fd72c301aa4cfcfe17a71
These hooks notify the HWASan runtime library whenever a library is loaded
or unloaded so that it can update its shadow memory.
Bug: 138159093
Test: walleye_hwasan-userdebug boots with+without https://reviews.llvm.org/D65770
Change-Id: I6caf2a6540ed2c0d94db444e806a3c7ba504cabb
As of aosp/1086558, stdatomic.h directly checks for the presence of
<atomic> instead of checking _USING_LIBCXX, so these lines are
obsolete.
Test: build
Change-Id: Icfdb47b3ad109421b3413041f3d3027e2c0b518e
The AArch64 ELF specification originally specified these relocations:
R_AARCH64_TLS_DTPREL64 1028
R_AARCH64_TLS_DTPMOD64 1029
However, the GNU toolchain implemented them as:
R_AARCH64_TLS_DTPMOD64 1028
R_AARCH64_TLS_DTPREL64 1029
The AArch64 ELF specification has been updated to replace the relocations
with R_AARCH64_TLS_IMPDEF1 and with R_AARCH64_TLS_IMPDEF2, where each
implementation can choose which interpretation it implements, but with a
suggestion to follow the GNU behavior.
Also: remove the 64 suffixes from the TLS relocations. The newest version
of the AArch64 ELF specification (version f) also omits the suffixes, as
do the glibc headers.
Bug: http://b/123385182
Bug: https://bugs.llvm.org/show_bug.cgi?id=40507
Test: build device, it boots
Change-Id: Ie98b7624752982973f0ac969d646454a86784cab
When relocating a DSO, it is not unusual to have consecutive
relocations using the same symbol. In this case, it is wasteful to
perform the same symbol lookup. This change implements an 1-entry
symbol cache so that symbol lookup results are reused in such scenario.
Test: On cuttlefish, enable STATS in linker_debug.h. Boot and see RELO
STATS in logcat showing cache hits. Hit rate seen is mostly
within 15% to 45%.
Change-Id: I84783d3b9a6ac9e39ed7fb45e58f6b3c012478d0
The demangler is moving out of libc++abi to save on bloat.
Test: make checkbuild
Bug: http://b/138245375
Change-Id: I7402894bb326f524388c9efdf86509dad50f2018
Previously, the linker configuration of an APEX would only be used for
executables located exactly in its `bin` directory. This change
relaxes this condition to also include executables located in any
directory under the APEX's `bin` directory.
This change is needed to support APEX binaries located in directories
under the APEX's `bin` directory that need to use the APEX's linker
configuration (e.g. ART gtests located in
`bin/art/{arm,arm64,x86,x86-64}` in the Testing Runtime APEX).
Test: Run ART gtests on device using the Testing Runtime APEX
Bug: 129534335
Change-Id: I0eac317eba856211a344fa00e66640aae10816ea
Before:
$ ldd toybox
toybox: not an ELF file
After
$ ldd toybox
toybox: cannot open
I've also sent a toybox patch upstream to make that:
$ ldd toybox
toybox: cannot open: No such file or directory
Test: manual
Change-Id: If961cdf6f2f2f8d4f6cdfcade9061e49a09d1f01
Ordinary executables have a PT_INTERP path of /system/bin/linker[64], but:
- executables using bootstrap Bionic use /system/bin/bootstrap/linker[64]
- ASAN executables use /system/bin/linker_asan[64]
gdb appears to use the PT_INTERP path for debugging the dynamic linker
before the linker has initialized the r_debug module list. If the linker's
l_name differs from PT_INTERP, then gdb assumes that the linker has been
unloaded and searches for a new solib using the linker's l_name path.
gdb may print a warning like:
warning: Temporarily disabling breakpoints for unloaded shared library "$OUT/symbols/system/bin/linker64"
If I'm currently debugging the linker when this happens, gdb apparently
doesn't load debug symbols for the linker. This can be worked around with
gdb's "sharedlibrary" command, but it's better to avoid it.
Previously, when PT_INTERP was the bootstrap linker, but l_name was
"/system/bin/linker[64]", gdb would find the default non-bootstrap linker
binary and (presumably) get confused about symbol addresses.
(Also, remove the "static std::string exe_path" variable because the
soinfo::realpath_ field is a std::string that already lasts until exit. We
already use it for link_map_head.l_name in notify_gdb_of_load.)
Bug: http://b/134183407
Test: manual
Change-Id: I9a95425a3a5e9fd01e9dd272273c6ed3667dbb9a
Rearrange the st_dev and st_ino checks to reduce the number of
comparison needed.
Test: Ran cameraserver on a Go device. Measured time spent in the linker
and saw ~1% speed-up.
Change-Id: I8e977ff37925eae3ba8348e7c4a01ce8af3b9b6d
A linker namespace lives longer than its caller. It is never deleted
once created in a process. Currently, the pointer to the name is simply
copied which results dangling reference when the name is actually from
temporary objects like std::object. Fixing the issue by strdup'ing the
name upon namespace creation.
Bug: 130388701
Test: atest CtsJniTestCases; the log does not show broken error messages
like
unexpected dlerror: dlopen failed: library "/system/lib64/android.frameworks.cameraservice.common@2.0.so" needed or dlopened by "/data/app/android.jni.cts-HP6GyGXYy5honHQAffUXgw==/lib/arm64/libjnitest.so" is not accessible for the namespace " mT?"
Change-Id: I25d9d76f8520f490755c189ded5659e6c9741f79
When parsing ld.config.txt, paths in *.search.paths properties are
resolved using resolved_paths(). This is causing a number of newfstatat
system calls during a process start-up and is contributing about 3.6%
of the entire linker initialization time.
Optimize the overhead by not resolving already resolved paths; resolved
paths are cached.
Bug: 132348336
Test: strace -C -e newfstatat ls shows the number of the syscalls is
reduced from 131 to 99
Test: simpleperf record --call-graph fp -f 10000000 -o /data/perf.data ls
simpleperf report -g -i /data/perf.data
shows percentage of get_paths() from 2.9% to 2.5%
Change-Id: I79823255a89f6e2a63b80c74e6768850276ab536
A proposed set of changes:
https://android-review.googlesource.com/q/topic:"no-dup-hwasans"
will cause the HWASAN runtime to be moved from /system/lib64 to
/system/lib64/bootstrap. This causes a problem in the case where libc is built
with HWASAN but init is not built with HWASAN. In this case, libc.so will have
a DT_NEEDED dependency on the HWASAN runtime but init will not. Currently,
init and other bootstrap executables arrange to load bootstrap libraries by
setting rpath, but rpath only has an effect on libraries directly depended
on by the main executable, not libraries indirectly depended on by it. This
means that the loading of the HWASAN runtime will fail.
Instead of relying on rpath to find the bootstrap libraries, modify the
bootstrap linker so that it searches the bootstrap library directory after
searching the rpath.
Change-Id: I297be32e04ecd316ee12b8e694588e1249e2bb89
Creating two huge buffers with std::vector, as it is now, is very
expensive, because it ends up being satisfied with mmap()/munmap().
Instead, we statically allocate a large string buffer while resizing
the other one to be a smaller and more appropriate size so that it can
be placed on the stack.
Note that this does mean that we increase linker data segment by 4KB
and that increases the overall system memory usage by about 400KB.
However, since that additional page is mostly zeroed with only some
ASCII text, it should be fairly easy to compress when it's swapped out
to ZRAM.
Test: Run with strace and observe no mmap/munmap around readlinkat().
Test: Measure average linker time of starting cameraserver on a Go
device and see ~3% speed-up.
Bug: 132783386
Change-Id: I600cc7a92be316ef67440a9a28c05d18de229f6c
Given that we have both linker and linker64, I didn't really want to have
to have ldd and ldd64, so this change just adds the --list option to the
linkers and a shell script wrapper "ldd" that calls the appropriate
linker behind the scenes.
Test: adb shell linker --list `which app_process32`
Test: adb shell linker64 --list `which date`
Test: adb shell ldd `which app_process32`
Test: adb shell ldd `which date`
Change-Id: I33494bda1cc3cafee54e091f97c0f2ae52d1f74b
Enable native bridge support for bionic libraries.
Makes it possible to use them in binaries for translated
architectures.
Bug: http://b/77159578
Test: make
Change-Id: Iccd4ad7aecfa5260cc15f09ca975d2e18987278a
The libstdc++ directory has no copyright headers, so it was a no-op
anyway.
The interesting part will be switching libc and libm over to genrules...
Test: N/A
Change-Id: Iec92562af40c451fdcb4a7468984878ec5dba2ce
ANDROID_DLEXT_WRITE_RELRO was inadvertently writing out the RELRO
section of all libraries loaded during a given dlopen() call instead of
only the main library; since the other libraries are loaded at
unpredictable addresses this additional data is rarely useful.
Fix this to only happen when the
ANDROID_DLEXT_RESERVED_ADDRESS_RECURSIVE flag is being used.
Bug: 128623590
Test: DlExtRelroSharingTest.CheckRelroSizes
Change-Id: I05e8651d06ce2de77b8c85fe2b6238f9c09691ad
ANDROID_DLEXT_WRITE_RELRO was causing the GNU RELRO sections of
libraries to become corrupted if more than one library was being loaded
at once (i.e. if the root library has DT_NEEDED entries for libraries
that weren't already loaded). The file offset was not being correctly
propagated between calls, so after writing out the (correct) RELRO data
to the file, it was mapping the data at file offset 0 for all libraries,
which corrupted the data for all but one of the libraries.
Fix this by passing file_offset as a pointer the same way that
phdr_table_map_gnu_relro does.
Bug: 128623590
Test: tbd
Change-Id: I196cd336bd5a67454e89fd85487356b1c7856871
Also update an rvalue reference to lvalue reference where it's
unnecessary to make it clearer.
Test: Build and boot cuttlefish.
Change-Id: Ib799666ab075523e6446f34c7da2b1eb311f98ab
In order to enable no-vendor-variant VNDK, we need a way to restrict a
namespace to only a list of whitelisted libraries. We add a new
"whitelisted" property for this. If the property is not set, all
libraries in the search paths are available in a namespace. If the
property is set, only the libraries named are available.
Bug: 119423884
Test: Boot with no-vendor-variant VNDK enabled using the new property.
Change-Id: Id808c1733c8e2c2c3462b04c72461f9698403571
- Show which executable is being linked, which linker config file is
being read, and which section in it is being used with, enabled on
$LD_DEBUG>=1.
- Show more info to follow the dlopen() process, enabled with "dlopen"
in the debug.ld.xxx property.
Test: Flash, boot, and look at logcat after "adb shell setprop debug.ld.all dlopen"
Bug: 120430775
Change-Id: I5441c8ced26ec0e2f04620c3d2a1ae860b792154
The *.mountpoint targets that installs /bionic/lib/lib*.so and
/bionic/bin/linker* are no longer needed.
Now, /system/lib/lib*.so and /system/bin/linker* are simply symlinks to
the corresponding files in the runtime apex. For example,
/system/lib/libc.so -> /apex/com.android.runtime/lib/bionic/libc.so
This is made possible because we now activate APEXes even before the
data partition is mounted. Before the data partition mounting, the
APEXes from the system partition are ativated. After the data partition
is mounted, updated APEXes in the partition (if any) are activated. As a
result, the symlink always points to the valid path regardless of
whether /data is mounted or not.
Bug: 125549215
Test: device boots
Change-Id: Ie7d83686abe00b3c436f9f9db75d4244200a0fc9
Make the num_free_blocks calculation clearer. There are
sizeof(page->bytes) bytes available for blocks, which is equal to
(kAllocateSize - 16). The previous calculation overestimates the amount
of space by 8 or 12 bytes, but that doesn't change the result because
both kAllocateSize and block_size_ are multiples of 16.
Bug: none
Test: boot device, bionic unit tests
Change-Id: Ia880cd5abc2a7bb4e9e6a8c0441d985f4df8fdc4
The tlsdesc_resolver_dynamic function is only defined on arm64, but the
code was taking its address on all targets. (Apparently the optimizer was
removing the entire deferred_tlsdesc_relocs loop?)
Bug: none
Test: bionic unit tests
Test: add -O0 to cc_defaults in linker/Android.bp, device boots
Change-Id: I899c78018cbe82aeaca7856c9f7b04300c1d3196
It's possible and sometimes beneficial for a library to have disjoint mappings
and for other libraries to be mapped into the gap between the mappings using
ANDROID_DLEXT_RESERVED_ADDRESS. See for example the proposal for partitioning
in lld [1].
Because the find_containing_library and do_dl_unwind_find_exidx functions use
a simple bounds check to figure out whether a pointer belongs to a library
they will, given a pointer into a library mapped into the gap of a library
with disjoint mappings, return a pointer to the soinfo for the outer library
instead of the inner one, because the outer library will appear before the
inner one in the solist.
From a user perspective this means that we won't be able to unwind the inner
library's frames on 32-bit ARM with libgcc, dladdr() will return information
for the outer library given a pointer to the inner one and dlopen() et al will
use the linker namespace of the outer library when called from the inner one
(although they will usually be the same).
To make this work correctly, make it so that once find_containing_library
sees a match for the bounds check, it examines the library's PT_LOADs to
make sure that there is a mapping for the given address. This is similar
to how libgcc and libunwind_llvm already handle finding the PT_GNU_EH_FRAME
on non-ARM32 platforms [2,3]. do_dl_unwind_find_exidx is reimplemented in
terms of find_containing_library.
[1] http://lists.llvm.org/pipermail/llvm-dev/2019-February/130583.html
[2] e739ac0e25/libunwind/src/AddressSpace.hpp (L523)
[3] https://android.googlesource.com/toolchain/gcc/+/master/gcc-4.9/libgcc/unwind-dw2-fde-dip.c#294
Test: /data/nativetest{,64}/bionic-unit-tests/bionic-unit-tests on walleye-userdebug
Change-Id: I368fe6ad3c470b3dff80f7d9b04253566d63a7d2
Introduce a new flag ANDROID_DLEXT_RESERVED_ADDRESS_RECURSIVE which
instructs the linker to use the reserved address space to load all of
the newly-loaded libraries required by a dlopen() call instead of only
the main library. They will be loaded consecutively into that region if
they fit. The RELRO sections of all the loaded libraries will also be
considered for reading/writing shared RELRO data.
This will allow the WebView implementation to potentially consist of
more than one .so file while still benefiting from the RELRO sharing
optimisation, which would otherwise only apply to the "root" .so file.
Test: bionic-unit-tests (existing and newly added)
Bug: 110790153
Change-Id: I61da775c29fd5017d9a1e2b6b3757c3d20a355b3
Life is easier if we just keep test code in the same directory as the
stuff it's testing...
Test: tests still build and pass
Change-Id: I9b35d689098bdc28a71d69645b0ca9fdd6ea0108
The log priorities and ids are in an NDK header, available to everyone.
Move CHECK into its own header for now. This would be better if it was
more like the <android-base/logging.h> CHECK family, but I don't have an
easy way to do that without lots of copy & paste, so punting for now.
Bug: https://issuetracker.google.com/issues/119713191
Test: boots
Change-Id: I4566be8a0a024fede0e2d257c98b908ec67af2a8
Bug: http://b/116873221
The coverage library, which depends on libc, cannot link with libraries
since they opt of libc.
Test: m NATIVE_COVERAGE=true COVERAGE_PATHS=bionic
Change-Id: Iaaca8bc30c8ba59e5d4f7092b1a0fce8e8c14c72
The change is a workaround of app compat issue until the app fixes the
issue.
Likely, the app does not only dlopen(), but open() the libicuuc.so
with the absolute path.
Note that platform uses the stub libandroidicu.so, not libicuuc directly.
This CL topic moves (not duplicates) libandroidicu from /system to
/apex. Even though /system/lib/libicuuc.so is kept in the default
namespace, but no one should be using it.
Some debug info after this CL:
$ adb shell cat /proc/<zygote pid>/maps | grep libicuuc
$ adb shell cat /proc/<app pid>/maps | grep libicuuc
Only /apex/com.android.runtime/lib64/libicuuc.so is found, not the
/system.
Bug: 124218500
Test: The app can be launched successfully
Test: CtsJniTestCases
Change-Id: I67a50f0ad934be567a63d4038215546c9b4e5f7e
Some of the greylist libraries can be in directories other than
/system/lib. For example, libnativehelper.so is moved from the directory
to /apex/com.android.runtime/lib.
Previous behavior for greylist libraries is to find the lib under
/system/lib while "within" the current namespace which is usually the
classloader-namespace. Since the libs no longer exist there, linker
fails to load it and no further attempt is made.
This change fixes the problem by loading the greylist libs "from" the
default namespace. Since there is a link from the default namespace to
the namespace where the moved libraries are intended to be loaded (e.g.,
the 'runtime' namespace for libnativehelper.so), the call is successful.
Bug: 124201397
Test: bionic-unit-tests
Change-Id: I11d15993d18896bdf663c408f9e40d8a607c9b80
When the linker is invoked on itself, (`linker64 /system/bin/linker64`),
the linker prints an error, because self-invocation isn't allowed. The
current method for detecting self-invocation fails because the second
linker instance can crash in a constructor function before reaching
__linker_init.
Fix the problem by moving the error check into a constructor function,
which finishes initializing libc sufficiently to call async_safe_fatal.
The only important thing missing is __libc_sysinfo on 32-bit x86. The aux
vector isn't readily accessible, so use the fallback int 0x80.
Bug: http://b/123637025
Test: bionic unit tests (32-bit x86)
Change-Id: I8be6369e8be3938906628ae1f82be13e6c510119
This reverts commit 1906e77665
and fixes dlopen(nullptr)
It's enabled for the following .so libraries
libicuuc.so
libicui18n.so
This CL does not move the .so files into a APEX.
Thus, no visible effect before moving .so files.
Bug: 121248172
Test: dlfcn#dlopen_from_nullptr
Test: Perform the following test after moving libicuuc to /apex
dlopen("/system/lib64/libicuuc.so") for targetSdkVersion < Q
Change-Id: Ia310ea566485fe00ca28e2203f4bffbd92243638
It's enabled for the following .so libraries
libicuuc.so
libicui18n.so
This CL does not move the .so files into a APEX.
Thus, no visible effect before moving .so files.
Bug: 121248172
Test: Perform the following test after moving libicuuc to /apex
dlopen("/system/lib64/libicuuc.so") for targetSdkVersion < Q
Change-Id: Ic00d281d59de924c01582c48d2a946e3f5f33a09
This is the second attempt to purge linker block allocators. Unlike the
previously reverted change which purge allocators whenever all objects
are freed, we only purge right before control leaves the linker. This
limits the performance impact to one munmap() call per dlopen(), in
most cases.
Bug: 112073665
Test: Boot and check memory usage with 'showmap'.
Test: Run camear cold start performance test.
Change-Id: I02c7c44935f768e065fbe7ff0389a84bd44713f0
The dynamic linker installed to /system partition is only for
bootstraping, i.e. to be used only by early processes. The existing path
/system/bin/linker[64] will be a symlink to the mount pointpoint
/bionic/bin/linker[64] which will be bind-mounted with either the
bootstrap linker or the default linker in the runtime APEX. Since the
path is taken by the symlink, bootstrap linker is moved to ./boostrap
subdirectory.
Bug: 120266448
Test: m linker; the linker is found under /system/bin/bootstrap/
Change-Id: I163e0c8a1738b94565a6b60f48e65841ba058a91
This change adds following files and symlinks:
Files:
/bionic/lib[64]/lib{c|dl|m}.so
/bionic/bin/linker[64]
Symlinks:
/system/lib[64]/lib{c|dl|m}.so -> /bionic/lib[64]/lib{c|dl|m}.so
/system/bin/linker[64] -> /bionic/bin/linker[64]
/system/bin/linker_asan[64] -> /bionic/bin/linker[64]
The files serve as mount points for either the bootstrap Bionic or the
default Bionic from the runtime APEX. init does the bind-mounting during
booting.
The symlinks are there to not change the ordinary paths to the bionic
files; there are many places that the paths are implied or hard-coded,
e.g., dlopen("/system/lib/libc.so") or DT_INTERP pointing to
/system/bin/linker in the vendor prebuilts.
Bug: 120266448
Test: m blueline, cf_x86, aosp_arm
The aforementioned files and symlinks are found
Change-Id: I97e38c29409ac0610dde285db8df6e94a7930094
Each TLSDESC relocation relocates a 2-word descriptor in the GOT that
contains:
- the address of a TLS resolver function
- an argument to pass (indirectly) to the resolver function
(Specifically, the address of the 2-word descriptor is passed to the
resolver.)
The loader resolves R_GENERIC_TLSDESC relocations using one of three
resolver functions that it defines:
- tlsdesc_resolver_static
- tlsdesc_resolver_dynamic
- tlsdesc_resolver_unresolved_weak
The resolver functions are written in assembly because they have a
restrictive calling convention. They're only allowed to modify x0 and
(apparently) the condition codes.
For a relocation to memory in static TLS (i.e. the executable or an solib
loaded initially), the loader uses a simple resolver function,
tlsdesc_resolver_static, that returns the static offset it receives from
the loader.
For relocations to dynamic TLS memory (i.e. memory in a dlopen'ed solib),
the loader uses tlsdesc_resolver_dynamic, which allocates TLS memory on
demand. It inlines the fast path of __tls_get_addr, then falls back to
__tls_get_addr when it needs to allocate memory. The loader handles these
dynamic TLS relocations in two passes:
- In the first pass, it allocates a table of TlsDynamicResolverArg
objects, one per dynamic TLSDESC relocation.
- In the second pass, once the table is finalized, it writes the
addresses of the TlsDynamicResolverArg objects into the TLSDESC
relocations.
tlsdesc_resolver_unresolved_weak returns a negated thread pointer so that
taking the address of an unresolved weak TLS symbols produces NULL.
The loader handles R_GENERIC_TLSDESC in a target-independent way, but
only for arm64, because Bionic has only implemented the resolver functions
for arm64.
Bug: http://b/78026329
Test: bionic unit tests
Test: check that backtrace works inside a resolver function and inside
__tls_get_addr called from a resolver
(gdbclient.py, b __tls_get_addr, bt)
Merged-In: I752e59ff986292449892c449dad2546e6f0ff7b6
Change-Id: I752e59ff986292449892c449dad2546e6f0ff7b6
Initialize a thread's DTV to an empty zeroed DTV. Allocate the DTV and
any ELF module's TLS segment on-demand in __tls_get_addr. Use a generation
counter, incremented in the linker, to signal when threads should
update/reallocate their DTV objects.
A generation count of 0 always indicates the constant zero DTV.
Once a DTV is allocated, it isn't freed until the thread exits, because
a signal handler could interrupt the fast path of __tls_get_addr between
accessing the DTV slot and reading a field of the DTV. Bionic keeps a
linked list of DTV objects so it can free them at thread-exit.
Dynamic TLS memory is allocated using a BionicAllocator instance in
libc_shared_globals. For async-signal safety, access to the
linker/libc-shared state is protected by first blocking signals, then by
acquiring the reader-writer lock, TlsModules::rwlock. A write lock is
needed to allocate or free memory.
In pthread_exit, unconditionally block signals before freeing dynamic
TLS memory or freeing the shadow call stack.
ndk_cruft.cpp: Avoid including pthread_internal.h inside an extern "C".
(The header now includes a C++ template that doesn't compile inside
extern "C".)
Bug: http://b/78026329
Bug: http://b/123094171
Test: bionic unit tests
Change-Id: I3c9b12921c9e68b33dcc1d1dd276bff364eff5d7
Generalize the omitted symbol and missing-TLS-segment behaviors to all TLS
relocations.
R_GENERIC_TLS_DTPMOD is a module ID, which starts at 1 for the executable.
R_GENERIC_TLS_DTPREL is an offset from the start of a module to a specific
TLS symbol.
binutils currently disagrees with Bionic about the values of
R_AARCH64_TLS_DTPMOD64 and R_AARCH64_TLS_DTPREL64, so disable
DTPMOD/DTPREL for now on arm64.
Bug: http://b/78026329
Test: bionic unit tests (Tests for dynamic TLS will be added later)
Change-Id: I05c28d6a1036bdd6127f605036679b7475689445
These dynamic table entries are used for lazy relocation of TLSDESC
relocations. Bionic relocates everything eagerly and can ignore these
entries.
Bug: http://b/78026329
Test: bionic unit tests
Test: manual
Change-Id: I788c4a95041fae66de2fbeab3babe0205631999c
Rename LinkerMemoryAllocator -> BionicAllocator
Rename LinkerSmallObjectAllocator -> BionicSmallObjectAllocator
libc and the linker need to share an instance of the allocator for
allocating and freeing dynamic ELF TLS memory (DTVs and segments). The
linker also continues to use this allocator.
Bug: http://b/78026329
Test: /data/nativetest/bionic-unit-tests-static
Test: /data/nativetest64/bionic-unit-tests-static
Test: /data/nativetest/linker-unit-tests/linker-unit-tests32
Test: /data/nativetest64/linker-unit-tests/linker-unit-tests64
Change-Id: I2da037006ddf8041a75f3eba2071a8fcdcc223ce
The <async_safe/log.h> include in linker_allocator.h was made
unnecessary when the STL dependency was removed.
Replace the linker_*.h header includes with private/bionic_*.h
includes.
This change prepares LinkerMemoryAllocator for use in libc allocating
dynamic ELF TLS memory.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: I6ad62bd513e71dd3f91a4b724a92b19a1eb6c3b4
Suppress the SELinux denial log spam by not calling realpath(3) when the
path does not exist or is not accessible for read, and then not auditing
access(2) failure.
Bug: 120996057
Test: copy ping to /data/local/tmp, run it, verify no errors
Test: run bionic-unit-tests, the tests pass
Change-Id: Ie6058bfc9524a9b5c50fc7183fdddea6a8fb9200
In addition to ORIGIN, LIB is now expanded to either lib or lib64
depending on the ABI in the DT_RUNPATH.
Test: To be added (can't be added to AOSP directly since there are
internal tests that have to be updated as well)
Change-Id: I34a8316f1765a27c731bc270ee25c0ce287d7eed
Given that the block allocator does not free allocated memory unless
all objects are freed, we can allocate a large chunk of memory at a
time. This gives us:
1. Higher space efficiency, especially when the objects being
allocated has a large size.
2. Much less mmap()/prctl() calls.
3. Much less vm_area_struct objects.
While this increases the size of virtual memory allocated, the number
of dirty pages would be unchanged or less.
Test result on a Go device:
- Zygote starts 24ms faster on average.
- Kernel vm_area_struct usage 138KB less, across the entire system.
Bug: 112073665
Test: Boot and check the result from 'showmap'.
Change-Id: Iece631a5081151dd1e5bf25dbddac9b264bcdcff
If all allocated memory from a block allocator is freed, it is a good
opportunity to purge all the pages allocated to reduce lingering dirty
pages.
Memory saving varies with the platform and what processes are running.
Measuring right after boot, this saves ~1.8MB on cuttelfish and ~1.3MB
on a 32-bit ARM device.
Bug: 112073665
Test: Boot and check memory usage with 'showmap'.
Change-Id: I53769e0ec9699f0b3645cdf281a2c0bbffb98676
The field was pointing into an element of an std::vector, but the address
of a vector element is invalidated when the vector is resized.
This bug was caught by the new elftls.shared_ie and
elftls_dl.dlopen_shared_var_ie tests.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: I7232f6d703a9e339fe8966a95b7a68bae2c9c420
* changes:
Handle R_GENERIC_TLS_TPREL relocations
Avoid a dlopen abort on an invalid TLS alignment
Initialize static TLS memory using module list
Record TLS modules and layout static TLS memory
StaticTlsLayout: add exe/tcb and solib layout
Avoids some unnecessary work during dlopen/dlclose/dlsym for most
cases.
Bug: 122471935
Test: fewer page faults during app startup
Change-Id: Ie886e1e671066af3c6f3a895f9a8126f209d6660
This relocation is used for static TLS's initial-exec (IE) accesses.
A TLS symbol's value is its offset from the start of the ELF module's
TLS segment. It doesn't make sense to add the load_bias to this value,
so skip the call to soinfo::resolve_symbol_address.
Allow TLS relocations to refer to an unresolved weak symbol. In that case,
sym will be non-zero, but lsi will be nullptr. The dynamic linker resolves
the TPREL relocation to 0, making &missing_weak_symbol equal the thread
pointer.
Recognize Gold-style relocations to STB_LOCAL TLS symbols/sections and
issue an error.
Remove the "case R_AARCH64_TLS_TPREL64", because the R_GENERIC_TLS_TPREL
case handles it.
Remove the no-op R_AARCH64_TLSDESC handler. It's better to issue an error.
dlopen_library_with_ELF_TLS now fails with a consistent error about an
unimplemented dynamic TLS relocation.
Bug: http://b/78026329
Test: bionic unit tests (elftls tests are added in a later CL)
Change-Id: Ia08e1b5c8098117e12143d3b4ebb4dfaa5ca46ec
If the alignment of a TLS segment in a shared object is invalid, return
an error through dlerror() rather than aborting the process.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: I60e589ddd8ca897f485d55af089f08bd3ff5b1fa
It can happen e.g. if android_dlopen_ext is called with an absolute path to
a file that is not accessible in the current namespace. The first
load_library call in find_library_internal will then open the file and
assign its fd to the task and return false. Then linked namespaces are
searched, and load_library gets called again and opens the same file and
overwrites the fd in the task without closing it first. (In one of the later
calls the namespace config might very well allow the file to be loaded and
the android_dlopen_ext call eventually returns successfully, and the process
continues with the leaked fd.)
The code could perhaps be changed to avoid opening the file repeatedly in
these cases, but the LoadTask class should arguably keep its state clean
anyway.
Bug: 113373927
Test: Flash and boot device with (and without) http://r.android.com/812674,
which moves libart.so to the runtime namespace and thus makes it load
/system/framework/*/boot*.oat files across the namespace boundary from
runtime to default.
Change-Id: Iae91b7c743c5f3f973506153ba52898ae72e6fee
Replace reserve_tcb with reserve_exe_segment_and_tcb, which lays out both
the TCB and the executable's TLS segment, accounting for the difference in
layout between variant 1 and variant 2 targets.
The function isn't actually called with a non-null TlsSegment* yet.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: Ibd6238577423a7d0451f36da7e64912046959796
This macro is necessary for using stdatomic.h and <atomic> in the same
libc++-based program. <atomic> is used implicitly by other STL headers,
and stdatomic.h is used (or will be used) in headers shared with libc.
libc only has access to stdatomic.h because it can't use an STL.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: I89b128df08b478287ef4e2867319e74dbcc30d4e
For ELF TLS "local-exec" accesses, the static linker assumes that an
executable's TLS segment is located at a statically-known offset from the
thread pointer (i.e. "variant 1" for ARM and "variant 2" for x86).
Because these layouts are incompatible, Bionic generally needs to allocate
its TLS slots differently between different architectures.
To allow per-architecture TLS slots:
- Replace the TLS_SLOT_xxx enumerators with macros. New ARM slots are
generally negative, while new x86 slots are generally positive.
- Define a bionic_tcb struct that provides two things:
- a void* raw_slots_storage[BIONIC_TLS_SLOTS] field
- an inline accessor function: void*& tls_slot(size_t tpindex);
For ELF TLS, it's necessary to allocate a temporary TCB (i.e. TLS slots),
because the runtime linker doesn't know how large the static TLS area is
until after it has loaded all of the initial solibs.
To accommodate Golang, it's necessary to allocate the pthread keys at a
fixed, small, positive offset from the thread pointer.
This CL moves the pthread keys into bionic_tls, then allocates a single
mapping per thread that looks like so:
- stack guard
- stack [omitted for main thread and with pthread_attr_setstack]
- static TLS:
- bionic_tcb [exec TLS will either precede or succeed the TCB]
- bionic_tls [prefixed by the pthread keys]
- [solib TLS segments will be placed here]
- guard page
As before, if the new mapping includes a stack, the pthread_internal_t
is allocated on it.
At startup, Bionic allocates a temporary bionic_tcb object on the stack,
then allocates a temporary bionic_tls object using mmap. This mmap is
delayed because the linker can't currently call async_safe_fatal() before
relocating itself.
Later, Bionic allocates a stack-less thread mapping for the main thread,
and copies slots from the temporary TCB to the new TCB.
(See *::copy_from_bootstrap methods.)
Bug: http://b/78026329
Test: bionic unit tests
Test: verify that a Golang app still works
Test: verify that a Golang app crashes if bionic_{tls,tcb} are swapped
Merged-In: I6543063752f4ec8ef6dc9c7f2a06ce2a18fc5af3
Change-Id: I6543063752f4ec8ef6dc9c7f2a06ce2a18fc5af3
(cherry picked from commit 1e660b70da)
When parsing a dir.${section} line in ld.config.txt, the linker uses
realpath() to resolve each directory path. If realpath() fails, the linker
uses the non-resolved path instead. Previously, it issued a warning to
stderr and logcat, but realpath() is expected to fail on /postinstall when
running a binary in /data/local/tmp, so downgrade the warning to an INFO()
notice.
Bug: http://b/120996057
Test: copy ping to /data/local/tmp, run it, verify no errors
Test: bionic unit tests
Change-Id: I2104084e12ea18840230997408559299bf3bddac
This includes:
- Blocks allocated by small object allocators are now aligned to
their block size.
- Remove redundant free_blocks_cnt and allocated_blocks_cnt, as they
sum up to a fixed number.
- Remove a redundant padding word in small_object_page_info on 32-bit
platform.
Test: Build and boot cuttlefish.
Change-Id: Ib922d7af739e3709e8a162c7f68e7f99bf95a914
The current implementation of LinkerSmallObjectAllocator keeps record
of pages in a vector, which uses its own page(s). This is at least a
page overhead per LinkerSmallObjectAllocator.
This change removes the page record vector by managing the pages in a
doubly linked list.
We also fix a bug where we are actually keeping up to 2 free pages
instead of just one.
The memory used by small objects when running 'dd', before this change:
72 KB [anon:linker_alloc_small_objects]
28 KB [anon:linker_alloc_vector]
After this change:
60 KB [anon:linker_alloc_small_objects]
Test: Boot cuttlefish and check memory used by linker.
Change-Id: I3468fa4d853c78b4bc02bfb84a3531653f74fb17
Instead of passing the address of a KernelArgumentBlock to libc.so for
initialization, use __loader_shared_globals() to initialize globals.
Most of the work happened in the previous CLs. This CL switches a few
KernelArgumentBlock::getauxval calls to [__bionic_]getauxval and stops
routing the KernelArgumentBlock address through the libc init functions.
Bug: none
Test: bionic unit tests
Change-Id: I96c7b02c21d55c454558b7a5a9243c682782f2dd
Merged-In: I96c7b02c21d55c454558b7a5a9243c682782f2dd
(cherry picked from commit 746ad15912)
Split __libc_init_main_thread into __libc_init_main_thread_early and
__libc_init_main_thread_late. The early function is called very early in
the startup of the dynamic linker and static executables. It initializes
the global auxv pointer and enough TLS memory to do system calls, access
errno, and run -fstack-protector code (but with a zero cookie because the
code for generating a cookie is complex).
After the linker is relocated, __libc_init_main_thread_late finishes
thread initialization.
Bug: none
Test: bionic unit tests
Change-Id: I6fcd8d7587a380f8bd649c817b40a3a6cc1d2ee0
Merged-In: I6fcd8d7587a380f8bd649c817b40a3a6cc1d2ee0
(cherry picked from commit 39bc44bb0e)
It was discovered that we were building some objects inconsistently due
to an optimization in cc_library to only build objects once and use them
for both the static and shared libraries. But static libraries didn't
get system_shared_libs set automatically, and we didn't notice that we
would have built the objects differently.
So static libraries now get the default system_shared_libs, we allow
adjusting that for static vs shared in a cc_library, and we disable the
optimization if the linked libraries are configured differently between
static and shared in a single cc_library.
This triggers dependency cycles for static libraries that libc/libdl
use, so fix those cycles here.
Test: treehugger
Change-Id: I3cf7fda161a05ec32e0c1e871999720d12a4d38e
* changes:
Use shared globals to init __progname + environ
Move the abort message to libc_shared_globals
Expose libc_shared_globals to libc.so with symbol
Initialize the __progname and environ global variables using
libc_shared_globals rather than KernelArgumentBlock.
Also: suppose the linker is invoked on an executable:
linker prog [args...]
The first argument passed to main() and constructor functions is "prog"
rather than "linker". For consistency, this CL changes the BSD
__progname global from "linker" to "prog".
Bug: none
Test: bionic unit tests
Change-Id: I376d76953c9436706dbc53911ef6585c1acc1c31
__libc_shared_globals() is available in dynamic modules as soon as
relocation has finished (i.e. after ifuncs run). Before ifuncs have run,
the android_set_abort_message() function already doesn't work because it
calls public APIs via the PLT. (If this matters, we can use a static
bool variable to enable android_set_abort_message after libc
initialization).
__libc_shared_globals() is hidden, so it's available in the linker
immediately (i.e. before relocation). TLS memory (e.g. errno) currently
isn't accessible until after relocation, but a later patch fixes that.
Bug: none
Test: bionic unit tests
Change-Id: Ied4433758ed2da9ee404c6158e319cf502d05a53
Previously, the address of the global variable was communicated from the
dynamic linker to libc.so using a field of KernelArgumentBlock, which is
communicated using the TLS_SLOT_BIONIC_PREINIT slot.
As long as this function isn't called during relocations (i.e. while
executing an ifunc), it always return a non-NULL value. If it's called
before its PLT entry is relocated, I expect a crash.
I removed the __libc_init_shared_globals function. It's currently empty,
and I don't think there's one point in libc's initialization where
shared globals should be initialized.
Bug: http://b/25751302
Test: bionic unit tests
Change-Id: I614d25e7ef5e0d2ccc40d5c821dee10f1ec61c2e
Free pages count should be incremented by 1 when a new page is
allocated. Without this fix, free pages count underflows and the
allocator ends up freeing a free page whenever the last object in that
page is freed. In other words, it doesn't hold onto a free page as
expected and thus we may see more mmap/munmap calls.
Test: Set breakpoint at the end of __linker_init and check
free_pages_cnt values are either 0 or 1.
Change-Id: I259a3a27329aab6835c21b4aa7ddda89dac9655b
These modules have issues running with execute-only memory. Disable it
in them until we can resolve the issues.
Bug: 77958880
Test: No more XOM-related crashes in these binaries
Change-Id: Ie6c957731155566c2bbe7dbb7a91b9583d9aff93
__sanitize_environment_variables is only called when getauxval(AT_SECURE)
is true.
Instead of scanning __libc_auxv, reuse getauxval. If the entry is missing,
getauxval will set errno to ENOENT.
Reduce the number of times that __libc_sysinfo and __libc_auxv are
initialized. (Previously, __libc_sysinfo was initialized 3 times for the
linker's copy). The two variables are initialized in these places:
- __libc_init_main_thread for libc.a (including the linker copy)
- __libc_preinit_impl for libc.so
- __linker_init: the linker's copy of __libc_sysinfo is still initialized
twice, because __libc_init_main_thread runs after relocation. A later
CL consolidates the linker's two initializations.
Bug: none
Test: bionic unit tests
Change-Id: I196f4c9011b0d803ee85c07afb415fcb146f4d65
When /foo/bar/baz is added to the search paths and if getattr (stat())
is not allowed on one of its parent paths, i.e., /foo and /foo/baz, the
path was thought as non-existent and wasn't added to the search paths of
the namespace.
Fixing the bug by adding the path if the path (though not the parents)
does exist.
Bug: 119656753
Test: m apex.test; m; device boots.
Change-Id: I21bca1fee9aa20688ce9b72192d3173821ad91a3
Change three things regarding the work around to the fact that init is
special:
1) Only first stage init is special, so we change the check to include
accessing /proc/self/exe, which if is available, means that we're
not first stage init and do not need any work arounds.
2) Fix the fact that /init may be a symlink and may need readlink()
3) Suppress errors from realpath_fd() since these are expected to fail
due to /proc not being mounted.
Bug: 80395578
Test: sailfish boots without the audit generated from calling stat()
on /init and without the errors from realpath_fd()
Change-Id: I266f1486b142cb9a41ec791eba74122bdf38cf12
This reverts commit 73de62e461.
Reason for revert: bug was fixed upstream and included into AOSP lld
Bug: 78115263
Test: make checkbuild
Change-Id: Ib9e4af1efdcdb547c7bd65cce4ce48d7f09ed217
When executing an executable in an APEX (i.e., /apex/<name>/bin),
ld.config.txt file is read from the same APEX, not from /system/etc.
Bug: 115787633
Test: m apex.test; adb push ...apex.test.apex /data/apex; adb reboot
Test: adb root; adb shell /apex/com.android.example.apex/bin/dex2oat
is runnable.
Change-Id: I6400251f99d24f2379dbaf655ecd84da02490617
Use <android-base/macros.h> instead where possible, and move the bionic
macros out of the way of the libbase ones. Yes, there are folks who manage
to end up with both included at once (thanks OpenGL!), and cleaning that
up doesn't seem nearly as practical as just making this change.
Bug: N/A
Test: builds
Change-Id: I23fc544f39d5addf81dc61471771a5438778895b
We don't have a host bionic version of
libclang_rt.asan-x86_64-android.so, so I'm using the android version,
which can't load liblog.so, since it's missing DT_RUNPATH that would
normally load liblog.so from a relative path to the .so.
Bug: 118058804
Test: run ASAN host_bionic
Change-Id: I58badcd5ed35bd1c7b786b4f1e2367a1011ff08d
This is use by Host Bionic to bootstrap into an embedded copy of the
linker by tweaking the AT_* values before calling in to the linker entry.
Similarly to 9729f35922, get the base
address from AT_PHDR, so that we're not relying on the relative offset
before relocation, which doesn't work with lld (at least with the
standard flags).
To find the offset to the linker code, we can still use an absolute
symbol created by extract_linker (which is currently hardcoded to 0x1000).
Instead of relying on something similar for the linker entry point,
we're now just reading the entry point from the linker's ELF header.
Then we get the address to the real _start function using
host_bionic_inject, which injects the value into a global variable after
the link step is finished. It also uses that opportunity to verify that
the linker is embedded as we expect it to be.
Bug: 31559095
Test: build with host bionic
Change-Id: I9d81ea77c51c079de06905da1ebe421fead1dc3b
The executable can be inside a zip file using the same syntax used for
shared objects: path.zip!/libentry.so.
The linker currently requires an absolute path. This restriction could be
loosened, but it didn't seem important? If it allowed non-absolute paths,
we'd need to decide how to handle:
- foo/bar (relative to CWD?)
- foo (search PATH / LD_LIBRARY_PATH, or also relative to CWD?)
- foo.zip!/bar (normalize_path() requires an absolute path)
The linker adjusts the argc/argv passed to main() and to constructor
functions to hide the initial linker argument, but doesn't adjust the auxv
vector or files like /proc/self/{exe,cmdline,auxv,stat}. Those files will
report that the kernel loaded the linker as an executable.
I think the linker_logger.cpp change guarding against (g_argv == NULL)
isn't actually necessary, but it seemed like a good idea given that I'm
delaying initialization of g_argv until after C++ constructors have run.
Bug: http://b/112050209
Test: bionic unit tests
Change-Id: I846faf98b16fd34218946f6167e8b451897debe5
Currently it normalizes the path to a string with a single uninitialized
byte. It should instead normalize it to "/".
Bug: none
Test: /data/nativetest/linker-unit-tests/linker-unit-tests32
Test: /data/nativetest64/linker-unit-tests/linker-unit-tests64
Change-Id: I06e0f7598d16acfa21875dad53efbc293cfeb44d
The argument to CloseArchive has type ZipArchiveHandle, but we're
passing it a ZipArchiveHandle*. The compiler doesn't detect the type
mismatch because ZipArchiveHandle is a typedef for void*.
Remove a duplicate close() call:
The fourth argument to OpenArchiveFd is "bool assume_ownership = true".
Even if the function fails, ownership of the fd is still transferred to
a ZipArchive object that's deleted when this code calls CloseArchive.
AFAIK, this code path is rarely or never hit.
Bug: none
Test: manual (eventually, 'linker64 /system!/foo')
Change-Id: I95d79809b6e118fb3c39c7b98b8055c8e324db1a
