Symbol lookup is O(L) where L is the number of libraries to search (e.g.
in the global and local lookup groups). Factor out the per-DSO work into
soinfo_do_lookup_impl, and optimize for the situation where all the DSOs
are using DT_GNU_HASH (rather than SysV hashes).
To load a set of libraries, the loader first constructs an auxiliary list
of libraries (SymbolLookupList, containing SymbolLookupLib objects). The
SymbolLookupList is reused for each DSO in a load group. (-Bsymbolic is
accommodated by modifying the SymbolLookupLib at the front of the list.)
To search for a symbol, soinfo_do_lookup_impl has a small loop that first
scans a vector of GNU bloom filters looking for a possible match.
There was a slight improvement from templatizing soinfo_do_lookup_impl
and skipping the does-this-DSO-lack-GNU-hash check.
Rewrite the relocation processing loop to be faster. There are specialized
functions that handle the expected relocation types in normal relocation
sections and in PLT relocation sections.
This CL can reduce the initial link time of large programs by around
40-50% (e.g. audioserver, cameraserver, etc). On the linker relocation
benchmark (64-bit walleye), it reduces the time from 131.6ms to 71.9ms.
Bug: http://b/143577578 (incidentally fixed by this CL)
Test: bionic-unit-tests
Change-Id: If40a42fb6ff566570f7280b71d58f7fa290b9343
Validate the list of defined versions explicitly, during library
prelinking, rather than implicitly as part of constructing the
VersionTracker in soinfo::link_image.
Doing the validation upfront allows removing the symbol lookup failure
code paths, which only happen on a library with invalid version
information.
Helps on the walleye 64-bit linker relocation benchmark (146.2ms ->
131.6ms)
Bug: none
Test: bionic unit tests
Change-Id: Id17508aba3af2863909f0526897c4277419322b7
Previously, during a find_libraries call that loaded a library, a
library was prelinked once for each DT_NEEDED reference to the library.
This CL has a negligible effect on the linker relocation benchmark
(146.9ms -> 146.2ms).
Bug: none
Test: bionic unit tests
Change-Id: I385f312b8acf8d35aa0af9722131fe367b5edd9b
The run-time of dl_iterate_phdr can be noticeable for exception-handling.
e.g. I tested extest on the GitHub issue by:
- disabling the dlpi_subs/dlpi_adds-based caching in libgcc
- adding a weaker "load base hint" caching in libgcc that doesn't need
the dlpi_subs/dlpi_adds fields, but still has to iterate over every
module to validate a cache hit
extest throws 10000 exceptions, and I saw a regression from ~1550ms
runtime on Q to about ~1950ms on master. This CL reduces the regression to
about ~1700ms.
Bug: https://github.com/android/ndk/issues/1062
Test: bionic unit tests
Change-Id: I099e97e1a20f5b2aa6737789e49d965170eb85a8
Ordinary executables have a PT_INTERP path of /system/bin/linker[64], but:
- executables using bootstrap Bionic use /system/bin/bootstrap/linker[64]
- ASAN executables use /system/bin/linker_asan[64]
gdb appears to use the PT_INTERP path for debugging the dynamic linker
before the linker has initialized the r_debug module list. If the linker's
l_name differs from PT_INTERP, then gdb assumes that the linker has been
unloaded and searches for a new solib using the linker's l_name path.
gdb may print a warning like:
warning: Temporarily disabling breakpoints for unloaded shared library "$OUT/symbols/system/bin/linker64"
If I'm currently debugging the linker when this happens, gdb apparently
doesn't load debug symbols for the linker. This can be worked around with
gdb's "sharedlibrary" command, but it's better to avoid it.
Previously, when PT_INTERP was the bootstrap linker, but l_name was
"/system/bin/linker[64]", gdb would find the default non-bootstrap linker
binary and (presumably) get confused about symbol addresses.
(Also, remove the "static std::string exe_path" variable because the
soinfo::realpath_ field is a std::string that already lasts until exit. We
already use it for link_map_head.l_name in notify_gdb_of_load.)
Bug: http://b/134183407
Test: manual
Change-Id: I9a95425a3a5e9fd01e9dd272273c6ed3667dbb9a
Introduce a new flag ANDROID_DLEXT_RESERVED_ADDRESS_RECURSIVE which
instructs the linker to use the reserved address space to load all of
the newly-loaded libraries required by a dlopen() call instead of only
the main library. They will be loaded consecutively into that region if
they fit. The RELRO sections of all the loaded libraries will also be
considered for reading/writing shared RELRO data.
This will allow the WebView implementation to potentially consist of
more than one .so file while still benefiting from the RELRO sharing
optimisation, which would otherwise only apply to the "root" .so file.
Test: bionic-unit-tests (existing and newly added)
Bug: 110790153
Change-Id: I61da775c29fd5017d9a1e2b6b3757c3d20a355b3
Each TLSDESC relocation relocates a 2-word descriptor in the GOT that
contains:
- the address of a TLS resolver function
- an argument to pass (indirectly) to the resolver function
(Specifically, the address of the 2-word descriptor is passed to the
resolver.)
The loader resolves R_GENERIC_TLSDESC relocations using one of three
resolver functions that it defines:
- tlsdesc_resolver_static
- tlsdesc_resolver_dynamic
- tlsdesc_resolver_unresolved_weak
The resolver functions are written in assembly because they have a
restrictive calling convention. They're only allowed to modify x0 and
(apparently) the condition codes.
For a relocation to memory in static TLS (i.e. the executable or an solib
loaded initially), the loader uses a simple resolver function,
tlsdesc_resolver_static, that returns the static offset it receives from
the loader.
For relocations to dynamic TLS memory (i.e. memory in a dlopen'ed solib),
the loader uses tlsdesc_resolver_dynamic, which allocates TLS memory on
demand. It inlines the fast path of __tls_get_addr, then falls back to
__tls_get_addr when it needs to allocate memory. The loader handles these
dynamic TLS relocations in two passes:
- In the first pass, it allocates a table of TlsDynamicResolverArg
objects, one per dynamic TLSDESC relocation.
- In the second pass, once the table is finalized, it writes the
addresses of the TlsDynamicResolverArg objects into the TLSDESC
relocations.
tlsdesc_resolver_unresolved_weak returns a negated thread pointer so that
taking the address of an unresolved weak TLS symbols produces NULL.
The loader handles R_GENERIC_TLSDESC in a target-independent way, but
only for arm64, because Bionic has only implemented the resolver functions
for arm64.
Bug: http://b/78026329
Test: bionic unit tests
Test: check that backtrace works inside a resolver function and inside
__tls_get_addr called from a resolver
(gdbclient.py, b __tls_get_addr, bt)
Merged-In: I752e59ff986292449892c449dad2546e6f0ff7b6
Change-Id: I752e59ff986292449892c449dad2546e6f0ff7b6
Initialize a thread's DTV to an empty zeroed DTV. Allocate the DTV and
any ELF module's TLS segment on-demand in __tls_get_addr. Use a generation
counter, incremented in the linker, to signal when threads should
update/reallocate their DTV objects.
A generation count of 0 always indicates the constant zero DTV.
Once a DTV is allocated, it isn't freed until the thread exits, because
a signal handler could interrupt the fast path of __tls_get_addr between
accessing the DTV slot and reading a field of the DTV. Bionic keeps a
linked list of DTV objects so it can free them at thread-exit.
Dynamic TLS memory is allocated using a BionicAllocator instance in
libc_shared_globals. For async-signal safety, access to the
linker/libc-shared state is protected by first blocking signals, then by
acquiring the reader-writer lock, TlsModules::rwlock. A write lock is
needed to allocate or free memory.
In pthread_exit, unconditionally block signals before freeing dynamic
TLS memory or freeing the shadow call stack.
ndk_cruft.cpp: Avoid including pthread_internal.h inside an extern "C".
(The header now includes a C++ template that doesn't compile inside
extern "C".)
Bug: http://b/78026329
Bug: http://b/123094171
Test: bionic unit tests
Change-Id: I3c9b12921c9e68b33dcc1d1dd276bff364eff5d7
The field was pointing into an element of an std::vector, but the address
of a vector element is invalidated when the vector is resized.
This bug was caught by the new elftls.shared_ie and
elftls_dl.dlopen_shared_var_ie tests.
Bug: http://b/78026329
Test: bionic unit tests
Change-Id: I7232f6d703a9e339fe8966a95b7a68bae2c9c420
* Initialize the exe's l_ld correctly, and initialize its l_addr field
earlier.
* Copy the phdr/phnum fields from the linker's temporary soinfo to its
final soinfo. This change ensures that dl_iterate_phdr shows the phdr
table for the linker.
* Change init_linker_info_for_gdb a little: use an soinfo's fields to
init the soinfo::link_map_head field, then reuse the new
init_link_map_head function to handle the linker and the executable.
Test: manual
Test: bionic-unit-tests
Bug: https://issuetracker.google.com/112627083
Bug: http://b/110967431
Change-Id: I40fad2c4d48f409347aaa1ccb98d96db89da1dfe
The tls_nodelete state should apply to load_group not
isolated soinfo. This actually also means that multiple
soinfos may have tls_counter on their dso_handles.
This change replaces TLS_NODELETE flag with secondary counter.
Note that access to the secondary counter (located inside soinfo)
is pretty expensive because it requires soinfo lookup by dso_handle
whereas dso_handle counter is much faster. This is why it is updated
only when dso_handle counter starts or hits 0.
Bug: http://b/80278285
Test: bionic-unit-tests --gtest_filter=dl*
Change-Id: I535583f6714e45fa2a7eaf7bb3126da20ee7cba9
We've been using #pragma once for new internal files, but let's be more bold.
Bug: N/A
Test: builds
Change-Id: I7e2ee2730043bd884f9571cdbd8b524043030c07
This change modifies the encoding used in SHT_RELR sections to a simpler
version that gives better results. This encoding was suggested by Andrew
Grieve and is described in this post on generic-abi@googlegroups.com:
https://groups.google.com/d/msg/generic-abi/bX460iggiKg/Pi9aSwwABgAJ
Bug: None
Test: Built image for marlin, flashed on device, ran arm and
aarch64 binaries containing '.relr.dyn' sections using
the new encoding.
Change-Id: I266affe0fbad91dc375995985a221cb02499447b
This change adds experimental support for SHT_RELR sections, proposed
here: https://groups.google.com/forum/#!topic/generic-abi/bX460iggiKg
Definitions for the new ELF section type and dynamic array tags, as well
as the encoding used in the new section are all under discussion and are
subject to change. Use with caution!
Bug: None
Test: Built image for marlin, flashed on device, ran arm and
aarch64 binaries containing '.relr.dyn' sections.
Change-Id: I2953ae932d3c42ae394e71f8fa058013758a1778
Introduce new flag to mark soinfo as TLS_NODELETE when
there are thread_local dtors associated with dso_handle
belonging to it.
Test: bionic-unit-tests --gtest_filter=dl*
Test: bionic-unit-tests-glibc --gtest_filter=dl*
Bug: https://github.com/android-ndk/ndk/issues/360
Change-Id: I724ef89fc899788f95c47e6372c38b3313f18fed
This change addresses multiple problems introduced by
02586a2a34
1. In the case of unsuccessful dlopen the failure guard is triggered
for two namespaces which leads to double unload.
2. In the case where load_tasks includes libraries from 3 and more
namespaces it results in incorrect linking of libraries shared between
second and third/forth and so on namespaces.
The root cause of these problems was recursive call to find_libraries.
It does not do what it is expected to do. It does not form new load_tasks
list and immediately jumps to linking local_group. Not only this skips
reference counting it also will include unlinked but accessible library
from third (and fourth and fifth) namespaces in invalid local group. The
best case scenario here is that for 3 or more namesapces this will
fail to link. The worse case scenario it will link the library
incorrectly with will lead to very hard to catch bugs.
This change removes recursive call and replaces it with explicit list of
local_groups which should be linked. It also revisits the way we do
reference counting - with this change the reference counts are updated after
after libraries are successfully loaded.
Also update soinfo_free to abort in case when linker tries to free same
soinfo for the second time - this makes linker behavior less undefined.
Test: bionic-unit-tests
Bug: http://b/69787209
Change-Id: Iea25ced181a98c6503cce6e2b832c91d697342d5
This change also replaces elf-hash with gnu-hash.
Test: make
Test: bionic-unit-tests --gtest_filter=dl*:Dl*
Change-Id: Ibc4026f7abc7e8002f69c33eebaf6a193f1d22eb
This commit updates interface of libdl.c.
1. android_init_namespaces is replaces with android_init_anonymous_namespace
2. added 2 arguments to android_create_namespace to specify linked namespace
and the list of shared libraries sonames.
3. symbol lookup does not get past boundary libraries (added check and test for it).
Bug: http://b/26833548
Bug: http://b/21879602
Test: bionic-unit-tests --gtest_filter=dl*:Dl*
Change-Id: I32921da487a02e5bd0d2fc528904d1228394bfb9
Do not hijack libdl.so methods but make libdl proxy calls to
loader instead. This will be replaces by calls to libc.so
once loader functionality is migrated.
Also add a lock to dl_unwind_find_exidx function call.
Test: bionic-unit-tests --gtest_filter=dl*:Dl*
Bug: http://b/27106625
Change-Id: Ic33a7109a86f4262798d63a35f4c61d15b0068bb
Do not hijack libdl.so methods but make libdl proxy calls to
loader instead. This will be replaces by calls to libc.so
once loader functionality is migrated.
Also add a lock to dl_unwind_find_exidx function call.
Test: bionic-unit-tests --gtest_filter=dl*:Dl*
Bug: http://b/27106625
Change-Id: I9e666e771e4bbca52151cfa7fc4c8677e1480818
Extract linker executable specific code to linker_main.cpp;
this part of code does not have a place in libdl.a/so.
Bug: http://b/14998480
Bug: http://b/30706810
Test: mm && run bionic-unit-tests
Change-Id: I90f7475e93a919b0f9525da22928089ad35b8f6c
Move soinfo and globals out of linker.cpp to
separate files.
Breaking up huge linker.cpp into smaller peaces
in order to make it easier to extract part of the
code that belongs to libdl.so and remove parts of
the code that do not belong to linker
(refactoring part 2 of many)
Change-Id: I868417f4b8d2b84d0e8265e354bc7977161497e2
