Commit graph

30216 commits

Author SHA1 Message Date
Christopher Ferris
75edf16f4d Update the native allocator document.
Add extra information I've been using to evaluate scudo.

Test: NA
Change-Id: Ie7ee65890b1a3804c38ba38de504144d43e410b2
2019-11-13 14:00:56 -08:00
Elliott Hughes
5e85d1b290 Merge "Update docs/status.md function counts for recent releases." 2019-11-13 15:38:57 +00:00
Treehugger Robot
f0210e9300 Merge "Rename # vndk tag to # llndk" 2019-11-13 06:36:36 +00:00
Jiyong Park
a4fe44e339 Rename # vndk tag to # llndk
The APIs that are tagged with # vndk are actually for LLNDK libraries.
Although LLNDK is part of VNDK, calling those APIs 'vndk' has given
users a wrong perception that the APIs don't need to be kept stable
because that's the norm for most of the VNDK libraries that are not
LLNDK.

In order to eliminate the misunderstanding, rename the tag to 'llndk' so
that people introducing new such API will realize what they are signing
themselves up for.

Bug: 143765505
Test: m
Merged-In: I56e49876410bd43723a80d0204a9aef21d20fca9
(cherry picked from commit 3e2cd44aa4)
Change-Id: I56e49876410bd43723a80d0204a9aef21d20fca9
2019-11-13 14:31:26 +09:00
Elliott Hughes
502dc0b9c3 Update docs/status.md function counts for recent releases.
Test: N/A
Change-Id: I99b7227334d0858f19ec7544a255fe47f265d810
2019-11-12 21:06:07 -08:00
Treehugger Robot
754b27d3dd Merge "Link libgcc_stripped into libc.so and libm.so with --whole-archive on ARM." 2019-11-12 23:36:29 +00:00
Peter Collingbourne
b061e77748 Link libgcc_stripped into libc.so and libm.so with --whole-archive on ARM.
This library was previously being statically linked into both libraries as a
consequence of the relocation to __aeabi_unwind_cpp_prX present in most object
files. However, after LLVM commit 1549b469, we no longer emit these relocations
on Android, so we need to link the library explicitly with --whole-archive. The
intent is to eventually stop linking libgcc into these libraries altogether,
but for now, we need to keep linking them in order to avoid breaking the build.

Change-Id: I275109527b7cbd6c4247b3fe348975d720626273
2019-11-12 14:26:23 -08:00
Raman Tenneti
afe44cc483 Merge "Revert "Revert "Make system property reads wait-free""" 2019-11-12 20:43:29 +00:00
Raman Tenneti
b481a2e743 Revert "Revert "Make system property reads wait-free""
This reverts commit de39d9242a.

Reason for revert: This revert is not needed

Change-Id: I34af8e5d75c724f6c4066fafbfc6bc7d58377601
2019-11-12 20:41:55 +00:00
Treehugger Robot
e5c900f011 Merge "Optimized L2 Cache value for Intel(R) Core Architectures." 2019-11-12 20:35:38 +00:00
Raman Tenneti
1b28efa2d1 Merge "Revert "Make system property reads wait-free"" 2019-11-12 18:26:48 +00:00
Raman Tenneti
de39d9242a Revert "Make system property reads wait-free"
This reverts commit 0cf90556de.

Reason for revert: Device boot failures - 144355953

Change-Id: Icd4fc8c7a1d06d688a6d37e6f1c1aa45563f711b
2019-11-12 18:24:06 +00:00
Treehugger Robot
985e678516 Merge "Make system property reads wait-free" 2019-11-12 17:14:30 +00:00
jaishank
2e50fa7cf8 Optimized L2 Cache value for Intel(R) Core Architectures.
Performance Gain:
AnTuTu             - 4.80%
3D Mark Sling Shot - 3.47%
BaseMarkGPU        - 5.51%
GeekBench          - 3.19%

Test: ./tests/run-on-host.sh 64

Change-Id: I6122835a3f5fd97cc291623d1062fe25843a2d93
Signed-off-by: jaishank <jaishankar.rajendran@intel.com>
2019-11-12 15:58:34 +00:00
Treehugger Robot
ebe5bfefe1 Merge "bionic: libc: remove -Wimplicit-fallthrough" 2019-11-11 21:32:43 +00:00
Daniel Colascione
0cf90556de Make system property reads wait-free
Right now, when we read a system property, we first (assuming we've
already looked up the property's prop_info) read the property's serial
number; if we find that the low bit (the dirty bit) in the serial
number is set, we futex-wait for that serial number to become
non-dirty. By doing so, we spare readers from seeing partially-updated
property values if they race with the property service's non-atomic
memcpy to the property value slot. (The futex-wait here isn't
essential to the algorithm: spinning while dirty would suffice,
although it'd be somewhat less efficient.)

The problem with this approach is that readers can wait on the
property service process, potentially causing delays due to scheduling
variance. Property reads are not guaranteed to complete in finite time
right now.

This change makes property reads wait-free and ensures that they
complete in finite time in all cases. In the new approach, we prevent
value tearing by backing up each property we're about to modify and
directing readers to the backup copy if they try to read a property
with the dirty bit set.

(The wait freedom is limited to the case of readers racing against
*one* property update. A writer can still delay readers by rapidly
updating a property --- but after this change, readers can't hang due
to PID 1 scheduling delays.)

I considered adding explicit atomic access to short property values,
but between binary compatibility with the existing property database
and the need to carefully handle transitions of property values
between "short" (compatible with atomics) and "long" (incompatible
with atomics) length domains, I figured the complexity wasn't worth it
and that making property reads wait-free would be adequate.

Test: boots
Bug: 143561649
Change-Id: Ifd3108aedba5a4b157b66af6ca0a4ed084bd5982
2019-11-11 11:17:00 -08:00
Nick Desaulniers
97c43095b9 bionic: libc: remove -Wimplicit-fallthrough
aosp/144287300 set it to a global cppflag.

The compiler upgrade can now check for this warning in C code.

This patch should be reverted once the BSD sources with instances of
-Wimplicit-fallthrough have been fixed.

Remove it from cflags, so that it's not re-enabled for C code until
fixed.

Bug: 139945549
Bug: 144287300
Test: mm
Change-Id: Ieca0d5b41634636477392e5209a41807f9b44bd4
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
2019-11-11 10:49:10 -08:00
Treehugger Robot
28fb781ef7 Merge "linker: add more context to link failure error." 2019-11-09 00:13:31 +00:00
Treehugger Robot
ffad55954b Merge "Stop using __out as a name." 2019-11-08 23:25:19 +00:00
Christopher Ferris
c686ab54c2 Merge "Rename iterate to malloc_iterate internally." 2019-11-08 22:10:56 +00:00
Elliott Hughes
f38e8ca149 Stop using __out as a name.
libc++ poisons `__out` because it's #defined on Windows. Rather than
hack libc++, let's just avoid that name. "src" and "dst" are far more
widely used than "in" and "out" for this purpose anyway.

Test: treehugger
Change-Id: I0db9997fd5f06f626dbf0ee967b52145395466b4
2019-11-08 13:51:12 -08:00
Christopher Ferris
6f517cd7a1 Rename iterate to malloc_iterate internally.
I have no idea why I used the iterate name internally which is
completely unlike every other function name. Change this to match
everyone else so that it's now malloc_iterate everywhere.

This is probably the last chance to change this before mainline
modules begin, so make everything consistent.

Test: Compiles, unit tests passes.
Change-Id: I56d293377fa0fe1a3dc3dd85d6432f877cc2003c
2019-11-08 11:49:14 -08:00
Martin Stjernholm
31600e8424 Merge "Revert submission of topic 'libdl_android-in-apex'" 2019-11-08 11:51:39 +00:00
Yuexi Ma
c74666eb3c Revert submission of topic 'libdl_android-in-apex'
Reason for revert: broke numerous major apps in app compat testing
Bug: 144100240

Change-Id: I50b440356bdbde3ac130da5d28d5ef20f8b63582
2019-11-07 19:18:53 +00:00
Jim Blackler
ca1834de7a Merge "Revert "fortify: allow diagnostics without run-time checks"" 2019-11-07 12:39:16 +00:00
Jim Blackler
2a83cb7357 Revert "fortify: allow diagnostics without run-time checks"
This reverts commit d7e11b8853.

Reason for revert: Droidcop-triggered revert due to breakage android-build.googleplex.com/builds/submitted/5992686/aosp_x86_64-eng/latest/view/logs/build_error.log, bug b/144081298

Change-Id: Ic6a492c4fcf2ea1bb324c44fb615a898bbc9a9b7
2019-11-07 12:21:47 +00:00
Martin Stjernholm
77e910d337 Merge "Add libdl_android to the Runtime APEX." 2019-11-07 11:01:36 +00:00
George Burgess IV
2fead5fd89 Merge "Revert "fortify: allow diagnostics without run-time checks"" 2019-11-07 07:10:38 +00:00
George Burgess IV
f359548bf1 Revert "fortify: allow diagnostics without run-time checks"
This reverts commit d7e11b8853.

Reason for revert: Breaks aosp_x86_64-eng. Will look into it and
unbreak when it's not almost midnight. :)

Change-Id: I21f76efe4d19c70d0b14630e441376d359a45b49
2019-11-07 07:07:38 +00:00
Treehugger Robot
03aa94c9a9 Merge "Restore _FORTIFY_SOURCE in versioner." 2019-11-07 02:42:58 +00:00
Treehugger Robot
d1dcb72882 Merge "fortify: allow diagnostics without run-time checks" 2019-11-07 01:34:11 +00:00
Christopher Ferris
7a3a5d8cd7 Merge "Fix allocations escaping malloc debug." 2019-11-06 23:31:33 +00:00
Elliott Hughes
f7db8507ee Restore _FORTIFY_SOURCE in versioner.
Bug: http://b/118991081
Test: treehugger
Change-Id: I2818d8c27ecc5a48666013cdd0eaa26ec7900688
2019-11-06 14:35:34 -08:00
Tom Cherry
b18158c3c9 Merge changes I01d061c4,I2e3f6ee7
* changes:
  Check that __system_property_set() doesn't leak FDs
  clang-format: Add IncludeBlocks: Preserve
2019-11-06 21:54:00 +00:00
Ryan Prichard
ca0d8990d3 Merge "linker: avoid -O0 memset while clearing temp_tcb" 2019-11-06 20:38:30 +00:00
Christopher Ferris
ff88fb0d3a Fix allocations escaping malloc debug.
When using a FILE object for some malloc debug functions, calling
fprintf will trigger an allocation to be put in the object. The problem
is that these allocations were not allocated by the malloc debug
wrapper and they get freed during the fclose as if they are malloc
debug allocation. In most cases, the code will detect the bad pointer
and leak the memory, but it might also cause a crash.

The fix is to avoid using fprintf so that no allocations are made
in the object that survive and need to be freed in the fclose call.

Change the MallocXmlElem.h to use a file decsriptor not a FILE object.

Add new unit and system tests to detect this case.

Bug: 143742907

Test: Ran unit and system tests.
Test: Ran bionic unit tests.
Change-Id: I524392de822a29483aa5be8f14c680e70033eba2
2019-11-06 10:42:42 -08:00
Elliott Hughes
d9585a2196 Merge "android_getaddrinfo_proxy: fix memory leak on failure." 2019-11-06 17:31:14 +00:00
Elliott Hughes
9a09137b0e Merge "Limit threads in pthread_leak#detach for low power devices." 2019-11-06 16:00:48 +00:00
Treehugger Robot
b450a7f426 Merge "search.h: add doc comments." 2019-11-06 03:58:32 +00:00
Martin Stjernholm
a3b0eab32f Add libdl_android to the Runtime APEX.
Before this it ended up in /system.

Test: build & boot
Test: adb shell find system apex -name libdl_android.so | xargs adb shell ls -l
  =>
  apex/com.android.runtime/lib/bionic/libdl_android.so
  apex/com.android.runtime/lib64/bionic/libdl_android.so
  apex/com.android.runtime@1/lib/bionic/libdl_android.so
  apex/com.android.runtime@1/lib64/bionic/libdl_android.so
  system/lib/bootstrap/libdl_android.so
  system/lib/libdl_android.so -> /apex/com.android.runtime/lib/bionic/libdl_android.so
  system/lib64/bootstrap/libdl_android.so
  system/lib64/libdl_android.so -> /apex/com.android.runtime/lib64/bionic/libdl_android.so
Bug: 135753770
Change-Id: Ibabb5714e8cc0454c77ae5d42d9650c345ff1b89
2019-11-06 00:28:38 +00:00
Ryan Prichard
190e22ab9c Merge changes Ieb9446c2,I7c5d7928
* changes:
  Use ifuncs in the linker
  Refactor linker/Android.bp for native-bridge
2019-11-05 23:48:07 +00:00
Ryan Prichard
94a8e859ed linker: avoid -O0 memset while clearing temp_tcb
With -O0, on arm64, Clang uses memset for this declaration:

    bionic_tcb temp_tcb = {};

arm64 doesn't currently have an ifunc for memset, but if it did, then this
line would crash when the linker is compiled with -O0. It looks like other
architectures would only use a memset call if the bionic_tcb struct were
larger.

Avoid memset by using a custom memclr function that the compiler optimizes
into something efficient.

Also add __attribute__((uninitialized)) to ensure that
-ftrivial-auto-var-init does not generate a call to memset. See this
change[1] in build/soong.

[1] If085ec53c619e2cebc86ca23f7039298160d99ae

Test: build linker with -O0, linker[64] --help works
Bug: none
Change-Id: I0df8065a362646de4fa021cae63a7d68ca3966b6
2019-11-05 14:56:09 -08:00
Josh Gao
1626957666 linker: add more context to link failure error.
This change makes it easier to diagnose mistakes in linker
configuration that result in a library being accidentally loaded in
multiple namespaces without its dependencies available everywhere.

Test: manually tested the error message
Test: bionic-unit-tests
Change-Id: I03a20507f8fc902c2445a7fbbf59767ffffd5ebf
2019-11-05 13:45:54 -08:00
Ryan Prichard
249757bae2 Use ifuncs in the linker
Using ifuncs allows the linker to select faster versions of libc functions
like strcmp, making linking faster.

The linker continues to first initialize TLS, then call the ifunc
resolvers. There are small amounts of code in Bionic that need to avoid
calling functions selected using ifuncs (generally string.h APIs). I've
tried to compile those pieces with -ffreestanding. Maybe it's unnecessary,
but maybe it could help avoid compiler-inserted memset calls, and maybe
it will be useful later on.

The ifuncs are called in a special early pass using special
__rel[a]_iplt_start / __rel[a]_iplt_end symbols. The linker will encounter
the ifuncs again as R_*_IRELATIVE dynamic relocations, so they're skipped
on the second pass.

Break linker_main.cpp into its own liblinker_main library so it can be
compiled with -ffreestanding.

On walleye, this change fixes a recent 2.3% linker64 start-up time
regression (156.6ms -> 160.2ms), but it also helps the 32-bit time by
about 1.9% on the same benchmark. I'm measuring the run-time using a
synthetic benchmark based on loading libandroid_servers.so.

Test: bionic unit tests, manual benchmarking
Bug: none
Merged-In: Ieb9446c2df13a66fc0d377596756becad0af6995
Change-Id: Ieb9446c2df13a66fc0d377596756becad0af6995
(cherry picked from commit 772bcbb0c2)
2019-11-05 13:37:51 -08:00
Ryan Prichard
80e40f01c8 Refactor linker/Android.bp for native-bridge
Define a "linker_bin_template" cc_defaults module that a native bridge
implementation can inherit to define a guest linker.

Break the debuggerd_init call off into separate
linker_debuggerd_{android,stub}.cpp files to allow opting in/out of the
debuggerd integration without needing to change how linker_main.cpp is
compiled. (This is necessary for a later commit that moves
linker_main.cpp into a new static library.)

Test: bionic unit tests
Bug: none
Merged-In: I7c5d79281bce1e69817b266dd91d43ea40f78522
Change-Id: I7c5d79281bce1e69817b266dd91d43ea40f78522
(cherry picked from commit 5adf402ee9)
2019-11-05 13:37:29 -08:00
Elliott Hughes
4aef93acff android_getaddrinfo_proxy: fix memory leak on failure.
The shadowing of `ai` meant that the freeaddrinfo() call outside the
loop would never see anything but NULL.

Bug: https://issuetracker.google.com/143928781
Test: treehugger
Change-Id: I1bf137f7933201eb8024603bfd569ff7bbc7f9b7
2019-11-05 12:42:14 -08:00
Tom Cherry
fd7216cc19 Check that __system_property_set() doesn't leak FDs
Bug: 143936364
Test: fail when b/143936364 repros
Test: pass when b/143936364 is fixed
Change-Id: I01d061c4fb4f37b0300ef418a19c88003038092c
2019-11-05 12:33:33 -08:00
Tom Cherry
ef18d440d9 clang-format: Add IncludeBlocks: Preserve
The same change and reasoning as
c077eefc1e5007a5c656183737e82d5c0a7df855 in build/soong, copied below:

As of [1], the Google format, which we default to, uses IncludeBlocks:
Regroup, which sorts all includes with <> and without .h into the top
group reserved for libc headers, e.g.:

+#include <android-base/properties.h>
+#include <android-base/strings.h>
 #include <ctype.h>
 #include <stdlib.h>
 #include <sys/system_properties.h>

 #include <iostream>

-#include <android-base/properties.h>
-#include <android-base/strings.h>

This is accurate to the Google C++ style guide, which states that
other library headers should be included with "", however we have a
strong tendency to use <> for these headers for Android, so we must
revert this change.

Note, clang-format allows regexes for creating the blocks as well,
however, short of including all possible libc headers, I don't think
there's a way to differentiate between <sys/wait.h> and
<other_lib/header.h>.

[1]:
62e3198c4f (diff-b689ebff2e51e675b346a88289e74542)

Test: clang-format doesn't do the above.
Change-Id: I2e3f6ee75fd725b620174082afcce4ee7f4efa66
2019-11-05 12:29:56 -08:00
Elliott Hughes
5d9ba9ea70 Merge "Close descriptor in ScopeFd destructor" 2019-11-05 16:12:18 +00:00
Johan Harvyl
89b09198b9 Close descriptor in ScopeFd destructor
The underlying descriptor is supposed to be closed when the ScopedFd
goes out of scope but due to a typo in reset() that never happens.

This change was inspired by an earlier implementation of ScopedFd in:
04dc91ae7 Load library using file handle.

Issue: 143918215
Test: Manual (verify fd usage on device)
Change-Id: I5664fa82d3e732113732e34a7ae6df3ca79d3cee
2019-11-05 12:20:23 +01:00