Commit graph

26706 commits

Author SHA1 Message Date
Evgenii Stepanov
be551f596f HWASan support in bionic.
* Allow sanitization of libc (excluding existing global sanitizers)
  and disallow sanitization of linker. The latter has not been
  necessary before because HWASan is the first sanitizer to support
  static binaries (with the exception of CFI, which is not used
  globally).
* Static binary startup: initialize HWASan shadow very early so that
  almost entire libc can be sanitized. The rest of initialization is
  done in a global constructor; until that is done sanitized code can
  run but can't report errors (will simply crash with SIGTRAP).
* Switch malloc_common from je_*  to __sanitizer_*.
* Call hwasan functions when entering and leaving threads. We can not
  intercept pthread_create when libc depends on libclang_rt.hwasan.
  An alternative to this would be a callback interface like requested
  here:
    https://sourceware.org/glibc/wiki/ThreadPropertiesAPI

All of the above is behind a compile-time check
__has_feature(hwaddress_sanitizer). This means that HWASan actually
requires libc to be instrumented, and would not work otherwise. It's
an implementation choice that greatly reduces complexity of the tool.
Instrumented libc also guarantees that hwasan is present and
initialized in every process, which allows piecemeal sanitization
(i.e. library w/o main executable, or even individual static
libraries), unlike ASan.

Change-Id: If44c46b79b15049d1745ba46ec910ae4f355d19c
2018-08-21 00:15:47 +00:00
Christopher Ferris
3fd45bba48 Merge "Add more descriptive text around app starting." 2018-08-17 14:23:15 +00:00
Christopher Ferris
ad935c8139 Add more descriptive text around app starting.
Test: NA
Change-Id: If3bcf060b11ec08835863236b0a1408aebe43355
2018-08-16 17:57:23 -07:00
Christopher Ferris
841c9c1ce2 Merge "Add new malloc benchmarks." 2018-08-16 01:09:17 +00:00
Treehugger Robot
b3f24b1b86 Merge "libc: enable -Wimplicit-fallthrough" 2018-08-15 19:31:33 +00:00
George Burgess IV
fa5410fdb7 libc: enable -Wimplicit-fallthrough
libc had some -Wimplicit-fallthrough warnings. They all seem to be
benign. We're trying to enable this flag globally, so we need to
annotate these breaks here.

Bug: 112564944
Test: Builds
Change-Id: I5afae694cc4cf26ad1a61e2c8ae91f00cda7c733
2018-08-15 10:11:08 -07:00
Christopher Ferris
0dc784431b Add new malloc benchmarks.
This runs through the trace of the allocations in a sql benchmark app
executed in the benchmark thread.

Add one benchmark with decay time set to 0 and another with decay time
set to 1.

Include a script that can generate a header file that can be used to
regenerate the data.

Bug: 112317428

Test: Builds, ran unit tests, ran benchmarks.
Change-Id: I62e287cc06b74b74bcc5a4bbee71b0fac0a196fd
2018-08-14 16:01:58 -07:00
Treehugger Robot
5285d5995b Merge "Fix benchmark-tests" 2018-08-13 19:22:33 +00:00
Haibo Huang
510dda3ea1 Fix benchmark-tests
1. fix all_benchmarks. Some new benchmarks were added.
2. move some benchmark suites to the right place.
3. add unit test for https://android-review.googlesource.com/c/platform/bionic/+/724785

Test: compile and run bionic-benchmarks-tests on device.
Change-Id: I2b686b51910a422b664f9ea968199f37944039c0
2018-08-11 23:43:03 -07:00
Treehugger Robot
a7573441c1 Merge "Add new strcmp/strncmp benchmark" 2018-08-11 08:23:14 +00:00
Adhemerval Zanella
43f2b77c8b Add new strcmp/strncmp benchmark
This patch adds a new strcmp/strcmp benchmark with different set of
inputs.  It cover all pre-defined range values (small, medium, large)
for aligned inputs and also small range for unaligned ones.

Test: Ran new string test suite with a single bionic iteration.
Change-Id: Ice78d74c3583a33158eaba3309c10df54044aa28
2018-08-11 07:20:53 +00:00
Haibo Huang
68c8dd73ae Merge "Add new memcmp benchmark" 2018-08-11 07:20:19 +00:00
Adhemerval Zanella
0e4fd755a0 Add new memcmp benchmark
This patch adds a new memcmp benchmark with different set of inputs.
It covers all pre-defined range values (small, medium, large) for
aligned inputs and also medium range for unaligned ones.

Test: Ran new string test suite with a single bionic iteration.
Change-Id: Icdb95220a62d46bd71fa6ffbbf4567a9a05b7685
2018-08-11 07:20:13 +00:00
Treehugger Robot
5288655f01 Merge "Add option to define ranges sets for benchmarks" 2018-08-11 01:23:02 +00:00
Adhemerval Zanella
eda94aa1ae Add option to define ranges sets for benchmarks
This patch add an option to use the pre-define set of ranges along
with macros AT_ONEBUF_MANUAL_ALIGN_* and AT_TWOBUF_MANUAL_ALIGN1_*.

The size argument can be either a number or a string representing
the sets of values:

  * SMALL for values between 1 and 256.
  * MEDIUM for values between 512 and 128KB.
  * LARGE for values between 256KB and 2048KB.

Test: Ran new string test suite with a single bionic iteration.
Change-Id: Ieda81ee9a5019991b0b2f97d4ca3a237127c5848
2018-08-11 00:27:27 +00:00
Treehugger Robot
0f06b34e33 Merge "Add hhb@ to OWNERS." 2018-08-10 18:20:21 +00:00
Elliott Hughes
ebbb8c8969 Add hhb@ to OWNERS.
Bug: N/A
Test: N/A
Change-Id: I86dd3425870a62a286c15bd2e08170104e5fc038
2018-08-10 10:14:13 -07:00
Bernie Innocenti
817f578bf4 Merge "Clear the resolver timeout when setting default __res_params" 2018-08-10 02:15:20 +00:00
Treehugger Robot
fce5dae8a3 Merge "Make the resolver retry timeout configurable via __res_params" 2018-08-10 01:55:09 +00:00
Treehugger Robot
a2650eaf1c Merge "seccomp: Allow read(2) and getdents(2) in x86_64 for UBSan's sake" 2018-08-09 23:47:14 +00:00
Treehugger Robot
f0fae7d8e5 Merge "Use double-precision routines from arm-optimized-routines" 2018-08-09 21:42:18 +00:00
Luis Hector Chavez
ea82982b87 seccomp: Allow read(2) and getdents(2) in x86_64 for UBSan's sake
This change allows the use of read(2) and getdents(2) so that UBSan can
work correctly on x86_64.

Bug: 111999822
Test: CtsWrapWrapDebugTestCases
Change-Id: I44aac17fffdffc5c3f5d87fb3d876af28c8044b2
2018-08-09 12:32:01 -07:00
Adhemerval Zanella
f6b101d3ec Use double-precision routines from arm-optimized-routines
This patch ues exp, exp2, log, log2, and pow from arm optimized
routines.  For pow on x86_64, although slight slower it simplifies
the code required on both bionic and arm-optimized-routines (so
there is no need to select and export the symbol based on
architecture).

Performance-wise the improvements are:

  x86_64    throughput    latency
  exp            1.16x      1.16x
  log            1.08x      0.95x
  exp2           1.27x      1.55x
  log2           1.40x      1.43x
  pow            0.77x      0.89x *

  * I tried to check if AVX2/FMA but without success.

  aarch64   throughput     latency
  exp            2.33x      2.16x
  exp2           1.99x      1.50x
  log            1.79x      1.43x
  log2           2.15x      1.80x
  pow            3.81x      3.07x

Test: ran bionic tests on static mode.
Change-Id: Ib16bf3280c5329fd257a3b3f0b6c4f2f3cb34deb
2018-08-09 11:22:24 -03:00
Bernie Innocenti
96c619f7db Clear the resolver timeout when setting default __res_params
This is currently a non-functional change, because base_timeout_msec is
never set to a non-default value (other than in unit tests).

Test: netd_unit_test --gtest_filter='ResolverTest.*'
Change-Id: I21989bab5cc9f55ee1392e30965eb405d720a28e
2018-08-09 17:58:35 +09:00
Treehugger Robot
3652bcd775 Merge "Add pow benchmark" 2018-08-09 02:27:33 +00:00
Treehugger Robot
de8eb12517 Merge "Move unique_fd fdsan tests into bionic." 2018-08-09 02:14:48 +00:00
Josh Gao
7266e91f62 Move unique_fd fdsan tests into bionic.
EXPECT_DEATH requires special handling to work with fdsan, because of
our fork detection. Instead of duplicating the EXPECT_FDSAN_DEATH macro
in the libbase tests as well, move the unique_fd fdsan tests here.

Test: bionic_unit_tests
Change-Id: Ia7b65d4560601d5a78d143aced887a6773b401c0
2018-08-09 01:09:08 +00:00
Treehugger Robot
24c0eb1f30 Merge "Add log and log2 benchmark" 2018-08-09 00:33:42 +00:00
Adhemerval Zanella
72fe1c8df6 Add pow benchmark
As for exp/exp2 benchmark, this patch add two benchmark for pow:
one which measures thoughput and one which measures latency.

The input data is the same as powf.

Test: ran 32-bit and 64-bit x86 tests on host
Change-Id: I04335fac9e76fb3f39935323dacf6b7be6a6f917
2018-08-08 18:04:48 -03:00
Adhemerval Zanella
56f4511c39 Add log and log2 benchmark
Similar to exp/exp2, this patch add two benchmarks for log and log2:
one which measures thoughput and one which measures latency.

The input data is based on logf/log2f one (reduced trace based on 2.8
billion samples extracted from specpu2017 521.wrf_r benchmark).

Test: ran 32-bit and 64-bit x86 tests on host

Change-Id: I51a9e2960e45d5619585e685eaa7634cc7be901b
2018-08-08 18:04:48 -03:00
Treehugger Robot
099835ad2a Merge "Add exp and exp2 benchmark" 2018-08-08 20:18:16 +00:00
Adhemerval Zanella
10d330af34 Add exp and exp2 benchmark
This patch add two benchmarks for both exp and exp2, one which measures
thoughput and one which measures latency. The latency benchmark works by
creating a dependency on the previous iteration and with a zero multiply
to avoid changing the input value.

The input is based on expf/exp2f benchmark (reduced trace based on 2.4
billion samples extracted from specpu2017 521.wrf_r benchmark).

Test: ran 32-bit and 64-bit x86 tests on host
2018-08-08 15:06:09 -03:00
Elliott Hughes
b7b2884a21 Merge "Reland "Retire GCC FORTIFY."" 2018-08-08 15:26:37 +00:00
Bernie Innocenti
58d13622a5 Make the resolver retry timeout configurable via __res_params
The primary motivation is speeding up tests for non-responsive
nameservers. It's also base infrastructure for future work to compute
retry timeouts dynamically.

Using default timeouts (RES_TIMEOUT = 5 seconds):
  ResolverTest.GetAddrInfoV6_nonresponsive (45141 ms)

With base_timeout_msec=100:
  ResolverTest.GetAddrInfoV6_nonresponsive (1264 ms)

Test: executed unsubmitted test from aosp/713993
Change-Id: Id902089ca69ca8d28032180ba51e8937262ef490
2018-08-08 15:45:19 +09:00
Treehugger Robot
f2fb3b11c6 Merge "seccomp: Generate the policy files at compile time" 2018-08-07 13:17:42 +00:00
Josh Gao
16d705671b Merge changes from topic "fdsan_debuggerd"
* changes:
  fdsan: expose the address of the table to the platform.
  fdsan: minor refactoring.
2018-08-07 10:02:16 +00:00
Josh Gao
e16082f717 fdsan: expose the address of the table to the platform.
Let the debuggerd handler get the address of the fdsan table so that
crash_dump can parse it and print it out in tombstones.

Test: debuggerd `pidof adbd`
Change-Id: I2297200cae52d0ff24479dba55d1ab4938583501
2018-08-06 18:48:52 -07:00
Josh Gao
e6dab7b55f fdsan: minor refactoring.
Make it easier for other code to parse the fdsan table themselves by
making it so that they can include the bionic_fdsan.h header to get a
struct definition with the layout without bringing in other
dependencies.

Test: treehugger
Change-Id: I3583ef113991234aa83cd193a4eb139ad06737c7
2018-08-06 18:47:29 -07:00
Treehugger Robot
fc97dafa86 Merge "fdsan: enable by default." 2018-08-07 00:08:43 +00:00
Christopher Ferris
9f93f22d7e Merge "Fix libc assembler documentation."
am: 5c14bf3932

Change-Id: I4376b8d8689a4e07c85d23d8a0097bbde4c2ca03
2018-08-06 15:32:27 -07:00
Josh Gao
4b7c1cc13f fdsan: enable by default.
Test: adb shell crasher fdsan_file
Change-Id: I6be0f7372ddb9d622a0bd8d1eeb80a4f747c99e3
2018-08-06 15:11:08 -07:00
Christopher Ferris
5c14bf3932 Merge "Fix libc assembler documentation." 2018-08-06 21:25:02 +00:00
Christopher Ferris
a473be2395 Fix libc assembler documentation.
Test: NA
Change-Id: I71485a3a44a56f2ff7ed590aec171464b57f56f6
2018-08-06 12:18:32 -07:00
Elliott Hughes
e9a8aa6515 Merge "Make all popen(3) file descriptors O_CLOEXEC."
am: f80e3f4413

Change-Id: I151a5abcc99acdecf4cbd6ccd369a30e31b15941
2018-08-06 12:05:24 -07:00
Elliott Hughes
f80e3f4413 Merge "Make all popen(3) file descriptors O_CLOEXEC." 2018-08-06 18:43:40 +00:00
Luis Hector Chavez
fa09b3c754 seccomp: Generate the policy files at compile time
This change avoids having to run the genseccomp.py script every time a
policy file is edited, and instead generates these files at
compile-time.

Bug: None
Test: m
Test: find out/soong/ -name x86_64_global_policy.cpp  # Shows files
Test: generated policies are equivalent to original policies
Change-Id: I12461fe0c5fb02c008c1b2503fbb994b8aa2f56b
2018-08-06 11:10:25 -07:00
Elliott Hughes
7601130f0e Merge "Revert "Move android-changes-for-ndk-developers.md into docs/.""
am: 782cd01b09

Change-Id: I0a1037287fc26b6148a93f419830e4a96abd1a08
2018-08-06 10:57:51 -07:00
Treehugger Robot
782cd01b09 Merge "Revert "Move android-changes-for-ndk-developers.md into docs/."" 2018-08-06 17:27:31 +00:00
Elliott Hughes
74866e160d Revert "Move android-changes-for-ndk-developers.md into docs/."
This reverts commit 1975196c87.

Reason for revert: https://android.googlesource.com/ doesn't follow symlinks :-(

Change-Id: I58038a03388ab5257d301debb494ce354013acfe
2018-08-06 15:40:04 +00:00
Luis Hector Chavez
c43bff9b8e Merge "genseccomp.py: Add a way to better find the clang prebuilt"
am: ea6aff66fa

Change-Id: I2b22aa39ef49dbc78f64e4f1310c184198a5455b
2018-08-03 20:02:20 -07:00