Commit graph

240 commits

Author SHA1 Message Date
Colin Cross
d80cbca76d Convert cc to AndroidMkEntries
Use AndroidMkEntries so the next patch can use ExtraFooters, which
doesn't exist in AndroidMkData.

Test: manually diff out/soong/Android.aosp_x86_64.mk
Change-Id: Ia3006b6747813693cf7e2b536030b21f3109f538
2020-02-25 19:53:04 -08:00
Ivan Lozano
9ac32c7674 Allow Linux hosts to use ubsan runtime sanitizers.
Linux host prebuilts for UBSan runtime are available now, so we can
enable these. There's a bit more work to be done for Windows/Darwin
support, so that's left to another CL.

Bug: 148289941
Test: Build host binary with integer overflow sanitization enabled.
Test: ./art/test/testrunner/run_build_test_target.py art-linux-bionic-x64

Change-Id: Ie98979116fa3bb8accb23d1788adcc28330a9530
2020-02-20 20:27:35 +00:00
Inseob Kim
eec88e1de8 Add vendor snapshot modules
This implements four modules (static/shared/header libraries, and
binaries) for vendor snapshot. These modules will override source
modules if BOARD_VNDK_VERSION != current.

Bug: 65377115
Test: 1) VNDK_SNAPSHOT_BUILD_ARTIFACTS=true m dist vndk vendor-snapshot
Test: 2) install snapshot under source tree
Test: 3) set BOARD_VNDK_VERSION and boot cuttlefish
Change-Id: I24ddb4c3aa6abeab60bbfd31bcbd8753e2592dc5
2020-02-07 02:59:24 +09:00
Inseob Kim
8471cdaced Implement vendor snapshot
Vendor snapshot can be captured with "m dist vendor-snapshot". With
vendor snapshot and vndk snapshot, older version of /vendor and newer
version of /system will be able to be built together by setting
BOARD_VNDK_VERSION to past vendor's version.

Only vendor modules under AOSP are to be captured. In detail, modules
under following directories are ignored:
- device/
- vendor/
- hardware/, except for interfaces/, libhardware/, libhardware_legacy/,
  and ril/

Test modules (cc_test, etc.) and sanitized modules are also ignored.

Bug: 65377115
Test: m dist vendor-snapshot
Change-Id: If7a2f6de7f36deee936930c0ccf7c47c4a0cebf6
2020-02-07 02:58:29 +09:00
Yifan Hong
1b3348d50a Add ramdisk image.
It is similar to recovery image.
Test: m nothing -j

Change-Id: I11389777c6bfb0c0d73bbb4c70091c1e70f44077
2020-01-22 16:54:18 -08:00
Mitch Phillips
734b4cb62f Add $ORIGIN to fuzzer-coverage sanitized .so's.
Build fuzzer-sanitized libraries with an $ORIGIN DT_RUNPATH. Android's linker
uses DT_RUNPATH, not DT_RPATH. When we deploy cc_fuzz targets and their
libraries to /data/fuzz/<arch>/lib, any transient shared library gets the
DT_RUNPATH from the shared library above it, and not the executable, meaning
that the lookup falls back to the system. Adding the $ORIGIN to the DT_RUNPATH
here means that transient shared libraries can be found colocated with their
parents.

This may have some interesting consequences if:

 1. Your fuzz target depends on a shared library which has `sanitize.fuzzer:
 false` (as the DT_RUNPATH won't have `$ORIGIN`, and so you may get missing
 libraries).
 2. A `SANITIZE_TARGET=fuzzer` platform has a shared object in two
 different directories (like system vs. ndk) and is depending on the linker
 implementation details to resolve this in some manner.

I don't believe either of these instances should reasonably happen in
practise.

Fixes: 145973404
Fixes: 145988908
Test: m example_fuzzer
Change-Id: I94cbf628fc1ce15c43283d72bdabd9817de1fef8
2019-12-10 17:35:55 +00:00
Treehugger Robot
e643cd2b61 Merge "Add -instcombine-lower-dbg-declare to hwasan flags." 2019-12-03 02:19:08 +00:00
Evgenii Stepanov
64bee4decf Add -instcombine-lower-dbg-declare to hwasan flags.
This is a workaround to improve hwasan reports involving stack
variables.

Bug: 112437883
Test: SANITIZE_TARGET=hwaddress, build and boot

Change-Id: I21ea2fdd2d1f04e2bb9b9847785c00a951030c74
2019-11-22 18:40:07 -08:00
Colin Cross
7228ecd5e3 Move cc.imageMutator into the android package
Prepare for making the image mutator available to all modules and
moving it between the os and arch mutators by moving it into the
android package and using an interface implemented by the module
types to control it.

Bug: 142286466
Test: No unexpected changes to out/soong/build.ninja
Change-Id: I0dcc9c7b5ec80edffade340c367f6ae4da34151b
2019-11-20 15:21:32 -08:00
Colin Cross
4af21ed26f Split local and global cflags
Native compiler flags are currently applied in approximately:
global cflags
local cflags
local include dirs
global include dirs
global conlyflags
local conlyflags
global cppflags
local cppflags

This means that a flag that is enabled in the global cppflags
cannot be disabled in the local cflags, and an Android.bp author
must know to disable it in the local cppflags.  A better order
would be:
global cflags
global conlyflags
global cppflags
local cflags
local conlyflags
local cppflags
local include dirs
global include dirs

We are mixing both the global and local cflags into a single
variable, and similar for conlyflags and cppflags, which
prevents reordering them.  This CL prepares to reorder them
by splitting the global and local cflags into separate variables.

Bug: 143713277
Test: m native
Change-Id: Ic55a8c3516c331dc5f2af9d00e59ceca9d3e6c15
2019-11-07 15:27:58 -08:00
Jooyung Han
0302a84ddc Soong generates vndk-related .libraries.txt files
This is a follow-up CL of I9a24f6975bd4b226a94f61a13d43857dcdce6b88

Generated files are passed to make via MakeVars:
- SOONG_LLNDK_LIBRARIES_FILE
- SOONG_VNDKCORE_LIBRARIES_FILE
- SOONG_VNDKSP_LIBRARIES_FILE
- SOONG_VNDKPRIVATE_LIBRARIES_FILE
- SOONG_VNDKCOREVARIANT_LIBRARIES_FILE
- SOONG_VNDK_LIBRARIES_FILE

Previously filenames were "guessed" from module names. Now VndkMutator
stores filenames as well and generate those files with them.

Bug: 142963962
Bug: 141450808
Test: m && device boots && TH
Change-Id: I0c248b707188f904df60ead50059fefe90bfd17f
2019-11-01 04:35:21 +09:00
Ivan Lozano
52767be335 Add support for Rust C libraries.
Adds the ability for rust modules to be compiled as C libraries, and
allows cc modules to depend on these rust-generated modules. This also
means that soong-rust should not have any dependencies on soong-cc aside
from what's required for testing.

There's a couple small fixes included as well:

 - A bug in libNameFromFilePath that caused issues when library's had
 "lib" in their name.
 - VariantName is removed from rust library MutatedProperties since this
 was unused.

Bug: 140726209
Test: Soong tests pass.
Test: Example cc_binary can include a rust shared library as a dep.
Test: m crosvm.experimental
Change-Id: Ia7deed1345d2423001089014cc65ce7934123da4
2019-10-28 22:09:01 -07:00
Ivan Lozano
183a3218e2 Add a common interface for cc linkable libraries.
Adds an interface, CcLinkableInterface, for cc linkable dependencies
which come from other toolchains such as Rust.

Bug: 140726209
Test: Soong tests pass, rust modules still compile.
Change-Id: I7378a46fad94fd0b735746aaf4e265fd2c2c04d8
2019-10-28 13:45:12 -07:00
Ryan Prichard
b49fe1bde7 Stop linking libdl.a into static bins
libdl.a has a no-op dlopen, which breaks static libraries that need a real
dlopen. Instead of automatically linking libdl.a into static executables,
make it optional.

Until recently, the libunwind_llvm.a unwinder, used on arm32, needed the
no-op dladdr, but it's now built using -D_LIBUNWIND_USE_DLADDR=0.

The HWASan run-time uses dlsym and dladdr, so add a libdl dependency for
HWASan-built static binaries. We could also remove the dependency from
libclang_rt.hwasan_static-*.a, but this is also easy to do.

Bug: http://b/141485154
Test: bionic unit tests, device boots, verify that static and dynamic
   executables can throw/catch an exception
Test: verify that a static executable using dlopen doesn't link (unless it
   adds an explicit dependency on libdl)

Change-Id: Ic52c3f336b671b4ed335e99c94a64dfe8614b618
2019-10-24 22:25:54 -07:00
Colin Cross
ad4a597c79 Merge "Add method to determine variations from a Target" 2019-10-17 18:52:20 +00:00
Colin Cross
0f7d2ef3ac Add method to determine variations from a Target
The arch variants are hardcoded in every module type.  Refactor
them out into a Target.Variations() method in preparation for
splitting the arch mutator into two, which will require using
different variations.

Test: m checkbuild
Change-Id: I28ef7cd5168095ac888fe77f04e27f9ad81978c0
2019-10-16 14:52:30 -07:00
Mitch Phillips
b8e593d600 Workaround unexported sancov symbols. Fix multiple sanitizer RT deps.
Fuzz targets currently have dependencies on multiple libclang_rt runtime
libraries when building with ASan/HWAsan on device. This is an error.

This happens as Soong adds the dependency on the ASan/HWASan shared
runtime library. These libraries should provide the required UBSan
components. The clang driver was previously being passed
-fsanitize=fuzzer-no-link at link time, and as it doesn't know about the
already-established dependency on ASan/HWASan, it mistakenly thinks that
there is not runtime providing the UBSan components.

This patch fixes that problem by not adding -fsanitize=fuzzer-no-link to
the link-time flags.

This revealed a underlying issue in the upstream runtime compilation.
Android uses emulated TLS, which changes the symbol names from
<my_symbol_name> to __emutls_v._<my_symbol_name>. In particular, this
fails to account for the '__sancov_lowest_stack' symbol, as it no longer
matches the linker script rule for '__sancov*', and the symbol is no
longer exported in the shared library variant of ASan/HWASan.

This patch works around the discovered issue, which is being tracked in
the linked bug. It disables stack depth instrumentation, and we no
longer depend on this symbol. This means we get a missing sanitizer
coverage feature when fuzzing, but shouldn't be too detrimental.

Bug: 142430592
Test: SANITIZE_TARGET=hwaddress m example_fuzzer && \
readelf -d example_fuzzer # ensure only ONE libclang_rt dep (in this
case, hwasan)

Change-Id: Iea6df55d592a801732511c9b690134367429d62a
2019-10-10 10:53:48 -07:00
Peter Collingbourne
1c648b8936 Allow sanitize attribute on cc_objects and propagate sanitize attributes into them.
Without this change, we can end up failing to sanitize parts of libraries
that are compiled as cc_objects, such as bcm_object.o in boringssl. This
is normally harmless (except that the sanitizer would fail to detect bugs
in unsanitized code), however boringssl in particular needs to be compiled
with HWASAN enabled in all translation units in order to avoid link errors
with newer toolchains that support HWASAN global instrumentation.

Change-Id: Ib8bd325f76b7852cab435bff79d6945768298a5c
2019-09-26 12:35:06 -07:00
Mitch Phillips
586b8a2042 Merge "Disable FORTIFY for fuzzer builds."
am: 1e7f14b1d5

Change-Id: Ifa6d2c356181af8005e8df12801141b17e5b99ae
2019-08-29 10:10:17 -07:00
Mitch Phillips
b9b3e79096 Disable FORTIFY for fuzzer builds.
Disables FORTIFY for fuzz target builds. We generally build with a
sanitizer and want the sanitizer to handle the bugs and provide us with
a nice stack trace.

Bug: 140187921
Test: m example_fuzzer && $ANDROID_HOST_OUT/fuzz/example_fuzzer (note
that you get a proper ASan report instead of a fortify crash)

Change-Id: I15a117b26c401a03785fb44c0f796588f366bc55
2019-08-28 12:41:07 -07:00
Yi Kong
e4a824d378 Merge "Switch to clang-r365631"
am: bdefa541d9

Change-Id: I4f8b4e9ce9e0a355b247b11effc5bb1e5ba07f63
2019-08-22 11:56:05 -07:00
Yi Kong
bdefa541d9 Merge "Switch to clang-r365631" 2019-08-22 18:35:54 +00:00
Yi Kong
5006a72ec9 Merge "Revert "Re-enable ASAN globals instrumentation for devices.""
am: 20cf012ad1

Change-Id: Ibffe87ad95d7336c9ffdd78eb67fd81ab4b3e1b1
2019-08-21 21:09:52 -07:00
Yi Kong
20cf012ad1 Merge "Revert "Re-enable ASAN globals instrumentation for devices."" 2019-08-22 03:43:51 +00:00
Yi Kong
da069087bf Revert "Re-enable ASAN globals instrumentation for devices."
This reverts commit cba311bcfc.

The build is actually still broken with ASAN globals instrumentation.
We accidentally suppressed ASAN by turning on the new pass manager and
the build breakage was not discovered.

Bug: 137312732
Bug: 139747256
Change-Id: I8110e1413468c4d760a59a8de816a72c9852a7c8
2019-08-22 00:46:36 +00:00
Yi Kong
55d2d90ea4 Merge "Disable new pass manager for ASAN build"
am: 5724333e22

Change-Id: I670af9bab9dca4284420cda4bca5ed5884e7d6a6
2019-08-21 14:47:29 -07:00
Yi Kong
20233a43db Disable new pass manager for ASAN build
ASAN build has some weird issues with the new pass manager, keep using
the legacy pass manager for now.

Test: presubmit
Bug: 139747256
Change-Id: I345afbf6bc4043f274a4e7545a895b59c65641b0
2019-08-21 20:38:51 +00:00
Yi Kong
d255c7bc2f Switch to clang-r365631
Test: build, boot, bionic-unit-test
Test: For Googlers, go/clang-r365631-testing
Bug: 131328001
Change-Id: Ib357d9044a72e41f6bde1e5ec2db34e90706124d
2019-08-15 04:47:34 +00:00
Jiyong Park
72969547dc Merge "Fix sanitizer dep"
am: e49256e564

Change-Id: I2199bff441c53410030ccf7f48fc386a7e123c4f
2019-08-11 10:45:55 -07:00
Treehugger Robot
e49256e564 Merge "Fix sanitizer dep" 2019-08-11 17:12:47 +00:00
Jiyong Park
1d1119f4bd Fix sanitizer dep
This change fixes a problem in sanitizerMutator where a module is linked
with of non-sanitized variant of a lib at build-time, but is linked with
the sanitized variant of the lib at run-time.

This happened because, for each sanitizer type, every shared libs are
split into non-sanitized and sanitized variants, and then either of the
variants are suppressed from Make so that it isn't installed to the
device.

This change fixes the problem by NOT splitting for shared libs; only the
sanitized variant is created if needed. Header libs, static libs and
shared libs for a few sanitizer types (asan/fuzzer) are however split
into two. This is because the static and headers libs become part of the
depending module, and asan/fuzzer require that the depending module and
the dependant module should be compiled for the same sanitizer.

Bug: 138103882
Bug: 138426065
Test: m com.android.runtime.debug
Check that libziparchive exists under both
/system/apex/com.android.runtime/[lib|lib64]

Change-Id: Ia447785c485c0d049e19477b32bc638bfe6f1608
2019-08-08 01:51:26 +09:00
Mitch Phillips
cead9bad66 Merge "Disable LTO when building with fuzzer support."
am: df7d2cc9f8

Change-Id: Ie4f0e1c6266cca662c4eb133ce2bd9ce85b1dcd9
2019-08-05 11:39:28 -07:00
Mitch Phillips
34b493fec5 Disable LTO when building with fuzzer support.
Bug: 131771163

LTO is currently broken when building with SANITIZE_TARGET=fuzzer. The
compiler bug is currently being addressed upstream (see linked bug), but
we have applied a local workaround in the build system to disable LTO
when building using the fuzzer config.

There is a bug here however. In the sanitizer mutator we explicitly
remove -flto and add -fno-lto. The sanitizer mutator runs after the LTO
mutator, so (in general) this works just fine. The problem exists when a
target specifies an explicit 'lto: { ... }' flag in their Android.bp. In
this case, the sanitizer mutator disables LTO, then the flags are parsed
from the Android.bp, re-enabling LTO.

This patch fixes this issue. If the sanitizer mutator has added the
-fsanitize=fuzzer-no-link flags, then the LTO mutator won't add the LTO
flags after this fact.

Test: Build a target with SANITIZE_TARGET=fuzzer (or a cc_fuzz target),
where there is an explitiy 'lto: { ... }' and watch it now succeed in
building.

Change-Id: I6643909417f666539c23469816926b806e204b06
2019-08-02 16:57:55 -07:00
Dan Albert
233d006ef7 Merge "Remove asanLibs."
am: 6303423549

Change-Id: I4c5bc78924613aa6c61399400f468f7ff7353a53
2019-07-22 11:29:26 -07:00
Dan Albert
55a18ba71a Remove asanLibs.
We already link the shared prebuilt from the toolchain. This is
redundant and the wrong version.

Test: make checkbuild
Bug: http://b/74067984
Change-Id: I4b5d23514158eebfb4eea2aa6761e076131c3224
2019-07-19 12:53:27 -07:00
Mitch Phillips
88cddcf297 Merge "Re-enable ASAN globals instrumentation for devices."
am: 68c9eed6b2

Change-Id: I8a29cd64cd1c9a60adcc1bc8858163a1589cc20e
2019-07-12 13:20:19 -07:00
Mitch Phillips
cba311bcfc Re-enable ASAN globals instrumentation for devices.
ASan globals instrumentation apparently broke the build in the pass.
This is now a customer feature to get this up and running, let's see if
it works.

Bug: 137312732

Test: Build the platform, deploy and run an asan-ified test (maybe
resolv_integration_test?).

Change-Id: Ic4ee0db617c290d8a3fec52db4b90bc4772696b8
2019-07-11 17:16:58 -07:00
Colin Cross
e0318dec1b Merge changes Ia77090da,I95a57f76 am: e264f22bec
am: 0240f5cea1

Change-Id: I61e3e45e533cb62c548fcd09b2bf5eb3890e0a38
2019-06-21 10:51:45 -07:00
Colin Cross
e264f22bec Merge changes Ia77090da,I95a57f76
* changes:
  Fix data races in finder_test.go
  Optimize sanitizerRuntimeDepsMutator
2019-06-21 17:01:23 +00:00
Mitch Phillips
a87c8005b4 Merge "Remove experimental pass manager for fuzzer builds." am: c4be0815d8
am: 07c9775cef

Change-Id: I333316ff6041b6139b86bed5469885b42b6be96d
2019-06-20 18:17:19 -07:00
Treehugger Robot
c4be0815d8 Merge "Remove experimental pass manager for fuzzer builds." 2019-06-21 00:51:26 +00:00
Colin Cross
0b90833ea8 Optimize sanitizerRuntimeDepsMutator
sanitizerRuntimeDepsMutator only modifies the currently visited
module, it can visit modules in parallel.

Also, stop recursing into modules that are not static dependencies,
and stop recursing if the module already has all modifications that
the mutator could make.

Test: m checkbuild
Change-Id: I95a57f763a91940f1854ba3c587a2f70e8baba97
2019-06-20 15:23:41 -07:00
Mitch Phillips
74384758a6 Remove experimental pass manager for fuzzer builds.
Sanitizer coverage is currently broken with the experimental pass
manager. See b/133876586 for more information. The patch is currently
being worked on upstream (https://reviews.llvm.org/D62888), but is not
ready yet. Hence, we disable it here (similar to the LTO bug).

Bug: 133876586
Test: Build anything with SANITIZE_TARGET='fuzzer', verify that it has
'sancov' symbols.

Change-Id: I4302a8cf05300015aaff137f40b61d5685dc3fea
2019-06-17 10:33:52 -07:00
Evgenii Stepanov
d7344ad0ca Disable new LLVM pass manager for HWASan. am: 1c69e83030
am: b85d386af5

Change-Id: I443bdbf313216591841272cbe51491fd6bd51fbd
2019-06-15 14:13:04 -07:00
Evgenii Stepanov
1c69e83030 Disable new LLVM pass manager for HWASan.
Current Android toolchain does not support hwasan with new pm.

Bug: 135298400
Test: manual, inspect the binaries for calls to __hwasan_init
Change-Id: I126c59e6ecabc3a1699e1bf34c546568b6e8a21f
2019-06-14 18:39:59 -07:00
Mitch Phillips
bfeade6424 Fix fuzzer builds [#2].
- Updates the fuzzer builds to use SANITIZE_TARGET='fuzzer' instead of
'coverage'.
- Removed an old dependency that made fuzzer builds without ASan
an error.
- Fixed up the build flags to allow fuzzers to be built. Previously, the
coverage flags were manually provided. As the toolchain has moved on,
these flags are no longer compatible with libFuzzer, and so I've updated
them to use the correct, compatible flags.
- Added a dependency mutator for fuzzer coverage.

Bug: 121042685
Test: With all patches in the bug merged, build a fuzzer using
'SANITIZE_TARGET=fuzzer mmma <your_fuzzer>'.

Change-Id: Ib6246980f77bc4babe587b1e88038aa12228fa83
Merged-In: Ib6246980f77bc4babe587b1e88038aa12228fa83
2019-05-30 16:49:42 +00:00
Mitch Phillips
5a6ea6cf3b Fix fuzzer builds [#2].
- Updates the fuzzer builds to use SANITIZE_TARGET='fuzzer' instead of
'coverage'.
- Removed an old dependency that made fuzzer builds without ASan
an error.
- Fixed up the build flags to allow fuzzers to be built. Previously, the
coverage flags were manually provided. As the toolchain has moved on,
these flags are no longer compatible with libFuzzer, and so I've updated
them to use the correct, compatible flags.
- Added a dependency mutator for fuzzer coverage.

Bug: 121042685
Test: With all patches in the bug merged, build a fuzzer using
'SANITIZE_TARGET=fuzzer mmma <your_fuzzer>'.

This is the qt-dev-plus-aosp cherrypick of aosp/954925. See the AOSP
commit for further information.

Change-Id: Ib6246980f77bc4babe587b1e88038aa12228fa83
2019-05-28 15:46:11 -07:00
Evgenii Stepanov
6c8b9a0296 Disable malloc fill in HWASan.
am: 2c6484e334

Change-Id: I84cd62a3cc76006c968136b69bf4d88137b2f934
2019-05-15 15:28:45 -07:00
Evgenii Stepanov
2c6484e334 Disable malloc fill in HWASan.
HWASan has a feature to fill malloc() memory with non-zero pattern.
This has found a bug or two in the past, but it also keep causing hard
to debug issues. The main problem is lack of diagnostics - use of
pattern-initialized memory is not an immediate crash. More often than
not it manifests as a subtle change in system behavior.

Having MemorySanitizer on Android would be nice...

This change does not affect any shipping configuration, but improves
stability of testing-only SANITIZE_TARGET=hwaddress build.

Bug: 132803232
Bug: 132652537
Bug: 131438232
Bug: 132810685
Test: SANITIZE_TARGET=hwasan on crosshatch successfully boots in ~100%
      attempt (up from ~99% currently).

Change-Id: I7dcbcdf1b8a5b8556e32690327b32f04879e5db1
2019-05-15 13:53:37 -07:00
Christopher Ferris
422687005a Merge "Remove libc_scudo library when building asan." am: d7e66d9e7c
am: 486b170bbc

Change-Id: Ifd1c9021bad87c4a74104be311a04ba73a4419b6
2019-05-13 17:30:56 -07:00
Christopher Ferris
d7e66d9e7c Merge "Remove libc_scudo library when building asan." 2019-05-14 00:12:33 +00:00
Inseob Kim
c012b19eb0 Merge "Attach global variables to Context" am: a1591ab08c
am: 66c5cac22b

Change-Id: Icaf7aa78580459fd76605ec66b5bbaac13a30c9e
2019-05-09 19:42:18 -07:00
Christopher Ferris
753d4a69d5 Remove libc_scudo library when building asan.
Running any executable built with asan and libc_scudo results in crashes.

Bug: 131810078

Test: Built the media processes with the sanitizer enabled verifying
Test: that the libc_scudo.so shared library is not listed in the
Test: mediaextractor and the mediaswcodec binaries.
Test: Build the media processes without the sanitizer enabled verifying
Test: that the libc_scudo.so shared library is listed in the
Test: mediaextractor and the mediaswcodec binaries.
Change-Id: Ie55a0414088787ec11d85fffcb04592ed4f23c57
2019-05-09 13:35:37 -07:00
Inseob Kim
9516ee9556 Attach global variables to Context
Global variables make testing difficult, and they should attached to
Context.

Bug: N/A
Test: m
Change-Id: Ic671dda755e99d036c7ddce0eed114496374d7ec
2019-05-09 12:29:58 +09:00
Evgenii Stepanov
d98ccb44f2 Disable hwasan memory stats.
Disable a debugging feature in hwasan that seems to be causing flaky selinux
denials in the boot test:

05-02 12:11:11.360  1663  1663 I auditd  : type=1400 audit(0.0:8): avc: denied { read } for comm="iptables-wrappe" path="/proc/1015/statm" dev="proc" ino=30968 scontext=u:r:netutils_wrapper:s0 tcontext=u:r:netmgrd:s0 tclass=file permissive=0

I could not reproduce this locally, but this feature has no users and
I've verified that things keep working without it.

This does not affect any shipped product.

Bug: 131438232
Bug: 112438058
Bug: 131845076
Test: crosshatch_hwasan boot test
Change-Id: I08d46d19151e6d5221cdd56fd6864dae224fae25
2019-05-02 12:52:32 -07:00
Peter Collingbourne
e726ba56ac Pass -fsanitize-hwaddress-abi=platform when building with hwasan.
Reduces size of walleye_hwasan-userdebug system.img by another 4.5MB.

Bug: 122363025
Change-Id: I6d15c237dc5f330ad574a73147b2b2c712fe125e
2019-03-21 16:22:00 -07:00
Peter Collingbourne
967511a4e8 Pass -hwasan-allow-ifunc when building with hwasan.
Now that the toolchain and bionic both support ifuncs in
static executables, we can turn this on. Reduces size of
walleye_hwasan-userdebug system.img by 11MB.

Change-Id: I2f19b22518153328eee3fdca79d0c851cc513c3a
2019-03-20 19:42:42 +00:00
Peter Collingbourne
bd19db0da6 Remove workarounds for old compiler versions.
LLVM r349610 was released, so these workarounds are no longer required.

Bug: 112907825
Change-Id: Ia6ba0034881f2ab657d8420df3047c21757c7d9b
2019-03-06 10:48:36 -08:00
Jiyong Park
f97782b18c Remove prefer_sanitize.* properties
The properties are no longer required as an APEX module is mutated with
required sanitizers which are gathered by scanning the sanitizers that
are enabled for its direct dependencies.

Bug: 124128094
Test: m on marlin
The extractor libs are found under /system/apex/com.android.media

Merged-In: I55961d400dcbac067a5c0dcecb90e399d4991a70
Change-Id: I55961d400dcbac067a5c0dcecb90e399d4991a70
(cherry picked from commit abda0eb76b)
2019-02-14 12:30:49 +09:00
Colin Cross
571cccfcbc Prepare for a type-safe OnceKey
Add an opaque OnceKey type and use it for all calls to Once in
build/soong.  A future patch will convert the arguments to
Once* to OnceKey once users outside build/soong have been updated.

Test: onceper_test.go
Change-Id: Ifcb338e6e603e804e507203c9508d30ffb2df966
2019-02-06 01:52:41 +00:00
Kostya Kortchinsky
d5275c8657 Add option to disable Scudo globally [Soong]
This adds an option in Soong to turn off Scudo globally.

Bug: 123228023
Test: enable Scudo for tombstoned, lunch marlin_svelte-eng && m -j, and
make sure that Scudo is not linked in
out/target/product/marlin/system/bin/tombstoned
Test: enable Scudo for tombstoned, lunch marlin-userdebug && m -j, and
make sure that Scudo is linked in
out/target/product/marlin/system/bin/tombstoned

Change-Id: I0b0992446953fc4074bde94507b66f92764c8143
Merged-In: I0b0992446953fc4074bde94507b66f92764c8143
2019-02-04 12:29:17 -08:00
Jiyong Park
a2aca28750 Use bootstrap linker for asan build
This change the asan version of
https://android-review.googlesource.com/c/platform/build/soong/+/879237

Bug: 123725034
Test: forrest run on asan_cf_x86_pasan
Test: SANITIZE_TARGET=address m init_second_stage
inspect /system/bin/init with readelf
DT_INTERP is set to /system/bin/bootstrap/linker_asan

Change-Id: I0fabe08cc63126aadec18ca8e9f2973c33411a4f
2019-02-02 13:28:57 +09:00
Evgenii Stepanov
93c3f5368d [hwasan] Remove deprecated compiler flag (NFC)
-hwasan-with-ifunc=0 matches the default value of the flag, and there
are no plans to change that.

Test: SANITIZE_TARGET=hwaddress keeps working
Change-Id: Ie46ffa6ae3c42a9677c3a9a3b5cdc6fb62c2c2f0
2019-02-01 22:56:03 +00:00
Jiyong Park
8222663f1b Fix: runtime apex is not instrumented with sanitizer
Fixing the problem that IsSanitizerEnabled always returns false due to
the different sanitizer names from config.SanitizerDevice() and
sanitizerType.String().

Bug: 123708856
Test: SANITIZE_TARGET=hwaddress m com.android.runtime.release
$ ls -al out/soong/.intermediates/art/build/apex/com.android.runtime.release/android_common_hwasan_com.android.runtime.release/image.apex/lib | wc -l
34
$ ls -al out/soong/.intermediates/art/build/apex/com.android.runtime.release/android_common_hwasan_com.android.runtime.release/image.apex/lib64 | wc -l
35
$ ls -al out/target/product/blueline/apex/com.android.runtime.release/lib | wc -l
34
$ ls -al out/target/product/blueline/apex/com.android.runtime.release/lib64 | wc -l
35

Test: inspect out/soong/build.ninja
- Variant of com.android.runtime.release is "android_common_hwasan_com.android.runtime.release"
- Dependencies of com.android.runtime.release are hwasan variants: e.g.,
  out/soong/.intermediates/bionic/libc/libc/android_arm64_armv8-2a_kryo385_core_shared_hwasan_com.android.runtime.release/libc.so

Test: ls out/soong/.intermediates/art/build/apex/com.android.runtime.release/android_common_hwasan_com.android.runtime.release/image.apex/lib64/*.so  | xargs readelf -d | grep libclang_rt.hwasan | wc -l
29

Change-Id: Idbb1f68a3ea6bdd02351d6f6048c3eedb9cf32d2
2019-02-01 11:26:04 +09:00
Jiyong Park
3b1746a212 Fix: deps to sanitizer runtime libs are with correct 'image' variant
This change fixes a bug that sanitizer runtime libs for non-core variant
(e.g. recovery, vendor, etc.) are not installed. It happened because the
dependency to the sanitizer runtime lib was without 'image' variant,
which in most case caused only the core variant - which is the first in
the image variants - of the lib to be installed.

Fixing the issue by correctly selecting image variant depending on the
location of the lib having dependency to the runtime lib.

Bug: 123525879
Test: SANITIZE_TARGET=hwaddress m out/target/product/blueline/boot.img
Test: SANITIZE_TARGET=address m out/target/product/blueline/boot.img
libclang_rt.*.so is under
out/target/product/blueline/root/recovery/system/lib64

Change-Id: Iea7d718d4971e36521f0a3f712a454de944cd7ac
2019-01-31 01:58:26 +09:00
Jiyong Park
388ef3f5aa Create sanitizer variants of APEX only when SANITIZE_TARGET is set
This fixes a problem that APEX is unconditionally mutated for all
sanitizer types. This can make an APEX to include sanitized version
of a lib even when SANITIZE_TARGET is not set. It can happen when lib X
is directly depended on by an APEX (e.g. via native_shared_libs) and X
has a sanitized variant due to the dependency from another library Y
which is force sanitized (via 'sanitize' property). In that case,
regardless of lib Y is in the APEX or not, the APEX chooses the
sanitized variant of lib X since the dependency from APEX to the lib is
created with AddFarVariationDependency.

Fixing this problem by mutating the APEX for a sanitizer type only when
the device is requested to be sanitized.

Bug: 122717287
Test: add libnetd_client to com.android.runtime APEX.
Inspect build.ninja to verify that libnetd_client.so in the runtime APEX
is not a sanitized one.

Change-Id: I918bc8407137d74c5456142b3a29de13df68c0b3
2019-01-29 07:10:31 +09:00
Jiyong Park
64a44f231d Make sure that ASAN runtime lib is the first DT_NEEDED one
Introduce a new tag earlySharedDepTag which is added in front of the
ordinary sharedDep dependencies. Dependency to the ASAN runtime lib is
added with the new tag.

Bug: 120894259
Bug: 121038155
Test: m; SANITIZE_TARGET=address m
Use readelf -d to see if the runtime lib is in the first DT_NEEDED one.

Change-Id: I90de6ab95df734a99995532d826564b13fe05316
2019-01-25 14:05:51 +09:00
Pirama Arumuga Nainar
6aa210243c Revert "Revert "Fix: sanitizerRuntimeMutator is not run for host""
Bug: http://b/123254047

This reverts commit 84757ea7ea.

Reason for revert: veridex asan failure is fixed.

Change-Id: Id03180b8d3d1dc1386fd7e4f800d64dc8578139c
2019-01-25 00:21:02 +00:00
Doug Horn
c32c6b0d79 Support building for Fuchsia.
This CL adds configs for the arm64 and x64 fuchsia
device targets, sets up the necessary linker flags,
and disables some functionality that is not currently
supported on Fuchsia.

Bug: 119831161
Test: Compile walleye, internal validation against
fuchsia_arm64-eng and fuchsia_x86_64-eng.
Change-Id: I2881b99d2e3a1995e2d8c00a2d86ee101a972c94
2019-01-24 18:59:29 +00:00
Dan Albert
84757ea7ea Revert "Fix: sanitizerRuntimeMutator is not run for host"
This reverts commit 1c29b0e6ec.

Reason for revert: Broke aosp_x86_64

Change-Id: I581200f1c271c3e1a39d5cdad84c782977b13054
2019-01-23 23:33:25 +00:00
Jiyong Park
1c29b0e6ec Fix: sanitizerRuntimeMutator is not run for host
Bug: 123254047
Test: m ASAN_OPTIONS=detect_leaks=0 SANITIZE_HOST=address
FORCE_BUILD_LLVM_COMPONENTS=true out/host/linux-x86/bin/adb
nm -an out/host/linux-x86/bin/adb  | grep -c asan
shows non-zero value

Change-Id: I030e08bd0db38585b1fdaecd00220406f1e8b371
2019-01-23 23:37:45 +09:00
Jiyong Park
379de2fab0 APEXes can be sanitized
This change first introduces the interface "Sanitizeable" that
module types other than cc.Module can use to be handled by the sanitizer
mutator. APEX module, by implementing the interface, gets sanitizer
variants.

In doing so, sanitizer.go is refactored so that modules have explicit
dependencies to the runtime sanitizer libraries. This allows the runtime
library to be packaged into the APEX when required. This also completes
the dependency graph; updating the prebuilt sanitizer runtime will
trigger rebuilding of modules using the runtime.

Bug: 120894259
Bug: 121038155
Test: SANITIZE_TARGET=hwaddress m apex.test
Test: TARGET_FLATTEN_APEX=true SANITIZE_TARGET=address m
Change-Id: Ia91576ff48cda3c996350308b75bf83fcf7c23d7
2019-01-13 00:48:47 +09:00
Evgenii Stepanov
3c5a52a477 [hwasan] Reduce history size to save memory.
Bug: 112437883
Test: adb shell echo $HWASAN_OPTIONS
Change-Id: Ifc4ab10e37efb1a0ce7ea77ad691947d24c2da2b
2018-12-18 17:02:44 -08:00
Ivan Lozano
7929bba218 Add option to avoid recovering with diagnostics.
Add a no_recover option that allows specifying which sanitizers in
diagnostics modes shouldn't recover. This can help debugging as we test
enabling sanitizers in new libraries since it'll cause tombstones to be
generated along with the diagnostics information.

Bug: 80195448
Bug: 110791537
Test: Compiled test module with this flag, checked compiler command.
Test: Test module crashed, tombstone contained diagnostics information.
Change-Id: Ie938923ef95716c2d49d6cd719ee223ad62755c6
2018-12-12 10:01:16 -08:00
Evgenii Stepanov
0a87b664c3 Disable hwasan frame descriptions.
Current implementation does not play nice with -gc-sections.

Bug: 120673911
Test: make SANITIZE_TARGET=hwaddress recovery-persist
Change-Id: I36cd37fb41c0c26c7e747e2c1dd5fadf7a31f4e7
2018-12-07 15:33:24 -08:00
Chih-Hung Hsieh
3567e62f97 Switch to clang-r346389.
* Add -fno-sanitize=implicit-integer-sign-change when
  there is any integer related sanitizer, and this check
  is not explicitly specified.
  Android core does not boot with this new sanitizer yet.
* Filter out -fno-sanitize=implicit-integer-sign-change
  from tooling flags.

Bug: 119329758
Bug: 119557795
Test: make checkbuild, boot, go/clang-r346389-testing
Change-Id: I709de569cb73d070fc4958e2b4387f4041bc5438
2018-12-04 19:50:45 +00:00
Evgenii Stepanov
98f5b0642c Add sanitizer runtime library to recovery when needed.
Make sanitized targets in recovery depend on ".recovery" flavor of
sanitizer runtime library. This forces installation of the runtime
library in recovery image as long as at least one target depends on it.

Test: hwasan build can boot into recovery
Bug: 112438058
Bug: 119582521
Change-Id: I1b7f9fa4ff5f29acb03c27074512ea24467a3d37
2018-11-29 23:22:20 +00:00
Peter Collingbourne
8c7e6e2626 Add an "scs" flag for enabling shadow call stack on targets.
Shadow call stack (SCS) is a security mitigation that uses a
separate stack (the SCS) for return addresses.

The effect of setting sanitize.scs on a shared library is to build
the library and all of its static library dependencies with SCS. This
is similar to CFI and the other sanitizers.

Bug: 112907825
Bug: 119557795
Change-Id: I82fb2b38b10eac911c4d2d120b74fea4af0622ad
2018-11-19 17:39:23 -08:00
Evgenii Stepanov
1e79844f15 Apply HWASan to libc-in-recovery.
HWASan is disabled in recovery strictly for code size, not for
correctness. With this change, HWASan binaries and libraries can be
used in recovery.

The real reason is the make build system which has no idea of the big
picture, and makes it hard to disable HWASan in _all_ recovery modules
recursively. A few static libraries have sneaked in.

Bug: 119582521
Test: sideload an OTA onto a hwasan device
Change-Id: I6536cc7bf5bfb37f939b585545a23c2068fbd47f
2018-11-15 17:34:18 -08:00
Peter Collingbourne
abde584fd1 Stop passing the gold plugin path to llvm-ar.
llvm-ar takes a --plugin argument but it is ignored for compatibility,
so passing it has no effect.

Change-Id: I4fc51d226d66cf2a43462d3d4ccc12e6e5ebb226
2018-11-08 20:13:16 -08:00
Yi Kong
8c3c0f50d0 Revert "Prevent clang from using the x18 register on arm64."
This reverts commit 98d8580c3a.

Change no longer needed with the new LLVM toolchain update.

Bug: 112907825
Bug: 111759196
Change-Id: Ife3ce47a602dd850ac84bb51abc70c0ca157776e
2018-11-08 21:07:08 +00:00
Peter Collingbourne
98d8580c3a Prevent clang from using the x18 register on arm64.
This is achieved via the compiler flag -ffixed-x18 for non-LTO builds
and the linker flag -plugin-opt -mattr=+reserve-x18 for LTO builds.

This change should be reverted once we upgrade past LLVM r340889
which does this by default on Android.

Bug: 112907825
Bug: 111759196
Change-Id: I05473ddbb98319d87d442425b4d715647eae3a38
2018-10-24 23:40:49 +00:00
Evgenii Stepanov
109029fb5c Define HWADDRESS_SANITIZER_GLOBAL_OPTIONS.
It will be exported in device environment as HWASAN_OPTIONS.

Bug: 112438058
Test: none
Change-Id: I574b46060b3193c9f2351116a91ebc35470ae803
2018-10-12 18:18:13 +00:00
Kostya Kortchinsky
ad73b2e18a Scudo minimal runtime support for Soong
Scudo is now compatible with the -fsanitize-minimal-runtime, and offers a new
dynamic library that doesn't bundle UBSan.

This patch adds support for this new library in Soong, preferring it over the
full one, unless a UBSan or diagnostic dependency is found.

Test: aosp compiled with m -j
Test: local test enabling Scudo for tombstoned
Change-Id: I17794131db148b33f8a8710ac43302cadf1af314
2018-10-11 11:55:16 -07:00
Ivan Lozano
ae6ae1d880 Apply minimal runtime to the rest of ubsan.
Make sure that the minimal runtime gets applied to the rest
of the sanitizers we might use from ubsan.

Bug: 80195448
Test: Build success and checked build flags.
Change-Id: I31b211b80bb1068c153a9b65454a2605933c9568
2018-10-09 08:05:27 -07:00
Dan Willemsen
8536d6b3b7 Remove GCC checks
Clang is always used now, so we can remove all the GCC checks. Removing
GCC-specific configuration will happen in the next CL.

Test: m
Change-Id: I4835ecf6062159315d0dfb07b098e60bff033a8a
2018-10-09 02:16:58 +00:00
Evgenii Stepanov
dbf1d4f884 (soong) Add -fsanitize= argument to assembly flags.
It allows use of sanitizer preprocessor macros (like __has_feature())
in assembly files.

Bug: 112438058
Test: SANITIZE_TARGET=hwaddress
Change-Id: Id3fed5492ab9473cbb13cd6c0485302412e8b1de
2018-09-07 10:23:07 -07:00
Evgenii Stepanov
d97a6e9483 Add "hwaddress" sanitizer.
This is a new sanitizer similar to ASan, but with a few differences
from the build system perspective:
* Only runs on AArch64.
* Supports static binaries.
* Bionic itself will be built with HWASan.
* Does not have any "if a library is sanitized than the binary must
  be, too" requirements unlike ASan. Even better, individual static
  libraries can be freely sanitized or not. We propagate "nosanitize"
  from binaries to static libraries anyway, because otherwise there
  is no good way to disable hwasan on a binary as a whole.

Same a CFI, we export a list of sanitized static libraries to make.
In fact, we export separate lists for vendor and regular libraries,
because it is possible for one to be sanitized without the other
(i.e. there can be mylib.hwasan.vendor w/o mylib.hwasan or vice
versa).

Bug: 112438058, 112709969
Test: manual, part of a bigger patch set

Change-Id: Ie4fdeb522ac03cf9684526882e84dfee3807b6a7
2018-08-16 13:18:47 -07:00
Pirama Arumuga Nainar
b6682c47b8 Merge "Do not enable sanitizers on Windows"
am: d7e5c05eac

Change-Id: I7398e6b08154c5bc7fad0507bd9e1dae8c14da0a
2018-06-26 17:05:30 -07:00
Pirama Arumuga Nainar
83d716c413 Do not enable sanitizers on Windows
Bug: http://b/69933068

Test: m native-host-cross SANITIZE_HOST=address
Change-Id: Ia9ff941fb569b0a2e9c4eef604fd165f94bca31a
2018-06-26 14:18:16 -07:00
Colin Cross
e98238b3ab Merge "Use WalkDeps instead of VisitDepsDepthFirst"
am: cbafd0fb48

Change-Id: Ic22ffa9ac8a5e3b5c29cd5e85664acb1afebc04f
2018-06-22 17:14:31 -07:00
Colin Cross
6b75360cbc Use WalkDeps instead of VisitDepsDepthFirst
VisitDepsDepthFirst is almost never correct, as it can't query
dependency tags of multiple dependencies between the same two modules.
Replace VisitDepsDepthFirst with WalkDeps in sanitize.go and
python.go.  Also verify the dependency tag before continuing to
recurse to ensure that they don't recurse through genrules and into
unrelated modules.

Test: m checkbuild
Change-Id: I2f7560126f56b51a40ec39dfbdcc18b5891489f7
2018-06-22 12:44:35 -07:00
Kostya Kortchinsky
9cba8204b1 Soong support for Scudo
am: d18ae5ce98

Change-Id: I0c0fcf29ccd99575de573a915c7da5d51b14cd45
2018-06-19 13:11:58 -07:00
Kostya Kortchinsky
d18ae5ce98 Soong support for Scudo
Scudo is a hardened usermode allocator that is part of LLVM's compiler-rt
project (home of the Sanitizers). clang allows for -fsanitize=scudo as a
possible command line option to link the shared Scudo library to a binary.

This patch add Scudo as a potential sanitize option. Scudo is not compatible
with ASan and TSan and will be disabled if either is enabled.

Bug: 72112048
Test: aosp compiled with m -j
Test: local experiment with scudo: true to ensure that a test target
(tombstoned) could be linked with scudo.

Change-Id: I76bb6c60891d4782f6665a112c4c2bf7c31645da
2018-06-19 09:46:09 -07:00
Vishwath Mohan
722b64e0ad Don't enable diagnostics for CFI when setting the CFI sanitizer. am: 05b002dea0
am: dc55e621bb

Change-Id: I04214a9fcb50363bd5d82362b366e5bf69355183
2018-05-31 11:03:56 -07:00
Vishwath Mohan
05b002dea0 Don't enable diagnostics for CFI when setting the CFI sanitizer.
The SetSanitizer() method in cc/sanitize.go was also enabling
diagnostics for CFI when invoked from the sanitizer mutator. We
shouldn't be doing this, so this CL disables that.

Bug: 79785496
Test: strings libziparchive.so | grep 'ubsan'
Change-Id: Iffe0d4e9c333ac5dfbec3f48e381c232ce5d4463
2018-05-30 20:52:39 -07:00
Vishwath Mohan
4cac067ca1 Disable CFI for vendor variants of all components
am: 9ccbba0200

Change-Id: I6a6a839475f58f92180bfd391035dd538df71a50
2018-05-29 16:36:13 -07:00
Vishwath Mohan
9ccbba0200 Disable CFI for vendor variants of all components
The current VNDK ABI snapshot expects that CFI is disabled for
components that either in the include paths (from cfi-common.mk) OR
enabled directly in the makefile/blueprint. However, on non-arm64
architectures, the build system was only disabling CFI for vendor
components in the include paths.

This topic corrects it by (a) removing the include paths check to
disable CFI for vendor variants (this CL), and (b) inheriting the CFI
include paths in all product configs to ensure that when we update the
ABI we're able to move to relying exclusively
on (PRODUCT_)CFI_INCLUDE_PATHS.

Bug: 66301104
Test: ABI matches for all architectures.
Change-Id: I648edf13346b18fd88b623682e8590ed44709e0d
2018-05-28 13:57:48 -07:00
Vishwath Mohan
40aaea2321 Restrict CFI_INCLUDE_PATHS to Arm64
am: 1c54f66bd5

Change-Id: I379932454f8f18d7029f3fa3eb1654f4f36997c1
2018-05-25 00:24:53 -07:00
Vishwath Mohan
1c54f66bd5 Restrict CFI_INCLUDE_PATHS to Arm64
This CL restricts CFI_INCLUDE_PATHS and PRODUCT_CFI_INCLUDE_PATHS to
Arm64 only.

Bug: 66301104
Test: x86 targets do not respect the include paths variables
Change-Id: I66ec2fb05435535aaf5d59fdfc7a170a4fdd4f26
2018-05-24 18:36:18 -07:00