Whenever any two Rust static libraries are included
as static libraries anywhere in a CC dependency tree, we sometimes
get duplicate symbol errors. To avoid this, we no longer
directly link multiple rust static libs to CC modules.
Instead, we build rust_ffi_rlib modules and produce the actual
static library that gets linked against the CC module based on
that CC module's full list of Rust rlib dependencies.
This introduces a new static_rlibs property for cc modules to
define the rust_ffi_rlib dependencies, which are then used to
generate the module above.
This CL is intended to deprecate rust_ffi_static. It leaves
rust_ffi_static and rust_ffi static variants in place until
the remaining rust_ffi_static declarations and uses can be
removed. In the meantime, rust_ffi_static produces
rust_ffi_rlib variants as well to make the transition easier.
Bug: 254469782
Test: m # with no changes
Test: m libapexsupport # with static_rlibs
Test: m libunwindstack # with static_rlibs
Test: m netsimd # with static_rlibs, no duplicate symbols
Test: m blueprint_tests # New Soong tests
Change-Id: I47e27ac967ef0cad46d398ebf59d8275929ae28a
LOCAL_STATIC_LIBRARIES, LOCAL_WHOLE_STATIC_LIBRARIES,
and LOCAL_HEADER_LIBRARIES are only exported to Make
so that it can generate module-info.json. Export
ModuleInfoJSONProvider from cc modules so that Soong can
generate the module-info.json entries, and remove the
properties from the generated Android.mk. This will prevent
Kati reanalysis when making some Android.bp changes.
Bug: 309006256
Test: Compare module-info.json
Test: Compare Kati's build.ninja
Change-Id: I6660f6802b9cea46eed553cac12f09a373eeb019
This wasn't needed when fuzzers had their own
lib directories, but because the lib directories
were combined, this caused interactions because
system and vendor libraries were mixed in the
same folder, sometimes overriding each other.
Comments are left with some details about
future issues. Bug has a series of recommendations
to prevent issue again.
The main goal of this patch is to get the fuzzers
working, not fix every issue here.
Bug: 307611931
Test: run vendor fuzzer and check lsof:
android.hardwar 22719 root txt REG 254,94 4350056 3455 /data/fuzz/x86_64/android.hardware.drm-service.<name>.aidl_fuzzer/vendor/android.hardware.drm-service.<name>.aidl_fuzzer
android.hardwar 22719 root mem REG 254,94 4350056 3455 /data/fuzz/x86_64/android.hardware.drm-service.<name>.aidl_fuzzer/vendor/android.hardware.drm-service.<name>.aidl_fuzzer
android.hardwar 22719 root mem REG 254,94 1047232 3464 /data/fuzz/x86_64/lib/vendor/libbase.so
android.hardwar 22719 root mem REG 254,94 4027304 3460 /data/fuzz/x86_64/lib/vendor/libcrypto.so
android.hardwar 22719 root mem REG 254,94 4021632 3467 /data/fuzz/x86_64/lib/vendor/libbinder.so
android.hardwar 22719 root mem REG 254,94 329408 3466 /data/fuzz/x86_64/lib/vendor/libcutils.so
android.hardwar 22719 root mem REG 254,94 508968 3463 /data/fuzz/x86_64/lib/vendor/libutils.so
android.hardwar 22719 root mem REG 254,94 592944 3465 /data/fuzz/x86_64/lib/vendor/libclang_rt.ubsan_standalone-x86_64-android.so
android.hardwar 22719 root mem REG 254,94 2328616 3462 /data/fuzz/x86_64/lib/vendor/libc++.so
and run a system fuzzer
servicemanager_ 27878 root txt REG 254,94 609736 67408 /data/fuzz/x86_64/servicemanager_fuzzer/servicemanager_fuzzer
servicemanager_ 27878 root mem REG 254,94 609736 67408 /data/fuzz/x86_64/servicemanager_fuzzer/servicemanager_fuzzer
servicemanager_ 27878 root mem REG 254,94 4390560 4074 /data/fuzz/x86_64/lib/libvintf.so
servicemanager_ 27878 root mem REG 254,94 13872 4077 /data/fuzz/x86_64/lib/libvndksupport.so
servicemanager_ 27878 root mem REG 254,94 13872 4077 /data/fuzz/x86_64/lib/libvndksupport.so
servicemanager_ 27878 root mem REG 254,94 13872 4077 /data/fuzz/x86_64/lib/libvndksupport.so
servicemanager_ 27878 root mem REG 254,94 11584 4086 /data/fuzz/x86_64/lib/libdl.so
and run a system fuzzer
servicemanager_ 27878 root txt REG 254,94 609736 67408 /data/fuzz/x86_64/servicemanager_fuzzer/servicemanager_fuzzer
servicemanager_ 27878 root mem REG 254,94 609736 67408 /data/fuzz/x86_64/servicemanager_fuzzer/servicemanager_fuzzer
servicemanager_ 27878 root mem REG 254,94 4390560 4074 /data/fuzz/x86_64/lib/libvintf.so
servicemanager_ 27878 root mem REG 254,94 13872 4077 /data/fuzz/x86_64/lib/libvndksupport.so
servicemanager_ 27878 root mem REG 254,94 13872 4077 /data/fuzz/x86_64/lib/libvndksupport.so
servicemanager_ 27878 root mem REG 254,94 13872 4077 /data/fuzz/x86_64/lib/libvndksupport.so
servicemanager_ 27878 root mem REG 254,94 11584 4086 /data/fuzz/x86_64/lib/libdl.so
Change-Id: I91a51d2f1cb537cfaae8379998078188f2b10a98
Convert all of the callers of OtherModuleProvider/OtherModuleHasProvider
to use the type-safe android.OtherModuleProvider API.
Bug: 316410648
Test: builds
Change-Id: Id77f514d68761a262d9ea830a601dbed804bbbe5
Remove the ConvertWithBp2build implementations from all the module
types, along with the related code.
Bug: 315353489
Test: m blueprint_tests
Change-Id: I212672286686a318893bc7348ddd5a5ec51e77a7
To generate module-info.json in Soong for b/309006256 Soong needs to
know the test data paths. Moving test data installation into Soong will
also help later for test suite packaging.
Add ModuleContext.InstallTestData that installs the files listed in a
[]DataPath alongside the test. The files will also be passed to Make
to allow it to continue packaging them into the test suites for now.
Update the module types that are producing LOCAL_TEST_DATA entries
in their Android.mk files to go through InstallTestData instead.
Bug: 311428265
Test: atest --host toybox-gtests --test-timeout=120000
Change-Id: Ia8b964f86e584ea464667fd86a48d754d118bead
Provider added for the following test modules in this change: art_cc_test, cc_benchmark, cc_fuzz, cc_test, cc_test_host, rust_test,and rust_test_host.
Bug: 296873595
Test: Manual test
Change-Id: I815680529bcbecacb3a2bdb8f3746053afdee48c
Rust shared library dependencies are not always bundled in cc_fuzz or
rust_fuzz modules, which can lead to difficult to debug runtime errors
when running these fuzzers. It can also be hard to determine which
dependencies need to be explicitly declared.
This CL makes sure that we bundle the appropriate transitive
dependencies for our fuzzers.
Bug: 249551848
Test: Soong tests
Test: m <fuzzer> # check data/fuzz/<arch>/lib dir contents
Change-Id: I957ca8898079b61e2ff20d750f8c92bf61ac394f
Prevent collisions between host and host cross fuzz packaging when
both are supported as fuzz targets by using a different value for
hostOrTargetString.
Test: builds with HOST_CROSS_OS := linux_musl
Change-Id: I0e82bab2987899cece94c89eb8398d1733ebbd15
Shared libraries in fuzz zip files may have a different filename than
their source file, for example when the source file is a prebuilt in
the source tree. Propagate the computed library name through
FilesToZip to soong_zip to use the correct name in the zip file.
Bug: 254867347
Test: builds
Change-Id: Ib0def8decdbec7842b8e1a9056fac373d743bac0
Some device specific prebuilts have a suffix that varies depending on the lunch target.
The resulting install dependency binary has the suffix.
The fuzzer build expects the presubmit binary without the suffix.
This change is to drop the suffix.
Test: make haiku
Change-Id: I8eadd84bf5797659b17bc428dca47b7c3f382aff
Fix an issue where rust_fuzz modules were
not correctly packaging and copying their
dependencies.
This is done by extending the CC fuzz packager
to simply handle both Rust and CC modules,
ensuring this doesn't get out of sync again.
Fuzzer related functions are added to the
Linkable interface to facilitate this.
There was a bug where the Make definitions for
Rust fuzzers were not being created as well,
and that is addressed here.
Bug: 249551848
Test: m android_log_fuzzer #check $OUT/data/fuzz/arm64/libs
Change-Id: I9b41153e0cf08ab510476b75003c3a3baccc3858
Previously, as all versioned sdk snapshots have been removed from
Android.bp files this method would always return false. This change
effectively replaces all calls to it with false, and then optimizes
away any unused code.
Bug: 260237150
Test: m nothing
Change-Id: I1b717ee8345e807bd888451f6e7e3c3a0d391ee2
Previously, various places duplicated the logic to check whether a
module was part of a versioned snapshot. This change replaces that code
with a call to the existing IsModuleInVersionedSdk(Module) func.
This makes the code consistent in preparation for a change that will
remove all calls to IsModuleInVersionedSdk(Module). It is separated out
from that change as it is easier to see that the replaced code is
equivalent to the IsModuleInVersionedSdk(Module) func when it is not
mixed in with other changes. It also makes the following change easier
to review.
Bug: 260237150
Test: m nothing
Change-Id: I44d665bf02d2fc97632a03b5a3d2225ea3c2827d
For "normal" fuzz targets, the binary goes in
/data/fuzz/<arch>/<fuzzer_name>/fuzzer_name>
with libraries in
/data/fuzz/<arch>/lib/
If 'vendor: true' is set, the fuzz target binary get placed in
/data/fuzz/<arch>/<fuzzer_name>/vendor/<fuzzer_name>.
We need an extra '..' in the rpath to link to the included libraries
correctly.
Bug: 254723623
Test: built libsrtp2-fuzzer, checked linking with ldd
Change-Id: I24282d7b252142ed795a6091fece2d56125329dc
This allows us fuzzers to get coverage feedback and mutate more
intelligently based on feedback from strcmp, memcpy, and similar
functions.
Bug: 208297094
Test: make and run 'example_fuzzer', build 'haiku'
Change-Id: Ic51c207f8644cda885fb3f453cc583e2ee1e8c1d
Make rust and cc fuzzers collect their shared libraries once in
GenerateAndroidBuildActions and store it for later use by the
packaging singleton. Also use android.OutputFileForModule to get
the paths. Together this will fix fuzzers that depend on architecture
specific prebuilt shared libraries that are missing a prebuilt for an
architecture when building with AllowMissingDependencies.
Bug: 250918230
Test: lunch aosp_riscv64-userdebug && m ALLOW_MISSING_DEPENDENCIES=true nothing
Change-Id: I154a6f3a767c883e9fe7067003615db73ee78e2d
Test: Built AFL fuzzers individually and built all using haiku command
and built libfuzzers individually and also by using haiku command. Ran
selected fuzzers manually to ensure fuzzing still worked.
Description: Previously we needed to add cc_afl_fuzz to build an afl fuzz binary,
however, to turn current libFuzzers into AFL fuzzers this would required
an update to each Android.bp file which is a lot of work, and would also
require an approval from each Android.bp file owner, which is even more
work.
To get around this (and also to match how AFL fuzzers are built in G3)
we will build AFL++ fuzz binaries by command line option FUZZ_FRAMEWORK.
When FUZZ_FRAMEWORK=AFL is set, all cc_fuzz modules will be built
for AFL rather than libFuzzer. Devs can also specify if a cc_fuzz module
is only for libFuzzer or AFL by using fuzzing_frameworks. If
fuzzing_frameworks is left blank then it will be assumed that the
cc_fuzz module can be built for all available fuzzing frameworks.
Change-Id: If57d3038f05e52775177eaeb26f8ed2bdc73443a
Revert submission 2164482-Build AFL from Command line option
Reason for revert: Unit test fails on MAC builds
Reverted Changes:
Ia7a822462:Updated the way we build AFL++ fuzz binaries
I994bde63b:Updated the way we build AFL++ fuzz binaries
Ie863853b4:Updated the way we build AFL++ fuzz binaries
I577d6ae7b:Updated the way we build AFL++ fuzz binaries
Ic4fd34f8d:Updated the way we build AFL++ fuzz binaries
Change-Id: Ic2cda7733d206d7ed178f1db5e38268bc915e57b
Test: Built AFL fuzzers individually and built all using haiku command
and built libfuzzers individually and also by using haiku command. Ran
selected fuzzers manually to ensure fuzzing still worked.
Description: Previously we needed to add cc_afl_fuzz to build an afl fuzz binary,
however, to turn current libFuzzers into AFL fuzzers this would required
an update to each Android.bp file which is a lot of work, and would also
require an approval from each Android.bp file owner, which is even more
work.
To get around this (and also to match how AFL fuzzers are built in G3)
we will build AFL++ fuzz binaries by command line option FUZZ_FRAMEWORK.
When FUZZ_FRAMEWORK=AFL is set, all cc_fuzz modules will be built
for AFL rather than libFuzzer. Devs can also specify if a cc_fuzz module
is only for libFuzzer or AFL by using fuzzing_frameworks. If
fuzzing_frameworks is left blank then it will be assumed that the
cc_fuzz module can be built for all available fuzzing frameworks.
Change-Id: Ia7a8224627f2de61606b410d1d1a56d7bdc0955f
Each conversion required defining a separate mutator, which will each
operate on _all_ modules and requires each to repeat checks whether the
mutator should operator. Instead, we introduce a single mutator and
modules can define a ConvertWithBp2build to implement bp2build
conversion for that module.
Test: bp2build.sh
Bug: 183079158
Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
libc's native_bridge shared variant needs to be installable: false,
make the installable property arch-variant and allow the static and
shared clauses to override it.
This relands I8e0eaed2169e89520b07bb0326bb97ac0b4a70ac with fixes to
allow only the shared variant to be marked installable: false.
Bug: 204136549
Test: m checkbuild
Change-Id: Ie579b5ba56ca21f4d9243088afd5078d8c122a68
Rust fuzzers were not packaging up their CC shared dependencies.
This would lead to fuzzers using the shared libraries included on
system, which may not be sanitized, leading to incorrect behavior.
This refactors the relevant code from CC and calls it from the Rust
build logic.
Bug: 202282599
Test: output rust fuzzer zip file includes shared dependencies.
Change-Id: I92196eb0141733797a67eae24f8e9aedea94c3bc
Without this, any libraries with vendor: true set won't be included.
Bug: 197917632
Test: make haiku, run several fuzz targets on a device
Change-Id: Icab8d3f17d1f07f49940ee0ea52d9312a925affa
Rust and cc fuzz packaging have common. This change is to put this
common methods and fields into fuzz_common.
Test: make haiku and make haiku-rust ran locally and verify corpus,
dict and config are packaged along with the executable.
Change-Id: I04bf535cd2597ed8b2bd3d8e6ea8c28f38bdc18b
Dedups the many repetitions of the code to obtain a *Prebuilt from a
module.
Bug: 177892522
Test: m nothing
Change-Id: I1ededbe9ee79e89ea6dd8882dfee4be0bf0b51b7
Using $out.rsp as the rsp file adds extra complexity around keeping
the $ unescaped. Make callers to FlagWithRspFileInputList provide
an explicit path for the rsp file instead.
Bug: 182612695
Test: rule_builder_test.go
Change-Id: I3f531d80c1efa8a9d09aac0a63790c5b11a9f0c6
libicu_* fuzzer depends on installable cc modules and non-installable
cc modules with the same name, because the non-installable modules are
stubs used in the compile-time, and shouldn't be used in runtime.
Bug: 175583039
Test: make haiku dist
Test: Run 2 arbitrary binaries from out/dist/fuzz-target-x86_64.zip on device
Test: Run 2 arbitrary binaries from out/dist/fuzz-host-x86_64.zip on host
Change-Id: I0ecf24bb83e3985da3b8f393adab8c87ab7400c6
Adds a PlatformSanitizable interface which both CC and Rust can
implement so that the sanitizer mutators in CC can sanitize Rust
shared/static libraries appropriately.
Bug: 147140513
Test: m nothing
Change-Id: Ib31103b6c4902a4d5df2565c0d7c981298d100a3
Bug: 176998722
Test: built fuzz target with these options specified, checked resulting
config.json
Change-Id: Id38668dd11e76eead4ec87bd225ba67e3b30b80d
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.
Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
The android.WriteFile rule takes careful escaping to produce the
right contents. Wrap it in an android.WriteFileRule that handles
the escaping.
Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
The stubs methods are very specific to cc for now, move them out
of LinkableInterface so they are not shared with rust. Instead,
create a cc.Module.library field that contains the libraryInterface
to simplify calling libraryInterface methods on cc modules.
Test: all Soong tests
Test: no change to Soong outputs
Change-Id: I0289d866ce1f7a765631fe3101a62b1b4988ba1c
Add vendor_ramdisk_available and vendor_ramdisk attribute to
various rules. When a vendor_ramdisk variant of a module is
generated, it is installed to $OUT/vendor-ramdisk.
It is similar to a ramdisk image.
Test: m nothing -j
Change-Id: Ib2d16459f3094dbe21c3bdb7c016cb4b2bf62765
This will be used to track who should be recognized for any CVEs
resulting from bugs found by a fuzz target.
Bug: 145745999
Test: n/a
Change-Id: I21e065ab8013d013b1d9f42981c729b996163387
