Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.
Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
The android.WriteFile rule takes careful escaping to produce the
right contents. Wrap it in an android.WriteFileRule that handles
the escaping.
Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
The stubs methods are very specific to cc for now, move them out
of LinkableInterface so they are not shared with rust. Instead,
create a cc.Module.library field that contains the libraryInterface
to simplify calling libraryInterface methods on cc modules.
Test: all Soong tests
Test: no change to Soong outputs
Change-Id: I0289d866ce1f7a765631fe3101a62b1b4988ba1c
Add vendor_ramdisk_available and vendor_ramdisk attribute to
various rules. When a vendor_ramdisk variant of a module is
generated, it is installed to $OUT/vendor-ramdisk.
It is similar to a ramdisk image.
Test: m nothing -j
Change-Id: Ib2d16459f3094dbe21c3bdb7c016cb4b2bf62765
This will be used to track who should be recognized for any CVEs
resulting from bugs found by a fuzz target.
Bug: 145745999
Test: n/a
Change-Id: I21e065ab8013d013b1d9f42981c729b996163387
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.
Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
cc_fuzz relies on an invariant that's not exactly true. We assume that
for each fuzz target, we'll only have a dependency on a single sanitized
variant of a shared library. In a few instances, this is proven not to
be true, as we end up with a transitive dependency on a shared library
with sanitizer coverage instrumentation, and one without sancov.
This results in breaking the packaging for some fuzz targets. This then
goes on to break `make haiku` in some scenarios.
While this isn't a completely technically correct solution (as we
basically resolve one of the sanitized variants pseduorandomly), it does
resolve the issue for now. Realistically, we should select *both* of
them, and set the DT_RUNPATHS on the shared libraries to point to the
dependencies that have the sanitization that they're expecting. In
practice - this shouldn't break sancov (we might just silently drop some
coverage) or hwasan (we might just silently drop some hwasanification).
I believe that the walk order of VisitDirectDeps is deterministic, and
as such this shouldn't affect the reproducability of fuzz target builds
(and thus won't blow up the Soong rebuilds). ccross@ or dwillemsen@ can
speak better to this than I can though.
Bug: 148306195
Bug: 151102177
Bug: 155123587
Test: lunch flame_hwasan-userdebug && make haiku
Change-Id: I8d4001d93da33e4e5d21f740beb88a20fcc26e2a
`make haiku` shouldn't build fuzz targets that are disabled on the
continuous fuzzing infra. This should reduce pressure on the repackager.
Bug: N/A
Test: make haiku && verify that example_fuzzer isn't there.
Change-Id: I6a687a3ab8cbb1eb75113e96aba2c4f9f8058010
cc_fuzz reexports transient dependeny DSO's to /data/fuzz through Make.
We intentionally export the non-stripped variants so that ClusterFuzz
can use the symbolized variant, but we don't re-export to
/symbols/data/fuzz. This means that tools like `stack` and
`hwasan_symbolize` can't pick up the symbols and don't know what to do.
Fix this by re-exporting to /symbols/ as well.
Bug: N/A
Test: make example_fuzzer && ls
$ANDROID_PRODUCT_OUT/symbols/data/fuzz/arm64/lib
Change-Id: Id0343c95a0a83e16e6f67f29ff6361fb4d757c05
This change is to add granularity for disabling
of the fuzzer from running on host, device or both.
Bug: 145824604
Test: make haiku successfully completes after change
Change-Id: I686b439e36c622108f7b5b3e3f4738b8295320a7
"data" field specifies data dependencies that will be installed in
fuzzer's output directory.
"data" behaves similar to "corpus", except "data" preserves directory
structure, e.g.
data: ["foo/bar.txt"]
is installed into
$OUT/data/fuzz/<arch>/<target>/data/foo/bar.txt
Test: build a fuzzer with data depenency, check data is installed
correctly
Change-Id: Ia1255026278435181b6d93f91f8f9ad39c96d07f
cc_fuzz ignored soong namespaces, and built all cc_fuzz modules into the
architecture package. This caused naming conflicts where two modules had
the same name but were in different namespaces.
Now, cc_fuzz will only build modules that are in the same namespace as
the product definition - resolving the duplicate definitions error.
Bug: 145240569
Test: m haiku
Change-Id: I0a0acd139426301586ac5e3aba8ce5527cb7a1b5
Use lists instead of maps to make the order and contents of fuzz
packaging rules reproducible.
Test: m nothing; touch Android.bp; m nothing
Change-Id: Id11323f7528e594370c391bfe82eb7367bc1bcd5
RPATH needs to be set to traverse the local directory's lib folder for
fuzz target packages.
Bug: N/A
Test: m fuzz
Change-Id: Idea4f04ce4c55315df7e808de5b2b4f06a88d82a
Currently, prebuilt shared libraries fail the linkable.CcLibrary()
check, as the module returned by NewPrebuiltSharedLibrary uses a
prebuiltLibraryLinker as the module.linker, rather than a
libraryDecorator (which is the check by CcLibrary()).
We also need to ensure that we discard LLNDK stubs libraries, so we
manually categorise and discard them as well. They unfortunately are are
cc.Modules that aren't CcLibraries, as they use a custom linker object
as well (stubDecorator).
Fixes: 144415986
Test: m fuzz
Change-Id: I3b85ef66d1602cb8c035a0a90bddf30674e2eb71
Instead of writing all fuzz target files into a huge zipfile (by number
of entries), we instead write each fuzz target and its files to a
zipfile, then coalesce them into a big zipfile.
This should simplify the repackager significantly, as no unzipping and
repackaging should be needed (just renaming).
This also solves b/144418402, where we are very close to hitting the
POSIX args limit on the soong_zip command for the device fuzz package,
due to the line length generated by all the entries in the huge zipfile.
Fixes: 144418402
Test: make fuzz
Change-Id: I89797e1bea2691ff4a33a227306f2f9083d8af6e
Change the ClusterFuzz package built by 'make fuzz' to now package
shared libraries per-fuzz target.
Changes documented in go/proposed-clusterfuzz-package
This has the impact of:
- This package is much easier to post-process in the Haiku export
cronjob.
- Shared libraries are now per-fuzz-target. This means that we will
have some level of duplication (libdl, libc, libc++, libclang_rt.*,
etc.) as these libraries will be in multiple fuzz target's dirs.
This *only* affects the fuzz-$target-$arch.zip file, not the fuzz targets in
$ANDROID_HOST_OUT/fuzz or $ANDROID_PRODUCT_OUT/data/fuzz. These two
install directories still use the global shared libraries in order to
preserve space on devices.
Bug: 144360679
Test: m fuzz
Change-Id: I5f19a13070e01d2d39598f9f215a983b398a9f37
Some fuzz target dependencies are generated at Soong-time in a
nondeterministic fashion. When these are interpreted by Make, they may
cause rebuilds due to being in a different order.
Ensure that the dependencies are always sorted, which should avoid
triggering unnecessary rebuilds.
Bug: 144250431
Test: make <module> && make <module>, ensure no rebuild.
Change-Id: I44edc1a3bb3f57be3e3f093f765241b41928fa4d
Additional context (for Googlers): go/android-fuzzing-shared
This patch adds the Soong frontend for shared library fuzzing. We
traverse dependencies at soong install time to find all transient shared
libraries that $module depends on. We then ask the Make backend to
depend on the shared library.
We also create the source:destination mappings between where the shared
libraries are built to where they should be installed to for fuzzing.
This is then depended on by the Make backend.
Bug: N/A
Test: m fuzz, note the contents of $ANDROID_PRODUCT_OUT/data/fuzz/lib,
and out/soong/fuzz-target-*.zip now has shared libraries.
Change-Id: Id7afbd34bc9c055110af96cd3c668b730d404aee
This is causing some build errors because the config.json file isn't
associated with a rule (because the rule looks for .txt).
Test: ran locally, checked output directory
Bug: 142551000
Change-Id: Idcdb226fa2d23055a8850b2a0545202921edc3b1
Unstripped binaries allow for online symbolization, and offline
symbolization using the debug info in the binary. Debug information
isn't stripped for host binaries, but is for target. Target should also
contain debugging info.
Bug: N/A
Test: m fuzz && # Check that fuzz targets in the fuzz-*.zip package have
debug info.
Change-Id: Ibd3002674d519e927340a50dfdfbf44f2d809d58
Adds the ability for rust modules to be compiled as C libraries, and
allows cc modules to depend on these rust-generated modules. This also
means that soong-rust should not have any dependencies on soong-cc aside
from what's required for testing.
There's a couple small fixes included as well:
- A bug in libNameFromFilePath that caused issues when library's had
"lib" in their name.
- VariantName is removed from rust library MutatedProperties since this
was unused.
Bug: 140726209
Test: Soong tests pass.
Test: Example cc_binary can include a rust shared library as a dep.
Test: m crosvm.experimental
Change-Id: Ia7deed1345d2423001089014cc65ce7934123da4
Specifically, this adds:
* Owner
* Disable (stop fuzzer from running in Haiku)
* Bug Component
* Bug Hotlist
The fields are all inside a new 'options' struct.
The values from these fields (if any) are written into a config file as
json.
Bug: 142551000
Test: ran locally with a modified build file and verified output in .zip
Change-Id: I86edf74c2cebe9912ac0ad203f99028be4062c8b
Re-plumb the fuzz corpora through an intermediates directory.
Previously, due to the way that LOCAL_TEST_DATA is implemented, we would
break when corpora were included:
a) outside of a corpus/ directory, or
b) not directly under the module directory (i.e. my_module/subdir/corpus)
Corpora are now written into an intermediates directory so that they
play nice with LOCAL_TEST_DATA, because unfortunately the key-value pair
required there is not as simple as source:destination.
Test: m example_fuzzer && ls \
$ANDROID_PRODUCT_OUT/data/fuzz/arm64/example_fuzzer/corpus
Bug: N/A
Change-Id: I5888b06ed349781c6712f1ae77cc3582283a9552
If fuzzer coverage sanitized shared libraries are not deployed - we shouldn't
break users during dynamic link time. I need to think more about how we
should solve this problem in earnest, but for now let's just disable
static linkage of libc++ on device.
Bug: 142671952
Test: Cherrypick ag/9550833 and ag/9544155, make and run the fuzz target
there.
Change-Id: I39dfd6ba314e7a610ef3f0b30f35383a5e3a1e4b
Corpus files called 'FOO' incorrectly go into a subdirectory in the
exported zipfile, e.g.:
my_fuzzer/corpus/FOO/FOO
This patch removes that extra, unnecessary intermediate directory,
e.g.:
my_fuzzer/corpus/FOO
Bug: 142432347
Test: m fuzz && unzip -l out/soong/fuzz-host-x86_64.zip
Change-Id: Id23d4dd2d8322a8ad0234f8acb6878ea4dc3c6e4
Add the fuzz packages to dist. This requires that the phony for 'make'
be define in make itself, rather than in Soong. See comments on other
patch in topic for more information.
Bug: 141026328
Test: m fuzz dist
Change-Id: I4cd476adcfa42e4b40911c4a32427a7b00c67cc3
Adds the `make fuzz` build rule. This offers a few features on top of
`make haiku`:
1. Identifies all fuzz targets through the build system, rather than
having to maintain a golden file.
2. Packages now exist in out/soong/ and are packaged on a per-architecture
basis. Packages also now contain fuzzer seed corpus && dictionary.
3. This gives us further extension options:
- Packaging shared libraries for shared library fuzzing.
- Presubmit tests for fuzzing. Each fuzz target could be made into a
regression test by adding the crash testcase to the corpus. We can
also ensure that example_fuzzer finds a bug with an ASan report as a
presubmit smoke test.
Bug: 141026328
Test: m fuzz && unzip -l out/soong/fuzz-$ARCH.zip
Change-Id: I7aaad616d6b194a3beaf908241a9817df9dfdce1
Adds the corpus and dictionary properties to the cc_fuzz target.
Propagates these entries to the makefile backend via LOCAL_FUZZ_DATA, in
a similar manner to LOCAL_TEST_DATA.
Bug: 141026328
Test: m example_fuzzer, fuzz target should have adjacent corpus/dict
files.
Change-Id: If5add5a597cc479f4e084bdafbd0fc175cfd6321
Adds the host fuzzing support back for 32-bit x86 targets, now that the
toolchain has been updated to export this library.
Bug: 133261679
Test: `m example_fuzzer` on a 32-bit target/cross
Change-Id: I28285d531a53eebd28c9e1b733035be48696b48f
Statically links libcxx in order to not require shared library
deployment during packing of fuzz targets.
Test: m example_fuzzer && ldd $ANDROID_HOST_OUT/fuzz/example_fuzzer.
Note that the ldd results don't include libc++.so.
Bug: 139880425
Change-Id: I86d1945768de53217d54cb762392c20714494416
fuzzers require linking to libclang_rt which does not support
linux_bionic. Therefore always disable this target.
Test: ./art/tools/build_linux_bionic_tests.sh
Bug: 138307504
Bug: 118058804
Change-Id: I12816f302d32d5ee846f90c62814744ae35c49bb
The fuzzer runtime is not present for darwin host modules, disable
cc_fuzz modules when targeting darwin.
Test: m on darwin
Change-Id: I9da319e00e36c3e0a5912329de7890a6409f1a55
Adds the cc_fuzz target via a cc_fuzz module. Also implements the
libclang runtime interface for the x86 toolchain to allow host-built
fuzzers.
Bug: 133261679
Bug: 137398545
Test: Build a fuzzer (with all topic patches), notice that you now
have a host-built fuzzer :)
Change-Id: I7fa069603415f40b3f12a002c253fca6e2aa1988
