This CL makes the following changes when building an eng variant to
speed up compilation, possibly at the expense of the performance of the
generated binary:
* Disables LTO
* Enables incremental compilation
* Sets codegen units to 256 to increase parallelism (default set when
enabling incremental compilation)
These changes reduce the time taken to compile all of Android's Rust
code from 16m20s to 12m10s.
Test: lunch aosp_oriole-eng && m clean && m rust
Bug: https://b.corp.google.com/issues/289094772
Change-Id: I97f1675c67ded69120b0c0e0fb5608aa9314a866
Upstream currently uses pthread keys to emulate TLS on Android because
it historically did not have it, and is going to move to LLVM emulated
TLS, which is needed to target L. Since we know the platform is Q+, we
can just use the new features.
Test: Treehugger
Change-Id: I7c7747d89cba736377cfeb4080573a89b801e8fe
To generate module-info.json in Soong for b/309006256 Soong needs to
know the test data paths. Moving test data installation into Soong will
also help later for test suite packaging.
Add ModuleContext.InstallTestData that installs the files listed in a
[]DataPath alongside the test. The files will also be passed to Make
to allow it to continue packaging them into the test suites for now.
Update the module types that are producing LOCAL_TEST_DATA entries
in their Android.mk files to go through InstallTestData instead.
Bug: 311428265
Test: atest --host toybox-gtests --test-timeout=120000
Change-Id: Ia8b964f86e584ea464667fd86a48d754d118bead
cc modules can use __ANDROID_VENDOR_API__ to read BOARD_API_LEVEL
that is the API level of the vendor surface.
Bug: 313822931
Test: check ninja commands to have -D__ANDROID_VENDOR_API__=<version>
Change-Id: Iceb5918cbfce0f24929d02d7e3caf1e9942b88e8
Disabled modules may not have a valid root path.
Bug: 313410568
Test: SOONG_GEN_RUST_PROJECT=1 m nothing
Change-Id: I253cca18493193245aa198adfce5fd75cf27cefb
Previously, we manually re-computed crate roots inside project_json for
rendering rust-project.json. In addition to added complexity, this meant
that generated sources and glob sources would not render correctly - it
would select e.g. `src/**.rs` or `:foo` as a crate root.
Use a centralized computation of crate roots instead.
Bug: 309943184
Test: SOONG_GEN_RUST_PROJECT=1 m nothing, compare rust-project.json
Change-Id: I0caddbf600d025a0041f45e69812cdd6f1761234
This makes it possible to call crateRootPath in situations where a
ModuleContext is unavailable.
Test: m nothing
Bug: 309943184
Change-Id: Iee20b0606954a18ca516cdac40917d0016f94a05
This was frequently misused (for example, in the prebuilts module, it
was used as a complex "assert(len(srcs))==1"), and can be superceded by
getCrateRoot anywhere it was used. It's now only called from
compiler.go, and can drop the second return parameter, as it was only
actually used by the prebuilt assert misuse.
Bug: 309943184
Test: m nothing
Change-Id: I6c92580bc8f0ecb7586c544056b5409e6dd280e7
Provider added for the following test modules in this change: art_cc_test, cc_benchmark, cc_fuzz, cc_test, cc_test_host, rust_test,and rust_test_host.
Bug: 296873595
Test: Manual test
Change-Id: I815680529bcbecacb3a2bdb8f3746053afdee48c
Fix various issues that prevent building or running tests with the
Go tools:
Remove testing/test_spec_proto/go.mod. There's already a go.mod at
the top level, and adding testing/test_spec_proto/go.mod breaks
compiling with the Go tools in IntelliJ or from the command line.
Fix go vet issues when trying to print []DataPath.
Fix aconfig_conversion_test.go and add it to Android.bp so it runs
in presubmit.
Test: go test ./...
Change-Id: I18af2f05631446fe507744a2951e4b02c7ab59be
stdlibs can be set per-target, it makes to sense to allow no_stdlibs to
be set per-target as well.
Bug: 310924208
Test: Android.bp with per-target no_stdlibs builds.
Change-Id: I1c3a458a849367d2c363bd5f65af77b8cce8c0e0
aosp/2768553 reverted aosp/2518976, but there was some code that was
moved in between these two commits. This commit removes a code block
for some envvars that were moved in aosp/2638675, and adds back the code
for the ANDROID_RUST_DARWIN envvar.
Change-Id: I9ac09c608fe64340535ac493ab7f07ecb9b9c335
* changes:
Revert "Decompress debug sections when creating debug info"
Revert "Compress debug info in clang builds with zstd"
Revert "Use zstd to compress debug info in cc libraries"
Revert "Use zstd to compress debug info in rust libraries"
* changes:
Use zstd to compress debug info in rust libraries
Use zstd to compress debug info in cc libraries
Compress debug info in clang builds with zstd
Decompress debug sections when creating debug info
We can save a significant amount of disk space by compressing debug info
in rust builds. This adds a flag that configures compressing debug info
at the link stage for rust binaries.
Test: m
Bug: 305277519
Change-Id: Ia85d8dc0e7bea35d9f9abfeae27a4610b3113e78
This CL changes the handling of Rust flags to only pass one of either
incremental or codegen-units. This is necessary because incremental
implies a codegen-units value of 128 and it may be overridden by the
separate codegen-units argument.
Test: TH
Change-Id: Ia474e4f9fc071d5054cb479ec9de550fa886677e
This is breaking sanitized host builds. Disable the sanitizer in Rust
for host targets until this can be resolved.
Bug: 304507701
Test: SANITIZE_HOST=address m rust
Change-Id: I12b0c3e217717de7a89b32198f4c4729709e99b3
rust_bindgen modules which defined cpp_std with a .h file were not
correctly emitting the `-x c++` flag. This CL addresses that, and
ensures that if either cpp_std or c_std is set then the appropriate
behavior occurs no matter the file extension.
Bug: 304269101
Test: Soong tests
Change-Id: I71a8ae30ac0ed502d9d3fbf2f3039b0c56529d39
Fix mac builds by removing the -lrt and -lgcc_s flags from darwin
rust compiles.
Bug: 293349612
Test: builds
Change-Id: I99a9fea963761730efc4d3236135ee6d83dbca57
This reverts commit e7c71c344d.
The intermediates created by the separated rust compile steps are very
large, on the order of 60GB. This is more than CI can support for now,
revert the split into compile and link phases. This comes at the expense
of incremental build time, especially when modifying C/C++ sources that
are dependencies of rust modules.
Bug: 293349612
Test: builds
Change-Id: I81169e7d0727330c8de5e7688dcdc87fe7b8d3b5
Prepare for changing rust rules by testing that output file names will
match. A build failure in b/301463284 was partially caused by output paths changing and resulting in old files building up on buildbots which filled the disk. This unit test will ensure that the file paths of rust modules do not change and cause the same failure.
Bug: 301463284
Bug: 286077158
Change-Id: I1baba2c63ad29a2c98e378fce0a2ed32bc319c1e
rust_protobuf were unable to import protos from other rust_protobuf
modules. This CL adds support for that. rust_protobuf modules which are
listed in rustlibs will have their modules imported into the generated
protobuf stub. Additionally, rust_protobuf modules which define
"exported_include_dirs" will export those include paths to dependent
rust_protobuf modules.
Bug: 301266700
Test: m rust
Change-Id: I132edffa4d77e0ac80a7ac934f873374c8e94c1b
PRODUCT_PRODUCT_VNDK_VERSION is set to 'current' by default. Now, we
can generate product variants without checking the
PRODUCT_PRODUCT_VNDK_VERSION build variable. Remove reading the
PRODUCT_PRODUCT_VNDK_VERSION variable from soong and generate product
variants by default.
Bug: 302255959
Test: m
Change-Id: I9a9b2076f4367c5ce9a393bbb206f8dee3884bd8
We only supported bp2build for rust_protobuf_host because the non-host toolchain for rust is not checked in yet. aosp/2759750 changed a converted module from rust_protobuf_host to rust_protobuf and broke CI.
This CL adds bp2build support for rust_protobuf but makes the target incompatible with android os for now untilnon-host rust toolchain is checked in.
Bug: 301956497
Test: b build //build/make/tools/aconfig:all --config=android
Change-Id: I739896c79f32674000c2603e394f16860a6fc57d
Prebuilt modules for some clang tools were created in aosp/2746469, but they do not exist for darwin builds. Adding the same modules in prebuilts/clang/host/darwin-x86/Android.bp results in duplicate module names. Instead we can just get a path to these files directly in Soong.
Test: CI
Test: https://android-build.googleplex.com/builds/abtd/run/L90700000963131594
Change-Id: I620cfd7b1ab34bd0959feea562d599b963717198
This was hardcoded in previously, and aosp/2746467 changed this to use
the HostPrebuiltTag instead which caused errors. Reverting back to
original implementation.
Change-Id: I8ed1919aa2ea737f7836fab39f9ee0663fb455d2
This no-op refactoring facilitates some upcoming functional changes for
"bp2build allowlist v2". The work requires that the bp2build conversion
mutator be changed from a TopDown mutator to a BottomUp mutator.
Refactoring all bp2build-related methods so that they use Bp2buildMutatorContext
makes it easier to make this functional change without touching tens of
files and multiple projects.
Bug: 285631638
Test: m bp2build
Change-Id: I3d1ef3064146e959c6f0dc315350fc9764bf2bd2
This commit adds support for compiling rust rules inside the sbox
sandbox. To compile a rust module with sandboxing enabled, the entry
point to the crate must be specified via the `crate_root` property, and
all input sources and compile-time data must be specified via the `srcs`
and `compile_data` properties.
Bug: 286077158
Change-Id: I8c9dc5cf7578037a583b4be2e2f73cf20ffd4408
This CL enables the following features for riscv64 Rust builds in
Android: V, Zba, Zbb, and Zbs
Test: m rust
Bug: 300313717
Bug: 300312461
Change-Id: I8712b1850a60a05e99a3e89e602bb56d2ccf2563
rustc 1.71.0 doesn't support any sanitizers for linux_musl_x86, disable
sanitizing for now.
Bug: 257334542
Test: m USE_HOST_MUSL=true host-native
Change-Id: I70819d2175c59bca1ff385d4aee48518d6b702e7
The crate_root property will be used to specify the entry point for a
rustc compilation. This will allow the srcs property to be used to
collect all src inputs to rustc rather than just the entry point.
Bug: 286077158
Test: m libnum_traits
Change-Id: I1a167182305dcc11cc927d562ceed622153111d3
Sandboxing the rust rules requires having explicit inputs for the rust
toolchain. This module type makes the rustc prebuilt and its
dependencies explicit as a module that can be depended on by all rust
rules.
Bug: 286077158
Test: rust sandboxing topic + go test
Change-Id: If2b80b32e329e6a6ee11ba824de868cf04714553
Since this is a one-off just for building libprotobuf that will be removed when we handle cargo output more generically (b/297364081), I didn't write a unit test for this CL.
Test: b build //external/rust/crates/protobuf:libprotobuf
Bug: 295925256
Change-Id: I00cf44d54be27a09c184a96c13b250a2e54e2d10
This adds support for correctly handling Rust sanitized snapshots,
ensuring they only have one variation.
The presence of multiple variations were causing build failures
when a rust_fuzz module for host was defined and a snapshot
build was requested.
This also sets -Z link-native-libraries=no on host modules
(in addition to device modules) to avoid emitting extra linkage
flags due to link attributes.
Bug: 282897366
Test: SOONG_SDK_SNAPSHOT_USE_SRCJAR=true m
Change-Id: Idf980c29145f11c530ad635a4eb5b01a1730ac24
The config constant is consumed by rust toolchain implemented in aosp/2709668.
Bug: 295918553
Test: go test
Change-Id: I0a28a802ad28f6ee24dfb1b5bdab72094534ae04
Rust modules are different from C++ modules in that they will install
their transitive shared libs alongside a binary. C++ modules don't
do this, you have to install the transitive shared lib manually. (but
they do install direct shared libs)
It does this by using InstallDepNeeded, a mechanism that adds
dependencies from installed files to the installed copies of
dependencies if InstallDepNeeded() returns true. This mechanism does
not end up tracking the installed files all the way to
FULL_SYSTEMIMAGE_DEPS.
We're attempting to make FULL_SYSTEMIMAGE_DEPS more accurate so that
we can track the files that should be installed properly, and remove
the need for `m installclean`.
Listing the libraries a binary uses in LOCAL_SHARED_LIBRARIES does
properly track them and end up listing them in FULL_SYSTEMIMAGE_DEPS.
Bug: 205632228
Test: Building the systemimage with a change to delete anything not in FULL_SYSTEMIMAGE_DEPS
Change-Id: I4ba75b40b3ac77250297209a851bc9ba377782f5
- Hide universal binary dependency for rust
Without a darwin_arm64 libstd prebuilt, many rust binaries are not
possible to build as a universal binary anyway. This fixes an
intermediate error with compiling rust FFI targets, where C++ shared
library dependencies for both architectures were combined into the same
linker command.
There is still an outstanding problem with C++ executable targets that
include rust FFI library targets, as the rust FFI library targets
fail to find a proper libstd to build the arm64 side of the universal
executable.
- Use dynamic_lib instead of soname
This matches similar logic in cc/library.go : linkerFlags for including
shared libraries.
Bug: 291830024
Test: m libkmr_cf_ffi (when enabled for darwin)
Change-Id: I1d6b6a7855c6649b93ab6592ea43b17c6e37b3a3
For device build, rust defaults to dylib linkage for rustlibs deps. `dylibs` prop was provided for flexibility.
By removing it, we're enforcing users to either use the default linkage (dylibs for device and rlibs for host) or rlibs prop explicitly. This means no dylibs for host modules. This makes sense because host modules always uses rlib linkage against libstd. The flexibility with dylibs prop opened room for linkage collisions because the dependencies don't link against libstd the same way.
Test: go test
Change-Id: I2fc221daa8a9bb42bdcf6d9823c723a4ddabe7b5
Adds support for host-based Rust fuzzers.
Bug: 282897366
Test: SANITZE_HOST="address" m <host_fuzzer>
Test: run fuzzer
Change-Id: Ibb951f651ef12e763778ebbf12e66769a7113920
- Don't pass `--as-needed` to the linker on Mac OS which is unsupported
there
- Use `--force_load` rather than `--Wl,--whole-archive` on Mac OS
- Scan `rustc`'s linker arguments for `-dylib` and `-dynamiclib`, which
it can use instead of `-shared` on Mac OS:
7bd81ee190/compiler/rustc_codegen_ssa/src/back/linker.rs (L319)
Test: m libhalf serde_derive
Bug: 291164566
Change-Id: Iecd6c2532fa31c9476834f49b109de98cbd2dccf
Register a missing dependency when a dependency returns an invalid
OptionalPath.
Fixes: 294070048
Test: rust_test.go
Change-Id: Ibccfc30d59c706308e99f11e372f7c9b60d9dc74
Add support for CARGO_PKG_VERSION_{MAJOR,MINOR,PATCH}.
Bug: 293602526
Test: m
Test: check build flags for cargo_env_compat module
Change-Id: I6b2796656e3a489cd077c2f42b0bbd04de610ba4
SCS has been supported in Rust since 1.64.0.
This adds SCS as a sanitizer option in Rust.
Bug: 168914033
Test: Built module with sanitize: { scs: true }
Change-Id: Ie54ac4693286878b98704cf052649a267777d764
'-Z sanitizer={hw}address' was not being passed if the fuzzer sanitizer
property was also set. Additionally, trying to use the fuzzer sanitizer
with the address sanitizer incorrectly linked in the hwasan runtime.
Bug: 293466009
Test: SANITIZE_TARGET="fuzzer address" m android_logger_fuzzer
Test: SANITIZE_TARGET="fuzzer hwaddress" m android_logger_fuzzer
Test: ldd <fuzzer_on_device> # ensure correct libraries linked
Test: Check build flags
Change-Id: I6b01c8808af07c642217b642af128ebf934f4bc6
Add a new Bindgen_flag_files property to bindgen modules
that let users specify files that contain extra flags for
bindgen.
Bug: 242243245
Test: presubmit
Change-Id: I21d987c08ac417c04aaa3bfb3b75d563fc662d5b
SCS has been supported in Rust since 1.64.0.
This adds SCS as a sanitizer option in Rust.
Bug: 168914033
Test: Built module with sanitize: { scs: true }
Change-Id: Ibdc116a335339af6d0b0d6dd0bb82c0358fa9087
There is no longer a reason to restrict dylibs on product, make them
available.
Bug: 204303985
Test: m
Change-Id: I315d22307f64d02633940f30ae600c63f16f0e7a
Adds dylib support for vendor and recovery images.
This changes the default linkage for vendor and recovery images to
dylib, which matches the platform default linkage. This also means that
by default, dylib-std variants are used for rlib dependencies.
Bug: 204303985
Test: Soong tests.
Test: m dist vendor-snapshot
Test: RECOVERY_SNAPSHOT_VERSION=current m dist recovery-snapshot
Change-Id: If84074b8615a70c45e7e162abeb853dc8c34d49a
Switch custom_bindgen to use any HostToolProvider
instead of just Rust modules, so any host binary can
be used as the bindgen wrapper.
Bug: 242243245
Test: presubmit
Change-Id: I9ba189b2857b3049dcf14842c178ce5846bab2f7
Use Go's generics for DepSets so they don't require a type-specific
wrapper and reflection.
Test: depsets_test.go
Change-Id: I22ba0b7d680d37d2cd05230b0f560d166c4dd20b
This removes the dylib-std suffix from rlib variants which link against
libstd dynamically. This is by far the common case.
This fixes an issue where 'm <rust_library_rlib>' would fail since Make
would expect the suffix to be included (e.g. 'm libfoo.dylib-std').
Bug: 173312517
Bug: 268086496
Test: m <rust_library_rlib> # without a suffix
Change-Id: I1e5778c57468302a8d672d5eb540047d8bb79314
Technically cargo package names and crate names can be different, but
this is rare and usually only a matter of presentation. Adding basic
support in this fashion lets several crates build without adding
significant machinery.
Bug: 276464273
Test: Build a crate which uses this environment variable
Change-Id: I4f4c8c9e277789de7a63be2457f00090e8c3e973
Rust libfuzzer flags have changed upstream, so update our set of flags.
This fixes the following fuzzer error:
ERROR: The size of coverage PC tables does not match the
number of instrumented PCs. This might be a compiler bug,
please contact the libFuzzer developers.
Bug: 249551848
Test: SANITIZE_TARGET="hwaddress fuzzer" m <fuzz_target>
Test: Test running the fuzzer.
Change-Id: I4f7df1fbe636133c646dfb0bd12757c947995ffe
Rust shared library dependencies are not always bundled in cc_fuzz or
rust_fuzz modules, which can lead to difficult to debug runtime errors
when running these fuzzers. It can also be hard to determine which
dependencies need to be explicitly declared.
This CL makes sure that we bundle the appropriate transitive
dependencies for our fuzzers.
Bug: 249551848
Test: Soong tests
Test: m <fuzzer> # check data/fuzz/<arch>/lib dir contents
Change-Id: I957ca8898079b61e2ff20d750f8c92bf61ac394f
This CL removes the custom_inner_attributes and mixed_integer_ops
features from the allow-list. This also allows us to remove the `-A
stable-features` flag.
Bug: 267698452
Test: m rust
Change-Id: I4770c2e9477337f112732c354bf477a99d11a467
Don't automatically assume that HWASAN or ASAN is enabled if Fuzzer is
enabled.
Bug: 178365482
Test: m bluetooth_stack_with_facade
Test: SANITIZE_TARGET=fuzzer m bluetooth_stack_with_facade
Test: SANITIZE_TARGET=address m bluetooth_stack_with_facade
Test: SANITIZE_TARGET=hwaddress m bluetooth_stack_with_facade
Test: SANITIZE_TARGET="fuzzer address" m bluetooth_stack_with_facade
Test: SANITIZE_TARGET="fuzzer hwaddress" m bluetooth_stack_with_facade
Change-Id: Ief8c0f899837c5889a8035782616025f1b0d54e7
