Whenever any two Rust static libraries are included
as static libraries anywhere in a CC dependency tree, we sometimes
get duplicate symbol errors. To avoid this, we no longer
directly link multiple rust static libs to CC modules.
Instead, we build rust_ffi_rlib modules and produce the actual
static library that gets linked against the CC module based on
that CC module's full list of Rust rlib dependencies.
This introduces a new static_rlibs property for cc modules to
define the rust_ffi_rlib dependencies, which are then used to
generate the module above.
This CL is intended to deprecate rust_ffi_static. It leaves
rust_ffi_static and rust_ffi static variants in place until
the remaining rust_ffi_static declarations and uses can be
removed. In the meantime, rust_ffi_static produces
rust_ffi_rlib variants as well to make the transition easier.
Bug: 254469782
Test: m # with no changes
Test: m libapexsupport # with static_rlibs
Test: m libunwindstack # with static_rlibs
Test: m netsimd # with static_rlibs, no duplicate symbols
Test: m blueprint_tests # New Soong tests
Change-Id: I47e27ac967ef0cad46d398ebf59d8275929ae28a
VNDK is deprecated in 24Q2, so soong should be tested with no device and
platform vndk versions. This change removes all VNDK related tests and
VNDK versions from soong-etc, soong-rust and soong-sysprop tests.
Bug: 330100430
Test: m nothing --no-skip-soong-tests passed
Change-Id: Ie34d23f0facab31078de54682f7cc78d37fcd4be
This is equivalent to specifying a dependency name different to the
package name in cargo, which some external crates do.
Bug: 308790322
Test: Built libgrpcio with aliases for protobuf
Change-Id: I2801222051fdd962460cc7f4900cec357f63b974
This reverts commit e7c71c344d.
The intermediates created by the separated rust compile steps are very
large, on the order of 60GB. This is more than CI can support for now,
revert the split into compile and link phases. This comes at the expense
of incremental build time, especially when modifying C/C++ sources that
are dependencies of rust modules.
Bug: 293349612
Test: builds
Change-Id: I81169e7d0727330c8de5e7688dcdc87fe7b8d3b5
PRODUCT_PRODUCT_VNDK_VERSION is set to 'current' by default. Now, we
can generate product variants without checking the
PRODUCT_PRODUCT_VNDK_VERSION build variable. Remove reading the
PRODUCT_PRODUCT_VNDK_VERSION variable from soong and generate product
variants by default.
Bug: 302255959
Test: m
Change-Id: I9a9b2076f4367c5ce9a393bbb206f8dee3884bd8
This commit adds support for compiling rust rules inside the sbox
sandbox. To compile a rust module with sandboxing enabled, the entry
point to the crate must be specified via the `crate_root` property, and
all input sources and compile-time data must be specified via the `srcs`
and `compile_data` properties.
Bug: 286077158
Change-Id: I8c9dc5cf7578037a583b4be2e2f73cf20ffd4408
Rust modules are different from C++ modules in that they will install
their transitive shared libs alongside a binary. C++ modules don't
do this, you have to install the transitive shared lib manually. (but
they do install direct shared libs)
It does this by using InstallDepNeeded, a mechanism that adds
dependencies from installed files to the installed copies of
dependencies if InstallDepNeeded() returns true. This mechanism does
not end up tracking the installed files all the way to
FULL_SYSTEMIMAGE_DEPS.
We're attempting to make FULL_SYSTEMIMAGE_DEPS more accurate so that
we can track the files that should be installed properly, and remove
the need for `m installclean`.
Listing the libraries a binary uses in LOCAL_SHARED_LIBRARIES does
properly track them and end up listing them in FULL_SYSTEMIMAGE_DEPS.
Bug: 205632228
Test: Building the systemimage with a change to delete anything not in FULL_SYSTEMIMAGE_DEPS
Change-Id: I4ba75b40b3ac77250297209a851bc9ba377782f5
For device build, rust defaults to dylib linkage for rustlibs deps. `dylibs` prop was provided for flexibility.
By removing it, we're enforcing users to either use the default linkage (dylibs for device and rlibs for host) or rlibs prop explicitly. This means no dylibs for host modules. This makes sense because host modules always uses rlib linkage against libstd. The flexibility with dylibs prop opened room for linkage collisions because the dependencies don't link against libstd the same way.
Test: go test
Change-Id: I2fc221daa8a9bb42bdcf6d9823c723a4ddabe7b5
Adds dylib support for vendor and recovery images.
This changes the default linkage for vendor and recovery images to
dylib, which matches the platform default linkage. This also means that
by default, dylib-std variants are used for rlib dependencies.
Bug: 204303985
Test: Soong tests.
Test: m dist vendor-snapshot
Test: RECOVERY_SNAPSHOT_VERSION=current m dist recovery-snapshot
Change-Id: If84074b8615a70c45e7e162abeb853dc8c34d49a
This removes the dylib-std suffix from rlib variants which link against
libstd dynamically. This is by far the common case.
This fixes an issue where 'm <rust_library_rlib>' would fail since Make
would expect the suffix to be included (e.g. 'm libfoo.dylib-std').
Bug: 173312517
Bug: 268086496
Test: m <rust_library_rlib> # without a suffix
Change-Id: I1e5778c57468302a8d672d5eb540047d8bb79314
Teach rustc to generate object files instead of a linked executable
for binary and shared library crates. This lets us skip the lengthy
Rust compilation phase when only the C++ dependencies of these crates
have changed.
This works using a Python script that replaces the linker invoked by
rustc and converts the linker command line into a rspfile that can be
used during the link phase. The script makes certain assumptions about
how rustc invokes the linker, but since we control the version of Rust
we use, we can update the script if those assumptions ever break. I've
asked the Rust developers about adding an official way to do this so
hopefully we won't need to rely on this hack in the future.
The rustc rule for Kythe has been altered to pass linker=true, somewhat
consistent with the main rustc rule, which now doesn't invoke the actual
linker either. `m xref_rust` still succeeds, but hopefully we can find
someone who knows more about the rest of the Kythe pipeline who can let
us know whether it should still work. Otherwise, let's use this CL as
an opportunity to find out whether the rest of the pipeline is happy
with linker=true, since otherwise we will find it hard to maintain the
Kythe rule in the future.
Change-Id: Ifb9c8689c1bd78c0afcf6fdfd111140d20377536
The library file for a cc_shared_library dependency is added to the linkFlags of the rustc compilation action, but no explicit dependency was made on it from a Ninja perspective if a TOC was also present. This change adds the explicit dependency on the library file whether or not a TOC is present.
Test: m crosvm
Bug: 275416061
Change-Id: I625b62762d9ba7b4fd2b8362285528e47f728dd4
The prohibition on Rust dylibs outside system still stands, but rlibs,
rust_ffi_static, and binaries will all work fine.
Test: m nothing (new soong tests added by this commit)
Test: Created sample product_specific rust_binary, checked out/ location
Bug: 165791368
Change-Id: I6453274064bb24b2019f38e57fc0d09b7c0fcf30
Rust installed files reside in "$MODULE_OUT/stripped/" when they are
stripped, otherwise they reside in "$MODULE_OUT". However, other parts
of Soong assume that installed files are always in $MODULE_OUT
(cc_modules place *unstripped* files in $MODULE_OUT/unstripped).
This notably causes problems when adding Rust modules as test data in
AndroidMkDataPaths. When Rust modules are parsed by AndroidMkDataPaths,
if they are stripped then they incorrectly get installed as test data
with the path:
<install_root>/<relative_install_path>/stripped/file.
This CL refactors how we handle Rust stripped output such that the
installed file always resides in $MODULE_OUT.
Bug: 171710847
Test: Installed files now always reside in $MODULE_OUT
Change-Id: I53a6ff57a0a5a55cd95ea78ae592ce22abfa20c9
This CL adds support for the MTE memtag_heap sanitizer. This is
controlled via inclusion of an ELF note.
Bug: 170672854
Test: Heap MTE-enabled Rust test binary triggers MTE
Change-Id: I2619818785e86a94667d02b30d102c83456b7925
Adds support for snapshotting Rust rlibs. This allows us
vendor-specific code that uses rlib-only linkage until dylib
snapshot support is added.
Bug: 184042776
Test: m nothing # new Soong tests pass
Test: Example test Rust vendor module builds
Test: m dist vendor-snapshot # includes rlibs
Change-Id: I4976d3e1efec0ee778cc97730d45be471dffb678
Adds support for platform vendor_available Rust FFI libraries and
binaries to be included in the vendor snapshot.
Because rlib and dylibs are not yet in snapshots, libstd cannot be
included in a vendor snapshot. As a result, vendor-specific Rust code
can't be guaranteed to work with the platform-provided vendor_available
modules built with a newer toolchain. For now, a check is added
indicating vendor-specific Rust code is unsupported.
This changes the linkage for vendor variants of these modules to default
to rlib linkage since dylibs cannot be included in the snapshot yet.
Bug: 184042776
Test: m nothing # new Soong tests pass
Change-Id: I502eaa4bb962eb87ff868fcf49b435f0d2f982e6
Modify bloaty's MeasureSizeForPath to allow a module to provide multiple
paths. This is used to measure both unstripped and stripped
libraries/binaries. Add unit test to ensure correct measurements are
generated for Rust.
Test: m out/soong/binary_sizes.pb.gz
Change-Id: I59439b77dbf1cf5ad71e1c02996a6a90938536b4
This adds Rust vendor image support for all module types except
Rust prebuilts.
Bug: 184042776
Test: New Soong tests.
Test: Example cc_library vendor module can depend on rust_ffi_shared.
Test: Example rust_library vendor-only module compiles.
Change-Id: Iaa30ad51fdaedcbf14687da5472581f6af62ff59
All version names will go through ApiLevelFromUser which triggers an
error when the name is not a valid one.
Bug: 175678607
Test: m
Change-Id: Id33bf64085603914d45ad7942cb8908a4734493f
Revert the static_lib behavior to the previous behavior (pass static
libs to the linker rather than via to rustc using `-lstatic=<lib>`). To
bundle static libraries into libraries, provide the whole_static_libs
property which retains the current static_libs behavior.
Passing all static libraries via -lstatic was resulting in odd bloat
where in some cases static symbols were duplicated in binaries and
libraries. This split makes it possible to be explicit about when static
libraries should be bundled.
Bug: 183182230
Test: mma system/bt; mma system/security/keystore2; mma external/rust
Change-Id: Ic2dde5d1542dca5ce145aa3a3fbd9ea54440d991
Replaces the rust specific rustTestCtx mechanism with the general test
fixtures mechanism as converting it to use preparers was not possible.
Also, removes usages of the buildDir variable and removes it as it is
no longer needed.
Bug: 181070625
Test: m nothing
Change-Id: I0176a7b6fb2d390ae23693f1e198da5124b4be63
Pass an "android_vndk" cfg flag that indicates this code
targets the vndk. This can be useful in instances where code
might need to behave differently.
This also includes a fix to make sure our vendor Soong tests
are correctly configured.
Bug: 179907868
Test: Soong tests pass.
Test: Example module emits new cfg flags.
Change-Id: I01cdf91f6f9d42cd8a759266d5170479664bf4bc
Passing '-lstatic' for dylib static dependencies results in some
unexpected bloat as the static libraries are included as
whole-archives.
For now, limit this to rlibs while we investigate.
Bug: 175886967
Bug: 179122837
Test: cd system/bt/; mma
Test: Checked size of dylibs.
Change-Id: Id8328e95771e2ca8503bc8df35248b1c39cabc8e
Pass static libraries to rustc via the "-lstatic" flag so that rustc can
reason how to handle these libraries. This means that these static
libraries get bundled into rlibs, reducing the need to redeclare these
dependencies in dependent modules.
Additionally, exported depFlags were never used anywhere, so remove this
from flagExporter.
Bug: 175886967
Bug: 179122837
Test: m
Test: Checked that static symbols aren't duplicated in a resulting
binary when it appears in a dependency graph more than once.
Change-Id: Ifeb3c384e4fad8704206997242a0156ddcb87d2c
Adds the ability to define data files that should be installed alongside
the test.
This also fixes a bug wherein rust_test properties were duplicated.
Bug: 171710847
Test: rust_test module with "data" property installs files to device.
Change-Id: I091489afaf7e76b751a33a28049590d9fb39fe5f
To allow grpc protobufs to include non-grpc protos in a singular
library, we need to allow them to be defined as part of the same module.
This CL merges the two previously distinct module types into
rust_protobuf, and adds a new property for declaring which protos
contain grpc definitions.
Bug: 172952239
Test: rust_grpcio modules converted to rust_protobuf modules and build.
Test: Example rust_protobuf module with both proto types builds.
Change-Id: I0e627fd97bc6f74de89d309e3344694a1e76586d
We will have some APEXes having rust binaries and libraries. So, adding
the support for the types of modules.
rust.Module now inherits from android.ApexModuleBase and implements
the android.ApexModule interface.
Bug: 172414324
Test: m
Exempt-From-Owner-Approval: rebased after +2 from the owner
Change-Id: I356ef4c45f782a6460f001e83af96d1710642d80
This further emulates the rust-protobuf gen_mod_rs flag by providing
support for generating a single module containing multiple protobuf
definitions.
Bug: 171361369
Test: New Soong tests.
Test: Example module containing multiple protos works.
Change-Id: I815f9628a8289ae512758073dac49bc4535abf01
Relands Ic22603a5c0718b5a21686672a7471f952b4d1017 with a minor
change to track libc++ dependencies for python hosts and after
a fix to an internal genrule that depended on transitively
installed java libraries (ag/13068670).
Soong currently assumes that installed files should depend on
installed files of all transitive dependencies, which results
in extra installed file dependencies through genrules, static
libs, etc.
Annotate dependency tags for dependencies for which the
installed files are necessary such as shared libraries
and JNI libraries.
This avoids extra installed files, and is also a first step
towards genrules using their own copy of tools instead of
the installed copy.
Bug: 124313442
Test: m checkbuild
Test: java.TestBinary
Test: cc.TestInstallSharedLibs
Test: deptag_test.go
Change-Id: I725871249d561428e6f67bba6a7c65b580012b72
Soong currently assumes that installed files should depend on
installed files of all transitive dependencies, which results
in extra installed file dependencies through genrules, static
libs, etc.
Annotate dependency tags for dependencies for which the
installed files are necessary such as shared libraries
and JNI libraries.
This avoids extra installed files, and is also a first step
towards genrules using their own copy of tools instead of
the installed copy.
Bug: 124313442
Test: m checkbuild
Test: java.TestBinary
Test: cc.TestInstallSharedLibs
Test: deptag_test.go
Change-Id: Ic22603a5c0718b5a21686672a7471f952b4d1017
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.
Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
This reverts commit 323dc60712.
Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests
Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
Move to a builder pattern to increase flexibility when generating the
test configuration. The testRust, testRustCov and testRustError are kept
as main entry points for standard tests. Add documentation.
Test: m nothing
Change-Id: I891bec982ff2d65413f150d2395edf0fb0d68a43
Putting t.Parallel() in each test makes them run in parallel.
Additional t.Parallel() could be added to each subtest, although
that requires making a local copy of the loop variable for
table driven tests.
Test: m checkbuild
Change-Id: I5d9869ead441093f4d7c5757f2447385333a95a4
The current state of linkage is that device targets always link
libstd dynamically except for rust_ffi_static which requires a static
libstd linkage. However this prevents producing rust_ffi_static
modules which depend on other Rust libraries as those dependencies
will link libstd dynamically and cause a collision. We also want our
rust_test modules to statically link in libstd as well.
This adds a linkage mutator for rlibs that creates a variant for each
libstd linkage. Dependent modules can then select the variant that
matches their linkage of libstd.
Also fixes an issue where installation paths were being generated for
rlibs and static libs even though they weren't being installed. This broke
when adding the linkage mutator as Make would complain about multiple
targets producing the same output.
Bug: 168729404
Test: rust_ffi_static module with other rustlib dependency can be built.
Change-Id: I955b484bf5809e8fc5517750c7f8df82d3ca8895
This adds a new SourceProvider module type to handle protobuf code
generation. See the new test for an example of how to call this.
Bug: 143953733
Test: New soong tests pass.
Test: Replacing genrules in crosvm with rust_protobuf modules.
Change-Id: Ie3117129cde37b8736bc18ee09bf5cde27c01c34
In order to support cc dependencies which do not start with the 'lib'
prefix, we can't pass them through the -l flag. Instead, we can pass
them directly to linker flags.
Bug: 166151658
Test: cd external/rust/; mma
Test: cd external/crosvm/; mma
Test: Test linking to a cc dep that does not begin with 'lib'
Change-Id: I5acbf3d3405e66446f3eae600b35683c4eb3d8a5
This allows SourceProvider modules to create rust_library variants so
that generated source can be referenced as an external crate rather than
via an include macro. This is done by including rust_bindgen modules
like any other library, as a dependency in either rlibs, dylibs, or
rustlibs.
This renames the stem and flags properties for rust_bindgen modules to
source_stem and bindgen_flags, respectively. This deconflicts with the
usage in baseCompiler.
This also removes 'subName' from the Module struct and moves it over to
SourceProvider, which was the only user. This allows us to set it in
baseSourceProvider's AndroidMk; setting it in Module's AndroidMk was
causing problems finding NOTICE files for bindgen library variants.
Bug: 159064919
Test: New Soong tests pass.
Test: Local test rust_binary can use rust_bindgen module as a crate.
Change-Id: Ieb2cb614c2dd0b5aa7120541d77f6f822a6a1806
Modules defined in the srcs property are automatically added as
dependencies with AddDependency(), which will use any variant available.
This can cause incorrect architecture bindings to be silently pulled in,
such as when a host module uses a rust_bindgen module that doesn't
create a host variant.
This moves populating depPaths.SrcDeps over to depsToPaths and adds a
check for SourceProviders to make sure the correct OS and architecture
is being used.
Bug: 161826371
Test: Soong no longer silently pulls in bindings for the wrong
target.
Test: New Soong test to catch this case passes.
Change-Id: I2b3651cf6fc7dabf4081434df1c455e637f5b3a4
Allow rust modules to depend on and use generated source code provided
by SourceProvider modules and genrule modules without resorting to
hardcoded output paths.
All generated sources are now copied to a dependent module's
intermediates directory, then OUT_DIR is set to point to that path when
calling rustc. This matches the common convention used in most rust
crates to include generated source code from the path defined in the
OUT_DIR environment variable.
A couple other small notable changes are included in this CL:
* prebuiltLibraries can no longer include generated source files as they
should be prebuilt.
* srcPathFromModuleSrcs now excludes the main source file from the
second return value so its a list of only the generated sources.
Bug: 159064919
Test: Local example rust_library compiles with rust_bindgen dependency.
Test: Local example rust_library compiles with genrule dependency.
Test: Collision detected when multiple providers produce similar output.
Test: New Soong tests pass.
Change-Id: I59f54a25368c680b9086420c47ec24ab8cd1de6b
