Move EarlyModuleContext, BaseModuleContext and ModuleContext out of
module.go and into early_module_context, base_module_context and
module_context.go respectively.
Test: builds
Change-Id: I52e6eb1589d1478233c1c55d770b395a16eaa1a3
This flattened singleton module explicitly lists by module name whether source or
prebuilt version of a module should be used. If a module appears in this
metadata module, it supersedes all other source vs prebuilts selection
mechanism
Implementation details
- Update the module dep chain from <source> --> <prebuilt> to <source>
--> <prebuilt> --> all_apex_contributions
- all_apex_contributions sets a provider which is bubbled up
to the source module. This requires changing `prebuilt_select` to a
bottom up mutator
- Update `usePrebuilt` to consult the new provider before falling back
to existing source vs selection mechanisms. If (foo|prebuilt_foo) is
listed in the selected `apex_contributions` modules, it
will be used superseding any other selection mechanisms.
- Update this depTag to IsMetaDepTag so that bootclasspath_fragment's
validation ignores this new dependency.
Test: Added unit tests to assert that this new mechanism supersedes
`use_source_config_var`
Bug: 308174768
Change-Id: I39a85639642711f3c96b6f18b94d626b55e80c66
These bazel-built modules will be installed into the system image
as part of the bazel rule, rather than going through the make staging
directory.
Bug: 297269187
Test: m bazel_sandwich
Change-Id: I96c6e58f8e0898b2ad92cb7069745ca2059a39f8
The data attributes for cc library rules is dropped by their starlark
macros, so serves no real purpose. Furthermore, this unused dependency
proves problematic for allowlist v2, as there are many cases at HEAD
where the corner-case "requires" dependency would otherwise mark a module
as unconvertible.
Fixes: 303307456
Test: New unit test
Test: Manual verification to unblock allowlist v2 `bp2build.sh` runs in
AOSP
Change-Id: I6ca6104b958d2b428fc2ca5b3fa794106571acca
This should be no-op, as the underlying mutator has not changed yet.
Some other refactoring is required and done in this CL:
- Delete some old, dead ApiBp2build code
- Fix casting to TopDownMutator when it's not necessary
This change is required to prepare for allowlist v2 work, as only
BottomUp mutators can AddDependency.
Bug: 285631638
Test: m nothing
Test: presubmits
Change-Id: I5212a5f5634cc13056195783e6df37ff8eb000da
In other words, if, in bp2build, module "foo" would generate "foo",
and "foo_two", and "foo_two" already exists in a build file,
bp2build should label "foo" as being unconvertible.
Fixes: 301321658
Fixes: 301312582
Bug: 285631638
Test: Unit tests
Test: Verified that `m bp2build` results in bit-for-bit identical
contents for out/soong/bp2build before and after this change.
Change-Id: Icbbdd69fce83579ec9b172d04b2bf1f294698f70
Instead, we return an error. This allows us to access some product
variable information earlier when it will not be used as an attribute
without panicing
Test: m nothing
Change-Id: Id094b2b9e1364a8d174d99b3824fa149fb235b3e
android_test is just an android_app, but the test bits are handled
purely in Starlark (tradefed_test_impl).
This enables building HelloWorldTests (android_test) with Bazel.
Bug: 297030120
Bug: 297029612
Test: WIP
Change-Id: I6d287c94b31698ea247da48e4cd6b5a75bc71e11
For explicit values of compile_multilib (e.g. 32 or 64), add an
incompatibilty constraint for the arches that do not have the correct
bitness.
Test: go test ./bp2build
Bug: 299135307
Change-Id: I4cd997de6804a5c6d89086d740f773936db1cab9
A single proto providing directory can be used by multiple soong modules. Some of these
can be
1. Host specific
2. Device specific
3. Both host and device
Since the generated proto_library can have 1:many mapping, it should
have an empty target_compatible_with. Compatiblity will be enforced at
the top-level {cc|java|python}_proto_library.
(This is a followup to aosp/2727054 which did not handle this correctly)
Test: Added a unit test
Change-Id: I09b3def70e3d043fd8ba0d1eb4ffff1910f097d1
In this Android.bp file
```
my_cc_defaults {
enabled: false,
soong_config_variables: {
my_bool_variable: {
conditions_default: {enabled: false},
}
}
}
```
The inner enabled: false is a no-op because the top-level enabled is
false. Currently, bp2build will raise an exception for this Android.bp
file.
However, it does not need to. `productVariableConfigEnableLabels` runs
only if the top-level enabled is false. If it sees enabled: false via
conditions_default, it should just ignore it since it is a no-op.
Test: go test ./bp2build
Bug: 210546943
Change-Id: I816f209eaf21de65ddfbc2893e5255be94bcaa11
If enabled does not appear inside `soong_config_vars`, we can ignore it.
Bug: 210546943
Test: go test ./bp2build
Change-Id: I9e4d51c3b683f262921449634f827915ce87dc8d
* changes:
Handle .proto files in different package for filegroups
Handle .proto files that end up in a different package
Allow creation of BazelTargets in a different directory
The current API restricts creation of targets to the directory of the
visited soong module. This CL proposes adding a `Dir` property in
`CommonAttributes` that can be used to create a bazel target in
a specific dir. The use case for this is to dynamically create
additional targets for proto_library that are adjacent to .proto files
(Bazel poses a strict requirement about proto_library being in the
same package as the .proto file, but Soong does not)
Usage is restricted to dirs that have an existing Android.bp file. There
are some places in bp2build where we use existence of Android.bp/BUILD
on filesystem to curate a compatible fully qualified path (e.g. headers).
If we use `CommonAttributes.Dir` to arbritraily create BUILD
files, then it might render those curated labels incompatible.
Test: go test ./bp2build
Change-Id: If9446700457eddfb389be9d9bde39087f67daa60
...instead of based on constraint settings.
Bug: 269577299
Test: m nothing and ./build/bazel/ci/bp2build.sh
Change-Id: Ib9caec79c92b8fd304e46be841de5612bd1637e3
This is prep work to add additional special handling for genrule
generated headers as there will be similar partitioning for those
headers.
Test: go test soong tests
Change-Id: Ib63e7e4f7554b2b7b7bc78b2825b20c05403216a
Use Go's generics for DepSets so they don't require a type-specific
wrapper and reflection.
Test: depsets_test.go
Change-Id: I22ba0b7d680d37d2cd05230b0f560d166c4dd20b
This also changes the expectation of ConvertWithBp2build. Each
implementation must either create one or more Bazel target modules, or
mark the module as unconvertible (with a specific reason).
Manually verified no runtime hit in AOSP
In AOSP, the metrics file size increases from 252K to 1.6M
This changes some effective module counts in bp2build metrics:
- Removes "package" modules from the module count list in
metrics, as these will not be converted like regular modules.
- Counts Handcrafted modules as being "unconverted", as bp2build is not
responsible for them.
Bug: 285631638
Test: Verified generated BUILD.bazel files are bit-for-bit identical
with this change
Test: Manually verified one case of each implemented reasonType
Change-Id: I308dd451d8f28379b15671dae9f931bd0446f5c1
In Soong, java libraries can use src files generated from xsd_config. In
Bazel, instead of providing srcs, java_xsd_config_library will provide a
jar. Update bp2build so that all src references to xsd_config modules in
srcs get routed to deps and exports attributes
This CL creates an interface in build/soong/android to get around a
circular dependency issue. The bp2build logic needs to exist in
soong-java, but soong-java does have soong-xsdc in its package path. Use
the interface and type assertions to special case xsd_config references
Bug: 211678537
Test: bp2build test in sibling CL in system/tools/xsdc
Change-Id: Ida924bb20b1fd7eb8beeef950b070d37a9c6f3b5
This is to differentiate soong intermediate directories for soong config
modules. This will help incremental build across different
devices.
Test result of building panther, building cheetah, and building panther
again:
Before this change
- build time: 02:57
- # of tasks: 31044
After this change
- build time: 01:48
- # of tasks: 1694
Build time includes build.ninja generating time (which is more than 1
minute), so the overriden artifacts become far fewer.
And "NINJA_ARGS='-n -d explain' m" only gave 4 "command line changed"
nodes.
Bug: 279362051
Test: see above
Change-Id: I4891cbe823ae21628465e5c6eb26a4837ccdd202
Bazel will fail on queries with missing dependencies. Instead, we check
for missing dependencies in mixed builds and we fall back to Soong when
we are aware of missing dependencies in a module.
Test: go test soong tests
Change-Id: I4f83752704970d8b43650d5b55ff35799c7bc625
This effectively undoes both If6c3ee82d588e2742c85cef7244c090c93f38b8e
and I682e4f1f477f3024f7719dfaa67006ef335e0640. SDK variants are now
installed again, which will fix unbundled builds of cc_test modules.
The platform variants used by com.android.virt are now packagable
even though they are not installable.
Fix the original problem in b/194403710 by adding a flag to platform
variants of modules in apexes that are not platform available, and
using that to prevent install and packaging dependencies. That
allows the HideFromMake flag to go back to being used for preventing
install dependencies but not packaging dependencies.
Test: TestPackagingWithSkipInstallDeps
Test: TestFileSystemShouldInstallCoreVariantIfTargetBuildAppsIsSet
Test: TestFileSystemShouldSkipApexLibraries
Bug: 194403710
Bug: 268582372
Fixes: 274443025
Change-Id: If5418df3ddbb940bd631caebdf38daa81e71f40e
Currently, there is little verification around allowlisted modules
actually being mixed-built. This flag would allow us to verify
that a module allowlisted is mixed-built for at least one variant.
Bug: 278910100
Test: m nothing --bazel-mode-staging --ensure-allowlist-integrity
Test: m nothing --bazel-mode-staging --ensure-allowlist-integrity
--bazel-force-enabled-modules=com.google.android.neuralnetworks (This
fails, as expected)
Test: build/soong/test/mixed_mode_test.sh
Change-Id: Icd5976f4f44f1a8caca1e5247d986642f7995f97
VNDK snapshot must include the information of the license texts and
kinds. Include them when packaging the snapshot prebuilt files.
This will be used to generate the VNDK snapshot files.
Bug: 270625053
Bug: 277317599
Test: development/vndk/snapshot/build.sh --build-artifacts
Change-Id: I0a39ff598efbb80faaf63807cdf5ce685fa074cc
Currently, non-apex variants of modules that are in apexes are not
exported to make unless they're apex_available to the platform. This
means that you can't `m` those modules directly.
However, there is a workaround in the apex androidmk implementation that
emits make rules for the removed modules, but just redirects them to
build the apex itself. We want to remove that, but one of the problems
with doing so is that you can no longer `m` many modules afterwards.
To fix that, unhide the apex's dependencies from make. To ensure they're
not installed, call SkipInstall() on them, and update SkipInstall() to
be more strict by setting `LOCAL_UNINSTALLABLE_MODULE := true`.
Bug: 254205429
Test: Presubmits
Change-Id: Ib094feb2c437ad50d8319c58caa997759e7ce32f
override_apex's bp2build converter had a bug where it was looking at
the product variables for the override_apex module itself instead of
for the base module it is overriding.
Fixes: 271424349
Test: go test
Change-Id: If1e2653d3751fa908faf0ab97dfa2e943ebe98ec
Previously, the symlink optimization for APEXes assumed that the target
of the symlinks are in the system partition. The assumption however
doesn't hold always because the file that was added to the APEX might be
with system_ext_specific: true or vendor: true.
Bug: 265598720
Test: m nothing
Change-Id: Ieb9a6769320c0ec697a88c0cae977e7d65288362
This reverts commit 8ec823cba1.
Reason for revert: DroidMonitor: Potential culprit for Bug b/262965953 - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted.
Change-Id: I236cc36981d8b30527ca286632727f8ca267e969
Some tests need to add custom tradefed options, but still want to
keep most of the soong autogenerated tradefed xml file.
Expose a test_options: { tradefed_options: [...] } property that
will allow tests to add more options to the autogenerated xml file.
Fixes: 184895128
Test: go test, and verified that the ninja files did not change for aosp_arm64
Change-Id: I75f7eb002c8325ce7cdc76e12e76e16195320620
Current InstallInVendor only checks vendor property, but there are more
properties to check if the module will be installed in the vendor. This
change adds two more cases (soc_specific and proprietary) to the
function.
Bug: 233294155
Test: N/A
Change-Id: Ic3cb4c52f5097cdcc6231c0e85a62ce3ad6094b3
1) Bp2build convert python_binary_host main attribute as LabelAttribute. Currently "main" attribute in python_binary_host is handled as string but for some modules (e.g certify_bootimg) the "main" attribute points to a file in its subpackage like "subpackage/file.py" and should be converted to "//.../subpackage:file.py".
2) Filter out duplicated labels in the merged list of "required" attributes of python_binary_host and its defaults.
Test: b build //system/tools/mkbootimg:certify_bootimg
Test: b build //build/make/tools/releasetools:check_target_files_signatures
Bug: 253081249
Bug: 253101186
Change-Id: Ic2cb4cadec2c1348da70af9f0730da9914d3a8ca
The module types in scope of this conversion are
1. cc_library and cc_library_shared (non-null llndk or stubs prop)
2. cc_library_headers (all)
For (2), we need some postprocessing on the results of the parser
bp2BuildParseBaseProps. This is necessary because arch and os specific
API exports need to be flattened and added to the generateed API headers
target along NoConfigAxis
e.g.
```
The api equivalent of
cc_library_headers (
name = "lifoo",
deps = select({
"//build/bazel/platforms/arch:arm": ["arm_deps"],
"//build/bazel/platforms/arch:arm64": ["arm64_deps"],
}),
)
should be
cc_api_library_headers (
name = "lifoo",
deps = ["arm_deps", "arm64_deps"],
)
```
For (1), we also need to generate 1:many header api targets so that
arch-specific deps can propagate arch metadata to the top-level
api_domain rule
Test: go test ./bp2build
Test: go test ./cc
Change-Id: Ie40cba1ac8e89f290b3d926c190d5e93abd52859
Report missing output files as missing dependencies when
AllowMissingDependencies is enabled. This will fix
AllowMissingDependencies builds where a dependency module is
present but missing support for a specific architecture.
Bug: 250918230
Test: lunch aosp_riscv64-userdebug && m ALLOW_MISSING_DEPENDENCIES=true nothing
Test: TestOutputFileForModule
Change-Id: I95e96e3e7cb3df7bf678ed628b45baf659addbad
Add it to the `CommonAttributes` struct and handle it in fillCommonBp2BuildModuleAttrs
Bug: 190817312
Test: treehugger
Change-Id: I7b21056e680384d4046372ee844512029721026e
Add a filter for cyclic references in required module names in bp2build.
Test: CI
Bug: 247985196
Bug: 225291837
Fixes: 234748998
Change-Id: Id1cfd0b2e5f217cdb01218ec3bbe11d5070cc9df
zeek@ has been tagging unit tests that cannot run on RBE with
"no-remote". While we don't yet have RBE support with standalone `b
test` yet, we can make sure to propagate the tags now.
Bug: 236973602
Test: new unit test
Change-Id: I6d0648f184626bb01c657068804f99ffb82907a3
