Focusing on the properties needed to remove soong config modules from
packages/modules/Virtualization.
- prebuilt_etc's src and srcs
- filegroup's srcs and exclude_srcs
- rust's cfgs
Bug: 342006386
Test: m nothing --no-skip-soong-tests
Change-Id: I6971da744a17955f98104948e6f9614776955782
android.ModuleBase already calls aconfigUpdateAndroidBuildActions()
that is the same with CollectDependencyAconfigFiles(). Remove the
CollectDependencyAconfigFiles() to avoid duplication with
aconfigUpdateAndroidBuildActions().
To make the aconfig information available in
GenerateAndroidBuildActions() of all modules, call
aconfigUpdateAndroidBuildActions() before calling
GenerateAndroidBuildActions() of each module.
Also, we don't need SetAconfigFileMkEntries(), which is a duplicate
of aconfigUpdateAndroidMkData()
Bug: 335363964
Test: diff `adb shell printflags` before and after the change.
Change-Id: I52808e442e9fed7db1eae7b7c5ed0b1c5ba74f5d
This change propagates the intermediateCacheFiles generated by the
aconfig_declarations to the static rdeps that are java modules or the
rdeps that are filegroups.
Test: m nothing
Bug: 329284345
Change-Id: I02431336c1aa0378d03248f3bb6edf2f57ec3b7f
Convert all of the callers of SetProvider to use the type-safe
android.SetProvider API.
Bug: 316410648
Test: builds
Change-Id: If58f4b5355264ddab2045bc3591a4eac19cd58fc
Remove the ConvertWithBp2build implementations from all the module
types, along with the related code.
Bug: 315353489
Test: m blueprint_tests
Change-Id: I212672286686a318893bc7348ddd5a5ec51e77a7
This is groundwork for allowlist v2, as such targets are still
effectively referencable via their file syntax; thus, rdep targets of
these filegroups should still be treated as convertible.
Fixes: 302025822
Test: Unit test update
Change-Id: I93445ff75e7c0570530725d086e47c1532833c06
This should be no-op, as the underlying mutator has not changed yet.
Some other refactoring is required and done in this CL:
- Delete some old, dead ApiBp2build code
- Fix casting to TopDownMutator when it's not necessary
This change is required to prepare for allowlist v2 work, as only
BottomUp mutators can AddDependency.
Bug: 285631638
Test: m nothing
Test: presubmits
Change-Id: I5212a5f5634cc13056195783e6df37ff8eb000da
This no-op refactoring facilitates some upcoming functional changes for
"bp2build allowlist v2". The work requires that the bp2build conversion
mutator be changed from a TopDown mutator to a BottomUp mutator.
Refactoring all bp2build-related methods so that they use Bp2buildMutatorContext
makes it easier to make this functional change without touching tens of
files and multiple projects.
Bug: 285631638
Test: m bp2build
Change-Id: I3d1ef3064146e959c6f0dc315350fc9764bf2bd2
Followup to aosp/2693190, this CL adds the support to filegroups.
<mod>__bp2build_converted is now an alias to a proto_library
target <mod>_proto. This proto_library will be created in a different
package if the .proto file exists in a different package.
Test: bp2build unit tests
Test: TH
Bug: 292583584
Change-Id: I8ca452aacf1a86dfc9e218464e38aab89afa5a29
This also changes the expectation of ConvertWithBp2build. Each
implementation must either create one or more Bazel target modules, or
mark the module as unconvertible (with a specific reason).
Manually verified no runtime hit in AOSP
In AOSP, the metrics file size increases from 252K to 1.6M
This changes some effective module counts in bp2build metrics:
- Removes "package" modules from the module count list in
metrics, as these will not be converted like regular modules.
- Counts Handcrafted modules as being "unconverted", as bp2build is not
responsible for them.
Bug: 285631638
Test: Verified generated BUILD.bazel files are bit-for-bit identical
with this change
Test: Manually verified one case of each implemented reasonType
Change-Id: I308dd451d8f28379b15671dae9f931bd0446f5c1
This reverts commit 2562585878.
Reason for revert: We now have aidl_library to remove this workaround to support specifying aidl headers
Change-Id: Idf03ebbea764f8c5b3c4fc1623f8f02aa321c8c1
1) Use attribute name "default_package_metadata" instead of
"default_applicable_licenses" in packages to better describe its use.
2) Add a filegroup "default_metadata_file" in packages to search for
METADATA file in each package.
3) Include "default_metadata_file" in each package's
"default_package_metadata" attribute.
Bug: 275472038
Test: CIs
Change-Id: I645c013c39e3190fd96c4a549d39a331aced16bd
In a follow-up CL, we should eventually disallow aidl.include_dirs in cc rules so that the deps are always explicit and compatible with Bazel migration.
Test: go test
Bug: 278059962
Change-Id: Ia786cc8634d03589dc008f10b01e6bb2b9f2c7f0
api providing modules (droidstubs, cc_library etc.) can refer to the api
file via a separate filegroup module. Therefore these modules should be
generated in the api_bp2build workspace as well
Test: m api_bp2build && build/bazel/bin/bazel build
--config=api_bp2build --config=android //build/orchestrator/apis:*
Change-Id: I77371bd94a2794770b01b98aaf84b1bc42810841
Bug: 240424572
Test: Manual tests:
1. m --dev-mode-staging com.android.adbd com.android.media.swcodec.
2. verify the DCLA libs from the two apexes have the same size and
sha1sum, and also match the libs in bazel-out.
3. empty the DCLA libs list in allowlist.go and repeat step 1
4. repeat step 2 and verify the opposite result
5. build git_master: mainline_modules_bundles-userdebug in ABTD
with the cl, then follow go/build-sideload-dcla-locally to
download the adbd and swcodec aab files, run the DCLA trimming
workflow locally, and verify the symlinks in the two trimmed
apexes are identical and also match the lib path in the DCLA
apex that was created by the workflow.
Change-Id: Ib2f8a29126a54829c0e10eba17b256a79930fd70
The Tags attribute is now available in CommonAttributes, and if we use
that one, we won't get errors with conflicting definitions of the
attribute if a global Tags value is added via CommonAttributes.
Test: m bp2build
Change-Id: I422ed817b84e6808ef60fe3599fe84332bc51d8e
Rather than individually denylisting filegroups until we prioritize a
solution for mixed builds that will correctly integrate into uses such
as proto, aidl, gensrcs, etc.
Test: mixed_droid
Change-Id: Iddbd391af7dd7cabc892b2b26dbc68e3aa506471
This is just a partial fix.
Bug: 246997908
Test: Manual build //frameworks/proto_logging/stats:libstats_proto_host_proto
Change-Id: I9e4e62bce22cb68bcd7f917c57f3d1438fcec716
Test: go test ./bp2build
Test: b build //frameworks/base/services/tests/servicestests/aidl:servicestests-aidl
Test: enable //packages/modules/NetworkStack/common/networkstackclient
&& disable restriction on Android SDK in javaLibraryBp2Build
&& b build //packages/modules/NetworkStack/common/networkstackclient:ipmemorystore-aidl-interfaces-V10-java
Change-Id: Ifb817daf09a3983ea1c84948ed9f02a79f95784b
In mixed builds currently, filegroup doesn't use path prop when creating the paths to the srcs. It defaults to ModuleDir.
Hence, when java.genAidlIncludeFlags [1] calls srcFile.Rel() to eventually create the AIDL include dir for AIDL flags, srcFile.Rel() returns the filepath relative to the module directory instead. This CL appends path prop to module dir when creating relativeRoot.
This fixes the bridge between converted filegroup that set path prop (e.g. libbinder_aidl) to unconverted module (for example, droidstubs). The fix is needed for the child CL aosp/2186599 to convert libbinder_aidl to Bazel. Without this fix, module-lib-api-stubs-docs-non-updatable (unconverted module that depends on libbinder_aidl) can't be built in mixed builds.
[1]: https://cs.android.com/android/platform/superproject/+/master:build/soong/java/gen.go;l=123?q=java%2Fgen.go
Test: go test
Bug: 243010121
Change-Id: Ic2dd2ab9199c62010303a5b8c611d722f4a4118d
filegroup.srcs must have a .rel() version of their paths
that is relative to the module directory. Previously in
mixed builds, the relative path wasn't filled in.
Bug: 229251008
Bug: 204244290
Test: go tests, and USE_BAZEL_ANALYSIS=1 m apexer with the following cl
Change-Id: I11a2a8b96139e0e7b956fa4c256a9a99c02f88f3
This will allow us to do json analysis on a filegroup's inputs as they
will now have a corresponding phony action.
Test: m json-module-graph and check module-actions.json
Change-Id: Id1ab681c01689eef06205d4a7fe8cf8b9110c628
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:
1. Queueing calls to Bazel is done in a new mutator instead of a full
soong_build pass. Normal soong_build flow is interrupted (via a
functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
as implementing MixedBuildsBuildable. In this interface, define the
request that must be queued to Bazel, and then subsequently define
how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
improves mixed build performance.
Result:
A 33% runtime improvement on soong analysis phase with mixed builds.
Caveats:
C++ BazelHandler handling still remains a bit of a mess; I did what
I could within this CL's scope, but this may require additional cleanup.
Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.
Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
Allow java_libraries that depend on protobufs to be converted with
bp2build.
Bug: 215230097
Test: build/bazel/ci/bp2build.sh
Change-Id: I3ce52389e7e4e82755605ee277c1e527a6aebc6b
Each conversion required defining a separate mutator, which will each
operate on _all_ modules and requires each to repeat checks whether the
mutator should operator. Instead, we introduce a single mutator and
modules can define a ConvertWithBp2build to implement bp2build
conversion for that module.
Test: bp2build.sh
Bug: 183079158
Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
This also removes the special-case filegroup from mixed builds
buildroot; no special handling is required. Since we're currently
hardcoding linux_x86_64 as our host platform, it should be fine
to continue assumping that hardcoded host for now.
Test: USE_BAZEL_ANALYSIS=1 m adbd
Change-Id: I35509f4eb33ba7a243fab4c34b35958f6f2fceab
target.kind was a field that existed for only a few months, and have
been removed from Bazel. This is needed to update to a newer Bazel version.
Also, this reverts filegroups to be queried for a specific arch again,
but leaving file targets in the common arch (which is correct, anyway).
Fixes: b/199363072
Test: mixed_libc
Test: presubmits
Change-Id: I3b8e5c43a39516d977d327a44a75750b2f251be3
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule
Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.
As initial user, the Soong `required` property will be
translated to Bazel's `data`.
Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
This builds cap_names.list.h, which uses an eponymous filegroup
"generate_cap_names_list.awk" in Soong, but uses the file target
directly in Bazel.
This also improve filegroup support for mixed builds, by issuing a
cquery call _without_ arch. Filegroups in Soong don't have configurable
properties, so don't generate Bazel filegroups into buildroot's
config_nodes (which was x86_64 by default).
The mixed_build_root now looks like this:
```
config_node(...)
config_node(...)
config_node(...)
config_node(...)
...
filegroup(name = "common",
srcs = ["@//bionic/linker:linker_sources_x86",
"@//bionic/libc:kernel_input_headers",
"@//system/timezone/apex:com.android.tzdata-androidManifest",
"@//external/libcap:generate_cap_names_list.awk",
"@//bionic/linker:linker_sources_arm64",
"@//bionic/linker:linker_sources",
"@//bionic/libc:libc_sources_shared_arm",
"@//bionic/linker:linker_sources_x86_64",
"@//bionic/libc:all_kernel_uapi_headers",
"@//build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal-file_contexts",
"@//system/core/libcutils:android_filesystem_config_header",
"@//bionic/libc:libc_sources_static",
"@//bionic/linker:linker_sources_arm",
"@//bionic/libc/tools:bionic-gensyscalls",
"@//bionic/tools:bionic-generate-version-script",
"@//bionic/libc:libc_sources_shared"],
)
mixed_build_root(name = "buildroot",
deps = [":x86",
":arm64",
":arm",
":common",
":x86_64"],
)
```
Test: CI
Fixes: 198595323
Fixes: 198235838
Change-Id: I6df9a14da556cf358d96e6a99b514f66a2638295
with the same name.
Also add capability to test for errors raised in bp2build mutators /
contexts.
This CL does two things to filegroups:
1) If the filegroup has only 1 source file with the same name as itself,
don't generate a filegroup target. Instead, dependents will depend
directly on the Bazel file target instead.
2) If the filegroup has more than 1 source file and 1 of them has the
same name as itself, the bp2build mutator will error out. If bp2build
on CI passes, it means that the source tree / product we're testing
against does not have such a case (which seems to be true for most
source trees).
Either way, this will allow us to be unblocked for most of the errant
filegroups (case 1) in the tree.
Test: CI
Test: New test cases in filegroup_conversion_test.go
Fixes: 194762573
Change-Id: I830c53efc8808569afe3c5f9f08436855bcdafed
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3
We avoid both of these by storing the information directly in the
underlying module.
Also as a fringe benefit, removes some necessary boilerplate for
conversion.
For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714
Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
Because it's commonly useful and not cc/ specific
Also export GenerateBazelBuildActions and update uses
Test: Existing tests pass
Change-Id: Ibc6858bb1129afba181a7686dda432defe33b00d
This CL adds support for cc_library to correctly split c, as and cpp
srcs in shared/static nested props, as well as splitting
the *filegroup* deps in those props, where each filegroup is expanded
into its own c, cpp and as srcs filegroups. This ensures that the
correct sources go into cc_library_static's underlying cc_libraries for
c, cpp and as sources respectively.
See the bp2build conversion test for a better visualization.
Bug: 183064430
Test: TH
Change-Id: I29add5140672d042adff65527d8b65f4a5f0a05b
