The Tags attribute is now available in CommonAttributes, and if we use
that one, we won't get errors with conflicting definitions of the
attribute if a global Tags value is added via CommonAttributes.
Test: m bp2build
Change-Id: I422ed817b84e6808ef60fe3599fe84332bc51d8e
Rather than individually denylisting filegroups until we prioritize a
solution for mixed builds that will correctly integrate into uses such
as proto, aidl, gensrcs, etc.
Test: mixed_droid
Change-Id: Iddbd391af7dd7cabc892b2b26dbc68e3aa506471
This is just a partial fix.
Bug: 246997908
Test: Manual build //frameworks/proto_logging/stats:libstats_proto_host_proto
Change-Id: I9e4e62bce22cb68bcd7f917c57f3d1438fcec716
Test: go test ./bp2build
Test: b build //frameworks/base/services/tests/servicestests/aidl:servicestests-aidl
Test: enable //packages/modules/NetworkStack/common/networkstackclient
&& disable restriction on Android SDK in javaLibraryBp2Build
&& b build //packages/modules/NetworkStack/common/networkstackclient:ipmemorystore-aidl-interfaces-V10-java
Change-Id: Ifb817daf09a3983ea1c84948ed9f02a79f95784b
In mixed builds currently, filegroup doesn't use path prop when creating the paths to the srcs. It defaults to ModuleDir.
Hence, when java.genAidlIncludeFlags [1] calls srcFile.Rel() to eventually create the AIDL include dir for AIDL flags, srcFile.Rel() returns the filepath relative to the module directory instead. This CL appends path prop to module dir when creating relativeRoot.
This fixes the bridge between converted filegroup that set path prop (e.g. libbinder_aidl) to unconverted module (for example, droidstubs). The fix is needed for the child CL aosp/2186599 to convert libbinder_aidl to Bazel. Without this fix, module-lib-api-stubs-docs-non-updatable (unconverted module that depends on libbinder_aidl) can't be built in mixed builds.
[1]: https://cs.android.com/android/platform/superproject/+/master:build/soong/java/gen.go;l=123?q=java%2Fgen.go
Test: go test
Bug: 243010121
Change-Id: Ic2dd2ab9199c62010303a5b8c611d722f4a4118d
filegroup.srcs must have a .rel() version of their paths
that is relative to the module directory. Previously in
mixed builds, the relative path wasn't filled in.
Bug: 229251008
Bug: 204244290
Test: go tests, and USE_BAZEL_ANALYSIS=1 m apexer with the following cl
Change-Id: I11a2a8b96139e0e7b956fa4c256a9a99c02f88f3
This will allow us to do json analysis on a filegroup's inputs as they
will now have a corresponding phony action.
Test: m json-module-graph and check module-actions.json
Change-Id: Id1ab681c01689eef06205d4a7fe8cf8b9110c628
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:
1. Queueing calls to Bazel is done in a new mutator instead of a full
soong_build pass. Normal soong_build flow is interrupted (via a
functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
as implementing MixedBuildsBuildable. In this interface, define the
request that must be queued to Bazel, and then subsequently define
how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
improves mixed build performance.
Result:
A 33% runtime improvement on soong analysis phase with mixed builds.
Caveats:
C++ BazelHandler handling still remains a bit of a mess; I did what
I could within this CL's scope, but this may require additional cleanup.
Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.
Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
Allow java_libraries that depend on protobufs to be converted with
bp2build.
Bug: 215230097
Test: build/bazel/ci/bp2build.sh
Change-Id: I3ce52389e7e4e82755605ee277c1e527a6aebc6b
Each conversion required defining a separate mutator, which will each
operate on _all_ modules and requires each to repeat checks whether the
mutator should operator. Instead, we introduce a single mutator and
modules can define a ConvertWithBp2build to implement bp2build
conversion for that module.
Test: bp2build.sh
Bug: 183079158
Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
This also removes the special-case filegroup from mixed builds
buildroot; no special handling is required. Since we're currently
hardcoding linux_x86_64 as our host platform, it should be fine
to continue assumping that hardcoded host for now.
Test: USE_BAZEL_ANALYSIS=1 m adbd
Change-Id: I35509f4eb33ba7a243fab4c34b35958f6f2fceab
target.kind was a field that existed for only a few months, and have
been removed from Bazel. This is needed to update to a newer Bazel version.
Also, this reverts filegroups to be queried for a specific arch again,
but leaving file targets in the common arch (which is correct, anyway).
Fixes: b/199363072
Test: mixed_libc
Test: presubmits
Change-Id: I3b8e5c43a39516d977d327a44a75750b2f251be3
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule
Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.
As initial user, the Soong `required` property will be
translated to Bazel's `data`.
Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
This builds cap_names.list.h, which uses an eponymous filegroup
"generate_cap_names_list.awk" in Soong, but uses the file target
directly in Bazel.
This also improve filegroup support for mixed builds, by issuing a
cquery call _without_ arch. Filegroups in Soong don't have configurable
properties, so don't generate Bazel filegroups into buildroot's
config_nodes (which was x86_64 by default).
The mixed_build_root now looks like this:
```
config_node(...)
config_node(...)
config_node(...)
config_node(...)
...
filegroup(name = "common",
srcs = ["@//bionic/linker:linker_sources_x86",
"@//bionic/libc:kernel_input_headers",
"@//system/timezone/apex:com.android.tzdata-androidManifest",
"@//external/libcap:generate_cap_names_list.awk",
"@//bionic/linker:linker_sources_arm64",
"@//bionic/linker:linker_sources",
"@//bionic/libc:libc_sources_shared_arm",
"@//bionic/linker:linker_sources_x86_64",
"@//bionic/libc:all_kernel_uapi_headers",
"@//build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal-file_contexts",
"@//system/core/libcutils:android_filesystem_config_header",
"@//bionic/libc:libc_sources_static",
"@//bionic/linker:linker_sources_arm",
"@//bionic/libc/tools:bionic-gensyscalls",
"@//bionic/tools:bionic-generate-version-script",
"@//bionic/libc:libc_sources_shared"],
)
mixed_build_root(name = "buildroot",
deps = [":x86",
":arm64",
":arm",
":common",
":x86_64"],
)
```
Test: CI
Fixes: 198595323
Fixes: 198235838
Change-Id: I6df9a14da556cf358d96e6a99b514f66a2638295
with the same name.
Also add capability to test for errors raised in bp2build mutators /
contexts.
This CL does two things to filegroups:
1) If the filegroup has only 1 source file with the same name as itself,
don't generate a filegroup target. Instead, dependents will depend
directly on the Bazel file target instead.
2) If the filegroup has more than 1 source file and 1 of them has the
same name as itself, the bp2build mutator will error out. If bp2build
on CI passes, it means that the source tree / product we're testing
against does not have such a case (which seems to be true for most
source trees).
Either way, this will allow us to be unblocked for most of the errant
filegroups (case 1) in the tree.
Test: CI
Test: New test cases in filegroup_conversion_test.go
Fixes: 194762573
Change-Id: I830c53efc8808569afe3c5f9f08436855bcdafed
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3
We avoid both of these by storing the information directly in the
underlying module.
Also as a fringe benefit, removes some necessary boilerplate for
conversion.
For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714
Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
Because it's commonly useful and not cc/ specific
Also export GenerateBazelBuildActions and update uses
Test: Existing tests pass
Change-Id: Ibc6858bb1129afba181a7686dda432defe33b00d
This CL adds support for cc_library to correctly split c, as and cpp
srcs in shared/static nested props, as well as splitting
the *filegroup* deps in those props, where each filegroup is expanded
into its own c, cpp and as srcs filegroups. This ensures that the
correct sources go into cc_library_static's underlying cc_libraries for
c, cpp and as sources respectively.
See the bp2build conversion test for a better visualization.
Bug: 183064430
Test: TH
Change-Id: I29add5140672d042adff65527d8b65f4a5f0a05b
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.
It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.
Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.
Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
Adds the test fixture support and converts a few tests to exercise the
code and show how it works.
Bug: 181070625
Test: m nothing
Change-Id: I0a2b40fff93b6041f9aa8c4ef0aba91da1bc8bf3
This allows more direct access to the bazel label and whether the module
is bp2build available.
Test: go test *
Change-Id: I23375835d20fa53d7d94127b2dc2d5bb20487bfb
This eliminates the need to remove quotes, delete attributes, and
re-checking that name has correct prefix. Additionally, this allows
assignment directly to the BazelTargetModuleProperties struct, which
allows defaulting unused fields and clarity of which field is being set.
Test: go test soong tests
Test: ran ./build/bazel/scripts/milestone-2/demo.sh
Change-Id: Ia9bfcce76234c793a4ddd5f29a661150f83341c9
This CL replaces the "__bp2build__" name prefix boilerplate with a props
creation function, and centralizes the prefixing in there.
Test: TH
Test: soong tests
Change-Id: Ic963199ab60dcce0d3361abff111cfa9acd4c21b
This CL adds a per-target allowlist to instruct bp2build on which modules it should generate Bazel targets for.
Test: soong tests
Change-Id: I869e66fce405c2c6689b381569b8cc0118cbcf76
This CL refactors the CreateBazelTargetModule API to minimize boilerplate, and to establish a well defined function signature for the expected metadata about a BazelTargetModule.
Test: soong tests
Test: TH
Change-Id: I474ff5a2b0db8deeed49ba4ca73b416ccb494fdd
This currently expands all globs, still need to support converting glob
syntax.
Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
The previous implementation relied on the implicit registration of Bp2Build mutators, resulting in test non-hermeticity. Refactor bp2build tests to explicitly specify the bp2build mutators under test.
Test: Soong tests
Test: TH
Change-Id: I9b9674bad1ea533b3bd31b07077a9e02c99b4c1d
This allows setting per-module make variables earlier in the build,
particularly for prebuilt_build_tool users like LEX/M4/BISON (which will
come in a later CL). I moved filegroup over because it's a simpler
common interface, but it doesn't strictly need it.
Test: Inspect out/soong/make_vars-*.mk, out/soong/late-*.mk
Test: treehugger
Change-Id: I38631129d82b0aad2438e83610bb6a97568bf932
This reverts commit 2019658b86.
Reason for revert: broke builds where makefiles were using M4 without depending upon it
Change-Id: I18304246e7a8a8f8c2620dd2c65ed77acd256be2
This allows setting per-module make variables earlier in the build,
particularly for prebuilt_build_tool users like LEX/M4/BISON. I moved
filegroup over because it's a simpler common interface, but it doesn't
strictly need it.
With this, the last user of the hardcoded cc.m4Cmd variable is gone.
Test: Inspect out/soong/make_vars-*.mk, out/soong/late-*.mk
Test: treehugger
Change-Id: I195b688131feac0c100c338a0749368aa5d50f4f
Move the logic from ctx.ExpandSources into android.PathsForModuleSrc
and ctx.ExpandSource into android.PathForModuleSrc, and deprecate
them. When combined with the pathDepsMutator this will let all
properties that take source paths also take filegroups or genrule
outputs, as long as they are tagged with `android:"path"`.
Test: All soong tests
Change-Id: I01625e76b5da19240e9649bf26a014eeeafcab8f
Replace ExpandSourcesSubDir with ExpandSources plus
PathsWithModuleSrcSubDir, which loops over the paths and uses
Join to create paths relative to subdir on any results that
are ModuleSrcPaths.
Test: All soong tests
Change-Id: I11a7face88641e2c26ccdca0a3117d5c38ab588e
Cleaned up the synopsis of filegroups module under android package.
Bug: b/128337482
Test: built the documentation manually and verified that the changes
took place.
Change-Id: Ib5544125e5462fca7c7aebc4bed4647bba11d56b
Add `android:"path"` to all properties that take paths to source
files, and remove the calls to ExtractSource(s)Deps, the
pathsDepsMutator will add the necessary SourceDepTag dependency.
Test: All soong tests
Change-Id: I488ba1a5d680aaa50b04fc38acf693e23c6d4d6d
Bug: http://b/64121881
Bug: http://b/78188880
- Allow filegroup's properties to be extended by a LoadHook
- Support a filegroup (':module') in a prebuilt's 'Srcs' property to
export files from a different path as the prebuilt's sources.
This change also includes a refactoring that moves genrule/filegroup.go
to android/filegroup.go so that FileGroupFactory is visible in
prebuilt_test.go.
Test: Test
https://android-review.googlesource.com/c/platform/development/+/469159
in clang-tools branch on Linux, Darwin. Test regular build in
aosp/master.
Change-Id: I3ff6215ab2e62955f039fd1086c31f1bd50ebcf6
