If we have a soong_config_module_type with target.android_<arch>.*
properties, we should only panic if a soong_config_var is used to select
on those properties. We already call `IsZero` in an upper function, but
we need to call it in `AddSoongConfigPropertiesFromTargetStruct` as well
because conditions_default gets its own struct.
Test: unit tests
Test: repro'd cmd in b/293206386#comment1, it passes now
Bug: 293206386
Change-Id: I23f14f83cc9be66487625e292e26b204dfea4f9a
These tests cannot be run with b since it uses functions that should not be
used outside tests, and it enforces this by looking at os.Args. Since
the pattern (`-test`) is missing in os.Args, these tests panic in b.
A module can be a dep of soong_build via two mechanisms
1. Listed in deps explicitly
2. Via plugin_for to create a reverse dep
This CL denylists (1) explicitly. e.g. soong-android. Another option
would have been to denylist by using Deps() of soong-build, but that
would denylist `golang-protobuf-proto` as well
This CL denylists (2) programmatically.
Note that this CL only denylists creation of the `go_test` target. We
will still create `go_binary` and `go_library` for soong since there
might be tools that depend on soong.
e.g.
```
--> depends on
dexpreot_gen --> soong-dexpreopt --> soong-android (test failure)
```
(The above example means that we might need to convert _some_ of soong
after all, but we can refactor the common bits to common utils. Keeping
this out of scope for now).
Test: b test //build/soong/... //system/tools/...
too)
Bug: 284483729
Change-Id: I809012bdd383febca30da95e991edfde5fd4d9b9
For go modules with non-empty testSrcs, we will create an additional
go_test target. To build a standalone test executable, we need to
include the source files in the compilation unit. This will be done
using the `embed` attribute
1. For tests of go_library, this is straightforward. It will embed the
go_library and "inherit" its .go files and deps
2. For tetss of go_binary, we need to create an additional go_source
target since go_binary cannot be embedded inside a go_test
Using `b test` for these tests revealed that certain tests are not
hermitic and rely on `testdata` files checked into the tree. This CL
introduces an allowlist to skip generating go_test targets for them.
Test: bp2build unit test
Test: TH
Bug: 284483729
Bug: 288491147
Change-Id: Ic736d655babc2f6067e4da75384900b7b8bdc2ed
cc_test bazel macro will default `gtest` to True (similar to Soong). So
we can skip explicitly setting this in the generated BUILD files. The
benefit will be that it will make the generated files less verbose,
without loss of information.
This will be implemented by changing its datatype to *bool from bool.
Test: go test ./bp2build
Test: TH
Change-Id: I284e10f1d58c8e7893b170209827f7d5084ca95e
This property implicitly adds `libgtest_isolated_main` to the static
libs of the test. bp2build will make this and `liblog` explicit in BUILD
files
Since the deps are made explicit in BUILD files, `isolated` becomes a
no-op for cc_test. Remove this property from cc_test.
Test: unit tests
Bug: 244432609
Change-Id: I189a7b6b62d9064f4b2abad49ac4975468046498
Soong defaults to false, while bp2build defaults to true. This CL fixes
bp2build's default to false to match Soong.
To prevent future differences, bp2build now uses the `isolated()`
function used by Soong. The type of the context object has been changed
to EarlyModuleContext so that the ctx object does not depend on
mutations that might not run in bp2build.
Test: TH
Bug: 244432609
Change-Id: I15346107896312961e8d12270cf2f9a2a48827e1
Embedding multiple includes from a genrule may be difficult to read,
instead we generate a header library that contains the headers and all
include dirs that Soong generates.
Test: go test bp2build tests
Change-Id: I590c74c133f015f27cccf5a2fd916153ad9c125e
This is currently implicitly added by the cc_test macro. This can cause
duplicate issues if users write a BUILD file as
```
cc_test (
name = "mytest",
deps = ["libgtest"],
)
```
Making this dep explicit in BUILD files prevents this issue.
The property `gtest` will not be removed as part of this CL. cc_test
macro will use this property to add some gtest specific copts
Test: unit tests
Bug: 249374572
Change-Id: Ife931e26367fd4aab302bedc709cf907e2c01245
This allows us to set product variables as build settings instead
of loading them from a target's provider, which further allows us
to read product config variables in transitions.
Bug: 287539062
Bug: 269577299
Test: Presubmits
Change-Id: I8497703f706162572ceb3486240e1eb02a37f5f6
* changes:
Respect package boundaries in bp2build conversion of go modules
Create a temporary denylist for go binaries used in mixed builds
Partial bp2build conversion of blueprint_go_binary
Partial bp2build conversion of bootstratp_go_package
bp2build's codegen context does not implement
BazelPathConversionContext. To reuse the utility function
transformPackagePaths, update its signature
(Also make deps of go_library unique to make the conversion resilient)
Test: go test ./bp2build
Change-Id: I126b1057d2b26bc6c7d3be2780f1b62d28323cf0
This allows us to rollout building _some_ go targets using rules_go
without affecting mixed builds.
Test: Presubmit
Bug: 284483729
Change-Id: I0ccb4c9b90614369147a380f44f7ae372ef9396e
This module type does not implement android.Module, and therefore we
cannot write a conventional bp2build converter for this module type.
Instead this has been special-cased inside bp2build/build_conversion.go.
There is one major deviation between Soong and Bazel's
go_binary/go_library. Soong
collects the deps in the transitve closure and puts them on compile/link
paths. Bazel OTOH, requires the direct imports to be listed in deps of the binary
explicitly (AFAIK). Since bp2build cannot determine the list of direct
imports from the list of transitive deps, put all the transitive deps in
`deps`
Test: unit tests
Test: TH
Bug: 284483729
Change-Id: I004aaf8607fef1697a0d9e7d018ad657b67778ac
This module type does not implement android.Module, and therefore we
cannot write a conventional bp2build converter for this module type.
Instead this has been special-cased inside bp2build/build_conversion.go.
Because of the above, we also do not have access to useful functions
available in the ctx object of ConvertWithBp2build. This includes
1. Finding the package (directory) of a dep. This requires getting a
handle of the underlying module from its name (string). To solve, we
do a pre-visit to collect this information. This did not increase the
wall time. On my machine, `m bp2build --skip-soong-tests` takes ~14s
before and after this CL
2. Converting srcs to labels. This requires glob and package boundary
resolution. This CL introduces a partial implementation for this
function. (glob patterns are not used in go tools)
For (1), I considered creating a `ModuleFromName` on
`blueprint.Context` instead of a pre-run, but this increased the time to ~27s.
Test: unit tests
Test: TH
Bug: 284483729
Change-Id: Ifeb029103d14947352556dba295591dd7038b090
Bazel does not support using .rscrip/.fs files as cc srcs.
For a module foo with foo srcs, Bp2build will create a bazel
target of rule class rscript_to_cpp with name foo_renderscript
which will have only the renderscript files as srcs.
Bp2build will also create a target foo with the expected cc rule class.
The foo target will have all other src files as srcs and will also
have the target foo_renderscript as another src.
Bug: 210509914
Test: bp2build testing & b build target rstest-latency
Change-Id: Ifdc55051a3595f5fcf54eab8b59e11e9351e141c
This involves some minor changes to testing infrastructure.
Bug: 285631638
Fixes: 286545783
Test: unit test
Change-Id: If64ba29308d99e63a1cc5526feaf077f2cb14478
In Soong, CFI being explicitly set to false overrides presence in
CFI_INCLUDE_PATHS, so this change causes Bazel to mimic the same
precedence.
Bug: 278789968
Test: Unit tests
Change-Id: I827b3d4272eac2002031970f1e67c9321979b14b
This also changes the expectation of ConvertWithBp2build. Each
implementation must either create one or more Bazel target modules, or
mark the module as unconvertible (with a specific reason).
Manually verified no runtime hit in AOSP
In AOSP, the metrics file size increases from 252K to 1.6M
This changes some effective module counts in bp2build metrics:
- Removes "package" modules from the module count list in
metrics, as these will not be converted like regular modules.
- Counts Handcrafted modules as being "unconverted", as bp2build is not
responsible for them.
Bug: 285631638
Test: Verified generated BUILD.bazel files are bit-for-bit identical
with this change
Test: Manually verified one case of each implemented reasonType
Change-Id: I308dd451d8f28379b15671dae9f931bd0446f5c1
So that it's easier to maintain by being closer to the actual long-term
usage.
Bug: 279095899
Test: m nothing
Change-Id: I0b7cfa2eeb48595dd3081b458bbd1156ec2184a3
This is a followup to aosp/2606989.
This flag is not necessary now that we're using one platform name
for all of mixed builds.
Also rename current_product to mixed_builds_product so that it's clear
that that this platform should only be used for mixed builds.
In addition, make the bazelrc files point to the named products again
instead of the mixed build product so that b builds will still have
qualified outputs, but mixed builds won't.
Test: Presubmit and kernel build tools abtd run
Change-Id: I7f764cf42cd1323f4b495d1320931f59a076ac63
This reverts commit 2562585878.
Reason for revert: We now have aidl_library to remove this workaround to support specifying aidl headers
Change-Id: Idf03ebbea764f8c5b3c4fc1623f8f02aa321c8c1
This is a followup to aosp/2606989.
This flag is not necessary now that we're using one platform name
for all of mixed builds.
Also rename current_product to mixed_builds_product so that it's clear
that that this platform should only be used for mixed builds.
In addition, make the bazelrc files point to the named products again
instead of the mixed build product so that b builds will still have
qualified outputs, but mixed builds won't.
Test: Presubmit
Change-Id: I5acbd18869589f67efb0ad3ebfda2c4d5a6a3850
This change adds bp2build converter for platform_compat_config module
type and its corresponding test case.
Bug: 269202028
Test: bp2build unit tests
Change-Id: I17d560c8f0e725efe69e151685d003c3dcb11f2f
To support stem, cc libraries will be generated in
<pacakge_name>/<label.name>. Update RUNPATH of java binaries so that it
can find its jni deps
Test: bp2build
Test: TH
Bug: 240563612
Change-Id: Ia2f70f0424a8526ee35856225c1a8cd1067a8570
By default, the artifacts generated by cc_binary and cc_library in Soong
track the module name. But Soong supports overidding this using the stem
property. e.g.
```
cc_library {
name: "foo.1.2",
stem: "foo",
}
```
will generate foo.so
This CL adds this property to bp2build
Test: bp2build unit tests
Bug: 240563612
Change-Id: I1dfed870d5bad450511b72c397d2355c01fa3b60
To support export of NDK APIs in multi-tree, we added functionality to
generate Bazel targets of ndk related Soong module types. Since this use
case does not exist anymore, deprecate this to prevent bitrot
The removed code includes
- `ConvertWithApi2Build` implementation of these libraries, which
generates the Bazel targets in the synthetic `api_bp2build` workspace
- (api) bp2build unit tests for these module types
Test: go build ./cc
Bug: 284029211
Change-Id: Id3278fa119e0ab87f31f39a3783197a81b655e43
This enables output paths of product agnostic targets (e.g. mainline
apex) to be identical when switching between target products, since the
product name is no longer embedded in:
1) the output path prefix in bazel-out
2) the PlatformOptions contribution to the configuration hash as part of a transition
Test: treehugger
Change-Id: I3fd0f7134209aa7eede3ed98412a1c5482689b57
panic in top down mutator "bp2build_conversion_bp2build"
for module "foo".
reflect: call of reflect.Value.NumField on interface Value
Test: go test ./bp2build
Change-Id: I9ee460ca38a37c6a6c1a0787159fa5f38b852b55
host_ldlibs are used during linking. Set these in `linkopts` for now.
Note that this CL does not do the `CheckBadHostLdLibs` validation of
Soong. There are some different ways to do this, and these are being
discussed in b/216626461. It is likely that we will need to create a new
property `host_ldlibs` to do the validation. But for now, re-use
`linkopts`.
Bug: 216626461
Test: bp2build unit tests
Change-Id: Id4c77e4460fb1fb003fa58ea27bab5b50ea8cefe
https://source.android.com/docs/core/architecture/aidl/aidl-backends#custom-parcelables
Some aidl files have custom parcelables with references to cpp headers. Hence, we need to pass the include dirs from cc libs to cc_aidl_library so that custom cpp headers are findable by the generated cpp/h files.
Test: go test
Bug: 278704136
Change-Id: I4ede1d23fbc04bdbfb69823d955c6df6b0a7b4ee
Introduce aidl.libs prop on cc libraries to pass in aidl_library. The goal is to eventually disallow aidl.include_dirs (a pattern for passing aidl headers dir for aidl compilation) and enforce aidl headers to be explicitly specified in Android.bp.
Bug: 278704136
Test: go test
Change-Id: Ia78bc11dfa12f47d2d1bb90dc65372ddb17f7e14
We now rely on a `with_features` clause to effectively disable
thumb when `arm_isa_arm` is enabled. Thus, we no longer need to
add `-arm_isa_thumb` in bp2build.
Bug: 261733820
Test: Unit tests
Change-Id: I9da85834855b297a75097ce3a11e101f55bd53f4
For cc_libraries containing .y/.yy srcs, we will generate an additional
cc_yacc_static_library target. This target will produce an .a file that
will be statically linked to the parent target.
Test: go test ./bp2build
Bug: 281546029
Change-Id: I094ba56f0a95869e5bc1e1d38b83c777192b7ddd
* changes:
Special case platform variant of bootstrap libs
Select stub/impl per apex variant
For test apexes, base_apex_name is the api domain
Broaden the granularity of config_setting from apex_name to api_domain
Print default val if all vals in axis match default val
1) Use attribute name "default_package_metadata" instead of
"default_applicable_licenses" in packages to better describe its use.
2) Add a filegroup "default_metadata_file" in packages to search for
METADATA file in each package.
3) Include "default_metadata_file" in each package's
"default_package_metadata" attribute.
Bug: 275472038
Test: CIs
Change-Id: I645c013c39e3190fd96c4a549d39a331aced16bd
For converting the art plugins to pure soong, it would be useful to
have a property that's qualified on both a soong config variable and
the OS. Soong had very little-known support for this by saying your
soong config variable changes the "target.android.cflags" property,
and we didn't supporting bp2building that. Add the bp2build support.
This cl also refactors product variable and soong variable bp2building
so that they're separate from each other, which I think makes the code
easier to understand.
Test: go test
Change-Id: Ic74dc75da8103fa2523da95c3560c9ce3c5e5672
Create a select statement for every api_domain a library could be
included in. The stub/impl
selection heuristics per apex domain are
1. If dep has stubs and the api domain appears in dep's apex_available, use
impl
2. If dep has stubs and the api domain does not appear in dep's
apex_available, use stubs
(Category 3: If dep does not have stubs and the apex does not appear in
dep's apex_available, then a separate apex_available validation in Bazel
will emit an error).
Platform variants have been special-cased for now to use equality of
apex_available for stub/impl selection
Test: go test ./bp2build
Bug: 272378496
Change-Id: Ibd29efd763c8863c7e6d2a9af0da30bbde07175d
apex_test do not "override" the source apexes that they test. However,
similar to override_apexes, test apexes have the same path
(/apex/<apex_name>) on device. Instead of creating another property,
reuse the existing base_apex_name property. The use case for this will
be for creating selection statements for stub/impl selection.
Test: go test ./bp2build
Change-Id: I4b7548e0e0fc920e407e1c5e5c00e87efc6e70c9
The use case for this is for creating a select statement for stub/impl
selection. Since Bazel propagates apex_name from the top-level apex,
the source apex and test apex builds a specific library in two different
configurations. We need select statements for both these two
configurations, but this metadata might not always exist in Android.bp
since test apexes are not necessary to be listed in Android.bp files.
To overcome this, the select statements will be created per api domain
instead. This CL uses a naming convention to infer the api domain of
apexes.
Test: go test ./bp2build
Change-Id: Iad2b45f736bc98a24d2ce1cf2f69aad67973092d
Although paths to resource files in a Bazel java_library should be
relative to the package, the directory structure in the resulting jar
will have resources under the full path from the top-level of the
workspace, e.g. if a library in "a/BUILD" has java_resouces as
"res/res.txt" then by default the res.txt file would appear under
"a/res/res.txt".
Fix this by adding a resource_strip_prefix in that case.
Test: Unit tests
Change-Id: If4325126f5c19a2a8fb83ee09bc3a95a18673fe3
To avoid verbosity, we currently dedupe keys in axis if its value
matches the value of //conditions:default. For axes where all values
might match the default value, we would effectively drop the common
value.
To fix this, we are now dropping the select statement and not the common
value.
Test: go test ./bp2build
Change-Id: Ic377b93ee2aba971753f6a5e7a62e15d1fcfa2bc
Instead of exporting it to soong_injection.
Bug: 279095899
Test: m nothing
Change-Id: I7b93af233b7450848a475512b5f5682ece773c09
Merged-In: I7b93af233b7450848a475512b5f5682ece773c09
These are created by bp2build in /build/bazel/rules/apex. Eventually
these config_settings should likely be colocated with the source apex
definition.
Another alternative was to make Bazel's apex a macro that generates a
config_setting. I did not pursue this further for now since it requires the
apex_available of every allowlisted cc_library to also be allowlisted.
This might not always be true (e.g. com.android.runtime)
Test: go test ./bp2build
Change-Id: Ibbb14b0d9c1491b3c79b7634a18d9d35b03922c1
The use case for this is creating config_setting(s) specific to an apex
variant and selecting stub/impl in that config_setting. We likely need
only a handful of such config_setting(s), but determining that list
requires iterating the build graph.
Test: go test ./bp2build
Change-Id: I9aa552e3d0bcf67513023c3a7d4bbf8fae464ee4
There are a number of instances where we are exporting information
from soong to bazel via soong_injection. This could be more bazel-centric
if the information was instead held in bzl files, and both bazel and
soong read it from there.
Add a starlark package that will run
//build/bazel/constants_exported_to_soong.bzl at initialization time,
and then results can be retreived with GetStarlarkValue.
Since changes to the starlark files mean that soong has to rerun,
add them as ninja deps.
Unfortunately, the starlark code has to be run at runtime rather than
pregenerating their results, because tests run from intellij wouldn't
go through any pregeneration steps. This means that starlark is run
multiple times during the build, once per test package and once per
primary builder invocation. (currently 3, could be reduced to 2 if we
made the symlink forest generation into its own standalone tool) The
starlark code we have so far in this cl is very fast, roughly half a
millisecond, so it's not a big deal for now, but something to keep an
eye on as we add more starlark constants.
Bug: 279095899
Test: go test
Change-Id: I1e7ca1df1d8d67333cbfc46e8396e229820e4476
This reverts commit 1db4348734.
Changes from original:
- extracted function to add the current version to stub
versions for bp2build.
- added libc++ to mixed builds denylist
Reason for revert: re-uploading with fix in topic
Change-Id: Ifa0ed456bf8cb4a7f861d6826263adfedb4fdd9c
Previously we were treating it as a shared library only, but
cc_test_library produces both a static and shared library.
Test: bp2build tests
Change-Id: I293d4246d581212b95765fa8f084301514d00dbf
in bp2build. The context here is that `-fvisibility=default` should
only be added for CFI if `-fvisibility=hidden` is not already
specified. This will be achieved using toolchain features. Note
that Soong itself never adds `-fvisibility=hidden`. This is only
ever added in the `cflags` property of a bp file.
Bug: 261733820
Test: Unit tests
Change-Id: Ib821e8c30a9cd03d2929b4bd2e771bec7b33fa66
This reverts commit ca438e6b72.
Reason for revert: ci post-submit failures in aosp-master-bazel of targets mixed-droid-clean and mixed-droid-incremental:
FAILED: ninja: 'out/target/product/generic_arm64/obj/SHARED_LIBRARIES/libc_intermediates/libc.so.toc', needed by 'out/target/product/generic_arm64/obj/EXECUTABLES/updater_intermediates/LINKED/updater', missing and no known rule to make it
12:21:27 ninja failed with: exit status 1
Change-Id: I081b499d23f2568cdf6227c4e3b0278164086b69
In a follow-up CL, we should eventually disallow aidl.include_dirs in cc rules so that the deps are always explicit and compatible with Bazel migration.
Test: go test
Bug: 278059962
Change-Id: Ia786cc8634d03589dc008f10b01e6bb2b9f2c7f0
Bug: 270408757
Test: go test
Test: m NeuralNetworksTest_shared_partial --bazel-mode-staging
&& verify that Ninja commands link libneuralnetworks via stubs
Change-Id: I19bf5a829cea7fd00c0f82511ad2bddbc40fae4f
There was a request for using b with python tests. bp2build python
tests exactly the same way as python binaries so that they can be
used with `b`.
Bug: None
Test: go test
Change-Id: Id68a6a73572745a4885b3e5bb1b8452e36baa982
This CL adds java_version and sdk_version support to bp2build
converters for
- java library
- java binary
- android library
- android binary
- android library import
Although java import doesn't support java_version and sdk_version, the
neverlink java_library wrapper around a java_import must specify a
sdk_version when targetting a device. "none" is used by convention.
Change-Id: I22a69dea2e351858368df69ed6a703b568d613ea
Bug: 215230098
Test: Presubmits
We don't want rules to load the product variables directly, because
then they won't be able to transition on them.
Break constant information that is safe to load outside of the
product vars file, so that we can make the product vars file have
more restricted visibility later.
Bug: 269577299
Test: m nothing
Change-Id: I848bff33e4f5798f51296ea3a2600615cab36985
There was an incrementality bug in the symlink forest generation that
was recently fixed. However, if the appropriate files don't get touched,
the issue will remain. Add a mechanism to clean old symlink forests
so that when we fix incrementality bugs we can ensure they don't remain
in any old out directories.
Bug: 276349152
Test: Presubmits
Change-Id: I36664dfb0ca7227e3e1f89de859ebccb808c5f15
It was possible for the merged content to end up back in the bp2build
generated file because there was a symlink from the symlink forest to
the bp2build generated file.
Remove the symlink if it exists.
Bug: 276349152
Test: m bp2build, add a handcrafted file in the same folder as a Android.bp file, m bp2build again, check that the symlink forest version is not a symlink
Change-Id: Id64aa3addebcf0c6b1728389f21ae246796aaf8d
See the other cl in this topic for more information.
Bug: 269577299
Test: b test --config=android //build/bazel/rules/apex:all
Change-Id: I1f20bc5f85398cd79dc5b7fe9d0bd4d93d442383
These must be added in the soong_injection code as opposed to just
defined loosely in checked-in bzl files because the product_vars
select statement must be updated to support the new platforms.
Bug: 269577299
Test: b test --config=android //build/bazel/...
Change-Id: I7bba9af214896dd3b5938bae70b7c0cea4f75e41
Note this doesn't entirely match Soong's logic but is an improvement to
allow linking against implementation when two cc modules are
apex_available to the same module.
It is not possible to recreate the logic for "directly in" without
significant changes to bp2build as we do not add dependencies nor run
apex mutators. Rather than trying to replicate this, we would be better
off refactoring Soong to no longer support the "directly in apex" logic
and require users to correctly specify apex_available.
Bug: 272378496
Test: go test conversion tests
Change-Id: I17ac426f9b4bdad0c2ab661484e5d994f63568ce
The static deps of the parent library can be necessary to build a
bp2build-generated cc_aidl_library target. We should add these deps as
implementation_deps so that they are accessible.
Bug: 250876486
Test: b build //frameworks/native/libs/gui/...
Change-Id: Ibe7c3598a684907473e2a4e040fb3976455a59e9
BUILD files of rdeps should depend on stubs via @api_surfaces
indirection. e.g. instead of depending on
//system/logging/liblog:liblog_stub_libs_current, it should depend on
@api_surfaces//module-libapi/current:liblog. This ensures that the
generated BUILD files are compatible with Multi-tree.
Update the unit tests for this change.
Test: TH
Change-Id: Ibcc36dcfbee7b1973b341485f015e67987564dcc
Since the source_library attribute of _cc_stub_library_shared was a
label attribute, the source library was added as a dependency of the
stub which doesn't always make sense. E.g. it caused validation actions
for the source library to run even when we weren't building the source
library.
This change converts the label attriubte to a string attribute so that
we don't add the dependency.
Bug: 263390551
Change-Id: I19c62d1e14847dff68ab37156452addaa119eaf9
Bazel boolean attributes can only support true or false states, but the
clang-tidy logic changes for modules which don't set a specific value.
Therefore, the tidy attribute should really be a tri-state of "true",
"false", or "" (unset).
Test: b test //build/bazel/rules/...
Bug: 268681923
Change-Id: I85fbaa612d175f475757db806cea6c018901fe3e
for java_library, java_binary, android_app & android_library
Change-Id: Ia8dec7e7d497eb41b2d8dcd8c44c3090d1d3b4bd
Bug: 271265771
Test: go test ./bp2build
The expected use case for this is to create aliases for stub libraries
in the @api_surfaces repository in build/bazel/api_surfaces.
This restricts the scope to just aliases. If we have a use case for
generating actual Bazel targets in another directory, a workaround could
be to generate the targets in the current directory (via
CreateBazelTargetModule) and aliases to it in the other directory
Test: go test ./bp2build
Change-Id: I6b63d9d018618d447fc7c260a2a94aaa00e57a4d
override_apex's bp2build converter had a bug where it was looking at
the product variables for the override_apex module itself instead of
for the base module it is overriding.
Fixes: 271424349
Test: go test
Change-Id: If1e2653d3751fa908faf0ab97dfa2e943ebe98ec
This is to match the behavior of BUILD file generation.
Test: m and all existing bp2build conversion tests
Bug: 271122205
Change-Id: I6320a65aaad06114817eaed722cc0a0939c57bc1
there is now support for resoure_strip_prefix in kt_jvm_library targets.
Test: built AnalyzerKt and updated go ./bp2build tests
Change-Id: I4a6fe45276d45519186b6f40a02db990511d6def
Add Alwayslink to bazelPrebuiltLibraryStaticAttributes
Also adjust the corresponding tests to espect these new targets.
Test: m libc
Test: b build --config=android //prebuilts/clang/host/linux-x86:libunwind-exported_alwayslink
Test: bp2build.sh
Bug: 267214124
Change-Id: I4d996aef55321f1827947f00b0eaa52d0a14413e
With this CL, we avoid deleting the bp2build directory and regenerating
all BUILD files. Instead, we regenerate BUILD files which have changes
on the filesystem, and delete old BUILD files which should no longer
exist.
This improves incremental Bazel server performance by about ~5 seconds.
Previously, Bazel would have cache misses for regenerated packages, and
would thus need to analyze them even if no meaningful changes took
place.
For ease of implementation, we avoid removing stale (empty) directories
from the bp2build workspace, but this should have no effect on the
build. (These will be removed on next clean)
Test: New integration tests
Test: Manual benchmarking in conjunction with persistent bazel server
Change-Id: I3e489ff403be34040122876012329060a2506366
Given that we can map them directly to disabling the
corresponding Bazel features: `-{link_,use_lib}crt`
Test: Existing (adapted) Unit Tests
Test: bp2build.sh
Change-Id: Ib502f6fb929ace8e86a1001e3cc21f399317500c
Only public, system, test, module_lib and system_server are
converted in order to generate api_fingerprint.txt in Bazel.
Test: java_sdk_library_conversion_test.go and TH
Bug: 266973526
Change-Id: I67a00806165e5afad3876b6cd5cdbc6b0dd65d8b
This changes bp2build codegen, symlink forest generation, and
soong_build so that they do not rewrite output files if the contents are
unchanged.
Bug: 266983462
Test: m droid
Test: canonical_perf.sh benchmarking
Test: Manually verified that rerunning analysis did not regenerate
out/soong/workspace/prebuilts/sdk/BUILD.bazel unless contents changed
Change-Id: I5ec227df7a32b53c7fa0d741fb1403a51931024b
So that it can be used in the bazel implementation.
Bug: 268006095
Test: m nothing, check that it's in out/soong/soong_injection
Change-Id: I1520dd874076dee7fa083648d0790b060d658e5a
Previously, symlink forest generation involved removing the entire
symlink forest and recreating it from scratch. With this change,
a) symlinks which need not change are untouched,
b) symlinks pointing to the wrong location are fixed, and
c) symlinks which should no longer exist are removed.
On AOSP on my local machine, this reduces the symlink forest generation
step from 2.5s to 1.1s clean, and 0.6s when a single file is added to
a source directory.
Bug: 257528847
Test: m bp2build, touch `fakefile` under the forest, remove a file
from the source tree, rerun m bp2build. Manually verify the new forest
does not retain the link to the deleted source file, and that fakefile
no longer exists in the forest.
Change-Id: I481371ae487e9419af6a3a4370c552578b07d650
Previously, Bazel was invoked during mixed builds with a stripped down
environment. This CL adds an allowlist of environment variables that are
passed to Bazel.
Test: WITH_TIDY=1 DISABLE_ARTIFACT_PATH_REQUIREMENTS=true mss tidy-packages-modules-NeuralNetworks --bazel-mode-dev
Change-Id: I23147bec59f6522953cf623e7bcaa0f1f99a75a3
Test: bp2build tests and locally built MusicKotlin
(cannot allowlist yet due to soong/bazel custom_package differences)
Change-Id: I1cbd0fe2af0e6eeed65cf8d57dc85c1f54646c50
This fixes two main sources of nondeterminism:
1. Fix a bug in the ConfigurationAxis comparator (which caused
ConfigurationAxis sorting to be nondeterministic)
2. Process C++ dependencies using the sorted ConfigurationAxis order. In
theory, the order in which dependencies are processed shouldn't
matter (as they should end up in different select stanzas). However,
in the case of InApex stubs, this is not the case; we now ensure
that lists are concatenated in a predictable order.
Added bonus: Some cleanup with SortConfigurationAxes which
makes use of go generics (this made it easier to debug this issue).
Will follow-up with regression tests.
Test: Manually verified that build.ninja checksum and BUILD.bazel checksums do not change after running `m nothing` 6 times in AOSP (with comment-only Android.bp changes in between each run)
Change-Id: I81168e45bdbbcd61ea95ff665cf6c4bc180aa4e0
This API surface is provided by platform and mainline modules to other
mainline modules. Rename it to module-lib API surface to align it with
the terminology in go/android-api-types
Test: go test ./bp2build
Test: go test ./cc
Change-Id: Ieb9f3214e66366fc2ceb6f47e2d6623acb537827
We were looping over a map, and thus processing files
in a non-deterministic order. Some tests check for errors
produced during the symlink forest creation and they would
flake due to this nondeterminism.
Test: m nothing
Change-Id: Ie1a4cf98e341e4a951f110b7d5611bb69c4ddd2b
Brings down time taken to fail from 18s to 13s (on aosp branch)
Test: temporarily coded random failure
Bug: N/A
Change-Id: Ib694eec977293f4dd7054e779d1b82b8cace93f6
Previously if there was a value set outside of product variables and one
inside a conditions default soong config var, we would ignore the
conditions default. Now, we maintain the conditions default and use
the root value in cases where there is not a value for a product
variable.
Test: go test soong tests
Change-Id: Ic991e3aebe5bb6039353f4e3d25625e7c5190f96
- This is a wrapper function for all files in soong_injection directory.
This should prevent an error in the workspace generated for
api_bp2build.
- Rename the existing CreateSoongInjectionFiles fn and make it package
private to prevent confusion
(The subsequent CL in this stack should contain a smoke test for
api_bp2build)
Test: b build //:empty --config=api_bp2build --config=android
Test: TH
Change-Id: Iddb0aa1aff2f709826edd587aa99fccddf80f08f
adds support for *-> android_library and android_library->* edges
Change-Id: I41d4e1d1b8106a17e67951d47e67b59ef3170d17
Test: manually inspected build files for libWallpaperPicker & android-suppor-v4
Create a
build/bazel/product_config/generated/products/<product_name>/BUILD
file that contains the platform definitions needed for
a particular product. Currently we just create it for the
current lunch target, but the idea is that eventually when
all product config is in starlark, all the products will
have their platform definitions in the tree at once.
Bug: 249685973
Test: Presubmits
Change-Id: I08c82ff28dcf62f09d3b1d2e3186a6b961e12f6e
This is a follow up of aosp/2336916. All the properties that have
variant_prepend tag are supported in bp2build.
Bug: 261644490
Test: TH and modified tests with updated expected behaviors
Change-Id: I13bb5d4d8fcd38bf153fa9083c9c945aec2dfa19
When a cc library has stubs, its static variant shouldn't be included in an apex to ensure the library with stubs isn't statically linked. See https://cs.android.com/android/platform/superproject/+/master:build/soong/apex/apex.go;l=2804;drc=89b01aeaa9e19377ff547baab791277719b8aaf3
This CL modifies bp2build so that it excludes apex_available tag from the static target if the library has stubs. This way, we can ensure that no apex can depend on the static target which results to static linking.
Test: go test
Bug: 255589949
Change-Id: Iedf248994b808436f2440570b094fd06d6284ae9
and Sanitize.Misc_undefined.
This is a redo of aosp/2277186 with changes to account for issues
in mixed builds. Specifically, for now we're disabling mixed builds
for modules that use ubsan. This bug tracks enabling it:
b/261058727.
Bug: 253428057
Test: Unit tests
Change-Id: Ia1277a7fa9a82f40998d7f3d2c40ea90f38201e7
- The converter runs for api_bp2build, not bp2build workspace
- Since droidstubs is an internal module created by java_sdk_library,
the conversion encompasses the latter as well
- Since droidstubs do not have an api_surface attribute, this conversion
uses naming convention to infer the api_surface represented by the api
file e.g. *stubs.source -> publicapi, *stubs.source.system -> systemapi)
- Also adds an SdkIntraCore enum to represent the API surface provided
by one core module to another
There is also ongoing work to check in java_api_contribution modules in
Soong. Once we have that, we can update this converter to operate on
that module type instead
Test: go test ./bp2build
Change-Id: Ia85828e04c738d9ffcc524856d7c3034ee29bbf9
The comments in aosp/2336916 are also addressed here
Test: TH and TestCcLibraryVariantPrependPropOrder
Change-Id: If1472658fcd4b5544dec2e2691049a180520c84e
osLinux is implicitly glibc, use //build/bazel/platforms/os:linux_glibc
for the configurability select key for it.
Bug: 259266326
Test: build/bazel/ci/mixed_libc.sh
Test: USE_HOST_MUSL=true build/bazel/ci/mixed_libc.sh
Change-Id: Ifbd121ab3d943f3fdfbd3f6c3e982d1eaaaa9ec1
Because when stubs_symbol_file is defined on cc_library_shared, has_stubs is already implied to be true.
Bug: 260771171
Test: go test
Change-Id: I3b9a2cfb9000411d1b5f22da87dc339267775799
The BUILD file will now have the needed attributes to run the tradefed
test rules.
Test: b test //packages/modules/adb:adbd_test --platforms=//build/bazel/platforms:android_target
Change-Id: I5cb84ccdde27a7fadbef479269df594344a29187
So that they the allowlists can be read from build_mainline_modules_bundles.sh.
Bug: 262302253
Test: m out/soong/bazel_prod_mixed_builds_enabled_list.txt out/soong/bazel_staging_mixed_builds_enabled_list.txt && cat out/soong/bazel_prod_mixed_builds_enabled_list.txt && echo && cat out/soong/bazel_staging_mixed_builds_enabled_list.txt
Change-Id: Id5f89ebb651a52dc5368681a20d6183703e05bd4
* changes:
Change out path to android_target-opt in bazel integration tests
Add check for handcrafted BUILD file in bp2build
Add bp2build conversion to reference to fdo_profile targets
ag/20469925 added a handcrafted BUILD file (allowlisted in ag/20597986)
to internal master. ag/c/platform/build/soong/+/20583593/12/cc/bp2build.go#818 generates references to the targets in the BUILD file. However in tm-dev, the BUILD file doesn't exist while the .afdo files do.
One solution is to cherry-pick the BUILD file to tm-dev. However, tm-dev doesn't have vendor/google/build/soong/bp2build_allowlist.go to check in the BUILD file in bp2build.
This CL adds a check that the BUILD file exists to avoid the failure as in https://android-build.googleplex.com/builds/pending/P45724500/aosp_cf_x86_64_phone-userdebug/latest/view/logs/build_error.log in tm-qpr-dev-plus-aosp branch.
Bug: 253540178
Test: go test
Change-Id: I47fb853015ca230afe3cefe1d37728bf714624be
Merged-In: I47fb853015ca230afe3cefe1d37728bf714624be
The fdo_profile targets are handcrafted in ag/20469925. Bp2build creates fdo_profile attribute on cc_library_shared and references to the fdo_profile target if the profile exists. This works under assumption that all afdo profiles have an associated Bazel fdo_profile target declared in the same folder/package.
As noted in b/253540178, this won't work when we check in the BUILD files (which is likely 6+ months away) because some profiles might only exist internally but not in AOSP. We will implement a long-term solution once we figure out all the requirements we need to support afdo both in Soong and Bazel.
Bug: 253540178
Test: go tests
Change-Id: Iebd5dd7a76583b4b2bf0d9ee56d58f247d313a54
Merged-In: Iebd5dd7a76583b4b2bf0d9ee56d58f247d313a54
Some properties in soong are marked "variant_prepend", which means
in bp2build output, variant properties(select ...) should come before
general properties.
Prepend property is added in StringListAttribute
stuct so that bp2build can handlle variant_prepend properly, if
Prepend is properly set in different module logics.
This change also coordinates cc library with Prepend.
Coordination with other module types will be in separate CLs.
Bug: 261644490
Test: TestCcLibraryHeadersSimple,
TestCcLibraryHeadersArchAndTargetExportSystemIncludes and TH
Change-Id: I3b61c491c54bbe91ba9aa2af9b4c84193c91ae7f
The libc_musl_crt* cc_object modules use header_libs to add
headers to the search path. Propagate static_libs, shared_libs
and header_libs to includes_deps.
Bug: 259266326
Test: TestCcObjectHeaderLib
Change-Id: I8db4d6886761426d3ece38c43ac868d3248f7a9f
Add bp2build support for cc_prebuilt_object to allow buiiding
musl crt objects that use prebuilt clang_rt.crtbegin and
clang_rt.crtend.
Bug: 259266326
Test: //extenral/musl:libc_musl builds
Change-Id: Ic2b0375bc6e7336d31c9e2cf51bdc2ea894185aa
and Sanitize.Misc_undefined.
This is a redo of aosp/2277186 with changes to account for issues
in mixed builds. Specifically, for now we're disabling mixed builds
for modules that use ubsan. This bug tracks enabling it:
b/261058727.
Bug: 253428057
Test: Unit tests
Change-Id: I9dbec1d99353f8cbe6751d5241283ab19592dc27
Previously, the fixture test infrastructure was hard coded to
initialize itself in preparation for invoking the PrepareBuildActions()
method. That meant it could not be used for testing the bp2build and
apiBp2build modes which required different initialization and called a
different method.
This change extracts that behavior into a FixtureTestRunner and adds an
implementation that allows it to test the above two modes. It then uses
that to implement the runBp2BuildTestCaseWithSetup method.
The TestPrebuiltLibraryAdditionalAttrs was the only test which broke as
it supplied an invalid path to the MockFS (it does not support using a
trailing / to represent an empty build directory). So, it was modified
to add a couple of files inside those directories instead.
Test: m nothing
Change-Id: I6798a4f761160af8d1bfed81d46de9628bda3eb9
The tags will then be read by an aspect applied from apex attributes to
validate that the deps are explicitly opt-in to the appropriate apex.
The semantics of this is the same as the Soong apex_available property.
This also adds the "base_apex_name" attr to override_apex conversion so
that apex_available checks continue to work on the non-Google and non-Go
apex names.
Test: presubmits
Fixes: 218841706
Change-Id: I19b3637da9aa47573b252d74f62b601fbdc3784d
Revert submission 2277186-ubsan_bp2build
Reason for revert: b/260630869 broke mixed_droid
Reverted Changes:
Idd9b33e53:bp2build conversion for Sanitize.Integer_overflow
I0d636e439:Add Sanitizer libs up the dependency tree
If113de7bc:Introduce toolchain features for sanitizers
Change-Id: Idf7e7b62f6fb6d4ff92341cc4c9c4b7426b70951
The axis for OS and in APEX was previously ignoring the excludes from
OS, which would result in excludes for non-Android OSes being
incorrectly ignored.
Test: go test Soong tests
Test: b build //hardware/libhardware:all
Bug: 260131489
Change-Id: Ie77f763bea0f473ac69a0c6b6bc3216e7359ad12
Bazel needs to recreate the neverallow behavior for absolute includes,
so export the list of include dirs to soong_injection.
Bug: 250931909
Test: aosp/2302925
Change-Id: I3bd345ea4cfb3181f07e4570466f2fc8c5fe32da
java_library modules with .kt srcs or common_srcs will be converted into
module type kt_jvm_library
Test: m bp2build, manually inspected build files for allowlisted modules
Bug: 258688914
Change-Id: I8293a11c8247b4b76358d0991f82c6b61b58adc3
Previously, it stored an android.Context (no pointer) which while it
worked (because the current contents are themselves pointers) it is
inconsistent with how the rest of the code references it and could
cause weird behavior if additional fields were added to the Context.
Test: m nothing
m bp2build
Change-Id: I9c70f925dde85e5b15431cc232b0c2eb6371ec6d
This is a second attempt at aosp/2273288 which got rolled back because
it turned out that:
1. We make ~120K symlinks in AOSP (!), all of which need to be deleted
2. System calls are sometimes slow
3. Golang spawns a new OS-level thread for each blocking system calls to
keep cores busy
All this together means that we sometimes had 10K system calls in
flight, which meant 10K OS-level threads, which is when Go gives up and
says "I created too many threads, please help".
The fix is to move the system calls into a pool of goroutines, which
soon end up on a pool of threads (since they mostly do blocking system
calls)
Test: Presubmits.
Change-Id: Ia9aefff3b0ed373f09bb6c8b2ec1d8b0f00b213b
Bug: b/256212479
Test: Prior to thi CL bp2build.symlink_forest event was missing in bp2build_metrics.pb after a clean mixed build
Change-Id: I53bfc4114a383c0d1f9c4c7945e7d4c69bc50b0c
Since Bazel's java_import requires a jars attribute to be specified,
the generated neverlink-duplicated module is of type java_library
Change-Id: I14a866dfc583507a9462add50d95060cbfe540c5
Bug: 244210934
Test: m bp2build, go test ./bp2build, manual inspection of generated Build and jar files
This makes symlink forest creation ca. 2x faster again, taking 2-3
seconds instead of 5.
Who would have thought that os.RemoveAll() is slow.
Test: Presubmits.
Change-Id: I91e41319c972dbf1113cf723e383c785433c18b9
This makes it take ~5 seconds on AOSP instead of ~10. Frankly, the
speedup is somewhat disappointing but at least the code is not
complicated.
Test: Presubmits.
Change-Id: Icf94d7ca8bd80c458d014f4cf4cc1be7138deaa6
This helps with incrementality a lot: the symlink forest must depend on
almost every directory in the source tree so that if a new file is added
or removed from *anywhere*, it is regenerated.
Previously, we couldn't do this without invoking bp2build, which is
quite wasteful because bp2build takes way more time than the symlink
forest creation, even though we do the latter in a very suboptimal way
at the moment.
This means that if a source file is added or removed (which does not
affect globs), we don't pay the cost of bp2build anymore.
Also refactored symlink_forest.go on the side. Too much state was being
passed around in arguments.
This change reimplements aosp/2263423 ; the semantics of not touching an
output file is the exact same as order-only inputs and the latter is a
bit fewer lines of code.
Test: Presubmits.
Change-Id: I565c580df8a01bacf175d56747c3f50743d4a4d4
bp2build (i.e. bootstrap.ninja#bp2build_workspace_marker) generate $OUT/soong/bp2build/**/BUILD.bazel files
Having them as a dependency would thus make bp2build_workspace_marker stale upon incremental builds (because we don't re-touch the marker file if it already exists)
See Also: https://android-review.googlesource.com/c/platform/build/soong/+/2263423
Bug: b/253450880
Test: NINJA_ARGS='-d explain' m --bazel-mode adbd
repeat and see if bp2build is rerun
prior to this CL, one would see the input /usr/local/google/home/usta/aosp/out/soong/bp2build/build/make/tools/BUILD.bazel trigger bp2build over and over again
Change-Id: I904cd333a5d6ef506fc4019eda7623ef96a1daa3
Soong supports string properties, but they are overloaded, and can mean
one of three things:
* path reference
* module reference
* string literal
Bazel has different types: label and string attributes. Thus there needs
to be a way to categorize them correctly in bp2build.
This CL introduces a new function to be used on properties like
apex_key.private_key / apex_key.public_key, as well as
android_app.certificate / apex.certificate.
It is important to disambiguate the prop betenn a string literal
attribute or file/rule target label attribute, so this functions does
just that. The new attributes are then further handled by their
respective macros (apex_key, android_binary, apex).
Bug: 253557437
Fixes: 253557437
Test: presubmits, new tests
Change-Id: Id8111cdd60d3aabcae7d17fe9da84d0ee3966023
Also disallow arch variant of genrule.out.
This is to be consistent with bazel where we are migrating to.
Bug: 254114674
Test: Manual
Change-Id: I685a2e64102b7bb68128b39931f0bc85878bc6de
In Soong, decodeMultilib, used to get multilib to determine the dep variations, return "first" if defaultMultilib is set to "common". apex sets defaultMultilib to "common" which means equivalent compileMultilib in bp2build for apex should be "first" (See new Soong unit tests for more context).
This CL fixes bp2build for apex to be more correct.
Bug: 251559512
Test: go tests
Change-Id: Id1cb4407980fc1fab91822c81326f37fb4adfa0a
1) Bp2build convert python_binary_host main attribute as LabelAttribute. Currently "main" attribute in python_binary_host is handled as string but for some modules (e.g certify_bootimg) the "main" attribute points to a file in its subpackage like "subpackage/file.py" and should be converted to "//.../subpackage:file.py".
2) Filter out duplicated labels in the merged list of "required" attributes of python_binary_host and its defaults.
Test: b build //system/tools/mkbootimg:certify_bootimg
Test: b build //build/make/tools/releasetools:check_target_files_signatures
Bug: 253081249
Bug: 253101186
Change-Id: Ic2cb4cadec2c1348da70af9f0730da9914d3a8ca
