This is a followup to aosp/2711093 which added support for
proto.include_dirs. local_include_dirs is simlar, except that is
relative to the module directory.
Test: go test ./bp2build
Bug: 285140726
Change-Id: I32ddc7371048672d6935f827c8aee9d767305e2c
This is a fix similar to aosp/2707793. This adds manual tags to the
top-level proto_library created in the root package, plus any
dynamically created proto_library targets for proto.include_dirs. These
proto_library targets might not specify their deps correctly and might
be unbuildable.
(We need to keep these targets because they provide a ProtoInfo for
cc/java/py source gen)
Test: go test ./bp2build
Change-Id: Ic00f05186327fcfcc8d33a0a2c0891ed619b7acb
The config constant is consumed by rust toolchain implemented in aosp/2709668.
Bug: 295918553
Test: go test
Change-Id: I0a28a802ad28f6ee24dfb1b5bdab72094534ae04
Since we no longer support "zip" APEX, we don't need "host" support.
For example, we don't need go/python binary support.
Bug: 279835185
Test: m
Change-Id: I6e8d2b205e42662f31866dc9ac7507524effd144
expandSrcsForBazel always prefixed : in OriginalModuleName. The
exceptions to this are filegroups that appear in a different soong
namespace. For these cases, we were not correctly substituting the soong
module name with the equivalent bazel label.
Test: go test ./bp2build
Change-Id: If090f3f8819835177c1f4d191b3eef6bb6e30ace
There are three types of tests, deviceless tests, device-driven tests
and host-driven device tests. But currently we don't have information
to get the type of a test and can't generate test targets on desired
types.
Test: b test //platform_testing/tests/example/native:hello_world_test
Test: b test //packages/modules/adb:adbd_test
Bug: 296312548
Change-Id: I3f022ef769636d508e055477623a4d1a6a1d9044
Soong's proto.include_dirs becomes the -I path for aprotoc cpp code
generation. This does not translate well to Bazel because it runs this
action in a sandbox. Even if we construct an -I path, the .proto files
will not be there. This CL attempts to handle this kind of dependency
automatically via bp2build.
For this hypothetical example
```
foo.proto # contains `import bar.proto"
Android.bp # cc_library {srcs:"foo.proto", p.include_dirs:["subdir"]},
subdir/bar.proto
```
Implementation details for CcProtoGen of foo
- Glob the labels of .proto files for each includeDir, and create a
proto_library that encapsulates them.
- Since Bazel poses a contraint that proto_library target needs to be
in the same pacakge as the .proto file, the proto_library might be created
in a subdirectory with an import_prefix
- Add bar's proto_library as transitive deps of foo's cc_proto_library.
This will be added to -I during aprotoc. We cannot add them to `deps`,
otherwise bar's symbols will be statically linked into foo's .a
file.
Implementation details for clang compile of foo
At the end of CcProtoGen, we have converted foo.proto
to .cpp/.h files. To compile them to .a files, we need the .h files
generated from bar.proto. Soong specifies this at the
top-level cc_library soong module, so add those deps as the
implementation deps for clang compile.
(Will add support for java in a follow-up CL)
Test: go test ./bp2build
Test: built some internal modules that were previously blocked by this
Bug: 285140726
Change-Id: I7e1f9f0d1b1ba916a7ba8278f6cfb342b381d695
Mostly so that genrules can have their command qualified on a product
variable.
Bug: 295910468
Test: m nothing
Change-Id: I28cc18a1b3b00368f4768a1326259a656ce970ec
This is a fix for aosp/2693190 that handled .proto files that end up in
different bazel packages. It did it by creating proto_library targets in
the correct bazel package.
Changing the granularity causes issues if the the new proto_library in
the subpackage imports a .proto file from a parent package or a
different package. e.g.
```
tmp
├── foo.proto
└── subdir/import_foo.proto # contains an `import "foo.proto"`
└── subdir/Android.bp # package boundary
├── Android.bp # contains a cc_library with foo.proto and
# subdir/import_foo.proto
```
At ToT, the ProtoInfo we provide to CcProtoGen is correct, but the
proto_library in subdir/BUILD will not compile because it does not have
a dep on the proto_library in ./BUILD
This CL creates a workaround by adding `manual` to the proto_library
targets. This CL is based on the assumption that the buildable unit in
bp2build is cc_library_*, and not proto_library necessarily (atleast
till we do a manual/automated cleanup)
Test: Created an integration test in build/bazel
Test: go test ./bp2build
Bug: 292583584
Bug: 246997908
Change-Id: I73120be2411967cb144f37ed4417f76ecf1a6ffa
So that users can use soong config variables / product variables
to adjust a genrule's command.
Bug: 295910468
Test: m nothing
Change-Id: I9fedf8d5d52e515c3fdb913411ce1b3fecb7ba81
Protos in another directory were using import prefix, which was
prepending the repository-relative path with the value, instead, we want
to strip the prefix of the directory of the module the protos were used
in such that they can be referenced at the appropriate relative path.
Reference on import_prefix:
https://bazel.build/reference/be/protocol-buffer#proto_library.import_prefix
Test: b build //packages/modules/adb:libfastdeploy_host
Change-Id: If050b0f5fc5103bd9cc5a99703bd604325aa4204
In this Android.bp file
```
my_cc_defaults {
enabled: false,
soong_config_variables: {
my_bool_variable: {
conditions_default: {enabled: false},
}
}
}
```
The inner enabled: false is a no-op because the top-level enabled is
false. Currently, bp2build will raise an exception for this Android.bp
file.
However, it does not need to. `productVariableConfigEnableLabels` runs
only if the top-level enabled is false. If it sees enabled: false via
conditions_default, it should just ignore it since it is a no-op.
Test: go test ./bp2build
Bug: 210546943
Change-Id: I816f209eaf21de65ddfbc2893e5255be94bcaa11
with 4000X clones of adbd, bp2build.Codegen time goes from 36s to 7s
Test: m bp2build and verify same BUILD.bazel files
Bug: 292281398
Change-Id: Ibd3b328a917fe5d1bdad67c67116da336f9033cb
If enabled does not appear inside `soong_config_vars`, we can ignore it.
Bug: 210546943
Test: go test ./bp2build
Change-Id: I9e4d51c3b683f262921449634f827915ce87dc8d
* changes:
Handle .proto files in different package for filegroups
Handle .proto files that end up in a different package
Allow creation of BazelTargets in a different directory
Followup to aosp/2693190, this CL adds the support to filegroups.
<mod>__bp2build_converted is now an alias to a proto_library
target <mod>_proto. This proto_library will be created in a different
package if the .proto file exists in a different package.
Test: bp2build unit tests
Test: TH
Bug: 292583584
Change-Id: I8ca452aacf1a86dfc9e218464e38aab89afa5a29
Bazel poses a strict requirement that .proto files and proto_library
must be in the same package. This CL handles this automatically by
creating the proto_library in a separate dir/package if necessary
Implementation details
- Partition the `srcs` by package. `srcs` has been computed using
`transformSubpackagePath`, so the information about packages is
available at this point
- Create a proto_library in each package by using
`CommonAttributes.Dir`. Collect all these additional libraries
and put them in `info.Proto_libraries` so that they get added as deps
of (cc|python|...)_proto_library
- Add an import_prefix to the proto_library in subpackages relative to
the current directory. This relies on the assumption that every src is
beneath the current directory (Soong will complain if a path in
Android.bp contains ../)
filegroup module type uses a separate code-path to create proto_library.
This will be handled in the next CL in stack.
Test: bp2build unit tests
Test: TH
Test: Built the failing internal module mentioned in
b/292583584#comment1
Bug: 292583584
Change-Id: I437fc89092321b26c5f0511387cde9e84084d6f9
The current API restricts creation of targets to the directory of the
visited soong module. This CL proposes adding a `Dir` property in
`CommonAttributes` that can be used to create a bazel target in
a specific dir. The use case for this is to dynamically create
additional targets for proto_library that are adjacent to .proto files
(Bazel poses a strict requirement about proto_library being in the
same package as the .proto file, but Soong does not)
Usage is restricted to dirs that have an existing Android.bp file. There
are some places in bp2build where we use existence of Android.bp/BUILD
on filesystem to curate a compatible fully qualified path (e.g. headers).
If we use `CommonAttributes.Dir` to arbritraily create BUILD
files, then it might render those curated labels incompatible.
Test: go test ./bp2build
Change-Id: If9446700457eddfb389be9d9bde39087f67daa60
That build doesn't set platform_sdk_version.
Bug: 269577299
Test: ./prebuilts/kernel-build-tools/build-prebuilts.sh on the kernel-build-tools branch
Change-Id: I68eb33bfb80770cef9c3a9c5e01f1ed746a2d069
If `srcs` contains a gensrcs/genrule module, the current bp2build module
will put it in the catch-all `srcs` attribute. This is reserved for .cpp
sources, so if the genrule produces a .proto/.aidl/... file, this will
fail.
This handles genrules that produce .proto files. To implement this, this
creates an additional partition that detects if the other module is a
genrule/gensrc that produces .proto files. If true, it will append it to
the proto partition.
This CL does not handle
- genrule that produce .c/.aidl/.yacc/.... files. They will continue to
be partitioned into the catch-all partition
- java modules
Test: unit tests
Test: TH
Bug: 293205700
Change-Id: Ib720fdf3a053ff5dd256e6faa632e3fa7776966d
...instead of based on constraint settings.
Bug: 269577299
Test: m nothing and ./build/bazel/ci/bp2build.sh
Change-Id: Ib9caec79c92b8fd304e46be841de5612bd1637e3
Since these modules have been converted, we should tag them as converted
in the dashboard.
Test: Put a print statement for `metrics` and made sure that the go
modules appear there
Bug: 294098662
Change-Id: I6282100111030a94f15f330916eaf41fcfc16e1a
Revert submission 2651299-bazel_sandwich
Reason for revert: b/293883239, checking if this breaks the build.
Reverted changes: /q/submissionid:2651299-bazel_sandwich
Change-Id: I5f1d4bb13d21e6599f5c353dcaba2375f5ec234d
The "bazel sandwich" is a mechanism for bazel to depend on make/soong
outputs. The name comes from the fact that bazel is now at the top
and bottom of the build graph. This is intended to allow us to work
on converting the partition builds to bazel while not all of the
dependencies of the partition have been converted.
It works by adding the bazel_sandwich_import_file rule, which emits a
dangling symlink that starts with bazel_sandwich:, and includes
information that the aquery handler in soong reads. The aquery handler
rewrites the symlink so that it points to a file generated by
make/soong, and adds a ninja dependency from the symlink to the file
it's targeting.
This allows us to depend on make-built files from bazel, but notably
it doesn't allow us to depend on analysis-time information from make.
This shouldn't be a problem for the partitions, but limits the use of
the bazel sandwich to similar, less complicated types of builds.
go/roboleaf-bazel-sandwich
Bug: 265127181
Test: m bazel_sandwich
Change-Id: Ic41bae7be0b55f251d04a6a95f846c50ce897adc
If we have a soong_config_module_type with target.android_<arch>.*
properties, we should only panic if a soong_config_var is used to select
on those properties. We already call `IsZero` in an upper function, but
we need to call it in `AddSoongConfigPropertiesFromTargetStruct` as well
because conditions_default gets its own struct.
Test: unit tests
Test: repro'd cmd in b/293206386#comment1, it passes now
Bug: 293206386
Change-Id: I23f14f83cc9be66487625e292e26b204dfea4f9a
These tests cannot be run with b since it uses functions that should not be
used outside tests, and it enforces this by looking at os.Args. Since
the pattern (`-test`) is missing in os.Args, these tests panic in b.
A module can be a dep of soong_build via two mechanisms
1. Listed in deps explicitly
2. Via plugin_for to create a reverse dep
This CL denylists (1) explicitly. e.g. soong-android. Another option
would have been to denylist by using Deps() of soong-build, but that
would denylist `golang-protobuf-proto` as well
This CL denylists (2) programmatically.
Note that this CL only denylists creation of the `go_test` target. We
will still create `go_binary` and `go_library` for soong since there
might be tools that depend on soong.
e.g.
```
--> depends on
dexpreot_gen --> soong-dexpreopt --> soong-android (test failure)
```
(The above example means that we might need to convert _some_ of soong
after all, but we can refactor the common bits to common utils. Keeping
this out of scope for now).
Test: b test //build/soong/... //system/tools/...
too)
Bug: 284483729
Change-Id: I809012bdd383febca30da95e991edfde5fd4d9b9
For go modules with non-empty testSrcs, we will create an additional
go_test target. To build a standalone test executable, we need to
include the source files in the compilation unit. This will be done
using the `embed` attribute
1. For tests of go_library, this is straightforward. It will embed the
go_library and "inherit" its .go files and deps
2. For tetss of go_binary, we need to create an additional go_source
target since go_binary cannot be embedded inside a go_test
Using `b test` for these tests revealed that certain tests are not
hermitic and rely on `testdata` files checked into the tree. This CL
introduces an allowlist to skip generating go_test targets for them.
Test: bp2build unit test
Test: TH
Bug: 284483729
Bug: 288491147
Change-Id: Ic736d655babc2f6067e4da75384900b7b8bdc2ed
cc_test bazel macro will default `gtest` to True (similar to Soong). So
we can skip explicitly setting this in the generated BUILD files. The
benefit will be that it will make the generated files less verbose,
without loss of information.
This will be implemented by changing its datatype to *bool from bool.
Test: go test ./bp2build
Test: TH
Change-Id: I284e10f1d58c8e7893b170209827f7d5084ca95e
This property implicitly adds `libgtest_isolated_main` to the static
libs of the test. bp2build will make this and `liblog` explicit in BUILD
files
Since the deps are made explicit in BUILD files, `isolated` becomes a
no-op for cc_test. Remove this property from cc_test.
Test: unit tests
Bug: 244432609
Change-Id: I189a7b6b62d9064f4b2abad49ac4975468046498
Soong defaults to false, while bp2build defaults to true. This CL fixes
bp2build's default to false to match Soong.
To prevent future differences, bp2build now uses the `isolated()`
function used by Soong. The type of the context object has been changed
to EarlyModuleContext so that the ctx object does not depend on
mutations that might not run in bp2build.
Test: TH
Bug: 244432609
Change-Id: I15346107896312961e8d12270cf2f9a2a48827e1
Embedding multiple includes from a genrule may be difficult to read,
instead we generate a header library that contains the headers and all
include dirs that Soong generates.
Test: go test bp2build tests
Change-Id: I590c74c133f015f27cccf5a2fd916153ad9c125e
This is currently implicitly added by the cc_test macro. This can cause
duplicate issues if users write a BUILD file as
```
cc_test (
name = "mytest",
deps = ["libgtest"],
)
```
Making this dep explicit in BUILD files prevents this issue.
The property `gtest` will not be removed as part of this CL. cc_test
macro will use this property to add some gtest specific copts
Test: unit tests
Bug: 249374572
Change-Id: Ife931e26367fd4aab302bedc709cf907e2c01245
This allows us to set product variables as build settings instead
of loading them from a target's provider, which further allows us
to read product config variables in transitions.
Bug: 287539062
Bug: 269577299
Test: Presubmits
Change-Id: I8497703f706162572ceb3486240e1eb02a37f5f6
* changes:
Respect package boundaries in bp2build conversion of go modules
Create a temporary denylist for go binaries used in mixed builds
Partial bp2build conversion of blueprint_go_binary
Partial bp2build conversion of bootstratp_go_package
bp2build's codegen context does not implement
BazelPathConversionContext. To reuse the utility function
transformPackagePaths, update its signature
(Also make deps of go_library unique to make the conversion resilient)
Test: go test ./bp2build
Change-Id: I126b1057d2b26bc6c7d3be2780f1b62d28323cf0
This allows us to rollout building _some_ go targets using rules_go
without affecting mixed builds.
Test: Presubmit
Bug: 284483729
Change-Id: I0ccb4c9b90614369147a380f44f7ae372ef9396e
This module type does not implement android.Module, and therefore we
cannot write a conventional bp2build converter for this module type.
Instead this has been special-cased inside bp2build/build_conversion.go.
There is one major deviation between Soong and Bazel's
go_binary/go_library. Soong
collects the deps in the transitve closure and puts them on compile/link
paths. Bazel OTOH, requires the direct imports to be listed in deps of the binary
explicitly (AFAIK). Since bp2build cannot determine the list of direct
imports from the list of transitive deps, put all the transitive deps in
`deps`
Test: unit tests
Test: TH
Bug: 284483729
Change-Id: I004aaf8607fef1697a0d9e7d018ad657b67778ac
This module type does not implement android.Module, and therefore we
cannot write a conventional bp2build converter for this module type.
Instead this has been special-cased inside bp2build/build_conversion.go.
Because of the above, we also do not have access to useful functions
available in the ctx object of ConvertWithBp2build. This includes
1. Finding the package (directory) of a dep. This requires getting a
handle of the underlying module from its name (string). To solve, we
do a pre-visit to collect this information. This did not increase the
wall time. On my machine, `m bp2build --skip-soong-tests` takes ~14s
before and after this CL
2. Converting srcs to labels. This requires glob and package boundary
resolution. This CL introduces a partial implementation for this
function. (glob patterns are not used in go tools)
For (1), I considered creating a `ModuleFromName` on
`blueprint.Context` instead of a pre-run, but this increased the time to ~27s.
Test: unit tests
Test: TH
Bug: 284483729
Change-Id: Ifeb029103d14947352556dba295591dd7038b090
Bazel does not support using .rscrip/.fs files as cc srcs.
For a module foo with foo srcs, Bp2build will create a bazel
target of rule class rscript_to_cpp with name foo_renderscript
which will have only the renderscript files as srcs.
Bp2build will also create a target foo with the expected cc rule class.
The foo target will have all other src files as srcs and will also
have the target foo_renderscript as another src.
Bug: 210509914
Test: bp2build testing & b build target rstest-latency
Change-Id: Ifdc55051a3595f5fcf54eab8b59e11e9351e141c
This involves some minor changes to testing infrastructure.
Bug: 285631638
Fixes: 286545783
Test: unit test
Change-Id: If64ba29308d99e63a1cc5526feaf077f2cb14478
In Soong, CFI being explicitly set to false overrides presence in
CFI_INCLUDE_PATHS, so this change causes Bazel to mimic the same
precedence.
Bug: 278789968
Test: Unit tests
Change-Id: I827b3d4272eac2002031970f1e67c9321979b14b
This also changes the expectation of ConvertWithBp2build. Each
implementation must either create one or more Bazel target modules, or
mark the module as unconvertible (with a specific reason).
Manually verified no runtime hit in AOSP
In AOSP, the metrics file size increases from 252K to 1.6M
This changes some effective module counts in bp2build metrics:
- Removes "package" modules from the module count list in
metrics, as these will not be converted like regular modules.
- Counts Handcrafted modules as being "unconverted", as bp2build is not
responsible for them.
Bug: 285631638
Test: Verified generated BUILD.bazel files are bit-for-bit identical
with this change
Test: Manually verified one case of each implemented reasonType
Change-Id: I308dd451d8f28379b15671dae9f931bd0446f5c1
So that it's easier to maintain by being closer to the actual long-term
usage.
Bug: 279095899
Test: m nothing
Change-Id: I0b7cfa2eeb48595dd3081b458bbd1156ec2184a3
This is a followup to aosp/2606989.
This flag is not necessary now that we're using one platform name
for all of mixed builds.
Also rename current_product to mixed_builds_product so that it's clear
that that this platform should only be used for mixed builds.
In addition, make the bazelrc files point to the named products again
instead of the mixed build product so that b builds will still have
qualified outputs, but mixed builds won't.
Test: Presubmit and kernel build tools abtd run
Change-Id: I7f764cf42cd1323f4b495d1320931f59a076ac63
This reverts commit 2562585878.
Reason for revert: We now have aidl_library to remove this workaround to support specifying aidl headers
Change-Id: Idf03ebbea764f8c5b3c4fc1623f8f02aa321c8c1
This is a followup to aosp/2606989.
This flag is not necessary now that we're using one platform name
for all of mixed builds.
Also rename current_product to mixed_builds_product so that it's clear
that that this platform should only be used for mixed builds.
In addition, make the bazelrc files point to the named products again
instead of the mixed build product so that b builds will still have
qualified outputs, but mixed builds won't.
Test: Presubmit
Change-Id: I5acbd18869589f67efb0ad3ebfda2c4d5a6a3850
This change adds bp2build converter for platform_compat_config module
type and its corresponding test case.
Bug: 269202028
Test: bp2build unit tests
Change-Id: I17d560c8f0e725efe69e151685d003c3dcb11f2f
To support stem, cc libraries will be generated in
<pacakge_name>/<label.name>. Update RUNPATH of java binaries so that it
can find its jni deps
Test: bp2build
Test: TH
Bug: 240563612
Change-Id: Ia2f70f0424a8526ee35856225c1a8cd1067a8570
By default, the artifacts generated by cc_binary and cc_library in Soong
track the module name. But Soong supports overidding this using the stem
property. e.g.
```
cc_library {
name: "foo.1.2",
stem: "foo",
}
```
will generate foo.so
This CL adds this property to bp2build
Test: bp2build unit tests
Bug: 240563612
Change-Id: I1dfed870d5bad450511b72c397d2355c01fa3b60
To support export of NDK APIs in multi-tree, we added functionality to
generate Bazel targets of ndk related Soong module types. Since this use
case does not exist anymore, deprecate this to prevent bitrot
The removed code includes
- `ConvertWithApi2Build` implementation of these libraries, which
generates the Bazel targets in the synthetic `api_bp2build` workspace
- (api) bp2build unit tests for these module types
Test: go build ./cc
Bug: 284029211
Change-Id: Id3278fa119e0ab87f31f39a3783197a81b655e43
This enables output paths of product agnostic targets (e.g. mainline
apex) to be identical when switching between target products, since the
product name is no longer embedded in:
1) the output path prefix in bazel-out
2) the PlatformOptions contribution to the configuration hash as part of a transition
Test: treehugger
Change-Id: I3fd0f7134209aa7eede3ed98412a1c5482689b57
panic in top down mutator "bp2build_conversion_bp2build"
for module "foo".
reflect: call of reflect.Value.NumField on interface Value
Test: go test ./bp2build
Change-Id: I9ee460ca38a37c6a6c1a0787159fa5f38b852b55
host_ldlibs are used during linking. Set these in `linkopts` for now.
Note that this CL does not do the `CheckBadHostLdLibs` validation of
Soong. There are some different ways to do this, and these are being
discussed in b/216626461. It is likely that we will need to create a new
property `host_ldlibs` to do the validation. But for now, re-use
`linkopts`.
Bug: 216626461
Test: bp2build unit tests
Change-Id: Id4c77e4460fb1fb003fa58ea27bab5b50ea8cefe
https://source.android.com/docs/core/architecture/aidl/aidl-backends#custom-parcelables
Some aidl files have custom parcelables with references to cpp headers. Hence, we need to pass the include dirs from cc libs to cc_aidl_library so that custom cpp headers are findable by the generated cpp/h files.
Test: go test
Bug: 278704136
Change-Id: I4ede1d23fbc04bdbfb69823d955c6df6b0a7b4ee
Introduce aidl.libs prop on cc libraries to pass in aidl_library. The goal is to eventually disallow aidl.include_dirs (a pattern for passing aidl headers dir for aidl compilation) and enforce aidl headers to be explicitly specified in Android.bp.
Bug: 278704136
Test: go test
Change-Id: Ia78bc11dfa12f47d2d1bb90dc65372ddb17f7e14
We now rely on a `with_features` clause to effectively disable
thumb when `arm_isa_arm` is enabled. Thus, we no longer need to
add `-arm_isa_thumb` in bp2build.
Bug: 261733820
Test: Unit tests
Change-Id: I9da85834855b297a75097ce3a11e101f55bd53f4
For cc_libraries containing .y/.yy srcs, we will generate an additional
cc_yacc_static_library target. This target will produce an .a file that
will be statically linked to the parent target.
Test: go test ./bp2build
Bug: 281546029
Change-Id: I094ba56f0a95869e5bc1e1d38b83c777192b7ddd
* changes:
Special case platform variant of bootstrap libs
Select stub/impl per apex variant
For test apexes, base_apex_name is the api domain
Broaden the granularity of config_setting from apex_name to api_domain
Print default val if all vals in axis match default val
1) Use attribute name "default_package_metadata" instead of
"default_applicable_licenses" in packages to better describe its use.
2) Add a filegroup "default_metadata_file" in packages to search for
METADATA file in each package.
3) Include "default_metadata_file" in each package's
"default_package_metadata" attribute.
Bug: 275472038
Test: CIs
Change-Id: I645c013c39e3190fd96c4a549d39a331aced16bd
For converting the art plugins to pure soong, it would be useful to
have a property that's qualified on both a soong config variable and
the OS. Soong had very little-known support for this by saying your
soong config variable changes the "target.android.cflags" property,
and we didn't supporting bp2building that. Add the bp2build support.
This cl also refactors product variable and soong variable bp2building
so that they're separate from each other, which I think makes the code
easier to understand.
Test: go test
Change-Id: Ic74dc75da8103fa2523da95c3560c9ce3c5e5672
Create a select statement for every api_domain a library could be
included in. The stub/impl
selection heuristics per apex domain are
1. If dep has stubs and the api domain appears in dep's apex_available, use
impl
2. If dep has stubs and the api domain does not appear in dep's
apex_available, use stubs
(Category 3: If dep does not have stubs and the apex does not appear in
dep's apex_available, then a separate apex_available validation in Bazel
will emit an error).
Platform variants have been special-cased for now to use equality of
apex_available for stub/impl selection
Test: go test ./bp2build
Bug: 272378496
Change-Id: Ibd29efd763c8863c7e6d2a9af0da30bbde07175d
apex_test do not "override" the source apexes that they test. However,
similar to override_apexes, test apexes have the same path
(/apex/<apex_name>) on device. Instead of creating another property,
reuse the existing base_apex_name property. The use case for this will
be for creating selection statements for stub/impl selection.
Test: go test ./bp2build
Change-Id: I4b7548e0e0fc920e407e1c5e5c00e87efc6e70c9
The use case for this is for creating a select statement for stub/impl
selection. Since Bazel propagates apex_name from the top-level apex,
the source apex and test apex builds a specific library in two different
configurations. We need select statements for both these two
configurations, but this metadata might not always exist in Android.bp
since test apexes are not necessary to be listed in Android.bp files.
To overcome this, the select statements will be created per api domain
instead. This CL uses a naming convention to infer the api domain of
apexes.
Test: go test ./bp2build
Change-Id: Iad2b45f736bc98a24d2ce1cf2f69aad67973092d
Although paths to resource files in a Bazel java_library should be
relative to the package, the directory structure in the resulting jar
will have resources under the full path from the top-level of the
workspace, e.g. if a library in "a/BUILD" has java_resouces as
"res/res.txt" then by default the res.txt file would appear under
"a/res/res.txt".
Fix this by adding a resource_strip_prefix in that case.
Test: Unit tests
Change-Id: If4325126f5c19a2a8fb83ee09bc3a95a18673fe3
To avoid verbosity, we currently dedupe keys in axis if its value
matches the value of //conditions:default. For axes where all values
might match the default value, we would effectively drop the common
value.
To fix this, we are now dropping the select statement and not the common
value.
Test: go test ./bp2build
Change-Id: Ic377b93ee2aba971753f6a5e7a62e15d1fcfa2bc
Instead of exporting it to soong_injection.
Bug: 279095899
Test: m nothing
Change-Id: I7b93af233b7450848a475512b5f5682ece773c09
Merged-In: I7b93af233b7450848a475512b5f5682ece773c09
These are created by bp2build in /build/bazel/rules/apex. Eventually
these config_settings should likely be colocated with the source apex
definition.
Another alternative was to make Bazel's apex a macro that generates a
config_setting. I did not pursue this further for now since it requires the
apex_available of every allowlisted cc_library to also be allowlisted.
This might not always be true (e.g. com.android.runtime)
Test: go test ./bp2build
Change-Id: Ibbb14b0d9c1491b3c79b7634a18d9d35b03922c1
The use case for this is creating config_setting(s) specific to an apex
variant and selecting stub/impl in that config_setting. We likely need
only a handful of such config_setting(s), but determining that list
requires iterating the build graph.
Test: go test ./bp2build
Change-Id: I9aa552e3d0bcf67513023c3a7d4bbf8fae464ee4
There are a number of instances where we are exporting information
from soong to bazel via soong_injection. This could be more bazel-centric
if the information was instead held in bzl files, and both bazel and
soong read it from there.
Add a starlark package that will run
//build/bazel/constants_exported_to_soong.bzl at initialization time,
and then results can be retreived with GetStarlarkValue.
Since changes to the starlark files mean that soong has to rerun,
add them as ninja deps.
Unfortunately, the starlark code has to be run at runtime rather than
pregenerating their results, because tests run from intellij wouldn't
go through any pregeneration steps. This means that starlark is run
multiple times during the build, once per test package and once per
primary builder invocation. (currently 3, could be reduced to 2 if we
made the symlink forest generation into its own standalone tool) The
starlark code we have so far in this cl is very fast, roughly half a
millisecond, so it's not a big deal for now, but something to keep an
eye on as we add more starlark constants.
Bug: 279095899
Test: go test
Change-Id: I1e7ca1df1d8d67333cbfc46e8396e229820e4476
This reverts commit 1db4348734.
Changes from original:
- extracted function to add the current version to stub
versions for bp2build.
- added libc++ to mixed builds denylist
Reason for revert: re-uploading with fix in topic
Change-Id: Ifa0ed456bf8cb4a7f861d6826263adfedb4fdd9c
Previously we were treating it as a shared library only, but
cc_test_library produces both a static and shared library.
Test: bp2build tests
Change-Id: I293d4246d581212b95765fa8f084301514d00dbf
in bp2build. The context here is that `-fvisibility=default` should
only be added for CFI if `-fvisibility=hidden` is not already
specified. This will be achieved using toolchain features. Note
that Soong itself never adds `-fvisibility=hidden`. This is only
ever added in the `cflags` property of a bp file.
Bug: 261733820
Test: Unit tests
Change-Id: Ib821e8c30a9cd03d2929b4bd2e771bec7b33fa66
This reverts commit ca438e6b72.
Reason for revert: ci post-submit failures in aosp-master-bazel of targets mixed-droid-clean and mixed-droid-incremental:
FAILED: ninja: 'out/target/product/generic_arm64/obj/SHARED_LIBRARIES/libc_intermediates/libc.so.toc', needed by 'out/target/product/generic_arm64/obj/EXECUTABLES/updater_intermediates/LINKED/updater', missing and no known rule to make it
12:21:27 ninja failed with: exit status 1
Change-Id: I081b499d23f2568cdf6227c4e3b0278164086b69
In a follow-up CL, we should eventually disallow aidl.include_dirs in cc rules so that the deps are always explicit and compatible with Bazel migration.
Test: go test
Bug: 278059962
Change-Id: Ia786cc8634d03589dc008f10b01e6bb2b9f2c7f0
Bug: 270408757
Test: go test
Test: m NeuralNetworksTest_shared_partial --bazel-mode-staging
&& verify that Ninja commands link libneuralnetworks via stubs
Change-Id: I19bf5a829cea7fd00c0f82511ad2bddbc40fae4f
There was a request for using b with python tests. bp2build python
tests exactly the same way as python binaries so that they can be
used with `b`.
Bug: None
Test: go test
Change-Id: Id68a6a73572745a4885b3e5bb1b8452e36baa982
This CL adds java_version and sdk_version support to bp2build
converters for
- java library
- java binary
- android library
- android binary
- android library import
Although java import doesn't support java_version and sdk_version, the
neverlink java_library wrapper around a java_import must specify a
sdk_version when targetting a device. "none" is used by convention.
Change-Id: I22a69dea2e351858368df69ed6a703b568d613ea
Bug: 215230098
Test: Presubmits
We don't want rules to load the product variables directly, because
then they won't be able to transition on them.
Break constant information that is safe to load outside of the
product vars file, so that we can make the product vars file have
more restricted visibility later.
Bug: 269577299
Test: m nothing
Change-Id: I848bff33e4f5798f51296ea3a2600615cab36985
There was an incrementality bug in the symlink forest generation that
was recently fixed. However, if the appropriate files don't get touched,
the issue will remain. Add a mechanism to clean old symlink forests
so that when we fix incrementality bugs we can ensure they don't remain
in any old out directories.
Bug: 276349152
Test: Presubmits
Change-Id: I36664dfb0ca7227e3e1f89de859ebccb808c5f15
It was possible for the merged content to end up back in the bp2build
generated file because there was a symlink from the symlink forest to
the bp2build generated file.
Remove the symlink if it exists.
Bug: 276349152
Test: m bp2build, add a handcrafted file in the same folder as a Android.bp file, m bp2build again, check that the symlink forest version is not a symlink
Change-Id: Id64aa3addebcf0c6b1728389f21ae246796aaf8d
See the other cl in this topic for more information.
Bug: 269577299
Test: b test --config=android //build/bazel/rules/apex:all
Change-Id: I1f20bc5f85398cd79dc5b7fe9d0bd4d93d442383
These must be added in the soong_injection code as opposed to just
defined loosely in checked-in bzl files because the product_vars
select statement must be updated to support the new platforms.
Bug: 269577299
Test: b test --config=android //build/bazel/...
Change-Id: I7bba9af214896dd3b5938bae70b7c0cea4f75e41
Note this doesn't entirely match Soong's logic but is an improvement to
allow linking against implementation when two cc modules are
apex_available to the same module.
It is not possible to recreate the logic for "directly in" without
significant changes to bp2build as we do not add dependencies nor run
apex mutators. Rather than trying to replicate this, we would be better
off refactoring Soong to no longer support the "directly in apex" logic
and require users to correctly specify apex_available.
Bug: 272378496
Test: go test conversion tests
Change-Id: I17ac426f9b4bdad0c2ab661484e5d994f63568ce
The static deps of the parent library can be necessary to build a
bp2build-generated cc_aidl_library target. We should add these deps as
implementation_deps so that they are accessible.
Bug: 250876486
Test: b build //frameworks/native/libs/gui/...
Change-Id: Ibe7c3598a684907473e2a4e040fb3976455a59e9
BUILD files of rdeps should depend on stubs via @api_surfaces
indirection. e.g. instead of depending on
//system/logging/liblog:liblog_stub_libs_current, it should depend on
@api_surfaces//module-libapi/current:liblog. This ensures that the
generated BUILD files are compatible with Multi-tree.
Update the unit tests for this change.
Test: TH
Change-Id: Ibcc36dcfbee7b1973b341485f015e67987564dcc
Since the source_library attribute of _cc_stub_library_shared was a
label attribute, the source library was added as a dependency of the
stub which doesn't always make sense. E.g. it caused validation actions
for the source library to run even when we weren't building the source
library.
This change converts the label attriubte to a string attribute so that
we don't add the dependency.
Bug: 263390551
Change-Id: I19c62d1e14847dff68ab37156452addaa119eaf9
Bazel boolean attributes can only support true or false states, but the
clang-tidy logic changes for modules which don't set a specific value.
Therefore, the tidy attribute should really be a tri-state of "true",
"false", or "" (unset).
Test: b test //build/bazel/rules/...
Bug: 268681923
Change-Id: I85fbaa612d175f475757db806cea6c018901fe3e
for java_library, java_binary, android_app & android_library
Change-Id: Ia8dec7e7d497eb41b2d8dcd8c44c3090d1d3b4bd
Bug: 271265771
Test: go test ./bp2build
The expected use case for this is to create aliases for stub libraries
in the @api_surfaces repository in build/bazel/api_surfaces.
This restricts the scope to just aliases. If we have a use case for
generating actual Bazel targets in another directory, a workaround could
be to generate the targets in the current directory (via
CreateBazelTargetModule) and aliases to it in the other directory
Test: go test ./bp2build
Change-Id: I6b63d9d018618d447fc7c260a2a94aaa00e57a4d
override_apex's bp2build converter had a bug where it was looking at
the product variables for the override_apex module itself instead of
for the base module it is overriding.
Fixes: 271424349
Test: go test
Change-Id: If1e2653d3751fa908faf0ab97dfa2e943ebe98ec
This is to match the behavior of BUILD file generation.
Test: m and all existing bp2build conversion tests
Bug: 271122205
Change-Id: I6320a65aaad06114817eaed722cc0a0939c57bc1
there is now support for resoure_strip_prefix in kt_jvm_library targets.
Test: built AnalyzerKt and updated go ./bp2build tests
Change-Id: I4a6fe45276d45519186b6f40a02db990511d6def
Add Alwayslink to bazelPrebuiltLibraryStaticAttributes
Also adjust the corresponding tests to espect these new targets.
Test: m libc
Test: b build --config=android //prebuilts/clang/host/linux-x86:libunwind-exported_alwayslink
Test: bp2build.sh
Bug: 267214124
Change-Id: I4d996aef55321f1827947f00b0eaa52d0a14413e
With this CL, we avoid deleting the bp2build directory and regenerating
all BUILD files. Instead, we regenerate BUILD files which have changes
on the filesystem, and delete old BUILD files which should no longer
exist.
This improves incremental Bazel server performance by about ~5 seconds.
Previously, Bazel would have cache misses for regenerated packages, and
would thus need to analyze them even if no meaningful changes took
place.
For ease of implementation, we avoid removing stale (empty) directories
from the bp2build workspace, but this should have no effect on the
build. (These will be removed on next clean)
Test: New integration tests
Test: Manual benchmarking in conjunction with persistent bazel server
Change-Id: I3e489ff403be34040122876012329060a2506366
Given that we can map them directly to disabling the
corresponding Bazel features: `-{link_,use_lib}crt`
Test: Existing (adapted) Unit Tests
Test: bp2build.sh
Change-Id: Ib502f6fb929ace8e86a1001e3cc21f399317500c
Only public, system, test, module_lib and system_server are
converted in order to generate api_fingerprint.txt in Bazel.
Test: java_sdk_library_conversion_test.go and TH
Bug: 266973526
Change-Id: I67a00806165e5afad3876b6cd5cdbc6b0dd65d8b
This changes bp2build codegen, symlink forest generation, and
soong_build so that they do not rewrite output files if the contents are
unchanged.
Bug: 266983462
Test: m droid
Test: canonical_perf.sh benchmarking
Test: Manually verified that rerunning analysis did not regenerate
out/soong/workspace/prebuilts/sdk/BUILD.bazel unless contents changed
Change-Id: I5ec227df7a32b53c7fa0d741fb1403a51931024b
So that it can be used in the bazel implementation.
Bug: 268006095
Test: m nothing, check that it's in out/soong/soong_injection
Change-Id: I1520dd874076dee7fa083648d0790b060d658e5a
Previously, symlink forest generation involved removing the entire
symlink forest and recreating it from scratch. With this change,
a) symlinks which need not change are untouched,
b) symlinks pointing to the wrong location are fixed, and
c) symlinks which should no longer exist are removed.
On AOSP on my local machine, this reduces the symlink forest generation
step from 2.5s to 1.1s clean, and 0.6s when a single file is added to
a source directory.
Bug: 257528847
Test: m bp2build, touch `fakefile` under the forest, remove a file
from the source tree, rerun m bp2build. Manually verify the new forest
does not retain the link to the deleted source file, and that fakefile
no longer exists in the forest.
Change-Id: I481371ae487e9419af6a3a4370c552578b07d650
Previously, Bazel was invoked during mixed builds with a stripped down
environment. This CL adds an allowlist of environment variables that are
passed to Bazel.
Test: WITH_TIDY=1 DISABLE_ARTIFACT_PATH_REQUIREMENTS=true mss tidy-packages-modules-NeuralNetworks --bazel-mode-dev
Change-Id: I23147bec59f6522953cf623e7bcaa0f1f99a75a3
Test: bp2build tests and locally built MusicKotlin
(cannot allowlist yet due to soong/bazel custom_package differences)
Change-Id: I1cbd0fe2af0e6eeed65cf8d57dc85c1f54646c50
This fixes two main sources of nondeterminism:
1. Fix a bug in the ConfigurationAxis comparator (which caused
ConfigurationAxis sorting to be nondeterministic)
2. Process C++ dependencies using the sorted ConfigurationAxis order. In
theory, the order in which dependencies are processed shouldn't
matter (as they should end up in different select stanzas). However,
in the case of InApex stubs, this is not the case; we now ensure
that lists are concatenated in a predictable order.
Added bonus: Some cleanup with SortConfigurationAxes which
makes use of go generics (this made it easier to debug this issue).
Will follow-up with regression tests.
Test: Manually verified that build.ninja checksum and BUILD.bazel checksums do not change after running `m nothing` 6 times in AOSP (with comment-only Android.bp changes in between each run)
Change-Id: I81168e45bdbbcd61ea95ff665cf6c4bc180aa4e0
