This is the bulk of the "allowlist v2" feature. It will disable bp2build
generation for modules which have transitive dependencies without a
bazel build definition.
This CL includes this mutator, but doesn't register it as a bp2build
mutator (outside of a few unit tests). This allows us to easily iterate
on completion of this feature and ensure there are no launch blockers
before we finalize the change in AOSP.
Bug: 285631638
Test: Unit tests
Change-Id: Ifb0a079c409ca19b02cafa3fab2efa0d3deebc50
- Allowlist an aidl module by package, not name (to support a small
module name change)
- Implement some unit test framework changes which facilitate better
aidl bp2build testing
- Support a convenience function to add a load hook for registering a
module as "has a bazel definition of a given target name"
Bug: 301676937
Test: m bp2build, verified the aidl target was generated before/after
this CL.
Test: Presubmits
Test: Ran bp2build progress and ensured that aidl_interface targets
under frameworks/ continued to appear converted
Change-Id: I62412057d6f61a2ce2bc39488c75af793eb14c94
This should be no-op, as the underlying mutator has not changed yet.
Some other refactoring is required and done in this CL:
- Delete some old, dead ApiBp2build code
- Fix casting to TopDownMutator when it's not necessary
This change is required to prepare for allowlist v2 work, as only
BottomUp mutators can AddDependency.
Bug: 285631638
Test: m nothing
Test: presubmits
Change-Id: I5212a5f5634cc13056195783e6df37ff8eb000da
This no-op refactoring facilitates some upcoming functional changes for
"bp2build allowlist v2". The work requires that the bp2build conversion
mutator be changed from a TopDown mutator to a BottomUp mutator.
Refactoring all bp2build-related methods so that they use Bp2buildMutatorContext
makes it easier to make this functional change without touching tens of
files and multiple projects.
Bug: 285631638
Test: m bp2build
Change-Id: I3d1ef3064146e959c6f0dc315350fc9764bf2bd2
Instead, we return an error. This allows us to access some product
variable information earlier when it will not be used as an attribute
without panicing
Test: m nothing
Change-Id: Id094b2b9e1364a8d174d99b3824fa149fb235b3e
Allowlist v2 will change bp2build by automatically disabling rdeps of
unconvertible modules. Many bp2build tests create bp2build stub modules
without an implementation,
This CL changes setup of such tests to also contain a BUILD file with
"stub implementations" of equivalent targets, to keep these test targets
convertible.
To verify this change in-place, this CL removes `bp2build_available:
false` from these dependencies.
This is a test-only change for bp2build tests.
Bug: 285631638
Test: m bp2build
Change-Id: I489480cbc4158a416b7abf57c35a6e2bc2ad6173
This involves some minor changes to testing infrastructure.
This is a rollforward of aosp/2628496 with a couple of minor changes:
- In ParseBuildFiles, filter out all build files that are kept due
to ShouldKeepExistingBuildFileForDir
- Add some minor test infrastructure for StubbedBuildDefinitions, with a
couple of proof of concept tests used to demonstrate its usage. This
pattern will become immensely more common as we implement allowlist v2
(as we will need to update all tests which today simulate build
definitions that have missing deps)
Bug: 285631638
Fixes: 286545783
Test: bp2build.sh
Test: m nothing
Change-Id: I7c3a03b02098e39dd8e51d327482b440f294478f
cc_stub_suite now has an additional attribute correponding to the api
surface. This attribute will be used to pass additional args to the stub
generator.
Update bp2build to set the api_surface of the apex stubs. This ensures
that its stubs are generated with "--systemapi --apex"
Test: unit tests
Bug: 298085502
Change-Id: If4c479f85e6e485b5c795a565a0c559f1f013bf5
This feature is obsolete.
This makes a large number of codepaths "dead code" (such as
module-specific implementations of ApiBp2build functionality). These
will be deleted in a followup CL.
Bug: 284029211
Test: Presubmits
Change-Id: Ib53b99f1fe8c24380d219caf44e9bb3b96724fa0
When sdk_version is unset, the build with fail with Bazel because
private APIs are do not build with Bazel at this time.
This instead prevents migrating the relevant target to Bazel in bp2build
Test: go tests
Change-Id: I31dabcf143de7933706d1b734ef21467a01b65e9
The current API restricts creation of targets to the directory of the
visited soong module. This CL proposes adding a `Dir` property in
`CommonAttributes` that can be used to create a bazel target in
a specific dir. The use case for this is to dynamically create
additional targets for proto_library that are adjacent to .proto files
(Bazel poses a strict requirement about proto_library being in the
same package as the .proto file, but Soong does not)
Usage is restricted to dirs that have an existing Android.bp file. There
are some places in bp2build where we use existence of Android.bp/BUILD
on filesystem to curate a compatible fully qualified path (e.g. headers).
If we use `CommonAttributes.Dir` to arbritraily create BUILD
files, then it might render those curated labels incompatible.
Test: go test ./bp2build
Change-Id: If9446700457eddfb389be9d9bde39087f67daa60
This involves some minor changes to testing infrastructure.
Bug: 285631638
Fixes: 286545783
Test: unit test
Change-Id: If64ba29308d99e63a1cc5526feaf077f2cb14478
This also changes the expectation of ConvertWithBp2build. Each
implementation must either create one or more Bazel target modules, or
mark the module as unconvertible (with a specific reason).
Manually verified no runtime hit in AOSP
In AOSP, the metrics file size increases from 252K to 1.6M
This changes some effective module counts in bp2build metrics:
- Removes "package" modules from the module count list in
metrics, as these will not be converted like regular modules.
- Counts Handcrafted modules as being "unconverted", as bp2build is not
responsible for them.
Bug: 285631638
Test: Verified generated BUILD.bazel files are bit-for-bit identical
with this change
Test: Manually verified one case of each implemented reasonType
Change-Id: I308dd451d8f28379b15671dae9f931bd0446f5c1
The use case for this is creating config_setting(s) specific to an apex
variant and selecting stub/impl in that config_setting. We likely need
only a handful of such config_setting(s), but determining that list
requires iterating the build graph.
Test: go test ./bp2build
Change-Id: I9aa552e3d0bcf67513023c3a7d4bbf8fae464ee4
This CL adds java_version and sdk_version support to bp2build
converters for
- java library
- java binary
- android library
- android binary
- android library import
Although java import doesn't support java_version and sdk_version, the
neverlink java_library wrapper around a java_import must specify a
sdk_version when targetting a device. "none" is used by convention.
Change-Id: I22a69dea2e351858368df69ed6a703b568d613ea
Bug: 215230098
Test: Presubmits
Since the source_library attribute of _cc_stub_library_shared was a
label attribute, the source library was added as a dependency of the
stub which doesn't always make sense. E.g. it caused validation actions
for the source library to run even when we weren't building the source
library.
This change converts the label attriubte to a string attribute so that
we don't add the dependency.
Bug: 263390551
Change-Id: I19c62d1e14847dff68ab37156452addaa119eaf9
override_apex's bp2build converter had a bug where it was looking at
the product variables for the override_apex module itself instead of
for the base module it is overriding.
Fixes: 271424349
Test: go test
Change-Id: If1e2653d3751fa908faf0ab97dfa2e943ebe98ec
This is to match the behavior of BUILD file generation.
Test: m and all existing bp2build conversion tests
Bug: 271122205
Change-Id: I6320a65aaad06114817eaed722cc0a0939c57bc1
Previously if there was a value set outside of product variables and one
inside a conditions default soong config var, we would ignore the
conditions default. Now, we maintain the conditions default and use
the root value in cases where there is not a value for a product
variable.
Test: go test soong tests
Change-Id: Ic991e3aebe5bb6039353f4e3d25625e7c5190f96
Create a
build/bazel/product_config/generated/products/<product_name>/BUILD
file that contains the platform definitions needed for
a particular product. Currently we just create it for the
current lunch target, but the idea is that eventually when
all product config is in starlark, all the products will
have their platform definitions in the tree at once.
Bug: 249685973
Test: Presubmits
Change-Id: I08c82ff28dcf62f09d3b1d2e3186a6b961e12f6e
Previously, the fixture test infrastructure was hard coded to
initialize itself in preparation for invoking the PrepareBuildActions()
method. That meant it could not be used for testing the bp2build and
apiBp2build modes which required different initialization and called a
different method.
This change extracts that behavior into a FixtureTestRunner and adds an
implementation that allows it to test the above two modes. It then uses
that to implement the runBp2BuildTestCaseWithSetup method.
The TestPrebuiltLibraryAdditionalAttrs was the only test which broke as
it supplied an invalid path to the MockFS (it does not support using a
trailing / to represent an empty build directory). So, it was modified
to add a couple of files inside those directories instead.
Test: m nothing
Change-Id: I6798a4f761160af8d1bfed81d46de9628bda3eb9
The axis for OS and in APEX was previously ignoring the excludes from
OS, which would result in excludes for non-Android OSes being
incorrectly ignored.
Test: go test Soong tests
Test: b build //hardware/libhardware:all
Bug: 260131489
Change-Id: Ie77f763bea0f473ac69a0c6b6bc3216e7359ad12
Previously, it stored an android.Context (no pointer) which while it
worked (because the current contents are themselves pointers) it is
inconsistent with how the rest of the code references it and could
cause weird behavior if additional fields were added to the Context.
Test: m nothing
m bp2build
Change-Id: I9c70f925dde85e5b15431cc232b0c2eb6371ec6d
The generated Bazel workspace will only contain api specific targets.
This is feasible since these targets do not have any cross dependencies
with the targets in the bp2build workspace
The advantages of a new mode are
1. Does not pollute bp2build workspace with api targets
2. Does not block api targets with the current allowlist conversion
mechansims in bp2build
(In the future we might want to combine these two workspaces)
A Soong module type will generate a Bazel target if it implements
ApiProvider interface
Test: m apigen
Test: m nothing
Change-Id: I69c57ca6539f932e0ad554ce84a87fb7936fdba0
For directories without an Android.bp file, if they contain a merged
checked in BUILD file, it becomes a package boundary in the symlink
forest. However, the current glob expansion is only aware of Android.bp
files, but not merged BUILD files, so it generates glob expansions
incorrectly for files that cross the package boundary.
This CL fixes that by making the package boundary function aware of the
keepExistingBuildFile allowlist. See the new test cases for example.
Also stop using the same global bp2buildConfig for every test case to
allow each test to define their own bp2build allowlists.
Bug: 185358476
Test: new unit tests
Change-Id: Ifcd1283480a09a642ef4343e5bbc599583524577
Returning errors throughout instead of calling panic()
Errors will be more useful for testing
Test: bp2build tests
Change-Id: I3b03f0a30e7a80878e91c7f0e2df5a94d9d6b780
This is to support the migration of conversion tests to be next
to the files they're testing
Bug: 242070498
Test: m bp2build
Change-Id: I66243d420743273e7ebad70886cfe690740182c4
Since we are putting conversion tests in the same
package as the module being converted from now on
Test: m bp2build
Change-Id: Ied53589d497fef70bfa11d8f6067459c9a50cd72
Instead of a StringListAttribute of length 1,
introduce a more appropriately reduced StringAttribute
so that e.g. `select`s work properly
Test: new cases in TestGenerateBazelTargetModules
Change-Id: I0ae0e4a51e39f85caf55b0d00459222ede6de79c
bp2buildTestCase
attrNameToString
runBp2BuildTestCase
makeBazelTargetNoRestrictions
The testing framework defined in the bp2build package can only be used
from within the package because many common testing functions are
private to the package. This prevents modules defined in Soong
plugins (e.g. system/tools/aidl/build) from testing bp2build conversions.
Test: go test ./bp2build
Change-Id: Ia867081327c5181d04687b13c4550e68e6a11f86
By default, most module types are only enabled for device. Converting
this behavior and host_supported/device_supported properties allows us
to skip building incompatible targets.
Test: soong tests
Test: bp2build.sh
Change-Id: If1da523b4cc8c4cbf2bb26da063d9923b662cc32
This is to avoid having it hardcoded in a fork of the
py_library rule.
Most import attributes should just be set to ".", but
our previous solution always hardcoded it to ".." instead,
for ndkstubgen. ndkstubgen uses pkg_path: "ndkstubgen",
i.e., it set pkg_path to the name of the folder that
contained the Android.bp file. In this specific scenario,
imports = ".." works. Recreate that behavior here as well,
because we don't handle pkg_path properly yet.
Fixes: 233081071
Test: build/bazel/ci/bp2build.sh
Change-Id: Ib5e6a8edf428c74d4b5947f0ec53a2151001367a
while cc_prebuilt_library_(shared|static) were already supported,
the raw cc_prebuilt_library module type was not supported by
bp2build. This CL provides that functionality.
Also did a little bit of refactoring and cleaning.
Bug: 225938765
Test: Confirm output BUILD.bazel files look as expected. Confirm
that build succeeds.
Change-Id: Ic004f88febc0e5b527f2ef420dbd96ecc55d2e09
To make testing easier, refactor existing module-global variables into a
struct that can be mocked.
Test: build/bazel/bp2build.go
Change-Id: I9d177677644ea743641a745b1839a3a8b29f902a
If a system shared lib is specified in shared_libs, this results in
duplicate values appearing, causing a failure in Bazel. This change
removes any system shared libraries that appear in shared libraries from
bionic OS axes where system_shared_libraries takes the default value.
Test: go soong tests
Test: temporarily allowlist directory with this issue, no longer hits
duplicate library failure.
Change-Id: I9dce570b73c24973f695b815bce8d50f7259798d
Previously we looked for proto as suffix, however, some filegroups use
"<foo>-proto-srcs" or "<bar>-proto-sources", instead we look for proto
as a distinct word in a filegroup name.
Test: go test soong tests
Change-Id: Icf916a84304a02617efff9768e5b82d5ffe658bd
Each conversion required defining a separate mutator, which will each
operate on _all_ modules and requires each to repeat checks whether the
mutator should operator. Instead, we introduce a single mutator and
modules can define a ConvertWithBp2build to implement bp2build
conversion for that module.
Test: bp2build.sh
Bug: 183079158
Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
