The logic is not 100% provably the same since HEAD was quite
confusing at some points, but I did make an effort to preserve
functional equivalence.
In case that effort was not enough, it should be pretty easy to
tweak the logic at HEAD since it's still quite malleable.
Bug: 231370928
Test: Presubmits.
Change-Id: I17b2efbfb5c4d0aedd922caed54ff8d857e578df
Previously we ran mutators in bp2build mode to add dependencies, now we
look up modules by name directly. Remove workarounds to allow bp2build
mode to not fail when adding/handling dependencies.
Test: m bp2build
Change-Id: Ibf6fd905150cac306e5c395902ef28f609f4df2a
These contained duplicate code, but evolved separately in the effort to
identify module type of those created with CreateModule. This refactors
to share a common implementation between the two.
Test: m json-module-graph and verify Type fields
Test: CI
Change-Id: Ifdb9a006d9b1bef7411f9ce3a4384797693b4bfc
Also fix some bugs pertaining to configurable attribute handling of bool
attributes and label sttributes, so that they may support values across
multiple different axes at the same time.
Test: unit tests for bp2build
Test: mixed_droid
Change-Id: I411efcfddf02d55dbc0775962068a11348a8bb2c
Each conversion required defining a separate mutator, which will each
operate on _all_ modules and requires each to repeat checks whether the
mutator should operator. Instead, we introduce a single mutator and
modules can define a ConvertWithBp2build to implement bp2build
conversion for that module.
Test: bp2build.sh
Bug: 183079158
Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule
Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.
As initial user, the Soong `required` property will be
translated to Bazel's `data`.
Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
Support three options for converting modules with unconverted
dependencies
1. (default) Warn when converting a module if it has unconverted deps.
2. Error when encountering a module with unconverted deps. (not hooked
up yet)
Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Test: BP2BUILD_ERROR_UNCONVERTED=1 build/bazel/ci/bp2build.sh with
unconverted deps -- get appropriate error
Bug: 181155349
Change-Id: Ifaabf0cd2e43e963366dc137159c705294165c3d
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3
We avoid both of these by storing the information directly in the
underlying module.
Also as a fringe benefit, removes some necessary boilerplate for
conversion.
For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714
Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
Refactor bp2build to retrieve modules directly by name, instead of via
DirectDeps. This functions properly as bp2build has no need for variant
information of the blueprint graph.
Test: USE_BAZEL_ANALYSIS=1 m fmtlib
Change-Id: Ief4b67bc56f24929871af772f3a742f07085bf8c
Adds PrepareForTestWithNeverallowRules to make it easy to test
neverallow rules. Avoid exporting any unnecessary neverallow related
methods from the android package.
Bug: 181070625
Test: m nothing
Change-Id: Idfc6955cb23f1a4d1790be7879388154b03f3980
cc_object crtbrand sets product_variable.platform_sdk_version.asflag
and will not compile correctly within mixed builds without it.
Only handles product_variables that expand product variables.
Bug: 181794963
Test: ~/aosp/build/bazel/scripts/milestone-2/demo.sh full
Change-Id: I293fcb18032aa51f63bb7b3de94abd6d1ec38180
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.
It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.
Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.
Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
This separates the collation of mutators from the registration of them
to allow the test infrastructure to sort the mutator order to match
that used at runtime.
Bug: 181953909
Test: m nothing
Change-Id: I01a073289d44417f327b0815c09eb1c033d464f2
In preparation for following changes that will ensure the order of
registration (and so execution) for mutators, singletons and
pre-singletons in tests match the order in runtime this change creates
the sortableComponent interface to allow those to be sorted in the same
way without having to resort to reflection.
By moving the registration code into each component type this change
also eliminates some unnecessary duplication of that code.
Bug: 181953909
Test: m nothing
Change-Id: I597b461b966c84faaeb13e7dff765f1fadd99981
This eliminates the need to remove quotes, delete attributes, and
re-checking that name has correct prefix. Additionally, this allows
assignment directly to the BazelTargetModuleProperties struct, which
allows defaulting unused fields and clarity of which field is being set.
Test: go test soong tests
Test: ran ./build/bazel/scripts/milestone-2/demo.sh
Change-Id: Ia9bfcce76234c793a4ddd5f29a661150f83341c9
This CL replaces the "__bp2build__" name prefix boilerplate with a props
creation function, and centralizes the prefixing in there.
Test: TH
Test: soong tests
Change-Id: Ic963199ab60dcce0d3361abff111cfa9acd4c21b
This CL refactors the CreateBazelTargetModule API to minimize boilerplate, and to establish a well defined function signature for the expected metadata about a BazelTargetModule.
Test: soong tests
Test: TH
Change-Id: I474ff5a2b0db8deeed49ba4ca73b416ccb494fdd
This currently expands all globs, still need to support converting glob
syntax.
Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
This CL brings the defaults mutators into bp2build, reusing the
implementation of flattening defaults from *_defaults module deps into
the modules themselves.
This is an approach to converge defaults semantics with Bazel, which
doesn't have existing equivalent abstractions, other than macros, which
require more infrastructure to be built.
Test: TH
Test: build_conversion_test.go
Change-Id: Ia1443f8e6c3eb277fa861bb96625425a22cfc6d7
This CL creates the framework necessary for generating
BazelTargetModules from regular Soong Android modules.
BazelTargetModules are code-generated into Bazel targets in BUILD files.
See the follow-up CL for examples of creating filegroup/genrule
BazelTargetModules.
Test: GENERATE_BAZEL_FILES=true m nothing # creates out/soong/bp2build
with no BUILD files, because there are no BazelTargetModules in the
module graph.
Change-Id: I33a96365bd439043b13af6db9e439592e9983188
Revert submission 1377717-metalics
Reason for revert: This has broken renderscript_mac target for aosp-master, see b/176909442
Reverted Changes:
I26ac54ca9:Define the standard license_kind rules.
I656486070:Export soong license data to make.
If9d661dfc:Export soong license data to make.
I97943de53:Add ability to declare licenses in soong.
Icaff40171:Rough-in license metadata support to make.
Ib8e538bd0:Add variables for notice deps, license kinds etc.
Change-Id: Iddf5468e0175831ddb642011f2768f87a5e4fcf2
This CL adds a new CONVERT_TO_BAZEL env var, and a bp2build goal. It
reuses the queryview architecture, but registers no mutators before
converting to BUILD files. This gives us a blank slate of the module
graph to work with, and create a shadow/alternate pipeline of mutators
to converge the Bazel BUILD graph to be semantically equivalent with the
Android.bp graph (after apply its current set of mutators).
The command to do so is:
$ CONVERT_TO_BAZEL=true m bp2build
To not clobber with queryview, the generated files are in
out/soong/bp2build.
Test: CONVERT_TO_BAZEL=true m bp2build && bazel query --config=bp2build //...
Test: m queryview && bazel query --config=queryview //..
Test: soong tests
Test: TH presubmit
Fixes: 174465461
Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I4dd0ccc73abc345d70a50ca2803d6f400cd8c863
Propagate new return values of the BottomUpMutatorContext.Add*
methods from https://github.com/google/blueprint/pull/316 to
Soong.
Test: m checkbuild
Change-Id: I5378db46adf2f31cc4222b4619cc39db53791610
Copy documentation from Blueprint for interface methods that are
wrapped into Soong interfaces.
Test: m checkbuild
Change-Id: Ib58c9201b8e7dfbf900a252e709f0a3cfed73286
AddFarVariationDependencies was broken, which allowed modules to add
dependencies on bootstrap Go modules using os and arch variations
even though boostrap Go modules did not have any variations. Mutate
bootstrap Go modules in the os and arch mutators so that they have the
expected os and arch variations.
This relands Ida7bc75a51ab1d2d38a6be11f574399097380cc9 with fixes
to also mutate *bootstrap.goPackage modules so that they can find a
matching *bootstrap.goBinary module when creating their pluginFor
dependency, and to create a variation for linux_bionic when it is
enabled so that linux_bionic builds can package Go binaries.
Also requries https://github.com/google/blueprint/pull/320 to
support multiple variants of bootstrap Go modules.
Test: all soong tests
Change-Id: Ibc627aa262c298a076d171a4c21b0b9c2a7b0ada
Previously, when java_sdk_library_import was preferred over a
java_sdk_library the latter ends up depending on the prebuilt child
modules created by the java_sdk_library_import instead of the source
child modules that it created itself. That was because all dependencies
on those source child modules were replaced with the corresponding
prebuilt child modules.
This change prevents those dependencies from being replaced to preserve
the dependencies from java_sdk_library onto its source child modules by
making the replacement conditional depending on the tag used. It also
updates the affected test.
Bug: 159902351
Test: m nothing
Change-Id: I4441b901dedfd44b9769df1ac2e248b94834cf85
Previously, java_sdk_library_import added the dependencies on its child
components in the deps mutator after prebuilts without a matching
source module are renamed to the source module name. That meant that
the java_sdk_library_import has to use the source module name and ended
up depending on the source module unless it was preferred.
This change adds a new component deps mutator that runs before the
PrebuiltRenameMutator so that the java_sdk_library_import can add
dependencies onto the prebuilt modules. It also updates an affected
test.
Bug: 159902351
Test: m nothing
Change-Id: I3576c4873302743e51aff547ea1497bef6d748ac
Previously, the only way for a module type to create modules (other
than defining its own mutator) was to register a LoadHook. However,
that is called before defaults are applied so any properties used in
that hook cannot take advantage of defaults, e.g. java_sdk_library
cannot use defaults to set its sdk_version property and have that
affect its child modules.
This change adds a new SetDefaultableHook() to DefaultableModule to
register a hook that is called after any defaults have been applied.
Also adds some tests to ensure that errors in the visibility property
introduced in a DefaultableHook are reported in the gather phase of
visibility processing.
A follow up change will switch java_sdk_library to use that instead
of the AddLoadHook() mechanism.
Bug: 155295806
Test: m checkapi
Change-Id: I13df3115f9e225f7324b6725eaeb16a78cc2538a
A follow up change will add a mechanism for a module to register a hook
that is called after any defaults have been applied. That is intended
for use by modules like java_sdk_library that create child modules that
are dependent upon properties that could be supplied by defaults.
Creating those child modules after prebuilts mutators are run will
cause problems if those child modules clash with prebuilts modules.
Moving the prebuilts mutators after the defaults mutators will fix
that.
Tests are currently being run with the mutators in different orders so
this change also cleans that up so they are consistent with the actual
code that is being run.
Bug: 155295806
Test: m checkbuild
Change-Id: I825c6df09058fb3a45db196661959eb332aca2f3
This was not previously possible due to LoadHooks being run after the
module was registered.
Bug: 155462403
Test: m nothing
Change-Id: Ia8383b9d1272bb12c8a83948753a0e4b0d98a650
This exposes test func on ValueMatcher and helpers relevant for testing neverallows.
Bug: 153333044
Test: m
Change-Id: Ied07cd33afa537f9d7aa3c33e59f4ac985901a0f
This is for use to add dependencies between specific module variants,
without getting the dependencies potentially duplicated as more
variants are created.
Moved genrule tool dependency registration to this phase, to avoid
potential splitting of its dependencies, and for consistency with the
same kind of dependency on dex2oat in https://r.android.com/1205730.
Test: m nothing
Bug: 145934348
Change-Id: I1f9e75354e359cef5f41d14e5016b0a67d7655d7
Make android.AddLoadHook wrap blueprint.AddLoadHook. Also pass
the config object to ParseBlueprintsFiles.
Test: all soong tests
Change-Id: I60c988b717d395f52498ec23ef7c9046d9861a6e
