This feature is toggled on with USE_PERSISTENT_BAZEL, which is off by
default. Those that opt-in will have a bazel server running between
builds (with a 3hr default TTL) which will greatly improve analysis on
subsequent builds. (As Bazel maintains a cache of analysis results).
Bug: 266983462
Test: Manual `m nothing` runs (timing with and without the feature)
Test: New integration test
Test: Presubmits
Change-Id: I3af4948baa0c490e9b87c48ffdbe9f67732586c7
Creation of build statements is largely parallelizable because each
action is independent apart from updates/reads to
depsetHashToArtifactPathsCache. Locally resulted in build statements
taking ~.45 seconds on staging mode to ~.02 seconds
Test: CI
Change-Id: Iab00c8394a9eab17353f71230885ff0870e17f24
This prevents unnecessary copies of data from the query by using a
pointer to the proto or its contents
Test: go test soong tests
Test: m nothing
Change-Id: I5962b27fe31f7103d2af6cef159f11747cff3ed9
Previously in mixed builds, only the tidy files for the boundary module
were built, whereas all of its transitive dependencies' tidy files were
not being built. Instead we should export the list of transitive tidy
files for a module so that we can run clang-tidy for the boundary module
as well as its dependencies.
Bug: 195029134
Test: WITH_TIDY=1 DISABLE_ARTIFACT_PATH_REQUIREMENTS=true mss tidy-packages-modules-NeuralNetworks --bazel-mode-dev
Change-Id: I463646d2ae1fc4aa075a54c264e1c34571c3fd5c
This fixes two main sources of nondeterminism:
1. Fix a bug in the ConfigurationAxis comparator (which caused
ConfigurationAxis sorting to be nondeterministic)
2. Process C++ dependencies using the sorted ConfigurationAxis order. In
theory, the order in which dependencies are processed shouldn't
matter (as they should end up in different select stanzas). However,
in the case of InApex stubs, this is not the case; we now ensure
that lists are concatenated in a predictable order.
Added bonus: Some cleanup with SortConfigurationAxes which
makes use of go generics (this made it easier to debug this issue).
Will follow-up with regression tests.
Test: Manually verified that build.ninja checksum and BUILD.bazel checksums do not change after running `m nothing` 6 times in AOSP (with comment-only Android.bp changes in between each run)
Change-Id: I81168e45bdbbcd61ea95ff665cf6c4bc180aa4e0
Instead of using a sentinel file that was created under bazel_workspace
prune them out. The sentinel file was being created under bazel_workspace,
which would be deleted and thus rendered useless from the perspective of
not retriggering a build.
Test: `touch Android.bp` then `NINJA_ARGS="-d explain" m --bazel-mode-staging adbd_test`
Bug: 265155778
Change-Id: If5d8f0ea7f3b8828fda0646faafd1a621f0cb27c
This piggybacks onto the ApexInfo cquery handler, so we're issuing a
single bazel query call that reads two providers in the starlark expr.
Also rename requiredDeps to makeModulesToInstall to differentiate it from
APEX's required/provided libs in the apex manifest.
Test: unit test
Test: mkdiff
Fixes: 263123189
Change-Id: Ib7e43f1586f29864eee8627dba3631bfaff27afa
The adbd_test androidmk definition is missing some cc-related variables
for cc_test in mixed builds. These variables should be populated from
information from Bazel.
Bug: 265758350
Change-Id: I59d017e2eb2f139188ba3383c457cc0055372b61
Previously if there was a value set outside of product variables and one
inside a conditions default soong config var, we would ignore the
conditions default. Now, we maintain the conditions default and use
the root value in cases where there is not a value for a product
variable.
Test: go test soong tests
Change-Id: Ic991e3aebe5bb6039353f4e3d25625e7c5190f96
The .capex is the compressed apex. Depending on various conditions, the
.capex should be installed, and thus the
apex.go#ProcessBazelQueryResponse should set the .capex as the main apex
output file.
Fixes: 264873511
Test: new unit test with install rules
Test: m com.android.adbd --bazel-mode-staging gives:
$ ls -lah
out/target/product/generic_arm64/system/apex/com.android.adbd.capex
-rw-r----- 1 jingwen primarygroup 3.6M Jan 10 14:44 out/target/product/generic_arm64/system/apex/com.android.adbd.capex
Change-Id: Ic3143cc6999f111428c6adb76760be8ddc4b6344
This is a follow up of aosp/2336916. All the properties that have
variant_prepend tag are supported in bp2build.
Bug: 261644490
Test: TH and modified tests with updated expected behaviors
Change-Id: I13bb5d4d8fcd38bf153fa9083c9c945aec2dfa19
The comments in aosp/2336916 are also addressed here
Test: TH and TestCcLibraryVariantPrependPropOrder
Change-Id: If1472658fcd4b5544dec2e2691049a180520c84e
osLinux is implicitly glibc, use //build/bazel/platforms/os:linux_glibc
for the configurability select key for it.
Bug: 259266326
Test: build/bazel/ci/mixed_libc.sh
Test: USE_HOST_MUSL=true build/bazel/ci/mixed_libc.sh
Change-Id: Ifbd121ab3d943f3fdfbd3f6c3e982d1eaaaa9ec1
This will allow the test_suites_{arm64, x86_64} CI targets on
git_master-bazel to build adbd_test with bazel in `m`.
This is currently implemented similar to the cc_binary handler to
hydrate the output path field.
The tradefed config is still currently handled by Soong in
testBinary#install, which is called from both ProcessBazelQueryResponse
and GenerateAndroidBuildActions. We'll move the tradefed config path to
a mixed builds handler when it's ready on the cc_test bazel rule side.
Tested with `atest` with cuttlefish. Since `atest` doesn't have a way to pass
--bazel-mode-staging to its internal call to Soong, so I manually tested
this by adding adbt_test to the prod allowlist, and confirmed that the
installed adbd_test is the same binary as the one in bazel-out.
Test: $ showcommands adbd_test | grep bazel-out | tail -n1
```
/bin/bash -c "rm -f
out/target/product/vsoc_x86_64/data/nativetest64/adbd_test/adbd_test &&
cp -f -d
out/bazel/output/execroot/__main__/bazel-out/android_x86_64_silvermont-opt-ST-59d4124ca041/bin/packages/modules/adb/adbd_test
out/target/product/vsoc_x86_64/data/nativetest64/adbd_test/adbd_test"
```
Test: Add adbd_test to prod allowlist and run 'lunch aosp_cf_x86_64_phone-userdebug; atest adbd_test', all tests pass.
Bug: 258426425
Bug: 261692372
Change-Id: I6be329f39d607af8daa1dc772b23e8dd0247418c
Some properties in soong are marked "variant_prepend", which means
in bp2build output, variant properties(select ...) should come before
general properties.
Prepend property is added in StringListAttribute
stuct so that bp2build can handlle variant_prepend properly, if
Prepend is properly set in different module logics.
This change also coordinates cc library with Prepend.
Coordination with other module types will be in separate CLs.
Bug: 261644490
Test: TestCcLibraryHeadersSimple,
TestCcLibraryHeadersArchAndTargetExportSystemIncludes and TH
Change-Id: I3b61c491c54bbe91ba9aa2af9b4c84193c91ae7f
The axis for OS and in APEX was previously ignoring the excludes from
OS, which would result in excludes for non-Android OSes being
incorrectly ignored.
Test: go test Soong tests
Test: b build //hardware/libhardware:all
Bug: 260131489
Change-Id: Ie77f763bea0f473ac69a0c6b6bc3216e7359ad12
When a target does not have a certain provider we need,
currently we just fail at the dictionary lookup. Change
it so that we fail with a message that tells us what
target had the issue.
Bug: 260131489
Test: Presubmits
Change-Id: I99fd018835c14eba0d4db54ba2769b2da1b278f6
This file contains the java APIs parsed by the 'dexdeps' tool within the gen_java_usedby_apex.sh.
Bug: 239084755
Fixes: 239084755
Test: presubmits
Change-Id: Ia271783a6be3ea3a343481306cde1aaba2166e88
The build uses unstripped binary/shared library to extract function
signatures, so for each each target of this kind Bazel should return
its unstripped version, too.
Fixes: 220164721
Test: treehugger
Change-Id: Id5f6143340519bf2ae98791a9e981d1306bb08d1
The module types in scope of this conversion are
1. cc_library and cc_library_shared (non-null llndk or stubs prop)
2. cc_library_headers (all)
For (2), we need some postprocessing on the results of the parser
bp2BuildParseBaseProps. This is necessary because arch and os specific
API exports need to be flattened and added to the generateed API headers
target along NoConfigAxis
e.g.
```
The api equivalent of
cc_library_headers (
name = "lifoo",
deps = select({
"//build/bazel/platforms/arch:arm": ["arm_deps"],
"//build/bazel/platforms/arch:arm64": ["arm64_deps"],
}),
)
should be
cc_api_library_headers (
name = "lifoo",
deps = ["arm_deps", "arm64_deps"],
)
```
For (1), we also need to generate 1:many header api targets so that
arch-specific deps can propagate arch metadata to the top-level
api_domain rule
Test: go test ./bp2build
Test: go test ./cc
Change-Id: Ie40cba1ac8e89f290b3d926c190d5e93abd52859
Add barebones riscv64-linux-android support. This should be enough
to add riscv64-specific entries to Android.bp files, but can't
actually compile anything until there are riscv64 toolchains.
Test: arch_test.go
Change-Id: I0dcc7e797d9352dd38243be908a7f19004ff3db1
This makes mixed builds resilient to a recent backward-incompatbile
Bazel feature which explicitly adds the main repository (`@`) prefix to
all labels which are stringified by `str(target.label)` in Starlark.
This implementation is compatible with both current Bazel and
Bazel@HEAD.
After a new Bazel release to AOSP, we can clean up this code a little to
only support Bazel with this feature active (and that we need not hedge
on either Bazel behavior).
Bug: 248106697
Test: `m --bazel-mode nothing` with both current Bazel and a new Bazel
Change-Id: Id53c8505bb9080d5073c844de7f1ee57ceceae46
Previous commit: commit fcc53f992b.
Test: 1. use `TARGET_BUILD_VARIANT=userdebug TARGET_PRODUCT=aosp_arm64 build/bazel/ci/mixed_droid.sh` to verify no error occurred.
2. revise the test cases under request_type_test. 3. pass the test cases
Bug: 242587802
Change-Id: I7f70e62f7b6f42fa5f211772888040a0d07a710b
also using a generic json_encode as an approximation for `json.encode`
Test: 1. use m --bazel-mode-dev nothing to verify no error occured. 2.
revise the test cases under request_type_test. 3. pass the test cases
Bug:242587802
Change-Id: I1288ecca1afd3e32f6473bcabae4ee2cb5838007
Fix: 232544453
Test: go tests
Test: b build //frameworks/av/media/module/foundation:libstagefright_foundation
Test: b build //frameworks/av/media/module/foundation:libstagefright_foundation (with in-apex hardcoded to True, non-apex hardcoded to False)
Change-Id: I7b5f356ff6ed31f89a9d9153396bc851d991c487
also using a generic json_encode as an approximation for `json.encode`
Test: USE_BAZEL_ANALYSIS=1 m libc
Bug: b/242587802
Change-Id: Ib83aeda3f99cc3966548b0cd47e72669422ed72d
* changes:
add bp2build unit tests for aidl_interface
AIDL source generation accounts for Bazel paths
convert .aidl srcs for java_library
add Partition method to LabelListAttribute
Bug: 231322789
Test: with this CL on internal master,
1) b build --verbose_failures //vendor/google/modules/AdbdGoogle:com.google.android.adbd --config=android_x86_64
2) adb install bazel-bin/vendor/google/modules/AdbdGoogle/com.google.android.adbd.apex, and adbd is activated successfully on cuttlefish device.
3) m mts && mts-tradefed run mts-adbd, and there is no failure with cuttlefish device
4) packages/modules/adb/test_device.py, and there is no failure with cuttlefish device.
Change-Id: I81b6f5336cacf35c68957ae2dac65f985b6eafb9
It can be tedious to split a LabelListAttribute on all of the
configuration axes, so LabelListAttribute.Partition can be used to do
this based on a predicate function.
Test: go test ./bazel
Change-Id: I78ff8c24ce9c86bab5896290d8afe979d56545b6
Use of the native CcSharedLibraryInfo results in using a superfluous
symlink for shared libraries in mixed builds. The new provider
propagates an unwrapped output file (real file, not a symlink).
Test: With related rules changes, manually verified that installed
mixed-build libc was not a symlink. (cf_x86_64, inspected system.img
with `m droid`)
Change-Id: I2a59b348fbfc2a381c229d66c0098e87674a9ddc
Instead of a StringListAttribute of length 1,
introduce a more appropriately reduced StringAttribute
so that e.g. `select`s work properly
Test: new cases in TestGenerateBazelTargetModules
Change-Id: I0ae0e4a51e39f85caf55b0d00459222ede6de79c
bp2buildTestCase
attrNameToString
runBp2BuildTestCase
makeBazelTargetNoRestrictions
The testing framework defined in the bp2build package can only be used
from within the package because many common testing functions are
private to the package. This prevents modules defined in Soong
plugins (e.g. system/tools/aidl/build) from testing bp2build conversions.
Test: go test ./bp2build
Change-Id: Ia867081327c5181d04687b13c4550e68e6a11f86
* Add ApexCqueryInfo to obtain apex artifacts used by the makefile
generator and downstream modules
* Refactor code common to GenerateAndroidBuildActions and ProcessBazelQueryResponse
* Implement android.MixedBuildBuildable for modules
* Enable mixed build for apex modules with payload_type:"image"
The first take 6a2b7c40b was setting compressed APEX suffix incorrectly, and
was reverted in 8a3c91494.
Fixes: 239925080 239695521 232085015
Test: treehugger
Change-Id: I1720f8db3c7cc773183d25a815d9b7eeaf7c73ad
* Add ApexCqueryInfo to obtain apex artifacts used by the makefile
generator and downstream modules
* Refactor code common to GenerateAndroidBuildActions and ProcessBazelQueryResponse
* Implement android.MixedBuildBuildable for modules
* Enable mixed build for apex modules with payload_type:"image"
Bug: 232085015
Test: treehugger
Change-Id: I9f1e03e5e7a5b5dde35a5db10c253069543ac973
Add toolchains to support cross compiling to aarch64-linux-musl and
arm-linux-musleabihf.
Bug: 236052820
Test: build arm and arm64 musl sysroots
Change-Id: I47a9322929baff2492c6e8db989ece01fcbeb133
By default, most module types are only enabled for device. Converting
this behavior and host_supported/device_supported properties allows us
to skip building incompatible targets.
Test: soong tests
Test: bp2build.sh
Change-Id: If1da523b4cc8c4cbf2bb26da063d9923b662cc32
Note go.mod file changes go version to 1.18 (for generics)
You might need to refresh/reload your IDE (e.g. IntelliJ)
Bug: N/A
Test: manually verified (e.g. ran `m nothing` successfully)
Change-Id: I2a6dadd673fd170961d1fd1e1756f7581e9a1091
Bug: 196084681
Test: b run //build/bazel/examples/python/protobuf:build_bazel_examples_python_protobuf_main --config=linux_x86_64
Change-Id: I4d806902d262351231f64686a5d24513a25d9749
This change will cause bp2build to generate genlex targets any
time a .l or .ll file is present in the srcs for a cc target and
add those genlex targets to the srcs attribute of the original
target.
Bug: 207408632
Test: unit tests
Change-Id: I1bce82c9d3c3d458eae1cef547ffae3d6e975134
This change constitutes a number of fixes which cause mixed builds to
have deterministic ninja file output:
1. Depsets are identified based on a hash of their contents instead of
an arbitrary ID integer from Bazel
2. Depset definitions in the ninja file are sorted by the above hashes
3. BuildStatements (action information from Bazel's aquery) are sorted
by their contents
Test: Ran `USE_BAZEL_ANALYSIS=1 m nothing` three times and verified the
md5sum of out/soong/build.ninja was identical all three runs.
Test: mixed_droid
Change-Id: Iffdf6cc62c31d76fbbfa78726827497516171f4f
