Revert submission 2651299-bazel_sandwich
Reason for revert: b/293883239, checking if this breaks the build.
Reverted changes: /q/submissionid:2651299-bazel_sandwich
Change-Id: I5f1d4bb13d21e6599f5c353dcaba2375f5ec234d
The "bazel sandwich" is a mechanism for bazel to depend on make/soong
outputs. The name comes from the fact that bazel is now at the top
and bottom of the build graph. This is intended to allow us to work
on converting the partition builds to bazel while not all of the
dependencies of the partition have been converted.
It works by adding the bazel_sandwich_import_file rule, which emits a
dangling symlink that starts with bazel_sandwich:, and includes
information that the aquery handler in soong reads. The aquery handler
rewrites the symlink so that it points to a file generated by
make/soong, and adds a ninja dependency from the symlink to the file
it's targeting.
This allows us to depend on make-built files from bazel, but notably
it doesn't allow us to depend on analysis-time information from make.
This shouldn't be a problem for the partitions, but limits the use of
the bazel sandwich to similar, less complicated types of builds.
go/roboleaf-bazel-sandwich
Bug: 265127181
Test: m bazel_sandwich
Change-Id: Ic41bae7be0b55f251d04a6a95f846c50ce897adc
The use case for this is for building rules_go's root builder which runs
into issues when built in a directory that contains a symlink to
prebuilts/go
The implementation will involve two changes of working dir
- `sbox` to change the working directory to
__SBOX_SANDBOX_DIR__
- the generated manifest will change the working
directory to mixed build execution root relative to that
Implemenation details
1. Create a unique intermediate path by hashing the outputs of a buildAction.
"out/bazel/output/execroot/__main__/" was deliberately not chosen as
the outpuDir for the sandbox because ruleBuilder would wipe it.
`sbox` will generate the files in __SBOX_SANDBOX_DIR__ and then place
the files in this intermediate directory.
2. After the files have been generated in (1), copy them to
out/bazel/output/execroot/__main__/...
3. For bazel depsets that are inputs of an action, copy the direct
artifacts into the sandbox instead of the phony target
4. Make sandboxing an opt-in. Currently we will only use it for
`GoToolchainBinaryBuild`
In the current implementation, (3) will increase the size of the ninja
file. With sboxing turned on for only GoToolchainBinaryBuild, this will
increase the size of the ninja file by around 1.3% on aosp's cf
Test: m com.android.neuralnetworks (will build soong_zip from source
using rules_go)
Test: OUT_DIR=out.other m com.android.neuralnetworks
Bug: 289102849
Change-Id: I7addda9af583ba0ff306e50c1dfa16ed16c29799
Propagate api_domain from top-level config_node(s) in mixed builds.
This ensures that cc_libraries are built in the correct config setting
in mixed builds
Test: m com.android.media.swcodec
Test: built and booted google variant of com.android.media.swcodec
Test: m ccodec_unit_test
Change-Id: I14e3c8e8358b5e90e71697584f9b0eceb018bfa9
- Creation of a bazel command is independent of test/real implementation
- Use a custom struct instead of cmd.Exec
- Move mock bazel runner to the test
Bug: 270989498
Test: m nothing
Test: USE_PERSISTENT_BAZEL=0 m nothing
Test: Treehugger
Change-Id: Ieec35dad5e21aac644d5b8dc79a92397d42db861
Bug: 240424572
Test: Manual tests:
1. m --dev-mode-staging com.android.adbd com.android.media.swcodec.
2. verify the DCLA libs from the two apexes have the same size and
sha1sum, and also match the libs in bazel-out.
3. empty the DCLA libs list in allowlist.go and repeat step 1
4. repeat step 2 and verify the opposite result
5. build git_master: mainline_modules_bundles-userdebug in ABTD
with the cl, then follow go/build-sideload-dcla-locally to
download the adbd and swcodec aab files, run the DCLA trimming
workflow locally, and verify the symlinks in the two trimmed
apexes are identical and also match the lib path in the DCLA
apex that was created by the workflow.
Change-Id: Ib2f8a29126a54829c0e10eba17b256a79930fd70
This also fixes determinism of these files (by ensuring that bazel
requests and their configurations are sorted, as these are used in the
soong_injection file output)
Bug: 266983462
Test: Manually verified soong_injection files are the same among
multiple runs
Test: Unit test
Test: m nothing
Change-Id: I1568930549cff0bc5676825434cc448d16ebdd4e
* Remove `Bp2buildCcLibraryStaticOnlyList` it is never set
* Remove the rest of the sutff related to it
* Streamline MixedBuildsEnabled
* Rename BazelContext.BazelAllowlisted to BazelContext.NameFilter
to reflect its action
* Rename bazelContext to mixedBuildBazelContext
* Fix minor warnings from Go static analyzer
Test: treehugger
Change-Id: Ie245c992f94bbfd5c7c23dd06917026200a28263
Write long bazel commands returned by aquery to a shell script
before executing them to work around command line length limits.
Bug: 261536423
Test: USE_HOST_MUSL=true build/bazel/ci/mixed_libc.sh
Change-Id: Id98a07800df82d1538e6b16b733488a1b9943bf3
https://android-review.git.corp.google.com/q/topic:bazel_using_txt added
a tree artifact output for an action. This CL adds "-r" to the rm calls
to support deleting the old tree artifact outputs.
Test: presubmits
Fixes: 257956807
Change-Id: Ia44b1bef4f8031fe75a239fb162972e8827e3c19
This makes mixed builds resilient to a recent backward-incompatbile
Bazel feature which explicitly adds the main repository (`@`) prefix to
all labels which are stringified by `str(target.label)` in Starlark.
This implementation is compatible with both current Bazel and
Bazel@HEAD.
After a new Bazel release to AOSP, we can clean up this code a little to
only support Bazel with this feature active (and that we need not hedge
on either Bazel behavior).
Bug: 248106697
Test: `m --bazel-mode nothing` with both current Bazel and a new Bazel
Change-Id: Id53c8505bb9080d5073c844de7f1ee57ceceae46
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:
1. Queueing calls to Bazel is done in a new mutator instead of a full
soong_build pass. Normal soong_build flow is interrupted (via a
functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
as implementing MixedBuildsBuildable. In this interface, define the
request that must be queued to Bazel, and then subsequently define
how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
improves mixed build performance.
Result:
A 33% runtime improvement on soong analysis phase with mixed builds.
Caveats:
C++ BazelHandler handling still remains a bit of a mess; I did what
I could within this CL's scope, but this may require additional cleanup.
Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.
Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
This also removes the special-case filegroup from mixed builds
buildroot; no special handling is required. Since we're currently
hardcoding linux_x86_64 as our host platform, it should be fine
to continue assumping that hardcoded host for now.
Test: USE_BAZEL_ANALYSIS=1 m adbd
Change-Id: I35509f4eb33ba7a243fab4c34b35958f6f2fceab
This will be the list of targets directly being requested + additional
targets necessary for mixed_root. This reduces the size of the cquery
file by ~300Kb. Additionally, it removes unnecessary targets, making it
ever so slightly easier to find what one is looking for in cquery.out
for debugging purposes.
Test: build/bazel/ci/mixed_droid.sh
Change-Id: I6d0e337c9ab7c13388459dee16415be0ab3c0506
This builds cap_names.list.h, which uses an eponymous filegroup
"generate_cap_names_list.awk" in Soong, but uses the file target
directly in Bazel.
This also improve filegroup support for mixed builds, by issuing a
cquery call _without_ arch. Filegroups in Soong don't have configurable
properties, so don't generate Bazel filegroups into buildroot's
config_nodes (which was x86_64 by default).
The mixed_build_root now looks like this:
```
config_node(...)
config_node(...)
config_node(...)
config_node(...)
...
filegroup(name = "common",
srcs = ["@//bionic/linker:linker_sources_x86",
"@//bionic/libc:kernel_input_headers",
"@//system/timezone/apex:com.android.tzdata-androidManifest",
"@//external/libcap:generate_cap_names_list.awk",
"@//bionic/linker:linker_sources_arm64",
"@//bionic/linker:linker_sources",
"@//bionic/libc:libc_sources_shared_arm",
"@//bionic/linker:linker_sources_x86_64",
"@//bionic/libc:all_kernel_uapi_headers",
"@//build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal-file_contexts",
"@//system/core/libcutils:android_filesystem_config_header",
"@//bionic/libc:libc_sources_static",
"@//bionic/linker:linker_sources_arm",
"@//bionic/libc/tools:bionic-gensyscalls",
"@//bionic/tools:bionic-generate-version-script",
"@//bionic/libc:libc_sources_shared"],
)
mixed_build_root(name = "buildroot",
deps = [":x86",
":arm64",
":arm",
":common",
":x86_64"],
)
```
Test: CI
Fixes: 198595323
Fixes: 198235838
Change-Id: I6df9a14da556cf358d96e6a99b514f66a2638295
These are just out/ and out/soong/ and the old names were quite
confusing.
Test: Presubmits.
Merged-In: I999ca62162062f27e449f83ffb00fa580d4065b8
Merged-In: I2fcd80a9e376a6842a06aa104a9f6f5c5e16f8c7
Change-Id: Ib481d2aac40df8da1b74174dc8ec576c1cb48be0
