Commit graph

25 commits

Author SHA1 Message Date
Yu Liu
e43124023c Support DCLA
Bug: 240424572
Test: Manual tests:
1. m --dev-mode-staging com.android.adbd com.android.media.swcodec.
2. verify the DCLA libs from the two apexes have the same size and
   sha1sum, and also match the libs in bazel-out.
3. empty the DCLA libs list in allowlist.go and repeat step 1
4. repeat step 2 and verify the opposite result
5. build git_master: mainline_modules_bundles-userdebug in ABTD
   with the cl, then follow go/build-sideload-dcla-locally to
   download the adbd and swcodec aab files, run the DCLA trimming
   workflow locally, and verify the symlinks in the two trimmed
   apexes are identical and also match the lib path in the DCLA
   apex that was created by the workflow.

Change-Id: Ib2f8a29126a54829c0e10eba17b256a79930fd70
2023-02-21 10:12:47 -08:00
Liz Kammer
690fbac200 Collect more metrics for aquery handling
Test: go test soong tests
Test: m nothing & verify metrics
Change-Id: Id19e004d90dfbaa1b1706c607d5250bf845c0f3b
2023-02-14 11:10:19 -05:00
Chris Parsons
3a8d0fbede Only write soong_injection files if changed
This also fixes determinism of these files (by ensuring that bazel
requests and their configurations are sorted, as these are used in the
soong_injection file output)

Bug: 266983462
Test: Manually verified soong_injection files are the same among
multiple runs
Test: Unit test
Test: m nothing

Change-Id: I1568930549cff0bc5676825434cc448d16ebdd4e
2023-02-03 18:29:02 -05:00
Sasha Smundak
39a301c963 Clean up mixed builds module names filtering
* Remove `Bp2buildCcLibraryStaticOnlyList` it is never set
* Remove the rest of the sutff related to it
* Streamline MixedBuildsEnabled
* Rename BazelContext.BazelAllowlisted to BazelContext.NameFilter
  to reflect its action
* Rename bazelContext to mixedBuildBazelContext
* Fix minor warnings from Go static analyzer

Test: treehugger
Change-Id: Ie245c992f94bbfd5c7c23dd06917026200a28263
2023-01-12 00:32:45 +00:00
Colin Cross
d5c7ddb4df Workaround bazel command too long
Write long bazel commands returned by aquery to a shell script
before executing them to work around command line length limits.

Bug: 261536423
Test: USE_HOST_MUSL=true build/bazel/ci/mixed_libc.sh
Change-Id: Id98a07800df82d1538e6b16b733488a1b9943bf3
2022-12-13 12:40:03 -08:00
Sasha Smundak
4975c82666 Provide Bazel timing breakdown for the mixed builds
Instead of single event in the build metrics
```
| soong_build.mixed_build.bazel          | 1:17.1 |
```
it will now show:
```
| soong_build.mixed_build.bazel          | 1:17.1 |
| soong_build.mixed_build.bazel.cquery   |   51.9 |
| soong_build.mixed_build.bazel.aquery   |   23.1 |
| soong_build.mixed_build.bazel.symlinks |    2.1 |
```

Test: treehugger
Change-Id: I84a7dda6e3122860da9aaa98bfa6afe33d392dcf
2022-11-16 18:44:08 -08:00
Wei Li
cbd181c8e8 Fix the following issues in bazel_handler:
1) Convert coverage path wildcard to regex values for --instrumentation_filter to Bazel.
2) Handle the error properly when aquery commmand returns error.

Bug: 258397103
Test: CIs
Change-Id: Ie7485c2c45f2eb0e1ca0dc60478aae04ab151b61
2022-11-16 08:59:23 -08:00
Jingwen Chen
f3b1ec39eb Call rm -rf in mixed build build statement preamble.
https://android-review.git.corp.google.com/q/topic:bazel_using_txt added
a tree artifact output for an action. This CL adds "-r" to the rm calls
to support deleting the old tree artifact outputs.

Test: presubmits
Fixes: 257956807
Change-Id: Ia44b1bef4f8031fe75a239fb162972e8827e3c19
2022-11-07 15:10:04 +00:00
Jason Wu
118fd2b443 Rollforward "Use proto output instead of jsonproto output for aquery""
Test: use m --bazel-mode-dev and mixed_droid.sh
Bug: 233798334

This reverts commit d27e28c2a7.

Change-Id: Ifd37ac449979330529dd25c92a5eca2f5a80827e
2022-10-27 18:41:50 +00:00
Jason Wu
d27e28c2a7 Revert "Use proto output instead of jsonproto output for aquery"
This reverts commit 6fe8721e12.

Reason for revert: Broke aosp_kernel-build-tools/linux

Change-Id: I502ec4c275cbc56e1024a42fadf68b308750939d
2022-10-27 05:18:40 +00:00
Jason Wu
6fe8721e12 Use proto output instead of jsonproto output for aquery
Test: use m --bazel-mode-dev and mixed_droid.sh
Bug: 233798334
Change-Id: Ib945359d3d05e6f51afe70432ec380e4e584175b
2022-10-26 17:07:48 -04:00
Chris Parsons
86dc2c268a Support main repository prefixing in cquery
This makes mixed builds resilient to a recent backward-incompatbile
Bazel feature which explicitly adds the main repository (`@`) prefix to
all labels which are stringified by `str(target.label)` in Starlark.

This implementation is compatible with both current Bazel and
Bazel@HEAD.

After a new Bazel release to AOSP, we can clean up this code a little to
only support Bazel with this feature active (and that we need not hedge
on either Bazel behavior).

Bug: 248106697
Test: `m --bazel-mode nothing` with both current Bazel and a new Bazel
Change-Id: Id53c8505bb9080d5073c844de7f1ee57ceceae46
2022-09-28 19:02:51 +00:00
Sasha Smundak
e3cf1abefc Reformat the test to make them more readable.
Test: treehugger
Change-Id: I28ccd531a5dadcee3cbdebd5f5ba6db1605662d9
2022-06-30 14:20:55 -07:00
Usta Shrestha
ef92225a67 null build upon repeated mixed build
no implicit deps on bazel-tools

Test: USE_BAZEL_ANALYSIS=1 ../bazel/ci/incremental_mixed_build.sh
Bug: b/216194240
Change-Id: Ibbd87c6a6cc2fddf21fba37a6bb4e72adc209576
2022-06-28 15:37:12 -04:00
Usta Shrestha
acd5a0c080 paths in depfiles are relative to $OUT_DIR
Test: USE_BAZEL_ANALYSIS=1 m libc droid
Bug: b/232250671
Change-Id: I7a570894371bd31339ab0cf3c619c30b3cf8cd73
2022-06-23 14:14:01 -04:00
Yu Liu
8d82ac58b4 Support cc code coverage for mixed build
Bug: 231322627
Test: Manual tests and unit tests
Change-Id: I786042af0d612192c54c3572f63a86a47174a242
2022-06-06 12:29:25 -07:00
Chris Parsons
f874e46153 Refactor mixed builds to only take one pass
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:

1. Queueing calls to Bazel is done in a new mutator instead of a full
   soong_build pass. Normal soong_build flow is interrupted (via a
   functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
   as implementing MixedBuildsBuildable. In this interface, define the
   request that must be queued to Bazel, and then subsequently define
   how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
   improves mixed build performance.

Result:
  A 33% runtime improvement on soong analysis phase with mixed builds.

Caveats:
  C++ BazelHandler handling still remains a bit of a mess; I did what
  I could within this CL's scope, but this may require additional cleanup.

Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.

Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
2022-05-20 10:04:13 -04:00
Liz Kammer
0940b89937 Handle arch/cpu variants in mixed builds
Test: mixed_libc.sh
Change-Id: Ib5084060199b8e1a5955bd51bc3fd793bb83a1eb
2022-03-21 11:13:49 -04:00
Chris Parsons
787fb36189 Add OS to configuration key in mixed builds
This also removes the special-case filegroup from mixed builds
buildroot; no special handling is required. Since we're currently
hardcoding linux_x86_64 as our host platform, it should be fine
to continue assumping that hardcoded host for now.

Test: USE_BAZEL_ANALYSIS=1 m adbd
Change-Id: I35509f4eb33ba7a243fab4c34b35958f6f2fceab
2021-10-19 16:55:52 -04:00
Liz Kammer
c19d5cdcff Restrict mixed build cquery to deps of depth 2.
This will be the list of targets directly being requested + additional
targets necessary for mixed_root. This reduces the size of the cquery
file by ~300Kb. Additionally, it removes unnecessary targets, making it
ever so slightly easier to find what one is looking for in cquery.out
for debugging purposes.

Test: build/bazel/ci/mixed_droid.sh
Change-Id: I6d0e337c9ab7c13388459dee16415be0ab3c0506
2021-10-06 18:16:58 -04:00
Jingwen Chen
1e347860d5 bp2build: allowlist //external/libcap/...
This builds cap_names.list.h, which uses an eponymous filegroup
"generate_cap_names_list.awk" in Soong, but uses the file target
directly in Bazel.

This also improve filegroup support for mixed builds, by issuing a
cquery call _without_ arch. Filegroups in Soong don't have configurable
properties, so don't generate Bazel filegroups into buildroot's
config_nodes (which was x86_64 by default).

The mixed_build_root now looks like this:

```
config_node(...)
config_node(...)
config_node(...)
config_node(...)

...

filegroup(name = "common",
    srcs = ["@//bionic/linker:linker_sources_x86",
            "@//bionic/libc:kernel_input_headers",
            "@//system/timezone/apex:com.android.tzdata-androidManifest",
            "@//external/libcap:generate_cap_names_list.awk",
            "@//bionic/linker:linker_sources_arm64",
            "@//bionic/linker:linker_sources",
            "@//bionic/libc:libc_sources_shared_arm",
            "@//bionic/linker:linker_sources_x86_64",
            "@//bionic/libc:all_kernel_uapi_headers",
            "@//build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal-file_contexts",
            "@//system/core/libcutils:android_filesystem_config_header",
            "@//bionic/libc:libc_sources_static",
            "@//bionic/linker:linker_sources_arm",
            "@//bionic/libc/tools:bionic-gensyscalls",
            "@//bionic/tools:bionic-generate-version-script",
            "@//bionic/libc:libc_sources_shared"],
)

mixed_build_root(name = "buildroot",
    deps = [":x86",
            ":arm64",
            ":arm",
            ":common",
            ":x86_64"],
)
```

Test: CI
Fixes: 198595323
Fixes: 198235838
Change-Id: I6df9a14da556cf358d96e6a99b514f66a2638295
2021-09-09 07:54:16 +00:00
Lukacs T. Berki
9f6c24a887 Rename BuildDir and NinjaBuildDir.
These are just out/ and out/soong/ and the old names were quite
confusing.

Test: Presubmits.
Merged-In: I999ca62162062f27e449f83ffb00fa580d4065b8
Merged-In: I2fcd80a9e376a6842a06aa104a9f6f5c5e16f8c7
Change-Id: Ib481d2aac40df8da1b74174dc8ec576c1cb48be0
2021-08-27 10:08:49 +02:00
Lukacs T. Berki
3069dd9d71 Make @soong_injection usable in standalone builds.
Test: Presubmits.
Change-Id: I942e627f9f7b7428834258ef3ccefb7f1f5c7606
2021-05-11 16:54:29 +02:00
Liz Kammer
286c9fafa6 Fix bazel_handler_test.go
Test: go test soong tests
Bug: 185963707
Change-Id: I738fead29c7054e9deaf03fd3e37d6862309885e
2021-04-21 08:46:34 -04:00
Liz Kammer
8d62a4f725 Add tests for bazel_handler
Test: go test
Test: m nothing
Test: bp2build generate && sync; mixed build libc
Change-Id: Id4307a9e735b50221a8effb963325aaa57c3c7d0
2021-04-12 15:59:06 -04:00