Commit graph

231 commits

Author SHA1 Message Date
Chris Parsons
ad876010fe Refactor mixed build allowlist handling
This refactoring prepares for introduction of bazel prod mode, an
alternative mechanism for mixed builds allowlist handling.

 * Decide bazel-mode as close to soong_build main as possible
 * BazelContext itself decides whether a module is allowlisted
 * Separate bp2build and mixed build allowlist

Test: m nothing, manually verified all modules are mixed build disabled
(via metrics)
Test: USE_BAZEL_ANALYSIS=1 m nothing, manually verified that mixed build
disabled/enabled modules are identical before and after change.

Change-Id: I0f55d8b85000cb4a871a099edc6d7d868d7df509
2022-08-23 13:15:12 -04:00
Vinh Tran
444154d5d3 Convert filegroup with AIDL srcs to aidl_library
Change-Id: I94c185744a86c812dc48e30b66e060361b9161cb
Test: USE_BAZEL_ANALYSIS=1 m libbinder
Test: USE_BAZEL_ANALYSIS=1 m module-lib-api-stubs-docs-non-updatable
Bug: 243010121
2022-08-18 14:01:15 -04:00
Sasha Smundak
a095406411 Consolidate Bazel conversion state into a single field in CommonProperties
Bazel conversion state needs to be cloned during module cloning, so it
should be registered via module.AddProperties. Until now it was done
implicitly as the modules converted so far added the whole
base.commonProperties field (and thus transitively all its subfields).
However, adding CommonProperties also adds its fields to the list of
the module's attributes, which is undesirable. The problem surfaced
while implementing Bazel conversion to the `license` rule.

Bug: 190817312
Test: treehugger
Change-Id: Id3de4ede8df81b21f00065a3a1bdc2d707391c3a
2022-08-03 10:46:19 -07:00
Wei Li
440c66a799 Merge "Change init process of bp2buildAllowlist so the Soong plugin of bp2build allowlist can hookup properly." 2022-05-20 20:36:40 +00:00
Chris Parsons
f874e46153 Refactor mixed builds to only take one pass
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:

1. Queueing calls to Bazel is done in a new mutator instead of a full
   soong_build pass. Normal soong_build flow is interrupted (via a
   functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
   as implementing MixedBuildsBuildable. In this interface, define the
   request that must be queued to Bazel, and then subsequently define
   how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
   improves mixed build performance.

Result:
  A 33% runtime improvement on soong analysis phase with mixed builds.

Caveats:
  C++ BazelHandler handling still remains a bit of a mess; I did what
  I could within this CL's scope, but this may require additional cleanup.

Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.

Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
2022-05-20 10:04:13 -04:00
Wei Li
d7736ec1b1 Change init process of bp2buildAllowlist so the Soong plugin of bp2build allowlist can hookup properly.
Bug: 216442475
Test: m nothing
Change-Id: I47d768e09aa290e23fb846404686dcf26c8caf41
2022-05-12 23:40:45 -07:00
MarkDacek
ff851b83b6 Log information for Mixed Builds modules.
Test: Output matches expected. https://paste.googleplex.com/5913495636803584?raw
Performance evaluated: https://docs.google.com/spreadsheets/d/1X7eOVBKEZUwUWl5i8CDfBo9yUeZrDPXWi2JYO4BEZt4/edit?resourcekey=0-co8crIFW9dpiedhCMkhAgw#gid=0

Change-Id: I88780c7cc52a189a72216c5e2e499c96574b3731
2022-05-03 18:11:32 +00:00
Sam Delmerico
24c5603815 refactor bp2build module allowlists into struct
To make testing easier, refactor existing module-global variables into a
struct that can be mocked.

Test: build/bazel/bp2build.go
Change-Id: I9d177677644ea743641a745b1839a3a8b29f902a
2022-04-14 13:31:05 +00:00
Liz Kammer
d5aa079291 Clean up allowlist contents.
Test: mixed_droid.sh
Test: bp2build.sh
Change-Id: I6d00a55400e7843899c71be3de9e52403ba26f55
2022-04-06 15:26:15 -04:00
Liz Kammer
6879cc0983 Denylist modules that depend on a rust binary
Test: b build //system/...
Test: CI
Change-Id: Ib01124ca22e1ed83e9b71a18607d60fb1a4fc8e2
2022-04-05 18:00:22 -04:00
Sam Delmerico
489831660c convert java_import jars with arch variants
Bug: 220168131
Test: go test ./bp2build -run TestJavaImport
Test: b build '...'
Change-Id: Ib485e6905e7be9d199af5295b23131c226fd6e96
2022-03-31 21:00:24 +00:00
Sam Delmerico
3c25ab1ac3 Merge changes I8e4c3f37,I89a32bde
* changes:
  add plugins to javaLibraryAttributes
  convert java_import_host with bp2build; enable jetifier
2022-03-30 21:08:12 +00:00
Treehugger Robot
d93afba1e6 Merge changes from topic "no-srcs-no-deps"
* changes:
  enable conversion of all java_import modules
  emulate java_library static_deps with Bazel exports
2022-03-30 21:06:46 +00:00
Sam Delmerico
5f83b491b4 convert java_import_host with bp2build; enable jetifier
Also, add java_imports that were blocked by jetifier binary.
b/221082840 was blocking the build of jetifier which these java_imports
depended on. That bug is now resolved, so we can unblock these targets.

Bug: 220168131
Test: b build //packages/apps/Car/libs/car-ui-lib/car-ui-androidx/...
Test: build/bazel/ci/bp2build.sh
Change-Id: I89a32bde30c133a44aeb7826afa11d4140433f5d
2022-03-30 19:19:13 +00:00
Sam Delmerico
a9b047a9da enable conversion of all java_import modules
Bug: 220168131
Test: b build '...'
Change-Id: Ib1eaf47b81ac19cb6e930e5f9f3fc2aebe02b0c1
2022-03-30 19:19:09 +00:00
Sam Delmerico
c016143602 emulate java_library static_deps with Bazel exports
In Soong, java_library can specify static_deps which are dependencies
that get aggregated into the final jar (akin to static linking). This is
useful because it allows dependencies higher up in the chain to compile
against the APIs exported by transitive dependencies. Bazel does not
support this functionality directly, but it can be emulated via the
exports attribute which makes any targets listed in the attribute public
to targets further up the chain.

Bug: 217236083
Bug: 219908977
Test: b build //external/error_prone:error_prone_core
Test: b build //external/bouncycastle:bouncycastle-host
Test: b build --platforms=//build/bazel/platforms:linux_x86
  //prebuilts/sdk/tools/jetifier/jetifier-standalone:jetifier
Change-Id: I2867e3f816de720a6f4bd9ff7a847d1b0c2da2d6
2022-03-30 18:58:38 +00:00
Paul Duffin
26f19919ea analyze_bcpf: Add --fix option
Add a --fix option that will cause the script to automatically fix the
issues that it finds. It uses the bpmodify tool to add values to the
bootclasspath_fragment's hidden_api properties.

This adds analyze_bcpf to bp2buildModuleDoNotConvertList as
analyze_bcpf depends on bpmodify which is a blueprint_go_binary which
is not yet supported by bazel.

Bug: 202154151
Test: m analyze_bcpf && analyze_bcpf --bcpf art-bootclasspath-fragment
      m analyze_bcpf && analyze_bcpf --bcpf art-bootclasspath-fragment --fix
      atest --host analyze_bcpf_test

Change-Id: I5ee52419b4829474f6dbeb47f86ab2aeb22b1382
2022-03-30 16:00:08 +01:00
Sam Delmerico
f241e72b57 Merge "add module type allowlist to bp2build" 2022-03-22 16:35:49 +00:00
Sam Delmerico
85d831a28e add module type allowlist to bp2build
This change also requires that java_import modules be prepended with the
"prebuilt_" prefix in the `bp2buildModuleAlwaysConvertList`. This fixes
a bug where trying to disable a java_import by name without the
"prebuilt_" prefix would also disable the corresponding source module if
it exists with the same name.

Test: build/bazel/ci/bp2build.sh
Change-Id: If75236891810e5112d70441653be4644d8fe5ecf
2022-03-17 23:43:07 +00:00
Jingwen Chen
f2504b64eb Allowlist //external/e2fsprogs/...
Test: CI
Bug: 204244290
Change-Id: Ie18b555fb165cd3b869a3d7fb2c4ad73aef4bad5
2022-03-17 16:52:13 +00:00
Jingwen Chen
53c29e2467 Build e2fsdroid with Bazel.
Test: CI
Bug: 204244290
Change-Id: If5f01da62cff2913fefc98aed7fe7b986e6e7a33
2022-03-17 15:33:38 +00:00
Treehugger Robot
7db79508c7 Merge "bp2build supports arch variant srcs for java_library" 2022-03-14 17:06:01 +00:00
Jingwen Chen
398a32d9b0 Allowlist //external/auto/android-annotation-stubs for bp2build.
Fixes breakage for bp2build-incremental.

Test: b build //external/...
Change-Id: Id493c119b6ba0f383c9d1e8bade8b8a0acf3efe6
2022-03-08 08:03:28 +00:00
Sam Delmerico
e91d0303f2 bp2build supports arch variant srcs for java_library
Bug: 209577426
Test: build/bazel/ci/bp2build.sh
Change-Id: I6799bda904d286616e580f2395601ce4f764180f
2022-03-07 16:15:56 +00:00
Jingwen Chen
7edadabd6c Build avbtool with Bazel.
Test: CI
Change-Id: Ic506fea3ba1fb61692f59d8e6f708115c1a327ab
2022-03-07 04:40:35 +00:00
Wei Li
083d4e072c Merge "Keep Bazel build file in build/bazel/vendor/google and prebuilts/bundletool." 2022-03-05 04:31:55 +00:00
Treehugger Robot
fe86b81aab Merge "prevent module being enabled if in existing BUILD file" 2022-03-04 20:37:21 +00:00
Sam Delmerico
d508a638c3 Merge "convert android_app certificate property, bp2build" 2022-03-04 17:30:31 +00:00
Sam Delmerico
94d26c2bc0 prevent module being enabled if in existing BUILD file
If a module is enabled via the module-level allow list,
bp2buildModuleAlwaysConvertList, then we should error if that same
module is hidden by an existing BUILD file.

Bug: 220875679
Test: add a module in a directory covered by an existing BUILD file
Change-Id: I5426bd9edac593690c0989b68a421ac1c3216010
2022-03-02 22:15:14 +00:00
Wei Li
171410478a Keep Bazel build file in build/bazel/vendor/google and prebuilts/bundletool.
Test: b test //build/bazel/tests/apex:build.bazel.examples.apex.minimal_mainline-module
Bug: 217838178

Change-Id: If94a2086db2a4afabe600b691f65b388ea19dcfe
2022-03-02 02:09:46 -08:00
Liz Kammer
6bd0d4e864 Merge "Disallow libc_musl_sysroot_libc++_headers" 2022-03-01 19:10:16 +00:00
Liz Kammer
878a7f1632 Disallow libc_musl_sysroot_libc++_headers
This depends on go modules and is musl specific, not a current focus.
Allows CI to be green again.

Test: bp2build.sh
Change-Id: I35520636b38eed3f829e176c4ba91517fd61364a
2022-03-01 09:08:23 -05:00
Sam Delmerico
fa1831c200 add a module-level allowlist to bp2build
We currently enabled modules in bp2build on a package level, and we
disable individual modules based on a denylist. To enable a specific
module individually, we must change the corresponding Android.bp file to
include a bp2build_available property. Instead, have a centralized list
of specifically enabled modules.

Bug: 220875679
Test: m bp2build
Change-Id: I908f2f2366b72affb04234fbe92ba03bf5a37338
2022-02-24 14:46:41 +00:00
Sam Delmerico
9dfb139b6d convert android_app certificate property, bp2build
The android_app certificate property can be converted to the
@android_rules debug_signing_keys attribute in Bazel after converting
the .pk8/.pem key pairs to a JKS keystore in Starlark.

Test: b build -s //frameworks/base/tests/appwidgets/AppWidgetHostTest
  and verify includes SignApk action with generated keystore
Bug: 194133023
Change-Id: I2c4276f94a7856fc68a7674e89742f887dca31b4
2022-02-18 20:17:01 +00:00
Yu Liu
9d7d9810e2 Merge "Enable cc_binary in mixed build." 2022-02-17 21:54:04 +00:00
Yu Liu
7f3605fee1 Enable cc_binary in mixed build.
Bug: 216114082
Test: Run mixed build on adbd apex and verify the pavkaged adbd binary;
add unit test.
Change-Id: Id11549a7952a7ce5ac02988b88b747bbfe8339ff

Change-Id: If01efe076c223886278f466b65628602bff3a964
2022-02-17 11:15:59 -08:00
Treehugger Robot
632d0a03ed Merge "Keep build/bazel/ci/dist/BUILD for bp2build." 2022-02-16 16:14:26 +00:00
Jingwen Chen
eb2a3fd367 Add error_prone_checkerframework_dataflow_nullaway. to bp2build denylist.
Bug: 219908977
Change-Id: Ia6a0879d9af7aabadb5cd5722ddfdf72f579e621
Test: CI
2022-02-16 13:03:29 +00:00
Treehugger Robot
948c340160 Merge "enable auto_service_plugin bazel build" 2022-02-09 17:48:49 +00:00
Jingwen Chen
941f0ab807 Keep build/bazel/ci/dist/BUILD for bp2build.
Test: CI
Change-Id: I008672cbf746185c40e7832b70aed6dc564c111c
2022-02-09 13:25:30 +00:00
Jingwen Chen
f6b4cd4bcd Omit musl related cc_genrules from bp2build.
These trigger a few failure modes (duplicate genrule srcs, failure in
matching config conditions, depending on unconverted soong_zip host
tool).

Test: CI
Bug: 218405924
Change-Id: I15c119803a057e94ef6a941df3b58103d42eb4e4
2022-02-08 04:47:48 +00:00
Treehugger Robot
217a70e713 Merge "Close opened paths" 2022-02-08 00:33:04 +00:00
Liz Kammer
0fe123dcdf Close opened paths
Test: m nothing
Change-Id: I7fcc8e2fdbca1cd231ff1e72b02156f12078072b
2022-02-07 10:17:35 -05:00
Sam Delmerico
4c7069f111 enable auto_service_plugin bazel build
Bug: 215229742
Test: b build --platforms=//build/bazel/platforms:linux_x86
 //external/auto/service:auto_service_plugin
Change-Id: If4ec6e063c603d4be45039d09aa2ec9b60370297
2022-02-04 22:09:03 +00:00
Liz Kammer
a20b7a72ab Disable targets broken on host due to missing deps
Test: bp2build.sh
Change-Id: I8617c97954deaa985a1ff67cf4ca4d97207f6913
2022-02-04 13:08:03 -05:00
Liz Kammer
35ca77edb6 Allowlist additional com.android.runtime blockers
To unblock mixed builds, uses a common mechanism to determine cc module
type across bp2build and mixed builds.

Test: mixed_droid.sh
Change-Id: I1f69ad0858cda549b7c95dbadd596e46df0f23b2
2022-02-03 11:17:05 -05:00
Sam Delmerico
fde9fb5c9c convert java static dependencies
This commit enables conversions of java_library_static and adds the
static_deps Soong attribute to Bazel deps for all Java libraries.
Currently, Soong handles static dependencies for Java differently, and
Bp2build may need to address that later (see b/217236083).

Bug: 215229742
Test: go test ./bp2build
Change-Id: I4f86780c634dca0a2cfb9294084b14e139cc2375
2022-02-01 22:59:04 +00:00
Romain Jobredeaux
428a366703 Basic bp2build converter for java_import.
This only supports the `jars` property/attribute.

Test: b build external/error_prone:error_prone_core_jars
Bug: 215229744
Change-Id: If0d9c8c4e9c1c560d35a501c20bef9361ef45e15
2022-01-29 09:44:37 -05:00
Romain Jobredeaux
f49ca9e40d Merge "Revert "Revert "Add static_libs support to android_app bp2build converter.""" 2022-01-21 20:51:06 +00:00
Sam Delmerico
cd1b80f067 bp2build converts java_genrule and java_genrule_host
Bp2build currently supports building genrules for cc modules, but does
not support building java_genrule* modules. This commit adds this
functionality.

Bug: 213480907
Test: go test ./bp2build
Change-Id: I473196c5bcf5582ba0c8faa65b5005f81ac973a4
2022-01-21 17:50:40 +00:00