Commit graph

240 commits

Author SHA1 Message Date
MarkDacek
756b2966b8 Add recursive deny to allowlists.
Also change behavior of the recursive check - the more specific
directory will be chosen now, for competing recursives.

This is part of an effort to enable mixed_libc and mixed_droid on
internal master - many denies are required and this would simplify the
effort.
Test: m nothing

Change-Id: I7de05337644d8e1804078c66669dd0b2d65f3e71
2022-10-13 20:40:27 +00:00
Spandan Das
5af0bd3e48 Create a new mode in soong_ui to generate API only BUILD files
The generated Bazel workspace will only contain api specific targets.
This is feasible since these targets do not have any cross dependencies
with the targets in the bp2build workspace

The advantages of a new mode are
1. Does not pollute bp2build workspace with api targets
2. Does not block api targets with the current allowlist conversion
   mechansims in bp2build
(In the future we might want to combine these two workspaces)

A Soong module type will generate a Bazel target if it implements
ApiProvider interface

Test: m apigen
Test: m nothing

Change-Id: I69c57ca6539f932e0ad554ce84a87fb7936fdba0
2022-10-04 17:32:56 +00:00
Yu Liu
609c3e074f Merge "Support proto modules with external references." 2022-09-15 16:00:32 +00:00
Yu Liu
10853f90ef Improve error message when a module is double enabled.
Bug: None
Test: Manual
Change-Id: Ie96dd76d628e8ac3f0da06b949e9bfb9a3259dce
2022-09-14 16:15:43 -07:00
Yu Liu
2aa806b52d Support proto modules with external references.
Bug: 236055697
Test: Manual testing and adding new unit tests.
Change-Id: I984c0ecb93f0023727a39a3af3921820337bf8c7
2022-09-14 13:02:53 -07:00
Chris Parsons
1bb58da17c Remove USE_BAZEL_ANALYSIS
Use --bazel-mode-dev instead.

Test: Treehugger
Bug: 244309479
Change-Id: I570d02651eea92375cd96efbeb8d986a9495e1ed
2022-09-07 14:58:03 -04:00
Cole Faust
ea602c5a41 Remove old BUILD file merging code
It's not needed anymore since aosp/2197837,
it should've been removed in that cl but I forgot.

Bug: 234167862
Test: ./build/bazel/ci/bp2build.sh
Change-Id: I3d67a6e1894ad401525070ad37d3158708898306
2022-08-31 15:12:47 -07:00
Cole Faust
324a92e404 Always merge build files
Previous behavior:

- Packge not listed in bp2buildKeepExistingBuildFile:
    - Use bp2build generated build file
- Package listed in bp2buildKeepExistingBuildFile:
    - Use handcrafted build file even if there were allowlisted bp2build
      modules in the same package.
- Package listed in bp2buildKeepExistingBuildFile and a soong module has
  a bp2build: { label } attribute:
    - Merge the handcrafted and bp2build generated build files

New behavior:

- Packge not listed in bp2buildKeepExistingBuildFile:
    - Use bp2build generated build file
- Package listed in bp2buildKeepExistingBuildFile:
    - Merge with bp2build generated build file.

Bug: 234167862
Test: ./build/bazel/ci/bp2build.sh
Change-Id: Ifbaf4f8f0f5158b5b2bd6d534eb2311e2e5f399b
2022-08-30 16:07:23 -07:00
Chris Parsons
ef615e5841 Add --bazel-mode and --bazel-mode-dev
This allows "bazel mixed builds prod mode", in additional to reworking
the mechanism in which mixed builds dev mode is enabled.

As a followup, CI scripts will be migrated to use the new flags, as
USE_BAZEL_ANALYSIS=1 is deprecated.

Test: Manually ran --bazel-mode with an allowlist verifying that the
module alone was enabled
Test: Manually verified --bazel-mode and --bazel-mode-dev cause a build
failure

Change-Id: If0d34360e60452f428b05828f4ec7596b7cb619a
2022-08-29 10:56:04 -04:00
Chris Parsons
ad876010fe Refactor mixed build allowlist handling
This refactoring prepares for introduction of bazel prod mode, an
alternative mechanism for mixed builds allowlist handling.

 * Decide bazel-mode as close to soong_build main as possible
 * BazelContext itself decides whether a module is allowlisted
 * Separate bp2build and mixed build allowlist

Test: m nothing, manually verified all modules are mixed build disabled
(via metrics)
Test: USE_BAZEL_ANALYSIS=1 m nothing, manually verified that mixed build
disabled/enabled modules are identical before and after change.

Change-Id: I0f55d8b85000cb4a871a099edc6d7d868d7df509
2022-08-23 13:15:12 -04:00
Vinh Tran
444154d5d3 Convert filegroup with AIDL srcs to aidl_library
Change-Id: I94c185744a86c812dc48e30b66e060361b9161cb
Test: USE_BAZEL_ANALYSIS=1 m libbinder
Test: USE_BAZEL_ANALYSIS=1 m module-lib-api-stubs-docs-non-updatable
Bug: 243010121
2022-08-18 14:01:15 -04:00
Sasha Smundak
a095406411 Consolidate Bazel conversion state into a single field in CommonProperties
Bazel conversion state needs to be cloned during module cloning, so it
should be registered via module.AddProperties. Until now it was done
implicitly as the modules converted so far added the whole
base.commonProperties field (and thus transitively all its subfields).
However, adding CommonProperties also adds its fields to the list of
the module's attributes, which is undesirable. The problem surfaced
while implementing Bazel conversion to the `license` rule.

Bug: 190817312
Test: treehugger
Change-Id: Id3de4ede8df81b21f00065a3a1bdc2d707391c3a
2022-08-03 10:46:19 -07:00
Wei Li
440c66a799 Merge "Change init process of bp2buildAllowlist so the Soong plugin of bp2build allowlist can hookup properly." 2022-05-20 20:36:40 +00:00
Chris Parsons
f874e46153 Refactor mixed builds to only take one pass
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:

1. Queueing calls to Bazel is done in a new mutator instead of a full
   soong_build pass. Normal soong_build flow is interrupted (via a
   functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
   as implementing MixedBuildsBuildable. In this interface, define the
   request that must be queued to Bazel, and then subsequently define
   how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
   improves mixed build performance.

Result:
  A 33% runtime improvement on soong analysis phase with mixed builds.

Caveats:
  C++ BazelHandler handling still remains a bit of a mess; I did what
  I could within this CL's scope, but this may require additional cleanup.

Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.

Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
2022-05-20 10:04:13 -04:00
Wei Li
d7736ec1b1 Change init process of bp2buildAllowlist so the Soong plugin of bp2build allowlist can hookup properly.
Bug: 216442475
Test: m nothing
Change-Id: I47d768e09aa290e23fb846404686dcf26c8caf41
2022-05-12 23:40:45 -07:00
MarkDacek
ff851b83b6 Log information for Mixed Builds modules.
Test: Output matches expected. https://paste.googleplex.com/5913495636803584?raw
Performance evaluated: https://docs.google.com/spreadsheets/d/1X7eOVBKEZUwUWl5i8CDfBo9yUeZrDPXWi2JYO4BEZt4/edit?resourcekey=0-co8crIFW9dpiedhCMkhAgw#gid=0

Change-Id: I88780c7cc52a189a72216c5e2e499c96574b3731
2022-05-03 18:11:32 +00:00
Sam Delmerico
24c5603815 refactor bp2build module allowlists into struct
To make testing easier, refactor existing module-global variables into a
struct that can be mocked.

Test: build/bazel/bp2build.go
Change-Id: I9d177677644ea743641a745b1839a3a8b29f902a
2022-04-14 13:31:05 +00:00
Liz Kammer
d5aa079291 Clean up allowlist contents.
Test: mixed_droid.sh
Test: bp2build.sh
Change-Id: I6d00a55400e7843899c71be3de9e52403ba26f55
2022-04-06 15:26:15 -04:00
Liz Kammer
6879cc0983 Denylist modules that depend on a rust binary
Test: b build //system/...
Test: CI
Change-Id: Ib01124ca22e1ed83e9b71a18607d60fb1a4fc8e2
2022-04-05 18:00:22 -04:00
Sam Delmerico
489831660c convert java_import jars with arch variants
Bug: 220168131
Test: go test ./bp2build -run TestJavaImport
Test: b build '...'
Change-Id: Ib485e6905e7be9d199af5295b23131c226fd6e96
2022-03-31 21:00:24 +00:00
Sam Delmerico
3c25ab1ac3 Merge changes I8e4c3f37,I89a32bde
* changes:
  add plugins to javaLibraryAttributes
  convert java_import_host with bp2build; enable jetifier
2022-03-30 21:08:12 +00:00
Treehugger Robot
d93afba1e6 Merge changes from topic "no-srcs-no-deps"
* changes:
  enable conversion of all java_import modules
  emulate java_library static_deps with Bazel exports
2022-03-30 21:06:46 +00:00
Sam Delmerico
5f83b491b4 convert java_import_host with bp2build; enable jetifier
Also, add java_imports that were blocked by jetifier binary.
b/221082840 was blocking the build of jetifier which these java_imports
depended on. That bug is now resolved, so we can unblock these targets.

Bug: 220168131
Test: b build //packages/apps/Car/libs/car-ui-lib/car-ui-androidx/...
Test: build/bazel/ci/bp2build.sh
Change-Id: I89a32bde30c133a44aeb7826afa11d4140433f5d
2022-03-30 19:19:13 +00:00
Sam Delmerico
a9b047a9da enable conversion of all java_import modules
Bug: 220168131
Test: b build '...'
Change-Id: Ib1eaf47b81ac19cb6e930e5f9f3fc2aebe02b0c1
2022-03-30 19:19:09 +00:00
Sam Delmerico
c016143602 emulate java_library static_deps with Bazel exports
In Soong, java_library can specify static_deps which are dependencies
that get aggregated into the final jar (akin to static linking). This is
useful because it allows dependencies higher up in the chain to compile
against the APIs exported by transitive dependencies. Bazel does not
support this functionality directly, but it can be emulated via the
exports attribute which makes any targets listed in the attribute public
to targets further up the chain.

Bug: 217236083
Bug: 219908977
Test: b build //external/error_prone:error_prone_core
Test: b build //external/bouncycastle:bouncycastle-host
Test: b build --platforms=//build/bazel/platforms:linux_x86
  //prebuilts/sdk/tools/jetifier/jetifier-standalone:jetifier
Change-Id: I2867e3f816de720a6f4bd9ff7a847d1b0c2da2d6
2022-03-30 18:58:38 +00:00
Paul Duffin
26f19919ea analyze_bcpf: Add --fix option
Add a --fix option that will cause the script to automatically fix the
issues that it finds. It uses the bpmodify tool to add values to the
bootclasspath_fragment's hidden_api properties.

This adds analyze_bcpf to bp2buildModuleDoNotConvertList as
analyze_bcpf depends on bpmodify which is a blueprint_go_binary which
is not yet supported by bazel.

Bug: 202154151
Test: m analyze_bcpf && analyze_bcpf --bcpf art-bootclasspath-fragment
      m analyze_bcpf && analyze_bcpf --bcpf art-bootclasspath-fragment --fix
      atest --host analyze_bcpf_test

Change-Id: I5ee52419b4829474f6dbeb47f86ab2aeb22b1382
2022-03-30 16:00:08 +01:00
Sam Delmerico
f241e72b57 Merge "add module type allowlist to bp2build" 2022-03-22 16:35:49 +00:00
Sam Delmerico
85d831a28e add module type allowlist to bp2build
This change also requires that java_import modules be prepended with the
"prebuilt_" prefix in the `bp2buildModuleAlwaysConvertList`. This fixes
a bug where trying to disable a java_import by name without the
"prebuilt_" prefix would also disable the corresponding source module if
it exists with the same name.

Test: build/bazel/ci/bp2build.sh
Change-Id: If75236891810e5112d70441653be4644d8fe5ecf
2022-03-17 23:43:07 +00:00
Jingwen Chen
f2504b64eb Allowlist //external/e2fsprogs/...
Test: CI
Bug: 204244290
Change-Id: Ie18b555fb165cd3b869a3d7fb2c4ad73aef4bad5
2022-03-17 16:52:13 +00:00
Jingwen Chen
53c29e2467 Build e2fsdroid with Bazel.
Test: CI
Bug: 204244290
Change-Id: If5f01da62cff2913fefc98aed7fe7b986e6e7a33
2022-03-17 15:33:38 +00:00
Treehugger Robot
7db79508c7 Merge "bp2build supports arch variant srcs for java_library" 2022-03-14 17:06:01 +00:00
Jingwen Chen
398a32d9b0 Allowlist //external/auto/android-annotation-stubs for bp2build.
Fixes breakage for bp2build-incremental.

Test: b build //external/...
Change-Id: Id493c119b6ba0f383c9d1e8bade8b8a0acf3efe6
2022-03-08 08:03:28 +00:00
Sam Delmerico
e91d0303f2 bp2build supports arch variant srcs for java_library
Bug: 209577426
Test: build/bazel/ci/bp2build.sh
Change-Id: I6799bda904d286616e580f2395601ce4f764180f
2022-03-07 16:15:56 +00:00
Jingwen Chen
7edadabd6c Build avbtool with Bazel.
Test: CI
Change-Id: Ic506fea3ba1fb61692f59d8e6f708115c1a327ab
2022-03-07 04:40:35 +00:00
Wei Li
083d4e072c Merge "Keep Bazel build file in build/bazel/vendor/google and prebuilts/bundletool." 2022-03-05 04:31:55 +00:00
Treehugger Robot
fe86b81aab Merge "prevent module being enabled if in existing BUILD file" 2022-03-04 20:37:21 +00:00
Sam Delmerico
d508a638c3 Merge "convert android_app certificate property, bp2build" 2022-03-04 17:30:31 +00:00
Sam Delmerico
94d26c2bc0 prevent module being enabled if in existing BUILD file
If a module is enabled via the module-level allow list,
bp2buildModuleAlwaysConvertList, then we should error if that same
module is hidden by an existing BUILD file.

Bug: 220875679
Test: add a module in a directory covered by an existing BUILD file
Change-Id: I5426bd9edac593690c0989b68a421ac1c3216010
2022-03-02 22:15:14 +00:00
Wei Li
171410478a Keep Bazel build file in build/bazel/vendor/google and prebuilts/bundletool.
Test: b test //build/bazel/tests/apex:build.bazel.examples.apex.minimal_mainline-module
Bug: 217838178

Change-Id: If94a2086db2a4afabe600b691f65b388ea19dcfe
2022-03-02 02:09:46 -08:00
Liz Kammer
6bd0d4e864 Merge "Disallow libc_musl_sysroot_libc++_headers" 2022-03-01 19:10:16 +00:00
Liz Kammer
878a7f1632 Disallow libc_musl_sysroot_libc++_headers
This depends on go modules and is musl specific, not a current focus.
Allows CI to be green again.

Test: bp2build.sh
Change-Id: I35520636b38eed3f829e176c4ba91517fd61364a
2022-03-01 09:08:23 -05:00
Sam Delmerico
fa1831c200 add a module-level allowlist to bp2build
We currently enabled modules in bp2build on a package level, and we
disable individual modules based on a denylist. To enable a specific
module individually, we must change the corresponding Android.bp file to
include a bp2build_available property. Instead, have a centralized list
of specifically enabled modules.

Bug: 220875679
Test: m bp2build
Change-Id: I908f2f2366b72affb04234fbe92ba03bf5a37338
2022-02-24 14:46:41 +00:00
Sam Delmerico
9dfb139b6d convert android_app certificate property, bp2build
The android_app certificate property can be converted to the
@android_rules debug_signing_keys attribute in Bazel after converting
the .pk8/.pem key pairs to a JKS keystore in Starlark.

Test: b build -s //frameworks/base/tests/appwidgets/AppWidgetHostTest
  and verify includes SignApk action with generated keystore
Bug: 194133023
Change-Id: I2c4276f94a7856fc68a7674e89742f887dca31b4
2022-02-18 20:17:01 +00:00
Yu Liu
9d7d9810e2 Merge "Enable cc_binary in mixed build." 2022-02-17 21:54:04 +00:00
Yu Liu
7f3605fee1 Enable cc_binary in mixed build.
Bug: 216114082
Test: Run mixed build on adbd apex and verify the pavkaged adbd binary;
add unit test.
Change-Id: Id11549a7952a7ce5ac02988b88b747bbfe8339ff

Change-Id: If01efe076c223886278f466b65628602bff3a964
2022-02-17 11:15:59 -08:00
Treehugger Robot
632d0a03ed Merge "Keep build/bazel/ci/dist/BUILD for bp2build." 2022-02-16 16:14:26 +00:00
Jingwen Chen
eb2a3fd367 Add error_prone_checkerframework_dataflow_nullaway. to bp2build denylist.
Bug: 219908977
Change-Id: Ia6a0879d9af7aabadb5cd5722ddfdf72f579e621
Test: CI
2022-02-16 13:03:29 +00:00
Treehugger Robot
948c340160 Merge "enable auto_service_plugin bazel build" 2022-02-09 17:48:49 +00:00
Jingwen Chen
941f0ab807 Keep build/bazel/ci/dist/BUILD for bp2build.
Test: CI
Change-Id: I008672cbf746185c40e7832b70aed6dc564c111c
2022-02-09 13:25:30 +00:00
Jingwen Chen
f6b4cd4bcd Omit musl related cc_genrules from bp2build.
These trigger a few failure modes (duplicate genrule srcs, failure in
matching config conditions, depending on unconverted soong_zip host
tool).

Test: CI
Bug: 218405924
Change-Id: I15c119803a057e94ef6a941df3b58103d42eb4e4
2022-02-08 04:47:48 +00:00