Commit graph

4348 commits

Author SHA1 Message Date
Vinh Tran
a1675a539f Merge changes from topics "bp2build-gensrcs", "bp2build-gensrcs-gendir"
* changes:
  Allowlist integration test for gensrcs
  Modify ConvertWithBp2build mutator to convert gensrcs to Bazel rule
2022-06-02 17:51:48 +00:00
Treehugger Robot
f57b54cda7 Merge "Make sure dist files have license metadata." 2022-06-02 17:37:46 +00:00
Bob Badour
5180438478 Make sure dist files have license metadata.
Bug: 151177513
Bug: 210912771

Test: m droid dist reportmissinglicenses

Change-Id: I0c85f6c49a3e9d9bb3219ed6ddfb939d90f80656
2022-06-01 21:07:25 -07:00
Bob Badour
afd99fd305 Add gen_notice test.
Test: m cts dist reportmissinglicenses
Change-Id: I1d630c3d14d27b7605ab13d204d34b6851a24d33
2022-06-01 17:57:00 -07:00
Vinh Tran
b69fd88773 Allowlist integration test for gensrcs
Tests are in aosp/2108410

Test: b build //build/bazel/examples/gensrcs/...
Bug: 179452413
Change-Id: Iea7da04d92ba7b6ac3d555d5f35765a48a68bf65
2022-06-01 20:28:39 +00:00
Cole Faust
91a2e1020f Allowlist apexer tools
Bug: 204244290
Test: ./build/bazel/ci/bp2build.sh
Change-Id: I6bfe64bd435b721eb61eed2578ec9df02c7583d1
2022-05-31 16:13:46 -07:00
Cole Faust
5317be265a Merge "Add bp2build converter for python protobuf files" 2022-05-31 19:22:32 +00:00
Jingwen Chen
24a060cf88 Denylist f2fscrypt because it's failing to compile for the host.
Bug: 234340806
Test: bp2build.sh (CI)
Change-Id: Id437611f35a387949c664ff4385c464c12ce51f5
2022-05-30 14:19:01 +00:00
Cole Faust
97fd87e4db Merge "Allowlist tools that apexer uses" 2022-05-27 20:21:39 +00:00
Treehugger Robot
0e3d36c582 Merge "Add filegroup srcs to json module graph" 2022-05-27 13:29:54 +00:00
Cole Faust
53b62098d0 Add bp2build converter for python protobuf files
Bug: 196084681
Test: b run //build/bazel/examples/python/protobuf:build_bazel_examples_python_protobuf_main --config=linux_x86_64
Change-Id: I4d806902d262351231f64686a5d24513a25d9749
2022-05-26 15:54:23 -07:00
Liz Kammer
5edc1411ee Add filegroup srcs to json module graph
This will allow us to do json analysis on a filegroup's inputs as they
will now have a corresponding phony action.

Test: m json-module-graph and check module-actions.json
Change-Id: Id1ab681c01689eef06205d4a7fe8cf8b9110c628
2022-05-26 16:14:33 -04:00
Cole Faust
ba95b11146 Allowlist tools that apexer uses
Apexer uses a lot of extra binaries, and most
of them already build fine but haven't been
allowlisted.

After this cl, only aapt2 and sefcontext_compile
will still be needed for apexer to run.

Bug: 233346273
Test: b build --config=linux_x86_64 //build/make/tools/zipalign:zipalign //external/f2fs-tools:make_f2fs //external/f2fs-tools:sload_f2fs //external/erofs-utils:make_erofs
Change-Id: I1b721d998c7d1f89d62fce30f565b8422e2dd74a
2022-05-26 12:54:09 -07:00
Trevor Radcliffe
5731d0fffd Merge "Generate genlex rules from bp2build for cc targets" 2022-05-25 19:38:15 +00:00
Sorin Basca
78ce8c23de Merge "Add experimental option to target Java 17" 2022-05-24 15:12:14 +00:00
Trevor Radcliffe
ef9c900ec3 Generate genlex rules from bp2build for cc targets
This change will cause bp2build to generate genlex targets any
time a .l or .ll file is present in the srcs for a cc target and
add those genlex targets to the srcs attribute of the original
target.

Bug: 207408632
Test: unit tests
Change-Id: I1bce82c9d3c3d458eae1cef547ffae3d6e975134
2022-05-24 14:42:51 +00:00
Liz Kammer
91f10eccc6 Merge "Handle multiple linkages in sdk snapshots" 2022-05-24 14:13:19 +00:00
Sorin Basca
ce720c3663 Add experimental option to target Java 17
Bug: 233029164
Test: EXPERIMENTAL_TARGET_JAVA_VERSION_17=true \
      EXPERIMENTAL_USE_OPENJDK17_TOOLCHAIN=true m
Change-Id: I80a9ba9eb356a460c1036e711e793c2ca6620f53
2022-05-24 13:05:17 +01:00
Treehugger Robot
52edad230a Merge "Add blockers of cc mainline modules to allowlist" 2022-05-23 18:55:01 +00:00
Treehugger Robot
5bec8d41ea Merge "Clean up some bp2build technical debt" 2022-05-23 18:01:04 +00:00
Liz Kammer
96320dfff8 Handle multiple linkages in sdk snapshots
Currently, if the same library is specified for multiple of native_libs,
native_shared_libs, and native_static_libs for different arch/oses,
there can be a few errors:

1. specifying a .so file as `srcs` within a cc_prebuilt_library rather
than being specified only for shared
2. the final type of prebuilt library is dependent on the arch/os

This change introduces:
* an ability for a member type to override the type for specified
  properties
  * checks for a library being used with incompatible member types
  * basing linkage nesting on the member type in addition to variants

  This will ensure that the correct library type is used, regardless of
  the order of iteration over oses/arches, and support nesting linkages
  where necessary but only one linkage variant exists.

Test: soong tests
Test: CI
Change-Id: I81dee013b09b99c34ca6c18f9cfcc12ee56d33d1
2022-05-23 10:51:22 -04:00
Bob Badour
0507921dcf gen_module visibility tests.
Test: m cts dist reportmissinglicenses

Change-Id: Ib2f8d0ad46ffe795d392166a74a7c6309b1246e2
2022-05-20 16:54:13 -07:00
Bob Badour
a5ea2479d3 Fix bug: allow multiple gen_notice modules.
Test: m cts dist reportmissinglicenses

Change-Id: I07963c83eaddfe363a04871e813b56fe7f1465ad
2022-05-20 16:54:00 -07:00
Wei Li
440c66a799 Merge "Change init process of bp2buildAllowlist so the Soong plugin of bp2build allowlist can hookup properly." 2022-05-20 20:36:40 +00:00
Christopher Parsons
489128b8ef Merge "Refactor mixed builds to only take one pass" 2022-05-20 14:13:49 +00:00
Chris Parsons
f874e46153 Refactor mixed builds to only take one pass
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:

1. Queueing calls to Bazel is done in a new mutator instead of a full
   soong_build pass. Normal soong_build flow is interrupted (via a
   functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
   as implementing MixedBuildsBuildable. In this interface, define the
   request that must be queued to Bazel, and then subsequently define
   how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
   improves mixed build performance.

Result:
  A 33% runtime improvement on soong analysis phase with mixed builds.

Caveats:
  C++ BazelHandler handling still remains a bit of a mess; I did what
  I could within this CL's scope, but this may require additional cleanup.

Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.

Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
2022-05-20 10:04:13 -04:00
Paul Duffin
e73d52cb9b Merge "Add custom java_sdk_library info to the SDK info file" 2022-05-20 13:59:19 +00:00
Bob Badour
335a4333fe Merge "Add gen_notice module." 2022-05-19 18:03:09 +00:00
Jingwen Chen
23ca2d2d26 bp2build: keep existing build files for prebuilts/build-tools recursively.
This lets us test https://cs.android.com/android/platform/superproject/+/master:prebuilts/build-tools/tests/BUILD.bazel;l=2;drc=d3dc3d5f20dee2a67dabc1a89dcc9ce93d6f5d44

Test: CI
Change-Id: Ic7c43e3ca655a222ed9113b31761f3cb0d8a45f3
2022-05-19 09:53:56 +00:00
Bob Badour
eef4c1c563 Add gen_notice module.
Refactor notices to support notices for multiple modules.

Enforce visibility and handle missing dependencies.

Bug: 213388645

Change-Id: Id6a81987f087419ad37d0cce57a71e8a7c4cd6e0
2022-05-18 16:38:19 -07:00
Paul Duffin
958806b8c8 Add custom java_sdk_library info to the SDK info file
Previously the SDK info file only contained basic common information
about each member. This change adds support for each member to
provide custom information to add to the info file.

It uses that mechanism to add the following:
* "dist_stem"
* "scopes" object containing:
  * for each scope a:
    "<scope>" object containing:
    * "current_api" - the path within the snapshot for the API's .txt
                      file.
    * "removed_api" - the path within the snapshot for the removed
                      API's .txt file.
    * "latest_api" - the path within the build to the latest finalized
                     API .txt file.
    * "latest_removed_api" - the path within the build to the latest
                             finalized removed API .txt file.

In order to access the latest API files it was necessary to add and
resolve dependencies on the module that makes them available. In order
to do that safely the code for creating the names of the modules was
refactored to avoid duplicating the name creation logic.

Bug: 204763318
Test: m nothing
Change-Id: Ica68abbd2b2c7c2b2b7877b502f96cc89f06fd68
2022-05-16 17:09:10 +00:00
Liz Kammer
be1315eb41 Add blockers of cc mainline modules to allowlist
Test: mixed_droid.sh
Change-Id: If14254d192c67a76e307fa2fc7d732a648879cb5
2022-05-13 19:51:51 -04:00
Liz Kammer
3bf97bd1c0 Clean up some bp2build technical debt
Previously we ran mutators in bp2build mode to add dependencies, now we
look up modules by name directly. Remove workarounds to allow bp2build
mode to not fail when adding/handling dependencies.

Test: m bp2build
Change-Id: Ibf6fd905150cac306e5c395902ef28f609f4df2a
2022-05-13 19:47:48 -04:00
Chris Parsons
0bfb1c0556 Deterministic aquery details in mixed builds
This change constitutes a number of fixes which cause mixed builds to
have deterministic ninja file output:

1. Depsets are identified based on a hash of their contents instead of
   an arbitrary ID integer from Bazel
2. Depset definitions in the ninja file are sorted by the above hashes
3. BuildStatements (action information from Bazel's aquery) are sorted
   by their contents

Test: Ran `USE_BAZEL_ANALYSIS=1 m nothing` three times and verified the
md5sum of out/soong/build.ninja was identical all three runs.
Test: mixed_droid

Change-Id: Iffdf6cc62c31d76fbbfa78726827497516171f4f
2022-05-13 13:45:56 -04:00
Wei Li
d7736ec1b1 Change init process of bp2buildAllowlist so the Soong plugin of bp2build allowlist can hookup properly.
Bug: 216442475
Test: m nothing
Change-Id: I47d768e09aa290e23fb846404686dcf26c8caf41
2022-05-12 23:40:45 -07:00
Treehugger Robot
1c87db50ca Merge "Clean up hard-coded min_sdk_version/apex_available" 2022-05-13 04:56:59 +00:00
Jingwen Chen
b184d3130b bp2build: allowlist system/memory/libmemunreachable
A new dep on this package was added in https://android-review.googlesource.com/c/platform/bionic/+/2092709

Started failing from ab/8573658

Change-Id: If53bc4d75f95aeca9f0dcf007d7ed700860058eb
Test: CI
2022-05-12 00:28:57 +00:00
Treehugger Robot
b53efcfc97 Merge "Switch the namespace tests to fully use test fixtures" 2022-05-11 22:43:28 +00:00
Paul Duffin
0fc6d32c82 Switch the namespace tests to fully use test fixtures
Indented the bp contents to make it easier to differentiate between
them and the directory in which they belong.

Bug: 181070625
Test: m nothing
Change-Id: Iae7495fb7c88769dc688006a41f4d21f57cf03b8
2022-05-11 17:30:58 +00:00
Sam Delmerico
79985819fe convert java_resources with bp2build
Test: b build --platforms=//build/bazel/platforms:linux_x86
  //external/jarjar:jarjar-binary and try to use on a jar
Change-Id: Id6f4e6937687fd575360fbacaeda55c41922636e
2022-05-10 14:32:52 +00:00
Jooyung Han
932d2c1932 Clean up hard-coded min_sdk_version/apex_available
Set min_sdk_version/apex_available for modules used by
com.android.permission APEX.

Bug: 147364041
Bug: 158059172
Test: m
Change-Id: I93ff72ed765354d44a2ccb5105b5f2820f0358f5
2022-05-06 12:21:04 +09:00
Jooyung Han
0ea5277adf Clean up minSdkVersionAllowlist
remove NN modules from the list.

Bug: 158059172
Test: m com.android.neuralnetworks
Change-Id: I40b25a06afd81a8fad44c853cf4ccff41300f87a
2022-05-06 11:06:45 +09:00
Lukács T. Berki
28ba9c472f Merge "Add a test for correctness of C++ compilation." 2022-05-04 12:06:38 +00:00
Lukacs T. Berki
e3487c8848 Add a test for correctness of C++ compilation.
This required the following:

- Adding Platform_base_sdk_extension_version to default soong.variables
- Teaching the symlink tree creation code to understand symlinks
- Making finder.go follow symlinks when requested

Adding yet another knob is unfortunate, but I can't allow that
unconditionally because the Android code base contains a number of
symlinks giving rise to infinite directory trees because they point back
to their parent and this seemed preferable to adding complicated logic
like "follow symlink but if only its fully resolved version does not
point under the source tree".

I could be convinced about the latter, though.

Test: Presubmits.
Change-Id: I453f6b7e5334771f5832c700db00f9d24ed1d82f
2022-05-04 09:12:01 +02:00
Mark Dacek
390df4c181 Merge "Log information for Mixed Builds modules. Test: Output matches expected. https://paste.googleplex.com/5913495636803584?raw Performance evaluated: https://docs.google.com/spreadsheets/d/1X7eOVBKEZUwUWl5i8CDfBo9yUeZrDPXWi2JYO4BEZt4/edit?resourcekey=0-co8crIFW9dpiedhCMkhAgw#gid=0" 2022-05-03 20:04:10 +00:00
MarkDacek
ff851b83b6 Log information for Mixed Builds modules.
Test: Output matches expected. https://paste.googleplex.com/5913495636803584?raw
Performance evaluated: https://docs.google.com/spreadsheets/d/1X7eOVBKEZUwUWl5i8CDfBo9yUeZrDPXWi2JYO4BEZt4/edit?resourcekey=0-co8crIFW9dpiedhCMkhAgw#gid=0

Change-Id: I88780c7cc52a189a72216c5e2e499c96574b3731
2022-05-03 18:11:32 +00:00
Treehugger Robot
a0bab18a47 Merge "Clean up minSdkVersionAllowlist" 2022-05-03 09:00:15 +00:00
Treehugger Robot
a6fcf9fd92 Merge "Remove unused uses_sdks property for apexes" 2022-05-03 03:24:59 +00:00
Jooyung Han
c743039b48 Clean up minSdkVersionAllowlist
These modules have min_sdk_version set. No need to keep them in
allowlist.

Bug: 158059172
Test: m
Change-Id: Id9d3c62f4da941f26212ce69637469d82758c935
2022-05-02 12:38:26 +09:00
Christopher Parsons
f485a95d3a Merge "Preserve depset structure from bazel aquery" 2022-04-29 14:49:31 +00:00