Commit graph

4360 commits

Author SHA1 Message Date
Sam Delmerico
a22e2c982d Merge "add multilib data_device_bins properties" 2022-06-08 14:07:11 +00:00
Trevor Radcliffe
74e8df0741 Merge "Enable Mixed Builds for Prebuilt Library" 2022-06-08 13:51:13 +00:00
Trevor Radcliffe
5d6fa4d857 Enable Mixed Builds for Prebuilt Library
This change provides support for mixed builds for
cc_prebuilt_library modules and allowlists some such modules.

Fixes: 225938765
Test: Unit tests, run mixed builds and verify correctness
Change-Id: I04396c79661df6b9a43907859e4f96d0191e8e1b
2022-06-07 20:36:53 +00:00
Romain Jobredeaux
41758c71be Merge "Allow empty zip file in extra files in install command." 2022-06-07 15:14:18 +00:00
Sam Delmerico
cc271e2065 add multilib data_device_bins properties
Some targets need to be able to specify the specific architecture for a
data_device_bin module. This commit adds new properties to allow
specification of first, both, 32, or 64 multilib properties.

Bug: 231448797
Bug: 232408185
Test: go test ./java -run TestDataDeviceBinsBuildsDeviceBinary
Change-Id: I457cf4b1a9ccb28b46042f874c96bd0a87009fab
2022-06-06 20:11:46 +00:00
Usta Shrestha
93a9b9f471 Merge "minor dead code: unused format string" 2022-06-06 17:00:34 +00:00
Treehugger Robot
85f98765d2 Merge "Tool to create license metadata for copied target." 2022-06-03 02:05:02 +00:00
Liz Kammer
62ac8a0707 Merge "Handle restricting to device only builds" 2022-06-02 20:29:27 +00:00
Vinh Tran
a1675a539f Merge changes from topics "bp2build-gensrcs", "bp2build-gensrcs-gendir"
* changes:
  Allowlist integration test for gensrcs
  Modify ConvertWithBp2build mutator to convert gensrcs to Bazel rule
2022-06-02 17:51:48 +00:00
Treehugger Robot
f57b54cda7 Merge "Make sure dist files have license metadata." 2022-06-02 17:37:46 +00:00
Usta Shrestha
2ccdb42098 minor dead code: unused format string
Using URL base64 encoding
(without padding, i.e. with trailing '=' or '==')
hash string width is 256/6 = 43 characters.

file size virtually unchanged,
  out/soong/build.ninja:  7,188,669,041 => 7,186,999,370 ie 1.6M

Bug: N/A
Test: manually verified (e.g. ran `m nothing` successfully)
Change-Id: I166d613e1fd857555da9611d420d6691806571c7
2022-06-02 11:37:20 -04:00
Bob Badour
5180438478 Make sure dist files have license metadata.
Bug: 151177513
Bug: 210912771

Test: m droid dist reportmissinglicenses

Change-Id: I0c85f6c49a3e9d9bb3219ed6ddfb939d90f80656
2022-06-01 21:07:25 -07:00
Bob Badour
5a1fcab110 Tool to create license metadata for copied target.
Bug: 213388645

Test: m copy_license_metadata
Test: m cts dist reportmissinglicenses

Change-Id: I75c196ceca221effb2d6c972e250f5d42854e42f
2022-06-01 18:56:26 -07:00
Bob Badour
afd99fd305 Add gen_notice test.
Test: m cts dist reportmissinglicenses
Change-Id: I1d630c3d14d27b7605ab13d204d34b6851a24d33
2022-06-01 17:57:00 -07:00
Vinh Tran
b69fd88773 Allowlist integration test for gensrcs
Tests are in aosp/2108410

Test: b build //build/bazel/examples/gensrcs/...
Bug: 179452413
Change-Id: Iea7da04d92ba7b6ac3d555d5f35765a48a68bf65
2022-06-01 20:28:39 +00:00
Cole Faust
91a2e1020f Allowlist apexer tools
Bug: 204244290
Test: ./build/bazel/ci/bp2build.sh
Change-Id: I6bfe64bd435b721eb61eed2578ec9df02c7583d1
2022-05-31 16:13:46 -07:00
Liz Kammer
dfeb120324 Handle restricting to device only builds
By default, most module types are only enabled for device. Converting
this behavior and host_supported/device_supported properties allows us
to skip building incompatible targets.

Test: soong tests
Test: bp2build.sh
Change-Id: If1da523b4cc8c4cbf2bb26da063d9923b662cc32
2022-05-31 17:46:48 -04:00
Cole Faust
5317be265a Merge "Add bp2build converter for python protobuf files" 2022-05-31 19:22:32 +00:00
Jingwen Chen
24a060cf88 Denylist f2fscrypt because it's failing to compile for the host.
Bug: 234340806
Test: bp2build.sh (CI)
Change-Id: Id437611f35a387949c664ff4385c464c12ce51f5
2022-05-30 14:19:01 +00:00
Cole Faust
97fd87e4db Merge "Allowlist tools that apexer uses" 2022-05-27 20:21:39 +00:00
Treehugger Robot
0e3d36c582 Merge "Add filegroup srcs to json module graph" 2022-05-27 13:29:54 +00:00
Cole Faust
53b62098d0 Add bp2build converter for python protobuf files
Bug: 196084681
Test: b run //build/bazel/examples/python/protobuf:build_bazel_examples_python_protobuf_main --config=linux_x86_64
Change-Id: I4d806902d262351231f64686a5d24513a25d9749
2022-05-26 15:54:23 -07:00
Liz Kammer
5edc1411ee Add filegroup srcs to json module graph
This will allow us to do json analysis on a filegroup's inputs as they
will now have a corresponding phony action.

Test: m json-module-graph and check module-actions.json
Change-Id: Id1ab681c01689eef06205d4a7fe8cf8b9110c628
2022-05-26 16:14:33 -04:00
Cole Faust
ba95b11146 Allowlist tools that apexer uses
Apexer uses a lot of extra binaries, and most
of them already build fine but haven't been
allowlisted.

After this cl, only aapt2 and sefcontext_compile
will still be needed for apexer to run.

Bug: 233346273
Test: b build --config=linux_x86_64 //build/make/tools/zipalign:zipalign //external/f2fs-tools:make_f2fs //external/f2fs-tools:sload_f2fs //external/erofs-utils:make_erofs
Change-Id: I1b721d998c7d1f89d62fce30f565b8422e2dd74a
2022-05-26 12:54:09 -07:00
Trevor Radcliffe
5731d0fffd Merge "Generate genlex rules from bp2build for cc targets" 2022-05-25 19:38:15 +00:00
Sorin Basca
78ce8c23de Merge "Add experimental option to target Java 17" 2022-05-24 15:12:14 +00:00
Trevor Radcliffe
ef9c900ec3 Generate genlex rules from bp2build for cc targets
This change will cause bp2build to generate genlex targets any
time a .l or .ll file is present in the srcs for a cc target and
add those genlex targets to the srcs attribute of the original
target.

Bug: 207408632
Test: unit tests
Change-Id: I1bce82c9d3c3d458eae1cef547ffae3d6e975134
2022-05-24 14:42:51 +00:00
Liz Kammer
91f10eccc6 Merge "Handle multiple linkages in sdk snapshots" 2022-05-24 14:13:19 +00:00
Sorin Basca
ce720c3663 Add experimental option to target Java 17
Bug: 233029164
Test: EXPERIMENTAL_TARGET_JAVA_VERSION_17=true \
      EXPERIMENTAL_USE_OPENJDK17_TOOLCHAIN=true m
Change-Id: I80a9ba9eb356a460c1036e711e793c2ca6620f53
2022-05-24 13:05:17 +01:00
Treehugger Robot
52edad230a Merge "Add blockers of cc mainline modules to allowlist" 2022-05-23 18:55:01 +00:00
Treehugger Robot
5bec8d41ea Merge "Clean up some bp2build technical debt" 2022-05-23 18:01:04 +00:00
Liz Kammer
96320dfff8 Handle multiple linkages in sdk snapshots
Currently, if the same library is specified for multiple of native_libs,
native_shared_libs, and native_static_libs for different arch/oses,
there can be a few errors:

1. specifying a .so file as `srcs` within a cc_prebuilt_library rather
than being specified only for shared
2. the final type of prebuilt library is dependent on the arch/os

This change introduces:
* an ability for a member type to override the type for specified
  properties
  * checks for a library being used with incompatible member types
  * basing linkage nesting on the member type in addition to variants

  This will ensure that the correct library type is used, regardless of
  the order of iteration over oses/arches, and support nesting linkages
  where necessary but only one linkage variant exists.

Test: soong tests
Test: CI
Change-Id: I81dee013b09b99c34ca6c18f9cfcc12ee56d33d1
2022-05-23 10:51:22 -04:00
Bob Badour
0507921dcf gen_module visibility tests.
Test: m cts dist reportmissinglicenses

Change-Id: Ib2f8d0ad46ffe795d392166a74a7c6309b1246e2
2022-05-20 16:54:13 -07:00
Bob Badour
a5ea2479d3 Fix bug: allow multiple gen_notice modules.
Test: m cts dist reportmissinglicenses

Change-Id: I07963c83eaddfe363a04871e813b56fe7f1465ad
2022-05-20 16:54:00 -07:00
Wei Li
440c66a799 Merge "Change init process of bp2buildAllowlist so the Soong plugin of bp2build allowlist can hookup properly." 2022-05-20 20:36:40 +00:00
Christopher Parsons
489128b8ef Merge "Refactor mixed builds to only take one pass" 2022-05-20 14:13:49 +00:00
Chris Parsons
f874e46153 Refactor mixed builds to only take one pass
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:

1. Queueing calls to Bazel is done in a new mutator instead of a full
   soong_build pass. Normal soong_build flow is interrupted (via a
   functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
   as implementing MixedBuildsBuildable. In this interface, define the
   request that must be queued to Bazel, and then subsequently define
   how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
   improves mixed build performance.

Result:
  A 33% runtime improvement on soong analysis phase with mixed builds.

Caveats:
  C++ BazelHandler handling still remains a bit of a mess; I did what
  I could within this CL's scope, but this may require additional cleanup.

Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.

Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
2022-05-20 10:04:13 -04:00
Paul Duffin
e73d52cb9b Merge "Add custom java_sdk_library info to the SDK info file" 2022-05-20 13:59:19 +00:00
Romain Jobredeaux
1cef629459 Allow empty zip file in extra files in install command.
Test: m
Bug: 229901709
Change-Id: I4c8d1b16f2232d4cda75467615779145d2c7315e
2022-05-19 14:50:27 -04:00
Bob Badour
335a4333fe Merge "Add gen_notice module." 2022-05-19 18:03:09 +00:00
Jingwen Chen
23ca2d2d26 bp2build: keep existing build files for prebuilts/build-tools recursively.
This lets us test https://cs.android.com/android/platform/superproject/+/master:prebuilts/build-tools/tests/BUILD.bazel;l=2;drc=d3dc3d5f20dee2a67dabc1a89dcc9ce93d6f5d44

Test: CI
Change-Id: Ic7c43e3ca655a222ed9113b31761f3cb0d8a45f3
2022-05-19 09:53:56 +00:00
Bob Badour
eef4c1c563 Add gen_notice module.
Refactor notices to support notices for multiple modules.

Enforce visibility and handle missing dependencies.

Bug: 213388645

Change-Id: Id6a81987f087419ad37d0cce57a71e8a7c4cd6e0
2022-05-18 16:38:19 -07:00
Paul Duffin
958806b8c8 Add custom java_sdk_library info to the SDK info file
Previously the SDK info file only contained basic common information
about each member. This change adds support for each member to
provide custom information to add to the info file.

It uses that mechanism to add the following:
* "dist_stem"
* "scopes" object containing:
  * for each scope a:
    "<scope>" object containing:
    * "current_api" - the path within the snapshot for the API's .txt
                      file.
    * "removed_api" - the path within the snapshot for the removed
                      API's .txt file.
    * "latest_api" - the path within the build to the latest finalized
                     API .txt file.
    * "latest_removed_api" - the path within the build to the latest
                             finalized removed API .txt file.

In order to access the latest API files it was necessary to add and
resolve dependencies on the module that makes them available. In order
to do that safely the code for creating the names of the modules was
refactored to avoid duplicating the name creation logic.

Bug: 204763318
Test: m nothing
Change-Id: Ica68abbd2b2c7c2b2b7877b502f96cc89f06fd68
2022-05-16 17:09:10 +00:00
Liz Kammer
be1315eb41 Add blockers of cc mainline modules to allowlist
Test: mixed_droid.sh
Change-Id: If14254d192c67a76e307fa2fc7d732a648879cb5
2022-05-13 19:51:51 -04:00
Liz Kammer
3bf97bd1c0 Clean up some bp2build technical debt
Previously we ran mutators in bp2build mode to add dependencies, now we
look up modules by name directly. Remove workarounds to allow bp2build
mode to not fail when adding/handling dependencies.

Test: m bp2build
Change-Id: Ibf6fd905150cac306e5c395902ef28f609f4df2a
2022-05-13 19:47:48 -04:00
Chris Parsons
0bfb1c0556 Deterministic aquery details in mixed builds
This change constitutes a number of fixes which cause mixed builds to
have deterministic ninja file output:

1. Depsets are identified based on a hash of their contents instead of
   an arbitrary ID integer from Bazel
2. Depset definitions in the ninja file are sorted by the above hashes
3. BuildStatements (action information from Bazel's aquery) are sorted
   by their contents

Test: Ran `USE_BAZEL_ANALYSIS=1 m nothing` three times and verified the
md5sum of out/soong/build.ninja was identical all three runs.
Test: mixed_droid

Change-Id: Iffdf6cc62c31d76fbbfa78726827497516171f4f
2022-05-13 13:45:56 -04:00
Wei Li
d7736ec1b1 Change init process of bp2buildAllowlist so the Soong plugin of bp2build allowlist can hookup properly.
Bug: 216442475
Test: m nothing
Change-Id: I47d768e09aa290e23fb846404686dcf26c8caf41
2022-05-12 23:40:45 -07:00
Treehugger Robot
1c87db50ca Merge "Clean up hard-coded min_sdk_version/apex_available" 2022-05-13 04:56:59 +00:00
Jingwen Chen
b184d3130b bp2build: allowlist system/memory/libmemunreachable
A new dep on this package was added in https://android-review.googlesource.com/c/platform/bionic/+/2092709

Started failing from ab/8573658

Change-Id: If53bc4d75f95aeca9f0dcf007d7ed700860058eb
Test: CI
2022-05-12 00:28:57 +00:00
Treehugger Robot
b53efcfc97 Merge "Switch the namespace tests to fully use test fixtures" 2022-05-11 22:43:28 +00:00