Commit graph

126 commits

Author SHA1 Message Date
Liz Kammer
a5f86249a4 Merge changes from topic "mixed-bp2build"
* changes:
  Support autoconverted modules in mixed builds
  Use handcrafted build targets in bp2build
2021-03-16 21:40:04 +00:00
Christopher Parsons
bc3f7e0276 Merge "Refactor and cleanup of cquery processing" 2021-03-16 21:12:01 +00:00
Chris Parsons
944e7d01aa Refactor and cleanup of cquery processing
Test: USE_BAZEL_ANALYSIS=1 m libc
Change-Id: Iaf9a92e84d39c132e2444a8aaafd79505a12b8ec
2021-03-16 14:39:16 -04:00
Liz Kammer
ba3ea16f14 Use handcrafted build targets in bp2build
If both bp2build_available and label are specified, label will be
preferred.

Initially, we copy the entire BUILD.bazel file. Eventually we may move
this to use bazel query for a more accurate result.

Test: go test *
Test: build/bazel/scripts/milestone-2/demo.sh full
Test: GENERATE_BAZEL_FILES=true m nothing
      edit bionic/libc/tools/BUILD.bazel
      GENERATE_BAZEL_FILES=true m nothing and verify changes picked up
Bug: 180516554
Change-Id: I43025583300e6b10d2c18032cd4a76237b578d59
2021-03-15 13:09:25 -04:00
Rupert Shuttleworth
2e4219be73 Add a way to remove duplicates from Bazel label lists, to e.g. avoid duplicate deps.
Test: Added unit tests, and also resolves bp2build error.
Change-Id: I6e01d9e233ec21dbb244e5682cbeebbc3a1ac0c4
2021-03-15 11:51:31 +00:00
Chris Parsons
8d6e433c1e Support cc_object modules in mixed builds
Test: With a handwritten conversion of crtbegin_so1, USE_BAZEL_ANALYSIS=1 m crtbegin_so1
Change-Id: I7c777d7f46b37aa1827cc04205e2014f9293bf35
2021-03-01 18:22:34 -05:00
Jingwen Chen
5d8644990b bp2build: add configurable attribute (select) support.
This CL adds a basic framework to support configurable string_list
attributes, selecting on the Arch variant (x86, x86_64, arm, arm64).

It offers fine-grained controls to map individual configurable
properties (arch_variant) to configurable Bazel attributes, starting
with the string_list type for the copts property for cc_object.

This design is primarily motivated to have minimal boilerplate in
bp2build mutators, allowing anyone to opt-in configurable attributes,
and modify intermediate states before passing them on into the
CreateBazelTargetModule instantiator.

Fixes: 178130668

Test: go tests
Test: build/bazel/scripts/milestone-2/demo.sh

Change-Id: Id6f04d7c560312a93e193d7ca4e1b7ceb6062260
2021-02-26 05:17:54 -05:00
Liz Kammer
fc46bc1ee4 Refactor BazelTargetModule
This eliminates the need to remove quotes, delete attributes, and
re-checking that name has correct prefix. Additionally, this allows
assignment directly to the BazelTargetModuleProperties struct, which
allows defaulting unused fields and clarity of which field is being set.

Test: go test soong tests
Test: ran ./build/bazel/scripts/milestone-2/demo.sh
Change-Id: Ia9bfcce76234c793a4ddd5f29a661150f83341c9
2021-02-23 08:29:40 -05:00
Treehugger Robot
b68036f44b Merge "Add LOCAL_LICENSE_KINDS to build/soong" 2021-02-10 07:18:29 +00:00
Jingwen Chen
fb4692a743 bp2build: refactor BazelTargetModule naming boilerplate.
This CL replaces the "__bp2build__" name prefix boilerplate with a props
creation function, and centralizes the prefixing in there.

Test: TH
Test: soong tests
Change-Id: Ic963199ab60dcce0d3361abff111cfa9acd4c21b
2021-02-08 04:10:18 -05:00
Bob Badour
02040de891 Add LOCAL_LICENSE_KINDS to build/soong
Added SPDX-license-identifier-Apache-2.0 to:
  Android.bp
  android/Android.bp
  android/soongconfig/Android.bp
  androidmk/Android.bp
  apex/Android.bp
  bazel/Android.bp
  bp2build/Android.bp
  bpf/Android.bp
  bpfix/Android.bp
  cc/Android.bp
  cc/config/Android.bp
  cc/libbuildversion/Android.bp
  cc/libbuildversion/tests/Android.bp
  cc/ndk_api_coverage_parser/Android.bp
  cc/ndkstubgen/Android.bp
  cc/symbolfile/Android.bp
  cmd/dep_fixer/Android.bp
  cmd/diff_target_files/Android.bp
  cmd/extract_apks/Android.bp
  cmd/extract_jar_packages/Android.bp
  cmd/extract_linker/Android.bp
  cmd/fileslist/Android.bp
  cmd/host_bionic_inject/Android.bp
  cmd/javac_wrapper/Android.bp
  cmd/merge_zips/Android.bp
  cmd/multiproduct_kati/Android.bp
  cmd/path_interposer/Android.bp
  cmd/pom2bp/Android.bp
  cmd/pom2mk/Android.bp
  cmd/sbox/Android.bp
  cmd/soong_build/Android.bp
  cmd/soong_env/Android.bp
  cmd/soong_ui/Android.bp
  cmd/zip2zip/Android.bp
  cmd/zipsync/Android.bp
  cuj/Android.bp
  dexpreopt/Android.bp
  dexpreopt/dexpreopt_gen/Android.bp
  env/Android.bp
  etc/Android.bp
  filesystem/Android.bp
  finder/Android.bp
  finder/cmd/Android.bp
  genrule/Android.bp
  jar/Android.bp
  java/Android.bp
  java/config/Android.bp
  kernel/Android.bp
  linkerconfig/Android.bp
  linkerconfig/proto/Android.bp
  makedeps/Android.bp
  partner/Android.bp
  phony/Android.bp
  python/Android.bp
  python/tests/Android.bp
  remoteexec/Android.bp
  rust/Android.bp
  rust/config/Android.bp
  scripts/Android.bp
  sdk/Android.bp
  sh/Android.bp
  shared/Android.bp
  symbol_inject/Android.bp
  symbol_inject/cmd/Android.bp
  sysprop/Android.bp
  tradefed/Android.bp
  ui/build/Android.bp
  ui/logger/Android.bp
  ui/metrics/Android.bp
  ui/metrics/proc/Android.bp
  ui/status/Android.bp
  ui/terminal/Android.bp
  ui/tracer/Android.bp
  xml/Android.bp
  zip/Android.bp
  zip/cmd/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD to:
  finder/fs/Android.bp
  third_party/zip/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work

Change-Id: Ia47ca14f16b8c9f84f9d533a07e5b00e2c04e8d4
2021-02-06 04:23:21 +00:00
Jingwen Chen
77e8b7b6d2 bp2build: add bazel_module: { bp2build_available } prop.
This CL adds a per-target allowlist to instruct bp2build on which modules it should generate Bazel targets for.

Test: soong tests
Change-Id: I869e66fce405c2c6689b381569b8cc0118cbcf76
2021-02-05 09:26:22 -05:00
Jingwen Chen
1fd14691dd bp2build: Refactor CreateBazelTargetModule API.
This CL refactors the CreateBazelTargetModule API to minimize boilerplate, and to establish a well defined function signature for the expected metadata about a BazelTargetModule.

Test: soong tests

Test: TH
Change-Id: I474ff5a2b0db8deeed49ba4ca73b416ccb494fdd
2021-02-05 09:24:55 -05:00
Liz Kammer
356f7d45c1 bp2build: convert paths/module refs to Bazel label
This currently expands all globs, still need to support converting glob
syntax.

Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
2021-02-04 13:45:56 -05:00
Jingwen Chen
40067de675 bp2build: support Starlark rules and load statements.
This CL adds support to bp2build for declaring the location of the
Starlark rule definition when creating BazelTargetModules. This is
needed for non-native rules that needs to be loaded from .bzl files
somewhere in the tree.

Since load statements are aggregated at the top of the BUILD file, away
from the targets that actually use them, this CL also introduces an
abstraction to group BazelTargets together and compute their load
statements and target string representations separately, allowing load
statements to be decoupled and written into a BUILD file before the
targets themselves.

Test: soong tests
Test: TH
Test: GENERATE_BAZEL_FILES=true m nothing && build/bazel/scripts/bp2build-sync.sh write && bazel cquery //bionic/...
Fixes: 178531760

Test: TH
Change-Id: Ie5f793a00006eb024eaef07ddd9fde7aaefc054e
2021-01-26 22:46:20 -05:00
Christopher Parsons
f6d2b82dde Merge "Support aquery depsets with depth" 2021-01-25 16:13:50 +00:00
Jingwen Chen
7385067640 bp2build: framework for generating BazelTargetModules.
This CL creates the framework necessary for generating
BazelTargetModules from regular Soong Android modules.
BazelTargetModules are code-generated into Bazel targets in BUILD files.

See the follow-up CL for examples of creating filegroup/genrule
BazelTargetModules.

Test: GENERATE_BAZEL_FILES=true m nothing # creates out/soong/bp2build
with no BUILD files, because there are no BazelTargetModules in the
module graph.

Change-Id: I33a96365bd439043b13af6db9e439592e9983188
2021-01-21 22:46:11 -05:00
Chris Parsons
943f243bc2 Support aquery depsets with depth
In Bazel aquery responses, Bazel represents action inputs by preserving
the form of the depset data structure which contains them.
(https://docs.bazel.build/versions/master/skylark/lib/depset.html)
Thus, Soong's aquery handler must appropriately "flatten" this
data structure in cases where the depset consists of multiple levels.

Test: m nothing
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I2ebacac1deea7538eb34ad1975594caae71091a2
2021-01-19 21:50:32 +00:00
Chris Parsons
4f069891ae Improve aquery-related validation and error handling
Also introduce test suite for aquery handling

Test: m nothing
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I2493d42782099ea0b575968fca38bce6f0d59015
2021-01-15 13:53:57 -05:00
Chris Parsons
affbb60baa Process v2 bazel analysis protos in aquery responses
Test: USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I549e136a2706aa555d5a2cf0aa38739e80791826
2021-01-07 16:36:38 -05:00
Chris Parsons
dbcb1ff469 Use aquery to declare bazel actions in the ninja file.
This effectively moves execution of Bazel actions outside of soong_build
and alongside ninja execution of the actual ninja files, whether that be
by ninja or by Bazel itself.

This almost allows for mixed builds and Bazel-as-Ninja-executor to
coexist, but requires hacks explained in b/175307058.

Test: Treehugger
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Test: lunch aosp_flame && USE_BAZEL=1 USE_BAZEL_ANALYSIS=1 m libc,
though this requires a hack of the main BUILD file. See b/175307058

Change-Id: Ia2f6b0f1057e8cea3809de66d8287f13d84b510c
2020-12-15 17:39:23 -05:00
Patrice Arruda
05ab2d0731 Enable bazel profiling in soong_build.
Bazel is executed several times during the execution of soong_build.
For each bazel execution, generate a profile and save under the
BAZEL_METRICS_DIR which is defined in soong_ui.

Bug: b/174479924
Test: * USE_BAZEL_ANALYSIS=1 USE_BAZEL=1 m nothing and checked
        if the cquery and graph build bazel profiles were generated.
      * Verified that the generated bazel profiles were uploaded
        to the local dev metrics pipeline.

Change-Id: I3d20204484dc6c5a1525a5d3eec1d62cfb33535b
2020-12-14 18:48:11 +00:00
Jingwen Chen
30f5aaaa77 Move bazel_module property to a common file, and add it to filegroup.
This enables prototyping against aosp/1441774 to demonstrate mixed
builds with converted BUILD files.

Test: TH Presubmit
Test: USE_BAZEL_ANALYSIS=1 m libc && prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-aosp_cf_x86_auto.ninja -t commands libc | grep bazel-out | wc -l # 2 build actions
Bug: 171263886

Change-Id: I7b5cd0449d043ba26a339a0ef98b562fc62e13c8
2020-11-22 22:01:44 -05:00
Jingwen Chen
7c6089ad95 Integrate bazelenv.sh environment variables into soong_ui environment.
This removes the need to source bazelenv.sh for USE_BAZEL_ANALYSIS, and
unifies mixed builds to use the checked in tools/bazel and bazelrc.

It also unifies all bazel-related output to be in out/bazel.

Without aosp/1502095, this change still requires toplevel_output_directories to be an empty
list, otherwise there'll be this error:

ERROR: Directories specified with toplevel_output_directories should be
ignored and can not be used as sources.

Test: With aosp/1441774: rm -rf out/ && lunch aosp_cf_x86_auto && USE_BAZEL_ANALYSIS=1 m libc && prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-aosp_cf_x86_auto.ninja -t commands libc | grep bazel-out | wc -l # 2 results
Change-Id: I69b217ec88da531415792bb6e04b6a194ca4718d
2020-11-17 18:58:03 -05:00
Chris Parsons
8b77a009e2 Change mixed soong/bazel builds to use USE_BAZEL_ANALYSIS
As a result, one can enable bazel-as-ninja-executor separately from
mixed builds.

Test: Manually verified building image with and without bazelenv.sh.
Change-Id: Ia97806fb41e1de850a80b9483ed8a5ff50d9dab2
2020-10-27 18:59:25 -04:00
Chris Parsons
f3c96efea4 Mixed bazel/soong build prototype for genrule
With this change, bazel_module is a specifiable property on
genrule module definitions. With bazel-enabled mode, soong_build will
defer to Bazel for information on these modules.

source build/soong/bazelenv.sh to enter bazel-enabled mode.

Test: Manually verified on bionic/libc genrules using aosp_cf_x86_phone-userdebug
Change-Id: I3619848186d50be7273a5eba31c79989b981d408
2020-10-08 22:46:23 -04:00