Commit graph

243 commits

Author SHA1 Message Date
Rupert Shuttleworth
b815168474 Support arch variations for export_system_include_dirs in cc_library_headers bp2build converter.
Test: Added unit test
Test: bp2build-sync.py write; bazel build //bionic/... works for more cc_library_static targets (in a parent CL)
Change-Id: Ib487216a4bcbc52958ff948722dae347b0d8b606
2021-04-07 16:05:44 +00:00
Liz Kammer
3f9e155f83 Handle simple prebuilt static libraries from bazel
Test: generate & sync BUILD files via bp2build && mixed build droid
Bug: 184192619
Change-Id: I27f0d76c88cbff25f3c7a805f3dfbb1eeaf8e771
2021-04-07 11:07:36 -04:00
Liz Kammer
f29df7cc52 Move cquery RequestType interface to bazel_handler
go idiom is to define the interface where it is used rather than where
it is defined. This makes it obvious that ParseResult is not a used part
of the interface, removing the need to return an interface{} and cast
results.

Test: go test soong tests
Test: generate & sync bp2build; mixed build libc
Change-Id: I0d8d99c1d8d0125588522cc86502286b83c91bf7
2021-04-07 10:56:22 -04:00
Christopher Parsons
b7c6a7ef36 Merge "Fix note_memtag bionic libraries in mixed builds" 2021-04-06 17:57:59 +00:00
Treehugger Robot
e09691ce78 Merge "Only include used requests in cquery starlark" 2021-04-06 16:05:20 +00:00
Chris Parsons
eefc9e6a62 Fix note_memtag bionic libraries in mixed builds
Required fixes:
  - Set MakeLinkType for libraries, even in mixed build mode.
  - Set snapshot header information to empty list, which passes
        validation logic for depending modules

Fixing these libraries also requires a Starlark change to
cc_library_static.bzl, which will be submitted separately.

Additionally, this adds better error messaging in the event that
output files are missing from a mixed-build library.

Test: USE_BAZEL_ANALYSIS=1 m libc
Test: USE_BAZEL_ANALYSIS=1 m runtime-module-sdk
Change-Id: Iad2c4d46359986fb0a43263292a15ed45fabbac7
2021-04-05 15:27:55 -04:00
Liz Kammer
66ffdb74ee Only include used requests in cquery starlark
Test: go test soong tests
Test: generate & sync bp2build, mixed build libc
Change-Id: I1fbf854cf31f40b7721788956d0f08da3bda2cba
2021-04-05 10:41:05 -04:00
Jingwen Chen
c1c2650532 Add target/os configurable string_list attrs.
Starting with copts for cc_object, with an extracted function that can
be shared with other cc_* module types.

Test: TH
Change-Id: I9025232e83a3dcd0ca243387486fafbdbd3e2d9b
2021-04-05 11:27:19 +00:00
Jingwen Chen
1a6bbbd990 Merge "Add os/target configurable selects for label list attributes." 2021-04-05 07:09:48 +00:00
Treehugger Robot
100d5d6732 Merge "Remove unused cquery requests." 2021-04-02 21:50:40 +00:00
Liz Kammer
5ad66aaf41 Merge "Add depfile handling for bazel_handler." 2021-04-02 17:06:07 +00:00
Liz Kammer
09f9231e91 Remove unused cquery requests.
Test: go test soong tests
Change-Id: Ie7a72bc73ccf4d9b8aa2a0b20c1674d3a5b499de
2021-04-02 11:03:15 -04:00
Jingwen Chen
91220d7334 Add os/target configurable selects for label list attributes.
This CL is pretty large, so I recommend starting with reading the newly
added tests for the expected behavior.

This change works in conjunction with the linked CLs in the Gerrit topic.
Those CLs add support for new platform() definitions for OS targets
specified in Soong's arch.go, which are configurable through
Android.bp's `target {}` property. It works similary to previous CLs
adding support for the `arch {}` property.

These configurable props are keyed by the OS: android, linux_bionic,
windows, and so on. They map to `select` statements in label list
attributes, which this CL enables for cc_library_headers' header_libs
and export_header_lib_headers props.

This enables //bionic/libc:libc_headers to be generated correctly, from:

    cc_library_headers {
        name: "libc_headers",
        target: {
            android: {
                header_libs: ["libc_headers_arch"],
                export_header_lib_headers: ["libc_headers_arch"],
            },
            linux_bionic: {
                header_libs: ["libc_headers_arch"],
                export_header_lib_headers: ["libc_headers_arch"],
            },
        },
        // omitted props
    }

to:

    cc_library_headers(
        name = "libc_headers",
        deps = [] + select({
            "//build/bazel/platforms/os:android": [
                ":libc_headers_arch",
            ],
            "//build/bazel/platforms/os:linux_bionic": [
                ":libc_headers_arch",
            ],
            "//conditions:default": [],
        }),
    )

Test: TH
Test: Verify generated //bionic/libc:libc_headers
Fixes: 183597786

Change-Id: I01016cc2cc9a71449f02300d747f01decebf3f6e
2021-04-02 08:17:34 +00:00
Liz Kammer
acacbc1166 Merge "Handle product_variable asflag for cc_object." 2021-03-31 18:43:55 +00:00
Liz Kammer
de116856fb Add depfile handling for bazel_handler.
Test: go test soong tests
Test: TODO mixed build change header, mixed build
Change-Id: I7c51faf2d5b1a8717cbab6bb0b3eb75c307fcd85
2021-03-30 15:55:02 -04:00
Liz Kammer
a060c4521e Handle product_variable asflag for cc_object.
cc_object crtbrand sets product_variable.platform_sdk_version.asflag
and will not compile correctly within mixed builds without it.

Only handles product_variables that expand product variables.

Bug: 181794963
Test: ~/aosp/build/bazel/scripts/milestone-2/demo.sh full
Change-Id: I293fcb18032aa51f63bb7b3de94abd6d1ec38180
2021-03-30 15:16:43 -04:00
Jingwen Chen
0702791a99 bp2build: arch-configurable selects for label list attrs.
This CL adds the configurable LabelListAttribute support to bp2build.

Test: go test
Change-Id: I2ef9e385d9cf1b1845988128eca1d8cda1ecb5e8
2021-03-30 13:22:48 +00:00
Liz Kammer
a5f86249a4 Merge changes from topic "mixed-bp2build"
* changes:
  Support autoconverted modules in mixed builds
  Use handcrafted build targets in bp2build
2021-03-16 21:40:04 +00:00
Christopher Parsons
bc3f7e0276 Merge "Refactor and cleanup of cquery processing" 2021-03-16 21:12:01 +00:00
Chris Parsons
944e7d01aa Refactor and cleanup of cquery processing
Test: USE_BAZEL_ANALYSIS=1 m libc
Change-Id: Iaf9a92e84d39c132e2444a8aaafd79505a12b8ec
2021-03-16 14:39:16 -04:00
Liz Kammer
ba3ea16f14 Use handcrafted build targets in bp2build
If both bp2build_available and label are specified, label will be
preferred.

Initially, we copy the entire BUILD.bazel file. Eventually we may move
this to use bazel query for a more accurate result.

Test: go test *
Test: build/bazel/scripts/milestone-2/demo.sh full
Test: GENERATE_BAZEL_FILES=true m nothing
      edit bionic/libc/tools/BUILD.bazel
      GENERATE_BAZEL_FILES=true m nothing and verify changes picked up
Bug: 180516554
Change-Id: I43025583300e6b10d2c18032cd4a76237b578d59
2021-03-15 13:09:25 -04:00
Rupert Shuttleworth
2e4219be73 Add a way to remove duplicates from Bazel label lists, to e.g. avoid duplicate deps.
Test: Added unit tests, and also resolves bp2build error.
Change-Id: I6e01d9e233ec21dbb244e5682cbeebbc3a1ac0c4
2021-03-15 11:51:31 +00:00
Chris Parsons
8d6e433c1e Support cc_object modules in mixed builds
Test: With a handwritten conversion of crtbegin_so1, USE_BAZEL_ANALYSIS=1 m crtbegin_so1
Change-Id: I7c777d7f46b37aa1827cc04205e2014f9293bf35
2021-03-01 18:22:34 -05:00
Jingwen Chen
5d8644990b bp2build: add configurable attribute (select) support.
This CL adds a basic framework to support configurable string_list
attributes, selecting on the Arch variant (x86, x86_64, arm, arm64).

It offers fine-grained controls to map individual configurable
properties (arch_variant) to configurable Bazel attributes, starting
with the string_list type for the copts property for cc_object.

This design is primarily motivated to have minimal boilerplate in
bp2build mutators, allowing anyone to opt-in configurable attributes,
and modify intermediate states before passing them on into the
CreateBazelTargetModule instantiator.

Fixes: 178130668

Test: go tests
Test: build/bazel/scripts/milestone-2/demo.sh

Change-Id: Id6f04d7c560312a93e193d7ca4e1b7ceb6062260
2021-02-26 05:17:54 -05:00
Liz Kammer
fc46bc1ee4 Refactor BazelTargetModule
This eliminates the need to remove quotes, delete attributes, and
re-checking that name has correct prefix. Additionally, this allows
assignment directly to the BazelTargetModuleProperties struct, which
allows defaulting unused fields and clarity of which field is being set.

Test: go test soong tests
Test: ran ./build/bazel/scripts/milestone-2/demo.sh
Change-Id: Ia9bfcce76234c793a4ddd5f29a661150f83341c9
2021-02-23 08:29:40 -05:00
Treehugger Robot
b68036f44b Merge "Add LOCAL_LICENSE_KINDS to build/soong" 2021-02-10 07:18:29 +00:00
Jingwen Chen
fb4692a743 bp2build: refactor BazelTargetModule naming boilerplate.
This CL replaces the "__bp2build__" name prefix boilerplate with a props
creation function, and centralizes the prefixing in there.

Test: TH
Test: soong tests
Change-Id: Ic963199ab60dcce0d3361abff111cfa9acd4c21b
2021-02-08 04:10:18 -05:00
Bob Badour
02040de891 Add LOCAL_LICENSE_KINDS to build/soong
Added SPDX-license-identifier-Apache-2.0 to:
  Android.bp
  android/Android.bp
  android/soongconfig/Android.bp
  androidmk/Android.bp
  apex/Android.bp
  bazel/Android.bp
  bp2build/Android.bp
  bpf/Android.bp
  bpfix/Android.bp
  cc/Android.bp
  cc/config/Android.bp
  cc/libbuildversion/Android.bp
  cc/libbuildversion/tests/Android.bp
  cc/ndk_api_coverage_parser/Android.bp
  cc/ndkstubgen/Android.bp
  cc/symbolfile/Android.bp
  cmd/dep_fixer/Android.bp
  cmd/diff_target_files/Android.bp
  cmd/extract_apks/Android.bp
  cmd/extract_jar_packages/Android.bp
  cmd/extract_linker/Android.bp
  cmd/fileslist/Android.bp
  cmd/host_bionic_inject/Android.bp
  cmd/javac_wrapper/Android.bp
  cmd/merge_zips/Android.bp
  cmd/multiproduct_kati/Android.bp
  cmd/path_interposer/Android.bp
  cmd/pom2bp/Android.bp
  cmd/pom2mk/Android.bp
  cmd/sbox/Android.bp
  cmd/soong_build/Android.bp
  cmd/soong_env/Android.bp
  cmd/soong_ui/Android.bp
  cmd/zip2zip/Android.bp
  cmd/zipsync/Android.bp
  cuj/Android.bp
  dexpreopt/Android.bp
  dexpreopt/dexpreopt_gen/Android.bp
  env/Android.bp
  etc/Android.bp
  filesystem/Android.bp
  finder/Android.bp
  finder/cmd/Android.bp
  genrule/Android.bp
  jar/Android.bp
  java/Android.bp
  java/config/Android.bp
  kernel/Android.bp
  linkerconfig/Android.bp
  linkerconfig/proto/Android.bp
  makedeps/Android.bp
  partner/Android.bp
  phony/Android.bp
  python/Android.bp
  python/tests/Android.bp
  remoteexec/Android.bp
  rust/Android.bp
  rust/config/Android.bp
  scripts/Android.bp
  sdk/Android.bp
  sh/Android.bp
  shared/Android.bp
  symbol_inject/Android.bp
  symbol_inject/cmd/Android.bp
  sysprop/Android.bp
  tradefed/Android.bp
  ui/build/Android.bp
  ui/logger/Android.bp
  ui/metrics/Android.bp
  ui/metrics/proc/Android.bp
  ui/status/Android.bp
  ui/terminal/Android.bp
  ui/tracer/Android.bp
  xml/Android.bp
  zip/Android.bp
  zip/cmd/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD to:
  finder/fs/Android.bp
  third_party/zip/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work

Change-Id: Ia47ca14f16b8c9f84f9d533a07e5b00e2c04e8d4
2021-02-06 04:23:21 +00:00
Jingwen Chen
77e8b7b6d2 bp2build: add bazel_module: { bp2build_available } prop.
This CL adds a per-target allowlist to instruct bp2build on which modules it should generate Bazel targets for.

Test: soong tests
Change-Id: I869e66fce405c2c6689b381569b8cc0118cbcf76
2021-02-05 09:26:22 -05:00
Jingwen Chen
1fd14691dd bp2build: Refactor CreateBazelTargetModule API.
This CL refactors the CreateBazelTargetModule API to minimize boilerplate, and to establish a well defined function signature for the expected metadata about a BazelTargetModule.

Test: soong tests

Test: TH
Change-Id: I474ff5a2b0db8deeed49ba4ca73b416ccb494fdd
2021-02-05 09:24:55 -05:00
Liz Kammer
356f7d45c1 bp2build: convert paths/module refs to Bazel label
This currently expands all globs, still need to support converting glob
syntax.

Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
2021-02-04 13:45:56 -05:00
Jingwen Chen
40067de675 bp2build: support Starlark rules and load statements.
This CL adds support to bp2build for declaring the location of the
Starlark rule definition when creating BazelTargetModules. This is
needed for non-native rules that needs to be loaded from .bzl files
somewhere in the tree.

Since load statements are aggregated at the top of the BUILD file, away
from the targets that actually use them, this CL also introduces an
abstraction to group BazelTargets together and compute their load
statements and target string representations separately, allowing load
statements to be decoupled and written into a BUILD file before the
targets themselves.

Test: soong tests
Test: TH
Test: GENERATE_BAZEL_FILES=true m nothing && build/bazel/scripts/bp2build-sync.sh write && bazel cquery //bionic/...
Fixes: 178531760

Test: TH
Change-Id: Ie5f793a00006eb024eaef07ddd9fde7aaefc054e
2021-01-26 22:46:20 -05:00
Christopher Parsons
f6d2b82dde Merge "Support aquery depsets with depth" 2021-01-25 16:13:50 +00:00
Jingwen Chen
7385067640 bp2build: framework for generating BazelTargetModules.
This CL creates the framework necessary for generating
BazelTargetModules from regular Soong Android modules.
BazelTargetModules are code-generated into Bazel targets in BUILD files.

See the follow-up CL for examples of creating filegroup/genrule
BazelTargetModules.

Test: GENERATE_BAZEL_FILES=true m nothing # creates out/soong/bp2build
with no BUILD files, because there are no BazelTargetModules in the
module graph.

Change-Id: I33a96365bd439043b13af6db9e439592e9983188
2021-01-21 22:46:11 -05:00
Chris Parsons
943f243bc2 Support aquery depsets with depth
In Bazel aquery responses, Bazel represents action inputs by preserving
the form of the depset data structure which contains them.
(https://docs.bazel.build/versions/master/skylark/lib/depset.html)
Thus, Soong's aquery handler must appropriately "flatten" this
data structure in cases where the depset consists of multiple levels.

Test: m nothing
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I2ebacac1deea7538eb34ad1975594caae71091a2
2021-01-19 21:50:32 +00:00
Chris Parsons
4f069891ae Improve aquery-related validation and error handling
Also introduce test suite for aquery handling

Test: m nothing
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I2493d42782099ea0b575968fca38bce6f0d59015
2021-01-15 13:53:57 -05:00
Chris Parsons
affbb60baa Process v2 bazel analysis protos in aquery responses
Test: USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I549e136a2706aa555d5a2cf0aa38739e80791826
2021-01-07 16:36:38 -05:00
Chris Parsons
dbcb1ff469 Use aquery to declare bazel actions in the ninja file.
This effectively moves execution of Bazel actions outside of soong_build
and alongside ninja execution of the actual ninja files, whether that be
by ninja or by Bazel itself.

This almost allows for mixed builds and Bazel-as-Ninja-executor to
coexist, but requires hacks explained in b/175307058.

Test: Treehugger
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Test: lunch aosp_flame && USE_BAZEL=1 USE_BAZEL_ANALYSIS=1 m libc,
though this requires a hack of the main BUILD file. See b/175307058

Change-Id: Ia2f6b0f1057e8cea3809de66d8287f13d84b510c
2020-12-15 17:39:23 -05:00
Patrice Arruda
05ab2d0731 Enable bazel profiling in soong_build.
Bazel is executed several times during the execution of soong_build.
For each bazel execution, generate a profile and save under the
BAZEL_METRICS_DIR which is defined in soong_ui.

Bug: b/174479924
Test: * USE_BAZEL_ANALYSIS=1 USE_BAZEL=1 m nothing and checked
        if the cquery and graph build bazel profiles were generated.
      * Verified that the generated bazel profiles were uploaded
        to the local dev metrics pipeline.

Change-Id: I3d20204484dc6c5a1525a5d3eec1d62cfb33535b
2020-12-14 18:48:11 +00:00
Jingwen Chen
30f5aaaa77 Move bazel_module property to a common file, and add it to filegroup.
This enables prototyping against aosp/1441774 to demonstrate mixed
builds with converted BUILD files.

Test: TH Presubmit
Test: USE_BAZEL_ANALYSIS=1 m libc && prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-aosp_cf_x86_auto.ninja -t commands libc | grep bazel-out | wc -l # 2 build actions
Bug: 171263886

Change-Id: I7b5cd0449d043ba26a339a0ef98b562fc62e13c8
2020-11-22 22:01:44 -05:00
Jingwen Chen
7c6089ad95 Integrate bazelenv.sh environment variables into soong_ui environment.
This removes the need to source bazelenv.sh for USE_BAZEL_ANALYSIS, and
unifies mixed builds to use the checked in tools/bazel and bazelrc.

It also unifies all bazel-related output to be in out/bazel.

Without aosp/1502095, this change still requires toplevel_output_directories to be an empty
list, otherwise there'll be this error:

ERROR: Directories specified with toplevel_output_directories should be
ignored and can not be used as sources.

Test: With aosp/1441774: rm -rf out/ && lunch aosp_cf_x86_auto && USE_BAZEL_ANALYSIS=1 m libc && prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-aosp_cf_x86_auto.ninja -t commands libc | grep bazel-out | wc -l # 2 results
Change-Id: I69b217ec88da531415792bb6e04b6a194ca4718d
2020-11-17 18:58:03 -05:00
Chris Parsons
8b77a009e2 Change mixed soong/bazel builds to use USE_BAZEL_ANALYSIS
As a result, one can enable bazel-as-ninja-executor separately from
mixed builds.

Test: Manually verified building image with and without bazelenv.sh.
Change-Id: Ia97806fb41e1de850a80b9483ed8a5ff50d9dab2
2020-10-27 18:59:25 -04:00
Chris Parsons
f3c96efea4 Mixed bazel/soong build prototype for genrule
With this change, bazel_module is a specifiable property on
genrule module definitions. With bazel-enabled mode, soong_build will
defer to Bazel for information on these modules.

source build/soong/bazelenv.sh to enter bazel-enabled mode.

Test: Manually verified on bionic/libc genrules using aosp_cf_x86_phone-userdebug
Change-Id: I3619848186d50be7273a5eba31c79989b981d408
2020-10-08 22:46:23 -04:00