Commit graph

134 commits

Author SHA1 Message Date
Rupert Shuttleworth
54e7841b90 Add cc_library_headers support to bp2build.
Test: Added cc_conversion_test.go.
Change-Id: Id4459d2c2fa02687a374cd9fb25d687e9678218c
2021-02-16 23:07:01 +00:00
Liz Kammer
ebfcf672ef Omit bazel.LabelList attrs if no path is specified
Test: go test build_conversion_test.go
Change-Id: Ibcd81049788e99422569fe74e5f3834df4a05122
2021-02-16 15:00:05 -05:00
Rupert Shuttleworth
fb97fdee6b Move sh_binary tests for bp2build into their own file.
Test: go test passes for refactored tests.
Change-Id: Ie094031f4b1f190aa711a37d40bacb182e277b26
2021-02-11 04:41:38 +00:00
Treehugger Robot
b68036f44b Merge "Add LOCAL_LICENSE_KINDS to build/soong" 2021-02-10 07:18:29 +00:00
Treehugger Robot
6f833664f0 Merge "Add sh_binary support to bp2build converter." 2021-02-09 16:29:55 +00:00
Rupert Shuttleworth
a1a56e8f82 Add sh_binary support to bp2build converter.
Test: Added a unit test in this CL.
Change-Id: I6e9b3c833a257414daf82e603b733e459535d206
2021-02-09 11:53:00 +00:00
Jingwen Chen
bb483cbcf5 bp2build: rename generated files to BUILD
The checked in files are named BUILD.bazel, while bp2build files are
named BUILD.

Test: GENERATE_BAZEL_FILES=true m nothing
Change-Id: If226f581b5075088f774dc4f5f9ee88259821835
2021-02-09 04:26:18 +00:00
Treehugger Robot
198158b9f7 Merge "bp2build: refactor BazelTargetModule naming boilerplate." 2021-02-09 03:21:44 +00:00
Jingwen Chen
ff8f1ce7d7 Merge "bp2build: make generated BUILD files publicly visible." 2021-02-09 00:38:03 +00:00
Treehugger Robot
41905bf733 Merge "bp2build: add bazel_module: { bp2build_available } prop." 2021-02-08 18:14:20 +00:00
Jingwen Chen
033bfa6e29 Merge "queryview: blocklist package module type." 2021-02-08 15:18:19 +00:00
Jingwen Chen
fb4692a743 bp2build: refactor BazelTargetModule naming boilerplate.
This CL replaces the "__bp2build__" name prefix boilerplate with a props
creation function, and centralizes the prefixing in there.

Test: TH
Test: soong tests
Change-Id: Ic963199ab60dcce0d3361abff111cfa9acd4c21b
2021-02-08 04:10:18 -05:00
Jingwen Chen
96af35bec6 queryview: blocklist package module type.
The package module type was recently introduced, and its module name is
the full package path. queryview uses the module name as the eventual
target name in the generated BUILD files, so this causes a crash as seen
in b/179605762.

Workaround this by not generating package modules in the queryview
directory.

Test: m queryview && bazel query --config=queryview //...
Fixes: 179605762
Change-Id: Iac2bd79d02bec47d6271583d106c184fb2f16e68
2021-02-08 00:49:32 -05:00
Bob Badour
02040de891 Add LOCAL_LICENSE_KINDS to build/soong
Added SPDX-license-identifier-Apache-2.0 to:
  Android.bp
  android/Android.bp
  android/soongconfig/Android.bp
  androidmk/Android.bp
  apex/Android.bp
  bazel/Android.bp
  bp2build/Android.bp
  bpf/Android.bp
  bpfix/Android.bp
  cc/Android.bp
  cc/config/Android.bp
  cc/libbuildversion/Android.bp
  cc/libbuildversion/tests/Android.bp
  cc/ndk_api_coverage_parser/Android.bp
  cc/ndkstubgen/Android.bp
  cc/symbolfile/Android.bp
  cmd/dep_fixer/Android.bp
  cmd/diff_target_files/Android.bp
  cmd/extract_apks/Android.bp
  cmd/extract_jar_packages/Android.bp
  cmd/extract_linker/Android.bp
  cmd/fileslist/Android.bp
  cmd/host_bionic_inject/Android.bp
  cmd/javac_wrapper/Android.bp
  cmd/merge_zips/Android.bp
  cmd/multiproduct_kati/Android.bp
  cmd/path_interposer/Android.bp
  cmd/pom2bp/Android.bp
  cmd/pom2mk/Android.bp
  cmd/sbox/Android.bp
  cmd/soong_build/Android.bp
  cmd/soong_env/Android.bp
  cmd/soong_ui/Android.bp
  cmd/zip2zip/Android.bp
  cmd/zipsync/Android.bp
  cuj/Android.bp
  dexpreopt/Android.bp
  dexpreopt/dexpreopt_gen/Android.bp
  env/Android.bp
  etc/Android.bp
  filesystem/Android.bp
  finder/Android.bp
  finder/cmd/Android.bp
  genrule/Android.bp
  jar/Android.bp
  java/Android.bp
  java/config/Android.bp
  kernel/Android.bp
  linkerconfig/Android.bp
  linkerconfig/proto/Android.bp
  makedeps/Android.bp
  partner/Android.bp
  phony/Android.bp
  python/Android.bp
  python/tests/Android.bp
  remoteexec/Android.bp
  rust/Android.bp
  rust/config/Android.bp
  scripts/Android.bp
  sdk/Android.bp
  sh/Android.bp
  shared/Android.bp
  symbol_inject/Android.bp
  symbol_inject/cmd/Android.bp
  sysprop/Android.bp
  tradefed/Android.bp
  ui/build/Android.bp
  ui/logger/Android.bp
  ui/metrics/Android.bp
  ui/metrics/proc/Android.bp
  ui/status/Android.bp
  ui/terminal/Android.bp
  ui/tracer/Android.bp
  xml/Android.bp
  zip/Android.bp
  zip/cmd/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD to:
  finder/fs/Android.bp
  third_party/zip/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work

Change-Id: Ia47ca14f16b8c9f84f9d533a07e5b00e2c04e8d4
2021-02-06 04:23:21 +00:00
Jingwen Chen
1c2317315c bp2build: make generated BUILD files publicly visible.
This is obviously not good build hygiene, but since Soong modules are in a global namespace, making BUILD files fully public will increase our velocity during the initial migration phases.

We'd want to lock visibility down to private by default as soon as possible, but for the purposes of migration, these will be public.

Test: bazel build //bionic/...
Change-Id: I99d62f3f8826712240decd325928435333870352
2021-02-05 09:38:15 -05:00
Jingwen Chen
77e8b7b6d2 bp2build: add bazel_module: { bp2build_available } prop.
This CL adds a per-target allowlist to instruct bp2build on which modules it should generate Bazel targets for.

Test: soong tests
Change-Id: I869e66fce405c2c6689b381569b8cc0118cbcf76
2021-02-05 09:26:22 -05:00
Jingwen Chen
1fd14691dd bp2build: Refactor CreateBazelTargetModule API.
This CL refactors the CreateBazelTargetModule API to minimize boilerplate, and to establish a well defined function signature for the expected metadata about a BazelTargetModule.

Test: soong tests

Test: TH
Change-Id: I474ff5a2b0db8deeed49ba4ca73b416ccb494fdd
2021-02-05 09:24:55 -05:00
Liz Kammer
356f7d45c1 bp2build: convert paths/module refs to Bazel label
This currently expands all globs, still need to support converting glob
syntax.

Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
2021-02-04 13:45:56 -05:00
Jingwen Chen
4d2c0878a3 bp2build: rename method for readability.
Rename GenerateSoongModuleTargets to GenerateBazelTargets to reflect its return type.

Test: TH
Change-Id: I5359c8f4365750790837a9d412946d0b7fae64b7
2021-02-02 07:06:56 -05:00
Jingwen Chen
041b184c92 bp2build: flatten *_defaults properties.
This CL brings the defaults mutators into bp2build, reusing the
implementation of flattening defaults from *_defaults module deps into
the modules themselves.

This is an approach to converge defaults semantics with Bazel, which
doesn't have existing equivalent abstractions, other than macros, which
require more infrastructure to be built.

Test: TH
Test: build_conversion_test.go
Change-Id: Ia1443f8e6c3eb277fa861bb96625425a22cfc6d7
2021-02-01 00:33:30 -05:00
Jingwen Chen
40067de675 bp2build: support Starlark rules and load statements.
This CL adds support to bp2build for declaring the location of the
Starlark rule definition when creating BazelTargetModules. This is
needed for non-native rules that needs to be loaded from .bzl files
somewhere in the tree.

Since load statements are aggregated at the top of the BUILD file, away
from the targets that actually use them, this CL also introduces an
abstraction to group BazelTargets together and compute their load
statements and target string representations separately, allowing load
statements to be decoupled and written into a BUILD file before the
targets themselves.

Test: soong tests
Test: TH
Test: GENERATE_BAZEL_FILES=true m nothing && build/bazel/scripts/bp2build-sync.sh write && bazel cquery //bionic/...
Fixes: 178531760

Test: TH
Change-Id: Ie5f793a00006eb024eaef07ddd9fde7aaefc054e
2021-01-26 22:46:20 -05:00
Jingwen Chen
a42d6417b3 Make bp2buildMutators registration local to TestContext.
The previous implementation relied on the implicit registration of Bp2Build mutators, resulting in test non-hermeticity. Refactor bp2build tests to explicitly specify the bp2build mutators under test.

Test: Soong tests

Test: TH
Change-Id: I9b9674bad1ea533b3bd31b07077a9e02c99b4c1d
2021-01-26 21:57:27 -05:00
Jingwen Chen
885ee7ad86 bp2build: support genrule $(location) and $(locations)
Soong genrules support $(location) and $(locations) cmd variable
shortcuts without labels. The shortcut is for the location of the first
tool module from the concatenation of the tools and tool_files
properties.

Bazel doesn't support this shortcut, so the bp2build mapping needs to
support it.

Documentation:
https://cs.android.com/android/platform/superproject/+/master:build/soong/genrule/genrule.go;l=95-96;drc=316e07c593ab66599c74725cf482dedbb32b2875
Code: https://cs.android.com/android/platform/superproject/+/master:build/soong/genrule/genrule.go;l=236-246;drc=316e07c593ab66599c74725cf482dedbb32b2875

Test: build_conversion_test.go
Test: TH

Change-Id: I8aa98ae460af3a3fbb0a7835572518680dc7ade1
2021-01-26 09:15:01 -05:00
Jingwen Chen
dcc329af9f Implement String() method for CodegenMode.
Test: build_conversion_test.go
Change-Id: I17bf5c9b366b0e7bda59f0b1e18f6272153a6ed7
2021-01-26 09:14:55 -05:00
Jingwen Chen
33832f96e0 bp2build: introduce CodegenMode.
The CodegenMode enum helps to differentiate between bp2build and queryview modes of generating BUILD files from the Soong module graph.

bp2build is used for generating BUILD files from an unconfigured Soong module graph for Soong->Bazel conversion, whereas QueryView is a front-end for querying the existing Soong configured module graph with Bazel query.

Test: go test

Test: TH
Change-Id: I5709f591f1ae08b2770adf272d8ce60c9ee1f46a
2021-01-26 00:30:30 -05:00
Jingwen Chen
f92e14b08d Merge "bp2build: automatically convert all genrules." 2021-01-25 22:18:42 +00:00
Jingwen Chen
467f8b6c46 Merge "queryview: prefix internal attribute names." 2021-01-25 22:18:32 +00:00
Jingwen Chen
288e2babde queryview: prefix internal attribute names.
There's a module type in internal main that uses the module_deps
property, which clashes with QueryView's module_deps attribute. A fix is
to prefix the attribute as soong_module_deps to avoid the conflict.

While we're at this, rename the other internal attribute names to be
consistent as well.

Test: soong tests

Test: m queryview && bazel query //... --config=queryview

Fixes: 178351366
Change-Id: I84961c8f58c0c0fac1a1c2cbc6bc625499ac895e
2021-01-25 04:46:07 -05:00
Jingwen Chen
4e4756d720 bp2build: rename variables for readability.
Test: go test
Change-Id: I103bbed32844bf7d24c63e7d908630294f7ab20a
2021-01-24 21:13:56 -05:00
Jingwen Chen
316e07c593 bp2build: automatically convert all genrules.
Results of `cquery //bionic/...`: https://paste.googleplex.com/5608885982330880?raw

Note that this CL does not contain the global module name mapping to
fully qualified label lookup mechanism, so while the BUILD targets are
queryable, they are not buildable (yet).

Test: GENERATE_BAZEL_FILES=true m nothing && ./build/bazel/scripts/bp2build-sync.sh write && bazel cquery //bionic/...

Change-Id: I6f2316222723d8b612d9b041ca82a188cd05bac9
2021-01-24 20:51:18 -05:00
Jingwen Chen
32b4ece0c3 bp2build: automatically convert all filegroups.
See build_conversion_test.go for expected outputs.

Test: build_conversion_test.go
Test: GENERATE_BAZEL_FILES=true m nothing &&
./build/bazel/scripts/bp2build-sync.sh write && bazel build
//bionic/libc/tools/...
Test: bazel query //... --config=queryview

Change-Id: I3c54b96c0812f1ea4ab2c95da1bff3d7c5cc4006
2021-01-23 02:26:20 -05:00
Jingwen Chen
7385067640 bp2build: framework for generating BazelTargetModules.
This CL creates the framework necessary for generating
BazelTargetModules from regular Soong Android modules.
BazelTargetModules are code-generated into Bazel targets in BUILD files.

See the follow-up CL for examples of creating filegroup/genrule
BazelTargetModules.

Test: GENERATE_BAZEL_FILES=true m nothing # creates out/soong/bp2build
with no BUILD files, because there are no BazelTargetModules in the
module graph.

Change-Id: I33a96365bd439043b13af6db9e439592e9983188
2021-01-21 22:46:11 -05:00
Jingwen Chen
daa54bcbba Implement code-generation step for bp2build.
Implement bp2build codegen as a discrete step that runs after an
alternatively registered pipeline of mutators, instead of a
presingleton.

bp2build codegen requires a Context that supports VisitAllModules and
PathContext, so this CL also makes a BpToBuildWrapperContext that
conforms to PathContext by adding two method implementations.

Test: GENERATE_BAZEL_FILES=true m nothing && bazel query //... --config=bp2build | wc -l # 31433
Test: m queryview && bazel query //... --config=queryview # 63638

Change-Id: I0dd359746584b228046d2d0ff00895f28f9bdfc3
2021-01-19 11:03:36 -05:00
Liz Kammer
2dd9ca422d Refactor queryview.
Splits queryview into queryview and bp2build. The latter runs as a
presingleton (could be converted to a singleton). This prevents needing
to run soong_ui a subsequent time to build the query. Queryview remains
as a separate step to prevent increasing runtime due to this generation
every time Soong runs.

Currently this is running as a presingleton as this gives a translation
of Android.bp files after only LoadHooks have run, no mutators.

Test: go tests
Test: m queryview && bazel query --config=queryview //...
Change-Id: If2ba28c7ef60fbd41f43bda6623d41c8c7d23a1b
2021-01-07 14:34:00 -05:00