Commit graph

463 commits

Author SHA1 Message Date
Sam Delmerico
24c5603815 refactor bp2build module allowlists into struct
To make testing easier, refactor existing module-global variables into a
struct that can be mocked.

Test: build/bazel/bp2build.go
Change-Id: I9d177677644ea743641a745b1839a3a8b29f902a
2022-04-14 13:31:05 +00:00
Treehugger Robot
b6e1614272 Merge "convert logtags in java srcs" 2022-04-08 22:37:22 +00:00
Treehugger Robot
a2a7b57924 Merge changes from topic "export-vars"
* changes:
  export Java variables to Bazel
  refactor Bazel variable export
2022-04-08 17:31:23 +00:00
Sam Delmerico
24da73c032 convert logtags in java srcs
Some java_* modules contain .logtag files in their srcs, but we should
separate these out into their own targets.

Test: enabled QuickSearchBox and build
Change-Id: I595722ca59686d450ff522987a15ce283d3607e0
2022-04-08 14:43:18 +00:00
Sam Delmerico
932c01cf9e export Java variables to Bazel
Test: build/bazel/bp2build.sh
Change-Id: Ia06f9265c9f96e6add6edbd0cee925fe37b430d3
2022-04-08 14:15:43 +00:00
Treehugger Robot
ef03b3d8ba Merge "support errorprone javacflags" 2022-04-05 17:46:08 +00:00
Sam Delmerico
489831660c convert java_import jars with arch variants
Bug: 220168131
Test: go test ./bp2build -run TestJavaImport
Test: b build '...'
Change-Id: Ib485e6905e7be9d199af5295b23131c226fd6e96
2022-03-31 21:00:24 +00:00
Sam Delmerico
58614c0414 support errorprone javacflags
Soong has a separate property specifically for errorprone javacflags
that we need to add to the Bazel javacopts.

Test: go test ./bp2build
Change-Id: I5299f45793d2b05a6a71b94dd17ceb797f407846
2022-03-31 14:38:07 +00:00
Sam Delmerico
3c25ab1ac3 Merge changes I8e4c3f37,I89a32bde
* changes:
  add plugins to javaLibraryAttributes
  convert java_import_host with bp2build; enable jetifier
2022-03-30 21:08:12 +00:00
Treehugger Robot
d93afba1e6 Merge changes from topic "no-srcs-no-deps"
* changes:
  enable conversion of all java_import modules
  emulate java_library static_deps with Bazel exports
2022-03-30 21:06:46 +00:00
Sam Delmerico
77267c7d9c add plugins to javaLibraryAttributes
Test: go test ./bp2build
Change-Id: I8e4c3f373c62c60ea5013cd8c585915a366c1f81
2022-03-30 19:19:13 +00:00
Sam Delmerico
c016143602 emulate java_library static_deps with Bazel exports
In Soong, java_library can specify static_deps which are dependencies
that get aggregated into the final jar (akin to static linking). This is
useful because it allows dependencies higher up in the chain to compile
against the APIs exported by transitive dependencies. Bazel does not
support this functionality directly, but it can be emulated via the
exports attribute which makes any targets listed in the attribute public
to targets further up the chain.

Bug: 217236083
Bug: 219908977
Test: b build //external/error_prone:error_prone_core
Test: b build //external/bouncycastle:bouncycastle-host
Test: b build --platforms=//build/bazel/platforms:linux_x86
  //prebuilts/sdk/tools/jetifier/jetifier-standalone:jetifier
Change-Id: I2867e3f816de720a6f4bd9ff7a847d1b0c2da2d6
2022-03-30 18:58:38 +00:00
Treehugger Robot
6876d5aa5f Merge "Dedupe libraries in both static and whole static" 2022-03-26 00:32:01 +00:00
Treehugger Robot
1c2d87ba83 Merge "Add metrics collection to soong_build" 2022-03-26 00:21:21 +00:00
Chris Parsons
715b08fd82 Add metrics collection to soong_build
Metrics are added to a materialized protobuf soong_build_metrics.pb

Recorded events have scoped names and recorded runtimes.
For example, mixedbuilds includes the following events:

mixedbuilds.prepare
mixedbuilds.analyze.write_files
mixedbuild     (this is a total of all mixedbuild events)

Test: Manually verified materialized protos for mixedbuild, bp2build,
and legacy build.

Change-Id: I2e72d4502c0373bee0279bb1320b2e4dd586129c
2022-03-25 17:19:44 -04:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
c78604e6a4 Use prebuilt_file for prebuilt_{etc,usr_share}
Bug: 215723302
Test: TestPrebuiltEtcNoSubdir
Test: bp2build.sh
Test: mixed_{libc,droid}.sh
Change-Id: I006bfe12beb770e11153c52eddf34a6ec7f9e363
2022-03-21 12:29:26 -04:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
b81f77ed24 Abstract prebuilt_etc into prebuilt_file
Bug: 215723302
Test: bp2build.sh
Test: mixed_{libc,droid}.sh
Change-Id: I0dab7a6209320e31cfdfe1ce28cacd4f37c11024
2022-03-21 11:21:59 -04:00
Liz Kammer
cc2c1efc9a Dedupe libraries in both static and whole static
In Soong, libraries can appear in both static and whole static libs;
however, in Bazel, this results in errors due to duplicate libraries.

Test: bp2build.sh
Change-Id: I9dda4f7a7812340d28a6160d0e25f421c9db6612
2022-03-21 09:03:29 -04:00
Yu Liu
fc603167f4 Support tailoring clang --target flag based on min sdk version.
Bug: 215748260
Test: Tested using aquery to verify the --target flag; add added unit

Change-Id: Ifb13509db7d1e110316aa44d55ec9cccaa4b83d1
2022-03-18 01:16:52 +00:00
Treehugger Robot
7db79508c7 Merge "bp2build supports arch variant srcs for java_library" 2022-03-14 17:06:01 +00:00
Treehugger Robot
b3bdf38453 Merge changes Ic3ef7557,If75daf11
* changes:
  add all existing bp2build tests to CI
  remove unconverted strip_import_prefix property
2022-03-07 20:50:21 +00:00
Sam Delmerico
d3a1cfa6d1 add all existing bp2build tests to CI
Test: go test ./bp2build
Change-Id: Ic3ef75573cda9ed10c74bf12860269922e197d60
2022-03-07 18:26:33 +00:00
Sam Delmerico
7a62909f35 remove unconverted strip_import_prefix property
The requirements for this property changed in aosp/1985613.

Test: go test ./bp2build
Change-Id: If75daf11c857cdd8a54c119e073e327cddf8543d
2022-03-07 16:28:50 +00:00
Sam Delmerico
e91d0303f2 bp2build supports arch variant srcs for java_library
Bug: 209577426
Test: build/bazel/ci/bp2build.sh
Change-Id: I6799bda904d286616e580f2395601ce4f764180f
2022-03-07 16:15:56 +00:00
Sam Delmerico
881d72022a bp2build java_libraries depend on Android SDK
Soong adds an implicit dependency on the Android SDK for all
java_libraries (and related modules). This feature becomes apparent
after converting arch-variant srcs for android_apps, so we should suport
this for now. Eventually, we will want to replace this forced dependency
with a Bazel toolchain workaround.

Test: build/bazel/ci/bp2build.sh
Change-Id: Ic28e8c7690f69294b4bdcb9bb78a6f1f031fe97e
2022-03-07 15:50:42 +00:00
Liz Kammer
e8303bd0ae Export variant/features via soong_injection
Test: build/bazel/ci/bp2build.sh
Change-Id: If70043dc9d020d5f4a243ba4b192b99c9c7e8563
2022-02-28 14:17:15 -05:00
Sam Delmerico
c768102bce convert java proto libraries with bp2build
Allow java_libraries that depend on protobufs to be converted with
bp2build.

Bug: 215230097
Test: build/bazel/ci/bp2build.sh
Change-Id: I3ce52389e7e4e82755605ee277c1e527a6aebc6b
2022-02-15 21:04:59 +00:00
Liz Kammer
7756c8f20e Correct bp2build canonical_from_root logic
The logic was inverted from what it should have been,
canonical_from_root --> no strip_import_prefix attribute.

Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_droid.sh
Change-Id: Ic6685d8f0b88279d4444bab3b5e03a544d225f77
2022-02-15 09:54:57 -05:00
Treehugger Robot
9deb87e8e2 Merge "Escape cc ldflags in bp2build conversion" 2022-02-09 23:05:01 +00:00
Treehugger Robot
5974423b06 Merge "Add package for printing starlark formatted data" 2022-02-09 04:48:33 +00:00
Liz Kammer
72beb34609 Add package for printing starlark formatted data
Bug: 216168792
Test: build/bazel/ci/bp2build.sh
Change-Id: I3a06b19396f7ffe1c638042cda7e731dd840f1d6
2022-02-08 17:32:28 -05:00
Yu Liu
75be7b94d6 Support bssl hash injection for libcrypto.
Bug: 215541108
Test: Build libcrypto with Bazel and verify the hash injected, add unit
test
Change-Id: I0375307aab0b1e60a8be298d680e1e0af1ebb454

Change-Id: I9133ce398e2024fe7834a292bf2f5ba3222d630b
2022-02-07 14:15:42 -08:00
Liz Kammer
f38a8379ff Escape cc ldflags in bp2build conversion
Bug: 217757720
Test: bp2build.sh
Change-Id: I704acfb01527f15ff762afeaaf4e0e328e4737d8
2022-02-04 15:43:08 -05:00
Sam Delmerico
75539d62ae add target_compatible_with stanza for host targets
Soong modules that are specific for the host platform (e.g.
java_library_host, cc_binary_host, java_genrule_host, etc.) should
not be built on the target platform (Android), so we add a
target_compatible_with attribute to skip this type of module on an
Android target build.

Bug: 215229742
Test: go test ./bp2build
Change-Id: Ifb76ef4e0dc4cb3adb6a64b5c375ce36f7973e48
2022-02-03 19:25:12 +00:00
Sam Delmerico
c06ea03c9e convert java_plugin with bp2build
Also, because java_plugin is only intended to be built for the host
platform to make additions to the javac compiler, a compatibility
requirement is added to java_plugin that ensure that it is not built
for android targets.

Bug: 215229742
Test: go test ./bp2build
Change-Id: I09902215a9f1e8b14d4de5a51328d08a7a0a4450
2022-02-01 22:59:19 +00:00
Romain Jobredeaux
428a366703 Basic bp2build converter for java_import.
This only supports the `jars` property/attribute.

Test: b build external/error_prone:error_prone_core_jars
Bug: 215229744
Change-Id: If0d9c8c4e9c1c560d35a501c20bef9361ef45e15
2022-01-29 09:44:37 -05:00
Wei Li
f034cb4d7b Add bp2build support for property "compressible" of apex module.
Bug: 214466457
Bug: 207551677
Test: b build //build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal
Test: b test //build/bazel/tests/apex:build.bazel.examples.apex.minimal_apex
Test: b test //build/bazel/tests/apex:build.bazel.examples.apex.minimal_capex

Change-Id: I6bf12c1b0c52d4486968bb430a67a3c3110766db
2022-01-25 19:42:43 +00:00
Liz Kammer
fe813c53a5 Merge "Delete toolchain_library" 2022-01-24 21:45:36 +00:00
Romain Jobredeaux
f49ca9e40d Merge "Revert "Revert "Add static_libs support to android_app bp2build converter.""" 2022-01-21 20:51:06 +00:00
Sam Delmerico
925a30496a Merge "bp2build converts java_genrule and java_genrule_host" 2022-01-21 19:43:27 +00:00
Sam Delmerico
cd1b80f067 bp2build converts java_genrule and java_genrule_host
Bp2build currently supports building genrules for cc modules, but does
not support building java_genrule* modules. This commit adds this
functionality.

Bug: 213480907
Test: go test ./bp2build
Change-Id: I473196c5bcf5582ba0c8faa65b5005f81ac973a4
2022-01-21 17:50:40 +00:00
Chris Parsons
492bd91a51 Add bp2build metrics per module type
For example, new metrics at HEAD highlight that 99 of 1379 total
cc_library_static modules are converted.

Test: m bp2build, printproto
Change-Id: I6cc4227124e9a130b75911f3e40e6585d731d00a
2022-01-21 11:52:24 -05:00
Romain Jobredeaux
355851b520 Revert "Revert "Add static_libs support to android_app bp2build converter.""
This reverts commit b33e0b8b5b.

Reason for revert: CI failure deemed to be a fluke

Change-Id: Ia804d531092921e4ad090740420c4a4e02e6ff00
2022-01-19 21:13:51 +00:00
Jingwen Chen
539d41b686 Merge "Support multilib in apex." 2022-01-19 11:11:05 +00:00
Yu Liu
4ae55d17f6 Support multilib in apex.
Bug: b/208325023
Test: Added unit tests, also tested with adbd apex build manually.
Change-Id: I47e04cd4eb5d05227f0a84683dcb66dff00e3514
2022-01-18 10:38:05 -08:00
Jingwen Chen
0ee88a6b65 Implement stubs.symbol_file and stubs.versions for cc_library_shared bp2build.
This CL turns the stubs.symbol_file and stubs.versions properties into
stubs_symbol_file and stubs_version attributes on the cc_shared_library
target. See associated build/bazel change on how these attributes are
used to generate stub libraries.

Bug: 207812332

Test: New tests
Test: CI
Change-Id: Ie23eafb9903a131d92ff4e251215e998cea0a763
2022-01-17 13:03:12 +00:00
Liz Kammer
718eb27f5b Delete toolchain_library
Test: m && ci
Change-Id: Iab5267e02e3559431d980c70a58b49c2b8b4e6de
2022-01-14 12:52:59 -05:00
Treehugger Robot
8f5399889b Merge "Start unblocking com.android.runtime deps" 2022-01-13 19:57:01 +00:00
Romain Jobredeaux
8a9a97b4a0 Merge "Revert "Add static_libs support to android_app bp2build converter."" 2022-01-13 17:03:26 +00:00
Romain Jobredeaux
b33e0b8b5b Revert "Add static_libs support to android_app bp2build converter."
This reverts commit de722acd96.

Reason for revert: broke bp2build CI

Change-Id: I535e8746e846c20f1a47c2a1eb0362276af49206
2022-01-13 15:16:12 +00:00
Romain Jobredeaux
24431448b2 Merge "Add static_libs support to android_app bp2build converter." 2022-01-13 15:00:44 +00:00
Sam Delmerico
443703ab32 Merge "Support enabled flag in product variable config" 2022-01-12 20:34:14 +00:00
Sam Delmerico
0e33c9d772 Support enabled flag in product variable config
Some Android.bp modules have `enabled: false` but only use a product
variable such as `source_build` to enable the module.  Currently b2build
does not handle this case at all. This commit adds the functionality
to support this use case.

Also, remove `__enabled` suffix in ProductVariable SelectKey.

Bug: 210546943
Test: go test ./bp2build
Topic: use_enabled_flag_product_variable_config
Change-Id: I459c17a84c172df010666391066bf4d11d19253e
2022-01-12 16:26:44 +00:00
Romain Jobredeaux
de722acd96 Add static_libs support to android_app bp2build converter.
With this change we can allow list WifiDirectDemo, as its dependencies
happen to have manual BUILD file conversions.

Test: ./build/bazel/scripts/run_presubmits.sh
Change-Id: I68b4c2b129fe750913f013384c87a5332fdcc9b6
2022-01-07 16:47:14 -05:00
Sam Delmerico
4e27229b61 Convert javacflags -> javacopts for java_binary_host
This has already been done for java_library modules, we need to add
similar support for java_binary_host.

Bug: 213456294
Test: go test ./bp2build
Change-Id: Idf7b631f3afe9f59e50f9e45bcbdc29fb5923cc6
2022-01-06 21:00:46 +00:00
Liz Kammer
daa09efb5e Start unblocking com.android.runtime deps
Adds support to warn rather than error about missing deps in order to
suport modules in build/soong with deps outside of build/sooong in
bp2build integration tests.

Test: build/bazel/ci/bp2build.sh
Change-Id: I1282bccd37a3fc9f33555d34e68d7f0873d8272c
2022-01-06 10:40:43 -05:00
Liz Kammer
5430953c82 bp2build: Remove duplicate system shared libs
If a system shared lib is specified in shared_libs, this results in
duplicate values appearing, causing a failure in Bazel. This change
removes any system shared libraries that appear in shared libraries from
bionic OS axes where system_shared_libraries takes the default value.

Test: go soong tests
Test: temporarily allowlist directory with this issue, no longer hits
      duplicate library failure.
Change-Id: I9dce570b73c24973f695b815bce8d50f7259798d
2021-12-22 15:53:13 -05:00
Christopher Parsons
948e851b83 Merge "Handle the 'enabled' property in bp2build" 2021-12-22 15:50:10 +00:00
Chris Parsons
58852a05f3 Handle the 'enabled' property in bp2build
Also fix some bugs pertaining to configurable attribute handling of bool
attributes and label sttributes, so that they may support values across
multiple different axes at the same time.

Test: unit tests for bp2build
Test: mixed_droid

Change-Id: I411efcfddf02d55dbc0775962068a11348a8bb2c
2021-12-21 16:37:32 -05:00
Liz Kammer
48c6ead90d sh_binary: Use pointers for optional attributes
Empty strings for filename and sub_dir were unnecessicarily always being
propagated to BUILD files.

Test: build/bazel/ci/bp2build.sh
Change-Id: I39d6af12b1decd1d3813b92daa862ebc025daa7d
2021-12-21 16:01:53 -05:00
Liz Kammer
cac7f690eb bp2build remove "-std" from cflags
Soong overrides "-std" flags when provided via cppflags or conlyflags;
however, any user-provided "-std=" cflag will be overridden by Soong's
std flag handling.

For Bazel, we _always_ allow user to override via user provided flags.
To prevent conflicts, we remove the silently ignored values from
Android.bp files in the bp2build conversion.

Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_droid.sh
Change-Id: I4c33b2ae593a7ff3ff8e3ad15ef3461354fc0c83
2021-12-21 11:49:01 -05:00
Liz Kammer
aabfb5dc47 bp2build: Expand check if filegroup contains proto
Previously we looked for proto as suffix, however, some filegroups use
"<foo>-proto-srcs" or "<bar>-proto-sources", instead we look for proto
as a distinct word in a filegroup name.

Test: go test soong tests
Change-Id: Icf916a84304a02617efff9768e5b82d5ffe658bd
2021-12-15 13:07:58 -05:00
Wei Li
2115d35101 Merge "Add the initial implementation of bp2build converter for java_library, java_library_host, java_binary_host and cc_library_host_shared so signapk tool can be built with Bazel." 2021-12-15 03:39:51 +00:00
Wei Li
bafb6d6612 Add the initial implementation of bp2build converter for java_library,
java_library_host, java_binary_host and cc_library_host_shared so
signapk tool can be built with Bazel.

Test: b build //build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal
Test: jarsigner -verify -verbose build.bazel.examples.apex.minimal.apex
Bug: 209876137
Bug: 196204358
Bug: 210158864
Bug: 210159074
Bug: 210158872
Change-Id: I855884159d25e69d8f9623792c376da820a1eb4c
2021-12-14 17:23:19 -08:00
Christopher Parsons
f793ba068c Merge "Add missing test files to build/soong" 2021-12-14 21:56:26 +00:00
Chris Parsons
1de1d4ee02 Add missing test files to build/soong
Test: m nothing
Change-Id: Icd331f0c2ac09caed0d4f885dd2eb6ec2080b681
2021-12-14 15:25:51 -05:00
Liz Kammer
92344259c5 Merge "Use one mutator for all bp2build conversion." 2021-12-14 19:10:10 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
8bb0d8f55d Merge "Log bp2build_metrics .pb" 2021-12-14 16:30:29 +00:00
Liz Kammer
be46fccc40 Use one mutator for all bp2build conversion.
Each conversion required defining a separate mutator, which will each
operate on _all_ modules and requires each to repeat checks whether the
mutator should operator. Instead, we introduce a single mutator and
modules can define a ConvertWithBp2build to implement bp2build
conversion for that module.

Test: bp2build.sh
Bug: 183079158
Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
2021-12-14 09:37:45 -05:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
947fdbfdee Log bp2build_metrics .pb
Also share `Save(pb proto.Message, filepath string)`

Bug: 201539536
Test: bp2build_metrics.pb has expected content & path
Test: m nothing
Test: {bp2build,mixed_{libc,droid}}.sh
Test: CI
Change-Id: I7d8ad87fca6a4b0355010090a527f5ae67b27c88
2021-12-13 13:03:44 -05:00
Liz Kammer
1263d9bdb1 Handle includes for generated headers
For generated headers, Soong introduces the package to the includes (and
re-exports as necessary).

Test: bp2build.sh
Change-Id: Iea9cda859ddaa3e5393f0f4533c15fc60222c408
2021-12-13 12:06:39 -05:00
Rupert Shuttleworth
484aa25875 Add initial support for use_version_lib.
Test: Resolves build error for BP2BUILD_VERBOSE=1 b build //packages/modules/adb:libadbd_core
Test: Added unit test

Bug: 208481704

Change-Id: I4022c686c7baf050de3f97295fe4654515622a66
2021-12-13 06:21:16 -05:00
Jingwen Chen
8d0bd355fd Merge "bp2build: Fix apex.binaries to be a LabelListAttribute." 2021-12-13 10:03:50 +00:00
Liz Kammer
8f3da38b70 Merge "Add test for performance testing" 2021-12-09 20:46:56 +00:00
Chris Parsons
91b81f0b50 Print number of converted modules
Previously this was equal to the number of generated targets, but with
recent cc_library changes, we now sometimes generate more than one BUILD
target for a single module. Thus, converted module count is a more
useful metric.

At time of writing, this outputs:
`Converted 390 Android.bp modules to 453 total generated BUILD targets.
Included 4 handcrafted BUILD targets. There are 45093 total Android.bp
modules.`

Test: m bp2build
Change-Id: I7d68880dd4c6bf649ca753837a7f3a9c0d73753a
2021-12-08 11:39:07 -05:00
Jingwen Chen
b07c901e0d bp2build: Fix apex.binaries to be a LabelListAttribute.
apex.binaries accepts a list of module names, which should be resolved to
their fully qualified labels. Bazel treats string_list and label_list
attributes differently, most notably with the latter adding dependency edges.

Test: apex_conversion_test.go
Bug: 209743852
Change-Id: Iafdc5c728e8658cce0e99d42f32e7bb6fe2f3168
2021-12-08 10:10:19 +00:00
Liz Kammer
95b278757e Add test for performance testing
This is to ensure benchmarking infra continues working.

Test: go test bp2build tests
Change-Id: I20927652cf427548f6ff58a0c0cb8d3fa5df34d7
2021-12-07 16:59:40 -05:00
Romain Jobredeaux
4526269c2e Merge "Initial bp2build converter for android_app." 2021-12-07 20:14:29 +00:00
Romain Jobredeaux
1282c42484 Initial bp2build converter for android_app.
The only supported attributes at this point are:
- srcs
- manifest
- package_name
- resource_dirs
as they most easily map to bazel's android_binary's srcs, manifest, custom_package, and resource_files respectively.
Allow-listing all apps that use these fields, along with sdk_version and dex_preopt. The latter 2 are ignored by the converter,
- sdk_version because we're currently relying on a single pre-built SDK,
- dex_preopt because,
   1. though it is not supported in Bazel builds yet, it doesn't prevent the apps from building, and
   2. the apps being converted only use the dex_preopt attribute to disable dex_preopt, which is what is happening anyway.

Change-Id: I4a4f771eeb8f60a1cd4844b2ac1ce3df7c070e73
Test: ./build/bazel/scripts/run_presubmits.sh
Bug: 198224074
Bug: 203688791
2021-12-07 09:41:49 -05:00
Chris Parsons
77acf2e5c1 bp2build: split full_cc_library into shared/static
Test: mixed_droid.sh in conjunction with rule changes
Test: bp2build.sh
Change-Id: If0577065fd39a0446eab16b62c15204d43207e19
2021-12-06 14:11:40 -05:00
Liz Kammer
85c0097d69 Merge "bp2build: support full/lite protos in cc libs" 2021-12-06 14:46:34 +00:00
Yu Liu
0a84e93b7f Merge "Support filename and sub_dir attributes in sh_binary" 2021-12-03 01:04:04 +00:00
Yu Liu
b302455879 Support filename and sub_dir attributes in sh_binary
Test: test by building system/timezone/apex:com.android.tzdata
Change-Id: I08114910fccbdacf6750e237d3e80ba37fde7651
2021-12-02 14:42:20 -08:00
Liz Kammer
12615dbbca bp2build: support full/lite protos in cc libs
Test: bp2build.sh
Bug: 200601772
Change-Id: I3a7e00546726bc63b5eb8d5604557c5988a5320b
2021-12-02 11:00:46 -05:00
Liz Kammer
46fb7aba4d Support empty strings in bp2build
Previously, could not set an empty string as a value of an attribute;
however, this is necessary in some cases. To not unnecessarily create an
empty string, use string pointers for attributes rather than strings.

Test: go test bp2build tests
Change-Id: I03b3a3567452d455246d22d81f86c317d06b7c39
2021-12-01 10:14:49 -05:00
Chris Parsons
79bd2b7761 bp2build implementation for c_std
Test: mixed_droid in conjunction with topic changes
Change-Id: Ic673c1b269f5082b490b32057eb60f3b73eb0940
2021-11-29 18:38:31 -05:00
Jingwen Chen
01812020c1 Add support for writing all Soong config variables into @soong_injection.
Also remove the need to use bp2build_available on
soong_config_module_types as we want to convert every single of them
into the tree for a complete soong_injection soong_config_variables.bzl
file.

The variables are split into their bool, value and string types
respectively, as they all need to be handled differently on the Bazel
product_platform side, as well as for generating constraint values and
settings. For example, value variables need to integrate with
TemplateVariableInfo, and string variables need to include the string
value itself into the select key/constraint value.

Sample soong_config_variables.bzl file: https://gist.github.com/jin/cef700bfb20c8656a931306dd71d47e1

Test: CI
Bug: 198556411
Change-Id: I8665dd1269a507edb37de62407ed3641564bea5c
2021-11-23 08:37:30 +00:00
Jingwen Chen
58ff6801f4 Fix bp2build select generation for inter-attribute soong config
variable usage.

There's bug a in the current soong_config_variable handling
implementation where a soong_config_variable sets conditions_default
value for an attr, and a non-conditions_default value for another attr.
This results in the former attr not properly setting the zero value for
the non-conditions_default select key, resulting in the pretty printer
omitting the attribute totally.

The current implementation in this CL ensures that the zero value is set
whenever this happens at the module level. This is seen in
library_linking_strategy_cc_defaults (see comments in code, and the new
tests)

Test: CI
Bug: 198556411

Change-Id: Ibaeb94508c51a7429fb7a08df610cbb5470f76d2
2021-11-23 08:37:30 +00:00
Jingwen Chen
84817de033 Add support for merging defaults soong_config_module_types into bp2build
select statements.

This supports defaults from the same or different namespaces, and
transitively defaults as well.

Test: soong unit tests
Test: CI
Change-Id: I99435bacfcfbfe20ad753b8021a1779531d7595a
2021-11-23 08:37:30 +00:00
Treehugger Robot
59a5cb072b Merge "Refactor ProductConfigProperties to use a struct key instead of an string key with hardcoded patterns." 2021-11-16 04:55:15 +00:00
Jingwen Chen
25825ca08d Refactor ProductConfigProperties to use a struct key instead of an
string key with hardcoded patterns.

This fixes a bug with label list conditions_default attrs where the
attribute values get clobbered in a map with the keys
"conditions_default" (with a default empty list) and
"acme__feature__conditions_default" (with a non-empty list) when
generating the LabelListAttribute.

Test: CI
Change-Id: I5429e40f747b7a0ed559f8a468a4831cd32df2c0
2021-11-16 02:50:22 +00:00
Liz Kammer
78cfdaa597 Refactor bp2build tests
Moves to specifying attributes as a map, such at it is possible to add
additional attributes conditionally. This is in particular useful once
supporting the `enabled` property which will add
`target_compatible_with`

Test: go test soong tests
Change-Id: Iade8eed1ce3acb1d1712a9ee3119d9ae59675624
2021-11-15 11:09:05 -05:00
Jingwen Chen
a47f28d28e bp2build: add support for soong_config_module_type.
Test: CI, go unit test
Bug: 198556411
Change-Id: Idf862904d51d822f92af0c072341c31b7a02fc64
2021-11-08 13:38:28 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
7fa0696eb7 cc_prebuilt_library_shared converted via bp2build
Bug: b/203699063
Test: bp2build/cc_prebuilt_library_shared_test.go
Test: USE_BAZEL_ANALYSIS=1 m nothing
Test: mixed_{libc,droid}.sh
Change-Id: I4c58224e88c31507a4b285f8c55bdc066bd47232
2021-10-28 11:04:50 -04:00
Liz Kammer
e6583482a8 bp2build: Handle export_generated_header property
The generated_header property resides in BaseCompilerProperties, while
export_generated_header resides in BaseLinkerProperties. Previously
bp2build handled these property structs separately; however, these two
related properties residing in separate structs requires restructuring
the code to allow access to both BaseCompilerProperties and
BaseLinkerProperties for a single axis/configuration combination to
resolve which generated headers are exported/not for the
axis/configuration..

Test: go test soong tests
Test: build/bazel/ci/bp2build.sh
Change-Id: Id150003637fd19d87e8dc5d6941e9f36dc4031dd
2021-10-26 15:10:41 -04:00
Liz Kammer
2dafba7da9 Merge "bp2build: Add support for cc_binary." 2021-10-20 14:02:39 +00:00
Liz Kammer
2ec92fc0a0 Merge "bp2build include generated hdrs in all src splits" 2021-10-19 20:51:00 +00:00
Liz Kammer
2b8004b1ef bp2build: Add support for cc_binary.
Bug: 197920036
Test: build/bazel/ci/bp2build.sh
Change-Id: I2c4200967653af15a330ab8cbaf796b70d43f32d
2021-10-19 14:12:39 -04:00
Liz Kammer
ae3994ed4c bp2build include generated hdrs in all src splits
Currently generated headers are only included in the srcs split;
however, if c or assembly sources depend on the headers, the
dependendency is lacking and can cause non-deterministic failures.

Test: build/bazel/ci/bp2build.sh
Change-Id: Ic354c88300a5fb1b85241f68889e904c33a81488
2021-10-19 14:09:49 -04:00
Chris Parsons
3b1f83d076 Add x86_host vars to config.bzl
Test: USE_BAZEL_ANALYSIS=1 m adbd
Test: Manually verified config.bzl contains various x86_host flags after
bp2build.
Test: Unit tests

Change-Id: Ie9201ea2be4cd1c6659bea088a797cedbae37403
2021-10-19 12:18:13 -04:00
Liz Kammer
51da93381c Merge "Update version_script to be handled as linkopt" 2021-10-15 14:22:27 +00:00
Liz Kammer
fdd72e6351 bp2build: Add special arch-variant targets.
Soong supports additional arch-variant targets based on arch/os aside
from strictly arch/os names. Handle the unconverted cases based solely
on arch/os.

Test: build/bazel/ci/bp2build.sh
Change-Id: I8dc987b1aa4a4ea646dd5b03a94b84297ba6113a
2021-10-13 09:01:56 -04:00
Liz Kammer
d2871189b1 Update version_script to be handled as linkopt
This allows us to handle this consistently between cc_binary and
cc_library* types.

Test: build/bazel/ci/bp2build.sh
Change-Id: I996f42bbe591215217c3d561662e775925b871ff
2021-10-13 08:36:58 -04:00
Liz Kammer
06f00c33a7 Merge "Do not sort after subtraction." 2021-10-13 12:27:48 +00:00
Jingwen Chen
e466cc7ea6 Merge "Convert cpp_std to be an attribute instead of a copt." 2021-10-12 23:13:53 +00:00
Liz Kammer
9bad9d645e Do not sort after subtraction.
This allows labels/strings to remain in their original order.

Test: go test bazel tests
Change-Id: I69f575df9e4a358fee4392ae48edf4550e463efb
2021-10-12 13:36:27 -04:00
Liz Kammer
7e1956643c Merge "bp2build: Improve handling of generated_sources" 2021-10-12 17:29:23 +00:00
Liz Kammer
222bdcff5c bp2build: Improve handling of generated_sources
Test: build/bazel/ci/bp2build.sh
Change-Id: Id5f8b6ae9bbf1e90d72854daafb8b1f86f7241ea
2021-10-11 14:15:51 -04:00
Jingwen Chen
5b11ab1a69 Convert cpp_std to be an attribute instead of a copt.
This attribute is then converted into a feature in cc_library_static.

Fixes: 202518741
Test: CI
Change-Id: I070b56a1e96680ffad0466d085caaab2e9308ebc
2021-10-11 17:44:33 +00:00
Liz Kammer
2649c7913a Merge "bp2build; Update handling of linker flags" 2021-10-08 20:09:32 +00:00
Jingwen Chen
97b8531492 Add bp2build support for cpp_std.
This converts cpp_std and gnu_extensions into a -std copt, if cpp_std is
specified or gnu_extensions is false if cpp_std is not specified.

I chose to go with this copts approach because the tradeoff is a much
simpler setting than adding a new attr(s) everywhere that uses features
to set the flag.

This approach limits the number of user-configurable knobs (since users
would then be able to set std in _both_ copts and the new attr). But it
does rely on the user copt overriding the toolchain's default gnu++17
version, which can mean a `-std` flag showing up twice in the action.

Fixes: b/202462232
Test: b build //system/libziparchive:libziparchive
Change-Id: I81dad029059461739b91f318d662e089edb46b84
2021-10-08 11:16:20 +00:00
Liz Kammer
0eae52e0de bp2build; Update handling of linker flags
Test: build/bazel/ci/bp2build.sh
Bug: 197920036
Change-Id: I6e3100574fa0e40bcd8cf0e6af0efd3310aa41bf
2021-10-07 16:07:40 -04:00
Jingwen Chen
6ada589f6e Add support for nocrt by translating it to link_crt in bp2build.
If nocrt is true, then the compilation for cc_shared_library,
cc_binary (shared or static binaries) will _not_ link against their
respective crtbegin and crtend libraries.

nocrt is true only for the Bionic libraries themselves. For everything
else that links against the Bionic runtime, crtbegin and crtend
libraries are used. This makes the "nocrt: false" case the majority.
Hence, if nocrt is explicitly false, we omit the generating attribute in
bp2build.

If nocrt is explicitly true (link_crt is false), the Starlark macro will
disable the link_crt cc_toolchain feature.

Test: new tests
Test: CI
Fixes: 187928070
Fixes: 197946668
Change-Id: I8947789930e599dc802d8eae440859257d044475
2021-10-07 14:04:39 +00:00
Liz Kammer
ba26bdc316 Merge "Handle arch-variant in prebuilt_etc" 2021-10-06 15:36:57 +00:00
Liz Kammer
dff00ea509 Handle arch-variant in prebuilt_etc
For cc_binary, changing the handling of version_script to no longer be a
LabelAttribute, which reduces test coverage of that type. Expand
prebuilt_etc to handle arch-variant srcs and correct a bug in
LabelAttribute.

Test: build/bazel/ci/bp2build.sh
Change-Id: I620bc3a43b28c1873f335e30099844ca79ce94c1
2021-10-04 13:44:34 -04:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
447f6c99c9 Bp2Build common properties auto-handling
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule

Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.

As initial user, the Soong `required` property will be
translated to Bazel's `data`.

Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
2021-10-04 14:43:04 +00:00
Christopher Parsons
d6558d15bc Merge "bp2build: Handle target.linux" 2021-10-01 21:54:16 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
19d399d4c5 Have python_*{,_host} handle arch-variants
Bug: 196081778
Test: TestPython*{,Host}ArchVariance
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: I89304e58f5bacd61534732bade4ad6bb5f2671c0
2021-10-01 19:51:02 +00:00
Chris Parsons
2dde0cb3de bp2build: Handle target.linux
Also refactor target.bionic to be handled not as its own configuration
axis, but instead to be grouped into os selects handling.

This allows us to remove libbase and its dependencies from the bp2build
denylist.

Test: mixed_droid.sh CI
Change-Id: I92f30074d286306207653fe37589835ae3db16c4
2021-10-01 14:56:39 -04:00
Chris Parsons
a37e195182 Support building libcrypto via mixed builds
This required the following fixes to bp2build:
  - Correctly handle Bionic_* and Linux_* targets
  - Correctly handle cc_object's selects
  - Generate linker_script, stl, and system_dynamic_deps for cc_object in bp2build

Test: USE_BAZEL_ANALYSIS=1 m adbd
Change-Id: I753fd18df8ae551fb69df07e4174527c5388f289
2021-09-30 12:30:36 -04:00
Liz Kammer
46f8dbe06a Merge "Refactor code for partitions c srcs" 2021-09-24 17:35:16 +00:00
Liz Kammer
57e2e7a78f Refactor code for partitions c srcs
To support protos (and other srcs that generate sources), we need to
partition further. Separate out into a separate common function.

Bug: 200601772
Test: build/bazel/ci/bp2build.sh
Change-Id: I7bf4cd96fd9a9fca4ccb3c96f21a04303201f891
2021-09-23 18:28:24 -04:00
Liz Kammer
7a210ac233 bp2build: Add support for export_.*headers props
Soong supports export_.*_headers properties, the libraries contained in
this list must also be within a shared/static/whole_static/header libs
property. For bp2build, we eliminate this duplication. The libraries
not listed in an export_.*_headers property will migrate to an attribute
prepended with implementation_, those in export_.*_headers will not have
a prefix.

Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Bug: 198241472
Change-Id: I3eb84c983ec5d241c8a568e411dfd5619d3184a7
2021-09-23 16:07:45 -04:00
Wei Li
bcd399479a Add bp2build converter for cc_genrule.
Test: cc_genrule_conversion_test.go

Change-Id: I19290b417d6336020a15ba7fa772ee0c76c58225
2021-09-23 05:41:20 +00:00
Jingwen Chen
be27f964f5 Merge "bp2build: Refactor metrics collection to use exported functions." 2021-09-22 14:16:12 +00:00
Christopher Parsons
ce0944fc96 Merge "bp2build: reference static variants directly" 2021-09-21 15:13:43 +00:00
Chris Parsons
953b35623a bp2build: reference static variants directly
Previously, when referencing a cc_library as a static_lib from a
converted module, the corresponding target would depend on the "root
target" of that cc_library. This resulted in a superfluous dependency on
the shared target of the cc_library dependency. This superfluous
dependency could sometimes result in cycles.

This change ensures the static target name is directly specified in such
cases.

Test: mixed_libc.sh CI
Test: USE_BAZEL_ANALYSIS=1 m fmtlib
Change-Id: I36ee9f0a9017b1e9d73df9e3174669107f0afd4f
2021-09-20 15:15:29 -04:00
Jingwen Chen
310bc8f5f4 bp2build: Refactor metrics collection to use exported functions.
This CL splits the metric count into handcrafted, generated, and
unconverted modules.

Test: CI
Test: m bp2build, the metrics reported are identical.
Change-Id: I80e3d2fd2989222bd40b6433b4a1924b4943d0ba
2021-09-20 11:54:00 +00:00
Jingwen Chen
afb84bd78f Fix bp2build-progress generating an empty converted_modules list, and
duplicate module names.

We were using the value receiver on AddConvertedModule before, which
didn't actually mutate the internal slice state.

Test: Run bp2build-progress.py
Change-Id: Ic60f8f6214a91dc129abf62c57dd0b85efffb283
2021-09-20 10:31:46 +00:00
Jingwen Chen
6117450ead bp2build: add converted modules to codegen metrics, and remove the
compat layer.

The list of converted modules is used for tracking bp2build progress.

The module_name_to_label aliases weren't really adding much value to
mixed builds, so let's drop the feature for now and simplify the code.

Test: CI
Test: USE_BAZEL_ANALYSIS=1 m droid
Change-Id: I3269344cfe6d1c72c108d66c6e820bc0ec19bb34
2021-09-17 08:47:16 +00:00
Jingwen Chen
16d90a8954 Remove __bp2build__ prefix trimming.
This is no longer needed after r.android.com/1792714

Test: CI
Change-Id: I808c3124cf1a4f3ebe5d25e41f346a818c3cc57c
2021-09-17 07:16:59 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
ac5097fcf4 Create bp2build converter for cc_shared_library
Factor out shareable code with cc_static_library

Remove `Linkstatic` field for shared cases,
because it's redundant with the rule

Bug: 198241907
Test: cc_library_shared_conversion_test.go
Test: mixed_{libc,droid}.sh
Change-Id: I5be3b66f812893bce41edb425fbd143ab22d9051
2021-09-16 15:00:56 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
a5474e36c5 Remove Linkstatic field for static cases
With the explicit implementation of cc_library_shared,
this is now redundant for both, so remove it.

Test: cc_library_static_conversion_test.go
Test: mixed_{libc,droid}.sh
Change-Id: I6e7733ca08c254562e95e70d93c9f519719d8a18
2021-09-16 15:00:47 +00:00
Liz Kammer
83d07a4c41 Merge "Expand handling of unconverted deps in bp2build" 2021-09-15 13:49:48 +00:00
Liz Kammer
adfade58a9 Merge "Bp2build: handle embedded structs as blueprint" 2021-09-15 12:39:29 +00:00
Liz Kammer
6eff323206 Expand handling of unconverted deps in bp2build
Support three options for converting modules with unconverted
dependencies
1. (default) Warn when converting a module if it has unconverted deps.
2. Error when encountering a module with unconverted deps. (not hooked
up yet)

Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Test: BP2BUILD_ERROR_UNCONVERTED=1 build/bazel/ci/bp2build.sh with
      unconverted deps -- get appropriate error
Bug: 181155349
Change-Id: Ifaabf0cd2e43e963366dc137159c705294165c3d
2021-09-14 19:55:55 +00:00
Liz Kammer
32a0339340 Bp2build: handle embedded structs as blueprint
For structs that are embedded, Blueprint does not nest under the
embedded name, flattening them into the original struct for blueprint
files (e.g.
9fd2ed93df:proptools/unpack_test.go;l=402-431;drc=3adb2409648d6f8b25354ac47f083dae87731f10).
We should do the same for bp2build.

This will also allow us to embed structs for bp2build conversion
allowing more reuse.

Test: go test bp2build tests
Change-Id: I9ce088462adaf59bffa80bea76cd488e31f98e9d
2021-09-14 14:41:36 -04:00
Liz Kammer
35687bc77a Split local/absolute include into attributes
Previously these were expanded into copts, requiring making all includes
absolute and duplicating includes to account for potentially generated
files. We now can handle both of these properly on the Bazel side, so
let's clean up build files a bit.

Test: bp2build.sh
Change-Id: I6c6160738cd6c269408c6c7a37010654d84f3c9d
2021-09-13 12:49:42 +00:00
Liz Kammer
8337ea4e7d Cleanup bp2build tests to reduce repetition
Test: go test bp2build tests
Change-Id: Ia96a1a8ed58d5be13fedf31e1667670efdfe3aef
2021-09-10 10:14:09 -04:00
Liz Kammer
5fad501aeb bp2build: Split export_{includes,system_includes}
The specification of exporting includes vs system includes has an impact
on inclusion sort order. Conflating the two caused some symbols to not
be resolved correctly.

Bug: 198403271
Test: build/bazel/ci/bp2build.sh
Test: USE_BAZEL_ANALYSIS=1 m libbacktrace_no_dex succeeds with libc++_*
      modules removed from mixed build denylist (would fail otherwise)
Change-Id: I08aff253d8962dc678ed10214b1c171330e0fe19
2021-09-09 15:59:05 -04:00
Treehugger Robot
4011ebb12d Merge "Cleanup unused code for bp2build conversion." 2021-09-09 13:12:04 +00:00
Rupert Shuttleworth
c5fa306162 Move genrule tests to genrule_conversion_test.go.
Test: go test

Change-Id: Ice1ed23d008d26cce0a4feca5bfcab51febcea1b
2021-09-09 06:19:02 -04:00
Jingwen Chen
5146ac02a1 bp2build: don't generate filegroups (or error) if it contains a file
with the same name.

Also add capability to test for errors raised in bp2build mutators /
contexts.

This CL does two things to filegroups:

1) If the filegroup has only 1 source file with the same name as itself,
don't generate a filegroup target. Instead, dependents will depend
directly on the Bazel file target instead.

2) If the filegroup has more than 1 source file and 1 of them has the
same name as itself, the bp2build mutator will error out. If bp2build
on CI passes, it means that the source tree / product we're testing
against does not have such a case (which seems to be true for most
source trees).

Either way, this will allow us to be unblocked for most of the errant
filegroups (case 1) in the tree.

Test: CI
Test: New test cases in filegroup_conversion_test.go

Fixes: 194762573
Change-Id: I830c53efc8808569afe3c5f9f08436855bcdafed
2021-09-09 07:12:13 +00:00
Liz Kammer
7cbf7738f2 Cleanup unused code for bp2build conversion.
This code to support bp2build-specific modules is no longer necessary as
we do not create bp2build modules after https://r.android.com/1792714

Test: build/bazel/ci/bp2build.sh
Change-Id: I20d3c42f7e3516554cfb537fef05ba40d222dbeb
2021-09-07 13:19:21 -04:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
560cb6685c python_{binary,library}{,_host} handle lib property
Have binary and library bp2build convert `libs` to
`deps` for bazel

Bug: 196083751
Test: python_{binary,library}_conversion_test.go
Test: build/bazel/ci/mixed_{libc,droid}.sh
Change-Id: I2d5f6ef2e83dd608910edb7adb2eb9a56d25293c
2021-08-27 14:10:33 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
dc212c0b24 Add python_library_host -> py_library bp2build support
Bug: 196081770
Test: bp2build/python_library_conversion_test.go
Test: build/bazel/ci/mixed_{libc,droid}.sh
Change-Id: I4da9938eb0b039f97b83badd2269af153c7edbcc
2021-08-26 16:03:34 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
0fc781c7c2 Add python_library -> py_library bp2build support
Bug: 196091681
Test: bp2build/python_library_conversion_test.go
Test: build/bazel/ci/mixed_{libc,droid}.sh
Change-Id: Ice87d75533c97fd9c139dc59de09a039e2713a01
2021-08-25 15:57:01 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
f5a3eacc07 Add License decl to testing.go
Test: Existing tests pass
Change-Id: I0d8b77d87801cb361036c8788f30bd80070fd7ff
2021-08-23 22:07:12 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
0da7ce6f39 Document testing & build_conversion purposes
Formatter also relocated some imports

Test: Existing tests pass
Change-Id: I48b1cb65400e463a6b4e91fd9fe76e13d8d81cd6
2021-08-23 22:04:53 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
ce0a07e249 Rename runPythonTestCase
To runBp2BuildTestCaseSimple because it's not Python-specific
but rather Bp2Build-specific

Name is suboptimal but given current functionality and use
we couldn't come up with a better one.

Test: Existing tests pass
Change-Id: I641cd051af741836b21ec6a7d6639516fb13028f
2021-08-23 22:02:20 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
1c92aeff4e Share runBp2BuildTestCase & runPythonTestCase
Pull along bp2buildTestCase as well
Because none are specific to their current hosts

Move `errored` up to the "generic" block vs.
the "custom" block

Test: Existing tests pass
Change-Id: Ie8d5322a5554ebd88f0c142a4c541791687be326
2021-08-23 19:20:29 +00:00
Liz Kammer
2ada09a546 Don't create a new module for bp2build conversion.
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3

We avoid both of these by storing the information directly in the
underlying module.

Also as a fringe benefit, removes some necessary boilerplate for
conversion.

For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714

Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
2021-08-17 15:57:09 -04:00
Liz Kammer
32b77cf8a8 Add some initial benchmarking for bp2build.
Test: go test -benchmark=. in bp2build directory
Change-Id: I8275c38461078cc6100fbc89837177de0edc9d68
2021-08-12 15:54:38 -04:00
Christopher Parsons
67d6ccecfc Merge "bp2build: handle system_shared_libs" 2021-08-09 19:11:00 +00:00
Chris Parsons
51f8c39261 bp2build: handle system_shared_libs
- If no system_shared_libs is specified, bp2build writes no attribute
value. In this case, the bazel library macros determine the correct
default behavior.
- If any system_shared_libs is specified for any variant, then bp2build
writes the value verbatim. This includes if an empty list is specified,
as this should override defaulting behavior.

Note this defaulting behavior is incomplete and will be incorrect in
corner cases. For example, if, in an Android.bp, system_shared_libs is
specified for os.linux_bionic but not for os.android, then the bazel
default for os.android will be incorrect. However, there are no current
modules in AOSP which fit this case.

As a related fix, supports static struct for cc_library_static.

Also, removes some elements from the bp2build denylist.

Test: mixed_droid CI
Change-Id: Iee5feeaaf05e8e7209c7a90c913173832ad7bf91
2021-08-09 11:41:09 -04:00
Rupert Shuttleworth
aa1777feef Merge "Add initial bp2build convert for prebuilt_etc modules." 2021-07-28 13:43:26 +00:00
Rupert Shuttleworth
569a23ae00 Merge "Add initial bp2build support for converting android_app_certificate modules." 2021-07-28 12:26:26 +00:00
Rupert Shuttleworth
378fc1b410 Add initial bp2build convert for prebuilt_etc modules.
Test: Added unit test.

Change-Id: Ie63674e4a867e355e4667b4240979b822f0f3c86
2021-07-28 08:22:25 -04:00
Rupert Shuttleworth
5c4881c8ec Add initial bp2build support for converting android_app_certificate modules.
Test: Added unit test.

Change-Id: Ib5bf1711eca5be772e1bed73663dcb86252e0b4a
2021-07-28 06:51:07 -04:00
Rupert Shuttleworth
9447e1ee12 Add support for prebuilts attribute to the bp2build APEX converter.
Test: Updated unit test.

Change-Id: I73dfa84d7a55c0b94393ad6257b5f16313422852
2021-07-28 06:03:17 -04:00
Rupert Shuttleworth
eb8c85abf5 Add initial bp2build converter support for apex_key modules.
Test: Added unit test.

Change-Id: I1ec2adfcef770ff91bcf35b89d9961f4d2b8a814
2021-07-28 01:45:54 -04:00
Rupert Shuttleworth
6e4950a061 Add some more properties to the bp2build APEX converter (second try)
In particular:

- AndroidManifest
- file_contexts
- key
- certificate
- min_sdk_version
- updatable
- installable
- native_shared_libs
- binaries

Test: Updated unit test

Change-Id: I1c6e8c4b6b24ce487f64e5d37bd594dbb000fe6f
2021-07-27 05:02:50 -04:00
Rupert Shuttleworth
0358478150 Revert "Add some more properties to the bp2build APEX converter."
This reverts commit 69bf4c0d60.

Reason for revert: Possibly broke CI (https://android-build.googleplex.com/builds/quarterdeck?branch=aosp-master-bazel&target=bp2build-incremental&lkgb=7266256&fkbb=7237028), rolling back just in case...

Change-Id: I14ebac4f47576a73285d1f724ae4499036e8e678
2021-07-27 02:37:32 +00:00
Rupert Shuttleworth
69bf4c0d60 Add some more properties to the bp2build APEX converter.
In particular:

- AndroidManifest
- file_contexts
- key
- certificate
- min_sdk_version
- updatable
- installable
- native_shared_libs
- binaries

Test: Updated unit test

Change-Id: I9ea13f021d63ec0f75006b3b0b23ec8178c7e9da
2021-07-26 07:32:36 -04:00
Treehugger Robot
2231f8a21c Merge "Remove default_shared_libs" 2021-07-23 21:18:51 +00:00
Christopher Parsons
2f1102fe58 Merge "Remove bp2build deps mutator" 2021-07-23 14:55:40 +00:00
Chris Parsons
5a34ffb350 Remove bp2build deps mutator
Refactor bp2build to retrieve modules directly by name, instead of via
DirectDeps. This functions properly as bp2build has no need for variant
information of the blueprint graph.

Test: USE_BAZEL_ANALYSIS=1 m fmtlib
Change-Id: Ief4b67bc56f24929871af772f3a742f07085bf8c
2021-07-22 18:09:34 -04:00
Colin Cross
6b8f4253eb Remove default_shared_libs
system_shared_libs has been modified to have the same behavior as
the newly added default_shared_libs, remove default_shared_libs in
favor of system_shared_libs.

This reverts Ia2349d84c70e503916f90a5d2702e135248f73df and renames
the default_shared_libs property in cc_object (which never had
system_shared_libs) to system_shared_libs.

Bug: 193559105
Test: m checkbuild
Change-Id: I46672e3a096b6ea94ff4c10e1c31e8fd010a163c
2021-07-22 12:10:11 -07:00
Colin Cross
cb0ac95bde Remove Fuchsia support from Soong
Bug: 194215932
Test: m checkbuild
Change-Id: Id7d3964d9417f8b0938af9b168bb4c00cebe9390
2021-07-21 20:37:46 -07:00
Liz Kammer
2d8056929e Merge "Handle target.bionic in bp2build." 2021-07-19 18:10:24 +00:00
Liz Kammer
01a16e8275 Handle target.bionic in bp2build.
Soong supports some hand-crafted target.<type> that match multiple os or
arch types to simplify configuring for similar targets. target.bionic is
used to match on both android and linux_bionic OSes for cases where they
should be handled the same way.

Test: build/bazel/ci/bp2build.sh
Change-Id: I47b6aaf3279e4d242c4fd0e12f24117eb98e0665
2021-07-19 12:53:58 +00:00
Jingwen Chen
f59a8e117e Fix apex bp2build.
apexBundle was missing InitBazelModule for it to have bazel_module properties.

Test: soong unit test
Fixes: 193866723
Change-Id: I5fb38c8b2e8edaecef44162491d399c0223d3fb1
2021-07-16 09:32:49 +00:00
Chris Parsons
69fa9f9e5e Seperate asflags and cflags
This fixes a bug which was a misunderstanding of soong properties:
Soong's cflags pertain only to C and C++ language, whereas bazel's copts
pertain to all three languages. This change ensures that asflags are
added as specifically asflags, and the 'copts' for the static library
macro pertains only to C and C++ languages.

This requires a somewhat hacky workaround for asflags, however: Since
assembly sources also need includepath-related flags, this duplicates
these flags between copts and asflags. To reduce verbosity of
bp2build-generated targets, this also ensures that asflags are omitted
in cases where there are no assembly sources.

Test: Mixed build droid CI
Change-Id: Ic0babed1f90d6dc82e5788638681ce5b995043f8
2021-07-13 14:06:14 -04:00
Rupert Shuttleworth
a9d76dd423 Add partial bp2build support for APEX targets (second try).
Test: Added unit test
Change-Id: Icbf0475251aac5d1f13bd35a18f90b6c70f3ae29
2021-07-02 07:36:07 -04:00
Colin Cross
137d7dafd8 Support shared_libs, static_libs and default_shared_libs in cc_object
cc_object files don't link against static or shared libraries, but they
can use headers from them.  Make cc_object use the default_shared_libs
headers, and add support for explicit static and shared library
dependencies.

Bug: 153662223
Test: go test ./cc/...
Change-Id: I9ab160ede06db1b135d217d72770c22f500cfe1b
2021-06-25 16:29:51 -07:00
Jingwen Chen
c63677b3c9 Direct Bazel builds from m.
This CL adds support to bp2build/Soong to dump a BUILD file under
out/soong/soong_injection/targets containing alias targets to their real
targets for every converted Soong module, regardless of whether they are
handcrafted or generated.

Test: TH
Change-Id: Ic1816fda5d019c395301618134fac68b3057d752
2021-06-21 06:12:34 +00:00
Jingwen Chen
c4dc9b4f08 bp2build: group shared/static attrs into a struct.
This makes bp2build generate these attrs into a Starlark dictionary,
passed into the cc_library macro directly. This makes the BUILD target
representation more similar to the Android.bp one, and also makes
it more legible.

Test: TH
Change-Id: I42b427cc4b22c6376d3d24e40b9af1692bb0c692
2021-06-16 05:40:37 +00:00
Liz Kammer
2d7bbe3883 Append _alwayslink to prebuilt whole static deps
This allows us to support prebuilt archives that are part of whole
static deps.

Test: build/bazel/ci/bp2build.sh
Bug: 190716727

Change-Id: I913dbd8f85a1974fbd53c0cbaa49211db18ea45f
2021-06-11 08:10:50 -04:00
Jingwen Chen
3d383bbeb9 bp2build: support strip properties.
This CL adds support to bp2build for generating all strip-related
properties into a strip dictionary as a parameter in the cc_library
macro.

With the dictionary, it's easy to organize related attributes and
directly expand into a stripped_shared_library's attributes.

Test: //build/bazel/tests/bionic:compare_libc_stripping
Test: TH
Fixes: b/187928597
Change-Id: Ifea68d48fe295e71a43b12876cb168c475a62187
2021-06-10 05:22:50 +00:00
Liz Kammer
d366c909ca Handle no_libcrt in bp2build.
Test: ci/bp2build.sh
Bug: 187928307
Change-Id: Ib80c4318169652b322e5d878c8784679e42f87dd
2021-06-08 17:18:22 -04:00
Liz Kammer
47535c51fa Handle excludes_{shared,static}_libs
Bug: 188497994
Test: bp2build.sh
Change-Id: I4a5ea40cbd804e8542fe33143e4926abc0c6164f
2021-06-04 10:15:36 -04:00
Liz Kammer
9abd62d133 Use maps in bazel *attribute types
This is to simplify the process of resolving label + exclude labels
across the various configuration axes we have and across the various
properties/modules that use this behavior.

Test: ci/bp2build.sh && ci/mixed_droid.sh
Change-Id: I8efae3e75ddb365384f5caaf5bb504a5206618d3
2021-06-03 17:37:56 -04:00
Jingwen Chen
14a8bda31e bp2build: split as, c, and cpp srcs for cc_library
This CL adds support for cc_library to correctly split c, as and cpp
srcs in shared/static nested props, as well as splitting
the *filegroup* deps in those props, where each filegroup is expanded
into its own c, cpp and as srcs filegroups. This ensures that the
correct sources go into cc_library_static's underlying cc_libraries for
c, cpp and as sources respectively.

See the bp2build conversion test for a better visualization.

Bug: 183064430

Test: TH
Change-Id: I29add5140672d042adff65527d8b65f4a5f0a05b
2021-06-03 05:00:37 +00:00
Treehugger Robot
72aedde483 Merge "Handle product vars *flags props in cc* modules" 2021-05-31 08:21:19 +00:00
Liz Kammer
ba7a9c5e4a Handle product vars *flags props in cc* modules
Test: build/bazel/ci/bp2build.sh
Bug: 188497994
Change-Id: Ifb379e370075d6a7bc55b82d79c210c31ef377e9
2021-05-27 13:57:01 -04:00
Rupert Shuttleworth
ffd4582b86 Add cc_prebuilt_library helpers for use by the llvm_prebuilt_library_static converter.
Test: bazel build //prebuilts/clang/host/linux-x86:prebuilt_libFuzzer
Test: bazel build //prebuilts/clang/host/linux-x86:prebuilt_libomp
Test: bazel build //prebuilts/clang/host/linux-x86:prebuilt_libunwind
Test: bazel build //prebuilts/clang/host/linux-x86:prebuilt_libunwind-exported

Change-Id: I9343e19a149b62785e8ee3096254776c19675032
2021-05-27 09:07:57 -04:00
Liz Kammer
f6840284b6 Merge "Support empty srcs for configurable attrs" 2021-05-27 12:56:29 +00:00
Rupert Shuttleworth
22cd2eb3d1 Add e.g. Target: { Android_arm: { ...} } support to LabelAttribute.
LabelListAttribute support was already added, but LabelAttribute support is needed for cc_import rules.

Test: Added unit test for version_script, which is the only supported LabelAttribute so far.

Change-Id: I4e86e7391586e0780623d06b794e7399f0ccd50e
2021-05-27 04:32:48 -04:00
Liz Kammer
2b07ec7b3b Support empty srcs for configurable attrs
Test: mixed build linkerconfig
Change-Id: I1ee56f7fa46ce2971ecae6d2ca9258631a32f46a
2021-05-26 16:58:54 -04:00
Jingwen Chen
bcf5304eab bp2build: support arch/target static/shared props.
This CL adds the support for static/shared cc_library props nested
within target and arch structs, and correctly sets them in the right
select statement.

Test: TH
Bug: 189183307
Change-Id: I48f7d62755ebab56fe03d3ecee2e15427eb552c9
2021-05-26 09:28:19 +00:00
Rupert Shuttleworth
c194ffbcf3 Make GetTargetProperties() aware of more complex targets, like 'android_arm', instead of just 'android'.
Test: Added new unit test and updated existing tests.

Test: bazel build //bionic/... //external/... //frameworks/... //system/...

Test: ./build/bazel/scripts/run_presubmits.sh

Change-Id: I250d1964f5cf42b92ddb929379d35d8c844423f7
2021-05-25 19:40:16 -04:00
Christopher Parsons
e28a176a99 Merge "Rollforward "Split asm and c flags and srcs in..."" 2021-05-25 18:59:39 +00:00
Chris Parsons
990c4f4fc3 Rollforward "Split asm and c flags and srcs in..."
This fixes a test and rolls forward I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4

Test: m nothing
Test: See I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4
Change-Id: I0e450c28e70087e406e7b562d7e772785f177379
2021-05-25 12:15:52 -04:00
Colin Cross
4f9d8d6482 Merge "Revert "Split asm and c flags and srcs in bp2build output"" 2021-05-25 15:21:36 +00:00
Colin Cross
52aa4e1fd4 Revert "Split asm and c flags and srcs in bp2build output"
Revert submission 1714835-roboleaf-asm-c

Reason for revert: TestCcLibraryStaticProductVariableSelects fails everywhere
Reverted Changes:
I28cf7437e:Split asm and c flags and srcs in bp2build output
I2b47e6b55:Split libraries by language in cc_library_static

Change-Id: I85d39a462f0a5b3f5ff3d685906813fab9f01358
2021-05-25 15:20:39 +00:00
Christopher Parsons
ed73b76513 Merge "Split asm and c flags and srcs in bp2build output" 2021-05-25 14:58:32 +00:00
Liz Kammer
e4982e8828 Mark test helpers as helpers.
This enables identifying the correct location of test failures.

from godocs:
Helper marks the calling function as a test helper function.
When printing file and line information, that function will be skipped.
Helper may be called simultaneously from multiple goroutines.

Test: go test bp2build tests
Change-Id: I52430e541494c78f5222c11510a4d273fe205a06
2021-05-25 14:41:43 +00:00
Jingwen Chen
962a8f0253 Merge changes I3ecdeaab,I43720641
* changes:
  Make BUILD file merging slightly smarter.
  bp2build/b: exit early in GENERATE_BAZEL_FILES=1.
2021-05-25 08:56:21 +00:00
Jingwen Chen
4910976314 Make BUILD file merging slightly smarter.
This change enables checked-in BUILD files like
prebuilts/clang/host/linux-x86/BUILD.bazel to be merged cleanly with the
bp2build generated one into the synthetic workspace.

The checked-in BUILD file contains a package() declaration that bp2build
also generates. To avoid double declaration, the BUILD file writer now
checks if the BazelTargets contain handcrafted targets. If so, it
delegates the package declaration to the handcrafted BUILD file instead.

This change also sorts the bp2build targets before the handcrafted ones,
and adds a section header to demarcate the two sets of targets.

Test: TH
Change-Id: I3ecdeaab3226b895b623daf0791d24a657f7a7c6
2021-05-25 07:01:55 +00:00
Chris Parsons
af24cdd99f Split asm and c flags and srcs in bp2build output
This allows removal of almost all current items from the mixed build
denylist, which were previously broken due to being unable to separately
control flags for compilations of different languages within the same
target.

Note that this does not appropriately implement asm/c srcs and flags for
either the shared variant or the static variant. This will require a
followup.

Test: bp2build.sh and mixed_libc.sh CI scripts
Test: Updated b2build tests

Change-Id: I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4
2021-05-24 21:49:14 -04:00