Commit graph

809 commits

Author SHA1 Message Date
Treehugger Robot
69bda98f50 Merge changes I816f209e,I9e4d51c3 into main
* changes:
  Handle enabled: false via conditions_default
  Handle nil enabled values
2023-08-08 19:35:54 +00:00
Spandan Das
846ce68a2f Handle enabled: false via conditions_default
In this Android.bp file
```
my_cc_defaults {
  enabled: false,
  soong_config_variables: {
    my_bool_variable: {
       conditions_default: {enabled: false},
    }
  }
}
```
The inner enabled: false is a no-op because the top-level enabled is
false. Currently, bp2build will raise an exception for this Android.bp
file.

However, it does not need to. `productVariableConfigEnableLabels` runs
only if the top-level enabled is false. If it sees enabled: false via
conditions_default, it should just ignore it since it is a no-op.

Test: go test ./bp2build
Bug: 210546943
Change-Id: I816f209eaf21de65ddfbc2893e5255be94bcaa11
2023-08-08 16:54:34 +00:00
Treehugger Robot
21b19f9ca8 Merge "Use string.Builder" into main 2023-08-08 14:35:15 +00:00
usta
da2a211bbd Use string.Builder
with 4000X clones of adbd, bp2build.Codegen time goes from 36s to 7s

Test: m bp2build and verify same BUILD.bazel files
Bug: 292281398
Change-Id: Ibd3b328a917fe5d1bdad67c67116da336f9033cb
2023-08-08 13:29:07 +00:00
Spandan Das
479e39f8fb Handle nil enabled values
If enabled does not appear inside `soong_config_vars`, we can ignore it.

Bug: 210546943
Test: go test ./bp2build

Change-Id: I9e4d51c3b683f262921449634f827915ce87dc8d
2023-08-08 02:19:38 +00:00
Spandan Das
09f6b1390c Merge changes I8ca452aa,I437fc890,If9446700 into main
* changes:
  Handle .proto files in different package for filegroups
  Handle .proto files that end up in a different package
  Allow creation of BazelTargets in a different directory
2023-08-08 00:51:52 +00:00
Spandan Das
df3ec82b62 Handle .proto files in different package for filegroups
Followup to aosp/2693190, this CL adds the support to filegroups.
<mod>__bp2build_converted is now an alias to a proto_library
target <mod>_proto. This proto_library will be created in a different
package if the .proto file exists in a different package.

Test: bp2build unit tests
Test: TH
Bug: 292583584
Change-Id: I8ca452aacf1a86dfc9e218464e38aab89afa5a29
2023-08-07 21:39:05 +00:00
Spandan Das
c53767e434 Handle .proto files that end up in a different package
Bazel poses a strict requirement that .proto files and proto_library
must be in the same package. This CL handles this automatically by
creating the proto_library in a separate dir/package if necessary

Implementation details
- Partition the `srcs` by package. `srcs` has been computed using
  `transformSubpackagePath`, so the information about packages is
  available at this point
- Create a proto_library in each package by using
  `CommonAttributes.Dir`. Collect all these additional libraries
  and put them in `info.Proto_libraries` so that they get added as deps
  of (cc|python|...)_proto_library
- Add an import_prefix to the proto_library in subpackages relative to
  the current directory. This relies on the assumption that every src is
  beneath the current directory (Soong will complain if a path in
  Android.bp contains ../)

filegroup module type uses a separate code-path to create proto_library.
This will be handled in the next CL in stack.

Test: bp2build unit tests
Test: TH
Test: Built the failing internal module mentioned in
b/292583584#comment1

Bug: 292583584

Change-Id: I437fc89092321b26c5f0511387cde9e84084d6f9
2023-08-07 19:18:03 +00:00
Spandan Das
3131d679f2 Allow creation of BazelTargets in a different directory
The current API restricts creation of targets to the directory of the
visited soong module. This CL proposes adding a `Dir` property in
`CommonAttributes` that can be used to create a bazel target in
a specific dir. The use case for this is to dynamically create
additional targets for proto_library that are adjacent to .proto files
(Bazel poses a strict requirement about proto_library being in the
same package as the .proto file, but Soong does not)

Usage is restricted to dirs that have an existing Android.bp file. There
are some places in bp2build where we use existence of Android.bp/BUILD
on filesystem to curate a compatible fully qualified path (e.g. headers).
If we use `CommonAttributes.Dir` to arbritraily create BUILD
files, then it might render those curated labels incompatible.

Test: go test ./bp2build

Change-Id: If9446700457eddfb389be9d9bde39087f67daa60
2023-08-07 19:17:03 +00:00
Treehugger Robot
38a72e717b Merge "cosmetic: unused argument" into main 2023-08-07 14:43:51 +00:00
Treehugger Robot
7660e71895 Merge "add bp2build subevents" into main 2023-08-07 14:43:01 +00:00
Cole Faust
946d02cb80 Remove final usages of product_vars_providing_rule
Bug: 269577299
Test: Presubmits
Change-Id: Iad1df30ba2ff0256e56919411d98c1bd9f7fa5a7
2023-08-04 14:58:05 -07:00
usta
40caf95004 cosmetic: unused argument
Test: m nothing
Bug: NA
Change-Id: I2a647e2e7c9c2fd3881c18e3cbb072b260cd9659
2023-08-04 16:53:31 -04:00
usta
aaf2fd13e1 add bp2build subevents
Test: verified new events in bp2build_metrics.pb
Bug: 286412441
Change-Id: Ib803c3a3d1a8e7eaae5fb4c01c7769a72f4664ff
2023-08-04 16:47:41 -04:00
Treehugger Robot
7d76e8cc9d Merge "Fix kernel build tools" into main 2023-08-04 17:15:10 +00:00
Cole Faust
d1acaa4169 Add a linux_86 config option
Test: b test --config=linux_x86 //build/bazel/rules/apex/...
Change-Id: I4b98935f8664b93162b28229134cf4b56603002b
2023-08-03 17:04:03 -07:00
Spandan Das
59b8a6687d Merge "cc Bp2build support for genrules that generate .proto file" into main 2023-08-03 21:51:11 +00:00
Cole Faust
95c5cf88bd Fix kernel build tools
That build doesn't set platform_sdk_version.

Bug: 269577299
Test: ./prebuilts/kernel-build-tools/build-prebuilts.sh on the kernel-build-tools branch
Change-Id: I68eb33bfb80770cef9c3a9c5e01f1ed746a2d069
2023-08-03 13:49:27 -07:00
Spandan Das
a99348dca4 cc Bp2build support for genrules that generate .proto file
If `srcs` contains a gensrcs/genrule module, the current bp2build module
will put it in the catch-all `srcs` attribute. This is reserved for .cpp
sources, so if the genrule produces a .proto/.aidl/... file, this will
fail.

This handles genrules that produce .proto files. To implement this, this
creates an additional partition that detects if the other module is a
genrule/gensrc that produces .proto files. If true, it will append it to
the proto partition.

This CL does not handle
- genrule that produce .c/.aidl/.yacc/.... files. They will continue to
  be partitioned into the catch-all partition
- java modules

Test: unit tests
Test: TH
Bug: 293205700
Change-Id: Ib720fdf3a053ff5dd256e6faa632e3fa7776966d
2023-08-03 18:56:05 +00:00
Cole Faust
87c0c33a65 Make bp2build-generated selects() based on product config build settings
...instead of based on constraint settings.

Bug: 269577299
Test: m nothing and ./build/bazel/ci/bp2build.sh
Change-Id: Ib9caec79c92b8fd304e46be841de5612bd1637e3
2023-08-03 10:12:09 -07:00
Treehugger Robot
687041c9e0 Merge "Add go modules to bp2build progress dashboard" into main 2023-08-02 12:02:19 +00:00
Spandan Das
41f1eeef91 Add go modules to bp2build progress dashboard
Since these modules have been converted, we should tag them as converted
in the dashboard.

Test: Put a print statement for `metrics` and made sure that the go
modules appear there
Bug: 294098662

Change-Id: I6282100111030a94f15f330916eaf41fcfc16e1a
2023-08-01 22:28:16 +00:00
Cole Faust
bc65a3fea8 Revert^2 "Initial implementation of the bazel sandwich"
c13fad8181

Change-Id: I478562c8fd89e62983feb5b52b62aad851d40f00
2023-08-01 10:27:57 -07:00
Jooyung Han
09195f4ffd Merge "Revert "Initial implementation of the bazel sandwich"" into main 2023-08-01 06:50:25 +00:00
Jooyung Han
c13fad8181 Revert "Initial implementation of the bazel sandwich"
Revert submission 2651299-bazel_sandwich

Reason for revert: b/293883239, checking if this breaks the build.

Reverted changes: /q/submissionid:2651299-bazel_sandwich

Change-Id: I5f1d4bb13d21e6599f5c353dcaba2375f5ec234d
2023-08-01 05:15:17 +00:00
Treehugger Robot
61a27f80cd Merge "Initial implementation of the bazel sandwich" into main 2023-07-31 23:00:55 +00:00
Cole Faust
a20d947329 Initial implementation of the bazel sandwich
The "bazel sandwich" is a mechanism for bazel to depend on make/soong
outputs. The name comes from the fact that bazel is now at the top
and bottom of the build graph. This is intended to allow us to work
on converting the partition builds to bazel while not all of the
dependencies of the partition have been converted.

It works by adding the bazel_sandwich_import_file rule, which emits a
dangling symlink that starts with bazel_sandwich:, and includes
information that the aquery handler in soong reads. The aquery handler
rewrites the symlink so that it points to a file generated by
make/soong, and adds a ninja dependency from the symlink to the file
it's targeting.

This allows us to depend on make-built files from bazel, but notably
it doesn't allow us to depend on analysis-time information from make.
This shouldn't be a problem for the partitions, but limits the use of
the bazel sandwich to similar, less complicated types of builds.

go/roboleaf-bazel-sandwich

Bug: 265127181
Test: m bazel_sandwich
Change-Id: Ic41bae7be0b55f251d04a6a95f846c50ce897adc
2023-07-31 11:53:41 -07:00
Treehugger Robot
dc3606a81d Merge "Gen a header library when genrules export includes" into main 2023-07-31 17:48:44 +00:00
Spandan Das
3671dcd39a Merge changes Ie292173d,I23f14f83 into main
* changes:
  Panic if soongConfig and productConfig values do not match
  Check if soong_config_var propertystruct is zero before panicking
2023-07-28 16:26:44 +00:00
Treehugger Robot
6677600f35 Merge "Emit gtest in BUILD files only if Android.bp sets it" into main 2023-07-27 23:52:46 +00:00
Spandan Das
474d43bda6 Check if soong_config_var propertystruct is zero before panicking
If we have a soong_config_module_type with target.android_<arch>.*
properties, we should only panic if a soong_config_var is used to select
on those properties. We already call `IsZero` in an upper function, but
we need to call it in `AddSoongConfigPropertiesFromTargetStruct` as well
because conditions_default gets its own struct.

Test: unit tests
Test: repro'd cmd in b/293206386#comment1, it passes now
Bug: 293206386
Change-Id: I23f14f83cc9be66487625e292e26b204dfea4f9a
2023-07-27 23:18:57 +00:00
Spandan Das
a11df791ed Merge changes from topic "go-test-bp2build" into main
* changes:
  Do not convert soong's tests in bp2build
  Create additional test target for go modules in bp2build
2023-07-27 23:16:29 +00:00
Spandan Das
89aa0f71c9 Do not convert soong's tests in bp2build
These tests cannot be run with b since it uses functions that should not be
used outside tests, and it enforces this by looking at os.Args. Since
the pattern (`-test`) is missing in os.Args, these tests panic in b.

A module can be a dep of soong_build via two mechanisms
1. Listed in deps explicitly
2. Via plugin_for to create a reverse dep

This CL denylists (1) explicitly. e.g. soong-android. Another option
would have been to denylist by using Deps() of soong-build, but that
would denylist `golang-protobuf-proto` as well

This CL denylists (2) programmatically.

Note that this CL only denylists creation of the `go_test` target. We
will still create `go_binary` and `go_library` for soong since there
might be tools that depend on soong.
e.g.
```
--> depends on
dexpreot_gen --> soong-dexpreopt --> soong-android (test failure)
```

(The above example means that we might need to convert _some_ of soong
after all, but we can refactor the common bits to common utils. Keeping
this out of scope for now).

Test: b test //build/soong/... //system/tools/...
too)
Bug: 284483729

Change-Id: I809012bdd383febca30da95e991edfde5fd4d9b9
2023-07-27 17:11:51 +00:00
Spandan Das
682e78686b Create additional test target for go modules in bp2build
For go modules with non-empty testSrcs, we will create an additional
go_test target. To build a standalone test executable, we need to
include the source files in the compilation unit. This will be done
using the `embed` attribute
1. For tests of go_library, this is straightforward. It will embed the
   go_library and "inherit" its .go files and deps
2. For tetss of go_binary, we need to create an additional go_source
   target since go_binary cannot be embedded inside a go_test

Using `b test` for these tests revealed that certain tests are not
hermitic and rely on `testdata` files checked into the tree. This CL
introduces an allowlist to skip generating go_test targets for them.

Test: bp2build unit test
Test: TH
Bug: 284483729
Bug: 288491147
Change-Id: Ic736d655babc2f6067e4da75384900b7b8bdc2ed
2023-07-27 17:11:42 +00:00
Spandan Das
e50fd111f3 Emit gtest in BUILD files only if Android.bp sets it
cc_test bazel macro will default `gtest` to True (similar to Soong). So
we can skip explicitly setting this in the generated BUILD files. The
benefit will be that it will make the generated files less verbose,
without loss of information.

This will be implemented by changing its datatype to *bool from bool.

Test: go test ./bp2build
Test: TH
Change-Id: I284e10f1d58c8e7893b170209827f7d5084ca95e
2023-07-26 17:42:21 +00:00
Spandan Das
d1cd3518a8 Support cc_test.isolated in bp2build
This property implicitly adds `libgtest_isolated_main` to the static
libs of the test. bp2build will make this and `liblog` explicit in BUILD
files

Since the deps are made explicit in BUILD files, `isolated` becomes a
no-op for cc_test. Remove this property from cc_test.

Test: unit tests
Bug: 244432609
Change-Id: I189a7b6b62d9064f4b2abad49ac4975468046498
2023-07-26 17:06:27 +00:00
Spandan Das
f5a8655cd8 Fix difference in default cc_test.isolated in soong and bp2build
Soong defaults to false, while bp2build defaults to true. This CL fixes
bp2build's default to false to match Soong.

To prevent future differences, bp2build now uses the `isolated()`
function used by Soong. The type of the context object has been changed
to EarlyModuleContext so that the ctx object does not depend on
mutations that might not run in bp2build.

Test: TH
Bug: 244432609
Change-Id: I15346107896312961e8d12270cf2f9a2a48827e1
2023-07-26 17:06:27 +00:00
Treehugger Robot
8fbc42cfd7 Merge "Create build settings for all product config variables" into main 2023-07-26 02:41:46 +00:00
Cole Faust
f055db60d4 Create build settings for all product config variables
Bug: 269577299
Test: Presubmits
Change-Id: Id8b7d65d657fa20ad8591e58d5173445b0e4f4df
2023-07-25 11:47:20 -07:00
Treehugger Robot
35448be862 Merge "Make libtest_* deps explicit in BUILD files" into main 2023-07-25 18:46:39 +00:00
Liz Kammer
0db0e34c68 Gen a header library when genrules export includes
Embedding multiple includes from a genrule may be difficult to read,
instead we generate a header library that contains the headers and all
include dirs that Soong generates.

Test: go test bp2build tests
Change-Id: I590c74c133f015f27cccf5a2fd916153ad9c125e
2023-07-24 13:57:17 -04:00
Spandan Das
651203de35 Make libtest_* deps explicit in BUILD files
This is currently implicitly added by the cc_test macro. This can cause
duplicate issues if users write a BUILD file as
```
cc_test (
  name = "mytest",
  deps = ["libgtest"],
)
```
Making this dep explicit in BUILD files prevents this issue.

The property `gtest` will not be removed as part of this CL. cc_test
macro will use this property to add some gtest specific copts

Test: unit tests
Bug: 249374572
Change-Id: Ife931e26367fd4aab302bedc709cf907e2c01245
2023-07-21 23:04:32 +00:00
Zi Wang
8348ce92f5 Merge "java_test macro needs srcs and deps to create .jar for tradefed_test_suite" into main 2023-07-21 20:48:13 +00:00
Romain Jobredeaux
cc1b67631c Add DeviceAbi to product variable to build setting export
Change-Id: Ibcb9e77a28bf0c4c0915f7013a7408f65429ae15
2023-07-21 10:05:28 -04:00
Cole Faust
88c8efb18c Add CFI product config variables to platform_mappings
Bug: 269577299
Fixes: 283130542
Test: b test --config=android //build/bazel/rules/cc/...
Change-Id: I2fc4094167c48b1f3b22bee6b7ada309278f7250
2023-07-20 09:45:36 -07:00
Spandan Das
958ca02582 Merge changes from topic "build_go_source_mixed_builds" into main
* changes:
  Delete aliases to prebuilts
  Add functionality to sandbox mixed build actions
2023-07-19 17:21:25 +00:00
Liz Kammer
0a681989ae Merge "Handle static binary repetition of system deps" into main 2023-07-19 14:16:52 +00:00
Zi Wang
7873f613c4 java_test macro needs srcs and deps to create .jar for tradefed_test_suite
Test: java_test_host_conversion_test.go and TH

Bug: 280452825
Change-Id: Ibf0c7eed415fc58a4d228f0347ab125fdc4466e4
2023-07-17 16:39:07 -07:00
Cole Faust
f8231dd0ea Platform mapping-based product config
This allows us to set product variables as build settings instead
of loading them from a target's provider, which further allows us
to read product config variables in transitions.

Bug: 287539062
Bug: 269577299
Test: Presubmits
Change-Id: I8497703f706162572ceb3486240e1eb02a37f5f6
2023-07-17 16:27:08 -07:00
Alix Espino
2a779ea829 Merge changes from topics "errProne_bp2build_manually_enabled", "error_prone_config" into main
* changes:
  Bp2build for errorprone modules that manually enabled/disabled it
  Make errorprone a configurable attribute for bazel conversion
2023-07-17 13:54:50 +00:00