Commit graph

819 commits

Author SHA1 Message Date
Alex Márquez Pérez Muñíz Díaz Puras Thaureaux
fecec28c34 Merge "bp2build: migrate export_{,system_}include_dirs -> *includes for cc_prebuilt_library{,_static,_shared}" 2023-04-17 21:01:21 +00:00
Vinh Tran
2562585878 Add aidl.deps prop to include aidl implicit deps explicitly in Android.bp
In a follow-up CL, we should eventually disallow aidl.include_dirs in cc rules so that the deps are always explicit and compatible with Bazel migration.

Test: go test
Bug: 278059962
Change-Id: Ia786cc8634d03589dc008f10b01e6bb2b9f2c7f0
2023-04-14 19:28:44 -04:00
Treehugger Robot
0c3682be50 Merge "mixed builds correctly reference stubs libs" 2023-04-14 23:07:21 +00:00
Cole Faust
ae31382119 Merge "bp2build support for python_test(_host)" 2023-04-14 20:49:01 +00:00
Romain Jobredeaux
141dcbb6cc Merge "Delete platform mapping specific bp2build code" 2023-04-14 17:34:13 +00:00
Sam Delmerico
ca438e6b72 mixed builds correctly reference stubs libs
Bug: 270408757
Test: go test
Test: m NeuralNetworksTest_shared_partial --bazel-mode-staging
  && verify that Ninja commands link libneuralnetworks via stubs
Change-Id: I19bf5a829cea7fd00c0f82511ad2bddbc40fae4f
2023-04-14 12:42:23 -04:00
MarkDacek
9c094ca153 Fix issue where bazel-force-enabled-modules aren't actually analyzed.
Bug: 273910287
Test: run mixed_mode_test.sh

Change-Id: I309c423b6e7f505ad1db0d9327e7de6b8298dfe9
2023-04-14 02:17:54 +00:00
Cole Faust
d82f036b89 bp2build support for python_test(_host)
There was a request for using b with python tests. bp2build python
tests exactly the same way as python binaries so that they can be
used with `b`.

Bug: None
Test: go test
Change-Id: Id68a6a73572745a4885b3e5bb1b8452e36baa982
2023-04-13 16:10:59 -07:00
Romain Jobredeaux
1a3382c0c2 Delete platform mapping specific bp2build code
Bug: 249685973
Test: Presubmits
Change-Id: Ib76a7659dfe57731100550c2cff4de960be2f1d7
2023-04-13 18:48:00 -04:00
Romain Jobredeaux
a3c029e7d5 Merge "Bp2build support for sdk_version and java_version." 2023-04-13 20:53:07 +00:00
Cole Faust
37d27c4884 Fix crash when missing platform sdk version
Fixes: 277890306
Test: Presubmits
Change-Id: I26658a90856cb057c631bb71ca0cd094ed648cf5
2023-04-12 10:32:18 -07:00
Jingwen Chen
aea34a38ca Merge "Revert^2 "Re-land test tzdata apex with bazel builds."" 2023-04-12 06:50:26 +00:00
Romain Jobredeaux
2eef2e13e9 Bp2build support for sdk_version and java_version.
This CL adds java_version and sdk_version support to bp2build
converters for
   - java library
   - java binary
   - android library
   - android binary
   - android library import

Although java import doesn't support java_version and sdk_version, the
neverlink java_library wrapper around a java_import must specify a
sdk_version when targetting a device. "none" is used by convention.

Change-Id: I22a69dea2e351858368df69ed6a703b568d613ea
Bug: 215230098
Test: Presubmits
2023-04-11 21:05:48 -04:00
Cole Faust
eb644cfb34 Break constant information out of product vars
We don't want rules to load the product variables directly, because
then they won't be able to transition on them.

Break constant information that is safe to load outside of the
product vars file, so that we can make the product vars file have
more restricted visibility later.

Bug: 269577299
Test: m nothing
Change-Id: I848bff33e4f5798f51296ea3a2600615cab36985
2023-04-11 15:38:43 -07:00
Alex Márquez Pérez Muñíz Díaz Puras Thaureaux
c641cc496b bp2build: migrate export_{,system_}include_dirs -> *includes for cc_prebuilt_library{,_static,_shared}
Migrate `export_include_dirs` property to attribute `export_includes`, handling arch/os-variants
Migrate `export_system_include_dirs` property to attribute `export_system_includes`, handling arch/os-variants

Test: cc_prebuilt_library_{,shared_,static_}conversion_test.go
Test: mixed_droid.sh
Bug: 229374533

Change-Id: I658a336a71265af0545c1c2db1a4f6eb613a7366
2023-04-11 18:56:41 +00:00
Jingwen Chen
6134211c2d Revert^2 "Re-land test tzdata apex with bazel builds."
fb8b1a69e9

Change-Id: Id5c0edd1cd4aacbb75a8019e0a5ada4e608fbf09
2023-04-11 14:55:27 +00:00
Treehugger Robot
dfa0a761f2 Merge "Clean old symlink forests" 2023-04-06 01:28:33 +00:00
Treehugger Robot
dd5653bbf0 Merge "Fix issue merging bp2build files with handcrafted ones" 2023-04-05 20:32:14 +00:00
Cole Faust
5db92090e1 Clean old symlink forests
There was an incrementality bug in the symlink forest generation that
was recently fixed. However, if the appropriate files don't get touched,
the issue will remain. Add a mechanism to clean old symlink forests
so that when we fix incrementality bugs we can ensure they don't remain
in any old out directories.

Bug: 276349152
Test: Presubmits
Change-Id: I36664dfb0ca7227e3e1f89de859ebccb808c5f15
2023-04-05 11:44:49 -07:00
Cole Faust
3f4f521711 Fix issue merging bp2build files with handcrafted ones
It was possible for the merged content to end up back in the bp2build
generated file because there was a symlink from the symlink forest to
the bp2build generated file.

Remove the symlink if it exists.

Bug: 276349152
Test: m bp2build, add a handcrafted file in the same folder as a Android.bp file, m bp2build again, check that the symlink forest version is not a symlink
Change-Id: Id64aa3addebcf0c6b1728389f21ae246796aaf8d
2023-04-05 09:08:38 -07:00
Romain Jobredeaux
15807fa7ed Merge "Bp2build converter for java_host_for_device." 2023-04-05 13:32:29 +00:00
Romain Jobredeaux
eb711b9f7c Support arch variants in java's StaticLibs in bp2build
Test: Presubmits
Bug: 276901800
Change-Id: I6058a726833ef10a5f470946e2d265b20fa547ce
2023-04-04 19:50:13 -04:00
Romain Jobredeaux
e7370eabe4 Bp2build converter for java_host_for_device.
Change-Id: I70a345c641bbed3223dc6222062fb6948f7dc6cb
Bug: 276710283
Test: Presubmit
2023-04-04 19:49:17 -04:00
Cole Faust
e28bde81fd Merge "Move the testing platforms out of soong_injection" 2023-03-31 00:17:33 +00:00
Cole Faust
117bb747a4 Move the testing platforms out of soong_injection
See the other cl in this topic for more information.

Bug: 269577299
Test: b test --config=android //build/bazel/rules/apex:all
Change-Id: I1f20bc5f85398cd79dc5b7fe9d0bd4d93d442383
2023-03-29 14:46:20 -07:00
Jingwen Chen
a8623da12e bp2build apex: convert canned_fs_config property.
Bug: 275280970
Test: bp2build unit test
Change-Id: Ic6867a640a5079cd206419ddd378d2357093dae6
2023-03-29 04:20:08 +00:00
Cole Faust
d3cc7c9bef Merge "Add testing android products" 2023-03-29 00:19:51 +00:00
Liz Kammer
f2571c4136 Merge "Propagate testonly for override_apex bp2build" 2023-03-28 20:57:45 +00:00
Cole Faust
ebc01a3225 Add testing android products
These must be added in the soong_injection code as opposed to just
defined loosely in checked-in bzl files because the product_vars
select statement must be updated to support the new platforms.

Bug: 269577299
Test: b test --config=android //build/bazel/...
Change-Id: I7bba9af214896dd3b5938bae70b7c0cea4f75e41
2023-03-28 11:27:30 -07:00
Liz Kammer
1a1c9df4e7 Propagate testonly for override_apex bp2build
Test: go test bp2build tests
Change-Id: I42c61687223c658237b3e4b0a0d6dd339946a6aa
2023-03-28 11:39:50 -04:00
Cole Faust
426b1c6c7f Merge "Rename utils.bzl to android_product.bzl" 2023-03-27 21:56:43 +00:00
Liz Kammer
48cdbeba29 Handle stubs within an apex with apex_available
Note this doesn't entirely match Soong's logic but is an improvement to
allow linking against implementation when two cc modules are
apex_available to the same module.

It is not possible to recreate the logic for "directly in" without
significant changes to bp2build as we do not add dependencies nor run
apex mutators. Rather than trying to replicate this, we would be better
off refactoring Soong to no longer support the "directly in apex" logic
and require users to correctly specify apex_available.

Bug: 272378496
Test: go test conversion tests
Change-Id: I17ac426f9b4bdad0c2ab661484e5d994f63568ce
2023-03-27 13:05:37 +00:00
Cole Faust
bd249821b9 Rename utils.bzl to android_product.bzl
Bug: 269577299
Test: b build --config=android `bmod com.android.adbd`
Change-Id: I93743a058a4f700e015cdd1c51544467a29d4fe6
2023-03-24 16:03:43 -07:00
Liz Kammer
2b3f56ed22 Add apex_available aidl + aidl&lang libraries
Test: b build com.android.neuralnetworks com.android.media.swcodec
Change-Id: I008b915b22e4c8c2cf1f0aee6cfdaf17374cb3a9
2023-03-24 15:14:53 -04:00
Sam Delmerico
512437b0b4 add parent static deps to cc_aidl_library targets
The static deps of the parent library can be necessary to build a
bp2build-generated cc_aidl_library target. We should add these deps as
implementation_deps so that they are accessible.

Bug: 250876486
Test: b build //frameworks/native/libs/gui/...
Change-Id: Ibe7c3598a684907473e2a4e040fb3976455a59e9
2023-03-21 16:49:06 -04:00
Alix Espino
895bc9463b Merge "Bp2build for android_app minsdkversion property" 2023-03-20 15:38:53 +00:00
Treehugger Robot
c887ac6a04 Merge "Depend on stubs via @api_surfaces" 2023-03-17 21:06:55 +00:00
Alix
e5085ebbc1 Bp2build for android_app minsdkversion property
Bug: 249265719
Test: go test ./bp2build
Test: insepcted generated build files
Change-Id: I02ed54a9fdcc88d98439e0a61626a0afc1fce639
2023-03-17 20:47:52 +00:00
Spandan Das
2518c022d1 Depend on stubs via @api_surfaces
BUILD files of rdeps should depend on stubs via @api_surfaces
indirection. e.g. instead of depending on
//system/logging/liblog:liblog_stub_libs_current, it should depend on
@api_surfaces//module-libapi/current:liblog. This ensures that the
generated BUILD files are compatible with Multi-tree.

Update the unit tests for this change.

Test: TH
Change-Id: Ibcc36dcfbee7b1973b341485f015e67987564dcc
2023-03-17 16:54:53 +00:00
Romain Jobredeaux
f5f6073ff5 Create an intermediate java library for every java_binary target
We're already doing this for kotlin libraries as java_binary does not
support kotlin sources.

Change-Id: I4967b3aca21dac8e0eaf54d48cc35d57a4c4923e
2023-03-16 21:11:30 +00:00
Sam Delmerico
c02e7d4017 Merge "remove stub dependency on source_library" 2023-03-16 17:43:11 +00:00
Treehugger Robot
feeee3ad26 Merge "make tidy attribute tri-state" 2023-03-16 15:42:21 +00:00
Sam Delmerico
5f90649036 remove stub dependency on source_library
Since the source_library attribute of _cc_stub_library_shared was a
label attribute, the source library was added as a dependency of the
stub which doesn't always make sense. E.g. it caused validation actions
for the source library to run even when we weren't building the source
library.

This change converts the label attriubte to a string attribute so that
we don't add the dependency.

Bug: 263390551
Change-Id: I19c62d1e14847dff68ab37156452addaa119eaf9
2023-03-16 10:01:22 -04:00
Sam Delmerico
63f0c93b09 make tidy attribute tri-state
Bazel boolean attributes can only support true or false states, but the
clang-tidy logic changes for modules which don't set a specific value.
Therefore, the tidy attribute should really be a tri-state of "true",
"false", or "" (unset).

Test: b test //build/bazel/rules/...
Bug: 268681923
Change-Id: I85fbaa612d175f475757db806cea6c018901fe3e
2023-03-15 10:10:43 -04:00
Alex Márquez Pérez Muñíz Díaz Puras Thaureaux
b157846a68 Merge "Revert "Revert "Produce _alwayslink sibling targets through bp2b..."" 2023-03-14 23:01:47 +00:00
Alix
f848bf8225 bp2build kotlinCFlags
for java_library, java_binary, android_app & android_library

Change-Id: Ia8dec7e7d497eb41b2d8dcd8c44c3090d1d3b4bd
Bug: 271265771
Test: go test ./bp2build
2023-03-14 13:57:22 +00:00
Alex Márquez Pérez Muñíz Díaz Puras Thaureaux
c353abdd53 Revert "Revert "Produce _alwayslink sibling targets through bp2b..."
Revert submission 2473787-revert-2467901-b/267214124-TLTTMLWFDS

Reason for revert: Fixed compilation errors

Reverted changes: /q/submissionid:2473787-revert-2467901-b/267214124-TLTTMLWFDS

Test: TOLIST
Bug: 271943047
Bug: 267214124

Change-Id: Ib08440377f6645cf26de4bda748d82c4dcd0b43a
2023-03-10 22:14:29 +00:00
Zi Wang
5c12bdfdd3 Merge "Sort expectTargets and actualTargets by name in bp2build test" 2023-03-09 18:17:51 +00:00
Spandan Das
d14a70d7ed Merge changes Ib004c2c3,I6b63d9d0
* changes:
  Create aliases for stubs in build/bazel/api_surfaces
  Add a method in bp2build to create aliases in another directory
2023-03-09 18:14:13 +00:00
Romain Jobredeaux
8062140a16 Merge "Call ResolveExcludes after arch-based sources computation in java.go" 2023-03-09 14:40:58 +00:00
Spandan Das
abedff0ca7 Add a method in bp2build to create aliases in another directory
The expected use case for this is to create aliases for stub libraries
in the @api_surfaces repository in build/bazel/api_surfaces.

This restricts the scope to just aliases. If we have a use case for
generating actual Bazel targets in another directory, a workaround could
be to generate the targets in the current directory (via
CreateBazelTargetModule) and aliases to it in the other directory

Test: go test ./bp2build
Change-Id: I6b63d9d018618d447fc7c260a2a94aaa00e57a4d
2023-03-08 21:40:26 +00:00
Romain Jobredeaux
3eaf747ae5 Call ResolveExcludes after arch-based sources computation in java.go
TEST: go test
Change-Id: I93a8ae5e0842dab03f73b0edb32841755d2ea435
2023-03-08 16:34:17 -05:00
Cole Faust
912bc8862e Use product variables from the overridden apex
override_apex's bp2build converter had a bug where it was looking at
the product variables for the override_apex module itself instead of
for the base module it is overriding.

Fixes: 271424349
Test: go test
Change-Id: If1e2653d3751fa908faf0ab97dfa2e943ebe98ec
2023-03-08 12:29:50 -08:00
Zi Wang
fba0a210e0 Sort expectTargets and actualTargets by name in bp2build test
This is to match the behavior of BUILD file generation.

Test: m and all existing bp2build conversion tests

Bug: 271122205
Change-Id: I6320a65aaad06114817eaed722cc0a0939c57bc1
2023-03-08 10:35:58 -08:00
Treehugger Robot
88e949945d Merge "Revert "Produce _alwayslink sibling targets through bp2build"" 2023-03-07 07:33:16 +00:00
Wilson Sung
68e126ee62 Revert "Produce _alwayslink sibling targets through bp2build"
Revert submission 2467901-b/267214124

Reason for revert: compile error

Reverted changes: /q/submissionid:2467901-b/267214124

Change-Id: I2235fd1457fda60b4613d5512f7fb20b7044a2db
Fix: 271943047
2023-03-07 04:16:53 +00:00
Alix Espino
09907dab55 Merge changes from topic "ktResourceStrip"
* changes:
  code cleanup for bp2build java_binary with kt srcs
  resource_strip_prefix support for kotlin srcs
2023-03-06 20:32:16 +00:00
Alix
5afd9fad98 code cleanup for bp2build java_binary with kt srcs
there is now support for resoure_strip_prefix in kt_jvm_library targets.

Test: built AnalyzerKt and updated go ./bp2build tests
Change-Id: I4a6fe45276d45519186b6f40a02db990511d6def
2023-03-03 23:07:51 +00:00
Alex Márquez Pérez Muñíz Díaz Puras Thaureaux
011b7cf063 Produce _alwayslink sibling targets through bp2build
Add Alwayslink to bazelPrebuiltLibraryStaticAttributes
Also adjust the corresponding tests to espect these new targets.

Test: m libc
Test: b build --config=android //prebuilts/clang/host/linux-x86:libunwind-exported_alwayslink
Test: bp2build.sh
Bug: 267214124
Change-Id: I4d996aef55321f1827947f00b0eaa52d0a14413e
2023-03-03 20:12:20 +00:00
Trevor Radcliffe
56b1a2b575 LTO Bp2build
Bug: 261733821
Test: Unit Tests
Change-Id: I8c3721d35c464e296012145b2e95a7f0866aac37
2023-03-02 17:45:01 +00:00
Christopher Parsons
647b8518ec Merge "Avoid deleting bp2build directory" 2023-03-02 14:22:13 +00:00
Chris Parsons
520e88b740 Avoid deleting bp2build directory
With this CL, we avoid deleting the bp2build directory and regenerating
all BUILD files. Instead, we regenerate BUILD files which have changes
on the filesystem, and delete old BUILD files which should no longer
exist.

This improves incremental Bazel server performance by about ~5 seconds.
Previously, Bazel would have cache misses for regenerated packages, and
would thus need to analyze them even if no meaningful changes took
place.

For ease of implementation, we avoid removing stale (empty) directories
from the bp2build workspace, but this should have no effect on the
build. (These will be removed on next clean)

Test: New integration tests
Test: Manual benchmarking in conjunction with persistent bazel server
Change-Id: I3e489ff403be34040122876012329060a2506366
2023-03-01 23:15:30 -05:00
Cole Faust
18994c73f1 Replace SortedStringKeys with SortedKeys
Now that we have generics.

Bug: 193460475
Test: presubmits
Change-Id: I1594fd8feb505175d5c09c03ef397e5ffd5b09cb
2023-02-28 16:51:32 -08:00
Alix Espino
0897df1466 Merge "Revert^2 "refactor of api_levels map for soong injection"" 2023-02-27 14:10:44 +00:00
Alix Espino
4fd7e74140 Revert^2 "refactor of api_levels map for soong injection"
a865f4e905

Change-Id: Ifea020ac83725df7131773ca587c56931f5cdf03
2023-02-24 14:46:43 +00:00
Matthias Männich
199584223a Merge "Revert "refactor of api_levels map for soong injection"" 2023-02-24 09:48:05 +00:00
Matthias Männich
a865f4e905 Revert "refactor of api_levels map for soong injection"
Revert submission 2441746-api_levels_refactor

Reason for revert: broke Android kernel build tools build: https://ci.android.com/builds/branches/aosp_kernel-build-tools/grid?

Reverted changes: /q/submissionid:2441746-api_levels_refactor

Change-Id: I35206879b93ec95ae325025cb764c6c34880dccf
2023-02-23 17:10:27 +00:00
Alix Espino
d3ffe6475e Merge "refactor of api_levels map for soong injection" 2023-02-23 16:35:35 +00:00
Yu Liu
10174ff208 bp2build of libraries with stubs should handle non-implementation deps
Bug: 266724299
Test: Unit test and manual tests
Change-Id: I7d7641676fbf3d3fae413b27d329a4b05b1cd3bf
2023-02-21 12:07:07 -08:00
Alix
c566ad8a4f refactor of api_levels map for soong injection
Change-Id: I58449fc84617e46727f23ab3d8dd5f118d2ee0d2
Bug: 249265719
Test: go test ./bp2build and locally ran bazel tests
2023-02-21 18:55:37 +00:00
Alex Márquez Pérez Muñíz Díaz Puras Thaureaux
01ec55ec92 Express no{,_lib}crt via features vs attrs
Given that we can map them directly to disabling the
corresponding Bazel features: `-{link_,use_lib}crt`

Test: Existing (adapted) Unit Tests
Test: bp2build.sh
Change-Id: Ib502f6fb929ace8e86a1001e3cc21f399317500c
2023-02-17 19:24:42 +00:00
Zi Wang
b2179e397a Add limited bp2build converter of java_sdk_library
Only public, system, test, module_lib and system_server are
converted in order to generate api_fingerprint.txt in Bazel.

Test: java_sdk_library_conversion_test.go and TH

Bug: 266973526
Change-Id: I67a00806165e5afad3876b6cd5cdbc6b0dd65d8b
2023-02-14 13:21:21 -08:00
Chris Parsons
1a12d03230 Avoid rewriting soong_build outputs if unchanged
This changes bp2build codegen, symlink forest generation, and
soong_build so that they do not rewrite output files if the contents are
unchanged.

Bug: 266983462
Test: m droid
Test: canonical_perf.sh benchmarking
Test: Manually verified that rerunning analysis did not regenerate
out/soong/workspace/prebuilts/sdk/BUILD.bazel unless contents changed

Change-Id: I5ec227df7a32b53c7fa0d741fb1403a51931024b
2023-02-10 15:34:06 -05:00
Cole Faust
9e384e2e6b Export apex_available_baseline to soong_injection
So that it can be used in the bazel implementation.

Bug: 268006095
Test: m nothing, check that it's in out/soong/soong_injection
Change-Id: I1520dd874076dee7fa083648d0790b060d658e5a
2023-02-08 17:43:09 -08:00
Usta (Tsering) Shrestha
3a49e9a068 Merge "reduce forest generation to be incremental" 2023-02-08 18:46:16 +00:00
Usta (Tsering) Shrestha
c4c07b12b6 reduce forest generation to be incremental
Previously, symlink forest generation involved removing the entire
symlink forest and recreating it from scratch. With this change,
a) symlinks which need not change are untouched,
b) symlinks pointing to the wrong location are fixed, and
c) symlinks which should no longer exist are removed.

On AOSP on my local machine, this reduces the symlink forest generation
step from 2.5s to 1.1s clean, and 0.6s when a single file is added to
a source directory.

Bug: 257528847
Test: m bp2build, touch `fakefile` under the forest, remove a file
from the source tree, rerun m bp2build. Manually verify the new forest
does not retain the link to the deleted source file, and that fakefile
no longer exists in the forest.

Change-Id: I481371ae487e9419af6a3a4370c552578b07d650
2023-02-07 06:23:43 +00:00
Sam Delmerico
cb3c52c766 export allowlist of environment variables to Bazel
Previously, Bazel was invoked during mixed builds with a stripped down
environment. This CL adds an allowlist of environment variables that are
passed to Bazel.

Test: WITH_TIDY=1 DISABLE_ARTIFACT_PATH_REQUIREMENTS=true mss tidy-packages-modules-NeuralNetworks --bazel-mode-dev
Change-Id: I23147bec59f6522953cf623e7bcaa0f1f99a75a3
2023-02-06 14:34:43 -05:00
Treehugger Robot
69ef681c13 Merge "bp2build kt for android_binary" 2023-02-02 23:34:59 +00:00
Alix
0856f9e227 bp2build kt for android_binary
Test: bp2build tests and locally built MusicKotlin
   (cannot allowlist yet due to soong/bazel custom_package differences)
Change-Id: I1cbd0fe2af0e6eeed65cf8d57dc85c1f54646c50
2023-02-02 20:26:01 +00:00
Treehugger Robot
9bedfd0d4b Merge "Rename API surface provided to mainline modules" 2023-01-30 23:06:22 +00:00
Christopher Parsons
94f8a60f5d Merge "Fix nondeterminism in bp2build" 2023-01-30 22:58:17 +00:00
Chris Parsons
7b3289b471 Fix nondeterminism in bp2build
This fixes two main sources of nondeterminism:

1. Fix a bug in the ConfigurationAxis comparator (which caused
   ConfigurationAxis sorting to be nondeterministic)
2. Process C++ dependencies using the sorted ConfigurationAxis order. In
   theory, the order in which dependencies are processed shouldn't
   matter (as they should end up in different select stanzas). However,
   in the case of InApex stubs, this is not the case; we now ensure
   that lists are concatenated in a predictable order.

Added bonus: Some cleanup with SortConfigurationAxes which
makes use of go generics (this made it easier to debug this issue).

Will follow-up with regression tests.

Test: Manually verified that build.ninja checksum and BUILD.bazel checksums do not change after running `m nothing` 6 times in AOSP (with comment-only Android.bp changes in between each run)

Change-Id: I81168e45bdbbcd61ea95ff665cf6c4bc180aa4e0
2023-01-30 21:26:57 +00:00
Treehugger Robot
32c52ad62f Merge "convert cc_object crt property in bp2build" 2023-01-30 17:07:37 +00:00
Jason Wu
47b0c8c620 Merge "Delete empty entries for covertedModules in bp2build_metrics" 2023-01-27 02:02:39 +00:00
Spandan Das
627fc3e436 Rename API surface provided to mainline modules
This API surface is provided by platform and mainline modules to other
mainline modules. Rename it to module-lib API surface to align it with
the terminology in go/android-api-types

Test: go test ./bp2build
Test: go test ./cc
Change-Id: Ieb9f3214e66366fc2ceb6f47e2d6623acb537827
2023-01-26 23:02:53 +00:00
Jason Wu
9948a86e2e Delete empty entries for covertedModules in bp2build_metrics
Test: `m nothing` and verify the bp2build_metrics
Bug: 257975713
Change-Id: Ib6b8a116bee20f3eef80cadaa0afac28a25e2121
2023-01-26 15:49:29 -05:00
Liz Kammer
0725a60e5d Merge "Correct global excludes not always being excluded" 2023-01-26 20:09:14 +00:00
Alix Espino
1db2ff3eaf Merge "arch variant support for libs property" 2023-01-26 19:11:23 +00:00
Liz Kammer
748d70786c Correct global excludes not always being excluded
Test: bp2build go tests
Fixes: 266617441
Change-Id: I82418ef8da4cca880d5adac98853805a18a9780e
2023-01-26 11:16:39 -05:00
Sam Delmerico
3fad8ed0d1 convert cc_object crt property in bp2build
Bug: 263905692
Change-Id: I71e3fc4a42954c2d1f9f631d012949b81f4536de
2023-01-26 10:21:58 -05:00
Treehugger Robot
7c30b6bad7 Merge "bp2build support .kt in android_library" 2023-01-25 15:47:49 +00:00
Trevor Radcliffe
6f0aef34e8 Merge "Export sanitizer vars for use in Bazel" 2023-01-24 17:29:47 +00:00
Alix
36795a74dc bp2build support .kt in android_library
Bug: 258690851
Test: bp2build tests and built target renderscript_toolkit
Change-Id: I43ddf5637c97e4849b63abd2423f29519d44c28f
2023-01-24 14:53:44 +00:00
Trevor Radcliffe
4f95ee96b8 Export sanitizer vars for use in Bazel
Bug: 265154269
Test: Unit tests
Change-Id: I46584159264476d76079e396c6ec294fb394acff
2023-01-23 20:28:06 +00:00
Cole Faust
c167d0f1d6 Add --config alias for musl host platform
Test: b build --config=linux_musl_x86_64 //build/bazel/examples/python/protobuf/...
Change-Id: Id665660ee7fe123d74ad78d42eb26d4a65e64a60
2023-01-23 10:17:21 -08:00
Alix
495cc8a6ff arch variant support for libs property
Test: bp2build tests and manually inspected Build file for libprotobuf-java-lite
Bug: 244210934
Change-Id: Id60d859eea2c23fa7a29784faeed5cb050d4193e
2023-01-19 20:42:24 +00:00
Trevor Radcliffe
3a79caa3b9 Merge "Add musl defaults in bp2build" 2023-01-19 16:01:04 +00:00
Alix Espino
fbd7fabf0a Merge "bp2build support libs for *->android_library_import" 2023-01-19 15:21:07 +00:00
Cole Faust
358ba4f0d8 Make symlink forest errors deterministic
We were looping over a map, and thus processing files
in a non-deterministic order. Some tests check for errors
produced during the symlink forest creation and they would
flake due to this nondeterminism.

Test: m nothing
Change-Id: Ie1a4cf98e341e4a951f110b7d5611bb69c4ddd2b
2023-01-18 15:02:03 -08:00
Alix
14101de2e1 bp2build support libs for *->android_library_import
Test: bp2build tests
Bug: 258688914
Change-Id: Ibeae04e4c006f84f902774821c92f463a7c709eb
2023-01-17 15:55:07 +00:00
Alix
53fae383d6 support libs for java_plugin -> *
Test: bp2build tests
Bug: 244210934
Change-Id: I7dbc820b23c4129485b0991aefb41e00e434d94e
2023-01-17 14:57:57 +00:00
Alix Espino
f02c7d882e Merge "libs support for android_app->* edge" 2023-01-17 14:31:18 +00:00
Usta Shrestha
071f6c29a3 for consistency symlink_tree failure = os.Exit(1)
Brings down time taken to fail from 18s to 13s (on aosp branch)

Test: temporarily coded random failure
Bug: N/A
Change-Id: Ib694eec977293f4dd7054e779d1b82b8cace93f6
2023-01-13 14:07:10 -05:00
Usta (Tsering) Shrestha
5c1b0ba692 Merge "metrics: count symlink/mkdir in symlink forest" 2023-01-13 04:01:55 +00:00
Usta Shrestha
da15c61add metrics: count symlink/mkdir in symlink forest
Bug: 260029212
Test: inspect out/bp2build_metrics.pb
Change-Id: Ia84c095a8d7b129cca629256b6c626c726cbfab1
2023-01-12 14:52:56 -05:00
Treehugger Robot
26248cf580 Merge changes I48426a8e,Ic991e3ae,I9768241f
* changes:
  bp2build apex min_sdk_version w/ soong config var
  Handle soong config vars for string attrs
  bp2build correctly handle empty soong_config_var
2023-01-12 19:11:27 +00:00
Alix
6c087cf01a libs support for android_app->* edge
Test: bp2build tests
Bug: 258688914
Change-Id: I2e24c484019c994b43509f350b3c090a40c3ba6a
2023-01-12 15:19:55 +00:00
Alix Espino
ed2491cf20 Merge "java_binary with kotlin sources" 2023-01-12 14:40:46 +00:00
Liz Kammer
9d2d41065c Handle soong config vars for string attrs
Previously if there was a value set outside of product variables and one
inside a conditions default soong config var, we would ignore the
conditions default. Now, we maintain the conditions default and use
the root value in cases where there is not a value for a product
variable.

Test: go test soong tests
Change-Id: Ic991e3aebe5bb6039353f4e3d25625e7c5190f96
2023-01-11 22:01:01 -08:00
Liz Kammer
f3963f8b9c bp2build correctly handle empty soong_config_var
Previously in cases where a variable value was set but empty in a bp
file we ignored the value in bp2build; however, the correct behavior is
to take the empty value for each case. The tests for soong config vars
with empty_prop_for_string_var and unused_string_var:
https://cs.android.com/android/platform/superproject/+/master:build/soong/android/soong_config_modules_test.go;l=243-283;drc=72beb34609217f3c98623fb6795b6417c7f0fc65
highlight the difference in when conditions default should be used or
not.

Test: go test bp2build tests
Change-Id: I9768241f013a1a1bc8fa23657a066b5b3f52f09b
2023-01-11 22:01:01 -08:00
Trevor Radcliffe
0d1b4029b8 Add musl defaults in bp2build
Bug: 261657184
Test: Unit Tests
Test: Manual Inspection of generated files
Change-Id: I69864abb410fddecd530fc75f2531aff33b33c97
2023-01-11 18:43:20 +00:00
Spandan Das
83e787e837 Add a new CreateSoongInjectionDirFiles function
- This is a wrapper function for all files in soong_injection directory.
This should prevent an error in the workspace generated for
api_bp2build.
- Rename the existing CreateSoongInjectionFiles fn and make it package
  private to prevent confusion

(The subsequent CL in this stack should contain a smoke test for
api_bp2build)

Test: b build //:empty --config=api_bp2build --config=android
Test: TH
Change-Id: Iddb0aa1aff2f709826edd587aa99fccddf80f08f
2023-01-11 03:21:00 +00:00
Vinh Tran
57b1e4064b Merge "Remove apex_available tag in cc_library_static generated from stubs-providing lib" 2023-01-10 15:30:11 +00:00
Paul Duffin
0c79abc971 Merge "Dedup registration of Soong config module types" 2023-01-10 10:33:22 +00:00
Zi Wang
709c59b2df Merge "Add variant_prepend support for all the properties in bp2build" 2023-01-10 05:54:29 +00:00
Alix
7c8eaebc4f java_binary with kotlin sources
Test:  bp2build conversion tests and inspected build file for AnalyzerKt
Change-Id: I0f6d5c3d371a9dbd000b74144ad68d5d8bd156cf
2023-01-09 20:09:37 +00:00
Paul Duffin
3229998d37 Dedup registration of Soong config module types
Bug: 264876909
Test: m nothing
Change-Id: I467a2090b32438cbbf88bbb453bb4960757ec37a
2023-01-09 16:48:46 +00:00
Cole Faust
501a7976be Merge "Bp2build support for multiple product configs" 2023-01-07 03:09:08 +00:00
Treehugger Robot
6228be4cd4 Merge "support libs for android_library" 2023-01-06 19:30:02 +00:00
Alix
82fb94e91e support libs for android_library
adds support for *-> android_library and android_library->* edges

Change-Id: I41d4e1d1b8106a17e67951d47e67b59ef3170d17
Test: manually inspected build files for libWallpaperPicker & android-suppor-v4
2023-01-05 21:13:57 +00:00
Cole Faust
b85d1a15cc Bp2build support for multiple product configs
Create a
build/bazel/product_config/generated/products/<product_name>/BUILD
file that contains the platform definitions needed for
a particular product. Currently we just create it for the
current lunch target, but the idea is that eventually when
all product config is in starlark, all the products will
have their platform definitions in the tree at once.

Bug: 249685973
Test: Presubmits
Change-Id: I08c82ff28dcf62f09d3b1d2e3186a6b961e12f6e
2023-01-05 12:49:14 -08:00
Alix Espino
d363771bc1 Merge "Libs support for * -> java_library_edge with kt sources" 2023-01-04 22:11:55 +00:00
Zi Wang
9f609db46a Add variant_prepend support for all the properties in bp2build
This is a follow up of aosp/2336916. All the properties that have
variant_prepend tag are supported in bp2build.

Bug: 261644490
Test: TH and modified tests with updated expected behaviors
Change-Id: I13bb5d4d8fcd38bf153fa9083c9c945aec2dfa19
2023-01-04 11:19:50 -08:00
Vinh Tran
55225e343d Remove apex_available tag in cc_library_static generated from stubs-providing lib
When a cc library has stubs, its static variant shouldn't be included in an apex to ensure the library with stubs isn't statically linked. See https://cs.android.com/android/platform/superproject/+/master:build/soong/apex/apex.go;l=2804;drc=89b01aeaa9e19377ff547baab791277719b8aaf3

This CL modifies bp2build so that it excludes apex_available tag from the static target if the library has stubs. This way, we can ensure that no apex can depend on the static target which results to static linking.

Test: go test
Bug: 255589949
Change-Id: Iedf248994b808436f2440570b094fd06d6284ae9
2023-01-03 21:44:07 +00:00
Trevor Radcliffe
db7e0263f4 bp2build conversion for Sanitize.Integer_overflow
and Sanitize.Misc_undefined.

This is a redo of aosp/2277186 with changes to account for issues
in mixed builds. Specifically, for now we're disabling mixed builds
for modules that use ubsan. This bug tracks enabling it:
b/261058727.

Bug: 253428057
Test: Unit tests
Change-Id: Ia1277a7fa9a82f40998d7f3d2c40ea90f38201e7
2022-12-29 21:21:23 +00:00
Spandan Das
87c149dccb Merge "Create (API) bp2build converters for droidstubs" 2022-12-29 17:58:45 +00:00
Spandan Das
0b555e323b Create (API) bp2build converters for droidstubs
- The converter runs for api_bp2build, not bp2build workspace
- Since droidstubs is an internal module created by java_sdk_library,
  the conversion encompasses the latter as well
- Since droidstubs do not have an api_surface attribute, this conversion
  uses naming convention to infer the api_surface represented by the api
  file e.g. *stubs.source -> publicapi, *stubs.source.system -> systemapi)
- Also adds an SdkIntraCore enum to represent the API surface provided
  by one core module to another

There is also ongoing work to check in java_api_contribution modules in
Soong. Once we have that, we can update this converter to operate on
that module type instead

Test: go test ./bp2build
Change-Id: Ia85828e04c738d9ffcc524856d7c3034ee29bbf9
2022-12-28 23:58:52 +00:00
Zi Wang
0f82844b75 Add variant_prepend test for cc_library
The comments in aosp/2336916 are also addressed here

Test: TH and TestCcLibraryVariantPrependPropOrder
Change-Id: If1472658fcd4b5544dec2e2691049a180520c84e
2022-12-28 11:18:11 -08:00
Treehugger Robot
aebab397b3 Merge "Swap error before setting to prevent nil error" 2022-12-22 08:12:06 +00:00
Liz Kammer
44bc9a3bee Swap error before setting to prevent nil error
Test: go test soong tests
Change-Id: Ib6e3519f0f596b3699882fe682e648dd69ed5645
2022-12-21 15:02:20 -05:00
Colin Cross
62e4fe15dc Merge changes I47b8fdd9,Ifbd121ab
* changes:
  Return linux as host OS from getConfigString when using musl
  Use //build/bazel/platforms/os:linux_glibc for osLinux
2022-12-21 19:33:14 +00:00
Vinh Tran
72e1d65f05 Merge "Remove creation of has_stubs in cc_library_shared by bp2build" 2022-12-21 03:16:08 +00:00
Colin Cross
133782e4bb Use //build/bazel/platforms/os:linux_glibc for osLinux
osLinux is implicitly glibc, use //build/bazel/platforms/os:linux_glibc
for the configurability select key for it.

Bug: 259266326
Test: build/bazel/ci/mixed_libc.sh
Test: USE_HOST_MUSL=true build/bazel/ci/mixed_libc.sh
Change-Id: Ifbd121ab3d943f3fdfbd3f6c3e982d1eaaaa9ec1
2022-12-20 15:29:31 -08:00
Yu Liu
295cdfc79c Merge "Enable code coverage for cc shared library" 2022-12-20 20:20:12 +00:00
Vinh Tran
83f88cdc8b Remove creation of has_stubs in cc_library_shared by bp2build
Because when stubs_symbol_file is defined on cc_library_shared, has_stubs is already implied to be true.

Bug: 260771171
Test: go test
Change-Id: I3b9a2cfb9000411d1b5f22da87dc339267775799
2022-12-19 13:56:11 -05:00
Cole Faust
abc182cc94 Merge "Allow exporting bazel mixed build allowlists to simple text files" 2022-12-19 03:09:11 +00:00
Kevin Dagostino
32edd1a608 Add tradefed attributes to cc_test bazel macro
The BUILD file will now have the needed attributes to run the tradefed
test rules.

Test: b test //packages/modules/adb:adbd_test --platforms=//build/bazel/platforms:android_target
Change-Id: I5cb84ccdde27a7fadbef479269df594344a29187
2022-12-18 04:38:16 +00:00
Yu Liu
f01a0f0ca9 Enable code coverage for cc shared library
Bug: None
Test: Manual and unit tests
Change-Id: I5064e4407d3451ff172ab203256f3c073cc0ad1b
2022-12-16 14:16:22 -08:00
Cole Faust
705968d043 Allow exporting bazel mixed build allowlists to simple text files
So that they the allowlists can be read from build_mainline_modules_bundles.sh.

Bug: 262302253
Test: m out/soong/bazel_prod_mixed_builds_enabled_list.txt out/soong/bazel_staging_mixed_builds_enabled_list.txt && cat out/soong/bazel_prod_mixed_builds_enabled_list.txt && echo && cat out/soong/bazel_staging_mixed_builds_enabled_list.txt
Change-Id: Id5f89ebb651a52dc5368681a20d6183703e05bd4
2022-12-15 15:40:36 -08:00
Alix
266bbf2f9c Libs support for * -> java_library_edge with kt sources
Bug: 244210934
Test: bo2build tests and built kotlinx_atomicfu
Change-Id: Id6eac2f104878e4d7902a32e846ef1cc87dd7863
2022-12-15 06:27:59 +00:00
Treehugger Robot
6441a125cf Merge changes from topics "bp2build_cc_object_dynamic_deps", "bp2build_cc_prebuilt_object"
* changes:
  Add support for headers from dependencies to bazel cc_object
  Add bp2build support for cc_prebuilt_object
2022-12-13 16:25:58 +00:00
Vinh Tran
6916999cd0 Merge changes from topic "afdo-in-cc-rules"
* changes:
  Change out path to android_target-opt in bazel integration tests
  Add check for handcrafted BUILD file in bp2build
  Add bp2build conversion to reference to fdo_profile targets
2022-12-13 00:12:03 +00:00
Treehugger Robot
331360e7d6 Merge "Add support for variant_prepend properties in bp2build for cc library" 2022-12-12 22:46:33 +00:00
Vinh Tran
bc9c8b494c Add check for handcrafted BUILD file in bp2build
ag/20469925 added a handcrafted BUILD file (allowlisted in ag/20597986)
to internal master. ag/c/platform/build/soong/+/20583593/12/cc/bp2build.go#818 generates references to the targets in the BUILD file. However in tm-dev, the BUILD file doesn't exist while the .afdo files do.

One solution is to cherry-pick the BUILD file to tm-dev. However, tm-dev doesn't have vendor/google/build/soong/bp2build_allowlist.go to check in the BUILD file in bp2build.

This CL adds a check that the BUILD file exists to avoid the failure as in https://android-build.googleplex.com/builds/pending/P45724500/aosp_cf_x86_64_phone-userdebug/latest/view/logs/build_error.log in tm-qpr-dev-plus-aosp branch.

Bug: 253540178
Test: go test
Change-Id: I47fb853015ca230afe3cefe1d37728bf714624be
Merged-In: I47fb853015ca230afe3cefe1d37728bf714624be
2022-12-12 12:17:49 -05:00
Vinh Tran
99270ea3c3 Add bp2build conversion to reference to fdo_profile targets
The fdo_profile targets are handcrafted in ag/20469925. Bp2build creates fdo_profile attribute on cc_library_shared and references to the fdo_profile target if the profile exists. This works under assumption that all afdo profiles have an associated Bazel fdo_profile target declared in the same folder/package.

As noted in b/253540178, this won't work when we check in the BUILD files (which is likely 6+ months away) because some profiles might only exist internally but not in AOSP. We will implement a long-term solution once we figure out all the requirements we need to support afdo both in Soong and Bazel.

Bug: 253540178
Test: go tests
Change-Id: Iebd5dd7a76583b4b2bf0d9ee56d58f247d313a54
Merged-In: Iebd5dd7a76583b4b2bf0d9ee56d58f247d313a54
2022-12-12 12:16:43 -05:00
Zi Wang
1cb11800f7 Add support for variant_prepend properties in bp2build for cc library
Some properties in soong are marked "variant_prepend", which means
in bp2build output, variant properties(select ...) should come before
general properties.
Prepend property is added in StringListAttribute
stuct so that bp2build can handlle variant_prepend properly, if
Prepend is properly set in different module logics.
This change also coordinates cc library with Prepend.
Coordination with other module types will be in separate CLs.

Bug: 261644490
Test: TestCcLibraryHeadersSimple,
TestCcLibraryHeadersArchAndTargetExportSystemIncludes and TH

Change-Id: I3b61c491c54bbe91ba9aa2af9b4c84193c91ae7f
2022-12-09 16:41:23 -08:00
Colin Cross
65ebc429e0 Add support for headers from dependencies to bazel cc_object
The libc_musl_crt* cc_object modules use header_libs to add
headers to the search path.  Propagate static_libs, shared_libs
and header_libs to includes_deps.

Bug: 259266326
Test: TestCcObjectHeaderLib
Change-Id: I8db4d6886761426d3ece38c43ac868d3248f7a9f
2022-12-09 13:58:58 -08:00
Colin Cross
c5075e917e Add bp2build support for cc_prebuilt_object
Add bp2build support for cc_prebuilt_object to allow buiiding
musl crt objects that use prebuilt clang_rt.crtbegin and
clang_rt.crtend.

Bug: 259266326
Test: //extenral/musl:libc_musl builds
Change-Id: Ic2b0375bc6e7336d31c9e2cf51bdc2ea894185aa
2022-12-09 13:58:58 -08:00
Trevor Radcliffe
682bfb2923 Revert "bp2build conversion for Sanitize.Integer_overflow"
This reverts commit f82482db31.

Reason for revert: http://b/261935841

Change-Id: I8155c2b17c6052b734efea43be06b1099ffedd32
2022-12-09 20:39:12 +00:00
Trevor Radcliffe
f82482db31 bp2build conversion for Sanitize.Integer_overflow
and Sanitize.Misc_undefined.

This is a redo of aosp/2277186 with changes to account for issues
in mixed builds. Specifically, for now we're disabling mixed builds
for modules that use ubsan. This bug tracks enabling it:
b/261058727.

Bug: 253428057
Test: Unit tests
Change-Id: I9dbec1d99353f8cbe6751d5241283ab19592dc27
2022-12-06 20:13:21 +00:00
Paul Duffin
4c0765a669 Support running bp2build tests in fixtures
Previously, the fixture test infrastructure was hard coded to
initialize itself in preparation for invoking the PrepareBuildActions()
method. That meant it could not be used for testing the bp2build and
apiBp2build modes which required different initialization and called a
different method.

This change extracts that behavior into a FixtureTestRunner and adds an
implementation that allows it to test the above two modes. It then uses
that to implement the runBp2BuildTestCaseWithSetup method.

The TestPrebuiltLibraryAdditionalAttrs was the only test which broke as
it supplied an invalid path to the MockFS (it does not support using a
trailing / to represent an empty build directory). So, it was modified
to add a couple of files inside those directories instead.

Test: m nothing
Change-Id: I6798a4f761160af8d1bfed81d46de9628bda3eb9
2022-12-02 11:14:17 +00:00
Jingwen Chen
c4c34e1eab Convert apex_available (for supported modules) to bazel tags.
The tags will then be read by an aspect applied from apex attributes to
validate that the deps are explicitly opt-in to the appropriate apex.

The semantics of this is the same as the Soong apex_available property.

This also adds the "base_apex_name" attr to override_apex conversion so
that apex_available checks continue to work on the non-Google and non-Go
apex names.

Test: presubmits
Fixes: 218841706
Change-Id: I19b3637da9aa47573b252d74f62b601fbdc3784d
2022-12-01 06:25:25 +00:00
Yu Liu
734fefdf90 Merge "Support abi check in bazel." 2022-11-30 17:57:42 +00:00
Alix
341484b3da Bp2Build for libs property in java_library
Bp2build for both *->java_library and java_library->* edges.

Change-Id: I2540c9af2ae2fe7677767d507647b1c6669b9bb7
Bug: 244210934
Test: ./bp2build testcase, manually inspected build files for hamcrest-library, hamcrest targets
2022-11-29 19:44:50 +00:00
Yu Liu
56ccb1aecd Support abi check in bazel.
Bug: 253498204
Test: Manual and unit test.
Change-Id: Id23c4f772f67c4ba05704eaad77322133474b42b
2022-11-29 11:25:10 -08:00
Jingwen Chen
ed6240d1bc Merge "Revert "bp2build conversion for Sanitize.Integer_overflow"" 2022-11-29 02:35:07 +00:00
Jingwen Chen
0ead79bd11 Revert "bp2build conversion for Sanitize.Integer_overflow"
Revert submission 2277186-ubsan_bp2build

Reason for revert: b/260630869 broke mixed_droid
Reverted Changes:
Idd9b33e53:bp2build conversion for Sanitize.Integer_overflow
I0d636e439:Add Sanitizer libs up the dependency tree
If113de7bc:Introduce toolchain features for sanitizers

Change-Id: Idf7e7b62f6fb6d4ff92341cc4c9c4b7426b70951
2022-11-28 22:44:35 +00:00
Trevor Radcliffe
202f2afbf8 Merge "bp2build conversion for Sanitize.Integer_overflow" 2022-11-28 21:45:07 +00:00
Trevor Radcliffe
548fc5dcec bp2build conversion for Sanitize.Integer_overflow
and Sanitize.Misc_undefined.

Bug: 253428057
Test: Unit tests
Change-Id: Idd9b33e537d1b1c80b0c96a9590327dae13fada7
2022-11-28 19:31:34 +00:00
Liz Kammer
ffc17e4edb Use excludes from OS axis for OsAndInApex
The axis for OS and in APEX was previously ignoring the excludes from
OS, which would result in excludes for non-Android OSes being
incorrectly ignored.

Test: go test Soong tests
Test: b build //hardware/libhardware:all
Bug: 260131489
Change-Id: Ie77f763bea0f473ac69a0c6b6bc3216e7359ad12
2022-11-23 10:01:30 -05:00
Sam Delmerico
775f2cb3cd Merge "export neverallow include dir list to Bazel" 2022-11-16 20:52:17 +00:00
Sam Delmerico
46d08b4e39 export neverallow include dir list to Bazel
Bazel needs to recreate the neverallow behavior for absolute includes,
so export the list of include dirs to soong_injection.

Bug: 250931909
Test: aosp/2302925
Change-Id: I3bd345ea4cfb3181f07e4570466f2fc8c5fe32da
2022-11-15 17:33:21 -05:00
Alix
8062f4dee7 bp2build for java_library with .kt srcs or common_srcs
java_library modules with .kt srcs or common_srcs will be converted into
module type kt_jvm_library

Test: m bp2build, manually inspected build files for allowlisted modules
Bug: 258688914
Change-Id: I8293a11c8247b4b76358d0991f82c6b61b58adc3
2022-11-15 20:01:29 +00:00
Sam Delmerico
1bde1cb668 Merge changes from topics "bp2build-tidy-disabled-srcs", "bp2build-tidy-timeout-srcs"
* changes:
  bp2build for tidy_timeout_srcs
  bp2build for tidy_disabled_srcs
2022-11-10 16:27:24 +00:00
Sam Delmerico
4c902d696a bp2build for tidy_timeout_srcs
Bug: 195029134
Test: m bp2build
Change-Id: I59fdc16d8c224f42167e3b5a3857f4ff2f546825
2022-11-08 17:18:52 -05:00
Sam Delmerico
c9b8fbdda0 bp2build for tidy_disabled_srcs
Bug: 195029134
Test: m bp2build
Change-Id: I9591439213dbf0ef68cd33151b3e32f6f6c68551
2022-11-08 17:17:52 -05:00
Paul Duffin
c639059fd8 Make CodegenContext reference *android.Context
Previously, it stored an android.Context (no pointer) which while it
worked (because the current contents are themselves pointers) it is
inconsistent with how the rest of the code references it and could
cause weird behavior if additional fields were added to the Context.

Test: m nothing
      m bp2build
Change-Id: I9c70f925dde85e5b15431cc232b0c2eb6371ec6d
2022-11-05 17:43:12 +00:00
Lukács T. Berki
72ab470a29 Merge "Multithread symlink forest removal." 2022-11-02 07:45:22 +00:00
Usta (Tsering) Shrestha
3bb3763afd Merge "include symlink metrics in bp2build_metrics.pb" 2022-10-31 20:00:36 +00:00
Lukacs T. Berki
bc5f731791 Multithread symlink forest removal.
This is a second attempt at aosp/2273288 which got rolled back because
it turned out that:

1. We make ~120K symlinks in AOSP (!), all of which need to be deleted
2. System calls are sometimes slow
3. Golang spawns a new OS-level thread for each blocking system calls to
   keep cores busy

All this together means that we sometimes had 10K system calls in
flight, which meant 10K OS-level threads, which is when Go gives up and
says "I created too many threads, please help".

The fix is to move the system calls into a pool of goroutines, which
soon end up on a pool of threads (since they mostly do blocking system
calls)

Test: Presubmits.
Change-Id: Ia9aefff3b0ed373f09bb6c8b2ec1d8b0f00b213b
2022-10-31 16:04:13 +00:00
usta
4f5d2c1e97 include symlink metrics in bp2build_metrics.pb
Bug: b/256212479
Test: Prior to thi CL bp2build.symlink_forest event was missing in bp2build_metrics.pb after a clean mixed build
Change-Id: I53bfc4114a383c0d1f9c4c7945e7d4c69bc50b0c
2022-10-31 11:38:10 -04:00
Alix
b4e09a0ada Bp2build Java libs for java_binary -> java_import edge
Since Bazel's java_import requires a jars attribute to be specified,
the generated neverlink-duplicated module is of type java_library

Change-Id: I14a866dfc583507a9462add50d95060cbfe540c5
Bug: 244210934
Test: m bp2build, go test ./bp2build, manual inspection of generated Build and jar files
2022-10-28 20:26:38 +00:00
Christopher Parsons
ed2873aea0 Revert "Multithread symlink forest removal."
This reverts commit 6b236f1607.

Reason for revert: Breaks ab/aosp-master-bazel incremental builds. Details on b/254338319

Change-Id: I37eeeda50cff0475d91e7926fdf74216975a0037
2022-10-28 15:43:48 +00:00
Lukacs T. Berki
6b236f1607 Multithread symlink forest removal.
This makes symlink forest creation ca. 2x faster again, taking 2-3
seconds instead of 5.

Who would have thought that os.RemoveAll() is slow.

Test: Presubmits.
Change-Id: I91e41319c972dbf1113cf723e383c785433c18b9
2022-10-28 07:15:55 +00:00
Lukács T. Berki
4bb30b59a9 Merge changes from topic "separate-symlink-forest-invocation"
* changes:
  Build the symlink tree on multiple threads.
  Create Bazel symlink forest in a separate process.
2022-10-28 07:05:00 +00:00
Lukacs T. Berki
647e7abfa2 Build the symlink tree on multiple threads.
This makes it take ~5 seconds on AOSP instead of ~10. Frankly, the
speedup is somewhat disappointing but at least the code is not
complicated.

Test: Presubmits.
Change-Id: Icf94d7ca8bd80c458d014f4cf4cc1be7138deaa6
2022-10-27 10:17:20 +00:00
Lukacs T. Berki
c541cd27fa Create Bazel symlink forest in a separate process.
This helps with incrementality a lot: the symlink forest must depend on
almost every directory in the source tree so that if a new file is added
or removed from *anywhere*, it is regenerated.

Previously, we couldn't do this without invoking bp2build, which is
quite wasteful because bp2build takes way more time than the symlink
forest creation, even though we do the latter in a very suboptimal way
at the moment.

This means that if a source file is added or removed (which does not
affect globs), we don't pay the cost of bp2build anymore.

Also refactored symlink_forest.go on the side. Too much state was being
passed around in arguments.

This change reimplements aosp/2263423 ; the semantics of not touching an
output file is the exact same as order-only inputs and the latter is a
bit fewer lines of code.

Test: Presubmits.
Change-Id: I565c580df8a01bacf175d56747c3f50743d4a4d4
2022-10-27 08:08:45 +00:00
Yu Liu
2ff53da9f8 Merge "Support apex_test." 2022-10-26 16:37:59 +00:00
Usta Shrestha
783ec5c72c Remove a self-dependency of bp2build
bp2build (i.e. bootstrap.ninja#bp2build_workspace_marker) generate $OUT/soong/bp2build/**/BUILD.bazel files
Having them as a dependency would thus make bp2build_workspace_marker stale upon incremental builds (because we don't re-touch the marker file if it already exists)

See Also: https://android-review.googlesource.com/c/platform/build/soong/+/2263423

Bug: b/253450880
Test: NINJA_ARGS='-d explain' m --bazel-mode adbd
      repeat and see if bp2build is rerun
prior to this CL, one would see the input /usr/local/google/home/usta/aosp/out/soong/bp2build/build/make/tools/BUILD.bazel trigger bp2build over and over again

Change-Id: I904cd333a5d6ef506fc4019eda7623ef96a1daa3
2022-10-25 13:46:20 +00:00
Yu Liu
4c212ce058 Support apex_test.
Bug: 249744489
Test: Manual and unit test.
Change-Id: I866debf7ffee54a0f4649b5e5425a8e3cc3042a1
2022-10-24 10:35:12 -07:00
Usta Shrestha
49d04e89d8 cosmetics
Test: m --bazel-mode nothing
Bug: b/239044236
Change-Id: Iaffc315c696f2fa19a2525009010d5964cf5a7d1
2022-10-21 20:09:23 +00:00
Treehugger Robot
95ac23eeeb Merge "Introduce BazelStringOrLabelFromProp." 2022-10-20 18:26:20 +00:00
Jingwen Chen
6817bbb3c8 Introduce BazelStringOrLabelFromProp.
Soong supports string properties, but they are overloaded, and can mean
one of three things:

* path reference
* module reference
* string literal

Bazel has different types: label and string attributes. Thus there needs
to be a way to categorize them correctly in bp2build.

This CL introduces a new function to be used on properties like
apex_key.private_key / apex_key.public_key, as well as
android_app.certificate / apex.certificate.

It is important to disambiguate the prop betenn a string literal
attribute or file/rule target label attribute, so this functions does
just that.  The new attributes are then further handled by their
respective macros (apex_key, android_binary, apex).

Bug: 253557437
Fixes: 253557437
Test: presubmits, new tests

Change-Id: Id8111cdd60d3aabcae7d17fe9da84d0ee3966023
2022-10-20 14:57:37 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
c5184ec089 Rename prebuilt_library_* to cc_prebuilt_library_*
Follow the convention of everything else in `cc`

Test: CI
Change-Id: I133d0b7119d4e9b357aada3467bc079f20dc0f60
2022-10-20 14:51:55 +00:00
Yu Liu
d6201013eb Make cc_genrule.srcs configurable.
Also disallow arch variant of genrule.out.

This is to be consistent with bazel where we are migrating to.

Bug: 254114674
Test: Manual
Change-Id: I685a2e64102b7bb68128b39931f0bc85878bc6de
2022-10-19 12:35:26 -07:00
Vinh Tran
827db14152 Merge "Default apex's compile_multilib to "first" in bp2build" 2022-10-17 14:12:29 +00:00
Vinh Tran
8f5310f90c Default apex's compile_multilib to "first" in bp2build
In Soong, decodeMultilib, used to get multilib to determine the dep variations, return "first" if defaultMultilib is set to "common". apex sets defaultMultilib to "common" which means equivalent compileMultilib in bp2build for apex should be "first" (See new Soong unit tests for more context).

This CL fixes bp2build for apex to be more correct.

Bug: 251559512
Test: go tests
Change-Id: Id1cb4407980fc1fab91822c81326f37fb4adfa0a
2022-10-13 16:59:30 -04:00
Wei Li
7d8f6182f9 Fix some issues in bp2build converter for python_binary_host.
1) Bp2build convert python_binary_host main attribute as LabelAttribute. Currently "main" attribute in python_binary_host is handled as string but for some modules (e.g certify_bootimg) the "main" attribute points to a file in its subpackage like "subpackage/file.py" and should be converted to "//.../subpackage:file.py".

2) Filter out duplicated labels in the merged list of "required" attributes of python_binary_host and its defaults.

Test: b build //system/tools/mkbootimg:certify_bootimg
Test: b build //build/make/tools/releasetools:check_target_files_signatures
Bug: 253081249
Bug: 253101186

Change-Id: Ic2cb4cadec2c1348da70af9f0730da9914d3a8ca
2022-10-12 17:43:20 -07:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
49b59385da Merge "Support cc_prebuilt_binary building with Bazel" 2022-10-10 21:37:35 +00:00
Spandan Das
4238c65a17 Converters for contributions to systemapi and vendorapi
The module types in scope of this conversion are
1. cc_library and cc_library_shared (non-null llndk or stubs prop)
2. cc_library_headers (all)

For (2), we need some postprocessing on the results of the parser
bp2BuildParseBaseProps. This is necessary because arch and os specific
API exports need to be flattened and added to the generateed API headers
target along NoConfigAxis
e.g.
```
The api equivalent of
cc_library_headers (
  name = "lifoo",
  deps = select({
  	"//build/bazel/platforms/arch:arm": ["arm_deps"],
  	"//build/bazel/platforms/arch:arm64": ["arm64_deps"],
	}),
)
should be
cc_api_library_headers (
  name = "lifoo",
  deps = ["arm_deps", "arm64_deps"],
)
```

For (1), we also need to generate 1:many header api targets so that
arch-specific deps can propagate arch metadata to the top-level
api_domain rule

Test: go test ./bp2build
Test: go test ./cc

Change-Id: Ie40cba1ac8e89f290b3d926c190d5e93abd52859
2022-10-07 23:48:00 +00:00
Treehugger Robot
41a7fedd39 Merge "Fix incrementality bug with merged BUILD files" 2022-10-07 22:21:26 +00:00
Cole Faust
fd88414b4c Fix incrementality bug with merged BUILD files
The merged build files should be added as ninja dependencies,
so we rerun when they're changed.

Fixes: 246552590
Test: m bp2build, m bp2build again and observe it didn't rerun. Then add a comment in external/protobuf/BUILD.bazel, run m bp2build again, and observe it reruns
Change-Id: I26ed035cc0a894500a192f9aa3371fb46519689b
2022-10-07 13:11:45 -07:00
Trevor Radcliffe
4e503ec09d Merge "Split ldflags in bp2build" 2022-10-07 17:31:29 +00:00
Treehugger Robot
d504adbead Merge "apex_key bp2build: disambiguate module and src deps." 2022-10-07 10:55:50 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
b12ff59f0b Support cc_prebuilt_binary building with Bazel
Bp2build-enable cc_prebuilt_binary -> cc_prebuilt_binary

Bug: 241415823
Test: cc_prebuilt_binary_conversion_test.go
Change-Id: I007deef8d44f68993012f2114314d1cb52cfbb0e
2022-10-06 19:39:09 +00:00
Jingwen Chen
34feb1471a apex bp2build: refactor module names to be more meaningful.
Numbered module names weren't obvious.

Test: presubmits
Change-Id: Iee921e2965d1664ed1bccd934ab80f913eff6356
2022-10-06 13:02:30 +00:00
Jingwen Chen
1d873331ad apex_key bp2build: disambiguate module and src deps.
(and remove unused keyName field.)

The private_key and public_key props of an apex_key can point to either
a module or a string. If it's a module, then respect it. If it's a
string, there's additional product variable lookup to find the apex_key
files in product_vars's DefaultAppCertificate parent dir.

This is similar to android_app_certificate.

Test: presubmits
Change-Id: Ib258da14cb0c2df8b5f817fcbc46afebcf225db8
2022-10-06 05:56:05 +00:00
Spandan Das
e191ae76b6 Merge "Move API providing modules from bp2build to api_bp2build workspace" 2022-10-05 19:40:01 +00:00
Spandan Das
ee08eb3c81 Merge "Create a new mode in soong_ui to generate API only BUILD files" 2022-10-05 01:37:28 +00:00
Trevor Radcliffe
6d91fe70e2 Merge "Update tests for cc_* converters" 2022-10-04 19:06:21 +00:00