Commit graph

56 commits

Author SHA1 Message Date
Cole Faust
6054cdf3b1 Generate android_certificate_directory
Previously, partners were required to add an
android_certificate_directory filegroup in their certificate
directories, and allowlist that BUILD file. Now, we generate the
filegroup automatically.

We're using a different name, generated_android_certificate_directory,
to avoid conflicts with already-checked-in filegroups.

Bug: 285777389
Test: b test //build/bazel/rules/apex/...
Change-Id: Ib1bde487acd79d58368faf0aad02ded0bcdaceb4
2023-09-12 15:23:36 -07:00
Chris Parsons
73f411b72c Delete ApiBp2build soong_build mode
This feature is obsolete.

This makes a large number of codepaths "dead code" (such as
module-specific implementations of ApiBp2build functionality). These
will be deleted in a followup CL.

Bug: 284029211
Test: Presubmits
Change-Id: Ib53b99f1fe8c24380d219caf44e9bb3b96724fa0
2023-09-05 17:09:51 -07:00
Liz Kammer
32c634bca9 Fix false pos in bp2build-prog due to prebulits
The prebuilt_ prefix was removed before writing converted modules, which
causes false positives when a prebuilt and from source module exist with
the same name but only one has been converted

Test: b run //build/bazel/scripts/bp2build_progress:bp2build_progress \
    -- report -m core-lambda-stubs-for-system-modules
Change-Id: Id00099780fb6af9fffcf745b509116a66bac8756
2023-08-25 17:42:42 -04:00
Vinh Tran
80f6b2147f Export RustDefaultVersion to Bazel
The config constant is consumed by rust toolchain implemented in aosp/2709668.

Bug: 295918553
Test: go test
Change-Id: I0a28a802ad28f6ee24dfb1b5bdab72094534ae04
2023-08-23 22:55:40 -04:00
usta
40caf95004 cosmetic: unused argument
Test: m nothing
Bug: NA
Change-Id: I2a647e2e7c9c2fd3881c18e3cbb072b260cd9659
2023-08-04 16:53:31 -04:00
Cole Faust
8a161be19b Move bazel environment variable list to bzl files
So that it's easier to maintain by being closer to the actual long-term
usage.

Bug: 279095899
Test: m nothing
Change-Id: I0b7cfa2eeb48595dd3081b458bbd1156ec2184a3
2023-06-14 15:45:12 -07:00
Cole Faust
3486740cc5 Use api_levels_released_versions from starlark
Instead of exporting it to soong_injection.

Bug: 279095899
Test: m nothing
Change-Id: I7b93af233b7450848a475512b5f5682ece773c09
Merged-In: I7b93af233b7450848a475512b5f5682ece773c09
2023-05-02 01:05:07 +00:00
Cole Faust
37d27c4884 Fix crash when missing platform sdk version
Fixes: 277890306
Test: Presubmits
Change-Id: I26658a90856cb057c631bb71ca0cd094ed648cf5
2023-04-12 10:32:18 -07:00
Jingwen Chen
aea34a38ca Merge "Revert^2 "Re-land test tzdata apex with bazel builds."" 2023-04-12 06:50:26 +00:00
Cole Faust
eb644cfb34 Break constant information out of product vars
We don't want rules to load the product variables directly, because
then they won't be able to transition on them.

Break constant information that is safe to load outside of the
product vars file, so that we can make the product vars file have
more restricted visibility later.

Bug: 269577299
Test: m nothing
Change-Id: I848bff33e4f5798f51296ea3a2600615cab36985
2023-04-11 15:38:43 -07:00
Jingwen Chen
6134211c2d Revert^2 "Re-land test tzdata apex with bazel builds."
fb8b1a69e9

Change-Id: Id5c0edd1cd4aacbb75a8019e0a5ada4e608fbf09
2023-04-11 14:55:27 +00:00
Cole Faust
18994c73f1 Replace SortedStringKeys with SortedKeys
Now that we have generics.

Bug: 193460475
Test: presubmits
Change-Id: I1594fd8feb505175d5c09c03ef397e5ffd5b09cb
2023-02-28 16:51:32 -08:00
Alix Espino
4fd7e74140 Revert^2 "refactor of api_levels map for soong injection"
a865f4e905

Change-Id: Ifea020ac83725df7131773ca587c56931f5cdf03
2023-02-24 14:46:43 +00:00
Matthias Männich
a865f4e905 Revert "refactor of api_levels map for soong injection"
Revert submission 2441746-api_levels_refactor

Reason for revert: broke Android kernel build tools build: https://ci.android.com/builds/branches/aosp_kernel-build-tools/grid?

Reverted changes: /q/submissionid:2441746-api_levels_refactor

Change-Id: I35206879b93ec95ae325025cb764c6c34880dccf
2023-02-23 17:10:27 +00:00
Alix
c566ad8a4f refactor of api_levels map for soong injection
Change-Id: I58449fc84617e46727f23ab3d8dd5f118d2ee0d2
Bug: 249265719
Test: go test ./bp2build and locally ran bazel tests
2023-02-21 18:55:37 +00:00
Cole Faust
9e384e2e6b Export apex_available_baseline to soong_injection
So that it can be used in the bazel implementation.

Bug: 268006095
Test: m nothing, check that it's in out/soong/soong_injection
Change-Id: I1520dd874076dee7fa083648d0790b060d658e5a
2023-02-08 17:43:09 -08:00
Sam Delmerico
cb3c52c766 export allowlist of environment variables to Bazel
Previously, Bazel was invoked during mixed builds with a stripped down
environment. This CL adds an allowlist of environment variables that are
passed to Bazel.

Test: WITH_TIDY=1 DISABLE_ARTIFACT_PATH_REQUIREMENTS=true mss tidy-packages-modules-NeuralNetworks --bazel-mode-dev
Change-Id: I23147bec59f6522953cf623e7bcaa0f1f99a75a3
2023-02-06 14:34:43 -05:00
Trevor Radcliffe
4f95ee96b8 Export sanitizer vars for use in Bazel
Bug: 265154269
Test: Unit tests
Change-Id: I46584159264476d76079e396c6ec294fb394acff
2023-01-23 20:28:06 +00:00
Spandan Das
83e787e837 Add a new CreateSoongInjectionDirFiles function
- This is a wrapper function for all files in soong_injection directory.
This should prevent an error in the workspace generated for
api_bp2build.
- Rename the existing CreateSoongInjectionFiles fn and make it package
  private to prevent confusion

(The subsequent CL in this stack should contain a smoke test for
api_bp2build)

Test: b build //:empty --config=api_bp2build --config=android
Test: TH
Change-Id: Iddb0aa1aff2f709826edd587aa99fccddf80f08f
2023-01-11 03:21:00 +00:00
Cole Faust
705968d043 Allow exporting bazel mixed build allowlists to simple text files
So that they the allowlists can be read from build_mainline_modules_bundles.sh.

Bug: 262302253
Test: m out/soong/bazel_prod_mixed_builds_enabled_list.txt out/soong/bazel_staging_mixed_builds_enabled_list.txt && cat out/soong/bazel_prod_mixed_builds_enabled_list.txt && echo && cat out/soong/bazel_staging_mixed_builds_enabled_list.txt
Change-Id: Id5f89ebb651a52dc5368681a20d6183703e05bd4
2022-12-15 15:40:36 -08:00
Sam Delmerico
46d08b4e39 export neverallow include dir list to Bazel
Bazel needs to recreate the neverallow behavior for absolute includes,
so export the list of include dirs to soong_injection.

Bug: 250931909
Test: aosp/2302925
Change-Id: I3bd345ea4cfb3181f07e4570466f2fc8c5fe32da
2022-11-15 17:33:21 -05:00
usta
4f5d2c1e97 include symlink metrics in bp2build_metrics.pb
Bug: b/256212479
Test: Prior to thi CL bp2build.symlink_forest event was missing in bp2build_metrics.pb after a clean mixed build
Change-Id: I53bfc4114a383c0d1f9c4c7945e7d4c69bc50b0c
2022-10-31 11:38:10 -04:00
Spandan Das
5af0bd3e48 Create a new mode in soong_ui to generate API only BUILD files
The generated Bazel workspace will only contain api specific targets.
This is feasible since these targets do not have any cross dependencies
with the targets in the bp2build workspace

The advantages of a new mode are
1. Does not pollute bp2build workspace with api targets
2. Does not block api targets with the current allowlist conversion
   mechansims in bp2build
(In the future we might want to combine these two workspaces)

A Soong module type will generate a Bazel target if it implements
ApiProvider interface

Test: m apigen
Test: m nothing

Change-Id: I69c57ca6539f932e0ad554ce84a87fb7936fdba0
2022-10-04 17:32:56 +00:00
Kevin Dagostino
60f562ac7e adding /out/soong_injection/metrics/converted_modules_path_map.json
This file can be used by atest/scripts/tools to determine which modules
have been converted and what their bazel build path is.

Test: b build //packages/modules/adb/...
will generate the file at /out/soong_injection/metrics/converted_modules_path_map.json
Also added a unit test to check that the file is created.

{
  "adbd": "//packages/modules/adb",
  "adbd_test": "//packages/modules/adb",
  "add_ext4_encrypt": "//external/e2fsprogs/contrib",
  ....
}

Change-Id: Ibd424c487840d84c8a4fb635828a73541220b36b
2022-09-20 04:01:13 +00:00
Sasha Smundak
8bea267caf Implement bp2build for the package module
* We are interested mostly in the conversion the `default_applicable_licenses`
  attribute, as `default_visibility` cannot be handled until every module's
  `visibility` is handled.
* Several referenced license modules had to be manually enabled for
  conversion, and likewise a few trivial Android.bp containing only
  package and license modules.
* As Bazel allows only a single `package` rule, the package rules in a
  manually converted BUILD.bazel files were removed (in
  external/protobuf and prebuilts/clang/host/linux-x86 trees).
* The converted package rule is emitted before the `load` statements per
  Bazel documentation.

Bug: 190817312
Test: treehugger
Change-Id: If8bf6fee1580177de3bb402923615bcd48923ed2
2022-09-14 13:49:27 -07:00
Sam Delmerico
263efde0a8 ignore versions_with_info property in queryview
Struct properties are not fully supported by queryview, so disable this
property for now.

Bug: 245727956
Test: m queryview && tools/bazel run --config=bp2build --config=ci --//build/bazel/rules/apex:apexer_verbose --config=linux_x86_64 //build/bazel/scripts/bp2build-progress:bp2build-progress -- graph -m com.android.neuralnetworks --use-queryview
Change-Id: I74da94da82a037234fb2a8fe17780ed0eb309a7b
2022-09-08 10:51:50 -04:00
Cole Faust
ea602c5a41 Remove old BUILD file merging code
It's not needed anymore since aosp/2197837,
it should've been removed in that cl but I forgot.

Bug: 234167862
Test: ./build/bazel/ci/bp2build.sh
Change-Id: I3d67a6e1894ad401525070ad37d3158708898306
2022-08-31 15:12:47 -07:00
Cole Faust
324a92e404 Always merge build files
Previous behavior:

- Packge not listed in bp2buildKeepExistingBuildFile:
    - Use bp2build generated build file
- Package listed in bp2buildKeepExistingBuildFile:
    - Use handcrafted build file even if there were allowlisted bp2build
      modules in the same package.
- Package listed in bp2buildKeepExistingBuildFile and a soong module has
  a bp2build: { label } attribute:
    - Merge the handcrafted and bp2build generated build files

New behavior:

- Packge not listed in bp2buildKeepExistingBuildFile:
    - Use bp2build generated build file
- Package listed in bp2buildKeepExistingBuildFile:
    - Merge with bp2build generated build file.

Bug: 234167862
Test: ./build/bazel/ci/bp2build.sh
Change-Id: Ifbaf4f8f0f5158b5b2bd6d534eb2311e2e5f399b
2022-08-30 16:07:23 -07:00
Jingwen Chen
7810e17e54 Export default apex version from updatable_modules.go to Bazel's
apex_toolchain through soong_injection.

This is necessary to keep both Bazel and Soong in sync.

Test: presubmit
Bug: 238723069
Fixes: 238153998

Change-Id: Ibf4a8ed91d358dc7800c10e4112485a52f0606bb
2022-08-02 10:11:36 +00:00
Sasha Smundak
0fd93e0756 Output informational messages only when BP2BUILD_VERBOSE is set.
Test: manual
Change-Id: Ieeb00a002e07b04449d70614ce205c47c1dd7bce
2022-07-18 13:34:44 -07:00
Jingwen Chen
d9d096ec7c queryview: don't generate 'for' attributes
'for' is a reserved keyword

Test: CI
Fixes: b/233579439
Change-Id: I7b94250574465353bbd6bfb82a7ea37ce9ae17a3
2022-05-23 09:38:39 +00:00
Sam Delmerico
932c01cf9e export Java variables to Bazel
Test: build/bazel/bp2build.sh
Change-Id: Ia06f9265c9f96e6add6edbd0cee925fe37b430d3
2022-04-08 14:15:43 +00:00
Yu Liu
fc603167f4 Support tailoring clang --target flag based on min sdk version.
Bug: 215748260
Test: Tested using aquery to verify the --target flag; add added unit

Change-Id: Ifb13509db7d1e110316aa44d55ec9cccaa4b83d1
2022-03-18 01:16:52 +00:00
Liz Kammer
e8303bd0ae Export variant/features via soong_injection
Test: build/bazel/ci/bp2build.sh
Change-Id: If70043dc9d020d5f4a243ba4b192b99c9c7e8563
2022-02-28 14:17:15 -05:00
Jingwen Chen
0ee88a6b65 Implement stubs.symbol_file and stubs.versions for cc_library_shared bp2build.
This CL turns the stubs.symbol_file and stubs.versions properties into
stubs_symbol_file and stubs_version attributes on the cc_shared_library
target. See associated build/bazel change on how these attributes are
used to generate stub libraries.

Bug: 207812332

Test: New tests
Test: CI
Change-Id: Ie23eafb9903a131d92ff4e251215e998cea0a763
2022-01-17 13:03:12 +00:00
Jingwen Chen
01812020c1 Add support for writing all Soong config variables into @soong_injection.
Also remove the need to use bp2build_available on
soong_config_module_types as we want to convert every single of them
into the tree for a complete soong_injection soong_config_variables.bzl
file.

The variables are split into their bool, value and string types
respectively, as they all need to be handled differently on the Bazel
product_platform side, as well as for generating constraint values and
settings. For example, value variables need to integrate with
TemplateVariableInfo, and string variables need to include the string
value itself into the select key/constraint value.

Sample soong_config_variables.bzl file: https://gist.github.com/jin/cef700bfb20c8656a931306dd71d47e1

Test: CI
Bug: 198556411
Change-Id: I8665dd1269a507edb37de62407ed3641564bea5c
2021-11-23 08:37:30 +00:00
Chris Parsons
3b1f83d076 Add x86_host vars to config.bzl
Test: USE_BAZEL_ANALYSIS=1 m adbd
Test: Manually verified config.bzl contains various x86_host flags after
bp2build.
Test: Unit tests

Change-Id: Ie9201ea2be4cd1c6659bea088a797cedbae37403
2021-10-19 12:18:13 -04:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
447f6c99c9 Bp2Build common properties auto-handling
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule

Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.

As initial user, the Soong `required` property will be
translated to Bazel's `data`.

Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
2021-10-04 14:43:04 +00:00
Liz Kammer
7a210ac233 bp2build: Add support for export_.*headers props
Soong supports export_.*_headers properties, the libraries contained in
this list must also be within a shared/static/whole_static/header libs
property. For bp2build, we eliminate this duplication. The libraries
not listed in an export_.*_headers property will migrate to an attribute
prepended with implementation_, those in export_.*_headers will not have
a prefix.

Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Bug: 198241472
Change-Id: I3eb84c983ec5d241c8a568e411dfd5619d3184a7
2021-09-23 16:07:45 -04:00
Jingwen Chen
6117450ead bp2build: add converted modules to codegen metrics, and remove the
compat layer.

The list of converted modules is used for tracking bp2build progress.

The module_name_to_label aliases weren't really adding much value to
mixed builds, so let's drop the feature for now and simplify the code.

Test: CI
Test: USE_BAZEL_ANALYSIS=1 m droid
Change-Id: I3269344cfe6d1c72c108d66c6e820bc0ec19bb34
2021-09-17 08:47:16 +00:00
Jingwen Chen
c63677b3c9 Direct Bazel builds from m.
This CL adds support to bp2build/Soong to dump a BUILD file under
out/soong/soong_injection/targets containing alias targets to their real
targets for every converted Soong module, regardless of whether they are
handcrafted or generated.

Test: TH
Change-Id: Ic1816fda5d019c395301618134fac68b3057d752
2021-06-21 06:12:34 +00:00
Jingwen Chen
4910976314 Make BUILD file merging slightly smarter.
This change enables checked-in BUILD files like
prebuilts/clang/host/linux-x86/BUILD.bazel to be merged cleanly with the
bp2build generated one into the synthetic workspace.

The checked-in BUILD file contains a package() declaration that bp2build
also generates. To avoid double declaration, the BUILD file writer now
checks if the BazelTargets contain handcrafted targets. If so, it
delegates the package declaration to the handcrafted BUILD file instead.

This change also sorts the bp2build targets before the handcrafted ones,
and adds a section header to demarcate the two sets of targets.

Test: TH
Change-Id: I3ecdeaab3226b895b623daf0791d24a657f7a7c6
2021-05-25 07:01:55 +00:00
Rupert Shuttleworth
0096079bbc Make bazel build //external/... work by only allowing "approved" existing BUILD files in the Bazel workspace.
Test: bazel build //bionic/... //external/... //frameworks/... //system/...

Test: ./build/bazel/scripts/run_presubmits.sh

Change-Id: I91865ca87c6535053e7a14d2526ff3ce0991bfea
2021-05-17 06:34:53 -04:00
Jingwen Chen
bf61afb7f7 bp2build: export some cc toolchain flags into Starlark.
This CL exports common/global/device/host clang/ld/ldd flags
from their Ninja variable initialization locations in
cc/config/global.go and cc/config/clang.go to make Bazel's cc_toolchain
and Soong's cc actions more consistent with each other.

This does not handle env-dependent or arch-specific toolchain flags
yet (logic in compiler.go and linker.go).

Test: TH
Bug: 187086342
Bug: 187084737
Bug: 186628704
Bug: 187857770
Change-Id: Ie403d7cd23f35160897b9dd902c799cbf1bd7f0c
2021-05-13 06:25:47 +00:00
Liz Kammer
78c68655e1 Write all BUILD files for queryview
Some BUILD files cannot be written to support bp2build as they already
exist in the tree; however, all BUILD files should be written for
queryview as they are not intended to coexist with handwritten BUILD
files.

Test: $ bazel query --config=queryview
  "somepath(//system/linkerconfig:linkerconfig--android_arm64_armv8-a ,
  //bionic/libc:libc_openbsd--android_arm64_armv8-a_static)"
Test: build/bazel/ci/bp2build.sh
Fixes: 186132826
Change-Id: I2cb26b7482e7eb5523cee33452be3753ad37e1b0
2021-04-22 14:40:17 -04:00
Rupert Shuttleworth
2a4fc3ecdc Generate BUILD files for every directory that has an Android.bp file.
Test: Added an integration test
Test: bazel build --package_path=out/soong/workspace //bionic/...

Change-Id: Ie34bd23ab3c5428e6c9c9919e5fb6fcb4e709adc
2021-04-21 07:15:50 -04:00
Jingwen Chen
6c309cd0ac bp2build: don't generate a WORKSPACE.
bp2build doesn't need to generate a WORKSPACE file.

Also cleanup conversion_test.go.

Test: TH
Change-Id: Iba2f4b2f71d72ccf8b148cd11d78c190f93560c4
2021-04-01 11:50:27 +00:00
Jingwen Chen
12b4c2706d bp2build: add allowlist for package-level conversions.
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.

It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.

Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.

Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
2021-03-24 02:27:19 -04:00
Liz Kammer
ba3ea16f14 Use handcrafted build targets in bp2build
If both bp2build_available and label are specified, label will be
preferred.

Initially, we copy the entire BUILD.bazel file. Eventually we may move
this to use bazel query for a more accurate result.

Test: go test *
Test: build/bazel/scripts/milestone-2/demo.sh full
Test: GENERATE_BAZEL_FILES=true m nothing
      edit bionic/libc/tools/BUILD.bazel
      GENERATE_BAZEL_FILES=true m nothing and verify changes picked up
Bug: 180516554
Change-Id: I43025583300e6b10d2c18032cd4a76237b578d59
2021-03-15 13:09:25 -04:00
Jingwen Chen
88ae408a0f queryview: add "size" to ignored prop.
Test: m queryview && bazel query //... --config=queryview
Change-Id: Ia84be6e2fc5b302b4a147f94bfa219a729b67be1
2021-02-24 19:55:50 -05:00