Commit graph

4911 commits

Author SHA1 Message Date
Usta (Tsering) Shrestha
1ad62c7073 Merge "ShouldKeepExistingBuldFileForDir look up by dir" 2022-11-19 03:55:46 +00:00
Sasha Smundak
4975c82666 Provide Bazel timing breakdown for the mixed builds
Instead of single event in the build metrics
```
| soong_build.mixed_build.bazel          | 1:17.1 |
```
it will now show:
```
| soong_build.mixed_build.bazel          | 1:17.1 |
| soong_build.mixed_build.bazel.cquery   |   51.9 |
| soong_build.mixed_build.bazel.aquery   |   23.1 |
| soong_build.mixed_build.bazel.symlinks |    2.1 |
```

Test: treehugger
Change-Id: I84a7dda6e3122860da9aaa98bfa6afe33d392dcf
2022-11-16 18:44:08 -08:00
Wei Li
c33c2e732a Merge "Fix the following issues in bazel_handler:" 2022-11-16 23:45:47 +00:00
Sam Delmerico
775f2cb3cd Merge "export neverallow include dir list to Bazel" 2022-11-16 20:52:17 +00:00
Wei Li
cbd181c8e8 Fix the following issues in bazel_handler:
1) Convert coverage path wildcard to regex values for --instrumentation_filter to Bazel.
2) Handle the error properly when aquery commmand returns error.

Bug: 258397103
Test: CIs
Change-Id: Ie7485c2c45f2eb0e1ca0dc60478aae04ab151b61
2022-11-16 08:59:23 -08:00
Alix Espino
3044987181 Merge "bp2build for java_library with .kt srcs or common_srcs" 2022-11-16 15:40:55 +00:00
Usta Shrestha
ea9996411d ShouldKeepExistingBuldFileForDir look up by dir
instead of scanning every entry in the map for prefix-match, search each ancestor of the dir

Bug: N/A
Test: bazel_test.go#TestShouldKeepExistingBuildFileForDir()
Change-Id: I7c9afa456cca5a70624ae1e9c9b96c43b0110c1e
2022-11-15 21:39:37 -05:00
Sam Delmerico
46d08b4e39 export neverallow include dir list to Bazel
Bazel needs to recreate the neverallow behavior for absolute includes,
so export the list of include dirs to soong_injection.

Bug: 250931909
Test: aosp/2302925
Change-Id: I3bd345ea4cfb3181f07e4570466f2fc8c5fe32da
2022-11-15 17:33:21 -05:00
Alix
8062f4dee7 bp2build for java_library with .kt srcs or common_srcs
java_library modules with .kt srcs or common_srcs will be converted into
module type kt_jvm_library

Test: m bp2build, manually inspected build files for allowlisted modules
Bug: 258688914
Change-Id: I8293a11c8247b4b76358d0991f82c6b61b58adc3
2022-11-15 20:01:29 +00:00
Bob Badour
3800b5a787 Only generated notices for built variants.
Test: banchan dcla-notice mainline_modules_arm64; m apps_only dist
Change-Id: I6b0e2eb64f2f3db87dfed68affb44fe141da3eaa
2022-11-14 14:17:55 -08:00
Paul Duffin
280765acc4 Merge "Improve error reporting when depending on prebuilt implementation jar" 2022-11-14 16:38:11 +00:00
Treehugger Robot
4d87828d15 Merge "Export ndk/aml arches to Bazel" 2022-11-14 16:17:41 +00:00
Paul Duffin
73d394dec2 Merge "Allow namespace config to be tested properly" 2022-11-14 14:08:40 +00:00
Paul Duffin
c61783b20d Improve error reporting when depending on prebuilt implementation jar
The sdk snapshot must not be including implementation code for boot
libraries, the implementation is provided by dex jars within the
corresponding APEX. However, the snapshot does need a module for each
boot library so that the build can seamlessly access the dex files from
the APEX.

A java_library boot library (like core-oj) is represented in the
snapshot by a java_import module which requires a jar file to be
provided, otherwise it is disabled. However, that is provided purely
to keep Soong happy and should never be used.

Previously, the snapshot would contain an empty file for the jar. As
an empty file is an invalid jar any tool (like compiler) that tried
to consume it would fail which was the correct behavior. Unfortunately,
the error message that was produced was not very helpful, it was just
some variant on `invalid file` which lead to a lot of bugs being
raised.

This change replaces that empty file with a reference to the output
from a genrule which runs a script which produces a more useful error
message, with information on how to fix the issue, and fails the build.

It also adds a Name() method to the SdkMemberProperties type as that is
needed in AddInternalModule() to construct the name of the additional
module.

Tested as follows:

In AOSP/master make the following changes:
1. Temporarily set visibility on core-oj and core-libart to
   //visibility:public.
2. Run packages/modules/common/build/mainline_modules_sdks.py to create
   the snapshots.

For each of the S, T and latest snapshots I did the following in the
s-aml-prebuilt-test, t-aml-prebuilt-test and aosp/master branches:

1. Created an Android.bp file containing the following:
  java_library {
    name: "broken",
    static_libs: [
      "prebuilt_core-libart",
      "prebuilt_core-oj",
    ],
  }

2. Fix the visibility issues and run `m broken` where it fails with an
   invalid file.

3. Delete the contents of the prebuilts/module_sdk/art/current/sdk
   directory.

4. Unpack the relevant version of the art-module-sdk snapshot into the
   directory.

5. Run `m broken` where it fails with the helpful message.

6. Test the instructions on how to use the ninja -t path tool to
   identify the cause of the problem and fix it.

Bug: 257969510
Test: See above.
Change-Id: I125bde2d7202afff84c97daebcef37e21c548a3a
2022-11-14 10:14:10 +00:00
Kevin Dagostino
92f33d91f3 Add tools/asuite/atest/ to keep existing BUILD file list
Test:  b build tools/asuite/... && b cquery tools/asuite/...
--output=files

Change-Id: I2696327a60e770d0929ccc1baa3b4af703a2269d
2022-11-12 04:43:17 +00:00
Liz Kammer
992918d21f Export ndk/aml arches to Bazel
Test: lunch ndk-eng; b build //bionic/libc --config=android
Change-Id: I1ec6f7d0f084aafe948f27111275537b6d9e9637
2022-11-11 15:27:56 -05:00
Vinh Tran
7a8362c252 Remove AfdoAdditionalProfileDirs product variable
AfdoAdditionalProfileDirs is unconnected to build/make/core/soong_config.mk and is currently unused. Removing AfdoAdditionalProfileDirs will cut down on complexity when we migrate the support of auto-fdo profiles to Bazel cc rules.

b/258684450 proposes an alternative to support the use cases that were meant to be supported by AfdoAdditionalProfileDirs.

Test: existing go tests
Bug: 253540178
Change-Id: I024e48598a560717a1d6c19b16bde3fe65ff1961
2022-11-11 03:29:31 +00:00
Liz Kammer
dbb9fe8e92 Merge "Increase testing of request_type" 2022-11-10 13:30:58 +00:00
Treehugger Robot
01dd619766 Merge changes from topic "clang-tidy-mixed-builds"
* changes:
  add tidy files to mixed builds
  add --norun_validations to bazel build action
2022-11-10 01:00:15 +00:00
Liz Kammer
1b7ed9bcbc Increase testing of request_type
Test: go test soong tests
Change-Id: Iacb773a91a4cc097320f06ea1d1669c5923f8076
2022-11-09 11:22:11 -05:00
Jingwen Chen
dece457443 Add com.android.tzdata to AOSP staging allowlist.
Test: presubmits
Change-Id: I7d9edb56273375579edf3b8a5466570df6073668
2022-11-09 09:30:03 +00:00
Jingwen Chen
1ec7785ec5 bazel apex: Add mixed builds support for <module>_using.xml file.
This file contains the java APIs parsed by the 'dexdeps' tool within the gen_java_usedby_apex.sh.

Bug: 239084755
Fixes: 239084755
Test: presubmits
Change-Id: Ia271783a6be3ea3a343481306cde1aaba2166e88
2022-11-09 06:30:03 +00:00
Romain Jobredeaux
64340c0539 Merge "Sort allowlist ascii-betically" 2022-11-08 21:57:45 +00:00
Paul Duffin
3f7bf9fa0d Allow namespace config to be tested properly
Previously, the NewTextContext created a context that always called
NewNameResolver with a export filter that always returned true.

This change fixes that by:
1. Changing NewNameResolver to take a Config parameter instead of a
   filter parameter and pushing the code to create the filter from the
   Config from newNameResolver() in cmd/soong_build/main.go into the
   NewNameResolver function.

2. Extracting a newTestContextForFixture that does not create a
   NameResolver or set it on the context. That avoids creating a
   NameResolver before the test has prepared the config.

3. Modify the fixture to create and set the NameResolver in the Context
   after the config has been prepared by the test.

4. Added test to verify that it all works.

Bug: 234825639
Test: m nothing
Change-Id: Ie4b13f18093dc01a0ab65a9ecfd143585d843b55
2022-11-08 17:38:56 +00:00
Chris Parsons
66fc74509f Refactor "staging dep on prod" allowlist dep
This allows branch-specific plugins to more easily add modules to
staging or prod mode allowlists (they need not separately ensure that
staging mode is a superset of prod mode)

Bug: 254447469
Test: TH
Test: Verified that adding tzdata modules via plugin (not part of
this CL) causes many outputs under bazel-out directories

Change-Id: I5d543e262a42cce324c59e3f9880b57ca209c13c
2022-11-08 05:54:19 +00:00
Jingwen Chen
1214001184 Merge "Call rm -rf in mixed build build statement preamble." 2022-11-07 23:03:43 +00:00
Liz Kammer
f1c70ed5ad Merge "Add apex_test for mixed builds" 2022-11-07 21:56:02 +00:00
Sam Delmerico
658a4da53b add --norun_validations to bazel build action
Validation actions were being run before the build environment was set
up and causing build errors. Turn off validation actions in this phase
because they will be run later in the Ninja invocation.

Bug: 195029134
Test: build/bazel/ci/mixed_libc.sh
Test: verify that Ninja build file contains commands to generate tidy
  files that are generated by these validation actions
Test: delete a tidy file in mixed builds mode, and verify that it is
  rebuilt
Change-Id: I371572d5662913c5637e39c8894eeb6ee53d00ac
2022-11-07 15:58:19 -05:00
Romain Jobredeaux
77488afdc1 Sort allowlist ascii-betically
Change-Id: I49b62fdc2446bd0a88305ce1243c80cc8b7bc680
2022-11-07 16:56:12 +00:00
Liz Kammer
0e255ef6e6 Add apex_test for mixed builds
Test: go test soong tests
Change-Id: I33327e1fe7d6a4bbf32890d9fbd5453c28a9c5b1
2022-11-07 11:11:37 -05:00
Jingwen Chen
f3b1ec39eb Call rm -rf in mixed build build statement preamble.
https://android-review.git.corp.google.com/q/topic:bazel_using_txt added
a tree artifact output for an action. This CL adds "-r" to the rm calls
to support deleting the old tree artifact outputs.

Test: presubmits
Fixes: 257956807
Change-Id: Ia44b1bef4f8031fe75a239fb162972e8827e3c19
2022-11-07 15:10:04 +00:00
Paul Duffin
780a185ede Avoid loss of ninja_deps events
Previously, writeDepFile was passed a copy of the main EventHandler to
which it then added an event. Adding an event requires appending
information to slices in the EventHandler structure. As the slices are
a copy the addition only affects the copy and are not present in the
EventHandler which is written out as part of the metrics.

This change fixes that by passing a pointer to an EventHandler into
writeDepFile. For consistency it also replaces a few other uses of
EventHandler with *EventHandler instead.

Bug: 257590265
Test: m nothing
      # Check generated text metrics to make sure they don't contain ninja_deps.
      # Apply this change.
      m nothing
      # Check generated text metrics to make sure they do contain  ninja_deps.
Change-Id: I0fa0f2b08844cb7c7b4e99b37602703deb43ad20
2022-11-07 14:12:11 +00:00
Treehugger Robot
d6dec6f51e Merge "Rename ApexCqueryInfo to ApexInfo" 2022-11-05 00:36:45 +00:00
Treehugger Robot
17d0a8509b Merge "Add missing t.Helper() to androidmk functions" 2022-11-05 00:34:42 +00:00
Treehugger Robot
979c953ac0 Merge "Make PackageVarContext implement PathGlobContext" 2022-11-04 20:50:04 +00:00
Liz Kammer
be6a712767 Rename ApexCqueryInfo to ApexInfo
cquery is redundant given this is in the cquery package.

Test: go test soong tests
Change-Id: Ib38794b712f548129eec964b8fd85bf8401a5f86
2022-11-04 16:24:37 -04:00
Liz Kammer
6be6906429 Add missing t.Helper() to androidmk functions
Test: go test soong tests
Change-Id: I534aef51afa91a0942f58b3987e9c1cb3f3c75cb
2022-11-04 16:24:32 -04:00
Colin Cross
662d61430f Make PackageVarContext implement PathGlobContext
Make PackageVarContext implement PathGlobContext by implementing
GlobWithDeps.  This will allow calls to ExistentPathForSource
inside a VariableFunc to use optimized glob dependencies instead of
falling back to AddNinjaFileDeps, which is resulting in extra
dependencies from soong_build on top level directories, triggering
extra Soong regenerations.

Remove the fallback path in ExistentPathForSource by making it take
a PathGlobContext, which is now a superset of PathContext.

Rewrite TestNinjaDeps to not rely on the unoptimized glob dependencies
in VariableFuncs and instead call ctx.Config().AddNinjaFileDeps
directly.

Bug: 257079828
Test: test_create_global_include_directory
Change-Id: I48cf189157d78b9252d339dbc9baeb27e4694807
2022-11-04 09:04:02 -07:00
Usta (Tsering) Shrestha
9478303673 Merge "add test for ShouldKeepExistingBuldFileForDir()" 2022-11-03 02:44:08 +00:00
Ibrahim Kanouche
2d1eb1238d Merge "Export SBOM utility to make" 2022-11-03 00:03:21 +00:00
Usta Shrestha
342f28f9bf add test for ShouldKeepExistingBuldFileForDir()
Bug: N/A
Test: Added
Change-Id: I15dc543f65bffdd745ecfe5c8fe521723eeffbc0
2022-11-02 17:21:41 -04:00
Ibrahim Kanouche
5797d8c7f5 Export SBOM utility to make
Bug: 257094286

Test: m compliance_sbom
Change-Id: I2a8de6350c3a5b24cd3b3fd03312955ce1a3dba9
2022-11-02 19:17:30 +00:00
Cole Faust
acf417b085 Merge "Revert^4 "Prevent unspecified values in soong_config_string_variables"" 2022-11-02 16:56:45 +00:00
Cole Faust
b0a91330bb Revert^4 "Prevent unspecified values in soong_config_string_variables"
This reverts commit b7b2e12475.

Reason for revert: Issues on -plus-aosp branches have been fixed

Change-Id: Ie2f7ce45502b999280e099d9c0f9d0f7d6b7ba08
2022-11-01 17:10:23 +00:00
Treehugger Robot
0b2788e833 Merge "Skip the noop Soong dep edge for api_domain and contributions" 2022-11-01 03:07:14 +00:00
Sam Delmerico
8604df6fb1 Merge "export clang-tidy variables to soong_injection" 2022-10-31 17:23:19 +00:00
Alix
b4e09a0ada Bp2build Java libs for java_binary -> java_import edge
Since Bazel's java_import requires a jars attribute to be specified,
the generated neverlink-duplicated module is of type java_library

Change-Id: I14a866dfc583507a9462add50d95060cbfe540c5
Bug: 244210934
Test: m bp2build, go test ./bp2build, manual inspection of generated Build and jar files
2022-10-28 20:26:38 +00:00
Christopher Parsons
057ad009c7 Merge "Remove adbd from bazel prod allowlist" 2022-10-28 16:19:35 +00:00
Jason Wu
1cb6739c01 Merge "Rollforward "Use proto output instead of jsonproto output for aquery""" 2022-10-28 14:47:52 +00:00
Lukács T. Berki
4bb30b59a9 Merge changes from topic "separate-symlink-forest-invocation"
* changes:
  Build the symlink tree on multiple threads.
  Create Bazel symlink forest in a separate process.
2022-10-28 07:05:00 +00:00