Commit graph

38348 commits

Author SHA1 Message Date
Jingwen Chen
53dfa408b4 Generate build-globs.ninja when running bp2build.
This fixes null bp2build builds.

When running bp2build, out/soong/.bootstrap/build-globs.ninja wasn't
generated, therefore none of the globs in out/soong/.primary/globs have
actions to generate them. This makes bp2build always re-run on a null
build, because it depends on those non-existent globs in its depfile.

Fix the above by generating a build-globs.ninja file with the globs that
bp2build encounters during conversion, and add tests to verify that null
builds and bp2build -> soong workflows correctly regenerates the
build-globs.ninja file.

Test: soong integration tests for null soong->soong, bp2build->bp2build and bp2build->soong builds.
Fixes: 193650250
Change-Id: Icfbad351dfa85541d1ea451f73e65cac87097926
2021-08-12 10:14:02 +00:00
Rupert Shuttleworth
755ceb01e3 Ensure directories in the Soong output dir have appropriate permissions, so that 'm clean' works.
In particular, Bazel sometimes creates directories whose files cannot be deleted.

Test: m clean used to fail after b build //system/timezone/apex:com.android.tzdata, but now it works
Test: Added integration test which fails without this change

Change-Id: I08c8feed21c31ec187fe40be513f7eb4865c8ac3
2021-08-12 05:07:11 -04:00
hamzeh
c0a671fc80 Moving common fuzzing code to fuzz package
Test: make haiku and make haiku-rust
Change-Id: Ife80cc10672f51bd6afbae7061cc9373a2a15e7d
2021-08-11 23:54:15 +00:00
Yabin Cui
60880e0517 Merge "Update clang version used for bindgen" 2021-08-11 21:12:27 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
ccb6a01324 Merge "Lift BazelHandler interface into android/" 2021-08-11 21:01:56 +00:00
Yuntao Xu
402e9b08e1 Add Platform_sdk_version_or_codename variable
1. Added a new product variable Platform_sdk_version_or_codename;
2. This variable is used to support the logic in the Android.bp file
when either the Platform_sdk_version or the Platform_sdk_codename is
used.

Bug: 195986342
Test: m CtsThemeHostTestCases (the module where the product variable is
used)
Test: TreeHugger

Change-Id: I6b952ce47fd4adc280aa3c7ef4185487558af9bc
2021-08-11 12:33:37 -07:00
Paul Duffin
1f0cc77010 Merge "Filter blocked entries from modular flag files" 2021-08-11 18:36:37 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
0d99045b69 Lift BazelHandler interface into android/
Because it's commonly useful and not cc/ specific
Also export GenerateBazelBuildActions and update uses

Test: Existing tests pass
Change-Id: Ibc6858bb1129afba181a7686dda432defe33b00d
2021-08-11 18:19:40 +00:00
Joel Galenson
adffbe9166 Merge "Add libselinux to the Rust allowlist so we can run its test." 2021-08-11 17:20:35 +00:00
Paul Duffin
280bae6d20 Filter blocked entries from modular flag files
Previously, the sdk snapshot would include all the entries from the
stub-flags.csv and all-flags.csv modular files generated by a single
bootclasspath_fragment. That included implementation details, i.e.
class members that are not part of a stable API or the hidden API which
meant that the sdk snapshots were implementation dependent.

This change removes the implementation details from the modular flag
files, i.e. those entries that are only marked as "blocked". When
comparing the files against the corresponding subset of the monolithic
files it assumes that any entries missing from the modular flag files
are blocked.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I6b67b2253cf029d6830b58a06ebb0c8fcaa0dd71
2021-08-11 17:29:47 +01:00
satayev
721f42d12a Merge "Fix tests to use correct fixtures for configuring (Apex)BootJars." 2021-08-11 16:13:20 +00:00
Treehugger Robot
00caefe11e Merge "Allow //external/bazel-skylib to use existing BUILD files" 2021-08-11 14:58:33 +00:00
Colin Cross
7131982a76 Merge "Collect lint reports from apps in apexes" 2021-08-11 14:15:35 +00:00
Colin Cross
88c5391dbe Merge "Remove libjemalloc5 from default musl libraries" 2021-08-11 14:14:48 +00:00
Lukács T. Berki
4ddcd8152b Merge changes I07b0ca98,I1c2901e1
* changes:
  Cut the multiproduct_kati -> soong-ui-build dep.
  Do not create a build.Config in multiproduct_kati.
2021-08-11 13:17:47 +00:00
Lukacs T. Berki
f656b8434b Cut the multiproduct_kati -> soong-ui-build dep.
This is done by moving SetupSignals() to its own little package.

There are a number of tiny little utility packages for soong_ui we might
be better of merging, but that's for another change (maybe)

Test: Presubmits.
Change-Id: I07b0ca98bfb8884ef4223d665e632183b9896a0d
2021-08-11 15:15:02 +02:00
Lukacs T. Berki
2c40569db0 Do not create a build.Config in multiproduct_kati.
This mostly cuts the dependency from multiproduct_kati to the build
package; the remaining logic is SetupSignals which will be dealt with in
a followup change.

This requires some code duplication (e.g. ensureFileExists() and
detectTotalRAM()).

This removes a Finder invocation added in aosp/452598; it looks like
it's not a measurable performance hit. The quoted half a second does not
matter much in build_test{1,2,3,4} which take an amount of time measured
in hours.

It also makes ANDROID_QUIET_BUILD unsupported for the UI of
multiproduct_kati; the function of that variable seems to be to suppress
the output from build actions, which don't exist in multiproduct_kati so
it should not matter. In addition, I checked whether anyone sets this
flag for multiproduct_kati and it doesn't appear to be the case.

Test: Presubmits.
Change-Id: I1c2901e1ea69c3a35429e6c967a7bba845676ef7
2021-08-11 15:14:47 +02:00
Jingwen Chen
916322541a Allow //external/bazel-skylib to use existing BUILD files
Bug: 190605804
Test: bp2build-incremental
Change-Id: Ida765465532c0e98d035eef4c703d13fbf554538
2021-08-11 12:19:17 +00:00
Paul Duffin
726d44a86d Merge "Remove member signature and inner classes from signature-patterns.csv" 2021-08-11 09:08:01 +00:00
Treehugger Robot
1dde1fade7 Merge "[scripts] Print full clang prebuilt dir from get_clang_version.py" 2021-08-11 06:57:27 +00:00
Lukács T. Berki
3eff609379 Merge "Make multiproduct_kati call soong_ui.bash ." 2021-08-11 06:12:06 +00:00
Treehugger Robot
95048e9625 Merge "Removed commented out code" 2021-08-11 05:30:45 +00:00
Colin Cross
0bcba80823 Merge "Add symbols for start and end of embedded linker" 2021-08-11 02:28:38 +00:00
Colin Cross
8355c1524f Collect lint reports from apps in apexes
Lint reports were only being collected from jars in apexes, not
apps.

Bug: 188772607
Test: mainline_modules_arm64 build
Change-Id: Ie84819343f8b8ab3f43496bbc9b7876d0d0d8e70
2021-08-10 19:25:59 -07:00
Yabin Cui
1424ea499d Update clang version used for bindgen
Bug: 190099451
Test: none
Change-Id: Ie0480288bb1be2e7d55d8dd01b8d006410bfc7a0
2021-08-10 15:13:01 -07:00
Christopher Parsons
9f7c388337 Merge "Support rtti in bp2build" 2021-08-10 20:05:44 +00:00
Yabin Cui
39ff202748 Merge "Revert^2 "Switch to clang r428724 13.0.1."" 2021-08-10 19:22:15 +00:00
Joel Galenson
e4d6390558 Add libselinux to the Rust allowlist so we can run its test.
Test: Build and atest
Change-Id: Id828f286b35decfd16166e9cbb78cdf477014bf6
2021-08-10 11:54:14 -07:00
LaMont Jones
b02043b84d Merge "Improve handling of errors from zipinfo" 2021-08-10 18:12:27 +00:00
Chris Wailes
90c4b14a30 Removed commented out code
A previous CL commented out code instead of removing it.  This CL fixes
this grave error.

Test: TH
Change-Id: I1daeaaa1ad90230aec639d603e7cdb9b61af251a
2021-08-10 10:21:02 -07:00
Romain Jobredeaux
d9eb0f0aa6 Merge "Exempt prebuilts/jdk and build/bazel/examples/java BUILD files from bp2build overwriting" 2021-08-10 16:29:35 +00:00
Treehugger Robot
6fd3047115 Merge "Support exporting device arch toolchain info" 2021-08-10 16:11:52 +00:00
Chris Parsons
2c7883941e Support rtti in bp2build
Rtti was already handled by bazel macros, so this change simply
propagates the rtti bit to these macros.

Test: Run bp2build, build //external/libcxxabi:all
Change-Id: I63296db2db868202874c8bd0b1de6310f7bb85c7
2021-08-10 11:58:07 -04:00
Yabin Cui
10bf3b8c00 Revert^2 "Switch to clang r428724 13.0.1."
786c1d41ef

Change-Id: Ic143d1b3375c8258e0e4274339524380de391676
2021-08-10 15:42:10 +00:00
Lukacs T. Berki
cef87b62e4 Make multiproduct_kati call soong_ui.bash .
This serves to not link parts of soong_ui (and eventually soong_build)
into a separate, weird binary. This is in turn good because they contain
any number of global variables and no one really thought about what
happens when two instances are executing at the same time in the same
address space.

This comes with a slight performance hit: 5 aosp_* projects build 152
seconds instead of 146. I suppose this is a price worth paying for a
clean design?

Test: presubmits.
Change-Id: I5623dcab2290f0fc392dd2ede597b9794a3d2a4e
2021-08-10 17:35:14 +02:00
satayev
abcd59731e Fix tests to use correct fixtures for configuring (Apex)BootJars.
This is in preparation to r.android.com/1740313 where setting correct
variables would be enforced (i.e. apex and non-apex boot jars must be in
config.ApexBootJars and config.BootJars correspondingly).

Bug: 191369843
Test: m nothing
Change-Id: Ic86680c1f7af53d229083b2cc58beb3ceccb4b6a
2021-08-10 15:35:52 +01:00
Paul Duffin
6ffdff853c Remove member signature and inner classes from signature-patterns.csv
Previously, the signature-patterns.csv file included a lot of
implementation details, e.g. the signatures of dex members or inner
classes that are not part of any API, including the hidden API.

This change will remove all member signatures and inner class names
from the file and replace them with just the outermost qualified class
name.

That will still leave some implementation details, e.g. the names of
implementation only classes and packages.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I9de6a2a6129e875e19f7ded5fae578cbdb584660
2021-08-10 14:01:51 +01:00
Paul Duffin
8f34b0e19d Merge "Use trie to store monolithic hidden API flags" 2021-08-10 09:26:56 +00:00
Chris Wailes
353405d9fb Clean up generated rust-project.json files
This CL removes the roots section from the rust-projects.json file as it
is no longer required by the rust-analyzer tool.

Bug: 162055556
Test: prebuilts/rust/tests/rust-analyzer-integration-test.sh
Change-Id: I85c3277632c88394bc52e709ecce83f6aa1f2e0e
2021-08-09 15:50:39 -07:00
LaMont Jones
4224c6216d Improve handling of errors from zipinfo
Bug: 153900481
Test: manual
Change-Id: Ib60578ead4ccaa7ba09f628a6c9281c0cc70e6ce
2021-08-09 19:31:39 +00:00
Christopher Parsons
67d6ccecfc Merge "bp2build: handle system_shared_libs" 2021-08-09 19:11:00 +00:00
Paul Duffin
c11f6679d8 Use trie to store monolithic hidden API flags
Previously, a simple map from dex signature to the flags was used to
store the monolithic hidden API flags. This change replaces that with a
trie that uses packages and class names extracted from the signature
to map from the signature to the flags.

The signature is broken down into pieces consisting of package names,
class names and the remaining member signature. They are used in turn
to navigate through nodes in the trie to a Leaf that contains the flags
associated with the signature.

The flags can be retrieved from the trie in a number of ways:
* Using the complete signature to retrieve the flags for a single
  member.
* Using a class name to retrieve the flags for all the members in a
  class and any nested classes.
* Using a package wildcard to retrieve the flags for all the members in
  the classes in that package.
* Using a recursive package wildcard to retrieve the flags for all the
  members in the classes in that package and nested packages.

This will allow a bootclasspath_fragment to select a subset of the
monolithic flags to compare against itself without specifying the
signatures of every member of that set.

Before creating the trie an attempt was made to compute the subset by
iterating over all the signatures in the monolithic flags and matching
against a regular expression created from the patterns but that was too
slow. It took minutes to run whereas using the simple map or the trie
only takes a few seconds.

Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: I36f5e319d3e7d62dd34305de1eec990a93cb3a89
2021-08-09 19:51:33 +01:00
Chris Parsons
51f8c39261 bp2build: handle system_shared_libs
- If no system_shared_libs is specified, bp2build writes no attribute
value. In this case, the bazel library macros determine the correct
default behavior.
- If any system_shared_libs is specified for any variant, then bp2build
writes the value verbatim. This includes if an empty list is specified,
as this should override defaulting behavior.

Note this defaulting behavior is incomplete and will be incorrect in
corner cases. For example, if, in an Android.bp, system_shared_libs is
specified for os.linux_bionic but not for os.android, then the bazel
default for os.android will be incorrect. However, there are no current
modules in AOSP which fit this case.

As a related fix, supports static struct for cc_library_static.

Also, removes some elements from the bp2build denylist.

Test: mixed_droid CI
Change-Id: Iee5feeaaf05e8e7209c7a90c913173832ad7bf91
2021-08-09 11:41:09 -04:00
Yabin Cui
01cad0cddc Merge "Revert "Switch to clang r428724 13.0.1."" 2021-08-08 21:02:29 +00:00
Yabin Cui
786c1d41ef Revert "Switch to clang r428724 13.0.1."
Revert submission 1743633-switch-to-clang-r428724

Reason for revert: fix build
Reverted Changes:
I824f3bc1c:Update bazel's clang version to clang-r428724
I6c4c26267:Adjust test for compiler update.
I94ab1f2dc:Switch to clang r428724 13.0.1.

Change-Id: I846e904b6e868d4e2be1bd8260ed59fc5464ef68
2021-08-08 20:59:59 +00:00
Yabin Cui
baeca672c1 Merge "Switch to clang r428724 13.0.1." 2021-08-08 18:20:22 +00:00
Romain Jobredeaux
748652e350 Exempt prebuilts/jdk and build/bazel/examples/java BUILD files from bp2build overwriting
Test: b build prebuilts/jdk/...
Test: b build build/bazel/examples/java/...
Change-Id: I6f82e4984ab2d8ec4c21edbf2dcba5f37b515fc9
Bug: 195786195
2021-08-06 22:09:36 +00:00
Colin Cross
795a9f99f4 Merge "Reland: Pass jars with resources to R8" 2021-08-06 18:33:23 +00:00
Yabin Cui
e87329c4c5 Switch to clang r428724 13.0.1.
Bug: http://b/190099451
Test: build and boot devices

Change-Id: I94ab1f2dc194ffcec5681708b558af26ced30aae
2021-08-06 18:28:08 +00:00
Colin Cross
03349188bc Merge changes Ia0c3826a,I92ad6635
* changes:
  Revert "Time out and dump stacks from R8 processes after 30 minutes"
  Add more debugging to run_with_timeout
2021-08-06 17:47:49 +00:00