Commit graph

38520 commits

Author SHA1 Message Date
LaMont Jones
b02043b84d Merge "Improve handling of errors from zipinfo" 2021-08-10 18:12:27 +00:00
Chris Wailes
90c4b14a30 Removed commented out code
A previous CL commented out code instead of removing it.  This CL fixes
this grave error.

Test: TH
Change-Id: I1daeaaa1ad90230aec639d603e7cdb9b61af251a
2021-08-10 10:21:02 -07:00
Romain Jobredeaux
d9eb0f0aa6 Merge "Exempt prebuilts/jdk and build/bazel/examples/java BUILD files from bp2build overwriting" 2021-08-10 16:29:35 +00:00
Treehugger Robot
6fd3047115 Merge "Support exporting device arch toolchain info" 2021-08-10 16:11:52 +00:00
Chris Parsons
2c7883941e Support rtti in bp2build
Rtti was already handled by bazel macros, so this change simply
propagates the rtti bit to these macros.

Test: Run bp2build, build //external/libcxxabi:all
Change-Id: I63296db2db868202874c8bd0b1de6310f7bb85c7
2021-08-10 11:58:07 -04:00
Yabin Cui
10bf3b8c00 Revert^2 "Switch to clang r428724 13.0.1."
786c1d41ef

Change-Id: Ic143d1b3375c8258e0e4274339524380de391676
2021-08-10 15:42:10 +00:00
Lukacs T. Berki
cef87b62e4 Make multiproduct_kati call soong_ui.bash .
This serves to not link parts of soong_ui (and eventually soong_build)
into a separate, weird binary. This is in turn good because they contain
any number of global variables and no one really thought about what
happens when two instances are executing at the same time in the same
address space.

This comes with a slight performance hit: 5 aosp_* projects build 152
seconds instead of 146. I suppose this is a price worth paying for a
clean design?

Test: presubmits.
Change-Id: I5623dcab2290f0fc392dd2ede597b9794a3d2a4e
2021-08-10 17:35:14 +02:00
satayev
abcd59731e Fix tests to use correct fixtures for configuring (Apex)BootJars.
This is in preparation to r.android.com/1740313 where setting correct
variables would be enforced (i.e. apex and non-apex boot jars must be in
config.ApexBootJars and config.BootJars correspondingly).

Bug: 191369843
Test: m nothing
Change-Id: Ic86680c1f7af53d229083b2cc58beb3ceccb4b6a
2021-08-10 15:35:52 +01:00
Paul Duffin
6ffdff853c Remove member signature and inner classes from signature-patterns.csv
Previously, the signature-patterns.csv file included a lot of
implementation details, e.g. the signatures of dex members or inner
classes that are not part of any API, including the hidden API.

This change will remove all member signatures and inner class names
from the file and replace them with just the outermost qualified class
name.

That will still leave some implementation details, e.g. the names of
implementation only classes and packages.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I9de6a2a6129e875e19f7ded5fae578cbdb584660
2021-08-10 14:01:51 +01:00
Paul Duffin
8f34b0e19d Merge "Use trie to store monolithic hidden API flags" 2021-08-10 09:26:56 +00:00
Chris Wailes
353405d9fb Clean up generated rust-project.json files
This CL removes the roots section from the rust-projects.json file as it
is no longer required by the rust-analyzer tool.

Bug: 162055556
Test: prebuilts/rust/tests/rust-analyzer-integration-test.sh
Change-Id: I85c3277632c88394bc52e709ecce83f6aa1f2e0e
2021-08-09 15:50:39 -07:00
LaMont Jones
4224c6216d Improve handling of errors from zipinfo
Bug: 153900481
Test: manual
Change-Id: Ib60578ead4ccaa7ba09f628a6c9281c0cc70e6ce
2021-08-09 19:31:39 +00:00
Christopher Parsons
67d6ccecfc Merge "bp2build: handle system_shared_libs" 2021-08-09 19:11:00 +00:00
Paul Duffin
c11f6679d8 Use trie to store monolithic hidden API flags
Previously, a simple map from dex signature to the flags was used to
store the monolithic hidden API flags. This change replaces that with a
trie that uses packages and class names extracted from the signature
to map from the signature to the flags.

The signature is broken down into pieces consisting of package names,
class names and the remaining member signature. They are used in turn
to navigate through nodes in the trie to a Leaf that contains the flags
associated with the signature.

The flags can be retrieved from the trie in a number of ways:
* Using the complete signature to retrieve the flags for a single
  member.
* Using a class name to retrieve the flags for all the members in a
  class and any nested classes.
* Using a package wildcard to retrieve the flags for all the members in
  the classes in that package.
* Using a recursive package wildcard to retrieve the flags for all the
  members in the classes in that package and nested packages.

This will allow a bootclasspath_fragment to select a subset of the
monolithic flags to compare against itself without specifying the
signatures of every member of that set.

Before creating the trie an attempt was made to compute the subset by
iterating over all the signatures in the monolithic flags and matching
against a regular expression created from the patterns but that was too
slow. It took minutes to run whereas using the simple map or the trie
only takes a few seconds.

Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: I36f5e319d3e7d62dd34305de1eec990a93cb3a89
2021-08-09 19:51:33 +01:00
Chris Parsons
51f8c39261 bp2build: handle system_shared_libs
- If no system_shared_libs is specified, bp2build writes no attribute
value. In this case, the bazel library macros determine the correct
default behavior.
- If any system_shared_libs is specified for any variant, then bp2build
writes the value verbatim. This includes if an empty list is specified,
as this should override defaulting behavior.

Note this defaulting behavior is incomplete and will be incorrect in
corner cases. For example, if, in an Android.bp, system_shared_libs is
specified for os.linux_bionic but not for os.android, then the bazel
default for os.android will be incorrect. However, there are no current
modules in AOSP which fit this case.

As a related fix, supports static struct for cc_library_static.

Also, removes some elements from the bp2build denylist.

Test: mixed_droid CI
Change-Id: Iee5feeaaf05e8e7209c7a90c913173832ad7bf91
2021-08-09 11:41:09 -04:00
Yabin Cui
01cad0cddc Merge "Revert "Switch to clang r428724 13.0.1."" 2021-08-08 21:02:29 +00:00
Yabin Cui
786c1d41ef Revert "Switch to clang r428724 13.0.1."
Revert submission 1743633-switch-to-clang-r428724

Reason for revert: fix build
Reverted Changes:
I824f3bc1c:Update bazel's clang version to clang-r428724
I6c4c26267:Adjust test for compiler update.
I94ab1f2dc:Switch to clang r428724 13.0.1.

Change-Id: I846e904b6e868d4e2be1bd8260ed59fc5464ef68
2021-08-08 20:59:59 +00:00
Yabin Cui
baeca672c1 Merge "Switch to clang r428724 13.0.1." 2021-08-08 18:20:22 +00:00
Romain Jobredeaux
748652e350 Exempt prebuilts/jdk and build/bazel/examples/java BUILD files from bp2build overwriting
Test: b build prebuilts/jdk/...
Test: b build build/bazel/examples/java/...
Change-Id: I6f82e4984ab2d8ec4c21edbf2dcba5f37b515fc9
Bug: 195786195
2021-08-06 22:09:36 +00:00
Colin Cross
795a9f99f4 Merge "Reland: Pass jars with resources to R8" 2021-08-06 18:33:23 +00:00
Yabin Cui
e87329c4c5 Switch to clang r428724 13.0.1.
Bug: http://b/190099451
Test: build and boot devices

Change-Id: I94ab1f2dc194ffcec5681708b558af26ced30aae
2021-08-06 18:28:08 +00:00
Colin Cross
03349188bc Merge changes Ia0c3826a,I92ad6635
* changes:
  Revert "Time out and dump stacks from R8 processes after 30 minutes"
  Add more debugging to run_with_timeout
2021-08-06 17:47:49 +00:00
Stephen Hines
fe3bab726d Merge "Revert "Make DWARF v5 the default debug information format."" 2021-08-06 17:04:41 +00:00
Paul Duffin
8d007e9919 Export signature patterns to sdk snapshot
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: Ic819def9b0c41e3b22fb98582cd39f6d27080a32
2021-08-06 14:11:14 +01:00
Paul Duffin
67b9d61ac2 Separate creation of signature patterns from overlap checking
Previously, the signatures used to select the subset of the monolithic
flags were simply the signatures read from the modular flags file. This
change moves the creation of the signature list into a separate script
that outputs the signatures to a file and then passes the path through
Soong from the bootclasspath_fragment modules that create it to the
platform_bootclasspath module that uses it to compare the modular
flags against the monolithic flags.

Currently, the signatures are the full signatures but follow up changes
will replace them with patterns (hence the name) that avoids having to
include implementation details in the hidden API flags that are output
as part of a bootclasspath_fragment's snapshot.

This change moves the stub flags related code next to the all flags
related code as they are treated in a similar way.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I2855bf6d05c91b8a09591664185750361c7e644f
2021-08-06 13:40:34 +01:00
Paul Duffin
47c456228c Merge "Extract a subset of the monolithic flags for comparison" 2021-08-06 09:08:29 +00:00
Paul Duffin
335f02ad21 Merge "Clarify symbols used in verify_overlaps.py" 2021-08-06 09:06:23 +00:00
Paul Duffin
a26cf315a8 Merge "Add tests for verify_overlaps script" 2021-08-06 09:05:45 +00:00
Colin Cross
a79a52c7c4 Reland: Pass jars with resources to R8
R8 will parse proguard files out of resources in injars.  Use the jar
with resources instead of the classes jar so that R8 can see the
proguard files.

R8 fails when an input jar contains dex files, so filter out dex files
from the input jar before passing it to r8.

This relands Ibb870ee9c70470336f542a3b7542dab86716dbf8 with a fix to
move the temporary input jar outside the output directory.  Putting it
in the output directory interacted poorly with RBE in local comparison
mode, which incorrectly deleted the output directory.

Bug: 195558228
Test: m checkbuild
Change-Id: Id4b091c3bd72d4c2904883e8793ec4b9499e4b78
2021-08-05 23:16:38 -07:00
Stephen Hines
bb1a17aecb Revert "Make DWARF v5 the default debug information format."
This reverts commit 5e13f86fc8.

libabigail isn't quite ready for DWARF v5 yet, so let's revert this
temporarily.

Bug: http://b/182004844
Bug: http://b/195152239
Bug: http://b/195698910
Test: Builds
Change-Id: I4b1dd8f9e64d19f37c90b8a2433cbb302c7d5105
2021-08-05 17:05:01 -07:00
Colin Cross
a832a04db2 Revert "Time out and dump stacks from R8 processes after 30 minutes"
This reverts commit 29c294b2cf.

The deadlock in R8 was fixed, and the timeout causes problems when
RBE is enabled and the R8 action gets stuck in a queue for 30
minutes.

Test: checkbuild
Change-Id: Ia0c3826aea3288e9db3bdbd4955e1e16c0a38fe3
2021-08-05 17:04:06 -07:00
Colin Cross
11ec0a8f8b Add more debugging to run_with_timeout
Print more messages in run_with_timeout to make it easier to
determine which log statements came before the timeout and the
on_timeout command.

Test: run_with_timeout_test.go
Change-Id: I92ad663529fa03d91185ba382654a73f0eba1ab7
2021-08-05 16:51:50 -07:00
Pirama Arumuga Nainar
238ca381c4 [scripts] Print full clang prebuilt dir from get_clang_version.py
e.g. clang-r416183b1 instead of just r416183b1.

Test: manually run the script
Change-Id: I993202ea97433f841e3127ecef46dec66fb9a8c9
2021-08-05 23:39:00 +00:00
Colin Cross
93fb53f5d0 Merge "Revert "Pass jars with resources to R8"" 2021-08-05 15:25:25 +00:00
Colin Cross
1d73c7316c Revert "Pass jars with resources to R8"
This reverts commit 05ca7c8769.

Reason for revert: b/195644067

Flaky failure with "missing in.jar" error.

Change-Id: Id2d6452de1e43951bc58f0bfb8e20d5ad65ce830
2021-08-05 15:16:11 +00:00
Paul Duffin
53a7607f7d Extract a subset of the monolithic flags for comparison
Previously, the compare_signature_flags compared the full set of
monolithic flags against the modular flags and just ignored any
monolithic flags for a signature that was not present in the modular
flags. It essentially performed two tasks, identifying the subset of
the monolithic flags against which the modular flags should be compared
and then doing the actual comparison.

This change separates those two tasks, by first creating a dict that is
the subset of the monolithic flags against which the modular flags will
be compared and then do the comparison.

At the moment the subset is simply the set of monolithic flags
associated with the signatures that are present in the modular flags.
Follow up changes will provide other ways to select the subset.

Bug: 194063708
Test: atest --host verify_overlaps_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: Icb547f071b814dd955f31255b0d35597f7ba4aa1
2021-08-05 14:19:57 +01:00
Paul Duffin
7be96337f2 Clarify symbols used in verify_overlaps.py
Switches symbols to use monolithic... and modular... to try and clarify
the purpose of those symbols.

Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: Ifc646edebe3d43b51dc3528fa883718399cb5254
2021-08-05 14:19:57 +01:00
Paul Duffin
428c65156b Add tests for verify_overlaps script
Refactor verify_overlaps to make it testable and add tests for the
comparison. It does make one significant change in behavior which is to
read each of the files produced by a bootclasspath_fragment into a dict
before comparison, rather than reading and comparing them a row at a
time. That allows it to reuse the code to read a CSV into a dict.

Bug: 194063708
Test: atest --host verify_overlaps_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: Ib70ac87fe089fc25e3bef18f367d4939bfc0cb8d
2021-08-05 14:16:51 +01:00
Colin Cross
ac33aac0aa Merge "Pass jars with resources to R8" 2021-08-05 01:41:32 +00:00
Colin Cross
05ca7c8769 Pass jars with resources to R8
R8 will parse proguard files out of resources in injars.  Use the jar
with resources instead of the classes jar so that R8 can see the
proguard files.

R8 fails when an input jar contains dex files, so filter out dex files
from the input jar before passing it to r8.

Bug: 195558228
Test: m checkbuild
Change-Id: Ibb870ee9c70470336f542a3b7542dab86716dbf8
2021-08-04 14:57:45 -07:00
Treehugger Robot
27ac52198d Merge "Update the Rust prebuilt version number to 1.54.0" 2021-08-04 21:28:20 +00:00
Treehugger Robot
1e28e36589 Merge "[rust] [coverage] Don't pass -C link-dead-code" 2021-08-04 20:51:47 +00:00
Spandan Das
da7f362a32 Update documentation for VisitDirectDeps
Calling this function for dependencies that are not android.Module
results in a hard error rather than a soft skip. Update the doc to
reflect this

Bug: 195369546
Test: TH
Change-Id: I18a0938c6536cf20f239cf98c601722aaecd51e6
2021-08-04 20:50:04 +00:00
Ryan Savitski
aa3c355820 eng builds: dexpreopt: stop explicitly removing minidebuginfo
This special-casing is actively unhelpful nowadays, as it breaks
callstack unwinding and symbolisation by tools that work using the
on-device debug info (e.g. perfetto's profilers). It's also highly
surprising that eng builds have less debug info than userdebug/user.

As far as I can tell, the history behind this case is (with some
internal-only references as the context is in the review comments): the
minidebuginfo was originally rolled out in device-specific makefiles
(e.g. ag/2433691). Eng builds were excluded, presumably due to time/size
concerns for rebuild+resync workflow, as well as the assumption that
engineers using eng builds already have the necessary debug info
off-device. Later, as some more makefile variables were introduced, the
conversation in ag/990059 shows that the overheads were no longer a
concern, and the last patchset generalised the option to include eng
builds. However at a later point, during device-specific makefile
cleanup, the eng-exclusion got restored: ag/2455645. This then ended up
in the makefile->soong transition, leaving us with the current special
casing that is undesireable.

Checking things with an internal master build on my workstation:
dexpreopting system services is 10s regardless of this minidebuginfo
flag, and the resulting odex size is 37MB vs 38MB. Overall, I only see 4
dexpreopt invocations as part of a local build, so the effect of this
patch should be negligible.

Bug: 194835631
Change-Id: I80de75a84259c056cdae0696fe611afe7e0162f5
2021-08-04 19:34:55 +01:00
Elvis Chien
9c99354465 Allow PRODUCT_CFI_INCLUDE_PATHS to work with blueprints also
but disable it for host modules like how it's done for Android.mk

Bug:
179233410

Test:
m -j32

Change-Id: I246cd9163e06997a2b50cd25688370690c8929af
2021-08-04 18:07:51 +08:00
Mathew Inwood
bf49705608 Merge "Update error message for new policy." 2021-08-04 09:59:33 +00:00
Pirama Arumuga Nainar
b13e8ac6c8 [rust] [coverage] Don't pass -C link-dead-code
Bug: http://b/186873963

Per jgalenson@, this was added for GCOV and is not needed for clang
coverage.

Test: http://go/abtd/run/L71000000950546158
Change-Id: I352669687cbe764d938acb94d9d4fe0c24a31a56
2021-08-04 05:06:22 +00:00
Treehugger Robot
03beb0ec77 Merge changes Ie2012adb,Ib0283ca6
* changes:
  static rust binaries are also prohibited in APEXes
  Revert "Revert "Prohibit static executable in APEX""
2021-08-04 04:35:20 +00:00
Treehugger Robot
8662a3793b Merge "Update documentation for genrule and sdk_library" 2021-08-04 00:14:31 +00:00
Alexander Smundak
408a4bece5 Merge "Improve dynamic inherit paths handling" 2021-08-03 20:14:39 +00:00