Commit graph

35827 commits

Author SHA1 Message Date
Jingwen Chen
4fabaf52b0 bp2build/b: exit early in GENERATE_BAZEL_FILES=1.
This CL fixes a typo in writeFakeNinjaFile to correctly write
a fake out/soong/build.ninja and its depfile. It also modifies bootstrap
phase to *not* run the main soong build phase (which takes more than a
minute) if GENERATE_BAZEL_FILES=1.

This change has the side effect that `GENERATE_BAZEL_FILES=1 m nothing`
no longer generates the real build.ninja, which is fine because one
shouldn't be using GENERATE_BAZEL_FILES=1 for that anyway (or, use
USE_BAZEL_ANALYSIS=1). This change has no effect on mixed builds.

Time on a change to Soong or any Android.bp files:

Before: bp2build_workspace_marker (~20 seconds) + build.ninja (1 min)
After: bp2build_workspace_marker (~20 seconds)

Time on the second of two consecutive `b build //bionic/...`: 2.070s

Test: TH
Test: Soong integration tests
Change-Id: I43720641815994caba97b8d165d7c3fc254cbd06
2021-05-25 03:42:38 +00:00
Dan Shi
a22ed550e9 Merge "Add unit_test test option in sh_test_host" 2021-05-25 03:00:40 +00:00
Chris Parsons
af24cdd99f Split asm and c flags and srcs in bp2build output
This allows removal of almost all current items from the mixed build
denylist, which were previously broken due to being unable to separately
control flags for compilations of different languages within the same
target.

Note that this does not appropriately implement asm/c srcs and flags for
either the shared variant or the static variant. This will require a
followup.

Test: bp2build.sh and mixed_libc.sh CI scripts
Test: Updated b2build tests

Change-Id: I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4
2021-05-24 21:49:14 -04:00
Treehugger Robot
c2d8fa0515 Merge "Clarify how metalava @SuppressLint works" 2021-05-25 00:21:39 +00:00
Martin Stjernholm
cb3ff1e412 Move logic to disable flattening in unbundled app mode from Soong to
make.

Avoids TARGET_FLATTEN_APEX lying about it, so we can trust it in
scripts, e.g. in art/build/apex/runtests.sh.

Test: banchan com.android.art; art/build/apex/runtests.sh
Bug: 179900989
Change-Id: I5361957c448e61634689968424450420479f6111
2021-05-25 00:28:27 +01:00
Dan Willemsen
beb51ac64c Add go2bp tool
This tool automatically writes Android.bp files based on information
discovered from Go, similar to pom2bp or Bazel's Gazelle. This
implementation is fairly limited, but is usable enough to import
libraries like protobuf without a lot of manual effort.

Change-Id: Iecc145bc7ab9b9e9c3739f77b694b3f4bc2c6396
2021-05-24 16:06:52 -07:00
Treehugger Robot
d01a80869d Merge "Handle product config vars in bp2build." 2021-05-24 22:34:28 +00:00
Paul Duffin
00ceb0e7f5 Merge "Tighten bootclasspath_fragment property validation" 2021-05-24 22:19:52 +00:00
Paul Duffin
6778edb558 Merge "Only export boot image files to APEX if they are actually created" 2021-05-24 22:19:36 +00:00
Dan Shi
b40deac018 Add unit_test test option in sh_test_host
This change allows sh_test_host to be included
in host-unit-tests suite when test option `unit_test` is set to true.

Bug: 183209767
Test: m host-unit-tests
Change-Id: I5b441d5f49468cb681eb414dade8a1309c6e810a
2021-05-24 12:06:23 -07:00
Liz Kammer
e3e4a5f2d8 Handle arch/os-specific product variables
Bug: 183595873
Test: go test bp2build tests
Change-Id: I36e93ae1eb2943555dd304d5bdf62d995e77b437
2021-05-24 14:32:38 -04:00
Liz Kammer
6fd7b3fee9 Handle product config vars in bp2build.
Test: bp2build ci  & mixed build libc
Bug: 183595873
Change-Id: I2d87434ff4df5a24efc5e3e38f087de035228934
2021-05-24 14:22:31 -04:00
Aurimas Liutikas
b23b745e19 Clarify how metalava @SuppressLint works
Bug: 186157121
Change-Id: Ia3316b80dcf16e7fb2a41983732e0be8d0657497
Test: None
2021-05-24 18:01:17 +00:00
Treehugger Robot
3078f3bddc Merge "Add SOONG_SDK_SNAPSHOT_VERSION support" 2021-05-24 17:58:17 +00:00
Paul Duffin
8018e50ddb Tighten bootclasspath_fragment property validation
Previously, due to legacy reasons, the property validation did not
require a contents property and allowed the image_name to be either
"art" or "boot". Those reasons no longer apply and so this change
requires a contents property and only allows the image_name to be set
to "art" if specified.

Bug: 177892522
Test: m nothing
Change-Id: I8855d6e5365ef0b55490e90e7b6c0081cf070ee5
2021-05-24 16:22:47 +01:00
Paul Duffin
58e0e769c9 Only export boot image files to APEX if they are actually created
Previously, the generation of the rules to create the boot image files
was separate from the code to provide those files to the APEX which
meant that it was possible for the APEX to try and use files that had
no rules to create them.

This changes avoids that by only exporting the files once the rules
have been created.

This necessitated a few changes to tests that were relying on the
previous behavior. Including removing completely the test that used an
image_name: "boot" as that is no longer a valid configuration name as
its functionality has been replaced by platform_bootclasspath.

A follow up change will make the validation of the properties of
bootclasspath_fragment stricter to prevent "boot" being used as the
image_name.

Bug: 177892522
Bug: 188680624
Test: lunch qemu_trusty_arm64-userdebug
      m droid dist
      - verify that before this change it fails and after this change
        it works.
      m com.android.art
      - verify that this change does not change the APEX contents with
        either qemu_trusty_arm64-userdebug or aosp_arm64-userdebug
Change-Id: I0497a151eb0731cbe6a1a7e7bbbb1e4dda75898f
2021-05-24 16:22:43 +01:00
Paul Duffin
32cf58a8fc Support removed API members in modular hidden API processing
Previously, the hidden API flags generated for a bootclasspath_fragment
did not include removed API members. That was because it did not supply
a file containing the dex signatures of the removed API members.

The monolithic hidden API processing uses combined-removed-dex which is
the output of a genrule that takes as input the *removed.txt files from
all the APIs and uses metalava to construct the dex signatures file.
This change does the equivalent for the *removed.txt files for the APIs
provided by a bootclasspath_fragment and then passes them to the rule
that generates the final all-flags.csv.

Bug: 179354495
Test: - Update packages/modules/RuntimeI18N to enable hidden API
        processing.
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - Before this change that fails as the flags generated for the
        i18n-bootclasspath-fragment differ from the monolithic flags.
	After this change that passes.
      - Verify that this change does not change any of the monolithic
        hidden API files.
      m com.android.i18n
      - Verify that the apex file before and after this change are byte
        for byte identical.
Change-Id: I6a21edb8a5231666e3f35b2c99a8687f36dd98fd
2021-05-24 15:05:21 +01:00
Paul Duffin
f1b358cb57 Support hidden API processing for fragments with dependencies
Previously, a bootclasspath_fragment that depended on classes provided
by another bootclasspath_fragment did not support hidden API processing
as it would not supply information about those dependencies.

This change adds support for that as follows. Each fragment:
1. Exports the transitive sets of stub dex jars for each of the public,
   system, test and core_platform APIs (where relevant).
2. Adds dependencies onto its dependent fragments.
3. Retrieves the API stubs dex jars from its dependent fragments and
   passes them to the "hiddenapi list" tool which will use them to
   resolve dependencies but will not output them to the generated
   flags.

Once the flags are generated the existing encoding functionality
encodes the flags into the dex files of the bootclasspath_fragment's
content modules which are then packaged into the apex.

Bug: 179354495
Test: m com.android.sdkext
      - verify that this does not change the contents of the apex files
Change-Id: I3e82a6dbb437f1e417e5d7e25aeb212e378603d0
2021-05-24 14:56:10 +01:00
satayev
703c67a0a9 Populate individual systemserverclasspath_fragments' proto configs.
Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: Ic0ae897e9baca75dcc022d84656496563645e74e
2021-05-24 13:41:20 +01:00
Paul Duffin
18cf19745e Remove bootclasspathApiInfo
Previously, the bootclasspathApiInfo was only used for tests and follow
up changes will need to provide the stub dex jars. This change moves
the stubJarsByKind into HiddenAPIInfo and removes bootclasspathApiInfo
and the corresponding provider.

Bug: 179354495
Test: m nothing
Change-Id: I5459c56de561c053ed671dc9d5cb3ee4820c0ee8
2021-05-24 13:32:56 +01:00
Paul Duffin
6237092391 Fix hidden API flags in com.android.i18n
Change 70cfdff3da changed the hidden API
flags in com.android.i18n as it stopped the i18n-bootclasspath-fragment
from making the hidden API flag files available for use by
platform-bootclasspath.

This change fixes that by exporting the flag files even if hidden API
flag generation is skipped.

Bug: 179354495
Test: m com.android.i18 out/soong/hiddenapi/hiddenapi-flags.csv
      - make sure that the flags in
        packages/modules/RuntimeI18n/apex/hiddenapi/hiddenapi-max-target-o-low-priority.txt
        are reflected in the core-icu4j dex files in the apex.
Change-Id: I9b5c7c74bd996ab447bc0e0452da5fd49191a35d
2021-05-24 13:32:55 +01:00
Paul Duffin
af99afa919 Rename hiddenAPIFlagFileInfo to HiddenAPIInfo
This reflects that it has expanded from its initial purpose to include
more than just flag files. It is exported for use in tests in other
packages.

Bug: 179354495
Test: m nothing
Change-Id: I9f780b20e18ce3a774e4aa04a276463070a64c34
2021-05-24 13:32:55 +01:00
Paul Duffin
1352f7c471 Separate input to flag generation from hiddenAPIFlagFileInfo
Encapsulating the information needed by hidden API processing in a
struct makes it easy to add additional information in future and allows
the code to populate that struct from various different sources to be
grouped together.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: I53805737dff36a3ae87aca5aad51cf46ae1361fe
2021-05-24 13:32:55 +01:00
Paul Duffin
1e6f5c4e63 Separate output of flag generation from hiddenAPIFlagFileInfo
HiddenAPIFlagOutput encapsulates the paths to the files produced by the
hidden API flag generation of a single bootclasspath_fragment. It is
returned from hidden API flag generation and is embedded within the
hiddenAPIFlagFileInfo so they can be passed to other modules.

Unlike the fields it replaces in hiddenAPIFlagFileInfo the fields in
HiddenAPIFlagOutput are of type Path not Paths which makes it easier to
use.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: I7373ba1001cac3a75eb54a23e62fa52f5013ee7f
2021-05-24 13:32:55 +01:00
Paul Duffin
438eb57a27 Separate monolithic hidden API processing from hiddenAPIFlagFileInfo
The hiddenAPIFlagFileInfo was being used for both the input and output
of bootclasspath_fragment and platform_bootclasspath and also to pass
information around to various hidden API rule methods. Supporting
multiple different uses in this way made it hard to reason about.

This change creates a separate structure for use by the
platform_bootclasspath. Follow up changes will split out other
functionality into separate types.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: Ia5c5f65ae5645486c42819c669a8601588217f88
2021-05-24 13:32:55 +01:00
satayev
227e745e56 Rename generate proto config file to match classpath type.
There is no need to leak soong module names.

Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: I04f4e181d2f42b9d71641980a2c7c4e8cbc8e426
2021-05-24 12:52:01 +01:00
Justin Yun
611e886ca7 Do not filter LLNDK lib from llndk.libraries.txt
llndk.libraries.txt is used to list the LLNDKs for the linkerconfig.
This is also required to define the VNDK snapshot modules of LLNDK
stub libraries.
The list had a filter for hwasan libraries, but the filter must be
used only for the LLNDK_LIBRARIES make variable, but not for the
llndk.libraries.txt file.

Bug: 189070366
Bug: 188716508
Test: m nothing
Test: LLNDK_LIBRARIES in make_vars-*.mk does not include hwasan.
Change-Id: I820dd1161892dc76700aa1d696ea2643764d2052
2021-05-24 20:44:44 +09:00
Lukacs T. Berki
c1cc3b9678 Split each test case to a different test function.
This is so that they can be individually debugged.

Test: Presubmits.
Change-Id: I7d929c4126bba7470aaa1c0def85bad65429ffdc
2021-05-23 20:41:14 +02:00
Paul Duffin
54c98f5b4a Perform hidden API encoding in bootclasspath_fragment
Previously, the apex content info was populated with hidden API encoded
dex jars retrieved directly from the java module. This change retrieves
the unencoded dex jars from the java module, encodes them and then
stores the result in the apex content info.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: Ib1b6eb8b62ac50e03b9e0d07c877ca70bb6f6d25
2021-05-23 11:21:38 +01:00
Paul Duffin
0916595b1c Refactor hiddenAPIEncodeDex for use by bootclasspath_fragment
Previously, if the method was called multiple times by the same module
the resulting build rules would all use the same temporary directory
which meant that if run in parallel they would conflict with each
other. This change fixes that by providing a jar specific temporary
directory so it can be used by bootclasspath_fragment to encode its
content modules.

Also, cleans up, simplifies and improves the documentation.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: I0ebe61abc8e16111c6e8a822eb96c57846b98461
2021-05-23 11:21:38 +01:00
Martin Stjernholm
22053dcbf2 Merge "Fix indexing bug when config.Arches() returns an empty array." 2021-05-22 18:53:42 +00:00
Christopher Parsons
dbfafc0a3c Merge "Propagate unexported deps via implementation_deps" 2021-05-21 21:29:59 +00:00
Pirama Arumuga Nainar
cb56f0166f Merge "[cc/coverage] Define a macro during coverage builds" 2021-05-21 20:47:00 +00:00
Martin Stjernholm
df298b3bed Fix indexing bug when config.Arches() returns an empty array.
This happens when com.android.art.host is not disabled in
prebuilts/build-tools/build-prebuilts.sh.

Test: m nothing
Test: env OUT_DIR=out prebuilts/build-tools/build-prebuilts.sh
  on the aosp-build-tools branch
Bug: 188922057
Change-Id: Ib21d17bb627e4cf7faee320febe65e1bb4566d4e
2021-05-21 21:07:26 +01:00
Spandan Das
fc1fbc428a Merge "Adding relative_install_path field to android_app_import" 2021-05-21 16:15:38 +00:00
Colin Cross
90463fe8cb Merge "Allow missing java_sdk_library files in AllowMissingDependencies builds" 2021-05-21 15:07:49 +00:00
satayev
a4cf238f4d Merge "Populate individual classpath_fragments' classpaths.proto configs." 2021-05-21 10:21:49 +00:00
Jingwen Chen
857d037566 Merge "bp2build: generate arm toolchain flags into @soong_injection." 2021-05-21 04:14:51 +00:00
Jingwen Chen
78257e5b6b Add a comment to absolutePath on why it's not exported.
Change-Id: Ib6595e8af3d845861d4c5988750e3c43280bb697
Test: TH
2021-05-21 02:37:00 +00:00
Bill Peckham
79610d55fa Merge changes from topic "far-dep-exists"
* changes:
  Check for product variant in addition to core variant.
  Plumb through OtherModuleFarDependencyVariantExists from blueprint.
2021-05-21 01:37:21 +00:00
Colin Cross
18f840c45e Allow missing java_sdk_library files in AllowMissingDependencies builds
java_sdk_library modules in builds with AllowMissingDependencies may
be missing defaults modules that cause them to look for api files that
they normally wouldn't need.  Move the error to runtime so it doesn't
block the build unless the branch tries to build that module.

Test: prebuilts/build-tools/build-prebuilts.sh in aosp-build-tools
Change-Id: I279b0cd8493779f972c0ac02235967c10b35a5a0
2021-05-20 18:00:37 -07:00
Brian Egizi
2aded7dd55 Fix unbound variable in kzip script
Change-Id: Id7f81677389cb853da064a43f1d5d54b2271f0c0
2021-05-21 00:51:44 +00:00
Brian Egizi
c3a800f611 Merge "Update kzip script to support superproject revision environment variable" 2021-05-21 00:20:52 +00:00
Spandan Das
d1fac64807 Adding relative_install_path field to android_app_import
Test: Ran go test ./java from build/soong
Bug: 149873762
Change-Id: I4abc1fb9e2386c7319b446b84aebaabacc882fd6
2021-05-20 23:10:26 +00:00
Jingwen Chen
51a1e1cf59 bp2build: generate arm toolchain flags into @soong_injection.
Other architectures will come later.

Bug: 187084665
Test: TH
Change-Id: Ifcc6773625e56b1fc20260d840380e7ee4ecee7f
2021-05-20 22:29:39 +00:00
Bill Peckham
1d3466a019 Merge "VNDK snapshot may provide LLNDK stub libraries" 2021-05-20 21:43:44 +00:00
Brian Egizi
4aa056eb9e Update kzip script to support superproject revision environment variable
This changes the superproject sha variable to use the
SUPERPROJECT_REVISION as the default value if no explicit sha is
defined. Only sets up the new default if the revision value matches a
sha.

This change is required since the builder provides a revision for the
superproject that is either pinned to a sha or a branch name. If it is
pinned to a sha, we want to use this as the fallback source for
SUPERPROJECT_SHA

Bug: b/149248752
Change-Id: I59569b0e4d2f360243f0adc5b2587130956b6b99
2021-05-20 19:36:13 +00:00
satayev
8fab6f86af Populate individual classpath_fragments' classpaths.proto configs.
To avoid duplicates on *CLASSPATH environ variables at runtime, remove
split entries from platform-*classpath, i.e. all updatable jars that
have their own classpath fragments should not appear in the
platform-*classpath's classpaths.proto config.

Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: Id2759ab8e106cc183e695bf3509a6ab60ab0ef2a
Merged-In: Id2759ab8e106cc183e695bf3509a6ab60ab0ef2a
2021-05-20 20:06:03 +01:00
Bill Peckham
4016d7b53e Check for product variant in addition to core variant.
It's possible that a module will have a product variant
but not a core variant. In that case, we still need to
apply the vendor suffix to the androidmk module name.

Bug: 188720375
Bug: 187963715
Test: Build vendor image using vendor snapshot.

Change-Id: I190fdd707ca23357d1c324c1b1f3318457625f03
2021-05-20 18:57:04 +00:00
Martin Stjernholm
ecf2dc4614 Merge "Skip TestDex2oatToolDeps on Darwin." 2021-05-20 16:52:30 +00:00