Commit graph

404 commits

Author SHA1 Message Date
Paul Duffin
fb9a7f99be Support generating sdk snapshots that can be selected by Soong config vars
Allows the builder of an sdk snapshot to specify a Soong config
variable that can be used to control whether the snapshot's prebuilts
are used or not.

Bug: 193523070
Test: m nothing
Change-Id: Ib09500ba61befc1202dff61dc06847c730ba9253
2021-07-14 23:19:53 +01:00
Treehugger Robot
af60f411f8 Merge "Revert "Flag to control if generated SDK prebuilts are preferred."" 2021-07-07 20:49:14 +00:00
Mathew Inwood
7e9ddbe8c8 Revert "Flag to control if generated SDK prebuilts are preferred."
This reverts commit d0b99cea0a.

Reason for revert: Not needed anymore. We will simple exclude the 
gernated Android.bp files from prebuilt drops instead, it keeps
things simpler.

Change-Id: I5b01b2e01197f9fceb4630c8ade95f0d1d0f33f4
2021-07-07 12:47:51 +00:00
Colin Cross
8c3339f74c Merge "Support shared_libs, static_libs and default_shared_libs in cc_object" 2021-07-01 19:00:40 +00:00
Treehugger Robot
1a58d1a815 Merge "Flag to control if generated SDK prebuilts are preferred." 2021-07-01 14:29:01 +00:00
Colin Cross
e19e6196b9 Merge "Add default_shared_libs property" 2021-06-30 14:27:20 +00:00
Mathew Inwood
d0b99cea0a Flag to control if generated SDK prebuilts are preferred.
This is needed because:
- the SDK prebuilt drop is being automated, including the Android.bp
- We want to control the prefer flag on a per-module level

It augments the existing SOONG_SDK_SNAPSHOT_PREFER which now overrides
this new flag when set to true.

Bug: 188427719
Test: m nothing
Change-Id: Ieb5ab9fab53a34c615345b7a9d19cadf713eec26
2021-06-29 15:55:27 +00:00
Colin Cross
137d7dafd8 Support shared_libs, static_libs and default_shared_libs in cc_object
cc_object files don't link against static or shared libraries, but they
can use headers from them.  Make cc_object use the default_shared_libs
headers, and add support for explicit static and shared library
dependencies.

Bug: 153662223
Test: go test ./cc/...
Change-Id: I9ab160ede06db1b135d217d72770c22f500cfe1b
2021-06-25 16:29:51 -07:00
Paul Duffin
5cca7c44e5 Support hidden API processing for modules that use platform APIs
Previously, hidden API processing could only be done by those
bootclasspath_fragment modules that either did not depend on any other
fragments (e.g. art-bootclasspath-fragment) or only depended on APIs
provided by other fragments (e.g. i18n-bootclasspath-fragment). That
meant that modules like com.android.os.statsd-bootclasspath-fragment
that depended on APIs provided by parts of the platform which are not
yet part of another bootclasspath_fragment could not perform hidden
API processing.

This change adds support for a bootclasspath_fragment to specify the
additional stubs needed to perform hidden API processing. It adds a new
additional_stubs property that can be used to specify the additional
stub libraries.

Most bootclasspath_fragments that need to use the property will need
access to the APIs provided by the android-non-updatable.* libraries.
Rather than have each fragment explicitly specify the correct module
for each scope it treats "android-non-updatable" as if it was a
java_sdk_library that can provide different jars for each scope.
Soong will handle mapping that to the correct android-non-updatable.*
module.

Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv \
        out/soong/hiddenapi/hiddenapi-index.csv \
        out/soong/hiddenapi/hiddenapi-stub-flags.txt \
        out/soong/hiddenapi/hiddenapi-unsupported.csv
      - make sure that this change does not change the contents.
      m TARGET_BUILD_APPS=Calendar nothing
Change-Id: Ia8b79830ed0e6d42100de03d76b0c51b7f6c8ade
2021-06-25 16:20:19 +01:00
Paul Duffin
ea8f808580 Ensure consistent APEX variation for java_sdk_library and java_sdk_library_import
Previously, a java_sdk_library with shared_library = true would create
a variation per APEX because it depends on an sdkLibraryXml module
which generates a file containing the APEX name. However, a shared
java_sdk_library_import would create a merged APEX variation. The
inconsistent variations caused failures in sdkDepsReplaceMutator.

This change ensures that both java_sdk_library and
java_sdk_library_import create an APEX specific variation when their
shared_library property is true.

Bug: 190499958
Test: m nothing
      - ran the above command after modifying the test to reproduce the
        problem and then after fixing to verify that it fixed the problem.
Change-Id: Iee81776a8569db3e871c40cbde14d248dfeb56e4
2021-06-24 13:33:01 +01:00
Paul Duffin
b97b1579d3 Remove duplicate component from sdk snapshot
Previously, an sdk snapshot could contain the following:
* A java_sdk_library_import module, e.g. "foo" which creates component
  modules "foo.stubs", etc.
* A corresponding versioned module, e.g. "sdk_foo@current" which
  created component modules "sdk_foo@current.stubs", etc.
* An internal (to the sdk snapshot) java_import for one of "foo"'s
  components, e.g. "sdk_foo.stubs"
* A corresponding versioned module, e.g. "sdk_foo.stubs@current".

That causes a few problems:
1. The "foo.stubs" is duplicated.
2. The names of the components created by the versioned
   java_sdk_library_import are invalid, as they append the component's
   suffix to the version and not the name before the version.

The latter causes problems when building against prebuilts and fixing
that causes the generated snapshot to be invalid because it contains
duplicate definitions of the "sdk_foo.stubs@current" module. One
explicitly in the Android.bp file and one created by the
"sdk_foo@current" module.

Removing the duplicates from the snapshot causes errors as the name
generated by the snapshot for the component module, i.e.
"sdk_foo.stubs@current" does not match the name generated by the
"sdk_foo@current", i.e. "sdk_foo@current.stubs".

This change fixes them together.

Bug: 179354495
Test: m nothing
Merged-In: I515f235fe21755b5275af12366e96c24c94c0273
Change-Id: I515f235fe21755b5275af12366e96c24c94c0273
(cherry picked from commit a1aa7387f7)
2021-06-23 21:38:14 +01:00
Paul Duffin
51227d875b Output bootclasspath_fragment's fragments property to sdk snapshot
Adds support for printing a PropertySet that has a property whose value
is an array of structs.

Bug: 179354495
Test: m nothing
      m conscrypt-module-sdk
      - check generated Android.bp file
Change-Id: I71be04188465610bcbea4d3c9a5e8204171a1eeb
2021-06-23 17:12:31 +01:00
Paul Duffin
e59cad2089 Merge "Refactor the printing of an sdk snapshot's Android.bp contents" 2021-06-23 09:20:22 +00:00
Colin Cross
9ecb42dd31 Add default_shared_libs property
Building against a locally-built libc besides bionic requires setting
system_shared_libs to avoid circular dependencies, but modules all
over the tree assume that system_shared_libs only affects modules
building against bionic.  Add a new default_shared_libs property
that applies to all modules, which will generally be set in
arch-specific clauses to only affect the desired set of modules.

Bug: 190084016
Test: m checkbuild
Change-Id: Ia2349d84c70e503916f90a5d2702e135248f73df
2021-06-22 16:54:40 -07:00
Paul Duffin
a08e4dcfef Refactor the printing of an sdk snapshot's Android.bp contents
Previously, all of the supported types were handled explicitly using
type switches. This change switches to use reflection instead to allow
it to work with unknown types such as custom structures, support for
which will be added in a follow up change.

Bug: 179354495
Test: m nothing
Change-Id: I197f3f1dbce6d80c0127c99bbe6f836d422a8db9
2021-06-22 19:07:22 +01:00
Paul Duffin
3302871b7d Add test for sdk that contains overlapping members
This change adds a test for an sdk that includes a java_sdk_library
directly as well as transitively includes one of the component modules
that it creates. It also adds a test for the names of a versioned
java_sdk_library_import's components as fixing one requires fixing the
other.

The use case added is one that does occur in the art-module-sdk. It has
a couple of pairs of java_sdk_library and java_system_modules that
cause both the java_sdk_library and one of its component modules to be
included in the sdk snapshot. That causes problems when building
against prebuilts.

e.g. The art-module-sdk_art.module.public.api.stubs module is a
duplicate of the art.module.public.api.stubs module created by the
art.module.public.api java_sdk_library_import. The former is added
because the art-module-public-api-stubs-system-modules depends on it.

A follow up change will avoid the component being added to the sdk
snapshot separately and instead cause the snapshot to use the
component created by the java_sdk_library_import in the snapshot.

Bug: 179354495
Test: m nothing
Change-Id: Ifdc1b4a5a7968db2ded3fdb7eb02f9cff77c7c3f
2021-06-22 16:07:27 +01:00
Paul Duffin
d061d40eb6 Fix monolithic hidden API processing with prebuilts
Prebuilt modules do not provide classesJars containing annotations.
Previously, the monolithic hidden API processing just used classesJars
from all the modules that provided them so when building against
prebuilts would have fewer classesJars than when building against
sources and so would produce different hidden API flags.

This change will generate the monolithic files from both classesJars
and files previously generated from hidden API processing. A fragment
that has performed hidden API processing will contribute its generated
files whereas standalone libraries and fragments which have not
performed hidden API processing will contribute classesJars.

Bug: 177892522
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      m SOONG_CONFIG_art_module_source_build=false out/soong/hiddenapi/hiddenapi-flags.csv
      - verify that the files are identical whether built from
        source or prebuilts.
Change-Id: I06f3c7df49626bec21a452bc9abf1bb9e7545e5c
2021-06-20 19:09:09 +01:00
Paul Duffin
c8ead41ba9 Move boot jars package check into platform_bootclasspath
Bug: 177892522
Bug: 189298093
Test: m check-boot-jars
      m SOONG_CONFIG_art_module_source_build=false check-boot-jars
      - Ran both commands with and without java.lang in the
        package_allowed_list.txt
Change-Id: Iba1a881c8f6b6919d5c0c0520eb3073658f3b8d2
2021-06-17 10:54:27 +01:00
Paul Duffin
7c47515c7f Make bootclasspath_fragment_sdk_test.go tests more realistic
In the input test fixture this makes sure that every
bootclasspath_fragment is part of an apex and every content module for
the fragment is listed in the appropriate boot jars config property. It
also adds a platform_bootclasspath fragment that references the
fragment.

In the snapshot test fixtures this adds a prebuilt_apex that exports
the fragment so that the platform_bootclasspath has access to the dex
implementation files.

Bug: 177892522
Test: m nothing
Change-Id: I6c73651a359052858232b06229b4433799dd94db
2021-06-10 12:36:44 +01:00
Paul Duffin
00ceb0e7f5 Merge "Tighten bootclasspath_fragment property validation" 2021-05-24 22:19:52 +00:00
Treehugger Robot
3078f3bddc Merge "Add SOONG_SDK_SNAPSHOT_VERSION support" 2021-05-24 17:58:17 +00:00
Paul Duffin
8018e50ddb Tighten bootclasspath_fragment property validation
Previously, due to legacy reasons, the property validation did not
require a contents property and allowed the image_name to be either
"art" or "boot". Those reasons no longer apply and so this change
requires a contents property and only allows the image_name to be set
to "art" if specified.

Bug: 177892522
Test: m nothing
Change-Id: I8855d6e5365ef0b55490e90e7b6c0081cf070ee5
2021-05-24 16:22:47 +01:00
Paul Duffin
2fef136885 Generate hidden API flags for a bootclasspath_fragment
This change adds support for generating the hidden API flags for the
contents of a bootclasspath_fragment. Currently, it will only work for
the art-bootclasspath-fragment as it has no support for creating
dependencies between bootclasspath_fragment modules which will be
needed for handling any other bootclasspath_fragment.

The hidden API flag generation added by this change is completely
separate to the normal hidden API processing and is not as yet encoded
in dex jars so will have no effect on the runtime.

The generated files are provided for use by other modules and copied
into the sdk snapshot. That is needed to allow the build to verify that
the hidden API flags generated by the individual bootclasspath_fragment
modules are consistent with the flags generated for the whole
bootclasspath, whether building from source or prebuilts.

Bug: 179354495
Test: m art-module-sdk
      m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - test that the former file is a subset of the latter and that
        where they overlap they are identical.
Change-Id: Ie27303e2960953db1b7abe95510e3bca4411b09a
2021-05-14 01:48:51 +01:00
Paul Duffin
f4600f6e6a Disallow shared libraries in bootclasspath_fragment contents
Bug: 177892522
Test: m nothing
Change-Id: I78c8ef8664ec1eb0fe3456a2de2cb956162ca0da
2021-05-14 00:39:24 +01:00
Paul Duffin
a10bd3c127 Add java_sdk_library in bootclasspath_fragment contents to sdk
A java_library specified in a bootclasspath_fragment's contents
property will be automatically added to the sdk containing that
bootclasspath_fragment. Previously, if that was attempted with a
java_sdk_library it would be added to the sdk as if it was a normal
java_boot_libs which would prevent the sdk from containing the
API specific artifact such as current.txt files and stub libraries
and sources.

This change fixes that and adds a java_sdk_library as a java_sdk_libs
module.

Bug: 177892522
Test: m nothing
Change-Id: Ided57b846ce5b8940c7e898c786fd77602582ea2
2021-05-13 21:25:42 +01:00
Treehugger Robot
820397deb2 Merge "Add support for SOONG_SDK_SNAPSHOT_USE_SRCJAR" 2021-05-13 16:00:40 +00:00
Paul Duffin
475daafb1b Make hidden API bootclasspath_fragment test more realistic
Adds the bootclasspath_fragment to an apex and also adds an sdk library
to the stub_libs property.

Bug: 179354495
Test: m nothing
Change-Id: I2a5fffe915eb62923cc60732352d4d050dec4398
2021-05-13 12:40:08 +01:00
Paul Duffin
22628d50f3 Add support for SOONG_SDK_SNAPSHOT_USE_SRCJAR
Test: m SOONG_SDK_SNAPSHOT_USE_SRCJAR=true ipsec-module-sdk
      - check generated snapshot.zip file.
Change-Id: I02991e2a60d7784984b308cff2c47ee809d61f01
2021-05-12 23:13:22 +01:00
Paul Duffin
43f7bf0efd Add SOONG_SDK_SNAPSHOT_VERSION support
SOONG_SDK_SNAPSHOT_VERSION=current will generate unversioned and
versioned prebuilts and a versioned snapshot module. This is the
default behavior. The zip file containing the generated snapshot will
be <sdk name>-current.zip.

SOONG_SDK_SNAPSHOT_VERSION=unversioned will generate unversioned
prebuilts only and the zip file containing the generated snapshot will
be <sdk name>.zip.

SOONG_SDK_SNAPSHOT_VERSION=<number> will generate versioned prebuilts
and a versioned snapshot module only. The zip file containing the
generated snapshot will be <sdk name>-<number>.zip.

Bug: 157884619
Test: m nothing
      m SOONG_SDK_SNAPSHOT_VERSION=current art-module-sdk
      - check that the generated Android.bp file has not changed
        from the default.
      m SOONG_SDK_SNAPSHOT_VERSION=none art-module-sdk
      - check that the generated Android.bp file does not contain
        versioned modules.
      m SOONG_SDK_SNAPSHOT_VERSION=2 art-module-sdk
      - check that the generated Android.bp file only contains
        version 2 of each module.
Change-Id: I087e9d7d3ad110508a3d6a39bca50cbb46b3ce82
2021-05-12 22:14:32 +01:00
Paul Duffin
6213170b3e Add package module to sdk snapshot that contains licenses
This ensures that the LSC license tool does not attempt to add a
package module to an sdk snapshot Android.bp.

Bug: 181569894
Test: m nothing
Change-Id: I37b66e0df56c5b9ec255f66cb4f2a066ea96a738
2021-05-11 08:38:30 +01:00
Paul Duffin
b0bb376efa Add license modules to the sdk
Adds initial support for adding license modules to the sdk, along with
any referenced license text files. There is a number of minor
improvements to be made but the core of the support is there and it
works for ART sdks.

Basically, this change will automatically add license modules
referenced from any sdk member module as an internal sdk member. An
internal module has an sdk snapshot specific name that will not
conflict with the source module and which cannot be referenced from
outside the sdk snapshot.

Bug: 181569894
Test: m art-module-sdk art-module-host-exports art-module-test-exports
      - diff output before and after this change
        made sure that every prebuilt had a licenses field
        and that the license modules were added
Change-Id: I0c5ccabf58f4ef487e42ef8e61a5b2a74c0e81af
2021-05-11 08:28:49 +01:00
Paul Duffin
e96108d797 Add baseline test for license with sdk
This change adds a test that includes license modules that are used by
modules which are part of an sdk but which does not yet copy the
license module into the snapshot. It includes the refactoring changes
needed to allow license modules to be used in an sdk test and provides
a baseline against which future changes can be compared.

Bug: 181569894
Test: m nothing
Change-Id: I60722f43cc9cc8375d97f46eb4c281e6c38987cd
2021-05-11 08:24:59 +01:00
Paul Duffin
0df49686b3 Add support for name-less modules and property comments
Bug: 181569894
Test: m nothing
Change-Id: Ia4da1d2a55a924db82ae999da455adedbaca47c0
2021-05-11 01:00:12 +01:00
Paul Duffin
1308205638 Allow an sdk member type not to provide a bp property
Bug: 181569894
Test: m nothing
Change-Id: I7e98f14cb377683457fba32fd05d6c614f78ffeb
2021-05-11 01:00:06 +01:00
Paul Duffin
0d4ed0ac56 Allow sdk snapshot to contain source module type
Bug: 181569894
Test: m nothing
Change-Id: I30513dd22e2719099a1a1ef63cacc98336d02e9f
2021-05-11 00:59:56 +01:00
Paul Duffin
83ad956ec4 Move setting of prefer to createMemberSnapshot
Test: m nothing
Change-Id: Iab1c048fe74a5e474f68ce4acc6c1b41381368ca
2021-05-11 00:05:53 +01:00
Paul Duffin
57f8359fe3 Derive deapexer module properties from transitive dependencies
Previously, the deapexer module properties were simply passed through
from the prebuilt_apex/apex_set modules. This change derives the
properties from the transitive dependencies of the modules so that it
can automatically include the contents of a bootclasspath_fragment
module without having to duplicate them in the exported_java_libs
property.

Part of this change involves moving the addition of dependencies onto
the prebuilt apex's contents from DepsMutator to ComponentDepsMutator
so that they can be visited in the createDeapexerModule() function. The
ComponentDepsMutator runs before prebuilts without matching sources are
renamed to match the source name which simplifies the process of adding
dependencies directly onto the prebuilts.

The deapexerDeps method was renamed as the dependencies are added both
for use by deapexer and also to create APEX variants for access by
platform_bootclasspath so the name was confusing.

Bug: 187266082
Test: m nothing
      m SOONG_CONFIG_art_module_source_build=false nothing
      - this fails but not because of missing APEX variants
Change-Id: Icb4f883e7d2b63739e5ff0dc6edb8a906d80189b
2021-05-10 11:51:20 +01:00
Paul Duffin
64fb526cd0 Add SOONG_SDK_SNAPSHOT_PREFER support
By default the generated snapshot has prefer: false. Building it with
SOONG_SDK_SNAPSHOT_PREFER=true will force it to generate prefer: true.

Bug: 157884619
Test: m nothing
      m SOONG_SDK_SNAPSHOT_PREFER=true art-module-sdk
      - check the generated Android.bp file to make sure it contains
        prefer: true
Change-Id: Ied297b32d9bd4822a140fa99016f38e234c50f64
2021-05-07 14:54:35 +01:00
Paul Duffin
2d3da31d41 Make all SdkMemberTypes support transitive member deps
Previously, only those SdkMemberTypes which had specific need to
automatically add some of their dependencies as sdk members would cause
the sdk to visit their transitive dependencies. However, as any module
can have dependencies on license modules and license modules need to be
included in the sdk then it needs to visit transitive dependencies of
all members.

So, this change removes the support for allowing an SdkMemberType to
control whether its transitive dependencies are visited and just visits
them all.

This does not have any effect on sdk snapshots as in order for a
dependency to be added to an sdk it needs to be added with a tag that
implements SdkMemberTypeDependencyTag and the only tags that implement
this are used by SdkMemberTypes that had enabled transitive members.

Bug: 181569894
Test: m art-module-sdk art-module-host-exports art-module-test-exports
      - verify that this change has no effect on the generated snapshots
Change-Id: If0293af0237aa7e39335e5b8383a41c023ff5853
2021-05-06 23:13:06 +01:00
Paul Duffin
b9e7a3ca7a Make licenseModule SdkAware
Making licenseModule SdkAware caused two breakages in the build. The
breakages were both caused by having an SdkAware module that was
depended upon by a versioned sdk snapshot but which was not itself
versioned and so did not have the member_name property set.

That occured because some default licenses have been added to the
packages containing prebuilts, e.g. prebuilts_runtime_license in
prebuilts/runtime/Android.bp. They apply to both the versioned and
unversioned members.

Once license support has been added to the sdk most of those will be
removed and replaced with properly versioned license modules. However,
in the meantime it is necessary to support that.

This change avoids the issue by checking to see whether the module is
itself versioned before relying on the member_name property. It also
improves the error message when a panic is recovered to make it easier
to identify where it originates.

Bug: 181569894
Test: m nothing
Change-Id: I0e7da2e0c4a30a6f814c2faab821b185aaed2135
2021-05-06 23:13:06 +01:00
Paul Duffin
525a590565 Make sdk tests more realistic
The tests use <sdk>_<module>_<version> as the format for a versioned
sdk member name but the format should be <sdk>_<module>@<version>. This
change corrects it and also fixes a similar issue in an error message.

Bug: 181569894
Test: m nothing
Change-Id: I8be0db4bcd0b6f4d6fbdf9e402ef7257fae8e18b
2021-05-06 23:13:06 +01:00
Mathew Inwood
7e554ec2de Merge "Ensure current.zip is put in the right place." 2021-05-06 15:21:04 +00:00
Mathew Inwood
60770e2250 Ensure current.zip is put in the right place.
Using the InstallFile as the snapshot ensure that the artifact
in out/soong/mainline-sdks/ is built when building the sdk
target.

Test: m ipsec-module-sdk
Change-Id: I45ce6001dbae3a7a9c4cf50f8d7d5d67f94dbcb3
2021-05-06 13:16:27 +00:00
Paul Duffin
62035b5991 Add baseline test for sdk snapshot env variables
Bug: 157884619
Test: m nothing
Change-Id: I56c14ddfec1c33b60fe70f56e4d59b90639657fe
2021-05-06 10:50:00 +01:00
Paul Duffin
f23bc472b0 Move configuration checks from getBootImageJar
The getBootImageJar function will be removed once the boot image
creation has been moved to the platform_bootclasspath and
bootclasspath_fragment module types. However, the consistency checks
that it performs are still useful so this change moves them out
first.

The ART boot image related checks are now performed in the
bootclasspath_fragment module type. A previous change accidentally
disabled the checks when the contents property was not empty which has
been fixed. Also, the error messages have been tweaked to make it clear
that the art-bootclasspath-fragment is now the source of truth as to
its contents not the configuration.

The framework boot image related checks are now performed in the
platform_bootclasspath module type.

Initially, this change included an extra check to make sure that
UpdatableBootJars comes from updatable APEXes but that broke because
framework-wifi and framework-tethering are not currently marked as
updatable in AOSP.

Bug: 177892522
Test: m nothing
Change-Id: I80fb600fa2c7cec4566b3461c6a33c4c6f0743f4
2021-04-30 12:06:28 +01:00
Paul Duffin
895c7140a9 Add stub_libs properties to bootclasspath_fragment snapshot
Bug: 177892522
Test: m nothing
Change-Id: I7644122c4abed8e8bc2cc7b58f408bee3bdc91f6
2021-04-28 22:02:00 +01:00
Paul Duffin
2ac45f019b Remove special handling of frameworks/base/config/boot-profile.txt
Previously, the method was run in repo manifests that did not include
the frameworks/base repository so it had to handle the file being
missing. However, now that this is being called from
platform_bootclasspath which is only defined in frameworks/base that
is no longer required.

Bug: 177892522
Test: m droid
Change-Id: I77fa5a204d1426a6be24a6f0b48e18f48f3dd908
2021-04-28 09:03:46 +01:00
Paul Duffin
eee466e439 Pass module to SdkMemberTypeDependencyTag.SdkMemberType
This change allows an SdkMemberTypeDependencyTag to select the
SdkMemberType to use to add a dependency based on the module.

Bug: 177892522
Test: m nothing
Change-Id: I2d6e51b615636dc7cd41da65808ef851dd8ae6ac
2021-04-27 23:23:20 +01:00
Paul Duffin
2dc665bb48 bootclasspath_fragment: Always output contents property in snapshot
Bug: 177892522
Test: m art-module-sdk
      - check generated snapshot contains contents property
Change-Id: I122dedba6600a199bfd83b01988da36ddfd09d63
2021-04-26 21:21:58 +01:00
Paul Duffin
e95b53a55d Automatically add bootclasspath_fragment contents to sdk
Previously, both a bootclasspath_fragment and its contents had to be
explicitly added to the sdk. This change means that adding a
bootclasspath_fragment to and sdk will automatically add its contents
as if they were added using java_boot_libs.

Bug: 177892522
Test: m nothing
Change-Id: I8f7e70649f272c9a109d4606571a2d12c44b7904
2021-04-26 21:21:58 +01:00
Paul Duffin
cd06467124 Generate snapshot member list properties from dependencies
Previously, the snapshot member list properties were simply copied from
the sdk variants. That worked because the snapshot member list
properties must include all exported (i.e. visible outside the sdk
snaphot) members and only members explicitly specified in the sdk
member list properties were exported. However, a preceding change
allowed members that were transitively added (and so not explicitly
specified on the sdk) to be exported.

This change ensures that those exported members are added to the
snapshot member properties which means that they will now work properly
with the sdk version mutators.

This does not add tests for this because no existing SdkMemberType
exports its transitive dependencies and there is currently no way to
register an SdkMemberType for testing only. A following change will
exercise this functionality.

Bug: 186290299
Test: m nothing
Change-Id: Iadbdd86f6dcdd183fa99f647222a43d412271501
2021-04-24 22:33:04 +01:00
Paul Duffin
a720811c38 Allow exporting of sdk members to be done per tag
Previously, every module added to an sdk directly through one of the
SdkMemberType specific properties, e.g. java_libs, was exported and
every module added automatically via a transitive dependencies was not
exported. This change allows that behavior to be customized per tag.

Bug: 186290299
Test: m art-module-sdk
      - verify that this change does not affect its contents.
Change-Id: I563b5bcd823e61c23cdb706cfcbb13337963d550
2021-04-24 22:30:26 +01:00
Paul Duffin
2d1bb89fe3 Extract the os specific snapshot module properties handling
Previously, the code was inlined in the addSnapshotModule() method
which made it difficult to follow, did not have clear separation
between the different pieces and duplicated functionality. This
change combines the dynamic and static properties together and then
separates the processing into collation, optimization and property
set updating methods.

Bug: 186290299
Test: m nothing
Change-Id: I311eebca2337cb67634e63d71da12305f4370c07
2021-04-24 22:30:18 +01:00
Paul Duffin
fbe470e7d3 Dedup snapshot module property set initialization
Previously, the code to add the compile_multilib property was
duplicated. This change moves the logic into the existing
addMemberPropertiesToPropertySet method, renames it to reflect new
behavior and renames and promotes the variantProperties struct from the
addSnapshotModule() method to the package level scope.

Bug: 186290299
Test: m nothing
Change-Id: I832a7fd89a3cf95467a1ff0807902c9be34c50b9
2021-04-24 16:50:47 +01:00
Paul Duffin
7b0259ff62 Make order of properties in snapshot module consistent
Previously the compile_multilib property was added after the member
list properties at the top level but before them in the target.os
level. This change makes it always be added before them.

Bug: 186290299
Test: m nothing
Change-Id: I01fe272933fd11f7f730cc379f60fb2ce70f2114
2021-04-24 16:50:47 +01:00
Paul Duffin
cc3132e496 Clarify behavior around handling of sdk members
Renames some types and adds some documentation to try and clarify how
the sdk handles the members.

Bug: 186290299
Test: m nothing
Change-Id: I4038276f5f4c473a96fe06aedc43050c90416877
2021-04-24 16:49:26 +01:00
Paul Duffin
2182726990 Rename sdkMemberRef to sdkMemberVariantDep
Previously, there were two types sdkMemberRef and sdkMember that seemed
as though the former was referring to the latter but that was not the
case. The former was actually information about the dependency the sdk
had on a specific member variant. The latter is all the information
about a specific member, including all its variants.

This change attempts to clarify that relationship, both through a
better type name and through some improved documentation.

Bug: 186290299
Test: m nothing
Change-Id: I1dcc0ddf3333da9e797bb23ae945d04729839014
2021-04-24 16:48:16 +01:00
Paul Duffin
26197a65da Extract addSnapshotModule from buildSnapshot
Bug: 186290299
Test: m nothing
Change-Id: I297d63366d71f692572e8e40f1e510b6e41859e6
2021-04-24 16:48:16 +01:00
Paul Duffin
023dba0a3f Add exported_bootclasspath_fragments to prebuilt_apex/apex_set
This is needed to allow a prebuilt_bootclasspath_fragment to be used
interchangeably with a bootclasspath_fragment in the
platform_bootclasspath module.

The platform_bootclasspath module depends on APEX specific variants of
bootclasspath_fragment modules. That works because the
bootclasspath_fragment modules are part of an apex and so have an APEX
specific variant which the platform_bootclasspath can specify.

Using a prebuilt_bootclasspath_fragment in place of a
bootclasspath_fragment requires that the prebuilt also has an APEX
specific variant.

Specifying exported_bootclasspath_fragments on a prebuilt_apex/apex_set
will cause it to create an APEX variant for the named module whcih will
allow it to be selected by the platform_bootclasspath module.

Bug: 186034565
Bug: 177892522
Test: m nothing
Change-Id: I7ddacc6498ec3a4a9f26c5f78b7f9a033e494d78
2021-04-23 12:12:37 +01:00
Paul Duffin
7c95555d79 bootclasspath_fragment: Add hidden API flag files to snapshot
These are needed at the moment to ensure the hidden API processing
generates the same set of flags whether it is being generated from
source or prebuilts. Soon these will be needed to ensure that the
hidden API flags generated for the individual modules are compatible
with previous releases.

Bug: 179354495
Test: m art-module-sdk and check snapshot zip contains the files
      and the generated Android.bp references them.
Change-Id: I9a3334cc48faa381bbbcbbb59479db719042796a
2021-04-21 23:54:22 +01:00
Paul Duffin
a57835e8e5 bootclasspath_fragment: Add contents to snapshot
Bug: 177892522
Test: m nothing
Change-Id: I54fe0537b758a0e3dacd34b139ef3eb21b8841fd
2021-04-21 23:54:22 +01:00
Paul Duffin
0b28a8d356 Replace boot image with bootclasspath fragment in sdk package
Bug: 177892522
Test: m nothing
Change-Id: I67589bccbde426de2caf513bd643d9484df86c1e
2021-04-21 23:54:17 +01:00
Treehugger Robot
1cd2357dd3 Merge "Do not output two trailing newlines in generated bp files." 2021-04-20 23:41:00 +00:00
Martin Stjernholm
ee9b24e6d2 Do not output two trailing newlines in generated bp files.
Test: m platform-mainline-sdk
  Check Android.bp in the generated platform-mainline-sdk-current.zip.
Bug: 156286550
Change-Id: Ifb99288f3b85c3b986270d48e2dc7d9e15fa7536
2021-04-20 15:54:21 +01:00
Paul Duffin
1267d875b6 java_sdk_library: Make dex stub jars available for hiddenapi
The hidden API processing needs access to dex jars for the API stubs in
order to determine which dex members are part of an API surface. The
dex stubs used for the monolithic file are provided by normal
java_library modules for legacy reasons. However, the APEXes that
contribute to the bootclasspath, and so need to perform hidden API
processing, typically provide stubs created by a java_sdk_library.

This change adds support to java_sdk_library/_import to make the dex
stub jars available when requested, that involves:
1. Adding compile_dex property to java_sdk_library_import and
   propagating it down the the java_import modules for the stubs. That
   is already handled for java_sdk_library.
2. Propagating the java_sdk_library compile_dex property to the
   java_sdk_library_import in the generated snapshot.
3. Refactoring and wiring to make the dex stubs jar available to other
   parts of Soong.

Bug: 179354495
Test: m nothing
Change-Id: I5895d4f2ba0b684870862b9429b2364865e4afc6
2021-04-16 18:48:20 +01:00
Paul Duffin
e1381887ae Migrate sdk tests away from checkAndroidBpContents
Replaces a single call to checkAndroidBpContents(...) with separate
calls to check the versioned and unversioned Android.bp files.

Test: m nothing
Bug: 179354495
Change-Id: I270bf73909958d97b2f298e8d7f6d10a1f75ae71
2021-04-16 18:48:20 +01:00
Jingwen Chen
2f6a21e957 Export OsTypeList through a function.
This is a cleanup CL to make the OsTypeList accessor consistent with
ArchTypeList by creating a function that creates a copy of the
underlying slice.

Test: TH
Change-Id: I346c5c3937bbdb8a373e4d7135b896a5274cc1f1
2021-04-05 07:35:06 +00:00
Treehugger Robot
25c47a43a5 Merge ""current" is implicitly added to stubs.versions" 2021-04-01 04:53:55 +00:00
Jiyong Park
d4a3a137ed "current" is implicitly added to stubs.versions
So far, when a library `libfoo` has `stubs.versions: ["10", "11"]`, then
`shared_libs: ["libfoo"]` is linked to the version 11 of the stub.

This requires the author of `libfoo` to manually update the property
whenever a new version is introduced. Otherwise, clients are not able
to use the newly added APIs because the latest stub is for an old
version.

This change eliminates the need for manual updating. "current" version
is always implicitly added to `stubs.versions`. It is added even when
nothing is set on the property, if `stubs.symbol_file` is set. i.e.

```
cc_library {
    name: "libfoo",
    stubs: {
        symbol_file: "libfoo.map.txt",
	// no versions: [...] needed
    },
}

cc_library {
    name: "a_client",
    shared_libs: ["libfoo"],
    apex_available: ["myapex"],
    min_sdk_version: "29",
}

apex {
    name: "myapex",
    native_shared_libraries: ["a_client"],
    min_sdk_version: "29",
}
```

`a_client` links to the "current" stub of `libfoo` that has all symbols
shown in the map file.

Note that, above doesn't mean that the client has unlimited access to
APIs that are introduced even after the min_sdk_version of the client
(29 in this example). The use of such APIs still has to be guarded with
`__builtin_available` check.

Bug: N/A
Test: m
Change-Id: I70bb1600c18e74d36c6b24c3569d2149f02aaf96
2021-04-01 09:58:53 +09:00
Paul Duffin
55e740e9a6 Remove varargs from RunTest(t *testing.T)
Use GroupFixturePreparers instead.

Bug: 182885307
Test: m nothing
Change-Id: Iaedb0ddc9d6a704f4d41363e705f3025a1291dc8
2021-03-31 16:03:59 +01:00
Colin Cross
045ed0f684 Merge changes I71a83e3a,I66101c0c,Ie387c8c4,Iea742e75
* changes:
  Strengthen metalava sandbox support using sbox
  Move metalava's output files into a subdirectory
  Fix lint warnings in droidstubs.go
  Split droidstubs out of droiddoc.go
2021-03-26 15:14:53 +00:00
Colin Cross
cb77f75aae Move metalava's output files into a subdirectory
Move all of the output files of the metalava rule into a single
subdirectory to make it compatible with sandboxing in sbox.

Test: TestDroidstubs
Change-Id: I66101c0c224dee702c8175e61c12cc9cd1aa8b93
2021-03-25 11:11:36 -07:00
Paul Duffin
bb9ff5108b Filter duplicate modules in platform_compat_config_singleton
Previously, unpacking a snapshot containing a
prebuilt_platform_compat_config into a source build would cause build
failure because of duplicate ids because the singleton would collate
ids from both prebuilts (versioned and unversioned) and source.

This change filters out versioned prebuilts and only uses prebuilts
that are preferred and source modules that have not been replaced by a
prebuilt.

Bug: 182402754
Test: m nothing
Change-Id: Idacbb34444e5156370df70bf88c6e8a7e2d67890
2021-03-25 12:53:25 +00:00
Paul Duffin
db462dd987 Disallow non-existent paths in sdk package
Test behavior was changed a while ago so that tests by default ignore
non-existent source paths (unless they explicitly check for/rely on
them). Prior to that CheckSnapshot() could detect when files were
missing from the snapshot but it no longer can.

This change disallows non-existent source files in all the sdk tests
which means that they are disallowed when processing the snapshots as
they use the same preparers as were used to process the sources.

This caused a test failure which has been temporarily ignored and has
a TODO and bug associated with it.

Bug: 183184375
Test: m nothing
Change-Id: I969d8515d20ef5ae515f2b5f93d8ed4e4f8ede75
2021-03-25 12:53:22 +00:00
Paul Duffin
1822a0a371 Improve sdk snapshot testing
The sdk produces snapshots that are expected to be unpacked in one of
the Android repos. Often that can lead to issues due to conflicts
between the source and prebuilts. This change attempts to avoid those
conflicts by testing the different ways that those files can be used.

With the existing test to cover adding the snapshot
This change will cause the sdk tests to check the following:
1) Snapshot on its own (already done).
2) Snapshot plus original source where the original source is
   preferred.
3) Snapshot plus original source where the snapshot is preferred.

It also adds the ability for tests to provide their own custom checkers
to verify the result of each of the previous tests.

This change reveals a number of bugs already present. Rather than
attempt to fix them this change adds the ability to specify error
handlers for the two cases that mix source and snapshot to allow those
errors to be temporarily ignored while allowing the majority of the
tests to benefit from this improvement. Each of those failures has a
TODO and bug associated with it.

Bug: 183184375
Test: m nothing
Change-Id: I105233195074dbe7a6422b6dfc5486e74398ea15
2021-03-25 12:50:18 +00:00
Paul Duffin
76e5c8a37f Convert test that disallows non existent paths to use fixtures
This change needed to add some additional files to the registered
files for PrepareForTestWithJavaDefaultModules because otherwise they
would fail when "TestAllowNonExistentPaths = false". Those files were
being added by the TestJavaLintRequiresCustomLintFileToExist (albeit in
some cases in different locations to that required by the default
modules but as the files are needed by the modules defined in
PrepareForTestWithJavaDefaultModules they should be defined in it.

A couple of other places also provided some files so moving them into
PrepareForTestWithJavaDefaultModules caused some conflicts which needed
to be resolved.

Bug: 183184375
Test: m nothing
Change-Id: I76ce9f1673c1c1c4000635b76b8377d582224bf1
2021-03-24 22:08:05 +00:00
Paul Duffin
ac94726ef2 Remove testSdkWithJava
Bug: 181070625
Test: m nothing
Change-Id: I9ef906a386cc87f69b166ec88e0b6c7388c3d06a
2021-03-24 15:47:28 +00:00
Paul Duffin
9ec86b14e7 Convert java_sdk_test.go tests to fixtures
Bug: 182638834
Test: m nothing
Change-Id: I6a1fabdd2c8385e5fbaef6985047f9d0bdceb209
2021-03-24 15:47:28 +00:00
Paul Duffin
001b2342f7 Add platform_compat_config to sdk
Bug: 182402754
Test: m nothing
Change-Id: Ife3f4f64fc116d62eb7c3cc10c50e00f19d1d81c
2021-03-22 19:13:40 +00:00
Paul Duffin
04b4a19fe6 Add TestBasicSdkWithBootImage
Added to reproduce the conditions that lead to the error reported in
the bug so they can be fixed. There were a number of issues that were
fixed in previous changes and this test verifies that they have been
fixed.

Bug: 182992071
Test: m nothing
Change-Id: I2197899b284a99973e698db314b15812f602b141
2021-03-22 19:13:40 +00:00
Paul Duffin
4a1d451405 Convert boot_image_sdk_test.go to fixtures
Bug: 182638834
Test: m nothing
Change-Id: I8c9cb399b15d04ec8465a3b3bb4b43561aef46fd
2021-03-22 19:13:22 +00:00
Paul Duffin
cf3ee2f87e Allow sdk package tests to optimize their test setup
Extracts the setup for apex from prepareForSdkTest to allow it to be
reused without using all of it. That will allow tests to optimize their
test setup.

Bug: 182638834
Test: m nothing
Change-Id: I2056103b15c2737a616ee29ff890c6af0722e6d2
2021-03-22 18:31:53 +00:00
Paul Duffin
c93c98e315 Use test fixtures in CheckSnapshot()
Using the preparer(s) that were used to run the test to verify the
integrity of the generated snapshot ensures that it will be verified in
the same environment as the snapshot was generated. This ensures that
as sdk tests are migrated to use fixtures that are optimized for each
test that they will benefit from those optimizations when checking the
snapshot.

Bug: 183184375
Test: m nothing
Change-Id: I62b383f9a1d9a77d1cabb101d9d1e4a976170fe3
2021-03-22 18:31:53 +00:00
Paul Duffin
89648f98fa Remove usages of FixtureFactory from misc packages
These packages have already been migrated to use per test build
directory so have no need for a FixtureFactory.

Bug: 183235980
Test: m nothing
Change-Id: I667d1d992caaf0f615de91f89efdae11c44986c2
2021-03-22 18:31:53 +00:00
Paul Duffin
6bef6fee3c Merge "Ensure that DepIsInSameApex is not called for ExcludeFromApexContentsTag" 2021-03-22 18:30:10 +00:00
Paul Duffin
c737facd2b Merge "Replace AssertPanic with AssertPanicMessageContains" 2021-03-22 08:58:28 +00:00
Paul Duffin
4c3e8e2d67 Ensure that DepIsInSameApex is not called for ExcludeFromApexContentsTag
The ExcludeFromApexContentsTag marker interface was added to avoid
every implementation of DepIsInSameApex() from having to deal with the
special tags, like PrebuiltDepTag. Unfortunately, when adding that
not all calls to DepIsInSameApex() were protected which meant that the
BootImageModule, which panics if it doesn't recognize a tag, was
causing failures. This change documents the need and improves the
consistency.

A follow up change will add a test for this.

Bug: 182992071
Test: m nothing
Change-Id: If0bf9a7447ebf7a0bb0c88e91951a7220d4af45c
2021-03-22 08:43:55 +00:00
Paul Duffin
d14cbc1e69 Add prepareForSdkTestWithJava
Bug: 182638834
Test: m nothing
Change-Id: I86cb08a640c7e39c1eaaf54926699f32a32ccc9b
2021-03-21 11:01:13 +00:00
Paul Duffin
9f4b3bbb7c Replace AssertPanic with AssertPanicMessageContains
Bug: 182885307
Test: m nothing
Change-Id: Idffa314285c90080796cc3df391de9c314eaa422
2021-03-20 12:08:51 +00:00
Paul Duffin
ebcb37bab1 Merge "Migrate sdk package to a per test build directory" 2021-03-18 17:05:11 +00:00
Paul Duffin
573989d821 Prevent ApexInfoMutator from creating unnecessary variants
Adds the AlwaysRequireApexVariantTag interface to enable
ApexInfoMutator to differentiate between a tag that is excluded from
apex contents but still requires an apex variant and a tag that is
excluded from apex contents and does not require an apex variant.

That is needed to support the sdkMemberVersionedDepTag which excludes
the target from being added to the APEX but requires an APEX variant.
A more detailed explanation is in the comments.

The AlwaysRequireApexVariant() method follows the pattern used in
ReplaceSourceWithPrebuilt of having a method that returns a bool to
trigger the behavior and not say ExcludeFromApexContentsTag that simply
relies on the tag implementing an interface to trigger. That is because
the former is more flexible and allows a tag type to parameterize the
behavior if necessary.

The tags that this will exclude from creating an apex variant are:
* PrebuiltDepTag - by the time the apex variant has been created any
  preferred prebuilts will have replaced the sources so there is no
  need to create an APEX variant if the only dependency path from the
  APEX to the prebuilt is via this tag.
* hiddenApiAnnotationsDependencyTag - the target of which is a purely
  build time artifect and MUST NEVER end up in the APEX.

It will also stop calling DepIsInSameApex for any dependency created
by the sdkMemberVersionedDepTag. Which will fix the issue reported in
the bug.

Bug: 182992071
Test: m nothing
Change-Id: I9569e488d6446ca45d3ea8f32a9b74524eb865df
2021-03-18 09:05:28 +00:00
Paul Duffin
abbf63d650 Migrate sdk package to a per test build directory
Bug: 182885307
Test: m nothing
Change-Id: I051f2572ce5e94e2a3a66cf7663797178b7d1506
2021-03-18 01:48:25 +00:00
Paul Duffin
36474d322b Stop sdk package depending on testing.T being embedded in TestResult
This change is in preparation for removing testing.T from TestResult.

Bug: 181070625
Test: m nothing
Change-Id: I67535aff0d894e6e3d8456b75540f340af853355
2021-03-13 15:59:33 +00:00
Paul Duffin
ebddef39dd Fix build failure in boot_image_sdk_test.go
Test: m nothing
Change-Id: I1f386fe4119ac9c3efc8d046142d02862af5fd99
2021-03-12 08:20:27 +00:00
Paul Duffin
32a8aadfaa Merge "Add prebuilt_boot_image and add boot_images to sdk" 2021-03-12 08:06:45 +00:00
Paul Duffin
c10ee77ea6 Merge changes Ifc96992e,Ic76523ba
* changes:
  Support test fixtures in sdk package
  Add apexFixtureFactory to apex package
2021-03-12 08:01:31 +00:00
Paul Duffin
fe9a9e3f7d Add missing // to clarify comment
Bug: 181070625
Test: m nothing
Change-Id: Ia22a1be7a05b3a9efd34f2bf70216eb046396be0
2021-03-11 17:41:01 +00:00
Paul Duffin
4a2a29ce6a Support test fixtures in sdk package
Bug: 181070625
Test: m nothing
Change-Id: Ifc96992e54c1b1d89a82b88ab27e555ae267a51e
2021-03-11 17:25:29 +00:00
Paul Duffin
981b94b460 Switch CheckSnapshot from a testSdkResult method to a function
This will allow the testSdkResult to be replaced with the TestResult
when switching sdk package to use test fixtures.

Bug: 181070625
Test: m nothing
Change-Id: Ieca63f4c189f5e804102aeb4073289ea03143b6e
2021-03-11 13:39:24 +00:00
Paul Duffin
8306f25679 Make testSdkResult compatible with android.TestResult
This change makes it easier to switch the sdk package over to using the
new fixture mechanism by removing inconsistencies between the
testSdkResult and TestResult structures.

Bug: 181070625
Test: m nothing
Change-Id: Ic4c06e08ea5060fd09123f2ca65580e18b4d2ef6
2021-03-11 13:38:56 +00:00