Commit graph

1514 commits

Author SHA1 Message Date
Paul Duffin
3bae068ee5 Generalize deapexer module to export any files
Previously, the deapexer module had to duplicate the java library
specific logic for constructing the path to the library's dex file in
the .apex file. That is not something that the deapexer needs to be
aware of, all it needs is a list of files that should be exported.

This change moves that logic into the prebuilt_apex/apex_set modules
and generalizes the deapexer module so that it can export any files
that are requested.

The deapexer module does still need to know which java modules need
access to exported files so it can add dependencies from them onto
itself. However, it does not need to know what the type of the module
is.

Bug: 186455808
Test: m nothing
      m SOONG_CONFIG_art_module_source_build=false nothing
Change-Id: I71c6f0f761efe3b6d66d54273786e98cd545811c
2021-05-10 14:26:37 +01:00
Paul Duffin
bb0dc13afe Simplify prebuilt_apex/apex_set common properties
Moves all properties common to prebuilt_apex and apex_set module types
into prebuiltCommonProperties. Removes selectedApexProperties from
Prebuilt as that duplicated, and masked, the same property in
prebuiltCommon.

Bug: 187266082
Test: m nothing
      m SOONG_CONFIG_art_module_source_build=false nothing
      - this fails but not because of missing APEX variants
Change-Id: Ice7dfb8e18bb526fdd5b41bcd518c6971564d81b
2021-05-10 12:01:29 +01:00
Paul Duffin
57f8359fe3 Derive deapexer module properties from transitive dependencies
Previously, the deapexer module properties were simply passed through
from the prebuilt_apex/apex_set modules. This change derives the
properties from the transitive dependencies of the modules so that it
can automatically include the contents of a bootclasspath_fragment
module without having to duplicate them in the exported_java_libs
property.

Part of this change involves moving the addition of dependencies onto
the prebuilt apex's contents from DepsMutator to ComponentDepsMutator
so that they can be visited in the createDeapexerModule() function. The
ComponentDepsMutator runs before prebuilts without matching sources are
renamed to match the source name which simplifies the process of adding
dependencies directly onto the prebuilts.

The deapexerDeps method was renamed as the dependencies are added both
for use by deapexer and also to create APEX variants for access by
platform_bootclasspath so the name was confusing.

Bug: 187266082
Test: m nothing
      m SOONG_CONFIG_art_module_source_build=false nothing
      - this fails but not because of missing APEX variants
Change-Id: Icb4f883e7d2b63739e5ff0dc6edb8a906d80189b
2021-05-10 11:51:20 +01:00
Paul Duffin
5dda3e387e Move creation of deapexer modules to separate mutator
Previously, deapexer modules were created by the LoadHook which meant
that the deapexer module could not use any information derived from the
dependencies of the prebuilt_apex/apex_set modules. This change moves
the creation into a separate mutator that runs after
ComponentDepsMutator and before DepsMutator. That means that a follow
up change can use information from dependencies added by the former
mutator in order to create the deapexer module and the deapexer module
can itself add dependencies onto other modules.

This change also dedups the logic to determine whether a deapexer
module is needed by pushing it down into the createDeapexerModule
method which was renamed to createDeapexerModuleIfNeeded to reflect its
conditional nature.

Bug: 187266082
Test: m nothing
      m SOONG_CONFIG_art_module_source_build=false nothing
Change-Id: I65316473ff1e4b2827ff48ab5a870a8ce5c0475a
2021-05-10 11:41:40 +01:00
Paul Duffin
b17d044628 Transitively add APEX variants for contents of prebuilt_apex/apex_set
This is part of the work needed to allow the exported_java_libs
property to be replaced by exported_bootclasspath_fragments.

Bug: 187266082
Test: m nothing
      m SOONG_CONFIG_art_module_source_build=false nothing
Change-Id: I2c1d70a390200b93163f9799719290c9d55a041c
2021-05-06 23:43:43 +01:00
Colin Cross
a5b1ea248f Merge "Ignore cc.Module.HideFromMake when producing flattened apex Android.mk entries" 2021-04-30 14:43:03 +00:00
Paul Duffin
f23bc472b0 Move configuration checks from getBootImageJar
The getBootImageJar function will be removed once the boot image
creation has been moved to the platform_bootclasspath and
bootclasspath_fragment module types. However, the consistency checks
that it performs are still useful so this change moves them out
first.

The ART boot image related checks are now performed in the
bootclasspath_fragment module type. A previous change accidentally
disabled the checks when the contents property was not empty which has
been fixed. Also, the error messages have been tweaked to make it clear
that the art-bootclasspath-fragment is now the source of truth as to
its contents not the configuration.

The framework boot image related checks are now performed in the
platform_bootclasspath module type.

Initially, this change included an extra check to make sure that
UpdatableBootJars comes from updatable APEXes but that broke because
framework-wifi and framework-tethering are not currently marked as
updatable in AOSP.

Bug: 177892522
Test: m nothing
Change-Id: I80fb600fa2c7cec4566b3461c6a33c4c6f0743f4
2021-04-30 12:06:28 +01:00
Colin Cross
76f0ba81d3 Ignore cc.Module.HideFromMake when producing flattened apex Android.mk entries
The modules in a flattened apex have suffixes that ensure they don't
conflict with the platform versions, so they can ignore HideFromMake.
This ensures that Make can install symbols for all libraries in
apexes.

Fixes: 180622230
Test: banchan com.android.adbd && m && ls $OUT/symbols/apex/com.android.adbd/lib
Change-Id: I76d1aba2eed6ae27d9a954bf88f40f0d00fc6a98
2021-04-29 13:54:49 -07:00
Colin Cross
203b421043 Remove obsolete llndk_library
Remove llndk_library in favor of cc_library with llndk.symbol_file.

Bug: 170784825
Test: m checkbuild
Test: TestLlndkLibrary
Change-Id: I43580976589a7a2a176d7442be53fa043c0c8324
2021-04-26 18:41:00 -07:00
Paul Duffin
ba6afd0dba Allow contents and image_name to be specified together
Previously, only one of the contents or image_name properties could be
specified at once which meant that there was no way to create a
prebuilt which lists its fixed contents while at the same time allowing
it to check that that the contents matched what the build configuration
required.

e.g. a prebuilt_bootclasspath_fragment that had image_name: "art",
could not list its contents and also check that those contents matched
the ART_APEX_JARS which the build configuration required.

This change allows contents and image_name to be specified together and
adds a check to make sure that the contents are consistent with the
configuration appropriate to the image_name. The check is only
performed for modules that are active so that a
prebuilt_bootclasspath_fragment which was created without coverage
enabled (the default) would not cause a build failure in a coverage
build unless it was preferred.

Bug: 177892522
Test: m nothing
Change-Id: Ie601f29f707b3f6030fa7d252afa2c4826cc9f8e
2021-04-26 21:21:58 +01:00
Paul Duffin
190fdef294 Delegate retrieval of dex boot jar for apex to the bootclasspath_fragment
The dex boot jar for the apex must have had hidden API flags encoded
into it. Currently, the hidden API processing is done within the java
modules themselves so the apex gets the dex boot jar from them.

However, as part of the hidden API modularization work the hidden API
encoding will be performed by the bootclasspath_fragment so this change
prepares for that by delegating the retrieval of the dex boot jars to
the bootclasspath_fragment, via BootclasspathFragmentApexContentInfo.

For the moment that simply delegates straight back to the java module
so this change does not change the build. It will however make it
easier to switch hidden API encoding to the bootclasspath_fragment in
future.

Bug: 179354495
Test: m com.android.art
      - verify that this change does not change its contents
Change-Id: I12eba333749be976bcc72661bb9d6be6cc3c56e3
2021-04-26 11:09:25 +01:00
Paul Duffin
cc33ec8838 Extract bootclasspathFragment related code into separate functions
The apexBundle.GenerateAndroidBuildActions is 400+ lines long which
makes it difficult to understand what is happening.

Bug: 177892522
Test: m com.android.art
      - verify that this change does not change its contents
Change-Id: I07cb31d246377869771bdb18fdc6291365adf399
2021-04-26 11:09:02 +01:00
Paul Duffin
e946b327f4 Rename BootImageInfo to BootclasspathFragmentApexContentInfo
Currently, it only contains ART boot image related information, i.e.
.art, .oat and .vdex files. However, follow up changes will extend that
to include other information from bootclasspath_fragment.

Bug: 177892522
Test: m nothing
Change-Id: I2b226131c0eccff0c739a18f265f90caa10a91d9
2021-04-26 10:05:27 +01:00
Paul Duffin
7771eba88e Rename BootImageModule to BootclasspathFragmentModule
Also renames files, tests, module types in a similar fashion.

There are still some references to image and boot image. They are kept
for the following reasons:
* image_name - this is the name of an ART boot image, i.e. the
  collection of .art/.oat/.vdex files.
* BootImageInfo - again this is related to the ART boot image.
* .../art_boot_images/... paths - ditto.

Bug: 177892522
Test: m nothing
Change-Id: Ie1f4738061d131fee75de48bc26a7601481bad4d
2021-04-23 16:59:26 +01:00
Paul Duffin
023dba0a3f Add exported_bootclasspath_fragments to prebuilt_apex/apex_set
This is needed to allow a prebuilt_bootclasspath_fragment to be used
interchangeably with a bootclasspath_fragment in the
platform_bootclasspath module.

The platform_bootclasspath module depends on APEX specific variants of
bootclasspath_fragment modules. That works because the
bootclasspath_fragment modules are part of an apex and so have an APEX
specific variant which the platform_bootclasspath can specify.

Using a prebuilt_bootclasspath_fragment in place of a
bootclasspath_fragment requires that the prebuilt also has an APEX
specific variant.

Specifying exported_bootclasspath_fragments on a prebuilt_apex/apex_set
will cause it to create an APEX variant for the named module whcih will
allow it to be selected by the platform_bootclasspath module.

Bug: 186034565
Bug: 177892522
Test: m nothing
Change-Id: I7ddacc6498ec3a4a9f26c5f78b7f9a033e494d78
2021-04-23 12:12:37 +01:00
Treehugger Robot
1552c7b178 Merge "Support paths for vintf_fragments from Soong." 2021-04-22 17:03:31 +00:00
Paul Duffin
6e0a62b773 Merge "bootclasspath_fragment: Add contents to snapshot" 2021-04-22 17:01:24 +00:00
Jaewoong Jung
6a328ff8ac Merge "Make apex.key and certificate overridable." 2021-04-22 15:37:22 +00:00
Paul Duffin
57d13c6892 Merge "Move monolithic stub flags generation to platform_bootclasspath" 2021-04-22 15:06:16 +00:00
Paul Duffin
7cbdf81e95 Merge changes I67589bcc,Ibcfae390
* changes:
  Replace boot image with bootclasspath fragment in sdk package
  bootclasspath_fragment must only depend on source contents
2021-04-22 14:02:43 +00:00
Paul Duffin
74431d57c6 Move monolithic stub flags generation to platform_bootclasspath
As part of that this change:
* Moves code that will be common to platform_bootclasspath and
  bootclasspath_fragment from hiddenapi_singleton.go into
  hiddenapi_modular.go.
* Fixes the tests in hiddenapi_singleton_test.go but intentionally
  does not rename them or move them into a more appropriate place so
  as to make it easier to see the differences. A TODO has been added
  and these will be cleaned up in a follow up change.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I680e4dab2e6bdf4a655fa9f255c195175904667e
2021-04-22 14:23:28 +01:00
Paul Duffin
a57835e8e5 bootclasspath_fragment: Add contents to snapshot
Bug: 177892522
Test: m nothing
Change-Id: I54fe0537b758a0e3dacd34b139ef3eb21b8841fd
2021-04-21 23:54:22 +01:00
Nikita Ioffe
6681ef3101 Merge "Don't compress apexes when building in unbundled mode" 2021-04-21 21:50:32 +00:00
Paul Duffin
6589805e2a bootclasspath_fragment must only depend on source contents
This change ensures that bootclasspath_fragment only depends on source
modules and prebuilt_bootclasspath_fragment only depends on prebuilt
modules.

It does that in two ways:
1. It adds the dependencies in ComponentDepsMutator method which is
   called before any renaming of prebuilts is done which makes it very
   easy to add a dependency directly onto either the source or prebuilt
   as required.
2. It uses a tag which prevents dependencies on a source module from
   being replaced with a dependency on a prebuilt module which ensures
   that a dependency on the source modules is not replaced with a
   dependency on a prebuilt module.

Bug: 177892522
Test: m nothing
Change-Id: Ibcfae39083afbc07fcf729ead3ed5f5d020845bf
2021-04-21 19:05:37 +01:00
Liz Kammer
7b3dc8a259 Support paths for vintf_fragments from Soong.
The property vintf_fragments accepts paths and references to other
modules; however, none of that is passed onto make, resulting in errors
if a non-local path is used.

Test: m
Bug: 184567830
Change-Id: If3b56ea8eec3b95b3a310b58bffd045cedd8ee52
2021-04-21 09:11:32 -04:00
Jaewoong Jung
4cfdf7de26 Make apex.key and certificate overridable.
Test: apex_test.go
Fixes: 185477325
Change-Id: I8992ac5606fa92f53f07e6870a034f2d3352e618
2021-04-20 16:21:24 -07:00
Paul Duffin
94f1963b6d Remove unused boot_images property and rename tag
Bug: 177892522
Test: m nothing
Change-Id: I9ffd6dda23aefea183f87e22497c17fb8a276ece
2021-04-20 14:46:08 +01:00
Nikita Ioffe
b6ea6c2d3c Don't compress apexes when building in unbundled mode
Test: TARGET_BUILD_APPS=com.android.neuralnetworks m apps_only
Bug: 185708645
Change-Id: I3fdf96f2b8decdff1422549bca413e7660ffb706
2021-04-20 10:20:15 +01:00
Jiyong Park
7b4fb51919 Merge changes Ib07d4410,I0b306292
* changes:
  ApexInfo doesn't pass MinSdkVersion as string, but as ApiLevel
  Remove nativeApiLevelFromUserWithDefault
2021-04-15 23:21:55 +00:00
Nikita Ioffe
5b94c8c007 Merge changes I83828f58,I1179235e
* changes:
  Generate hashtree for compressed apexes
  Don't compress apexes produced by apex_test module
2021-04-15 11:40:43 +00:00
Jiyong Park
4eab21d5a2 ApexInfo doesn't pass MinSdkVersion as string, but as ApiLevel
ApexInfo is not part of the properties struct. It can handle structs
having private fields.

Bug: 1663140
Test: m
Change-Id: Ib07d4410f0ce187c9de347da34b84b814b2eb537
2021-04-15 15:17:54 +09:00
Nikita Ioffe
bc03588bec Generate hashtree for compressed apexes
Bug: 184615428
Test: flashed build on device, checked compressed apex has hashtree
Change-Id: I83828f581e84504c47047e7692817da5c44a6ec1
2021-04-14 21:35:24 +01:00
Nikita Ioffe
05b999ecb2 Don't compress apexes produced by apex_test module
AFAIU, apex_test is used by modules to create a test version of their
apex for e2e testing of the update flow. Since it's a regular pattern
for test_apex to use the same apex_defaults as the "production" apex,
and due to the fact that we are only going to compress pre-installed
version of the apex, it makes sense to explicitly not compress
test apexes.

Bug: 185082717
Test: build test_com.android.media verified it's not compressed
Test: atest sdkextensions_e2e_tests
Change-Id: I1179235efe068fefc582369a4218ba967e7efbf3
2021-04-14 21:08:13 +01:00
Paul Duffin
00b2bfdea5 Move hidden API index file rule to platform_bootclasspath
This change moves the monolithic hidden API index file creation rule
from the hiddenAPIIndexSingleton into the platform_bootclasspath. It
also moves the corresponding test from java/hiddenapi_singleton_test.go
to java/platform_bootclasspath_test.go.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ia295d0f7ae9b51ea816f16921aa42339ed91704e
2021-04-14 18:54:29 +01:00
Jiyong Park
f398995a64 Merge "SdkSpec is fully using ApiLevel" 2021-04-14 13:27:05 +00:00
Paul Duffin
8f146b99e6 prebuilt_apex created ApexInfo must not include prebuilt_ prefix
As part of the work to modularize the hiddenAPI processing the
generation of the monolithic hidden API index file needs to be moved
to the platform_bootclasspath module type. Doing that broke the
TestBootDexJarsFromSourcesAndPrebuilts tests which checks the inputs to
the rule that creates that file. Fixing that required added a
platform_bootclasspath module to the test fixture for those tests which
highlighted an issue with the prebuilt_apex module.

Previously, when the prebuilt_apex created apex variants it would use
its own name as the apex variant name, even when that name included the
prebuilt_ prefix. That broke the platform_bootclasspath logic as it was
looking for apex variants for "myapex" but the only ones available were
"prebuilt_myapex".

This change ensures that it always uses the unprefixed name and fixes
the TestNoUpdatableJarsInBootImage to match. This also adds some
improved error reporting in platform_bootclasspath which helped debug
this problem.

Bug: 177892522
Test: m nothing
Change-Id: I3e88b5cec767f77dcc0e94b3ae38b499d07eadf0
2021-04-13 19:15:27 +01:00
Paul Duffin
60264a0b99 Improve realism of boot jar tests
Boot jars, updatable boot jars and art apex jars are part of two
separate but related configuration objects, the main Config struct
(actually the nested productVariables struct) and the dexpreopt
specific GlobalConfig. The fields in both are initialized from the same
data in the make config files but handled separately.

Previously each test that used one of the configuration objects would
generally just initialize the one it used. That would make the test
sensitive to the specific configuration object that was used. A
refactoring that change the code from using one configuration object to
the other would cause the test to fail.

Also, some tests would inadvertently create invalid configurations by
setting ArtApexJars without also setting BootJars. While the ability to
create invalid configurations is useful (and there are some tests that
exist to verify the behavior in that case) most tests should not be
using them.

This change simplifies the configuration of the tests and improves
their realism by:
1. Providing a new FixtureConfigureBootJars method that takes a set of
   boot jars and sets ArtApexJars, and BootJars in the
   dexpreopt.GlobalConfig and BootJars in the product variables too.
2. Providing a new FixtureConfigureUpdatableBootJars method that takes
   a set of boot jars and sets UpdatableBootJars in both the
   dexpreopt.GlobalConfig and productVariables.
3. Migrating existing tests to use these new methods.

Some tests still use the dexpreopt.FixtureSet...Jars() methods directly,
generally to create invalid configurations.

Bug: 177892522
Test: m nothing
Change-Id: I4d8f0b9762cfcc7ae6383bef08563d7c3fa13955
2021-04-13 14:48:34 +01:00
Liz Kammer
bda284bb19 Merge "Support paths for init_rc from Soong." 2021-04-12 19:05:27 +00:00
Jiyong Park
a417f881bd Merge "Shared lib dependencies from rlib are included in APEX" 2021-04-08 23:31:28 +00:00
Paul Duffin
49ab1d38de Merge "Allow platform_bootclasspath to specify contributing fragments" 2021-04-08 20:34:19 +00:00
Paul Duffin
d32118c5c4 Merge "Add dependencies from platform_bootclasspath to contents" 2021-04-08 20:34:03 +00:00
Paul Duffin
62d8c3b110 Allow platform_bootclasspath to specify contributing fragments
Adds the fragments property to the platform_bootclasspath to allow the
fragments that contribute to it to be specified.

Bug: 177892522
Test: m nothing
Change-Id: I14f83d9336f6984442c7315cc86dfdd0a0fd2d20
2021-04-08 18:53:04 +01:00
Paul Duffin
b432df9cda Add dependencies from platform_bootclasspath to contents
Adds a FinalDeps mutator to add dependencies from the
platform_bootclasspath to the configured boot jars which can be from
either the platform or any apex. It adds dependencies for every
configured boot jar, whether in ArtApexJars, BootJars or
UpdatableBootJars.

At the moment the dependencies are only used for testing purposes
but following changes will make more use of them.

Bug: 177892522
Test: m nothing
Change-Id: I981305bf45bc20539a3d36987252f490e2b885cc
2021-04-08 18:53:04 +01:00
Jiyong Park
94e22fd35e Shared lib dependencies from rlib are included in APEX
This change fixes a bug that shared lib dependencies of an rlib is not
installed to the APEX even when the rlib is part of the APEX.

Bug: N/A
Test: m
Change-Id: I88fe461584499839d8018d6b4292374592e7562b
2021-04-08 18:20:39 +09:00
Jiyong Park
9231537fe2 SdkSpec is fully using ApiLevel
Previously, SdkSpec was constructed only from the user string. It didn't
make use of the Config struct where information about the latest stable
SDK version, etc. is recorded. As a result, the build system couldn't
check if the sdk version "current" is referring to the in-development
(i.e.  not-yet-frozen) SDK version or the latest stable version.
"current" was always assumed to be in-development (IsPreview() returns
true) even when Platform_sdk_final == true.

As the first step for fixing that, this change requires
android.EarlyModuleContext to be passed when constructing SdkSpec from
the user string.

In the following changes, "current" will be mapped to either
FutureApiLevel (10000) or one of the FinalApiLevels() depending on
whether the platform SDK was finalized or not.

Bug: 175678607
Test: m
Change-Id: Ifea12ebf147ecccf12e7266dd382819806571543
2021-04-08 11:27:24 +09:00
Liz Kammer
0c4f71cd96 Support paths for init_rc from Soong.
The property init_rc accepts paths and references to other modules;
however, none of that is passed onto make, resulting in errors if a
non-local path is used.

Test: m
Bug: 184567830
Change-Id: Idbbf9de66c5182784d055e1cd64bcef34a7dbd79
2021-04-07 16:26:55 -04:00
Paul Duffin
f58fd9aec1 Add deapexer support for apex_set
Bug: 181267622
Test: m droid
Change-Id: I630f9f2c33d037bf4a7011d2131a7d28f213ae8b
2021-04-07 20:39:13 +01:00
Paul Duffin
2470467d42 Separate apex extraction from the ApexSet
This follows the same pattern as is used by the prebuilt_apex in order
to make files encapsulated with a prebuilt .apex file available for use
by the rest of the build. It separates the extraction of the
appropriate apex from the zip supplied to apex_set into a separate
module type whose output is used by the apex_set and will (in a follow
up change) be used by the deapexer module too.

Bug: 181267622
Test: m droid
Change-Id: Icd13ce1f21845494e875d3e092abd4a9d2efca57
2021-04-07 20:36:28 +01:00
Paul Duffin
dfd3326b65 Move deapexer related functionality to prebuiltCommon
Refactoring in preparation for adding deapexer support to apex_set.

Bug: 181267622
Test: m droid
Change-Id: I8e7661ff4cfbe68f62e14d3bdf20d9356934e351
2021-04-07 20:35:35 +01:00
Paul Duffin
11216db1d3 Separate apex selection from apex.Prebuilt
This dedups the apex selection (which was previously being done by both
the apex.Prebuilt module type and the Deapexer module type) by
separating it out into its own module.

The apex.Prebuilt module retrieves the selected apex from its
selected_apex property which is simply initialized with
":<selector-module-name>".

The Deapexer module retrieves the selected apex it should use from its
src property which is initialized in the same way. That makes it easy
for the Deapexer module to be reused by the apex_set in a similar way.

Bug: 181267622
Test: m droid
Change-Id: I90cfb55698d35a97dcf28b95afcb1f58584bc90c
2021-04-07 10:11:10 +01:00