While enabling prebuilts in T we hit b/229932396 which was caused by
some parts of the build depending on the prebuilt updatable-media jar
which used to be a full implementation jar but which is now an invalid
jar as the snapshot must not be including implementation details. We
fixed the issue in T but we are hitting the same problem in S with the
M-2022-07.
That is the first train in which the prebuilt updatable-media module
provides an invalid jar, prior to that it was always providing an
implementation jar. This change tweaks the sdk snapshot generation
code to use an implementation jar for updatable-media in the S
snapshot to avoid partners having to cherry pick changes similar to
those needed to fix b/229932396 in T.
Bug: 239121291
Test: packages/modules/common/build/mainline_modules_sdks.sh
# Check that S media snapshot includes implementation jar.
# Check that S art snapshot includes invalid jar.
# Check that T media snapshot includes invalid jar.
Change-Id: Ib49484d00a60b4ed7f8268e04f9c10a3498edb56
Specifying an apex in the apexes propety will cause all the
*classpath_fragments that are contents of the APEX to be automatically
added as members of the sdk and appear in the snapshot.
The purpose of this change is to dedup the APEX and sdk definitions and
try and avoid some of the issues that we have been finding while
attempting to build against the prebuilts.
Two tests, one each for bootclasspath_fragment and
systemserverclasspath_fragment, have been refactored to compare the
output when adding the *fragment to the sdk directly of via the APEX.
That ensures switching to use the APEX will not change the sdk snapshot
unless it was previously missing a *fragment.
There was also a slight difference in where the hidden API flags were
copied from. That should have no impact on the output as the flags are
identical.
The sdk snapshot generation needed some tweaks to avoid generating a
prebuilt for the APEX.
Bug: 232401814
Test: m nothing
Change-Id: I7aaf16a3a0ab4bebf97765d1484215cc008dc4b8
Previously, when targeting the S release the generated sdk snapshot
would contain prebuilt_systemserverclasspath_fragment modules even
though they were only added in T.
This allows SdkMemberTypes to specify the set of target build releases
they support and ignores them when targeting an unsupported target
build release.
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
# Check that the for-S-build snapshots do not include SSCPFs.
Bug: 237718221
Change-Id: I2df08c2fcebf9b866695d691572a9d3783758b17
Currently, if the same library is specified for multiple of native_libs,
native_shared_libs, and native_static_libs for different arch/oses,
there can be a few errors:
1. specifying a .so file as `srcs` within a cc_prebuilt_library rather
than being specified only for shared
2. the final type of prebuilt library is dependent on the arch/os
This change introduces:
* an ability for a member type to override the type for specified
properties
* checks for a library being used with incompatible member types
* basing linkage nesting on the member type in addition to variants
This will ensure that the correct library type is used, regardless of
the order of iteration over oses/arches, and support nesting linkages
where necessary but only one linkage variant exists.
Test: soong tests
Test: CI
Change-Id: I81dee013b09b99c34ca6c18f9cfcc12ee56d33d1
Previously the SDK info file only contained basic common information
about each member. This change adds support for each member to
provide custom information to add to the info file.
It uses that mechanism to add the following:
* "dist_stem"
* "scopes" object containing:
* for each scope a:
"<scope>" object containing:
* "current_api" - the path within the snapshot for the API's .txt
file.
* "removed_api" - the path within the snapshot for the removed
API's .txt file.
* "latest_api" - the path within the build to the latest finalized
API .txt file.
* "latest_removed_api" - the path within the build to the latest
finalized removed API .txt file.
In order to access the latest API files it was necessary to add and
resolve dependencies on the module that makes them available. In order
to do that safely the code for creating the names of the modules was
refactored to avoid duplicating the name creation logic.
Bug: 204763318
Test: m nothing
Change-Id: Ica68abbd2b2c7c2b2b7877b502f96cc89f06fd68
In preparation for a refactoring that will dedup the registry code.
Bug: 195754365
Test: m nothing
Change-Id: I93e8485d588f5b4d6b5e2967da5dbef7b6c831ad
Currently, every sdk member of a specific module type has to be treated
in the same way as every other sdk member of that type. e.g. it is not
possible for an sdk member to use different variants to other members
of the same type.
Adding a new member type for each different way to treat the members is
not scalable as if there were N different ways treat a member then it
would require 2^N types for all the possible combinations.
This adds a new traits mechanism that allows the behavior of member
types to be customized per sdk member. Each member type can specify a
list of supported traits and customize its behavior based on which
traits are required for each member. A trait can be supported by
multiple different member types.
Bug: 195754365
Test: m nothing
Change-Id: I165ac80d208c0402d2a9ffa8085bba29562c19b7
Makes it adhere to the go standard practice of prefixing documentation
comments with the name of the type/func/method.
Bug: 195754365
Test: m nothing
Change-Id: Idf3fe827edc9b6d67d12a99a4b27539ac938ea95
Previously, some of the sdk member type related types and variables
included the words "member type" whereas others only used the word
"member" and did not include "type". This changes makes them all
include the word "type" to differentiate them from other types related
to sdk members.
Bug: 195754365
Test: m nothing
Change-Id: I913daaa1d436fd5dc857c2af0ffdee6471dcc6ed
Replaces the BottomUpMutatorContext parameter with a new
SdkDependencyContext type that extends BottomUpMutatorContext. This is
to allow the sdk to pass additional information to the implementations
of that method to allow the behavior to be more finely tuned.
Bug: 195754365
Test: m nothing
Change-Id: I69c6d2c523934eb67d7a7e6c55c241e9b8a81773
This reverts commit 22ff0aaf51.
The workaround to support hidden API generation with prebuilts by
passing the full implementation jars through to the SDK snapshot is no
longer needed as the monolithic hidden API flag generation uses the
prebuilt flag files that are already part of the snapshot.
This change reverts that previous workaround.
Bug: 192868581
Test: - Update the prebuilts
DIST_DIR=$PWD/dist TARGET_BUILD_VARIANT=userdebug art/build/build-art-module.sh --skip-apex
packages/modules/ArtPrebuilt/update-art-module-prebuilts.py --local-dist=dist --skip-cls --skip-apex
- Build hidden API flags to make sure it does not fail
Change-Id: Idb3fde6f7dcb171677316d8794a4af91ede1f7e0
Previously, hidden API processing could only be done by those
bootclasspath_fragment modules that either did not depend on any other
fragments (e.g. art-bootclasspath-fragment) or only depended on APIs
provided by other fragments (e.g. i18n-bootclasspath-fragment). That
meant that modules like com.android.os.statsd-bootclasspath-fragment
that depended on APIs provided by parts of the platform which are not
yet part of another bootclasspath_fragment could not perform hidden
API processing.
This change adds support for a bootclasspath_fragment to specify the
additional stubs needed to perform hidden API processing. It adds a new
additional_stubs property that can be used to specify the additional
stub libraries.
Most bootclasspath_fragments that need to use the property will need
access to the APIs provided by the android-non-updatable.* libraries.
Rather than have each fragment explicitly specify the correct module
for each scope it treats "android-non-updatable" as if it was a
java_sdk_library that can provide different jars for each scope.
Soong will handle mapping that to the correct android-non-updatable.*
module.
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv \
out/soong/hiddenapi/hiddenapi-index.csv \
out/soong/hiddenapi/hiddenapi-stub-flags.txt \
out/soong/hiddenapi/hiddenapi-unsupported.csv
- make sure that this change does not change the contents.
m TARGET_BUILD_APPS=Calendar nothing
Change-Id: Ia8b79830ed0e6d42100de03d76b0c51b7f6c8ade
Previously, an sdk snapshot could contain the following:
* A java_sdk_library_import module, e.g. "foo" which creates component
modules "foo.stubs", etc.
* A corresponding versioned module, e.g. "sdk_foo@current" which
created component modules "sdk_foo@current.stubs", etc.
* An internal (to the sdk snapshot) java_import for one of "foo"'s
components, e.g. "sdk_foo.stubs"
* A corresponding versioned module, e.g. "sdk_foo.stubs@current".
That causes a few problems:
1. The "foo.stubs" is duplicated.
2. The names of the components created by the versioned
java_sdk_library_import are invalid, as they append the component's
suffix to the version and not the name before the version.
The latter causes problems when building against prebuilts and fixing
that causes the generated snapshot to be invalid because it contains
duplicate definitions of the "sdk_foo.stubs@current" module. One
explicitly in the Android.bp file and one created by the
"sdk_foo@current" module.
Removing the duplicates from the snapshot causes errors as the name
generated by the snapshot for the component module, i.e.
"sdk_foo.stubs@current" does not match the name generated by the
"sdk_foo@current", i.e. "sdk_foo@current.stubs".
This change fixes them together.
Bug: 179354495
Test: m nothing
Merged-In: I515f235fe21755b5275af12366e96c24c94c0273
Change-Id: I515f235fe21755b5275af12366e96c24c94c0273
(cherry picked from commit a1aa7387f7)
Adds support for printing a PropertySet that has a property whose value
is an array of structs.
Bug: 179354495
Test: m nothing
m conscrypt-module-sdk
- check generated Android.bp file
Change-Id: I71be04188465610bcbea4d3c9a5e8204171a1eeb
Previously, only those SdkMemberTypes which had specific need to
automatically add some of their dependencies as sdk members would cause
the sdk to visit their transitive dependencies. However, as any module
can have dependencies on license modules and license modules need to be
included in the sdk then it needs to visit transitive dependencies of
all members.
So, this change removes the support for allowing an SdkMemberType to
control whether its transitive dependencies are visited and just visits
them all.
This does not have any effect on sdk snapshots as in order for a
dependency to be added to an sdk it needs to be added with a tag that
implements SdkMemberTypeDependencyTag and the only tags that implement
this are used by SdkMemberTypes that had enabled transitive members.
Bug: 181569894
Test: m art-module-sdk art-module-host-exports art-module-test-exports
- verify that this change has no effect on the generated snapshots
Change-Id: If0293af0237aa7e39335e5b8383a41c023ff5853
Making licenseModule SdkAware caused two breakages in the build. The
breakages were both caused by having an SdkAware module that was
depended upon by a versioned sdk snapshot but which was not itself
versioned and so did not have the member_name property set.
That occured because some default licenses have been added to the
packages containing prebuilts, e.g. prebuilts_runtime_license in
prebuilts/runtime/Android.bp. They apply to both the versioned and
unversioned members.
Once license support has been added to the sdk most of those will be
removed and replaced with properly versioned license modules. However,
in the meantime it is necessary to support that.
This change avoids the issue by checking to see whether the module is
itself versioned before relying on the member_name property. It also
improves the error message when a panic is recovered to make it easier
to identify where it originates.
Bug: 181569894
Test: m nothing
Change-Id: I0e7da2e0c4a30a6f814c2faab821b185aaed2135
The tests use <sdk>_<module>_<version> as the format for a versioned
sdk member name but the format should be <sdk>_<module>@<version>. This
change corrects it and also fixes a similar issue in an error message.
Bug: 181569894
Test: m nothing
Change-Id: I8be0db4bcd0b6f4d6fbdf9e402ef7257fae8e18b
This change allows an SdkMemberTypeDependencyTag to select the
SdkMemberType to use to add a dependency based on the module.
Bug: 177892522
Test: m nothing
Change-Id: I2d6e51b615636dc7cd41da65808ef851dd8ae6ac
Previously, every module added to an sdk directly through one of the
SdkMemberType specific properties, e.g. java_libs, was exported and
every module added automatically via a transitive dependencies was not
exported. This change allows that behavior to be customized per tag.
Bug: 186290299
Test: m art-module-sdk
- verify that this change does not affect its contents.
Change-Id: I563b5bcd823e61c23cdb706cfcbb13337963d550
Hiddenapi processing currently requires access to the class
implementation jars for libraries on the bootclasspath which means that
they need to be provided as part of the prebuilts. This change modifies
the java_boot_libs property on the sdk to make those files available.
Modularization of the hiddenapi processing will hopefully remove the
need for these to be exported so this should be temporary.
Bug: 178361284
Test: m art-module-sdk
check generated snapshot zip contains implementation jars
Change-Id: I9e94662dddb0ddb85a477ae6d27e533085147e88
The build has some implicit dependencies (via the boot jars
configuration) on a number of modules, e.g. core-oj, apache-xml, that
are part of the java boot class path and which are provided by mainline
modules (e.g. art, conscrypt, runtime-i18n) but which are not otherwise
used outside those mainline modules.
As they are not needed outside the mainline modules adding them to
the sdk/module-exports as either java_libs, or java_header_libs would
end up exporting more information than was strictly necessary. This
change adds the java_boot_libs property to allow those modules to be
exported as part of the sdk/module_exports without exposing any
unnecessary information.
Some points to note:
* The java_import has to have a valid file for the src property
otherwise it will be disabled.
* The src property is supposed to reference a jar file but the
java_boot_libs property will make it reference an empty file (not
an empty jar) so that any attempt to use that file as a jar, e.g.
compiling against it, will cause a build failure.
* The name of the file passed to the src property should make it
clear that the file is not intended to be used.
* The test makes sure that only the jar file is copied to the
snapshot.
Test: m nothing
Bug: 171061220
Change-Id: I175331e4c8e3874ab70a67cdc2f76ed1576e41eb
Both will create a nested property set, that may be merged with an
existing one.
Test: m nothing
Bug: 151303681
Change-Id: I30696ba3eb8960ca6fa54c9ee2cf6229ab9f5da9
Necessary to avoid problems with implicitly enabled host OS'es, e.g.
linux_glibc getting enabled when we only supply a linux_bionic linker
in runtime-module-host-exports. That will then cause a non-functional
prebuilt to take precedence over source if the prebuilt is preferred.
We don't do this for device since we so far only support a single
device OS (android).
This introduces the notion that SDK member types can be host OS
dependent or not. That way java members with host prebuilts don't get
restricted to a specific host OS.
Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh runtime-module-host-exports
Check that the generated Android.bp correctly disables the bionic
linker prebuilt for linux_glibc.
Test: art/build/apex/runtests.sh
on master-art with an updated runtime SDK snapshot
Test: art/tools/buildbot-build.sh {--host,--target}
on master-art with an updated runtime SDK snapshot
Bug: 160349757
Change-Id: Idad7ef138cdbcbd209d390bf6c10ca8365d4619f
Previously, it was not possible to navigate directly from the interface
methods in SdkAware to the implementations in SdkBase as the former
embeds android.Module but the latter did not implement those methods
and so did not implement SdkAware. Finding the implementation of those
methods required explicitly searching for them.
This change extracts sdkAwareWithoutModule interface to allow use of
the IDE capability to navigate directly to implementations of an
interface method.
Test: m nothing
Change-Id: I0cd25180ea0081a54681b58e3202ff9491563131
There was a special case in cc_prebuilt_binary only, which resulted in
prebuilt libraries getting both 32 and 64 bit variants even when their
sources only had one of them, and the other variant would be defunct
since it wouldn't get any prebuilt artefact.
This moves the handling of compile_multilib completely to the common
update code, so that SDK members don't need to deal with it. It doesn't
take SDK member defaults into account, which means a bit more
boilerplate in the snapshots, but it's simpler and less error prone
(different SDK member types have different defaults).
This relands https://r.android.com/1359962 after disabling the SDK
tests on mac once and for all.
Bug: 151303681
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I05f6603b2ac9b8676b25c3e297165ca23284e9cc
There was a special case in cc_prebuilt_binary only, which resulted in
prebuilt libraries getting both 32 and 64 bit variants even when their
sources only had one of them, and the other variant would be defunct
since it wouldn't get any prebuilt artefact.
This moves the handling of compile_multilib completely to the common
update code, so that SDK members don't need to deal with it. It doesn't
take SDK member defaults into account, which means a bit more
boilerplate in the snapshots, but it's simpler and less error prone
(different SDK member types have different defaults).
Bug: 151303681
Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh art-module-host-exports
Check that the generated Android.bp passes Soong.
Change-Id: Ib73444c6788ee1c78480bdb103aa2b8ae8f2c63c
Previously, preferring a prebuilt of an sdk/module_exports's member
would cause the sdk/module_exports to depend on the prebuilt instead
of the source and cause problems with the build.
This chance prevents the dependency from an sdk/module_exports to its
members from being replaced with prebuilts.
Bug: 160785918
Test: m nothing
Change-Id: Iee4bcd438c11929e30fb5766701b05a0e89956d9
Adds a filter mechanism that can exclude property values from being
included in the common value extraction. That is needed to prevent the
snapshot mechanism from generating invalid output for properties that
are ignored on host (and have their values cleared) and which are not
tagged with `android:"arch_variant"`.
Changes:
* Updates the documentation of SdkMemberType to explain what effect
the 'ignored-on-host' tag has.
* Adds some tests for this new mechanism.
Bug: 155628860
Test: m nothing
Change-Id: Ibafdb6e921ba5abe505bd8a91ca5a1d9c9b5d0cb
Previously, the snapshot code did not know whether a specific property
could be arch specific or not and assumed that they all were which
meant that it could generate snapshots containing arch specific values
for properties that are not arch specific and so would fail when
unpacked.
This change requires arch specific fields in SdkMemberProperties to be
tagged as such using `android:"arch_variant"` (just as in module input
property structures). Any property without that must have properties
that are common across all variants.
Bug: 155628860
Test: m nothing
Change-Id: I3df60f0b53ba02ec2c55a80c7da058eac5909d26
Revert submission 1302576
Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...
Change-Id: Id7eba0bdde5c579e10e9b42d94a7cfab5f34995f
Revert submission 1302576
Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...
Change-Id: I2a7ac0ef0232177eefc26542c11dc675d6f4cab2
Adds a filter mechanism that can exclude property values from being
included in the common value extraction. That is needed to prevent the
snapshot mechanism from generating invalid output for properties that
are ignored on host (and have their values cleared) and which are not
tagged with `android:"arch_variant"`.
Changes:
* Updates the documentation of SdkMemberType to explain what effect
the 'ignored-on-host' tag has.
* Adds some tests for this new mechanism.
Bug: 155628860
Test: m nothing
Change-Id: I7ebd333079619dba546bc8c4911d567e0287b676
Previously, the snapshot code did not know whether a specific property
could be arch specific or not and assumed that they all were which
meant that it could generate snapshots containing arch specific values
for properties that are not arch specific and so would fail when
unpacked.
This change requires arch specific fields in SdkMemberProperties to be
tagged as such using `android:"arch_variant"` (just as in module input
property structures). Any property without that must have properties
that are common across all variants.
Bug: 155628860
Test: m nothing
Change-Id: Ifc8116e11d987cfe7aec2eeaa964f3bbf36b5dc2
The DepIsInSameApex() and RequiredSdks() methods were defined in a few
places to avoid having to depend on the whole ApexModule/SdkAware
interfaces directly. However, that has a couple of issues:
1) It duplicates functionality making it difficult to change, changes
to the definitions outside the main interfaces do not cause compile
time failures, instead they result in a runtime change in behavior
which can be difficult to debug.
2) IDE navigation (specifically in Intellij) does not detect that the
duplicate definitions can resolve to the definitions in the main
interface.
This change extracts the methods into their own interfaces and reuses
those interfaces instead of duplicating the methods to fix both of
these issues.
Bug: 152878661
Test: m nothing
Change-Id: I0cfdf342a14eb0bfb82b1bd17e0633d81c7facfb
Migrates system modules and droid stubs over to use the new API for
creating the snapshot modules and removes the old API.
Test: m nothing
Change-Id: Ia825767f1f7ee77f68cfe00f53e09e6f6bfa027f
Previously, java snapshot properties (java_library and java_test)
relied on the properties not being optimized when there was a single os
type and instead being added directly to the common os type properties.
However, that means that the behavior is inconsistent for other member
types depending on whether there was one os type or not.
This change updates the java sdk member handling to support
optimization. This involved:
1) Adding AidlIncludeDirs field to librarySdkMemberProperties to
specify the aidl include dirs instead of extracting that from the
library field.
2) Renaming jarToExport to JarToExport (in both
library/testSdkMemberProperties)to allow it to be optimized.
3) Adding MemberType() and Name() methods to SdkMemberPropertiesContext
to avoid having to store the former in the properties struct and
retrieve the latter from the library/test fields.
4) Removing the now unused library/test fields from the properties
structures.
5) Separating the processing of the jar/test config in
AddToPropertySet(...) as they may be optimized separately.
6) Ditto for the jar/aidl include dirs.
7) While doing this work I noticed that although the contents of the
aidl include dirs are copied into the snapshot the java_import does
not make use of them. Raised bug 151933053 and added TODO to track
that work.
Bug: 142935992
Test: m nothing
Change-Id: Iba9799e111ca5672b2133568163d8c49837ba9cd
Previously passing additional information to the implementations of
AddPrebuiltModule() or the SdkMemberProperties interface would have
required making changes to the API. This change added an
SdkMemberContext object into which additional information can easily
be added without requiring changes to existing implementations.
The BuildSnapshot() method was not modified because it is deprecated
and will be removed in a follow up change.
It also switches the API from passing variants as android.SdkAware to
android.Module. That is for a couple of reasons:
1) SdkAware is designed for managing the relationship between the
module and the SDK, not for generating the output snapshot. As such
there is nothing in SdkAware that is needed for generating the
output snapshot.
2) Accepting android.Module instead makes it easier to use the
underlying code for generating the snapshot module as well as the
individual member modules.
This is in preparation for a number of improvements and bug fixes in
both the snapshot creation code and implementations to address found
while trying to built the platform against ART prebuilts.
Bug: 151937654
Test: m nothing
Change-Id: Iac10f1200c0f283aa35402167eec8f9aeb65a38e
