* changes:
Copy exportable artifacts to module sdk snapshot
Enable hiddenapi check for exportable stubs
Add defaults support for bootclasspath_fragment module type
This change modifies the copy rules of the sdk_library when generating
the module sdk snapshots so that the exportable artifacts (annotations
zip, api file, removed api file, srcjar file) are copied instead of the
everything artifacts.
In order to satisfy the prebuilts compatibility, this feature is flag
guarded by the build flag "RELEASE_HIDDEN_API_EXPORTABLE_STUBS"
Test: m --no-skip-soong-tests
Bug: 315027929
Change-Id: I8bf5bb1f196aa9b5db78ba4677caa5f4337e3021
This change modifies the dependencies of the hiddenapi to always depend
on the exportable stubs, instead of the currently utilized everything
stubs.
To support this, the full api surface exportable stubs are defined in a
separate change at the `frameworks/base` project. Note that the full api
surface exportable stubs are only used for the hiddenapi purpose, and
`sdk_version` continues to utilize the currently existing everything
stubs.
Currently, this feature is hidden behind the build flag
"RELEASE_HIDDEN_API_EXPORTABLE_STUBS". This feature will be fully
enabled once metalava fully supports handling of the flagged apis.
Test: ENABLE_HIDDENAPI_FLAGS=true m
Bug: 317426356
Change-Id: I109b7cd27b20ceffcdf1766ab8106b0c276be2b3
Every module belonging to a single mainline module family will be
hidden from make, except the one which has been flagged using
apex_contributions
Details
- Introduce a new `source_apex_name` property to prebuilt_apex and
override_apex. This property will be used to identify the source
equivalent of a prebuilt soong apex module.
- Create an N-ary tree from source to prebuilt(s). The tree wil be
rooted at the source module.
- In a subsequent mutator, visit every node in the tree(s). Query
apex_contributions and store the handle of the node which is "active"
(if any)
- In the same mutator, do another pass over the tree. Invoke
`HideFromMake` on every node which is not "active". The two-pass
approach is needed PrebuiltSelectionInfoProvider does not know about
the inter source-prebuilt dependency, this dependency can only be
known by doing a graph walk of the N-ary tree.
Some tangential implementation details
- Each prebuilt apex has an internal deapxer module that is responsible
for generating the deapex ninja rules. The name of this internal
module uses the BaseModuleName (without the prebuilt_ prefix). Since
we can have multiple prebuilt soong modules in trunk stable, change
this to follow the name of the prebuilt module in order to avoid name
collisions. Update existing unit tests accordingly
Bug: 316179314
Test: go test ./apex -run TestInstallationRulesForMultipleApexPrebuilts
Test: m nothing --no-skip-soong-tests
Test: presubmits
Change-Id: I58aa99d5e6a9859954614e6db9a8e9e2e581642d
This change generates rules for "exportable" stubs in the droidstubs
module.
Given that there are a lot of overlap between the currently existing
"everything" stubs rule and the newly introducing "exportable" stubs
rule, the currently existing metalava rule commands are modularized to
be utilized in the "exportable" stubs rule commands.
The currently existing build actions are modularized in the followings:
- commonMetalavaStubCmd(...): metalava commands that are required for
generation of both "everything", "exportable", and potentially
"runtime" stubs
- everythingOptionalCmd(...): metalava commands that are dependent on
"everything" stubs and not dependent on flagged apis annotations, such
as api lint, released api check
Based on this modularization, the "everything" stubs are now generated
in everythingStubCmd(...), which calls commonMetalavaStubCmd(...) and
everythingOptionalCmd(...).
Similarly, the "exportable" stubs are generated in
optionalStubCmd(stubsType=Exportable, ...), which calls
commonMetalavaStubCmd(...) and appends additional flags. Runtime stubs
can be generated similarly in the future with
optionalStubCmd(stubsType=Runtime, ...).
"everything"-related artifacts will now be created in
`everything/` subdirectory, and "exportable"-related artifacts will be
created in `exportable/` subdirectory. For example, the outdir of a
droidstubs module "foo" would look like below:
```
foo
|-- everything
| |-- foo_api.txt
| |-- foo-stubs.srcjar
|
|-- exportable
|-- foo_api.txt
|-- foo-stubs.srcjar
```
The module generates the build rules for the "exportable" stubs
regardless of whether the module defines the `aconfig_declarations`
property or not. All APIs marked with `@FlaggedApis` annotations are
stripped out for the "exportable" stubs when the `aconfig_declarations`
property is not defined. On the other hand, only the `@FlaggedApis` that
are specified in the aconfig_declarations module and are enabled will be
included (and all others are stripped) when the `aconfig_declarations`
propety is defined.
Test: go test ./java && BUILD_FROM_SOURCE_STUBS=true m
Bug: 315490657
Change-Id: I300273cd2a62fa978b046c0268e3a67c35e22b08
Convert all of the callers of OtherModuleProvider/OtherModuleHasProvider
to use the type-safe android.OtherModuleProvider API.
Bug: 316410648
Test: builds
Change-Id: Id77f514d68761a262d9ea830a601dbed804bbbe5
Some users want to repackage the results of an sdk module.
Genrules have 3 variants: linux, android, and common_os. The common_os
one produces a snapshot zip file that users want. In order to get
access to it, we need a genrule in the same variant, so create
an sdk_genrule for that.
The sdk_genrule shouldn't have linux/android variants either, because
those other variants would get errors when trying to depend on the
sdk modules because the snapshot zip doesn't exist in those other
variants. The code in arch.go needs to be tweaked to allow a common_os
variant without the other variants.
Bug: 315962165
Test: m dist out/dist/art_release.zip
Change-Id: Idc9b3cae7a525d71aed6bafa0f8724a89f75a94b
Add a test for that checks the SDK snapshot contains sdk library
definitions for modules with alowed minSdkVersion, but that doesn't
contain definitions for modules with disallowed minSdkVersions.
Bug: 289183551
Test: go test ./sdk
Change-Id: I194526f8eba4dacb52a345fa67cf7371e722c237
If a header jar couldn't be built (for example when an API generating
annoation processor is in use) the implementation jar is reused as the
header jar. If the implementation jar contains an annotation processor
listed in META-INF/services/javax.annotation.processing.Processor then
later javac executions with the implementation jar in the classpath
could attempt to run the annotation processors unexpectedly. Remove
the META-INF/services directory when using an implementation jar as
a header jar.
Bug: 290933559
Test: builds
Change-Id: I40d48644bc5a09a9564dc2c4b38f627edd00fcf8
In the past, dexpreopt for boot jars was very inflexible, and it was
incredibly hard to make a change that is as simple as adding a jar to a
boot image. Boot image generation was handled by
"platform_bootclasspath" and "bootclasspath_fragment" separately. This
caused not only code duplication but also the inflexiblity as such a
design did not fit today's use cases, where a boot image may take jars
from multiple mainline modules and the platform, and a mainline module
can contribute to multiple boot images. The design casued a huge
maintenance burden as any change to the boot image cost multi-week
efforts.
In recent years, efforts have been made to improve this a bit by a bit.
This change is another step towards making dexpreopt reasonable.
After this change, all boot images are generated by "dex_bootjars",
which is in build/soong and is therefore available on both the full
source tree and the thin manifest (master-art). The change decouples
profile generation/extraction from boot image generation. Profiles for
mainline modules are still handled by "bootclasspath_fragment"
because they need to be packed into APEXes when building mainline
modules and extracted from APEXes whem building the system image from
prebuilt modules. Boot images are not handled by
"bootclasspath_fragment" anymore.
Bug: 290583827
Test: m (all existing tests are still passing)
Test: Manually checked that the boot images are exactly the same as
before.
Change-Id: Ib5a5f401bee334ffcab5c26618e0c8888b84575a
* changes:
Change the profile path on host.
Extract duplicate code to common helper functions.
Fix dumpOatRules.
Remove Modules() from BootclasspathFragmentApexContentInfo.
Fix some tests for dexpreopt and remove unnecessary tests.
This CL is to prepare for the changes in http://r.android.com/2652081.
See the description of that CL for the reasons.
Some tests were set up in a fragile way and were easy to be broken when
the implementation changes. This CL is a pure test change that fixes
them.
This CL also removes TestNoUpdatableJarsInBootImage. That test checks
what should go to the ART boot image and what should go to the
platform one, which does not apply today because boot images are not
associated with mainline modules anymore: a boot image may take jars
from multiple mainline modules and the platform, and a mainline module
can contribute to multiple boot images. In practice, we have ART jars in
the platform boot image, and we are going to add core-icu4j and
consrypt to the ART boot image, which is now for testing only.
Bug: 290583827
Test: m nothing
Change-Id: I22c45cbf6f853b030b68edb51197854e9c53a02e
If a systemserverclasspath_fragment only contains libraries that have a
higher min_sdk_version than the target build release version, then we
should not export the systemserverclasspath_fragment. Before this
change, the fragment was exported with an empty `contents` property
which caused errors after being dropped as a prebuilt.
Bug: 289183551
Test: go test ./sdk
Change-Id: Ifefc6880228e4dd37f5e42b2bda31a83df785375
Rename java_library created inside sdk_library with the ".from-source"
suffix, and set it as static lib of the top level java_library, which
gets java_api_library instead as static lib during from-text stub build.
Test: m nothing && m nothing --build-from-text-stub
Bug: 286446015
Change-Id: I32e8ea264987e9f9df05e462292bd54e45074912
After this change, there is a clear separation between tests that are
related to dexpreopt and tests that are not. The former uses
PrepareForTestWithDexpreopt, while the latter uses
PrepareForTestWithJavaDefaultModules. The benefit is that the latter
will no longer affected by any dexpreopt changes.
Bug: 280776428
Test: m nothing
Change-Id: Ib957765b9287d51c082e0a33cee17a6bb56daeef
Instead of exporting it to soong_injection.
Bug: 279095899
Test: m nothing
Change-Id: I7b93af233b7450848a475512b5f5682ece773c09
Merged-In: I7b93af233b7450848a475512b5f5682ece773c09
This relands aosp/2465355. The original CL was submitted as part of a
stack which broke tm and udc. Those branches contain soong modules with
min_sdk_version of type (kind+level)
Bug: 208456999
Test: go test ./sdk (top of CL stack)
Change-Id: I627f69b1fd71ab386e9026a2fc1566bad786bf1d
This relands aosp/2478277. The original CL was submitted as part of a
stack that broke tm and udc since those branches contains soong modules
that have min_sdk_version of type (kind+level).
Test: go test ./sdk
Bug: 208456999
Change-Id: Ie5d7f2597092a51c6e74e7b3cab2f81c02906d5e
This is needed to determine whether we need to deapex the profile or
not.
Bug: 241823638
Test: m nothing
Test: -
1. Patch ag/20582608 PS2.
2. TARGET_BUILD_APPS=com.android.wifi packages/modules/common/build/mainline_modules_sdks.sh
3. See the `dex_preopt` property being written.
Change-Id: I05177388c24be1ab315247ea85e7e3a8600cebd3
Previously, the memberMutator was responsible for adding dependencies
from the sdk to its members. It could not just implement DepsMutator()
as it had to be run before the memberDepsMutator which itself had to be
run before the depsMutator.
As the memberDepsMutator has been removed by a previous change this
change removes the memberMutator and just performs that functionality
in the standard DepsMutator(BottomMutatorContext) method.
Bug: 260237150
Test: m nothing
BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Run the above before and after this change and compare the
# out/dist/*mainline-sdks directory contents to ensure that there are no
# changes.
Change-Id: I264bdeee84f2349e30e5c24dd4861e945640d020
Previously, the memberDepsMutator was responsible for tracking the
membership of modules in an sdk/module_exports (or corresponding
snapshot). That was only used to determine whether a module was in a
versioned sdk snapshot so that it could be treated differently by the
build. As there are no versioned sdk snapshots these can now be
removed.
This change removes the memberDepsMutator, along with SdkAware and
related types that are no longer used.
Bug: 260237150
Test: m nothing
Change-Id: I2668679a42d5b3f33f9e06d5b59804dfdae5f6da
Previously, the sdk snapshot code used SdkAware instead of Module to
ensure that all its members had implemented SdkAware. However, it never
used any of the methods provided by SdkAware, or if it did it no longer
does. So, this change replaces usages of SdkAware with Module in
preparation for deleting it completely.
Bug: 260237150
Test: m nothing
Change-Id: Ia89e02394f27b2da776f0cf0f0bc86835a03433a
In preparation for removing support for versioned snapshots from Soong
this change rewrites TestBasicSdkWithBootclasspathFragment to use
non-versioned prebuilts making it more realistic in the process.
Bug: 260237150
Test: m nothing
Change-Id: I99d1ad9b480657bb805c3d48e99e715052f4d00a
The min_sdk_version was added to the java_import in Tiramisu. This
change will propagate the min_sdk_version set on a java_library to the
java_import snapshot. If possible the min_sdk_version will be resolved
into a numerical version to ensure consistent behavior across build
releases.
Bug: 260560424
Test: m nothing
BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Ran above before and after this change and made sure that only
# Tiramisu, UpsideDownCake and latest were changed and those changes
# were the addition of a min_sdk_version property.
#
# Ran the following command in tm-mainline-prod:
# lunch cf_x86_64_phone-userdebug
# m ART_MODULE_BUILD_FROM_SOURCE=false nothing`.
#
# It failed with:
# module "prebuilt_okhttp-norepackage" variant "android_common": should support min_sdk_version(33) for "AdServicesApk": min_sdk_version is not specified.
#
# Unpacked the Tiramisu art snapshot generated with this change into
# tm-mainline-prod and reran the previous command and it succeeded.
Change-Id: I9d9d730845554fc175d17f38c038e4e3c7d39e07
The following environment variables are no longer used when generating
sdk snapshots:
* SOONG_SDK_SNAPSHOT_PREFER - we can control the modules that are
preferred at build time now.
* SOONG_SDK_SNAPSHOT_USE_SOURCE_CONFIG_VAR - this is done as a
post-processing step by the mainline_module_sdks.sh script to avoid
having to regenerate the ninja files for every module.
* SOONG_SDK_SNAPSHOT_VERSION - support for this was removed a long time
ago but was still referenced from dumpvars.go.
Bug: 259095197
Test: m nothing
# Searched for uses of these variables but could not find any
# outside the build.
Change-Id: Id2b32b29ff6f09faffea43292f549ba13acd956d
The sdk snapshot must not be including implementation code for boot
libraries, the implementation is provided by dex jars within the
corresponding APEX. However, the snapshot does need a module for each
boot library so that the build can seamlessly access the dex files from
the APEX.
A java_library boot library (like core-oj) is represented in the
snapshot by a java_import module which requires a jar file to be
provided, otherwise it is disabled. However, that is provided purely
to keep Soong happy and should never be used.
Previously, the snapshot would contain an empty file for the jar. As
an empty file is an invalid jar any tool (like compiler) that tried
to consume it would fail which was the correct behavior. Unfortunately,
the error message that was produced was not very helpful, it was just
some variant on `invalid file` which lead to a lot of bugs being
raised.
This change replaces that empty file with a reference to the output
from a genrule which runs a script which produces a more useful error
message, with information on how to fix the issue, and fails the build.
It also adds a Name() method to the SdkMemberProperties type as that is
needed in AddInternalModule() to construct the name of the additional
module.
Tested as follows:
In AOSP/master make the following changes:
1. Temporarily set visibility on core-oj and core-libart to
//visibility:public.
2. Run packages/modules/common/build/mainline_modules_sdks.py to create
the snapshots.
For each of the S, T and latest snapshots I did the following in the
s-aml-prebuilt-test, t-aml-prebuilt-test and aosp/master branches:
1. Created an Android.bp file containing the following:
java_library {
name: "broken",
static_libs: [
"prebuilt_core-libart",
"prebuilt_core-oj",
],
}
2. Fix the visibility issues and run `m broken` where it fails with an
invalid file.
3. Delete the contents of the prebuilts/module_sdk/art/current/sdk
directory.
4. Unpack the relevant version of the art-module-sdk snapshot into the
directory.
5. Run `m broken` where it fails with the helpful message.
6. Test the instructions on how to use the ninja -t path tool to
identify the cause of the problem and fix it.
Bug: 257969510
Test: See above.
Change-Id: I125bde2d7202afff84c97daebcef37e21c548a3a
Previously, in a build containing source and prebuilt art
bootclasspath_fragments, the bootImageVariant.licenseMetadataFile was
set twice with the source always being set after the prebuilt and
so winning.
This change only sets bootImageVariant.licenseMetadataFile for the
active module so it will use the prebuilt's license file if that is
preferred.
Bug: 245956352
Test: m nothing
Change-Id: I948c7e5123169452f67c85ad98c4bbdb90a5d2de
Most of the fields in the bootImageConfig/Variant structs are assigned
inside a Once func so are guaranteed to be only set once. However, some
are assigned outside. This change adds comprehensive tests for those
structs and verifies that the constant fields are preserved and the
mutated fields have the correct value.
The check for the constant fields is added in a new TestBootImageConfig
test.
The check for the mutated fields is added into
TestSnapshotWithBootclasspathFragment_ImageName as that test checks an
art bootclasspath_fragment in the following configurations:
* source on its own
* prebuilt on its own
* source and prebuilt with source preferred
* source and prebuilt with prebuilt
It reveals a couple of interesting facts:
* All the *installs fields are set to the same value irrespective of
whether the source or prebuilt is preferred. The information is
constructed solely from information already within the
bootImageConfig/Variant and so can be moved within Once.
* The licenseMetadataFile is incorrect when prebuilt is preferred.
That is due to both the source and prebuilt modules setting it and
the source module always wins as the source module depends on the
prebuilt so always runs its GenerateAndroidBuildActions after it.
Those issues will be cleaned up in following changes.
Bug: 245956352
Test: m nothing
Change-Id: If917cfbcb3b1c842a8682d51cc1ee1fed1c51add
Bug: 249769908
Test: packages/modules/common/mainline_modules_sdks.sh
# With modification to add UpsideDownCake to list of releases
# to build with.
Change-Id: I050811e7d9a5d0a8cf54a42c03ad889defa3a251
The API finalization process for platform releases copies various files
from the apistubs/ dist directory into prebuilts/sdk/<n>/... Having
them use the same name in both places makes that process simpler. For
most modules the name of the file is derived from the name of the
module by appending a suffix but unfortunately, for some modules that
does not work. e.g. the conscrypt.txt file is produced by the
conscrypt.module.public.api module. The dist_stem property was added to
java_sdk_library to allow the stem name of the file to differ from the
module name.
The API finalization process for extension APIs does something similar
as it extracts various files from the snapshots and copies them into
the appropriate extension API specific directory in
prebuilts/sdk/extensions/<n>. Instead of copying files from the
apistubs/ dist directory (which is not built for trains) it copies them
from sdk snapshots that are built as part of the train.
Previously, the sdk snapshot used to derive the name of the files
within the snapshot from the name of the module, it ignored the
dist_stem property. This change causes it to use the dist_stem property
to make it consistent with the apistubs/ dist directory naming.
The file name is created in
sdkLibrarySdkMemberProperties.AddPropertyToSet() which does not have
access to the dist_stem property. So, it has to be supplied in the
sdkLibrarySdkMemberProperties instance in the new Stem property.
Bug: 248258460
Test: m nothing
BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh --build-release=latest
# Ran the previous command before and after and make sure that the
# conscrypt, art and icu sdk snapshots use the dist_stem value but
# none of the other snapshots are affected.
Change-Id: Ied52003de63dcdb86a252a39bb8781f85d51a6ff
Fields tagged with `sdk:"keep"` will keep the value even if it would
normally be cleared because it was common across a number of structs.
This will allow a module type to specify the same value across all
structs populated directly from variants and have it be copied into all
common property structs without clearing it from the variant specific
structs.
Bug: 248258460
Test: m nothing
Change-Id: I147d946b11fd8766a7d875d9206e8f5034f585d6
"ignore" is a better description of the effect that this tag has on the
optimization that is done as part of the sdk snapshot generation.
"keep" implies that something special is done with the field during the
optimization process but actually the optimization process completely
ignores the property, just as it does with unexported fields.
Bug: 248258460
Test: m nothing
Change-Id: Idfdfd87f927d641d4d301cbbde72b463c89155bf
When an sdk snapshot is targeted at release X then it cannot include
bootclasspath fragment libraries which are not present in that build as
otherwise it causes build failures. It should also not include any
unsupported libraries, i.e. libraries that cannot work on that release.
This change causes sdk snapshot to exclude libraries that have a
min_sdk_version > target build release
It also ensures that hidden API flags do not include any information
from excluded libraries.
Bug: 240406019
Test: BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Ran the previous command with and without this change to make
# sure that this change excludes framework-connectivity-t library from the
# tethering sdk snapshot for S, including from the hidden API flag files.
Change-Id: I57969b85a12e9e5a3fc76c055b260cec5d5f7d7f
Previously, the build releases only included named releases so did not
have a way to represent the latest build release. This adds the current
build release (named after the current API level) to represent that.
Bug: 240406019
Test: m nothing
Change-Id: Ib8336da716b447448b23bc9684ce3be1ab78648a
Previously, hidden API properties were only allowed on
bootclasspath_fragment and platform_bootclasspath module types. This
change allows them to be specified on java_sdk_library modules too. It
involves the following changes:
1. Add the properties to the java.Module.
2. Populate and provide a HiddenAPIPropertyInfo struct from
java_sdk_library modules.
3. Modify bootclasspath_fragment to merge information gathered from its
content libraries as if it was specified on the fragment itself.
Bug: 240406019
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
# Ran the previous command with and without this change to make
# sure that this change does not change the sdk snapshot
# contents.
Change-Id: I64eb71c2039ddc14cf380689d0cec7ec221f5b88
Previously, the platform_compat_config modules needed to be explicitly
listed in the sdk snapshot and the apex. This change will automatically
export them to the sdk snapshot when the apex is listed in its apexes
property.
Bug: 232401814
Test: m tethering-module-sdk
# Before this change the generated snapshot did not contain the
# platform_compat_config, after this change it did.
m art-module-sdk
# As that explicitly specifies the platform_compat_config in its
# compat_configs property this change has no effect.
Change-Id: Ia854b9a52db2b1619fca41a387ce98d7f9f9efe9
The order of the files in the sdk snapshot was stable but depended on
how the sdk was built. If the snapshot was created directly then the
files have one order, if it was created by merging in files from other
zips then it has a different order.
This change ensures files are in alphabetical order no matter how the
snapshot zip is constructed.
Bug: 232401814
Test: m media-module-sdk
# Check the contents of the following zips to ensure that they are sorted
# after this change.
# out/soong/.intermediates/packages/modules/Media/apex/media-module-sdk/common_os/media-module-sdk-current.unmerged.zip
# out/soong/mainline-sdks/media-module-sdk-current.zip
Change-Id: Ie97e0119c07a1f34a2b1d3ea6895f0e76cd195a8
