If a header jar couldn't be built (for example when an API generating
annoation processor is in use) the implementation jar is reused as the
header jar. If the implementation jar contains an annotation processor
listed in META-INF/services/javax.annotation.processing.Processor then
later javac executions with the implementation jar in the classpath
could attempt to run the annotation processors unexpectedly. Remove
the META-INF/services directory when using an implementation jar as
a header jar.
Bug: 290933559
Test: builds
Change-Id: I40d48644bc5a09a9564dc2c4b38f627edd00fcf8
Rename java_library created inside sdk_library with the ".from-source"
suffix, and set it as static lib of the top level java_library, which
gets java_api_library instead as static lib during from-text stub build.
Test: m nothing && m nothing --build-from-text-stub
Bug: 286446015
Change-Id: I32e8ea264987e9f9df05e462292bd54e45074912
This is needed to determine whether we need to deapex the profile or
not.
Bug: 241823638
Test: m nothing
Test: -
1. Patch ag/20582608 PS2.
2. TARGET_BUILD_APPS=com.android.wifi packages/modules/common/build/mainline_modules_sdks.sh
3. See the `dex_preopt` property being written.
Change-Id: I05177388c24be1ab315247ea85e7e3a8600cebd3
The min_sdk_version was added to the java_import in Tiramisu. This
change will propagate the min_sdk_version set on a java_library to the
java_import snapshot. If possible the min_sdk_version will be resolved
into a numerical version to ensure consistent behavior across build
releases.
Bug: 260560424
Test: m nothing
BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Ran above before and after this change and made sure that only
# Tiramisu, UpsideDownCake and latest were changed and those changes
# were the addition of a min_sdk_version property.
#
# Ran the following command in tm-mainline-prod:
# lunch cf_x86_64_phone-userdebug
# m ART_MODULE_BUILD_FROM_SOURCE=false nothing`.
#
# It failed with:
# module "prebuilt_okhttp-norepackage" variant "android_common": should support min_sdk_version(33) for "AdServicesApk": min_sdk_version is not specified.
#
# Unpacked the Tiramisu art snapshot generated with this change into
# tm-mainline-prod and reran the previous command and it succeeded.
Change-Id: I9d9d730845554fc175d17f38c038e4e3c7d39e07
The sdk snapshot must not be including implementation code for boot
libraries, the implementation is provided by dex jars within the
corresponding APEX. However, the snapshot does need a module for each
boot library so that the build can seamlessly access the dex files from
the APEX.
A java_library boot library (like core-oj) is represented in the
snapshot by a java_import module which requires a jar file to be
provided, otherwise it is disabled. However, that is provided purely
to keep Soong happy and should never be used.
Previously, the snapshot would contain an empty file for the jar. As
an empty file is an invalid jar any tool (like compiler) that tried
to consume it would fail which was the correct behavior. Unfortunately,
the error message that was produced was not very helpful, it was just
some variant on `invalid file` which lead to a lot of bugs being
raised.
This change replaces that empty file with a reference to the output
from a genrule which runs a script which produces a more useful error
message, with information on how to fix the issue, and fails the build.
It also adds a Name() method to the SdkMemberProperties type as that is
needed in AddInternalModule() to construct the name of the additional
module.
Tested as follows:
In AOSP/master make the following changes:
1. Temporarily set visibility on core-oj and core-libart to
//visibility:public.
2. Run packages/modules/common/build/mainline_modules_sdks.py to create
the snapshots.
For each of the S, T and latest snapshots I did the following in the
s-aml-prebuilt-test, t-aml-prebuilt-test and aosp/master branches:
1. Created an Android.bp file containing the following:
java_library {
name: "broken",
static_libs: [
"prebuilt_core-libart",
"prebuilt_core-oj",
],
}
2. Fix the visibility issues and run `m broken` where it fails with an
invalid file.
3. Delete the contents of the prebuilts/module_sdk/art/current/sdk
directory.
4. Unpack the relevant version of the art-module-sdk snapshot into the
directory.
5. Run `m broken` where it fails with the helpful message.
6. Test the instructions on how to use the ninja -t path tool to
identify the cause of the problem and fix it.
Bug: 257969510
Test: See above.
Change-Id: I125bde2d7202afff84c97daebcef37e21c548a3a
The API finalization process for platform releases copies various files
from the apistubs/ dist directory into prebuilts/sdk/<n>/... Having
them use the same name in both places makes that process simpler. For
most modules the name of the file is derived from the name of the
module by appending a suffix but unfortunately, for some modules that
does not work. e.g. the conscrypt.txt file is produced by the
conscrypt.module.public.api module. The dist_stem property was added to
java_sdk_library to allow the stem name of the file to differ from the
module name.
The API finalization process for extension APIs does something similar
as it extracts various files from the snapshots and copies them into
the appropriate extension API specific directory in
prebuilts/sdk/extensions/<n>. Instead of copying files from the
apistubs/ dist directory (which is not built for trains) it copies them
from sdk snapshots that are built as part of the train.
Previously, the sdk snapshot used to derive the name of the files
within the snapshot from the name of the module, it ignored the
dist_stem property. This change causes it to use the dist_stem property
to make it consistent with the apistubs/ dist directory naming.
The file name is created in
sdkLibrarySdkMemberProperties.AddPropertyToSet() which does not have
access to the dist_stem property. So, it has to be supplied in the
sdkLibrarySdkMemberProperties instance in the new Stem property.
Bug: 248258460
Test: m nothing
BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh --build-release=latest
# Ran the previous command before and after and make sure that the
# conscrypt, art and icu sdk snapshots use the dist_stem value but
# none of the other snapshots are affected.
Change-Id: Ied52003de63dcdb86a252a39bb8781f85d51a6ff
When an sdk snapshot is targeted at release X then it cannot include
bootclasspath fragment libraries which are not present in that build as
otherwise it causes build failures. It should also not include any
unsupported libraries, i.e. libraries that cannot work on that release.
This change causes sdk snapshot to exclude libraries that have a
min_sdk_version > target build release
It also ensures that hidden API flags do not include any information
from excluded libraries.
Bug: 240406019
Test: BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Ran the previous command with and without this change to make
# sure that this change excludes framework-connectivity-t library from the
# tethering sdk snapshot for S, including from the hidden API flag files.
Change-Id: I57969b85a12e9e5a3fc76c055b260cec5d5f7d7f
The order of the files in the sdk snapshot was stable but depended on
how the sdk was built. If the snapshot was created directly then the
files have one order, if it was created by merging in files from other
zips then it has a different order.
This change ensures files are in alphabetical order no matter how the
snapshot zip is constructed.
Bug: 232401814
Test: m media-module-sdk
# Check the contents of the following zips to ensure that they are sorted
# after this change.
# out/soong/.intermediates/packages/modules/Media/apex/media-module-sdk/common_os/media-module-sdk-current.unmerged.zip
# out/soong/mainline-sdks/media-module-sdk-current.zip
Change-Id: Ie97e0119c07a1f34a2b1d3ea6895f0e76cd195a8
While enabling prebuilts in T we hit b/229932396 which was caused by
some parts of the build depending on the prebuilt updatable-media jar
which used to be a full implementation jar but which is now an invalid
jar as the snapshot must not be including implementation details. We
fixed the issue in T but we are hitting the same problem in S with the
M-2022-07.
That is the first train in which the prebuilt updatable-media module
provides an invalid jar, prior to that it was always providing an
implementation jar. This change tweaks the sdk snapshot generation
code to use an implementation jar for updatable-media in the S
snapshot to avoid partners having to cherry pick changes similar to
those needed to fix b/229932396 in T.
Bug: 239121291
Test: packages/modules/common/build/mainline_modules_sdks.sh
# Check that S media snapshot includes implementation jar.
# Check that S art snapshot includes invalid jar.
# Check that T media snapshot includes invalid jar.
Change-Id: Ib49484d00a60b4ed7f8268e04f9c10a3498edb56
Previously, the code for selecting specific versions of sdk snapshots
was removed (along with the uses_sdks property). That makes versioned
snapshots useless so this change removes all the code and tests that
generated those versioned snapshots.
Bug: 232546567
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
Change-Id: Ib6d1b72bc8399fbb39075494ae37da92f4b28d03
Previously the SDK info file only contained basic common information
about each member. This change adds support for each member to
provide custom information to add to the info file.
It uses that mechanism to add the following:
* "dist_stem"
* "scopes" object containing:
* for each scope a:
"<scope>" object containing:
* "current_api" - the path within the snapshot for the API's .txt
file.
* "removed_api" - the path within the snapshot for the removed
API's .txt file.
* "latest_api" - the path within the build to the latest finalized
API .txt file.
* "latest_removed_api" - the path within the build to the latest
finalized removed API .txt file.
In order to access the latest API files it was necessary to add and
resolve dependencies on the module that makes them available. In order
to do that safely the code for creating the names of the modules was
refactored to avoid duplicating the name creation logic.
Bug: 204763318
Test: m nothing
Change-Id: Ica68abbd2b2c7c2b2b7877b502f96cc89f06fd68
Some build scripts need to know information about the contents of an
sdk, such as what APIs it provides (via java_sdk_library). Rather than
duplicate that information in the scripts or attempt to access that
information (where available) by looking at the contents of the
snapshot this change generates a JSON file that sits alongside the
snapshot itself.
The info file can be generated without generating the snapshot zip file
but whenever a snapshot zip file is generated the info is generated
too. The info file sits alongside the zip file in out/mainline-sdks.
Bug: 204763318
Test: m art-module-sdk
m dist
Change-Id: I289530bb21693dc6443826c24c17c9b5d85d2d8b
Marks the `scopeProperties.AnnotationsZip` property as only being
supported on T builds and above.
Bug: 204763318
Test: m nothing
Change-Id: Ie59396287c08da77a6a1f15d4be202429e011e17
Similar to java_boot_libs, java_systemserver_libs only provide access
to their dex implementation jar for use by dexpreopting and do not
provide an actual implementation jar.
This is used in the subsequent CL.
Bug: 194150908
Test: m nothing
Change-Id: Ib2e7d5e6c002314a0231f46934a766f4a27e610a
* changes:
Consolidate the code to resolve a deapexer module dependency.
Propagate the dex jar path as an OptionalPath which is either valid or invalid with a message.
invalid with a message.
This will allow propagating any error from the deapexer module for
prebuilt APEXes to the location where the dex jars get used. It's only
at those points that we can raise errors about not being able to
extract files from the deapexer modules if they are invalid, and this
way we avoid encoding knowledge there about why they may be invalid.
To keep the refactoring limited it intentionally does not change any of
the existing logic for when dexJarFiles are set or not (non-nil vs nil
prior to this change), although there may be opportunity to use this
for more conditions when dex jars aren't available.
The refactoring is also not extended to
dexpreopt.ClassLoaderContextMap.
Test: m nothing
Bug: 192006406
Change-Id: I68986dccd9a9b3fee4d24caa1947ea17a36caedc
The annotations zip file is produced by the "main" sdk build and is
primarily consumed by android studio.
In order to support building the main SDK without requiring the sources
of all modules, we are adding module SDK artifacts that allows
reconstructing these outputs. The annotations zip contains XML files
which should be fairly easy to merge from all the individual parts.
Bug: 187397779
Test: unit tests in this CL
Test: m sdkextensions-sdk and inspect output
Change-Id: I955cae720e6f1382936836ee1d8fb11003f51b7d
This reverts commit 22ff0aaf51.
The workaround to support hidden API generation with prebuilts by
passing the full implementation jars through to the SDK snapshot is no
longer needed as the monolithic hidden API flag generation uses the
prebuilt flag files that are already part of the snapshot.
This change reverts that previous workaround.
Bug: 192868581
Test: - Update the prebuilts
DIST_DIR=$PWD/dist TARGET_BUILD_VARIANT=userdebug art/build/build-art-module.sh --skip-apex
packages/modules/ArtPrebuilt/update-art-module-prebuilts.py --local-dist=dist --skip-cls --skip-apex
- Build hidden API flags to make sure it does not fail
Change-Id: Idb3fde6f7dcb171677316d8794a4af91ede1f7e0
Previously, permitted_packages were not copied to the sdk snapshot.
This change corrects that.
Bug: 193763688
Test: m nothing
- Added unit tests, which all failed and then fixed the tests.
Change-Id: I4560987f746f78c0ae706058195b6db4bea438aa
Previously, an sdk snapshot could contain the following:
* A java_sdk_library_import module, e.g. "foo" which creates component
modules "foo.stubs", etc.
* A corresponding versioned module, e.g. "sdk_foo@current" which
created component modules "sdk_foo@current.stubs", etc.
* An internal (to the sdk snapshot) java_import for one of "foo"'s
components, e.g. "sdk_foo.stubs"
* A corresponding versioned module, e.g. "sdk_foo.stubs@current".
That causes a few problems:
1. The "foo.stubs" is duplicated.
2. The names of the components created by the versioned
java_sdk_library_import are invalid, as they append the component's
suffix to the version and not the name before the version.
The latter causes problems when building against prebuilts and fixing
that causes the generated snapshot to be invalid because it contains
duplicate definitions of the "sdk_foo.stubs@current" module. One
explicitly in the Android.bp file and one created by the
"sdk_foo@current" module.
Removing the duplicates from the snapshot causes errors as the name
generated by the snapshot for the component module, i.e.
"sdk_foo.stubs@current" does not match the name generated by the
"sdk_foo@current", i.e. "sdk_foo@current.stubs".
This change fixes them together.
Bug: 179354495
Test: m nothing
Merged-In: I515f235fe21755b5275af12366e96c24c94c0273
Change-Id: I515f235fe21755b5275af12366e96c24c94c0273
(cherry picked from commit a1aa7387f7)
This change adds a test for an sdk that includes a java_sdk_library
directly as well as transitively includes one of the component modules
that it creates. It also adds a test for the names of a versioned
java_sdk_library_import's components as fixing one requires fixing the
other.
The use case added is one that does occur in the art-module-sdk. It has
a couple of pairs of java_sdk_library and java_system_modules that
cause both the java_sdk_library and one of its component modules to be
included in the sdk snapshot. That causes problems when building
against prebuilts.
e.g. The art-module-sdk_art.module.public.api.stubs module is a
duplicate of the art.module.public.api.stubs module created by the
art.module.public.api java_sdk_library_import. The former is added
because the art-module-public-api-stubs-system-modules depends on it.
A follow up change will avoid the component being added to the sdk
snapshot separately and instead cause the snapshot to use the
component created by the java_sdk_library_import in the snapshot.
Bug: 179354495
Test: m nothing
Change-Id: Ifdc1b4a5a7968db2ded3fdb7eb02f9cff77c7c3f
The tests use <sdk>_<module>_<version> as the format for a versioned
sdk member name but the format should be <sdk>_<module>@<version>. This
change corrects it and also fixes a similar issue in an error message.
Bug: 181569894
Test: m nothing
Change-Id: I8be0db4bcd0b6f4d6fbdf9e402ef7257fae8e18b
The hidden API processing needs access to dex jars for the API stubs in
order to determine which dex members are part of an API surface. The
dex stubs used for the monolithic file are provided by normal
java_library modules for legacy reasons. However, the APEXes that
contribute to the bootclasspath, and so need to perform hidden API
processing, typically provide stubs created by a java_sdk_library.
This change adds support to java_sdk_library/_import to make the dex
stub jars available when requested, that involves:
1. Adding compile_dex property to java_sdk_library_import and
propagating it down the the java_import modules for the stubs. That
is already handled for java_sdk_library.
2. Propagating the java_sdk_library compile_dex property to the
java_sdk_library_import in the generated snapshot.
3. Refactoring and wiring to make the dex stubs jar available to other
parts of Soong.
Bug: 179354495
Test: m nothing
Change-Id: I5895d4f2ba0b684870862b9429b2364865e4afc6
Replaces a single call to checkAndroidBpContents(...) with separate
calls to check the versioned and unversioned Android.bp files.
Test: m nothing
Bug: 179354495
Change-Id: I270bf73909958d97b2f298e8d7f6d10a1f75ae71
* changes:
Strengthen metalava sandbox support using sbox
Move metalava's output files into a subdirectory
Fix lint warnings in droidstubs.go
Split droidstubs out of droiddoc.go
Move all of the output files of the metalava rule into a single
subdirectory to make it compatible with sandboxing in sbox.
Test: TestDroidstubs
Change-Id: I66101c0c224dee702c8175e61c12cc9cd1aa8b93
Test behavior was changed a while ago so that tests by default ignore
non-existent source paths (unless they explicitly check for/rely on
them). Prior to that CheckSnapshot() could detect when files were
missing from the snapshot but it no longer can.
This change disallows non-existent source files in all the sdk tests
which means that they are disallowed when processing the snapshots as
they use the same preparers as were used to process the sources.
This caused a test failure which has been temporarily ignored and has
a TODO and bug associated with it.
Bug: 183184375
Test: m nothing
Change-Id: I969d8515d20ef5ae515f2b5f93d8ed4e4f8ede75
The sdk produces snapshots that are expected to be unpacked in one of
the Android repos. Often that can lead to issues due to conflicts
between the source and prebuilts. This change attempts to avoid those
conflicts by testing the different ways that those files can be used.
With the existing test to cover adding the snapshot
This change will cause the sdk tests to check the following:
1) Snapshot on its own (already done).
2) Snapshot plus original source where the original source is
preferred.
3) Snapshot plus original source where the snapshot is preferred.
It also adds the ability for tests to provide their own custom checkers
to verify the result of each of the previous tests.
This change reveals a number of bugs already present. Rather than
attempt to fix them this change adds the ability to specify error
handlers for the two cases that mix source and snapshot to allow those
errors to be temporarily ignored while allowing the majority of the
tests to benefit from this improvement. Each of those failures has a
TODO and bug associated with it.
Bug: 183184375
Test: m nothing
Change-Id: I105233195074dbe7a6422b6dfc5486e74398ea15
This change is in preparation for removing testing.T from TestResult.
Bug: 181070625
Test: m nothing
Change-Id: I67535aff0d894e6e3d8456b75540f340af853355
This will allow the testSdkResult to be replaced with the TestResult
when switching sdk package to use test fixtures.
Bug: 181070625
Test: m nothing
Change-Id: Ieca63f4c189f5e804102aeb4073289ea03143b6e
This change makes it easier to switch the sdk package over to using the
new fixture mechanism by removing inconsistencies between the
testSdkResult and TestResult structures.
Bug: 181070625
Test: m nothing
Change-Id: Ic4c06e08ea5060fd09123f2ca65580e18b4d2ef6
Hiddenapi processing currently requires access to the class
implementation jars for libraries on the bootclasspath which means that
they need to be provided as part of the prebuilts. This change modifies
the java_boot_libs property on the sdk to make those files available.
Modularization of the hiddenapi processing will hopefully remove the
need for these to be exported so this should be temporary.
Bug: 178361284
Test: m art-module-sdk
check generated snapshot zip contains implementation jars
Change-Id: I9e94662dddb0ddb85a477ae6d27e533085147e88
The build has some implicit dependencies (via the boot jars
configuration) on a number of modules, e.g. core-oj, apache-xml, that
are part of the java boot class path and which are provided by mainline
modules (e.g. art, conscrypt, runtime-i18n) but which are not otherwise
used outside those mainline modules.
As they are not needed outside the mainline modules adding them to
the sdk/module-exports as either java_libs, or java_header_libs would
end up exporting more information than was strictly necessary. This
change adds the java_boot_libs property to allow those modules to be
exported as part of the sdk/module_exports without exposing any
unnecessary information.
Some points to note:
* The java_import has to have a valid file for the src property
otherwise it will be disabled.
* The src property is supposed to reference a jar file but the
java_boot_libs property will make it reference an empty file (not
an empty jar) so that any attempt to use that file as a jar, e.g.
compiling against it, will cause a build failure.
* The name of the file passed to the src property should make it
clear that the file is not intended to be used.
* The test makes sure that only the jar file is copied to the
snapshot.
Test: m nothing
Bug: 171061220
Change-Id: I175331e4c8e3874ab70a67cdc2f76ed1576e41eb
Setting this to true by default is dangerous as it can mask bugs. Create
a dedicated attribute for java_sdk_library to enable this behavior
instead. The default will be flipped in a future CL when all the current
offenders have been fixed.
Fix all the tests to have the right API files.
Bug: 176092454
Test: m nothing
Change-Id: Ieab94bcb74abf8d018365a56fb447fe3dbd46957
This reverts commit 7f97957ded.
Reason for revert: breaks sdk snapshots b/173508731
Bug: 173508731
Test: Ran prebuilts/runtime/update.py and then m nothing
Before revert it failed
After revert it worked
Change-Id: I9c081681fac589e37788a0d592435e3224011c58
The droidstubs support in sdk/module_exports was a temporary measure
to work around the fact that some APIs were being defined by direct use
of droidstubs instead of java_sdk_library.
This change removes that support as those APIs have all been switched
from droidstubs to use java_sdk_library so droidstubs support is no
longer needed.
Bug: 168301990
Test: m nothing
Change-Id: I3517bed29b030438a0423a6cb8c248992a988222
Some java_sdk_library modules do not have any stubs for some API
scopes. That results in an empty ".srcjar" being created for them which
ends up not creating a directory for the stubs sources when that
snapshot is unzipped. Previously, that would cause a build failure as
the generated java_sdk_library_import module used the directory, which
did not exist, in its stub_srcs property.
This change switches the stubs_srcs property to use a glob pattern of
"**/*.java" relative to the directory instead of using the directory
directly. When the directory does not exist the glob pattern is
resolved to an empty set of paths and does not break the build.
Bug: 172811712
Test: Add sdkextensions-sdk sdk module (local patch)
m sdk-extensions (local patch)
unpack generated snapshot
m nothing - results in build failure due to missing directory
Make this change and repeat above except this time the build works.
Change-Id: I691ffbfdc01ba89bbcaf647dcbb7dfebc3c8aec2
Also fix //apex_available:platform that could get lost in the generated
apex_available properties. It only happened if a module didn't specify
an apex_available setting and then got extra entries from
apex.BaselineApexAvailable.
Test: Build ART SDK with libnativeloader_lazy in the SDK, and check
that the generated prebuilt module has //apex_available:platform
(before https://r.android.com/1480559).
Change-Id: I9df20aee63455932a7e558993f0f26769cb02792
