The sdk snapshot must not be including implementation code for boot
libraries, the implementation is provided by dex jars within the
corresponding APEX. However, the snapshot does need a module for each
boot library so that the build can seamlessly access the dex files from
the APEX.
A java_library boot library (like core-oj) is represented in the
snapshot by a java_import module which requires a jar file to be
provided, otherwise it is disabled. However, that is provided purely
to keep Soong happy and should never be used.
Previously, the snapshot would contain an empty file for the jar. As
an empty file is an invalid jar any tool (like compiler) that tried
to consume it would fail which was the correct behavior. Unfortunately,
the error message that was produced was not very helpful, it was just
some variant on `invalid file` which lead to a lot of bugs being
raised.
This change replaces that empty file with a reference to the output
from a genrule which runs a script which produces a more useful error
message, with information on how to fix the issue, and fails the build.
It also adds a Name() method to the SdkMemberProperties type as that is
needed in AddInternalModule() to construct the name of the additional
module.
Tested as follows:
In AOSP/master make the following changes:
1. Temporarily set visibility on core-oj and core-libart to
//visibility:public.
2. Run packages/modules/common/build/mainline_modules_sdks.py to create
the snapshots.
For each of the S, T and latest snapshots I did the following in the
s-aml-prebuilt-test, t-aml-prebuilt-test and aosp/master branches:
1. Created an Android.bp file containing the following:
java_library {
name: "broken",
static_libs: [
"prebuilt_core-libart",
"prebuilt_core-oj",
],
}
2. Fix the visibility issues and run `m broken` where it fails with an
invalid file.
3. Delete the contents of the prebuilts/module_sdk/art/current/sdk
directory.
4. Unpack the relevant version of the art-module-sdk snapshot into the
directory.
5. Run `m broken` where it fails with the helpful message.
6. Test the instructions on how to use the ninja -t path tool to
identify the cause of the problem and fix it.
Bug: 257969510
Test: See above.
Change-Id: I125bde2d7202afff84c97daebcef37e21c548a3a
Previously, in a build containing source and prebuilt art
bootclasspath_fragments, the bootImageVariant.licenseMetadataFile was
set twice with the source always being set after the prebuilt and
so winning.
This change only sets bootImageVariant.licenseMetadataFile for the
active module so it will use the prebuilt's license file if that is
preferred.
Bug: 245956352
Test: m nothing
Change-Id: I948c7e5123169452f67c85ad98c4bbdb90a5d2de
Most of the fields in the bootImageConfig/Variant structs are assigned
inside a Once func so are guaranteed to be only set once. However, some
are assigned outside. This change adds comprehensive tests for those
structs and verifies that the constant fields are preserved and the
mutated fields have the correct value.
The check for the constant fields is added in a new TestBootImageConfig
test.
The check for the mutated fields is added into
TestSnapshotWithBootclasspathFragment_ImageName as that test checks an
art bootclasspath_fragment in the following configurations:
* source on its own
* prebuilt on its own
* source and prebuilt with source preferred
* source and prebuilt with prebuilt
It reveals a couple of interesting facts:
* All the *installs fields are set to the same value irrespective of
whether the source or prebuilt is preferred. The information is
constructed solely from information already within the
bootImageConfig/Variant and so can be moved within Once.
* The licenseMetadataFile is incorrect when prebuilt is preferred.
That is due to both the source and prebuilt modules setting it and
the source module always wins as the source module depends on the
prebuilt so always runs its GenerateAndroidBuildActions after it.
Those issues will be cleaned up in following changes.
Bug: 245956352
Test: m nothing
Change-Id: If917cfbcb3b1c842a8682d51cc1ee1fed1c51add
When an sdk snapshot is targeted at release X then it cannot include
bootclasspath fragment libraries which are not present in that build as
otherwise it causes build failures. It should also not include any
unsupported libraries, i.e. libraries that cannot work on that release.
This change causes sdk snapshot to exclude libraries that have a
min_sdk_version > target build release
It also ensures that hidden API flags do not include any information
from excluded libraries.
Bug: 240406019
Test: BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Ran the previous command with and without this change to make
# sure that this change excludes framework-connectivity-t library from the
# tethering sdk snapshot for S, including from the hidden API flag files.
Change-Id: I57969b85a12e9e5a3fc76c055b260cec5d5f7d7f
Previously, hidden API properties were only allowed on
bootclasspath_fragment and platform_bootclasspath module types. This
change allows them to be specified on java_sdk_library modules too. It
involves the following changes:
1. Add the properties to the java.Module.
2. Populate and provide a HiddenAPIPropertyInfo struct from
java_sdk_library modules.
3. Modify bootclasspath_fragment to merge information gathered from its
content libraries as if it was specified on the fragment itself.
Bug: 240406019
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
# Ran the previous command with and without this change to make
# sure that this change does not change the sdk snapshot
# contents.
Change-Id: I64eb71c2039ddc14cf380689d0cec7ec221f5b88
Specifying an apex in the apexes propety will cause all the
*classpath_fragments that are contents of the APEX to be automatically
added as members of the sdk and appear in the snapshot.
The purpose of this change is to dedup the APEX and sdk definitions and
try and avoid some of the issues that we have been finding while
attempting to build against the prebuilts.
Two tests, one each for bootclasspath_fragment and
systemserverclasspath_fragment, have been refactored to compare the
output when adding the *fragment to the sdk directly of via the APEX.
That ensures switching to use the APEX will not change the sdk snapshot
unless it was previously missing a *fragment.
There was also a slight difference in where the hidden API flags were
copied from. That should have no impact on the output as the flags are
identical.
The sdk snapshot generation needed some tweaks to avoid generating a
prebuilt for the APEX.
Bug: 232401814
Test: m nothing
Change-Id: I7aaf16a3a0ab4bebf97765d1484215cc008dc4b8
Previously, the split_packages, single_packages and package_prefixes
properties were all optional and the split_packages defaulted to ["*"].
As that value conflicted with the other package properties that meant
that split_packages always had to be specified even if it was to just
set it to an empty array.
This change requires at least one of them to be specified and defaults
split_packages to an empty list which means it is not required,
although it can be helpful to make that explicit.
Bug: 194063708
Test: m nothing
Change-Id: I5a4c2d68e72e39f5c4a2441326dfce8685fc8ff2
Previously, the code for selecting specific versions of sdk snapshots
was removed (along with the uses_sdks property). That makes versioned
snapshots useless so this change removes all the code and tests that
generated those versioned snapshots.
Bug: 232546567
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
Change-Id: Ib6d1b72bc8399fbb39075494ae37da92f4b28d03
We will need the profile when we generate the primary boot image on
device.
Bug: 203492478
Test: Run `banchan com.android.art x86_64 && m` and see
`$ANDROID_PRODUCT_OUT/apex/com.android.art/etc/boot-image.prof`.
Test: Run `lunch aosp_cf_x86_64_phone-userdebug && m` and see both
`$ANDROID_PRODUCT_OUT/apex/com.android.art/etc/boot-image.prof` and
`$ANDROID_PRODUCT_OUT/system/etc/boot-image.prof`, in different
sizes.
Test: Start Cuttlefish with the built image and see both
`/apex/com.android.art/etc/boot-image.prof` and
`/system/etc/boot-image.prof` on device.
Change-Id: Id879dc49b234133dfbb9563814328661a1f4a6c0
Previous change that was reverted: https://r.android.com/1835222
An additional test was added that revealed a bug in the previous change
which has been fixed here.
Previously, the behavior of the stub_flags and all_flags properties
was different between S and T. In S they contained paths for the
complete set of stub flags and all the encoded flags. However, in T
they contained filtered sets of flags which if used in S would prevent
build checks from detecting possible inconsistencies. Also, a new
signature_patterns property was added in T that is not supported in S.
This change creates separate properties/files for T and reverts the
behavior of the properties/files that were added in S back to how they
behaved in S. The new properties are called filtered_stub_flags and
filtered_flags.
The S and T properties are tagged with the appropriate
supported_build_releases tag to ensure that they are only output when
specifically targeted.
Bug: 197842263
Test: m nothing
Change-Id: I1ce0a3d6623dabf73e32af1a7457b9b444fc3b7c
Previously, the behavior of the stub_flags and all_flags properties
was different between S and T. In S they contained paths for the
complete set of stub flags and all the encoded flags. However, in T
they contained filtered sets of flags which if used in S would prevent
build checks from detecting possible inconsistencies. Also, a new
signature_patterns property was added in T that is not supported in S.
This change creates separate properties/files for T and reverts the
behavior of the properties/files that were added in S back to how they
behaved in S. The new properties are called filtered_stub_flags and
filtered_flags.
The S and T properties are tagged with the appropriate
supported_build_releases tag to ensure that they are only output when
specifically targeted.
Bug: 197842263
Test: m nothing
Change-Id: Iec8b9c539796c507245b69c0aed980fde6d8694f
Historically, this file was used by the `--updatable-bcp-packages-file`
flag for dex2oat. The flag is no longer needed and is being deprecated.
Unfortunately, we cannot remove `permitted_packages` and all the code
related to it because we still need it for checking the module
compatibility with Q and R.
Bug: 200241946
Test: m nothing
Change-Id: Ie3be54fd47a847ba0caf627d12da76b415d99466
Previously, the sdk snapshot would include all the entries from the
stub-flags.csv and all-flags.csv modular files generated by a single
bootclasspath_fragment. That included implementation details, i.e.
class members that are not part of a stable API or the hidden API which
meant that the sdk snapshots were implementation dependent.
This change removes the implementation details from the modular flag
files, i.e. those entries that are only marked as "blocked". When
comparing the files against the corresponding subset of the monolithic
files it assumes that any entries missing from the modular flag files
are blocked.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I6b67b2253cf029d6830b58a06ebb0c8fcaa0dd71
This is in preparation to r.android.com/1740313 where setting correct
variables would be enforced (i.e. apex and non-apex boot jars must be in
config.ApexBootJars and config.BootJars correspondingly).
Bug: 191369843
Test: m nothing
Change-Id: Ic86680c1f7af53d229083b2cc58beb3ceccb4b6a
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: Ic819def9b0c41e3b22fb98582cd39f6d27080a32
Previously, the signatures used to select the subset of the monolithic
flags were simply the signatures read from the modular flags file. This
change moves the creation of the signature list into a separate script
that outputs the signatures to a file and then passes the path through
Soong from the bootclasspath_fragment modules that create it to the
platform_bootclasspath module that uses it to compare the modular
flags against the monolithic flags.
Currently, the signatures are the full signatures but follow up changes
will replace them with patterns (hence the name) that avoids having to
include implementation details in the hidden API flags that are output
as part of a bootclasspath_fragment's snapshot.
This change moves the stub flags related code next to the all flags
related code as they are treated in a similar way.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I2855bf6d05c91b8a09591664185750361c7e644f
Note that ART apex boot jars and core-icu4j are exceptions here as they
are not part of ApexBootJars. ART apex boot jars are defined in their
own variable, while core-icu4j is treated as a regular non-updatable
boot jar.
Bug: 191127295
Test: atest CtsClasspathsTestCases
Change-Id: I3cea3d82ef521655a1a5ffa8cae2258ab9d08bfc
This reverts commit 22ff0aaf51.
The workaround to support hidden API generation with prebuilts by
passing the full implementation jars through to the SDK snapshot is no
longer needed as the monolithic hidden API flag generation uses the
prebuilt flag files that are already part of the snapshot.
This change reverts that previous workaround.
Bug: 192868581
Test: - Update the prebuilts
DIST_DIR=$PWD/dist TARGET_BUILD_VARIANT=userdebug art/build/build-art-module.sh --skip-apex
packages/modules/ArtPrebuilt/update-art-module-prebuilts.py --local-dist=dist --skip-cls --skip-apex
- Build hidden API flags to make sure it does not fail
Change-Id: Idb3fde6f7dcb171677316d8794a4af91ede1f7e0
Previously, bootclasspath_fragment modules would only perform hidden
API processing if they provided some stub libraries and fragments. That
was needed because the bootclasspath_fragment modules were added before
Soong supported hidden API processing on all the different modules and
before they all provided the necessary information that hidden API
processing required.
This change stops hidden API being conditional as it is no longer
required as it has been enabled on all existing bootclasspath_fragment
modules.
Bug: 179354495
Test: m nothing
Change-Id: I0cbf11986adff1f2f967b96f86e6bfe0e9b8b1ef
Previously, the java_import and java_sdk_library_import modules did not
make their permitted_packages available to the rule that generates the
updatable-bcp-packages.txt file. This change corrects that.
Bug: 193763688
Test: m nothing
- Added unit tests, which all failed and then fixed the tests.
Change-Id: If0706e4551a331b48d383123088e63924dded48b
Previously, permitted_packages were not copied to the sdk snapshot.
This change corrects that.
Bug: 193763688
Test: m nothing
- Added unit tests, which all failed and then fixed the tests.
Change-Id: I4560987f746f78c0ae706058195b6db4bea438aa
Previously, a java_sdk_library with shared_library = true would create
a variation per APEX because it depends on an sdkLibraryXml module
which generates a file containing the APEX name. However, a shared
java_sdk_library_import would create a merged APEX variation. The
inconsistent variations caused failures in sdkDepsReplaceMutator.
This change ensures that both java_sdk_library and
java_sdk_library_import create an APEX specific variation when their
shared_library property is true.
Bug: 190499958
Test: m nothing
- ran the above command after modifying the test to reproduce the
problem and then after fixing to verify that it fixed the problem.
Change-Id: Iee81776a8569db3e871c40cbde14d248dfeb56e4
Adds support for printing a PropertySet that has a property whose value
is an array of structs.
Bug: 179354495
Test: m nothing
m conscrypt-module-sdk
- check generated Android.bp file
Change-Id: I71be04188465610bcbea4d3c9a5e8204171a1eeb
Prebuilt modules do not provide classesJars containing annotations.
Previously, the monolithic hidden API processing just used classesJars
from all the modules that provided them so when building against
prebuilts would have fewer classesJars than when building against
sources and so would produce different hidden API flags.
This change will generate the monolithic files from both classesJars
and files previously generated from hidden API processing. A fragment
that has performed hidden API processing will contribute its generated
files whereas standalone libraries and fragments which have not
performed hidden API processing will contribute classesJars.
Bug: 177892522
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
m SOONG_CONFIG_art_module_source_build=false out/soong/hiddenapi/hiddenapi-flags.csv
- verify that the files are identical whether built from
source or prebuilts.
Change-Id: I06f3c7df49626bec21a452bc9abf1bb9e7545e5c
Bug: 177892522
Bug: 189298093
Test: m check-boot-jars
m SOONG_CONFIG_art_module_source_build=false check-boot-jars
- Ran both commands with and without java.lang in the
package_allowed_list.txt
Change-Id: Iba1a881c8f6b6919d5c0c0520eb3073658f3b8d2
In the input test fixture this makes sure that every
bootclasspath_fragment is part of an apex and every content module for
the fragment is listed in the appropriate boot jars config property. It
also adds a platform_bootclasspath fragment that references the
fragment.
In the snapshot test fixtures this adds a prebuilt_apex that exports
the fragment so that the platform_bootclasspath has access to the dex
implementation files.
Bug: 177892522
Test: m nothing
Change-Id: I6c73651a359052858232b06229b4433799dd94db
Previously, due to legacy reasons, the property validation did not
require a contents property and allowed the image_name to be either
"art" or "boot". Those reasons no longer apply and so this change
requires a contents property and only allows the image_name to be set
to "art" if specified.
Bug: 177892522
Test: m nothing
Change-Id: I8855d6e5365ef0b55490e90e7b6c0081cf070ee5
This change adds support for generating the hidden API flags for the
contents of a bootclasspath_fragment. Currently, it will only work for
the art-bootclasspath-fragment as it has no support for creating
dependencies between bootclasspath_fragment modules which will be
needed for handling any other bootclasspath_fragment.
The hidden API flag generation added by this change is completely
separate to the normal hidden API processing and is not as yet encoded
in dex jars so will have no effect on the runtime.
The generated files are provided for use by other modules and copied
into the sdk snapshot. That is needed to allow the build to verify that
the hidden API flags generated by the individual bootclasspath_fragment
modules are consistent with the flags generated for the whole
bootclasspath, whether building from source or prebuilts.
Bug: 179354495
Test: m art-module-sdk
m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
m out/soong/hiddenapi/hiddenapi-flags.csv
- test that the former file is a subset of the latter and that
where they overlap they are identical.
Change-Id: Ie27303e2960953db1b7abe95510e3bca4411b09a
A java_library specified in a bootclasspath_fragment's contents
property will be automatically added to the sdk containing that
bootclasspath_fragment. Previously, if that was attempted with a
java_sdk_library it would be added to the sdk as if it was a normal
java_boot_libs which would prevent the sdk from containing the
API specific artifact such as current.txt files and stub libraries
and sources.
This change fixes that and adds a java_sdk_library as a java_sdk_libs
module.
Bug: 177892522
Test: m nothing
Change-Id: Ided57b846ce5b8940c7e898c786fd77602582ea2
Adds the bootclasspath_fragment to an apex and also adds an sdk library
to the stub_libs property.
Bug: 179354495
Test: m nothing
Change-Id: I2a5fffe915eb62923cc60732352d4d050dec4398
Previously, the deapexer module properties were simply passed through
from the prebuilt_apex/apex_set modules. This change derives the
properties from the transitive dependencies of the modules so that it
can automatically include the contents of a bootclasspath_fragment
module without having to duplicate them in the exported_java_libs
property.
Part of this change involves moving the addition of dependencies onto
the prebuilt apex's contents from DepsMutator to ComponentDepsMutator
so that they can be visited in the createDeapexerModule() function. The
ComponentDepsMutator runs before prebuilts without matching sources are
renamed to match the source name which simplifies the process of adding
dependencies directly onto the prebuilts.
The deapexerDeps method was renamed as the dependencies are added both
for use by deapexer and also to create APEX variants for access by
platform_bootclasspath so the name was confusing.
Bug: 187266082
Test: m nothing
m SOONG_CONFIG_art_module_source_build=false nothing
- this fails but not because of missing APEX variants
Change-Id: Icb4f883e7d2b63739e5ff0dc6edb8a906d80189b
The getBootImageJar function will be removed once the boot image
creation has been moved to the platform_bootclasspath and
bootclasspath_fragment module types. However, the consistency checks
that it performs are still useful so this change moves them out
first.
The ART boot image related checks are now performed in the
bootclasspath_fragment module type. A previous change accidentally
disabled the checks when the contents property was not empty which has
been fixed. Also, the error messages have been tweaked to make it clear
that the art-bootclasspath-fragment is now the source of truth as to
its contents not the configuration.
The framework boot image related checks are now performed in the
platform_bootclasspath module type.
Initially, this change included an extra check to make sure that
UpdatableBootJars comes from updatable APEXes but that broke because
framework-wifi and framework-tethering are not currently marked as
updatable in AOSP.
Bug: 177892522
Test: m nothing
Change-Id: I80fb600fa2c7cec4566b3461c6a33c4c6f0743f4
Previously, the method was run in repo manifests that did not include
the frameworks/base repository so it had to handle the file being
missing. However, now that this is being called from
platform_bootclasspath which is only defined in frameworks/base that
is no longer required.
Bug: 177892522
Test: m droid
Change-Id: I77fa5a204d1426a6be24a6f0b48e18f48f3dd908
Previously, both a bootclasspath_fragment and its contents had to be
explicitly added to the sdk. This change means that adding a
bootclasspath_fragment to and sdk will automatically add its contents
as if they were added using java_boot_libs.
Bug: 177892522
Test: m nothing
Change-Id: I8f7e70649f272c9a109d4606571a2d12c44b7904
This is needed to allow a prebuilt_bootclasspath_fragment to be used
interchangeably with a bootclasspath_fragment in the
platform_bootclasspath module.
The platform_bootclasspath module depends on APEX specific variants of
bootclasspath_fragment modules. That works because the
bootclasspath_fragment modules are part of an apex and so have an APEX
specific variant which the platform_bootclasspath can specify.
Using a prebuilt_bootclasspath_fragment in place of a
bootclasspath_fragment requires that the prebuilt also has an APEX
specific variant.
Specifying exported_bootclasspath_fragments on a prebuilt_apex/apex_set
will cause it to create an APEX variant for the named module whcih will
allow it to be selected by the platform_bootclasspath module.
Bug: 186034565
Bug: 177892522
Test: m nothing
Change-Id: I7ddacc6498ec3a4a9f26c5f78b7f9a033e494d78
These are needed at the moment to ensure the hidden API processing
generates the same set of flags whether it is being generated from
source or prebuilts. Soon these will be needed to ensure that the
hidden API flags generated for the individual modules are compatible
with previous releases.
Bug: 179354495
Test: m art-module-sdk and check snapshot zip contains the files
and the generated Android.bp references them.
Change-Id: I9a3334cc48faa381bbbcbbb59479db719042796a
