This is needed to determine whether we need to deapex the profile or
not.
Bug: 241823638
Test: m nothing
Test: -
1. Patch ag/20582608 PS2.
2. TARGET_BUILD_APPS=com.android.wifi packages/modules/common/build/mainline_modules_sdks.sh
3. See the `dex_preopt` property being written.
Change-Id: I05177388c24be1ab315247ea85e7e3a8600cebd3
Previously, the memberMutator was responsible for adding dependencies
from the sdk to its members. It could not just implement DepsMutator()
as it had to be run before the memberDepsMutator which itself had to be
run before the depsMutator.
As the memberDepsMutator has been removed by a previous change this
change removes the memberMutator and just performs that functionality
in the standard DepsMutator(BottomMutatorContext) method.
Bug: 260237150
Test: m nothing
BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Run the above before and after this change and compare the
# out/dist/*mainline-sdks directory contents to ensure that there are no
# changes.
Change-Id: I264bdeee84f2349e30e5c24dd4861e945640d020
Previously, the memberDepsMutator was responsible for tracking the
membership of modules in an sdk/module_exports (or corresponding
snapshot). That was only used to determine whether a module was in a
versioned sdk snapshot so that it could be treated differently by the
build. As there are no versioned sdk snapshots these can now be
removed.
This change removes the memberDepsMutator, along with SdkAware and
related types that are no longer used.
Bug: 260237150
Test: m nothing
Change-Id: I2668679a42d5b3f33f9e06d5b59804dfdae5f6da
Previously, the sdk snapshot code used SdkAware instead of Module to
ensure that all its members had implemented SdkAware. However, it never
used any of the methods provided by SdkAware, or if it did it no longer
does. So, this change replaces usages of SdkAware with Module in
preparation for deleting it completely.
Bug: 260237150
Test: m nothing
Change-Id: Ia89e02394f27b2da776f0cf0f0bc86835a03433a
In preparation for removing support for versioned snapshots from Soong
this change rewrites TestBasicSdkWithBootclasspathFragment to use
non-versioned prebuilts making it more realistic in the process.
Bug: 260237150
Test: m nothing
Change-Id: I99d1ad9b480657bb805c3d48e99e715052f4d00a
The min_sdk_version was added to the java_import in Tiramisu. This
change will propagate the min_sdk_version set on a java_library to the
java_import snapshot. If possible the min_sdk_version will be resolved
into a numerical version to ensure consistent behavior across build
releases.
Bug: 260560424
Test: m nothing
BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Ran above before and after this change and made sure that only
# Tiramisu, UpsideDownCake and latest were changed and those changes
# were the addition of a min_sdk_version property.
#
# Ran the following command in tm-mainline-prod:
# lunch cf_x86_64_phone-userdebug
# m ART_MODULE_BUILD_FROM_SOURCE=false nothing`.
#
# It failed with:
# module "prebuilt_okhttp-norepackage" variant "android_common": should support min_sdk_version(33) for "AdServicesApk": min_sdk_version is not specified.
#
# Unpacked the Tiramisu art snapshot generated with this change into
# tm-mainline-prod and reran the previous command and it succeeded.
Change-Id: I9d9d730845554fc175d17f38c038e4e3c7d39e07
The following environment variables are no longer used when generating
sdk snapshots:
* SOONG_SDK_SNAPSHOT_PREFER - we can control the modules that are
preferred at build time now.
* SOONG_SDK_SNAPSHOT_USE_SOURCE_CONFIG_VAR - this is done as a
post-processing step by the mainline_module_sdks.sh script to avoid
having to regenerate the ninja files for every module.
* SOONG_SDK_SNAPSHOT_VERSION - support for this was removed a long time
ago but was still referenced from dumpvars.go.
Bug: 259095197
Test: m nothing
# Searched for uses of these variables but could not find any
# outside the build.
Change-Id: Id2b32b29ff6f09faffea43292f549ba13acd956d
The sdk snapshot must not be including implementation code for boot
libraries, the implementation is provided by dex jars within the
corresponding APEX. However, the snapshot does need a module for each
boot library so that the build can seamlessly access the dex files from
the APEX.
A java_library boot library (like core-oj) is represented in the
snapshot by a java_import module which requires a jar file to be
provided, otherwise it is disabled. However, that is provided purely
to keep Soong happy and should never be used.
Previously, the snapshot would contain an empty file for the jar. As
an empty file is an invalid jar any tool (like compiler) that tried
to consume it would fail which was the correct behavior. Unfortunately,
the error message that was produced was not very helpful, it was just
some variant on `invalid file` which lead to a lot of bugs being
raised.
This change replaces that empty file with a reference to the output
from a genrule which runs a script which produces a more useful error
message, with information on how to fix the issue, and fails the build.
It also adds a Name() method to the SdkMemberProperties type as that is
needed in AddInternalModule() to construct the name of the additional
module.
Tested as follows:
In AOSP/master make the following changes:
1. Temporarily set visibility on core-oj and core-libart to
//visibility:public.
2. Run packages/modules/common/build/mainline_modules_sdks.py to create
the snapshots.
For each of the S, T and latest snapshots I did the following in the
s-aml-prebuilt-test, t-aml-prebuilt-test and aosp/master branches:
1. Created an Android.bp file containing the following:
java_library {
name: "broken",
static_libs: [
"prebuilt_core-libart",
"prebuilt_core-oj",
],
}
2. Fix the visibility issues and run `m broken` where it fails with an
invalid file.
3. Delete the contents of the prebuilts/module_sdk/art/current/sdk
directory.
4. Unpack the relevant version of the art-module-sdk snapshot into the
directory.
5. Run `m broken` where it fails with the helpful message.
6. Test the instructions on how to use the ninja -t path tool to
identify the cause of the problem and fix it.
Bug: 257969510
Test: See above.
Change-Id: I125bde2d7202afff84c97daebcef37e21c548a3a
Previously, in a build containing source and prebuilt art
bootclasspath_fragments, the bootImageVariant.licenseMetadataFile was
set twice with the source always being set after the prebuilt and
so winning.
This change only sets bootImageVariant.licenseMetadataFile for the
active module so it will use the prebuilt's license file if that is
preferred.
Bug: 245956352
Test: m nothing
Change-Id: I948c7e5123169452f67c85ad98c4bbdb90a5d2de
Most of the fields in the bootImageConfig/Variant structs are assigned
inside a Once func so are guaranteed to be only set once. However, some
are assigned outside. This change adds comprehensive tests for those
structs and verifies that the constant fields are preserved and the
mutated fields have the correct value.
The check for the constant fields is added in a new TestBootImageConfig
test.
The check for the mutated fields is added into
TestSnapshotWithBootclasspathFragment_ImageName as that test checks an
art bootclasspath_fragment in the following configurations:
* source on its own
* prebuilt on its own
* source and prebuilt with source preferred
* source and prebuilt with prebuilt
It reveals a couple of interesting facts:
* All the *installs fields are set to the same value irrespective of
whether the source or prebuilt is preferred. The information is
constructed solely from information already within the
bootImageConfig/Variant and so can be moved within Once.
* The licenseMetadataFile is incorrect when prebuilt is preferred.
That is due to both the source and prebuilt modules setting it and
the source module always wins as the source module depends on the
prebuilt so always runs its GenerateAndroidBuildActions after it.
Those issues will be cleaned up in following changes.
Bug: 245956352
Test: m nothing
Change-Id: If917cfbcb3b1c842a8682d51cc1ee1fed1c51add
Bug: 249769908
Test: packages/modules/common/mainline_modules_sdks.sh
# With modification to add UpsideDownCake to list of releases
# to build with.
Change-Id: I050811e7d9a5d0a8cf54a42c03ad889defa3a251
The API finalization process for platform releases copies various files
from the apistubs/ dist directory into prebuilts/sdk/<n>/... Having
them use the same name in both places makes that process simpler. For
most modules the name of the file is derived from the name of the
module by appending a suffix but unfortunately, for some modules that
does not work. e.g. the conscrypt.txt file is produced by the
conscrypt.module.public.api module. The dist_stem property was added to
java_sdk_library to allow the stem name of the file to differ from the
module name.
The API finalization process for extension APIs does something similar
as it extracts various files from the snapshots and copies them into
the appropriate extension API specific directory in
prebuilts/sdk/extensions/<n>. Instead of copying files from the
apistubs/ dist directory (which is not built for trains) it copies them
from sdk snapshots that are built as part of the train.
Previously, the sdk snapshot used to derive the name of the files
within the snapshot from the name of the module, it ignored the
dist_stem property. This change causes it to use the dist_stem property
to make it consistent with the apistubs/ dist directory naming.
The file name is created in
sdkLibrarySdkMemberProperties.AddPropertyToSet() which does not have
access to the dist_stem property. So, it has to be supplied in the
sdkLibrarySdkMemberProperties instance in the new Stem property.
Bug: 248258460
Test: m nothing
BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh --build-release=latest
# Ran the previous command before and after and make sure that the
# conscrypt, art and icu sdk snapshots use the dist_stem value but
# none of the other snapshots are affected.
Change-Id: Ied52003de63dcdb86a252a39bb8781f85d51a6ff
Fields tagged with `sdk:"keep"` will keep the value even if it would
normally be cleared because it was common across a number of structs.
This will allow a module type to specify the same value across all
structs populated directly from variants and have it be copied into all
common property structs without clearing it from the variant specific
structs.
Bug: 248258460
Test: m nothing
Change-Id: I147d946b11fd8766a7d875d9206e8f5034f585d6
"ignore" is a better description of the effect that this tag has on the
optimization that is done as part of the sdk snapshot generation.
"keep" implies that something special is done with the field during the
optimization process but actually the optimization process completely
ignores the property, just as it does with unexported fields.
Bug: 248258460
Test: m nothing
Change-Id: Idfdfd87f927d641d4d301cbbde72b463c89155bf
When an sdk snapshot is targeted at release X then it cannot include
bootclasspath fragment libraries which are not present in that build as
otherwise it causes build failures. It should also not include any
unsupported libraries, i.e. libraries that cannot work on that release.
This change causes sdk snapshot to exclude libraries that have a
min_sdk_version > target build release
It also ensures that hidden API flags do not include any information
from excluded libraries.
Bug: 240406019
Test: BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Ran the previous command with and without this change to make
# sure that this change excludes framework-connectivity-t library from the
# tethering sdk snapshot for S, including from the hidden API flag files.
Change-Id: I57969b85a12e9e5a3fc76c055b260cec5d5f7d7f
Previously, the build releases only included named releases so did not
have a way to represent the latest build release. This adds the current
build release (named after the current API level) to represent that.
Bug: 240406019
Test: m nothing
Change-Id: Ib8336da716b447448b23bc9684ce3be1ab78648a
Previously, hidden API properties were only allowed on
bootclasspath_fragment and platform_bootclasspath module types. This
change allows them to be specified on java_sdk_library modules too. It
involves the following changes:
1. Add the properties to the java.Module.
2. Populate and provide a HiddenAPIPropertyInfo struct from
java_sdk_library modules.
3. Modify bootclasspath_fragment to merge information gathered from its
content libraries as if it was specified on the fragment itself.
Bug: 240406019
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
# Ran the previous command with and without this change to make
# sure that this change does not change the sdk snapshot
# contents.
Change-Id: I64eb71c2039ddc14cf380689d0cec7ec221f5b88
Previously, the platform_compat_config modules needed to be explicitly
listed in the sdk snapshot and the apex. This change will automatically
export them to the sdk snapshot when the apex is listed in its apexes
property.
Bug: 232401814
Test: m tethering-module-sdk
# Before this change the generated snapshot did not contain the
# platform_compat_config, after this change it did.
m art-module-sdk
# As that explicitly specifies the platform_compat_config in its
# compat_configs property this change has no effect.
Change-Id: Ia854b9a52db2b1619fca41a387ce98d7f9f9efe9
The order of the files in the sdk snapshot was stable but depended on
how the sdk was built. If the snapshot was created directly then the
files have one order, if it was created by merging in files from other
zips then it has a different order.
This change ensures files are in alphabetical order no matter how the
snapshot zip is constructed.
Bug: 232401814
Test: m media-module-sdk
# Check the contents of the following zips to ensure that they are sorted
# after this change.
# out/soong/.intermediates/packages/modules/Media/apex/media-module-sdk/common_os/media-module-sdk-current.unmerged.zip
# out/soong/mainline-sdks/media-module-sdk-current.zip
Change-Id: Ie97e0119c07a1f34a2b1d3ea6895f0e76cd195a8
While enabling prebuilts in T we hit b/229932396 which was caused by
some parts of the build depending on the prebuilt updatable-media jar
which used to be a full implementation jar but which is now an invalid
jar as the snapshot must not be including implementation details. We
fixed the issue in T but we are hitting the same problem in S with the
M-2022-07.
That is the first train in which the prebuilt updatable-media module
provides an invalid jar, prior to that it was always providing an
implementation jar. This change tweaks the sdk snapshot generation
code to use an implementation jar for updatable-media in the S
snapshot to avoid partners having to cherry pick changes similar to
those needed to fix b/229932396 in T.
Bug: 239121291
Test: packages/modules/common/build/mainline_modules_sdks.sh
# Check that S media snapshot includes implementation jar.
# Check that S art snapshot includes invalid jar.
# Check that T media snapshot includes invalid jar.
Change-Id: Ib49484d00a60b4ed7f8268e04f9c10a3498edb56
Specifying an apex in the apexes propety will cause all the
*classpath_fragments that are contents of the APEX to be automatically
added as members of the sdk and appear in the snapshot.
The purpose of this change is to dedup the APEX and sdk definitions and
try and avoid some of the issues that we have been finding while
attempting to build against the prebuilts.
Two tests, one each for bootclasspath_fragment and
systemserverclasspath_fragment, have been refactored to compare the
output when adding the *fragment to the sdk directly of via the APEX.
That ensures switching to use the APEX will not change the sdk snapshot
unless it was previously missing a *fragment.
There was also a slight difference in where the hidden API flags were
copied from. That should have no impact on the output as the flags are
identical.
The sdk snapshot generation needed some tweaks to avoid generating a
prebuilt for the APEX.
Bug: 232401814
Test: m nothing
Change-Id: I7aaf16a3a0ab4bebf97765d1484215cc008dc4b8
Previously, when targeting the S release the generated sdk snapshot
would contain prebuilt_systemserverclasspath_fragment modules even
though they were only added in T.
This allows SdkMemberTypes to specify the set of target build releases
they support and ignores them when targeting an unsupported target
build release.
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
# Check that the for-S-build snapshots do not include SSCPFs.
Bug: 237718221
Change-Id: I2df08c2fcebf9b866695d691572a9d3783758b17
Previously, the split_packages, single_packages and package_prefixes
properties were all optional and the split_packages defaulted to ["*"].
As that value conflicted with the other package properties that meant
that split_packages always had to be specified even if it was to just
set it to an empty array.
This change requires at least one of them to be specified and defaults
split_packages to an empty list which means it is not required,
although it can be helpful to make that explicit.
Bug: 194063708
Test: m nothing
Change-Id: I5a4c2d68e72e39f5c4a2441326dfce8685fc8ff2
Previously, the code for selecting specific versions of sdk snapshots
was removed (along with the uses_sdks property). That makes versioned
snapshots useless so this change removes all the code and tests that
generated those versioned snapshots.
Bug: 232546567
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
Change-Id: Ib6d1b72bc8399fbb39075494ae37da92f4b28d03
Currently, if the same library is specified for multiple of native_libs,
native_shared_libs, and native_static_libs for different arch/oses,
there can be a few errors:
1. specifying a .so file as `srcs` within a cc_prebuilt_library rather
than being specified only for shared
2. the final type of prebuilt library is dependent on the arch/os
This change introduces:
* an ability for a member type to override the type for specified
properties
* checks for a library being used with incompatible member types
* basing linkage nesting on the member type in addition to variants
This will ensure that the correct library type is used, regardless of
the order of iteration over oses/arches, and support nesting linkages
where necessary but only one linkage variant exists.
Test: soong tests
Test: CI
Change-Id: I81dee013b09b99c34ca6c18f9cfcc12ee56d33d1
Previously the SDK info file only contained basic common information
about each member. This change adds support for each member to
provide custom information to add to the info file.
It uses that mechanism to add the following:
* "dist_stem"
* "scopes" object containing:
* for each scope a:
"<scope>" object containing:
* "current_api" - the path within the snapshot for the API's .txt
file.
* "removed_api" - the path within the snapshot for the removed
API's .txt file.
* "latest_api" - the path within the build to the latest finalized
API .txt file.
* "latest_removed_api" - the path within the build to the latest
finalized removed API .txt file.
In order to access the latest API files it was necessary to add and
resolve dependencies on the module that makes them available. In order
to do that safely the code for creating the names of the modules was
refactored to avoid duplicating the name creation logic.
Bug: 204763318
Test: m nothing
Change-Id: Ica68abbd2b2c7c2b2b7877b502f96cc89f06fd68
Some build scripts need to know information about the contents of an
sdk, such as what APIs it provides (via java_sdk_library). Rather than
duplicate that information in the scripts or attempt to access that
information (where available) by looking at the contents of the
snapshot this change generates a JSON file that sits alongside the
snapshot itself.
The info file can be generated without generating the snapshot zip file
but whenever a snapshot zip file is generated the info is generated
too. The info file sits alongside the zip file in out/mainline-sdks.
Bug: 204763318
Test: m art-module-sdk
m dist
Change-Id: I289530bb21693dc6443826c24c17c9b5d85d2d8b
This change adds the build_number.txt file into the sdk snapshot to
make it easier to debug issues arising in partner builds caused by the
snapshots.
The file is added to the root of the snapshot and is called:
snapshot-creation-build-number.txt
Bug: 201295663
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
# Check the contents of various snapshots.
Change-Id: Ib3ac0cea5c9315cad0b9a3c25d88759097e37163
Make this consistent with how code names are used in the rest of Soong.
Bug: 204763318
Test: m nothing
Change-Id: I1f2f40310df9d619db6aaaad3ed7339bb5e79194
Marks the `scopeProperties.AnnotationsZip` property as only being
supported on T builds and above.
Bug: 204763318
Test: m nothing
Change-Id: Ie59396287c08da77a6a1f15d4be202429e011e17
Adds support for traversing into a field that is of type:
map[...]*struct{...}
This is needed to allow java_sdk_library to mark scope specific
properties, e.g. public.annotations as being target build release
specific.
It was necessary to change the Scope field from:
Scope map[*apiScope]scopeProperties
to:
Scope map[*apiScope]*scopeProperties
That is because there is no way in go to change the field of a struct
value of a map. i.e. you cannot do the following, not even using
reflection:
Scope[apiScopePublic].AnnotationsZip = nil
Bug: 204763318
Test: m nothing
Change-Id: Id103f70f55d4202971321ef4925cbec4b55f8136
Minor restructuring of the build_release pruning code to make it easier
to add logic for handling fields of maps of structs. That includes:
1. Moving some code that is specific to clearing a selected field
inside the associated if block.
2. Replacing an if with a switch.
3. Improving the error handling by separating the reporting of the
container that broke from information about which field could not be
set. That allows follow up code to provide information about the map
key instead.
The tests were restructed by:
1. Switching from using AssertDeepEquals to compare the structs to
comparing the output of marshalling the structs to JSON. That was
for a couple of reasons. Firstly, because JSON will marshal (and so
allow comparison of) the contents of pointers to structs whereas
AssertDeepEquals will just compare the pointers themselves.
Secondly, because JSON can pretty print the output and make it
easier to read.
2. Using a func to create a new instance of the input structure for
each test. That is to allow the test to modify the input structure,
e.g. by clearing a field in a struct that is pointed to by a map.
The test previously relied on the input structure being immutable
and passed by value but a follow up change will change that by
adding a map field that contains pointers to structs.
Bug: 204763318
Test: m nothing
Change-Id: I84dc99621497b7263e30466895b823eb02cb2b56
We will need the profile when we generate the primary boot image on
device.
Bug: 203492478
Test: Run `banchan com.android.art x86_64 && m` and see
`$ANDROID_PRODUCT_OUT/apex/com.android.art/etc/boot-image.prof`.
Test: Run `lunch aosp_cf_x86_64_phone-userdebug && m` and see both
`$ANDROID_PRODUCT_OUT/apex/com.android.art/etc/boot-image.prof` and
`$ANDROID_PRODUCT_OUT/system/etc/boot-image.prof`, in different
sizes.
Test: Start Cuttlefish with the built image and see both
`/apex/com.android.art/etc/boot-image.prof` and
`/system/etc/boot-image.prof` on device.
Change-Id: Id879dc49b234133dfbb9563814328661a1f4a6c0
Previous change that was reverted: https://r.android.com/1835222
An additional test was added that revealed a bug in the previous change
which has been fixed here.
Previously, the behavior of the stub_flags and all_flags properties
was different between S and T. In S they contained paths for the
complete set of stub flags and all the encoded flags. However, in T
they contained filtered sets of flags which if used in S would prevent
build checks from detecting possible inconsistencies. Also, a new
signature_patterns property was added in T that is not supported in S.
This change creates separate properties/files for T and reverts the
behavior of the properties/files that were added in S back to how they
behaved in S. The new properties are called filtered_stub_flags and
filtered_flags.
The S and T properties are tagged with the appropriate
supported_build_releases tag to ensure that they are only output when
specifically targeted.
Bug: 197842263
Test: m nothing
Change-Id: I1ce0a3d6623dabf73e32af1a7457b9b444fc3b7c
This was reverted unnecessarily along with the change that actually
broke the build.
Previously, the sdk snapshot was assumed to be generated for the
current build system. This change adds support for the
SOONG_SDK_SNAPSHOT_TARGET_BUILD_RELEASE environment variable which can
be used to select the target build release in which the generated
snapshot will be used.
At the moment this will only affect the properties that are output but
if/when it becomes necessary it can also be used to control more
aspects of the snapshots such as supported member types or even
members.
This change does not modify any properties that are build release
specific. That will come in following changes. However, it does add a
test that targets build release S which defines a baseline for
properties that will be affected in following changes. That baseline
makes it easier to see the effect of those follow up changes.
Bug: 197842263
Test: m nothing
Change-Id: If4b452237f105382550d2842c8010249afbc7432
Previously, the behavior of the stub_flags and all_flags properties
was different between S and T. In S they contained paths for the
complete set of stub flags and all the encoded flags. However, in T
they contained filtered sets of flags which if used in S would prevent
build checks from detecting possible inconsistencies. Also, a new
signature_patterns property was added in T that is not supported in S.
This change creates separate properties/files for T and reverts the
behavior of the properties/files that were added in S back to how they
behaved in S. The new properties are called filtered_stub_flags and
filtered_flags.
The S and T properties are tagged with the appropriate
supported_build_releases tag to ensure that they are only output when
specifically targeted.
Bug: 197842263
Test: m nothing
Change-Id: Iec8b9c539796c507245b69c0aed980fde6d8694f
Previously, the sdk snapshot was assumed to be generated for the
current build system. This change adds support for the
SOONG_SDK_SNAPSHOT_TARGET_BUILD_RELEASE environment variable which can
be used to select the target build release in which the generated
snapshot will be used.
At the moment this will only affect the properties that are output but
if/when it becomes necessary it can also be used to control more
aspects of the snapshots such as supported member types or even
members.
This change does not modify any properties that are build release
specific. That will come in following changes. However, it does add a
test that targets build release S which defines a baseline for
properties that will be affected in following changes. That baseline
makes it easier to see the effect of those follow up changes.
Bug: 197842263
Test: m nothing
Change-Id: I5cfde5e9e86af522b0d5030d301e116b0233fd62
