This is a followup to aosp/2999198 and adds information about apps.
Each app will have an entry in this file with the following properties
- Name, mandatory
- Is_prebuilt, mandatory
- Prebuilt_info_file_path, optional
Implementation details
- Move prebuiltInfoProvider out of build/soong/apex to
build/soong/android. This allows build/soong/java to use it.
- Introduce a new `prebuilt_info` prop to `android_app_set` and
`android_app_import`
- All app module types will set a prebuiltInfoProvider in
GenerateAndroidBuildActions, including the source app module types
Test: m nothing --no-skip-soong-tests
Test: m out/soong/prebuilt_info.json
Test: ls -l out/soong/prebuilt_info.json --human-readable
-rw------- 1 spandandas primarygroup 317K Mar 11 23:46 out/soong/prebuilt_info.json
Test: #modified trunk_staging.locally to select prebuilts of some
mainline apps. Spot-checked that `is_prebuilt` and
`prebuilt_info_file_path` get populated appropriately
Bug: 327480225
Change-Id: I5078e0ec26c9568194550909962b90111a5223f7
out/soong/prebuilt_info.json will contain information about whether
source or prebuilt of an apex was used. If prebuilt is used, it will
print the path to its prebuilt_info file. This file will be
used to pick a matching set of MTS test prebuilts.
Each apex will have an entry in this file with the following properties
- Name, mandatory
- Is_prebuilt, mandatory
- Prebuilt_info_file_path, optional
Implementation details
- Introduce a new `prebuilt_info` prop to `apex_set` and `prebuilt_apex`
- All apex module types will set a prebuiltInfoProvider in
GenerateAndroidBuildActions, including the source apex module types
- Create a `apex_prebuiltinfo_singleton` that visits all apex modules.
It uses `IsHideFromMake` to filter out the unselected variants of a
specific apex. This new singleton will create prebuilt_info.json
- Dist prebuilt_info.json for droidcore
Test: m nothing --no-skip-soong-tests
Test: m out/soong/prebuilt_info.json
Test: ls -l out/soong/prebuilt_info.json --human-readable
-rw------- 1 spandandas primarygroup 25K Mar 11 23:46 out/soong/prebuilt_info.json
Test: #modified trunk_staging.locally to select prebuilts of some
mainline modules. Spot-checked that `is_prebuilt` and
`prebuilt_info_file_path` get populated appropriately
Bug: 327480225
Change-Id: I65c73010142b034ad1d2b3d05ef493be034e8d74
There are a couple of instances in apex/ and java/ that rely on the naming
convention that the jars exported by prebuilt apexes follow the name of
the java_sdk_library_import, minus the prebuilt_ prefix. With muliple
module sdk prebuilts in trunk stable, this naming convention might not
be valid. Use `source_module_name` instead.
```
prebuilt_sscp_fragment {name: "", contents: ["service-foo.v2"]}
java_import {name: "service-foo.v2", source_module_name: "service-foo"},
```
We should use service-foo and not service-foo.v2 because
1. The prebuilt apex contains service-foo.dex
2. PRODUCT_*JARS will contain service-foo and not service-foo.v2
For clarity, this CL does not drop the current mechanism where prebuilt bcp
fragments create a dependency edge to prebuilt java or java_sdk_library
imports. There is still some discussion around whether we can remove
that completely (b/320711431). If we were to do that, then the
Android.bp files will become
```
prebuilt_sscp_fragment {name: "", contents: ["service-foo"]}
```
Bug: 322175508
Test: Added a unit test
Test: In internal, lunch cf_x86_64_only_phone-next-userdebug && m
nothing # .ninja files identical
Test: In internal, created a parallel set of v2 prebuilts of
java_sdk_library_import and prebuilt_bcp_fragments for adservices && m
nothing # build passes
Change-Id: Ia899d75e826fa1a559368d706eaa65835f748d40
For prebuilts, the dexpreopt rules of system server jars are now
generated from the context of the top-level prebuilt apex and not in the
context of the shim java_import modules. Since
`dex_preopt.profile_guided` property is defined in java_import, this
needs to be bubbled up to the top-level apex. This will be done using
deapxerInfo. If profile_guided of a transitive java_import is true, the deapexed .prof file will be
set as dexreopter.inputProfilePathOnHost before invoking
dexpreopter.dexpreopt. This ensures that only that java_import undergoes
profile guided dexpreopt, and not every other transitive java_import
Test: go test ./apex -run TestPrebuiltStandaloneSystemserverclasspathFragmentContents
Test: lunch cf_x86_64_only_phone-next-userdebug && m
$ANDROID_PRODUCT_OUT/system/framework/oat/x86_64/apex@com.android.art@javalib@service-art.jar@classes.odex
Test: du -sh
$ANDROID_PRODUCT_OUT/system/framework/oat/x86_64/apex@com.android.art@javalib@service-art.jar@classes.odex
24K
Bug: 308790457
Change-Id: Ibf46ecb400b3f126b243fc8d27b08d9a1aa4cc97
For prebuilts, java.dexpreopt gets called twice
1. in the context of the top-level prebuilt apex
2. in the context of the java_library shim
Only the artifacts of (1) get installed. The artifacts generated by 2)
are unused. However the args used to generate the artifacts of (1) are
incorrect. It uses moduleName(ctx) to do special-handling of apex system
server jars. This special-handling does not happen in (1), so although
dexpreopt artficats get generated, they get generated with the wrong
args.
To fix this, add an additional parameter in java.dexpreopt to pass the
libraryName explicitly.
Details
- Delete moduleName function. This was used to determine the java
library name, which is no longer safe
- Add a libraryName parameter to java.dexpreopt
- Most module types will use j.Name() as libraryName
- prebuilt_apex and apex_set will iterate over its `contents` and use
each element as libraryName when invoking java.dexpreopt
With the correct args passed from (1), we can drop j.dexpreopt from (2)
completely. Dropping (2) also breaks profile guided dexpreopt tests.
These currently operate on (2). They will be moved to (1) in the next CL
of this stack.
Test: presubmits
Test: lunch cf_x86_64_auto-trunk_staging-userdebug && m nothing (this
was a postsubmit failure with aosp/2923733)
Test: art_standalone_dexpreopt_tests on next https://android-build.corp.google.com/builds/abtd/run/L86000030001579256
Test: art-gtest on git_master-art-host https://android-build.corp.google.com/builds/abtd/run/L07800030001550262
Bug: 308790457
Bug: 322255144
Change-Id: I8eb604c82f1fa5289d3cd1a20084d56e4d7485e3
This provider contains information about the contents of an apex and is
set during the post deps phase of top-level source and prebuilt apexes.
java.dexpreoptDisabled needs to distinguish between platform and apex
variants of a library to prevent creating duplicate ninja rules. It does
so by looking at ApexInfoProvider. This provider gets set on the ctx of
the java_library being dexpreopted.
In case of prebuilts, these rules are being moved to the top-level
prebuilt apexes. Move ApexInfoBundleProvider from apex to android so
that java.dexpreoptDisabled can consume this provider without running
into a golang circular dependency. (java.dexpreoptDisabled will use this
provider in the next CL of this stack).
Test: go build ./android ./apex
Change-Id: I665abd9d0591eeb17b273cc822beb20690e6af09
(This was missed in aosp/2876756, which copied only the bcp jars)
This hardcoded location is used by dex2oat to compile the dexpreopt
artifacts. The copy rules are currently generated by java_(sdk)_library
for source builds, and their prebuilt counterparts in prebuilt builds.
After this change, the copy rule will be bifurcated between source and
prebuilt builds
1. For source builds, it will come from java_(sdk)_library
2. For prebuilt builds, it will come from the top-level prebuilt apex.
Since there can be multiple prebuilt apexes in trunk stable,
HideFromMake will be used to determine which deapexed jar to copy.
The bifurfaction is expected to be temporary. It is needed for now since
the `apex_contributions` which will be used for source vs prebuilt
selection have not been populated completely.
Test: Added a unit test
Test: Presubmits
Test: git_master-art-host:
art-gtest (https://android-build.corp.google.com/builds/abtd/run/L40800030001459791)
Test: git_main:art_standalone_tests
(https://android-build.corp.google.com/builds/abtd/run/L09000030001463855)
Bug: 308790457
Change-Id: I3105d3b3a7e5c41cb601d07806f4ea483a61b50a
Every module belonging to a single mainline module family will be
hidden from make, except the one which has been flagged using
apex_contributions
Details
- Introduce a new `source_apex_name` property to prebuilt_apex and
override_apex. This property will be used to identify the source
equivalent of a prebuilt soong apex module.
- Create an N-ary tree from source to prebuilt(s). The tree wil be
rooted at the source module.
- In a subsequent mutator, visit every node in the tree(s). Query
apex_contributions and store the handle of the node which is "active"
(if any)
- In the same mutator, do another pass over the tree. Invoke
`HideFromMake` on every node which is not "active". The two-pass
approach is needed PrebuiltSelectionInfoProvider does not know about
the inter source-prebuilt dependency, this dependency can only be
known by doing a graph walk of the N-ary tree.
Some tangential implementation details
- Each prebuilt apex has an internal deapxer module that is responsible
for generating the deapex ninja rules. The name of this internal
module uses the BaseModuleName (without the prebuilt_ prefix). Since
we can have multiple prebuilt soong modules in trunk stable, change
this to follow the name of the prebuilt module in order to avoid name
collisions. Update existing unit tests accordingly
Bug: 316179314
Test: go test ./apex -run TestInstallationRulesForMultipleApexPrebuilts
Test: m nothing --no-skip-soong-tests
Test: presubmits
Change-Id: I58aa99d5e6a9859954614e6db9a8e9e2e581642d
hiddenapi and dexpreopt require boot and system server jars from apexes.
When building with prebuilts, this comes via
java_import/java_sdk_library_import, which acts as a hook for
prebuilt_apex/apex_set. If we have multiple apexes in the tree, this
hook becomes 1:many. This CL prepares dex_bootjars to select the right
deapexerd .jar files when mutliple prebuilts exist.
Implementation details
- Update prebuilt module types (prebuilt_apex/apex_set) and source
apexes to set a map of
library name to dex jar path on host.
- dex_bootjars will access the path of the .dex jar on host via the
provider. These then
copied/installed to the right locations.
This CL does not drop the old mechanism to get the dex file (i.e. by
creating a dep on java_library). Once all mainline
modules have been flagged using apex_contributions, the old mechanism
will be dropped
Bug: 308790457
Test: git_master-art-host:art-gtest https://android-build.corp.google.com/builds/abtd/run/L21500030000926533
Test: git_main:art_standalone_dexpreopt_tests https://android-build.corp.google.com/builds/abtd/run/L99000030000891212
Test: Added a unit test that checks that the right .jar is selected
when multiple prebuilts exists
Change-Id: I6ef94135b9303a35135810930af4b641df13a583
Generating boot image requires a .prof file provided by the ART apex.
When building with prebuilts, this comes via the
prebuilt_bootclasspath_fragment module, which acts as a shim for
prebuilt_apex/apex_set. If we have multiple prebuilt apexes in the tree,
this shim becomes 1:many. This CL prepares dex_bootjars to select the
right .prof file when multiple prebuilts exist.
Implementation details
- Update deps mutator of dex_bootjars to create a dep on
all_apex_contributions. The latter contains information about which
apex is selected in a specific release configuration. dex_bootjars
will create a dependency on the selected apex in a postdeps phase
mutator.
- All apex module types (apex, prebuilt_apex and apex_set) will set a
provider that contains info about the location of the .prof file on
host
- dex_bootjars will access the provider of the selected apex to get the
location of the .prof file
This CL does not drop the old mechanism to get the .prof file (i.e. by
creating a dep on {prebuilt_}bootclasspath_fragment). Once all mainline
modules have been flagged using apex_contributions, the old mechanism
will be dropped
Bug: 308790457
Test: Added a unit test that checks that the right .prof is selected
when multiple prebuilts exists
Change-Id: I40fdb21416c46bed32f6ff187ce5153711ec2c69
FindDeapexerProviderForModule raises an exception if multiple apexes in
the tree has an export dep on the java module. In prepartation to
support multiple prebuilts, move this error check out of
FindDeapexerProviderForModule and into rdeps. i.e. raise an exception
only if an rdep calls DexJarBuildPath
- This should be a no-op for now.
- In the short-term future, a java import module will be allowed to have
multiple deapexers. An error will be raised if anyone actually tries
to depend on the dexjar
- In the long-term future, this function will be removed. All processing
will be done at the prebuilt apex level and not at the prebuilt java
library level
Since this check now happens in the moduleCtx of rdeps, add some
additional props to unit tests to ensure that it does not exit early on
unrelated validation checks (e.g. hidden_api prop is not set)
Test: go test ./apex ./java
Bug: 308790457
Change-Id: I3323d993c1ea8f43305834cae8e65b6fe41dfefd
dexpreopt of apex system server from prebuilts involves three soong
modules
1. prebuilt_apex / apex_set
2. an internal deapexer module created by the prebuilt apex
3. java_import/java_sdk_library
(3) acts as a shim for the deapexer to set the dexjar extracted from the
prebuilt apex. This methodolody requires a 1:1 correspondence across the
three modules
This breaks down when we have multiple versions of the same prebuilt
apex in the tree. In preparation for this, move the dexpreopt
processing from (3) to (1). Each prebuilt_apex will create the necessary
rules for dexpreopting the jars deapexed from itself. In the future,
apex_contributions will be used to pick which service-foo.{odex|.vdex} to
install depending on which prebuilt apex is selected.
Implementation details
- Embed dexpreopter in prebuiltApex structs so that this module type can
register the dexpreopt rules. Since a single apex can have multiple
system server jars, this also requires creating an additional scope in
dexpreopt.go to prevent name collisions
- Add the dexpreopt modules as required in initApexFilesForAndroidMk
- Add the depreopt modules to androidMk in AndroidMkEntries. Drop the
equivalent from java_import and java_sdk_library_import
Bug: 308790457
Test: existing soong unit tests
Test: lunch cf_x86_64_phone-next-userdebug && m out/target/product/vsoc_x86_64/system/apex/com.google.android.adservices.apex
Test: Verified that the above command installs
/out/target/product/vsoc_x86_64/system/framework/oat/x86_64/apex@com.android.adservices@javalib@service-adservices.jar@classes.{odex|vdex} and the equivalent files of service-sdksandbox
Test: presubmits
Change-Id: I01cea8956d2857fb864b415e73d3d2686d069b5e
Convert all of the callers of SetProvider to use the type-safe
android.SetProvider API.
Bug: 316410648
Test: builds
Change-Id: If58f4b5355264ddab2045bc3591a4eac19cd58fc
Installed files should only depend on other installed files, change
the deps arguments of the ctx.Install* methods from Paths to
InstallPaths.
Bug: 311428265
Test: builds
Change-Id: I1ebef60a943bdbe907744cc43aa985371ac56d32
Merged-In: I1ebef60a943bdbe907744cc43aa985371ac56d32
Instead of listing all apexes in the source tree, now each apex emits
its own fragment for apexkeys.txt, which is pointed by
LOCAL_APEX_KEYS_FILE. Makefile collects apexkeys.txt from installed apex
files. This is to avoid listing unrelated apexes (not installed,
testdata, unexported namespaces, etc.)
Bug: 304914238
Test: m apexkeys.txt
Test: m blueprint-tests
Change-Id: Iefbe6e486cb418955584ad1a282455307e90be95
Since we no longer support "zip" APEX, we don't need "host" support.
For example, we don't need go/python binary support.
Bug: 279835185
Test: m
Change-Id: I6e8d2b205e42662f31866dc9ac7507524effd144
We want to make FULL_SYSTEMIMAGE_DEPS more accurate so that we can
remove the need for `m installclean`.
Currently, the compat symlinks are installed by adding them as
dependencies of other installed files. Doing this doesn't end up
propagaing the compat symlinks all the way to FULL_SYSTEMIMAGE_DEPS.
Add them to LOCAL_SOONG_INSTALL_SYMLINKS, which will cause them to
be propagated to FULL_SYSTEMIMAGE_DEPS.
Bug: 205632228
Test: Full build with a change to delete files not in FULL_SYSTEMIMAGE_DEPS from the staging dir
Change-Id: Idc7fc0a0dbf7c8ac3f0628c16413cd1df69a7821
Revert submission 2718295-colefaust_track_apex_compat_symlinks
Reason for revert: To validate if this change is causing the build breakage.
Reverted changes: /q/submissionid:2718295-colefaust_track_apex_compat_symlinks
Change-Id: I1896530219cf984efdf9ca7f4b14d8784367f242
We want to make FULL_SYSTEMIMAGE_DEPS more accurate so that we can
remove the need for `m installclean`.
Currently, the compat symlinks are installed by adding them as
dependencies of other installed files. Doing this doesn't end up
propagaing the compat symlinks all the way to FULL_SYSTEMIMAGE_DEPS.
Add them to LOCAL_SOONG_INSTALL_SYMLINKS, which will cause them to
be propagated to FULL_SYSTEMIMAGE_DEPS.
Bug: 205632228
Test: Full build with a change to delete files not in FULL_SYSTEMIMAGE_DEPS from the staging dir
Change-Id: I0600ccf8d863d5e29e044c82293ece730fc98d43
We don't need this anymore because we are going to compile ART jars and
framework jars together.
Bug: 280776428
Test: m
Change-Id: I070157530449a1bb5779e25984c367df3dde7b36
Add skip-sdk-check to skip checking the SDK version when extracting an
APK/APEX from an App Set Bundle. This can be used when the platform SDK
version is not defined and the APEXs/APKs use SHA based SDK versions.
This check should not be set to true for non Beta dessert releases
Bug: 274518686
Test: # Add SHA targeting modules to platform
m SOONG_SKIP_APPSET_SDK_CHECK=true #Build Passes
m SOONG_SKIP_APPSET_SDK_CHECK=false #Build Fails
m #No config supplied, build fails
Change-Id: I1919437d3410f09c991e1de39031bd88e1f8246a
Generally, when building system images from prebuilt APEXes the APEXes
should be released versions and the build ensures that is the case when
extracting .apex files from prebuilt .apks files. However, it is useful
to test building against prebuilt APEXes before finalization to flush
out any issues early on. Previously, the build would fail in that case.
This change adds support for using the SOONG_ALLOW_PRERELEASE_APEXES
environment variable to allow prerelease APEXes to be suppported.
Bug: 259958590
Test: m MODULE_BUILD_FROM_SOURCE=false droid
# Above fails because it could not find non-prelease APEXes in the
# prebuilt .apks files.
m SOONG_ALLOW_PRERELEASE_APEXES=true MODULE_BUILD_FROM_SOURCE=false droid
# Above works.
Change-Id: I20b8e5db0b9fd2a21434be25652195fae929bd3f
The apex and prebuilt_apex module types make the output APEX available
using ":<apex>" syntax, e.g. for use as a data file in a test. This
change adds compatible behavior to apex_set to ensure consistent
behavior when building against prebuilts.
Bug: 259958590
Test: m MODULE_BUILD_FROM_SOURCE=false nothing
Change-Id: Ic6f105976cfdd6af4920cc1e11ab0b4ff72e6a6f
This leads to an arm64 apex being used on a device that is mixed
x86_64 with nativebridge=arm64. A device like that doesn't appear
to work with arm64 binaries. For example, the boringssl-self-check
binary crashes on boot.
Bug: 260115309
Test: unit test
Test: boot emulator with this combination
Change-Id: Ic4a91974290a05b1799f755fcf52ef226d68f4c2
There are no riscv64 apex prebuilts, which breaks the build when
provenance_metadata singelton tries to run a tool on all the input
apexes that don't exist. Hack it to provide the arm64 apex for now
if no riscv64 apex is specified, which will allow the build to pass.
Test: lunch aosp_riscv64-userdebug && m droid
Change-Id: Ic39936539803615ea4b7f817daf7b59ab7c40e7e
This add support for src field in bp file.
Signed-off-by: Chen Guoyin <chenguoyin.cgy@linux.alibaba.com>
Signed-off-by: Mao Han <han_mao@linux.alibaba.com>
Change-Id: I9393696951db375d53436f9d5a9841b6a072cea3
Arch-specfic prebuilt_apex modules may be missing prebuilts for new
architectures. Mark the module as missing dependencies and allow
build analysis to continue when AllowMissingDependencies is set.
Bug: 250918230
Test: lunch aosp_riscv64-userdebug && m ALLOW_MISSING_DEPENDENCIES=true nothing
Change-Id: I79f0c20a0c9443fe443f9ed9bc846aa649f6b6a6
This allows apex_set's set config to use other soong modules as a dependency
Bug:249169376
Change-Id: I8da20c6f474328d63d9aa3811f94e3e04b89ea4c
Test: Added a test case to check soong module assignment to the set field of an apks_set rule
This change is similar to aosp/1947127, but for prebuilts.
After this change, if `bootImageConfig.installDirOnDevice` is set to a
path outside of the APEX, the build system will build a boot image from
the dex files and the profile extracted from the prebuilt APEX.
Otherwise, it keeps the current behavior: extracting the boot image from
the prebuilt APEX.
This is a no-op change. Current behavior is not affected.
Bug: 211973309
Test: m nothing
Test: -
On internal master:
1. Patch aosp/1947128.
2. Patch ag/16743847 and ag/16746804.
3. m SOONG_CONFIG_art_module_source_build=false com.google.android.art
4. See the boot image being installed in `/system/framework/<arch>`.
Change-Id: I24ca525309fecaf3ab7a67960fbf118cd00ecd1d
apex_set modules failed to set prebuiltCommon.installedFile, which
caused LOCAL_SOONG_INSTALLED_MODULE to be empty and base_rules.mk
to create a duplicate install rule. Set it to the path to the
apex file installed by Soong.
Bug: 204136549
Bug: 209867137
Test: m SOONG_CONFIG_art_module_source_build=false
Change-Id: Ia7fec09598823343242ebd44f1732e6bba21d027
Move apex module installation rules into Soong by overriding
InstallBypassMake.
Soong installs don't support post install commands, so move the
symlinks into separate rules and add dependencies on them.
This relands If65d283abc86f18ad266da0bf16fe95971a0bf9c with fixes
to install files into $OUT/apex and I606286e971b55d9d1fc4dcd0fbd476962de5fa79
with a fix for reversed logic.
Bug: 204136549
Test: m checkbuild
Change-Id: Ie65c53ebc7911efacdb8e11ba49059448f03c658
Move apex module installation rules into Soong by overriding
InstallBypassMake.
Soong installs don't support post install commands, so move the
symlinks into separate rules and add dependencies on them.
Bug: 204136549
Test: m checkbuild
Change-Id: If65d283abc86f18ad266da0bf16fe95971a0bf9c
A prebuilt_apex/apex_set module can make a dex implementation jar
available for use by Make targets but it does not have access to a real
classes implementation jar; the jar in the corresponding sdk is just an
empty file. Exporting that invalid file to Make means that the phony
target created for the module depends on an invalid classes jar which
causes the build to fail.
This change removes the logic that make classes and header jars
available to Make so that building the phony target no longer fails.
Bug: 202521005
Test: m TARGET_PRODUCT=armv8 TARGET_PRODUCT_VARIANT=eng TARGET_BUILD_UNBUNDLED=true SOONG_ALLOW_MISSING_DEPENDENCIES=true core-icu4j
- above was performed in master-art.
Change-Id: If667b36b988e2d331acd85f60f3369778838d034
This is for exporting the contents of systemserverclasspath_fragment for
dexpreopting.
Bug: 194150908
Test: manual - 1. Patch aosp/1818020 and aosp/1834534
2. m SOONG_CONFIG_art_module_source_build=false com.android.art
Change-Id: I7d2d2e02869d8a523f7c0efbbff81706672a95c5
