Previously, the patterns in SYSTEM_OTHER_ODEX_FILTER only matched
subpaths of /system. Broaden this to match /<p> as well as /system/<p>.
Because SYSTEM_OTHER_ODEX_FILTER includes system_ext and product apps,
the immediate effect of this is that the odex and vdex files of
apps on those partitions move to system_other for A/B devices.
This makes the product and system_ext partitions smaller, which
in turn makes OTAs smaller.
Bug: 141707536
Test: soong unit tests
Test: (adb shell dumpsys package com.android.stk &&
adb shell dumpsys package com.android.quicksearchbox) | grep -i dexopt -A3;
verify dexpreopt state is 'prebuilt' for both
Test: adb shell find /data/dalvik-cache -name '*QuickSearch*.vdex' | wc -l == 1
Test: adb shell find /data/dalvik-cache -name '*Stk*.vdex' | wc -l == 1
Change-Id: I0cf141014b9ddf32159cff47b0325d9c78b549f2
This doesn't affect the cases with explicitly set `uses_libs` or
`optional_uses_libs`, only the cases when the library implicitly added
to the manifest_fixer by Soong. "org.apache.http.legacy" should be
handled in the same way as "android.test.base", except that they have
been added in different SDK versions.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I4fa1f082bfdb67f227f432717731a7e3d790913c
Previously default install path was provided only for comatibility
libraries that are added explicitly via `uses_libs`/`optional_uses_libs`
properties. This didn't take into account compatibility libraries that
are added by Soong when it computes transitive closure of SDK library
dependencies. As a result, install path in such cases remained unknown,
and the corresponding compatibility library was omitted from class
loader context, which caused 'ClassLoaderContext shared library size
mismatch' errors at first boot on device.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: If661272f46803299c8640f7f5ef161c8217d07e4
Allow default install paths only for compatibility libraries. For other
libraries that are explicitly specified in `uses_libs` and
`optional_uses_libs` unknown install path should be an error.
Bug: 132357300
Test: lunch cf_x86_phone-userdebug && m
Change-Id: I2209c90a939a8aa46c42e13bb42d09c07e4d0895
For some dependencies, like stubs, the SDK library may not be found at
build time (either because the implementation library is not among the
dependencies of the dexpreopted module, or because it's part of a
prebuilt, or because it's missing from the build altogether). In such
cases dexpreopt is useless, because dex2oat does not have access to the
full classpath (unless the &-classpath is used). Therefore do not
dexpreopt in such cases.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: If289088cfd103011ccb16165e95a97b30fd31b81
Add a new helper SortedIntKeys similar to SortedStringKeys.
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I08a43ec2cae7d1a82531295aca1a0658e3a0dd6f
Previously only the names were collected, and later used in the
manifest_fixer to add missing <uses-library> entries to the manifest.
Now we also need to collect build-time and on-device paths, to be used
in class loader context for dexpreopt. This commit only collects paths,
but does not pass them to dexpreopt yet.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I34b229ee68f16ba215ba03770feadb4d890ec2bf
With the addition of apexes and /system_ext some of the bootclasspath
and system server jars have moved from /system to the new locations.
This has been implemented by using lists of colon-separated strings
called "apex-jar pairs" (although "apex" was misleading as it could
refer to "platform" or "system_ext", not necessarily a real apex).
Using the colon-separated string representation is inconvenient, as it
requires splitting and reassembling the list components many times,
which harms performance and makes error handling difficult. Therefore
this patch refactors the colon-separated lists into a struct that
hides the implementation details.
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: Id248ce639a267076294f4d4d73971da2f2f77208
This allows to reuse SDK version comparison routine from other Python
scripts. With the addition of non-numeric versions comparison has become
more complex, and the deleted shell script did it incorrectly: it used
comparisons like `[[ "S" -lt 28 ]]` which results in "true" as strings
are converted to zero in numeric context. This resulted in adding legacy
libraries to class loader context of apps targeting recent SDK versions.
The error was masked because currently there is only one non-AOSP app
that uses the script (GoogleDialer), and it targets numeric SDK version.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I26e752e29b5453fd3e29d5dbfbe4d9df9c0a55b7
There are essentially three different cases: 1) system server dexpreopt,
2) the case when uses-library information is known, and 3) the case when
uses-library information is unknown and the &-classpath hack is used.
This patch reorganizes the code into three branches that correspond to
the above cases.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: Ife2d01a612e0a608ae1346d7007741498e938bc5
This script belongs with other manifest-related scripts, and the future
plan is to rewrite it in Python and share common functionality with
other scripts.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I4b90129e5023ca1a2c818fc160c34b0b7da943ef
Different names of the same field in Make and in Soong resulted in
the field being dropped when unmarshalling the JSON from Make to Soong.
As a result, the list of optional used libraries for Make modules was
always empty.
Test: lunch aosp_cf_x86_phone-userdebug && m
Test: Cherry-pick in internal master and observe that the logcat error
"ClassLoaderContext shared library size mismatch" for GoogleDialer
has disappeared:
terminal-1 $ lunch cf_x86_phone-userdebug && m && launch_cvd
terminal-2 $ adb wait-for-device && adb root \
&& adb logcat | grep 'size mismatch'
(nothing after the change, one error message before the change)
Bug: 132357300
Change-Id: I57820646fa7c1d8af01b2cbc8a3521da450719c0
This is a preliminary CL before fixing on-device paths to DEX jars.
It groups together the inormation about on-host build paths and
on-device install paths to library DEX jars.
This CL changes the structure of module dexpreopt.config files
generated by the build system. Aside of that, no functional changes.
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I059654be7670f2ba66248d9c49b9694a0591f9c1
The order is non-optimal if some jar X precedes its dependency Y. In
that case dexpreopt will be unable to resolve any rerefences from X to
Y. Raise an error unless the product sets variable
PRODUCT_BROKEN_SUBOPTIMAL_ORDER_OF_SYSTEM_SERVER_JARS to true in the
makefile. This is to prevent regressions in existing products that
currently have correct order.
Test: lunch cf_x86_phone-userdebug && m nothing
Test: lunch aosp_car_arm && m nothing
Bug: 140451054
Change-Id: Iafe8fe0e992deb628e38f2321d4601a4804a3c79
Merged-In: Iafe8fe0e992deb628e38f2321d4601a4804a3c79
(cherry picked from commit fb863c87c4)
Deduplicate code for conditional class loader contexts that depend
on the target SDK version in the manifest.
Test: lunch aosp_cf_phone-userdebug && m
Change-Id: I69d51b212bf502b2659d43f124ea3922f90e1881
This library no longer exists on device and "org.apache.http.legacy"
should be used instead. The error lingered in the code for a while
because the code was not exercised at all. (The only app that sets
EnforceUsesLibraries is GoogleDialer (not in AOSP), and it specifies
"org.apache.http.legacy" explicitly as an optional used library.)
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I05fa115476315d35a8877fc42bd1c2042f72f2de
The first component is the apex name, or a special name "platform"
if the boot jar is a platform jar rather than a part of some apex.
This is a prerequisite change for moving core-icu4j to a separate
com.android.i18n apex.
Old one-column format is still supported, but all unqualified
components of PRODUCT_BOOT_JARS get "platform:" prepended to them
after reading the product makefiles.
Test: aosp_walleye-userdebug boots
Bug: 138994281
Change-Id: Ic229159fdcdaf6182210a53b63850a389dd786fc
Previously there was a discrepancy between the generated paths for
Soong modules that used "out/soong/system_server_jars" and Make modules
that used "out/system_server_jars". This happened because the default
output directory is $OUT/soong for the normal Soong config and jsut $OUT
for the reduced "null config" created by dexpreopt_gen.
As a result, class loader context for system server jars defined in Make
was referring to nonexistent jars, which caused non-fatal dex2oat errors
(the build was not broken, so the errors remained unnoticed):
Failed to determine oat file name for dex location out/system_server_dexjars/[...]
Could not open dex files from location: out/system_server_dexjars/[...]
Test: aosp_walleye-userdebug boots
Test: cherry-picked in master-arc-dev that has a Make system server jar
arc-services, `lunch lunch cheets_x86-userdebug && m`, check that the
generated dexpreopt.sh for arc-services contains paths starting with
"out/soong/system_server_jars" rather than "out/system_server_jars".
Bug: 140712566
Change-Id: Ia7ea8ac383e32042c31d64971cdc8101ea3068cd
Soong already uses build directory as a part of the generated output
path. Besides, trying to append it second time breaks if the build
directory is an absolute path.
Test: frameworks/rs/build_rs.py (uses absolute build directory path)
Test: walleye-userdebug boots
Bug: 152235239
Change-Id: Ie03c9e688013b9a3e6bc859c936d89c538ded76a
Merged-In: Ie03c9e688013b9a3e6bc859c936d89c538ded76a
(cherry picked from commit f798c3ec32)
The null pointer dereference was caused by trying to get device name
from a config passed to dexpreopt. For Make modules, the config is a
stub that doesn't have a device name. The fix is to use build directory
instead of device name (it is provided both by the full config passed
to Soong modules and by the stub config passed to Make modules).
Test: walleye-userdebug boots.
Test: aosp_bertha_arm-userdebug builds and generates arc-services.odex.
Bug: 152193534
Change-Id: I7adc173646cae6eaee188c6672629a5a942aea06
Merged-In: I7adc173646cae6eaee188c6672629a5a942aea06
(cherry picked from commit 9790f1058b)
This reworks CL https://r.android.com/1180134 as follows:
1) Do not reorder the list of system server jars passed from Make to
Soong via the product variable PRODUCT_SYSTEM_SERVER_JARS. This means
that for some products the order of jars on the system server
classpath may be non-optimal: a jar X that depends on Y may be
dexpreopted before Y, so that all references to the classes and
methods from Y wil be unresolved.
Unfortunately for such products, fixing the order is not a simple
matter of rearranging their PRODUCT_SYSTEM_SERVER_JARS, because the
conflicts may arise when the product-specific variable gets merged
with the common variable.
2) As a consequence of 1), do not add artificial dependencies between
system server jars: this is now impossible, as it would create
circular dependencies for those products that have non-optimal order
of jars.
3) Copy dex files for system server jars to a predefined build location.
This is necessary because at the time when Soong generates class
loader context for k-th jar, it needs to know the paths to jars 1 ..
(k-1), and it might have not processed those jars yet (so Soong can't
query the paths from the modules).
This approach is similar to the way Soong handles bootclasspath jars.
4) Do not exclude from dexpreopting system server jars that are not
Soong modules (those that are Make modules). The previous CL excluded
them because Make modules do not have ModuleContext. But it turns out
that ModuleContext is not necessary, as all the information is passed
via the dexpreopt config.
Test: aosp_walleye-userdebug boots and there are no messages in the
logcat regarding class loader context mismatch:
$ adb logcat | grep 'mismatch'
# empty
Test: Class loader contexts in the oat files for system server jars
match expectations:
$ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/com.android.location.provider.odex 2>/dev/null | grep '^classpath'
classpath = PCL[]
$ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/services.odex 2>/dev/null | grep '^classpath'
classpath = PCL[/system/framework/com.android.location.provider.jar*1989208671]
$ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/ethernet-service.odex 2>/dev/null | grep '^classpath'
classpath = PCL[/system/framework/com.android.location.provider.jar*1989208671:/system/framework/services.jar*4040443083:/system/framework/services.jar!classes2.dex*2210087472]
Test: The phone boots and logcat has no scary messages related to
class loader contexts:
$ lunch aosp_walleye-userdebug && m
$ adb reboot bootloader && fastboot flashall -w && adb wait-for-device
$ adb root
$ adb shell stop
$ adb logcat -c
$ adb shell setprop dalvik.vm.extra-opts -verbose:oat
$ adb shell start
$ adb logcat | egrep -io 'system_server: .*load.*/system/framework.*'
system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 0
system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 0
system_server: Loading /system/framework/oat/arm64/services.odex with executable: 0
system_server: Successfully loaded /system/framework/oat/arm64/services.odex with executable: 0
system_server: Loading /system/framework/oat/arm64/ethernet-service.odex with executable: 0
system_server: Successfully loaded /system/framework/oat/arm64/ethernet-service.odex with executable: 0
system_server: Loading /system/framework/oat/arm64/wifi-service.odex with executable: 0
system_server: Successfully loaded /system/framework/oat/arm64/wifi-service.odex with executable: 0
system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
system_server: Loading /system/framework/oat/arm64/services.odex with executable: 1
system_server: Successfully loaded /system/framework/oat/arm64/services.odex with executable: 1
system_server: Loading /system/framework/oat/arm64/ethernet-service.odex with executable: 1
system_server: Successfully loaded /system/framework/oat/arm64/ethernet-service.odex with executable: 1
system_server: Loading /system/framework/oat/arm64/wifi-service.odex with executable: 1
system_server: Successfully loaded /system/framework/oat/arm64/wifi-service.odex with executable: 1
system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
Bug: 141785760
Bug: 140451054
Bug: 148944771
Bug: 147017252
Change-Id: I33c4087f8bfacd0ecb89877aa150b75360d06710
Merged-In: I33c4087f8bfacd0ecb89877aa150b75360d06710
(cherry picked from commit a4a83b0ef9)
Exempt-From-Owner-Approval: cherry-pick.
Rename the two prefix-in-list funcs so that their usages are clearer.
Also find and replace all the code that essentially does either.
This introduces additional loops in some places, but I think the added
readability and simplicity outweighs the performance degradation, which
should be negligible anyway.
Test: m nothing
Test: TreeHugger
Change-Id: I37e2276ca0c815105ed0031f23c0b1264b480e4f
Also disable dexpreopting for host. These are necessary to avoid adding
dependencies on dex2oat in various non-platform builds where they will
break.
Since we cannot assume at least one module enables dexpreopting now,
the two dexpreopt singletons are silently disabled if there has been no
call to dexpreopt.GetGlobalSoongConfig.
Bug: 145934348
Bug: 148312086
Bug: 148319588
Bug: 148690468
Test: m
Test: env OUT_DIR=out-tools prebuilts/build-tools/build-prebuilts.sh
on the aosp-build-tools branch
Test: build/soong/soong_ui.bash --make-mode static_sdk_tools dist DIST_DIR=out-dist BUILD_HOST_static=1
on internal (cf b/148312086#comment8)
Test: build/soong/soong_ui.bash --make-mode dist DIST_DIR=out-apps TARGET_BUILD_APPS=Launcher3 TARGET_BUILD_VARIANT=userdebug
on internal without art/ and external/vixl/ (cf b/148319588)
Change-Id: I240dade7204b87fc2d12181534ab23439eca8b46
Should cut down on a bit of copying, and also required for an upcoming
CL that'll change GetCachedGlobalSoongConfig.
Test: m nothing
Bug: 145934348
Change-Id: I6bed737d9b061b5239cc603ad881f4ccd4312e43
binary module.
This uses the Once cache for GlobalSoongConfig to propagate the dex2oat
path from a module dependency to the singletons (both the one that
writes out dexpreopt_soong.config and the one that creates the
dexpreopted boot images). Unless dexpreopting is disabled altogether
through DisablePreopt in dexpreopt.config, that means:
- We must ensure at least one module registers a dex2oat tool
dependency and resolves a GlobalSoongConfig using it, or else the
singletons fail. That means we litter dex2oat dependencies in java
modules even when they won't get dexpreopted and hence don't really
need them.
- We still assume there's at least one java_library or android_app in
the build.
This relands https://r.android.com/1205730 without changes - the
necessary fixes are in the child CLs.
Bug: 145934348
Test: m
(check that out/soong/dexpreopt_soong.config points to dex2oatd64)
Test: env USE_DEX2OAT_DEBUG=false m
(check that out/soong/dexpreopt_soong.config points to dex2oat64)
Test: env OUT_DIR=out-tools prebuilts/build-tools/build-prebuilts.sh
on the aosp-build-tools branch
Change-Id: I66661711b317d1e4ec434861982919bdde19b575
dexpreopt package.
Preparation for a future CL that will need to get the make-written
GlobalConfig from within dexpreopt.
Also rename the Load*Config functions to Parse*Config, since they don't
actually load the config files anymore.
This relands https://r.android.com/1211982.
Bug: 145934348
Test: m
Change-Id: Icb9332a93811d77d2d8b06e983b92501b180a358
caching of it.
Introduce a Once cache for GlobalSoongConfig to allow it to get binary
tool paths from ordinary module dependencies (coming in a future CL)
that are then reused in singletons.
This relands https://r.android.com/1205729.
Bug: 145934348
Test: m
Change-Id: I039d6e204bee5ddc16d8e2d85057fbec20e326fe
This patch excludes non-Soong system server jars from dexpreopting.
System server jars should be dexpreopted together for better
performance: each jar should have all preceding system server jars
in its class loader context (that is passed to dex2oat and recorded
in the resulting .oat file to be used by the loader).
This means that we need a total order on system server jars. The
default order provided by PRODUCT_SYSTEM_SERVER_JARS is not good, as
it does not always respect genuine dependencies between jars (counter-
examples are rare, but non-trivial to fix: b/148219586).
This patch adds a post deps mutator pass that creates additional
dependencies and enforces global order. These are later used to
generate precise class loader contexts and system server classpath.
Test: Class loader contexts in the oat files for system server jars
match expectations:
$ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/com.android.location.provider.odex | grep '^classpath'
classpath = PCL[]
$ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/services.odex | grep '^classpath'
classpath = PCL[/system/framework/com.android.location.provider.jar*1919890654]
$ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/wifi-service.odex | grep '^classpath'
classpath = PCL[/system/framework/com.android.location.provider.jar*1919890654:/system/framework/services.jar*4269704903:/system/framework/services.jar!classes2.dex*134345935]
...
Test: The phone boots and logcat has no scary messages related to
class loader contexts:
$ lunch aosp_walleye-userdebug && m
$ adb reboot bootloader && fastboot flashall -w && adb wait-for-device
$ adb root
$ adb shell stop
$ adb logcat -c
$ adb shell setprop dalvik.vm.extra-opts -verbose:oat
$ adb shell start
$ adb logcat | egrep -i 'system_server: .*load.*/system/framework'
02-03 14:14:26.912 5016 5016 I system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
02-03 14:14:26.914 5016 5016 I system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
02-03 14:14:26.914 5016 5016 I system_server: Loading /system/framework/oat/arm64/services.odex with executable: 1
02-03 14:14:26.916 5016 5016 I system_server: Successfully loaded /system/framework/oat/arm64/services.odex with executable: 1
02-03 14:14:26.927 5016 5016 I system_server: Loading /system/framework/oat/arm64/wifi-service.odex with executable: 1
02-03 14:14:26.933 5016 5016 I system_server: Successfully loaded /system/framework/oat/arm64/wifi-service.odex with executable: 1
02-03 14:14:26.933 5016 5016 I system_server: Loading /system/framework/oat/arm64/ethernet-service.odex with executable: 1
02-03 14:14:26.934 5016 5016 I system_server: Successfully loaded /system/framework/oat/arm64/ethernet-service.odex with executable: 1
02-03 14:14:26.946 5016 5016 I system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 0
02-03 14:14:26.947 5016 5016 I system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 0
02-03 14:14:26.947 5016 5016 I system_server: Loading /system/framework/oat/arm64/services.odex with executable: 0
02-03 14:14:26.948 5016 5016 I system_server: Successfully loaded /system/framework/oat/arm64/services.odex with executable: 0
02-03 14:14:26.948 5016 5016 I system_server: Loading /system/framework/oat/arm64/wifi-service.odex with executable: 0
02-03 14:14:26.948 5016 5016 I system_server: Successfully loaded /system/framework/oat/arm64/wifi-service.odex with executable: 0
02-03 14:14:26.949 5016 5016 I system_server: Loading /system/framework/oat/arm64/ethernet-service.odex with executable: 0
02-03 14:14:26.949 5016 5016 I system_server: Successfully loaded /system/framework/oat/arm64/ethernet-service.odex with executable: 0
02-03 14:14:30.480 5016 5016 I system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
02-03 14:14:30.481 5016 5016 I system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
Bug: 141785760
Bug: 140451054
Bug: 148944771
Change-Id: Idac678dbd1f5fe0e381ce8e0e3561423f8a31389
This is to unify boot jars with system server jars:
PRODUCT_SYSTEM_SERVER_JARS and PRODUCT_UPDATABLE_SYSTEM_SERVER_JARS
do not overlap.
Test: aosp_walleye-userdebug boots
Exempt-From-Owner-Approval: cherry-pick.
Bug: 146363577
Change-Id: If001049a46f8e9d38921dabeedbd61e971ebfe73
Merged-In: If001049a46f8e9d38921dabeedbd61e971ebfe73
(cherry picked from commit ef4358e536)
Framework and other dex files are used without image.
Test: taimen-userdebug boots when built with
DEXPREOPT_USE_ART_IMAGE=true
Test: Check logcat for checksum verification failures.
(Build ART with extra logging in OatFileAssistant.)
Test: Check that bootclasspath-checksums from some prebuilt
oat files (say input.odex) contain only one image
checksum followed by dex file checksums with
grep -az -A1 -E '^bootclasspath-checksums$' <oat-file> | \
xargs -0 echo | gawk '{print $2}'
Bug: 119800099
Change-Id: I65c2f247656e41f2c37df1ecb9e06af7dabab76e
Revert submission 1211982-dex2oat-soong-dep
Reason for revert: Build failures. See b/148312086.
Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...
Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: Ice3990225635a737e49e9aed7373f06516fccea3
Revert submission 1211982-dex2oat-soong-dep
Reason for revert: Build failures. See b/148312086.
Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...
Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: I6b656afb5feaad70d958b9d38b6c6eab7b03fba1
Revert submission 1211982-dex2oat-soong-dep
Reason for revert: Build failures. See b/148312086.
Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...
Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: I0cf6fef04434fd3c0b7957ee9abdcaabeb20b9d8
module.
This uses the Once cache for GlobalSoongConfig to propagate the dex2oat
path from a module dependency to the singletons (both the one that
writes out dexpreopt_soong.config and the one that creates the
dexpreopted boot images). Unless dexpreopting is disabled altogether
through DisablePreopt in dexpreopt.config, that means:
- We must ensure at least one module registers a dex2oat tool
dependency and resolves a GlobalSoongConfig using it, or else the
singletons fail. That means we litter dex2oat dependencies in java
modules even when they won't get dexpreopted and hence don't really
need them.
- We still assume there's at least one java_library or android_app in
the build.
Bug: 145934348
Test: m
(check that out/soong/dexpreopt_soong.config points to dex2oatd64)
Test: env USE_DEX2OAT_DEBUG=false m
(check that out/soong/dexpreopt_soong.config points to dex2oat)
Test: env OUT_DIR=out-tools prebuilts/build-tools/build-prebuilts.sh
on the aosp-build-tools branch
Change-Id: I38317f2d5e5cf6822f87daae25745767c8a3656c
package.
Preparation for a future CL that will need to get the make-written
GlobalConfig from within dexpreopt.
Also rename the Load*Config functions to Parse*Config, since they don't
actually load the config files anymore.
Bug: 145934348
Test: m
Change-Id: I71df11c1e042ca8135d273a7263e9539ea3cd68f
it.
Introduce a Once cache for GlobalSoongConfig to allow it to get binary
tool paths from ordinary module dependencies (coming in a future CL)
that are then reused in singletons.
Bug: 145934348
Test: m
Change-Id: I440a09dba7d337965a196527566b0966a18e3653
This relands I12a0f907753fefd1997ab8b4ea2ac331234093cf along with
a fix to blueprint for absolute paths.
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.
Change-Id: I24ac485677aa102eec1a2521d16820da6ee1ae77
Fixes: 146437378
Test: m checkbuild
Test: m OUT_DIR=/tmp/out nothing
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.
Fixes: 146437378
Test: m checkbuild
Change-Id: I12a0f907753fefd1997ab8b4ea2ac331234093cf
The dexpreopt global config is now split into the part that is generated
from make (in build/make/core/dex_preopt_config.mk) and the part that is
generated from Soong. Since the goal is to generate the dex2oat path from
Soong dependencies, the old GlobalConfig.Tools struct is simply repurposed
for the Soong generated config, although the intention is to allow more
settings to migrate from make to Soong, and hence from GlobalConfig to
GlobalSoongConfig.
Since the new dexpreopt_soong.config is written from a Soong-created ninja
rule, it doesn't need to be rewritten to out/soong/<device>/ like the old
make-created config file.
Test: m
Test: env USE_DEX2OAT_DEBUG=false m
(check that out/soong/dexpreopt_soong.config points to dex2oat instead of dex2oatd)
Bug: 145934348
Change-Id: Ifd45c4a08e2ec55b86f4a93f0d85bd39cf2cf189
Earlier CL Ida40dfae8c83bf7c2e737d5c7ea418e1197ad826 introduced
Soong-generated Make variable 'DEXPREOPT_IMAGE_LOCATIONS'. That CL was
erroneous in that it did not take JIT-zygote config into account and
generated identical location for "boot" and "apex" boot images.
This caused build breakages, because in case of JIT-zygote config the
two variables 'DexPreoptImages' and 'DexPreoptImageLocations' in the
module's dexpreopt.config were out of sync: 'DexPreoptImages' was
for the "apex" image, and 'DexPreoptImageLocations' was for the "boot"
image.
CL I9a91fc48e54d7d43abec2cb2b5a11e3581db380b introduced a workaround
for this problem: incorrect 'DexPreoptImageLocations' from the module
dexpreopt.config was ignored, and instead boot image location was
manually reconstructed from 'DexPreoptImages'. This workaround would
not work when we start using boot image extension and location will
become more complex.
This CL fixes the way 'DexPreoptImageLocations' is generated by
spliting the 'DEXPREOPT_IMAGE_LOCATIONS' variable in two variables
depending on the boot image flavour "boot" of "apex". This is
aligned with the way other similar variables are generated.
Test: aosp_walleye-userdebug boots.
Test: walleye_jitzygote-userdebug builds
(on git_rvc-release branch with this CL cherry-picked).
Change-Id: I93415227564522bce4250d281d561e708a022101
This fixes JIT-zygote builds (b/146462581). The fix is a workaround:
boot image location is computed from boot image filename instead of
using the pre-computed location from the module dexpreopt config.
Currently parts of the dexpreopt module config use boot image name
that is hard-coded in Makefiles, and other parts use boot image name
that is generated by Soong. This is exacerbated by the fact that
"default" boot image config in JIT-zygote builds consists partly of
the "apex" boot image, and partly of "boot" boot image.
Test: walleye_jitzygote-userdebug builds.
Bug: 146462581
Change-Id: I9a91fc48e54d7d43abec2cb2b5a11e3581db380b
The filesystem object was available through ModuleContext.Fs(), but
gives too much access to the filesystem without enforicing correct
dependencies. In order to support sandboxing the soong_build
process move the filesystem into the Config. The next change will
make it private.
Bug: 146437378
Test: all Soong tests
Change-Id: I5d3ae9108f120fd335b21efd612aefa078378813
This patch splits the system boot image in two parts:
- The ART boot image. This is the primary boot image that is
included in the ART apex and contains dexpreopted Core Libraries.
- The framweork boot image extension. It depends on the ART boot
image and contains framework libraries.
The third "apex" boot image (used in the JIT-zygote experiment)
remains unchanged; it is a monolithic primary boot image that
contains both libcore and framework libraries.
Dexpreopting of APKs now uses the framework boot image extension
(which in turn pulls in the ART boot image as a dependency).
Test: m
Test: phone boots:
lunch aosp_walleye-userdebug && m \
&& adb reboot bootloader && fastboot flashall -w
Bug: b/119800099
Exempt-From-Owner-Approval: rebased after getting approval.
Change-Id: Ida40dfae8c83bf7c2e737d5c7ea418e1197ad826
Use apex:jar pair values for updatable boot jars to align with
updatable system_server values.
Test: Compiles & flashed device. Ensured that the bootclasspath & system_server class
paths remain the same.
Change-Id: I1d6dfe3325d9091b7c724458be4e6471f9086666
This reverts commit 01f6b0a656.
Reason for revert: Build failure is not reproducible.
Forrest build on the same build ID 6033773 and same target
cf_x86_phone-userdebug_coverage finished successfully.
Change-Id: I5077f8332aa0b8037e324b89d41f35b86b8cf216
This reverts commit 61c325ebcc.
Reason for revert: broken build git_master/cf_x86_phone-userdebug_coverage (likely caused by these changes).
Exempt-From-Owner-Approval: revert.
Change-Id: I88ddd3af3a6c4ffdaa1fbb881d965356c5c75ad3
Test: m
Test: The list of updatable system server jars is empty now,
so I tested that the filer works with a manual experiment:
- temporarily add ethernet-service to PRODUCT_UPDATABLE_SYSTEM_SERVER_JARS
- m nothing
- fgrep -e 'ethernet-service' $ANDROID_BUILD_TOP/out/soong/build.ninja | grep dexpreopt
- expect empty output (no dexpreopt command is generated)
Change-Id: I0b231e823d5a5a97632daa2b5eb7be3e06782004
Allow system_server jars delivered via apex. Regular system_server
jars are located in /system/framework folder. But, jars delivered via
apex are mounted at /apex/<module_name>/javalib.
Bug: 144722612
Bug: 141785760
Test: Compiles
Change-Id: Ia40bb91d2e05b2601a52eac28a985fe2d8da3481
Stripping is incompatible with ART module updatability.
Bug: 65154345
Bug: 138851227
Test: build and observe no change in output (stripping is not used by
default).
Change-Id: I4bbc01d9979605b7dd2f73ee99a74c1e817c3e8c
Previously, the patterns in SYSTEM_OTHER_ODEX_FILTER only matched
subpaths of /system. Broaden this to match /<p> as well as /system/<p>.
Because SYSTEM_OTHER_ODEX_FILTER includes system_ext and product apps,
the immediate effect of this is that the odex and vdex files of
apps on those partitions move to system_other for A/B devices.
This makes the product and system_ext partitions smaller, which
in turn makes OTAs smaller.
Bug: 141707536
Test: soong unit tests
Test: (adb shell dumpsys package com.android.stk &&
adb shell dumpsys package com.android.quicksearchbox) | grep -i dexopt -A3;
verify dexpreopt state is 'prebuilt' for both
Test: adb shell find /data/dalvik-cache -name '*QuickSearch*.vdex' | wc -l == 1
Test: adb shell find /data/dalvik-cache -name '*Stk*.vdex' | wc -l == 1
Change-Id: I6e1e6d89c5822d48b522e7e6d77a363ed9f0e05b
...and move odex/vdex files into the dir corresponding to their
source app's partition.
Bug: 141707536
Test: soong unit test
Test: lunch aosp_blueline && m
Change-Id: I90a7a6c97e0eb2706951a2317a1a65a59ca974fa
Add some more test combinations, in particular apps in
/product and /system/product, as well as some different
patterns.
Bug: 141707536
Test: soong unit test
Change-Id: I9ea431770132d95a4411c7903402993806e1f8c8
Instead of just depending on the main .art file (boot.art, etc), also
expose the dependencies to the .oat/.vdex files (boot.oat/boot.vdex),
and all of the module files that get implicitly loading (boot-ext.*,
boot-framework.*, etc)
This is necessary for RBE, where the rule only gets the files that it
depends upon.
Test: treehugger
Test: build a system image with RBE
Change-Id: I0c7051f18582f1891d3398b46763b1521e4326c8
Ports 09f3b97f4b488cd3a7b7d72038b173575b02c162 (Add support for
preopt with uses-libraries) from Make to Soong to support verifying
and preopting shared libraries.
This reapplies Id25f55f07a55120bebe2a9b32c094209efc85c8b with fixes
for unbundled builds and builds with ALLOW_MISSING_DEPENDENCIES=true
set.
Bug: 132357300
Test: app_test.go
Test: m checkbuild
Change-Id: I964309a68ec4ed081f3f3154879c71048ecb5455
Ports 09f3b97f4b488cd3a7b7d72038b173575b02c162 (Add support for
preopt with uses-libraries) from Make to Soong to support verifying
and preopting shared libraries.
Bug: 132357300
Test: app_test.go
Test: m checkbuild
Change-Id: Id25f55f07a55120bebe2a9b32c094209efc85c8b
Verifying the <uses-library> tags in the manifest does not need
to be part of dexpreopt, move it out to a separate rule.
Bug: 132357300
Test: m checkbuild
Change-Id: I9d37872953b46f37ae77804819dc4eb8e2da0657
Writing $OUT/dexpreopt.config during build setup and then using
it during the build causes bad incremental builds if a cleanspec
removes $OUT/dexpreopt.config. Create a rule in Soong to write
out a copy so that it is a normal build artifact.
Bug: 132379813
Test: m checkbuild
Change-Id: I7755a7f8c1d6561fc83022a7f4cf7e4a64e989c9
To get performance benefits here for apps that aren't installed
through the play store.
Bug: 130217075
Bug: 131310042
Test: make
(cherry picked from commit 8910722e71)
Merged-In: I46dc3e4a588be52f359577ca1f0fd2dcd440c4bf
Change-Id: Ia98c4e5447b63f6362a4bd5c617a518784f0ef53
On-device compilation will always see paths starting with /product. This CL
makes sure preopt does the same.
Test: m && check odex files
Bug: 126415372
Change-Id: I5764e7f22bffb97b7093f351d7b97bbe5fb1f2f1
Copying p.rel doesn't work, as rel needs to match the end of the
value in path. Apply the same transformation to p.rel as p.path.
Test: paths_test.go
Change-Id: I42d676c6c4fc18d9852c1a73f25e5a791d7553d0
There are no more Make paths being used in Soong now that
dexpreopting and hiddenapi are in Soong. Use the Path types
in the inputs to RuleBuilder, and fix all users of RuleBuilder.
This reapplies I886f803d9a3419a43b2cae412537645f94c5dfbf with
fixes to disable preopt for Soong-only builds when the global
dexpreopt.config doesn't exist.
Test: all soong tests
Test: m checkbuild
Change-Id: I4dae9ecd5de22f062f9478ec8f0747f099cf8190
There are no more Make paths being used in Soong now that
dexpreopting and hiddenapi are in Soong. Use the Path types
in the inputs to RuleBuilder, and fix all users of RuleBuilder.
Test: all soong tests
Test: m checkbuild
Change-Id: I886f803d9a3419a43b2cae412537645f94c5dfbf
Bug: 124789867
Test: build with coverage, the boot classpath has jacoco.jar in /system/framework, not the apex
Change-Id: Iaa0762b5a674282be95710d0e1fb166a58ec893b
Add a utility method that returns a copy of a slice of strings.
This is primarily useful when appending to a string slice to avoid
accidentally reusing the backing array.
Test: util_test.go
Change-Id: I7801fc7ca19e27ddc9f1b1b452788b723c7f619c
strip.sh was including the dependencies of dexpreoptRule instead
of stripRule, resulting in incorrect rebuilds.
Bug: 124529711
Test: m && m
Change-Id: I4b40f7c040442a03918c16a46ecf777f5cc30130
Add a RuleBuilderInstalls type for a slice of RuleBuilderInstalls,
and give it a String() method that returns the list of installs
in the format that is convenient for passing to Make.
Test: rule_builder_test.go
Change-Id: I2e9cd9abf4dfb0ad312d0a6662f1567baf9cd222
This reverts commit 67e8ec1973.
Test: build && atest android.text.cts.EmojiTest#testEmojiGlyphWebView on Cuttlefish
Exempt-From-Owner-Approval: Got +2 from Colin, latest PS is a rebase across conflicts.
Change-Id: I99faf0f2ec698d70c107516bd43756b9ddcb90d0
Make ArchType implement the encoding.TextMarshaller and
encoding.TextUnmarshaller interfaces so that it can be used
as a value in the dexpreopt config structs that are passed
through JSON files.
Test: m checkbuild
Change-Id: Ie4c12443e7ee5fe43f42d5403bcde12d62f617e2
Instead of passing both the module and global dexpreopt image
selection into the dexpreopt package, determine the correct
dexpreopt image in the java package.
Also stop using the boot image "location" as an input, only
track the real path, and then convert it to the "location"
that dex2oat expects when constructing the command line.
Test: m checkbuild
Change-Id: I2be2b5917e09fd57428dab27849c0153fdc75722
dexpreopt doesn't need to know about UseEmbeddedDex, it has
already been taken into account for UncompressedDex.
Test: m checkbuild
Change-Id: I5508aa43bb8bdb06e20cf93bc5171a5e78833983
Add a few convenience methods, document all the methods and add
examples that would show up in the godoc if we were to actually
generate it.
Test: rule_builder_test.go
Change-Id: I270fed605ffec34e6f5b36fde0dc9ca52694b741
Move dexpreopt.Script to android.RuleBuilder so that the builder
style can be used in more places. Also add tests for it.
Test: rule_builder_test.go
Change-Id: I92a963bd112bf033b08899e930094b908acfcdfd
Add a no_stripping property and pass it to dexpreopt to disable
stripping for a module.
Bug: 122610462
Test: dexpreopt_test.go
Change-Id: I5a4b005633bb8b1ea373e9eeb420aa0999de17ab
If dexpreopt is disabled for a module then classes.dex must not be
stripped.
Bug: 121377197
Test: m WITH_DEXPREOPT_BOOT_IMG_AND_SYSTEM_SERVER_ONLY=true
Change-Id: Icfa48804cf02291874ac6623c9b9297821ac8ce6
It is useful to have the exact dex2oat invocation used to generate
artifacts for some tests. This adds the --write-invocation-to= flag to
the build to save this information.
Test: m droid
Bug: 119332327
Change-Id: Id7664cf6d05bbb1c580f94584cdeb96084d5799c
If dexpreopt is disabled inside dexpreopt_gen there won't be any installed files and the install directory won't be created, causing soong_zip to fail. Create the directory ahead of time.
Test: treehugger
Change-Id: Icf17d0bcd617eeeafdcd50ab56359fa8d6612822
Port the dexpreopt logic from Make to the dexpreopt package in Soong,
and use it to dexpreopt Soong modules. The same package is also
compiled into the dexpreopt_gen binary to generate dexpreopt scripts
for Make modules.
This relands Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb and
I462182638bd57b1367b5bfb0718e975c11ae66f7, along with multiple fixes
to depsfile generation in dexpreopt_gen that caused .odex files for
modules in defined make to be missing dependencies on boot.art, and
a fix to not dexpreopt and strip tests.
Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
only expected changes to dexpreopt outputs on system_other
(.vdex files for privileged Soong modules no longer incorrectly
contain .dex contents).
Test: OUT_DIR=$PWD/out m
Test: NINJA_ARGS="-t deps out/target/product/sailfish/obj/APPS/Contacts_intermediates/dexpreopt.zip" m
Change-Id: I6bb2c971cee65d2338839753aa0d84939f335b1b
Port the dexpreopt logic from Make to the dexpreopt package in Soong,
and use it to dexpreopt Soong modules. The same package is also
compiled into the dexpreopt_gen binary to generate dexpreopt scripts
for Make modules.
Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
only expected changes to dexpreopt outputs on system_other
(.vdex files for privileged Soong modules no longer incorrectly
contain .dex contents).
Change-Id: Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb
