Before this change, dexpreopt was often broken with optional libraries.
This was because the CLC construction was done in Soong at an early
stage, where we don't have sufficient information to determine whether
an optional library is installed or not.
For example, the "Settings" package uses an optional library called
"androidx.window.extensions". On some devices, the library is installed,
but on some other devices, it's not. Soong always adds the library to
the CLC, meaning the CLC is wrong for devices which don't have the
library. This change fixes the problem. See the tests below.
After this change, the CLC construction is done by a Python script
invoked at a very late stage. It uses product_packages.txt, which is
generated by Make, to determine whether an optional library is
installed or not, and filter out libraries that are not installed.
Note that optional libraries are still added as dependencies by Soong.
This is because dependencies have to be added at an early stage. This
means what dex2oat eventually uses will be a subset of the dependencies,
which is fine.
Bug: 282877248
Test: m
Test: atest construct_context_test
Test: -
1. lunch aosp_cf_x86_64_phone-userdebug && m
2. Check the .invocation file of the "Settings" package (defined in
.bp file)
3. See androidx.window.extensions
Test: -
1. lunch aosp_redfin-userdebug && m
2. Check the .invocation file of the "Settings" package (defined in
.bp file)
3. Don't see androidx.window.extensions
Test: Check the .invocation file of the "Dialer" package (defined in
.mk file)
Test: -
1. Build a Pixel 5 system image and flash it to a Pixel 5 device.
2. adb shell pm art dump
3. See "reason=prebuilt" instead of "reason=vdex".
(https://diff.googleplex.com/#key=fB6Ls9q2QGSN, before: left,
after: right)
Change-Id: Ia112bd7c2328373e68db6bffb74bf34030f683d8
Bug: 280440941
Test: -
1. Patch ag/22302622 to disable dexpreopt.
2. lunch aosp_cf_x86_64_phone-userdebug && m
3. See app profiles still generated.
Test: -
1. Patch ag/20592051 to enable profile for service-art.
2. banchan com.android.art x86_64 && m
3. See the profile for service-art generated.
Change-Id: I4e721b475b84a2f667bbccc030a8947078f26bb0
After this change, if profile-guided compilation is enabled for an APEX
system server jar, the profile will be installed next to the jar with
the ".prof" suffix, ("javalib/<name>.jar.prof"). This file will later be
used by odrefresh and dexpreopt from prebuilt APEX.
Bug: 241823638
Test: m nothing
Test: -
1. Patch ag/20581649 PS2.
2. banchan com.android.btservices x86_64 && m
3. Check that "javalib/service-bluetooth.jar.prof" exists in the APEX.
Change-Id: Ibcfc6257dade92bd40a6d4b7368148717d0863b9
This change is a no-op change. It doesn't enable userfaultfd GC by
default. ENABLE_UFFD_GC=true can be passed to the build system to enable
userfaultfd GC for testing purposes.
Bug: 242553398
Test: -
1. lunch aosp_oriole-userdebug
2. ENABLE_UFFD_GC=true m
3. See "--runtime-arg -Xgc:CMC" in .invocation files.
Change-Id: I789b49a71b9604fd41bf1ef77d0ac5bac4cbdf25
Bug: 245383870
Bug: 155630745
Test:m nothing
Test:-
1.Add a jar installed in the system_ext partition toSYSTEMSERVERJARS
(e.g.b/245383870 PRODUCT_SYSTEM_SERVER_JARS_EXTRA += system_ext:oem-services)
2.Build a ROM
3.See the odex and vdex files generated in $ANDROID_PRODUCT_OUT/system_ext/framework/oat/
4.Flash the ROM to a device
5.Check the maps information of the system_server , the newly added odex has the r-xp flags
6.Run `adb shell am restart;adb root;adb shell setprop dalvik.vm.extra-opts
-verbose:deopt,heap,jit,oat,image,dex,verifier;`
7.Check ClassLoaderContext classpath size matching information
8.Run `atest art_standalone_odrefresh_tests`
`atest art_standalone_dexpreopt_tests`
`atest CtsSdkExtensionsTestCases`
Change-Id: Ib1083ee130b5fef9c23d56523868fa1e29be879a
Signed-off-by: liulvping <liulvping@xiaomi.com>
The motivation is to reduce file size and in-process memory footprint
for "services" (which is currently the only system server jar that has a
profile). For other system server jars that have no profile keep using
"speed", as enabling "speed-profile" for them would completely disable
AOT-compiled code.
Compare .odex file size:
- before: 44360 services.odex
- after: 24968 services.odex
Bug: b/237399630
Test: lunch aosp_sargo-userdebug && m
Change-Id: I844b9607c496d3d6e7048dc6bb8cd958ecbaa441
dexpreopt.GetGlobalConfig uses a OncePer to only parse the config once.
If the config contains missing paths, which can happen in partial
manifest branches, the missing path errors will only be reported on
the first caller to GetGlobalConfig. Fix the nondeterminism by
wrapping the PathContext and capturing any path errors and reporting
them to all callers to GetGlobalConfig.
Bug: 207813628
Test: tradefed branch builds
Change-Id: I869c3ae49a819b1251b94b73e0487aa594cd5e22
This reverts commit 0b1c70efbc.
The reverted commit was based on the idea that uses-libraries that are
explicitly specified in build files should not be implicitly added to
the manifest, as that would mean that anything added to the build files
will flow to the manifest.
Although this logic is correct, it prevents propagation of
uses-libraries from dependencies, which is wrong: if a library has an
explicit uses-library property in Android.bp, this property is expected
to be propagated to the library's dependencies. Failing to do so would
mean that every user of that library has to add uses-library property to
their build files, which doesn't scale (see b/214255490 for example).
Bug: 214255490
Test: lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd \
&& adb wait-for-device && adb root \
&& adb logcat | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
# empty output, no errors at boot
Change-Id: I6f420e76a89aa2f37be99f877711736640f2c361
Previously "any" was serialized as its numeric value, 10000. But other
code in makefiles and scripts expects string "any", and dexpreopt.config
files generated by Make (for Android.mk modules) have "any" not 10000.
Bug: 214255490
Test: lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd \
&& adb wait-for-device && adb root \
&& adb logcat | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
# empty output, no errors at boot
Change-Id: Id5e80eb8a90d9786b5cb999c172aaecb44952f76
A number of tests in the cts/tests/signature/api-check check for the
accessibility of classes from the android.test.base,
android.test.runner and android.test.mock libraries. Some tests expect
to find the classes other do not. Unfortunately, the tests use
libraries, specifically compatibility-device-util-axt, that depend on
the android.test... libraries which causes Soong to implicitly add
<uses-library> entries to the manifest so that they will be accessible
at runtime. That causes the tests that do not expect to find the
classes to fail.
Bug: 209607558
Test: m nothing
Change-Id: I54c194ab23d5a70df790ece3fe98f2b3d6a1c1f6
Standalone system server jars are dynamically loaded by system server
using a `PathClassLoader` whose parent is `SYSTEMSERVERCLASSPATH`. They
are listed in `PRODUCT_STANDALONE_SYSTEM_SERVER_JARS` and
`PRODUCT_APEX_STANDALONE_SYSTEM_SERVER_JARS` in Makefile. We need to
dexpreopt them to achieve better performance.
Bug: 203198541
Test: m nothing
Test: -
1. Add a standalone system server jar (e.g., by patching
aosp/1906158)
2. Build a system image.
3. See the odex and vdex files generated in
$ANDROID_PRODUCT_OUT/system/framework/oat/
4. Flash the image to a device.
5. Run `atest art_standalone_dexpreopt_tests`.
Change-Id: I358a62d34989c5c8eba12e18fe6167e0b72ff69d
We will need the profile when we generate the primary boot image on
device.
Bug: 203492478
Test: Run `banchan com.android.art x86_64 && m` and see
`$ANDROID_PRODUCT_OUT/apex/com.android.art/etc/boot-image.prof`.
Test: Run `lunch aosp_cf_x86_64_phone-userdebug && m` and see both
`$ANDROID_PRODUCT_OUT/apex/com.android.art/etc/boot-image.prof` and
`$ANDROID_PRODUCT_OUT/system/etc/boot-image.prof`, in different
sizes.
Test: Start Cuttlefish with the built image and see both
`/apex/com.android.art/etc/boot-image.prof` and
`/system/etc/boot-image.prof` on device.
Change-Id: Id879dc49b234133dfbb9563814328661a1f4a6c0
This CL updates platform_systemserverclasspath,
systemserverclasspath_fragment, and
prebuilt_systemserverclasspath_fragment to write entries for
STANDALONE_SYSTEMSERVER_JARS to systemserverclasspath.pb. A new property
`standalone_contents` is aded to `systemserverclasspath_fragment` and
`prebuilt_systemserverclasspath_fragment` to list the standalone system
server jars in the APEX, and entries will be written to
systemserverclasspath.pb accordingly at build time.
To add more context, these entries will be consumed by derive_classpath
in order to generate an environment variable
PRODUCT_STANDALONE_SYSTEM_SERVER_JARS. The environment variable will
then be comsumed by odrefresh to determine what jars to preopt on early
boot.
Note that the variable should not end with "CLASSPATH" because the list
is not used by runtime as a classpath. It is just a colon-separated list
of jars. System server loads the jars separately with paths hardcoded in
the code.
Bug: 203198541
Test: manual -
1. Add some jars to PRODUCT_STANDALONE_SYSTEM_SERVER_JARS
2. Add some other jars to PRODUCT_APEX_STANDALONE_SYSTEM_SERVER_JARS
and standalone_contents.
3. Build an image.
4. Flash the image to a device.
5. adb shell echo \$STANDALONE_SYSTEMSERVER_JARS
6. See the correct list of jars.
Change-Id: I09a6fd1d3db85c194330da9b751702a9bf069e26
Tests were using the same value for OutDir and SoongOutDir, separate
them to allow a test that needs to distinguish between them.
Test: all soong tests
Change-Id: Ibd78758c133a7f07bead3f51b699052676f92fbd
This change adds a Soong module that stores the paths to the compilation
artifacts of system server jars in a variable, which will then be
consumed by Make to check if the artifacts are installed. When the check
fails, it means that dexpreopting is not working for some system server
jars and needs to be fixed.
Bug: 201371822
Test: m nothing
Test: manual -
1. Add "service-permission" to DEXPREOPT_DISABLED_MODULES (https://cs.android.com/android/platform/superproject/+/master:build/make/core/product_config.mk?q=DEXPREOPT_DISABLED_MODULES)
2. m nothing
3. See the error:
Missing compilation artifacts. Dexpreopting is not working for some system server jars
Offending entries:
system/framework/oat/x86_64/apex@com.android.permission@javalib@service-permission.jar@classes.odex
system/framework/oat/x86_64/apex@com.android.permission@javalib@service-permission.jar@classes.vdex
Change-Id: I4816f19668f1dae180a34adafdbfa448c97aa0db
Previously, if an implementation jar (Host) was not available to the
build it would panic when trying to generate the JSON representation of
the CLC. That prevents builds with missing implementation jars from
working even if those jars are never actually built.
This change defers the build failure until it is actually built.
Test: TARGET_PRODUCT=armv8 TARGET_BUILD_VARIANT=eng ./art/test/testrunner/run_build_test_target.py -j80 art-no-prebuild
- run above in master-art before and after this change.
Bug: 202366925
Change-Id: I60a78a8bf6c13b83a9dceb5c43019a9e21f0b637
This is necessary in order to expose the copy rule to Ninja. Otherwise
Ninja has a build rule that depends on the copied jar (namely, dexpreopt
commands for all subsequent system server jars that have the current one
in class loader context), but no rule that creates it, so Ninja fails
with a "no rule to make ..." error.
The change only affects system server ed in Android.mk
Previously the problem existed, but was hidden by the fact that the only
system server jar defined in Android.mk was the last one on the list, so
no other jar depended on it. Now that updatable apex jars are also
dexpreopted (https://r.android.com/1828115) the problem was uncovered.
Bug: 200297762
Test: lunch bertha_x86_64-userdebug && m
Change-Id: Idbf2eacdf7ae89847aa9d755ef688eda3720ce89
This reverts commit 92346c4832.
Reason for revert: Fixed build error.
The build error is fixed by ag/15841934. This CL remains unchanged. This
CL will be submitted AFTER ag/15841934 is submitted.
Bug: 200024131
Test: 1. Patch this CL and ag/15841934 into internal master.
2. sudo vendor/google/build/build_test.bash
Change-Id: I5f2b8357846fc7dda56e25ebe6ffb095e8047ec8
The path to the artifacts will in the form of
/system/framework/oat/<arch>/<encoded-jar-path>@classes.{odex,vdex,art},
where <encoded-jar-path> is the path to the jar file with "/" replaced
by "@". For example,
/system/framework/oat/x86_64/apex@com.android.art@javalib@service-art.jar@classes.odex
There will be a follow-up CL to update ART runtime to recognize
artifacts in that path.
Test: m com.android.art
Bug: 194150908
Change-Id: Ic89fd63c4b1cd565684cead83fc91dae3bc97a4c
They both used to return out/soong. Unfortunately, the tests seem to
assume the old behavior, so I had to keep tests inconsistent and I was
scared of dexpreopt, too.
Test: Presubmits.
Change-Id: Ib00c7956fd2cdb28cc11d481c17e5a3ff5ad563a
These are just out/ and out/soong/ and the old names were quite
confusing.
Test: Presubmits.
Merged-In: I999ca62162062f27e449f83ffb00fa580d4065b8
Merged-In: I2fcd80a9e376a6842a06aa104a9f6f5c5e16f8c7
Change-Id: Ib481d2aac40df8da1b74174dc8ec576c1cb48be0
These properties specify libraries that cannot be implicitly inferred by
Soong. If these properties are added to Android.bp, this can only be for
the reason that there is a <uses-library> tag in the manifest which is
unknown to the build system. Adding them to the manifest_fixer doesn't
make sense: if they are not in the manifest, they should be removed from
Android.bp as well.
Bug: 132357300
Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
$ adb wait-for-device && adb root && adb logcat \
| grep -E 'ClassLoaderContext [a-z ]+ mismatch'
# empty grep output, no errors
Change-Id: Ic6eb5268a954ef3be7f06a181ec72af99000c547
Previously manifest_fixer used a naive way to distiniguish optional libs
from required ones: it checked if a library is on the list of optional
compatibility libraries. This works for compatibility libs, but not for
other libs.
Now we properly track optionality through all stages of the build,
starting with the addition of the library as a dependency (here's where
the `uses_libs`/`optional_uses_libs` distinction kicks in), store it in
dependency tag and propagate to class loader context, and from there to
the manifest_fixer.
The tests have been updated accordingly.
Bug: 196377222
Test: lunch bertha_x86_64-userdebug && m droid dist cts mts
Change-Id: I3631ce59ebe47116ce7a9b3d33a86f636846ef0f
This is in preparation to r.android.com/1740313 where setting correct
variables would be enforced (i.e. apex and non-apex boot jars must be in
config.ApexBootJars and config.BootJars correspondingly).
Bug: 191369843
Test: m nothing
Change-Id: Ic86680c1f7af53d229083b2cc58beb3ceccb4b6a
This special-casing is actively unhelpful nowadays, as it breaks
callstack unwinding and symbolisation by tools that work using the
on-device debug info (e.g. perfetto's profilers). It's also highly
surprising that eng builds have less debug info than userdebug/user.
As far as I can tell, the history behind this case is (with some
internal-only references as the context is in the review comments): the
minidebuginfo was originally rolled out in device-specific makefiles
(e.g. ag/2433691). Eng builds were excluded, presumably due to time/size
concerns for rebuild+resync workflow, as well as the assumption that
engineers using eng builds already have the necessary debug info
off-device. Later, as some more makefile variables were introduced, the
conversation in ag/990059 shows that the overheads were no longer a
concern, and the last patchset generalised the option to include eng
builds. However at a later point, during device-specific makefile
cleanup, the eng-exclusion got restored: ag/2455645. This then ended up
in the makefile->soong transition, leaving us with the current special
casing that is undesireable.
Checking things with an internal master build on my workstation:
dexpreopting system services is 10s regardless of this minidebuginfo
flag, and the resulting odex size is 37MB vs 38MB. Overall, I only see 4
dexpreopt invocations as part of a local build, so the effect of this
patch should be negligible.
Bug: 194835631
Change-Id: I80de75a84259c056cdae0696fe611afe7e0162f5
Note that ART apex boot jars and core-icu4j are exceptions here as they
are not part of ApexBootJars. ART apex boot jars are defined in their
own variable, while core-icu4j is treated as a regular non-updatable
boot jar.
Bug: 191127295
Test: atest CtsClasspathsTestCases
Change-Id: I3cea3d82ef521655a1a5ffa8cae2258ab9d08bfc
A Java module may depend on a stub library. In that case an additional
dependency on the implementation library is created, and it is used to
add the implementation library to class loader context. We should not
attempt to add the stubs library as well (previously the attempt to add
it happend after the implemention was added to CLC, to the attempt was
unsuccessful).
Raise an error if someone tries to add the same library with different
build/instal paths.
Also, rename local variable `implicitSdkLib` to `sdkLib` to better
reflect its meaning.
Bug: 193425964
Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
$ adb wait-for-device && \
adb root && \
adb logcat | \
grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C1
# empty output, no errors
Change-Id: I01c1bdd23f9d118d891d0b806e7e3b4d78896a34
This is needed when some source libraries get replaced with prebuilt
variants: CLC should contain one entry for a library (the prebuilt one
or the source one, whichever of them is preferred). Because the prebuilt
module name starts with "prebuilt_" prefix, previously Soong considered
such libraries as two different ones libraries and added both to CLC.
Bug: 193425964
Test: mark "prebuilt_android.net.ipsec.ike" as preferred and `m nothing`
(before this CL it would fail the build, now it builds fine).
Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
$ adb wait-for-device && \
adb root && \
adb logcat | \
grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C 1
# empty output, no errors
Change-Id: Icc42533d9915060d7fffda12aa93b9d18dc4f83d
