aosp/2984037 disabled dexperopt of the source apex system server jars
when prebuilts are selected. This was done to prevent duplicate
installation and dex2oat deps.
AOSP art has some additional variants like com.android.art.debug. In
source builds, this apex should contain service-art.jar and
service-art.jar.prof (a dexpeopt artifact). We have a test to check this
(`art_apex_test.py`). If we disable dexpreopt of source sdlib when
prebuilts are selected, this test needs to be disabled. This is the
behavior at ToT.
This CL changes the behavior to enable running this test even when
com.google.android.art prebuilt is active in a specific release
configuraiton. To prevent collisions that prompted aosp/2984037, this CL
special-cases the installation and dex2oat rules instead of disabling
dexpreopt of the source sdklib altogether.
b/331665856 tracks the principled solution to prevent duplicate
dexpreopt rules.
Implementation:
Add a new copyApexSystemServerJarDex arg to GenerateDexpreoptRule API.
If true, the dexjar file will be copied to
out/soong/system_server_jars/. For soong modules, the value of this will
be the inverse of disableSourceApexVariant. Since none of the apex
system server jars are in make, this will be a noop in dexpreopt_gen
Bug: 331665856
Test: modified trunk_staging.scl locally to select
art.google.contributions.prebuilt; mmma art; (with the sibling CL in
topic)
Change-Id: Idb59e424f83d126cdc8b1671dde358745979bd8d
This is a no-op change for a majority of cases.
Before this change, the contruction of the manifest check inputs is
confusing. It mutates uses_libs properties in place just for the
manifest check, by replacing module names with library names for
direct dependencies and merging library names from CLC for both direct
denpendencies and transitive denpendencies, and then constructs manifest
check inputs from those mutated uses_libs properties. This is
error-prone and leads to insistency: the goal is to check that the CLC
matches the manifest, but the inputs to the check don't reflect the CLC.
After this change, we no longer mutate uses_libs properties in place.
Instead, we maintain a separate list of missing denpendencies, and then
construct manifest check inputs directly from the CLC for all existing
libraries, no matter they are direct or transtive, and from the separate
list of missing libraries. This change makes the logic more
consistent and straightforward, and it also allows us to easily do the
next change, which is to propagate transtive missing denpendencies.
In fact, this change revealed several bugs around library optionality
and order in CLC construction, and fixed them.
Bug: 331528424
Test: m --no-skip-soong-tests
Ignore-AOSP-First: Depends on internal changes. Will cherry-pick once merged.
Merged-In: I0de82e76c47995b54aba9efd41538d950256a95f
Change-Id: I0de82e76c47995b54aba9efd41538d950256a95f
How it works:
1. build/make/core/Makefile generates a txt file with the kernel
version, which is taken from an explicit BOARD_KERNEL_VERSION value,
or extracted from the kernel image on the source tree, or extracted
from the kernel image extracted from the prebuilt boot.img.
The file is saved at
$ANDROID_PRODUCT_OUT/obj/PACKAGING/check_vintf_all_intermediates/kernel_version.txt.
2. If PRODUCT_ENABLE_UFFD_GC is "default", meaning the GC type needs to
be determined by the kernel version, build/make/core/Makefile copies
kernel_version.txt to
out/soong/dexpreopt/kernel_version_for_uffd_gc.txt.
3. build/soong/dexpreopt/config.go writes the the UFFD GC flag to
out/soong/dexpreopt/uffd_gc_flag.txt. The flag is determined by an
explicit PRODUCT_ENABLE_UFFD_GC value or by contruct_uffd_gc_flag.py,
which reads kernel_version_for_uffd_gc.txt and determines the flag
accordingly.
4. dex2oat takes the UFFD GC flag from uffd_gc_flag.txt.
5. post_process_props.py mangles ro.dalvik.vm.enable_uffd_gc based on
the same logic.
Bug: 321751629
Bug: 319554951
Bug: 318763448
Bug: 319648491
Test: m --no-skip-soong-tests nothing
Test: atest uffd_gc_utils_test
Test: Build with `OVERRIDE_ENABLE_UFFD_GC=default m` for device with no
UFFD support -
1. Check the existence of `-Xgc:CMC` in
out/soong/dexpreopt_arm64/dex_bootjars/android/system/framework/arm64/boot.invocation
(dex2oat invocation for a boot image)
2. Check the existence of `-Xgc:CMC` in
out/soong/.intermediates/packages/apps/Settings/Settings/android_common/dexpreopt/Settings/oat/arm64/package.invocation
(dex2oat invocation for an app defined in .bp)
3. Check the existence of `-Xgc:CMC` in
$ANDROID_PRODUCT_OUT/obj/APPS/Dialer_intermediates/oat/arm64/package.invocation
(dex2oat invocation for an app defined in .mk)
4. Check the value of ro.dalvik.vm.enable_uffd_gc in
$ANDROID_PRODUCT_OUT/product/etc/build.prop
Test: Build with `OVERRIDE_ENABLE_UFFD_GC=default m` for device with
UFFD support, and do the steps above.
Test: Build with `OVERRIDE_ENABLE_UFFD_GC=true m`, and do the steps
above.
Test: Build with `OVERRIDE_ENABLE_UFFD_GC=false m`, and do the steps
above.
Change-Id: I035ad32233e49e2a30ce11f6c7c318a648511ef8
(This was missed in aosp/2876756, which copied only the bcp jars)
This hardcoded location is used by dex2oat to compile the dexpreopt
artifacts. The copy rules are currently generated by java_(sdk)_library
for source builds, and their prebuilt counterparts in prebuilt builds.
After this change, the copy rule will be bifurcated between source and
prebuilt builds
1. For source builds, it will come from java_(sdk)_library
2. For prebuilt builds, it will come from the top-level prebuilt apex.
Since there can be multiple prebuilt apexes in trunk stable,
HideFromMake will be used to determine which deapexed jar to copy.
The bifurfaction is expected to be temporary. It is needed for now since
the `apex_contributions` which will be used for source vs prebuilt
selection have not been populated completely.
Test: Added a unit test
Test: Presubmits
Test: git_master-art-host:
art-gtest (https://android-build.corp.google.com/builds/abtd/run/L40800030001459791)
Test: git_main:art_standalone_tests
(https://android-build.corp.google.com/builds/abtd/run/L09000030001463855)
Bug: 308790457
Change-Id: I3105d3b3a7e5c41cb601d07806f4ea483a61b50a
This was added by r.android.com/513843 to force dexpreopt some GMS core
modules even if dexpreopt is disabled, to avoid some memory usage
regression. We no longer need it because dexpreopt is never disabled on
production builds.
Bug: 313505540
Test: m
Change-Id: I605b7569c17ee715cd4df167768e25aaf51bcd37
OnlyPreoptBootImageAndSystemServer was for making the eng build faster.
This change makes it even faster.
Bug: 309011617
Test: Presubmit
Change-Id: If9001e99593b57deafa7b56f0ca6b3d4d80f56d3
Starting from aosp/2594905, dexpreopt depends on
`$PRODUCT_OUT/product_packages.txt`. When PRODUCT_PACKAGES changes,
dexpreopt has to rerun for all apps. This is not ideal.
After this change, dexpreopt uses a per-app product_packages.txt that is
filtered by the app's dependencies, and it uses `rsync --checksum` to
prevent the file's mtime from being changed if the contents don't change.
This avoids unnecessary dexpreopt reruns.
Bug: 288218403
Test: m
Test: Change PRODUCT_PACKAGES and see no dexpreopt reruns.
Change-Id: I5788a9ee987dfd0abfd7d91cbcef748452290004
The conversion of FirstUniqueStrings to be implemented on top of the
generic firstUnique accidentally used a different threshold to switch
from the list implementation to the map implementation. Modify the
threshold of firstUnique to match the old value from FirstUniqueStrings
now that it doesn't have the reflection overhead. While we're at it,
also make firstUnique make a copy of the list, and make FirstUniqueStrings
a pure wrapper around firstUnique.
Test: BenchmarkFirstUniqueStrings
Change-Id: Icc2febea663142c508ff2e4be65a8a68121631d5
Use Go's generics for DepSets so they don't require a type-specific
wrapper and reflection.
Test: depsets_test.go
Change-Id: I22ba0b7d680d37d2cd05230b0f560d166c4dd20b
Before this change, dexpreopt was often broken with optional libraries.
This was because the CLC construction was done in Soong at an early
stage, where we don't have sufficient information to determine whether
an optional library is installed or not.
For example, the "Settings" package uses an optional library called
"androidx.window.extensions". On some devices, the library is installed,
but on some other devices, it's not. Soong always adds the library to
the CLC, meaning the CLC is wrong for devices which don't have the
library. This change fixes the problem. See the tests below.
After this change, the CLC construction is done by a Python script
invoked at a very late stage. It uses product_packages.txt, which is
generated by Make, to determine whether an optional library is
installed or not, and filter out libraries that are not installed.
Note that optional libraries are still added as dependencies by Soong.
This is because dependencies have to be added at an early stage. This
means what dex2oat eventually uses will be a subset of the dependencies,
which is fine.
Bug: 282877248
Test: m
Test: atest construct_context_test
Test: -
1. lunch aosp_cf_x86_64_phone-userdebug && m
2. Check the .invocation file of the "Settings" package (defined in
.bp file)
3. See androidx.window.extensions
Test: -
1. lunch aosp_redfin-userdebug && m
2. Check the .invocation file of the "Settings" package (defined in
.bp file)
3. Don't see androidx.window.extensions
Test: Check the .invocation file of the "Dialer" package (defined in
.mk file)
Test: -
1. Build a Pixel 5 system image and flash it to a Pixel 5 device.
2. adb shell pm art dump
3. See "reason=prebuilt" instead of "reason=vdex".
(https://diff.googleplex.com/#key=fB6Ls9q2QGSN, before: left,
after: right)
Change-Id: Ia112bd7c2328373e68db6bffb74bf34030f683d8
Bug: 280440941
Test: -
1. Patch ag/22302622 to disable dexpreopt.
2. lunch aosp_cf_x86_64_phone-userdebug && m
3. See app profiles still generated.
Test: -
1. Patch ag/20592051 to enable profile for service-art.
2. banchan com.android.art x86_64 && m
3. See the profile for service-art generated.
Change-Id: I4e721b475b84a2f667bbccc030a8947078f26bb0
After this change, if profile-guided compilation is enabled for an APEX
system server jar, the profile will be installed next to the jar with
the ".prof" suffix, ("javalib/<name>.jar.prof"). This file will later be
used by odrefresh and dexpreopt from prebuilt APEX.
Bug: 241823638
Test: m nothing
Test: -
1. Patch ag/20581649 PS2.
2. banchan com.android.btservices x86_64 && m
3. Check that "javalib/service-bluetooth.jar.prof" exists in the APEX.
Change-Id: Ibcfc6257dade92bd40a6d4b7368148717d0863b9
This change is a no-op change. It doesn't enable userfaultfd GC by
default. ENABLE_UFFD_GC=true can be passed to the build system to enable
userfaultfd GC for testing purposes.
Bug: 242553398
Test: -
1. lunch aosp_oriole-userdebug
2. ENABLE_UFFD_GC=true m
3. See "--runtime-arg -Xgc:CMC" in .invocation files.
Change-Id: I789b49a71b9604fd41bf1ef77d0ac5bac4cbdf25
Bug: 245383870
Bug: 155630745
Test:m nothing
Test:-
1.Add a jar installed in the system_ext partition toSYSTEMSERVERJARS
(e.g.b/245383870 PRODUCT_SYSTEM_SERVER_JARS_EXTRA += system_ext:oem-services)
2.Build a ROM
3.See the odex and vdex files generated in $ANDROID_PRODUCT_OUT/system_ext/framework/oat/
4.Flash the ROM to a device
5.Check the maps information of the system_server , the newly added odex has the r-xp flags
6.Run `adb shell am restart;adb root;adb shell setprop dalvik.vm.extra-opts
-verbose:deopt,heap,jit,oat,image,dex,verifier;`
7.Check ClassLoaderContext classpath size matching information
8.Run `atest art_standalone_odrefresh_tests`
`atest art_standalone_dexpreopt_tests`
`atest CtsSdkExtensionsTestCases`
Change-Id: Ib1083ee130b5fef9c23d56523868fa1e29be879a
Signed-off-by: liulvping <liulvping@xiaomi.com>
The motivation is to reduce file size and in-process memory footprint
for "services" (which is currently the only system server jar that has a
profile). For other system server jars that have no profile keep using
"speed", as enabling "speed-profile" for them would completely disable
AOT-compiled code.
Compare .odex file size:
- before: 44360 services.odex
- after: 24968 services.odex
Bug: b/237399630
Test: lunch aosp_sargo-userdebug && m
Change-Id: I844b9607c496d3d6e7048dc6bb8cd958ecbaa441
dexpreopt.GetGlobalConfig uses a OncePer to only parse the config once.
If the config contains missing paths, which can happen in partial
manifest branches, the missing path errors will only be reported on
the first caller to GetGlobalConfig. Fix the nondeterminism by
wrapping the PathContext and capturing any path errors and reporting
them to all callers to GetGlobalConfig.
Bug: 207813628
Test: tradefed branch builds
Change-Id: I869c3ae49a819b1251b94b73e0487aa594cd5e22
This reverts commit 0b1c70efbc.
The reverted commit was based on the idea that uses-libraries that are
explicitly specified in build files should not be implicitly added to
the manifest, as that would mean that anything added to the build files
will flow to the manifest.
Although this logic is correct, it prevents propagation of
uses-libraries from dependencies, which is wrong: if a library has an
explicit uses-library property in Android.bp, this property is expected
to be propagated to the library's dependencies. Failing to do so would
mean that every user of that library has to add uses-library property to
their build files, which doesn't scale (see b/214255490 for example).
Bug: 214255490
Test: lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd \
&& adb wait-for-device && adb root \
&& adb logcat | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
# empty output, no errors at boot
Change-Id: I6f420e76a89aa2f37be99f877711736640f2c361
Previously "any" was serialized as its numeric value, 10000. But other
code in makefiles and scripts expects string "any", and dexpreopt.config
files generated by Make (for Android.mk modules) have "any" not 10000.
Bug: 214255490
Test: lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd \
&& adb wait-for-device && adb root \
&& adb logcat | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
# empty output, no errors at boot
Change-Id: Id5e80eb8a90d9786b5cb999c172aaecb44952f76
A number of tests in the cts/tests/signature/api-check check for the
accessibility of classes from the android.test.base,
android.test.runner and android.test.mock libraries. Some tests expect
to find the classes other do not. Unfortunately, the tests use
libraries, specifically compatibility-device-util-axt, that depend on
the android.test... libraries which causes Soong to implicitly add
<uses-library> entries to the manifest so that they will be accessible
at runtime. That causes the tests that do not expect to find the
classes to fail.
Bug: 209607558
Test: m nothing
Change-Id: I54c194ab23d5a70df790ece3fe98f2b3d6a1c1f6
Standalone system server jars are dynamically loaded by system server
using a `PathClassLoader` whose parent is `SYSTEMSERVERCLASSPATH`. They
are listed in `PRODUCT_STANDALONE_SYSTEM_SERVER_JARS` and
`PRODUCT_APEX_STANDALONE_SYSTEM_SERVER_JARS` in Makefile. We need to
dexpreopt them to achieve better performance.
Bug: 203198541
Test: m nothing
Test: -
1. Add a standalone system server jar (e.g., by patching
aosp/1906158)
2. Build a system image.
3. See the odex and vdex files generated in
$ANDROID_PRODUCT_OUT/system/framework/oat/
4. Flash the image to a device.
5. Run `atest art_standalone_dexpreopt_tests`.
Change-Id: I358a62d34989c5c8eba12e18fe6167e0b72ff69d
We will need the profile when we generate the primary boot image on
device.
Bug: 203492478
Test: Run `banchan com.android.art x86_64 && m` and see
`$ANDROID_PRODUCT_OUT/apex/com.android.art/etc/boot-image.prof`.
Test: Run `lunch aosp_cf_x86_64_phone-userdebug && m` and see both
`$ANDROID_PRODUCT_OUT/apex/com.android.art/etc/boot-image.prof` and
`$ANDROID_PRODUCT_OUT/system/etc/boot-image.prof`, in different
sizes.
Test: Start Cuttlefish with the built image and see both
`/apex/com.android.art/etc/boot-image.prof` and
`/system/etc/boot-image.prof` on device.
Change-Id: Id879dc49b234133dfbb9563814328661a1f4a6c0
This CL updates platform_systemserverclasspath,
systemserverclasspath_fragment, and
prebuilt_systemserverclasspath_fragment to write entries for
STANDALONE_SYSTEMSERVER_JARS to systemserverclasspath.pb. A new property
`standalone_contents` is aded to `systemserverclasspath_fragment` and
`prebuilt_systemserverclasspath_fragment` to list the standalone system
server jars in the APEX, and entries will be written to
systemserverclasspath.pb accordingly at build time.
To add more context, these entries will be consumed by derive_classpath
in order to generate an environment variable
PRODUCT_STANDALONE_SYSTEM_SERVER_JARS. The environment variable will
then be comsumed by odrefresh to determine what jars to preopt on early
boot.
Note that the variable should not end with "CLASSPATH" because the list
is not used by runtime as a classpath. It is just a colon-separated list
of jars. System server loads the jars separately with paths hardcoded in
the code.
Bug: 203198541
Test: manual -
1. Add some jars to PRODUCT_STANDALONE_SYSTEM_SERVER_JARS
2. Add some other jars to PRODUCT_APEX_STANDALONE_SYSTEM_SERVER_JARS
and standalone_contents.
3. Build an image.
4. Flash the image to a device.
5. adb shell echo \$STANDALONE_SYSTEMSERVER_JARS
6. See the correct list of jars.
Change-Id: I09a6fd1d3db85c194330da9b751702a9bf069e26
Tests were using the same value for OutDir and SoongOutDir, separate
them to allow a test that needs to distinguish between them.
Test: all soong tests
Change-Id: Ibd78758c133a7f07bead3f51b699052676f92fbd
This change adds a Soong module that stores the paths to the compilation
artifacts of system server jars in a variable, which will then be
consumed by Make to check if the artifacts are installed. When the check
fails, it means that dexpreopting is not working for some system server
jars and needs to be fixed.
Bug: 201371822
Test: m nothing
Test: manual -
1. Add "service-permission" to DEXPREOPT_DISABLED_MODULES (https://cs.android.com/android/platform/superproject/+/master:build/make/core/product_config.mk?q=DEXPREOPT_DISABLED_MODULES)
2. m nothing
3. See the error:
Missing compilation artifacts. Dexpreopting is not working for some system server jars
Offending entries:
system/framework/oat/x86_64/apex@com.android.permission@javalib@service-permission.jar@classes.odex
system/framework/oat/x86_64/apex@com.android.permission@javalib@service-permission.jar@classes.vdex
Change-Id: I4816f19668f1dae180a34adafdbfa448c97aa0db
Previously, if an implementation jar (Host) was not available to the
build it would panic when trying to generate the JSON representation of
the CLC. That prevents builds with missing implementation jars from
working even if those jars are never actually built.
This change defers the build failure until it is actually built.
Test: TARGET_PRODUCT=armv8 TARGET_BUILD_VARIANT=eng ./art/test/testrunner/run_build_test_target.py -j80 art-no-prebuild
- run above in master-art before and after this change.
Bug: 202366925
Change-Id: I60a78a8bf6c13b83a9dceb5c43019a9e21f0b637
This is necessary in order to expose the copy rule to Ninja. Otherwise
Ninja has a build rule that depends on the copied jar (namely, dexpreopt
commands for all subsequent system server jars that have the current one
in class loader context), but no rule that creates it, so Ninja fails
with a "no rule to make ..." error.
The change only affects system server ed in Android.mk
Previously the problem existed, but was hidden by the fact that the only
system server jar defined in Android.mk was the last one on the list, so
no other jar depended on it. Now that updatable apex jars are also
dexpreopted (https://r.android.com/1828115) the problem was uncovered.
Bug: 200297762
Test: lunch bertha_x86_64-userdebug && m
Change-Id: Idbf2eacdf7ae89847aa9d755ef688eda3720ce89
This reverts commit 92346c4832.
Reason for revert: Fixed build error.
The build error is fixed by ag/15841934. This CL remains unchanged. This
CL will be submitted AFTER ag/15841934 is submitted.
Bug: 200024131
Test: 1. Patch this CL and ag/15841934 into internal master.
2. sudo vendor/google/build/build_test.bash
Change-Id: I5f2b8357846fc7dda56e25ebe6ffb095e8047ec8
The path to the artifacts will in the form of
/system/framework/oat/<arch>/<encoded-jar-path>@classes.{odex,vdex,art},
where <encoded-jar-path> is the path to the jar file with "/" replaced
by "@". For example,
/system/framework/oat/x86_64/apex@com.android.art@javalib@service-art.jar@classes.odex
There will be a follow-up CL to update ART runtime to recognize
artifacts in that path.
Test: m com.android.art
Bug: 194150908
Change-Id: Ic89fd63c4b1cd565684cead83fc91dae3bc97a4c
