Commit graph

96 commits

Author SHA1 Message Date
Jiakai Zhang
389a647320 Dexpreopt standalone system server jars.
Standalone system server jars are dynamically loaded by system server
using a `PathClassLoader` whose parent is `SYSTEMSERVERCLASSPATH`. They
are listed in `PRODUCT_STANDALONE_SYSTEM_SERVER_JARS` and
`PRODUCT_APEX_STANDALONE_SYSTEM_SERVER_JARS` in Makefile. We need to
dexpreopt them to achieve better performance.

Bug: 203198541
Test: m nothing
Test: -
  1. Add a standalone system server jar (e.g., by patching
     aosp/1906158)
  2. Build a system image.
  3. See the odex and vdex files generated in
     $ANDROID_PRODUCT_OUT/system/framework/oat/
  4. Flash the image to a device.
  5. Run `atest art_standalone_dexpreopt_tests`.
Change-Id: I358a62d34989c5c8eba12e18fe6167e0b72ff69d
2021-12-20 15:07:26 +00:00
Jiakai Zhang
0a0a2fbea9 Add a build-time check for dexpreopting system server jars.
This change adds a Soong module that stores the paths to the compilation
artifacts of system server jars in a variable, which will then be
consumed by Make to check if the artifacts are installed. When the check
fails, it means that dexpreopting is not working for some system server
jars and needs to be fixed.

Bug: 201371822
Test: m nothing
Test: manual -
  1. Add "service-permission" to DEXPREOPT_DISABLED_MODULES (https://cs.android.com/android/platform/superproject/+/master:build/make/core/product_config.mk?q=DEXPREOPT_DISABLED_MODULES)
  2. m nothing
  3. See the error:
    Missing compilation artifacts. Dexpreopting is not working for some system server jars
    Offending entries:
    system/framework/oat/x86_64/apex@com.android.permission@javalib@service-permission.jar@classes.odex
    system/framework/oat/x86_64/apex@com.android.permission@javalib@service-permission.jar@classes.vdex
Change-Id: I4816f19668f1dae180a34adafdbfa448c97aa0db
2021-10-29 11:55:20 +00:00
Ulya Trafimovich
29e95d94bf Move copy rule for system server jars from dexpreopt_gen to makefiles.
This is necessary in order to expose the copy rule to Ninja. Otherwise
Ninja has a build rule that depends on the copied jar (namely, dexpreopt
commands for all subsequent system server jars that have the current one
in class loader context), but no rule that creates it, so Ninja fails
with a "no rule to make ..." error.

The change only affects system server ed in Android.mk

Previously the problem existed, but was hidden by the fact that the only
system server jar defined in Android.mk was the last one on the list, so
no other jar depended on it. Now that updatable apex jars are also
dexpreopted (https://r.android.com/1828115) the problem was uncovered.

Bug: 200297762
Test: lunch bertha_x86_64-userdebug && m
Change-Id: Idbf2eacdf7ae89847aa9d755ef688eda3720ce89
2021-09-21 14:08:32 +01:00
Jiakai Zhang
519c5c82e5 Revert^2 "Preopt APEX system server jars."
This reverts commit 92346c4832.

Reason for revert: Fixed build error.

The build error is fixed by ag/15841934. This CL remains unchanged. This
CL will be submitted AFTER ag/15841934 is submitted.

Bug: 200024131
Test: 1. Patch this CL and ag/15841934 into internal master.
  2. sudo vendor/google/build/build_test.bash

Change-Id: I5f2b8357846fc7dda56e25ebe6ffb095e8047ec8
2021-09-16 06:25:26 +00:00
Adrian Roos
92346c4832 Revert "Preopt APEX system server jars."
This reverts commit ca9bc98e0c.

Reason for revert: breaks build
Bug: 200024131

Change-Id: Ide07b4c4d267370ae31107b1598b2f878c701282
2021-09-15 14:11:07 +00:00
Jiakai Zhang
ca9bc98e0c Preopt APEX system server jars.
The path to the artifacts will in the form of
/system/framework/oat/<arch>/<encoded-jar-path>@classes.{odex,vdex,art},
where <encoded-jar-path> is the path to the jar file with "/" replaced
by "@". For example,
/system/framework/oat/x86_64/apex@com.android.art@javalib@service-art.jar@classes.odex

There will be a follow-up CL to update ART runtime to recognize
artifacts in that path.

Test: m com.android.art
Bug: 194150908
Change-Id: Ic89fd63c4b1cd565684cead83fc91dae3bc97a4c
2021-09-15 09:01:07 +00:00
Ryan Savitski
aa3c355820 eng builds: dexpreopt: stop explicitly removing minidebuginfo
This special-casing is actively unhelpful nowadays, as it breaks
callstack unwinding and symbolisation by tools that work using the
on-device debug info (e.g. perfetto's profilers). It's also highly
surprising that eng builds have less debug info than userdebug/user.

As far as I can tell, the history behind this case is (with some
internal-only references as the context is in the review comments): the
minidebuginfo was originally rolled out in device-specific makefiles
(e.g. ag/2433691). Eng builds were excluded, presumably due to time/size
concerns for rebuild+resync workflow, as well as the assumption that
engineers using eng builds already have the necessary debug info
off-device. Later, as some more makefile variables were introduced, the
conversation in ag/990059 shows that the overheads were no longer a
concern, and the last patchset generalised the option to include eng
builds. However at a later point, during device-specific makefile
cleanup, the eng-exclusion got restored: ag/2455645. This then ended up
in the makefile->soong transition, leaving us with the current special
casing that is undesireable.

Checking things with an internal master build on my workstation:
dexpreopting system services is 10s regardless of this minidebuginfo
flag, and the resulting odex size is 37MB vs 38MB. Overall, I only see 4
dexpreopt invocations as part of a local build, so the effect of this
patch should be negligible.

Bug: 194835631
Change-Id: I80de75a84259c056cdae0696fe611afe7e0162f5
2021-08-04 19:34:55 +01:00
satayev
492b17d533 Rename UpdatableSystemServerJars to ApexSystemServerJars.
This is consistent with naming for boot jars, e.g. ApexBootJars.

Bug: 191127295
Test: presubmit
Change-Id: I6e8828d55ac86b7f5260ed7f8d4eca1c0b3dced8
2021-07-28 14:04:49 +01:00
Jeongik Cha
4dda75e73e Add usesTargetFiles option in dexpreopt_gen
For running dex2oat on the target_files, the paths should be use the
device install path instead of the path starting with $(OUT).
So add usesTargetFiles option and basePath option which indicates
extracted path. With those options, the path is replaced with
$(basePath)/$(device path)

And also, add DexPreoptImageDeviceLocations in the config which refers
to the boot image path(without arch) on the device. Because
DexPreoptImage related device path was missing.

Bug: 158843648
Test: dexpreopt_gen -usesTargetFiles -basePath (extract path) and then
check if paths in the generated shell script are based on on-device
path.

Change-Id: I9667fadbf3b7c6f770e0d1bcbee5d67c1ecd8a3d
2021-05-15 00:29:49 +09:00
Treehugger Robot
564fce4578 Merge "Rename fields in dexpreopt config" 2021-05-08 01:47:39 +00:00
Jeongik Cha
a596909342 Rename fields in dexpreopt config
Add 'host' into the name of fields regarding path on the host side to
distinguish between paths on the device(which will be added in the
following commit), and paths on the host.

Bug: 158843648
Test: build and flash, and then
  adb wait-for-device \
    && adb -s $S root \
    && adb -s $S logcat \
    | grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C 1
Change-Id: Ib2645ed51591ba2f4b726c115b401ad2bd6675da
2021-05-07 21:40:23 +09:00
satayev
9a6f87ebd6 Make SystemServerJars ConfiguredJarList.
Consistent with plumbing of boot jars. SystemServerJars only support /system/framework/ jars.

Bug: 180105615, 155630745
Test: m && launch_cvd
Change-Id: I58b005b7c4103c8e250090e995b1d9b2f9ed4a76
2021-05-05 14:24:02 +01:00
Vladimir Marko
982e384a63 Fix profileCommand to remove the profile first.
This fixes a discrepancy between clean and incremental
builds exposed by
    https://android-review.googlesource.com/1671710 .

Test: m
Bug: 148067697
Change-Id: I0de078045075417a4123f7c3821c723f1e8e6b68
2021-04-29 09:10:30 +01:00
Vladimir Marko
230bd421a8 profman: Clean up output profile type options.
Test: m  # Check output boot profiles with hexdump.
Bug: 148067697
Change-Id: Ic20f415ad4eee1fd6396f320b757420884764d2c
2021-04-23 15:19:16 +01:00
Jeongik Cha
33a3a8182f Made ManifestPath OptionalPath instead of Path
because it is actually an optional value.

Bug: 158843648
Test: m nothing
Change-Id: If323eacf6a7aa83dcf5ce4825ad8cabeeaa97ac9
2021-04-15 09:12:49 +09:00
Ulya Trafimovich
dd622951a4 Fix AAPT path in dexpreopt.
Dexpreopt uses AAPT to parse `targetSdkVersion` from the manifest, so it
the generated dexpreopt.sh script that calls AAPT must depend on it,
otherwise AAPT might not be built by the time it is used. Tool
dependencies are handled via the global Soong config and the
DEXPREOPT_GEN_DEPS variable that Soong generates for Make. This config
always uses Soong tool paths, like out/soong/host/linux-x86/bin/aapt
rather than out/host/linux-x86/bin/aapt.

This CL fixes a mistake in dexpreopt rule that used context-dependent
AAPT path (so, when called from Make, the dependency was on Soong tool,
but a Make tool was actually used, so it failed sometimes).

Bug: 132357300
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I1f0ab4afac98e6239f324e7f3571d670fd7a36cd
2021-03-25 12:52:49 +00:00
Ulya Trafimovich
bd6b076f80 Allow modules that don't run verify_uses_libraries to have nonempty CLC.
There are modules that would have passed verify_uses_library check, but
don't run it for some reason (the check gets enabled either with an
explicit setting, or if the module has nonempty <ues-library> lists in
the build properties). Previously all such modules were assumed to have
empty CLC, which is not always true. In particular, compatibility
libraries are ignored, which affected e.g. Calendar and messaging apps.

This CL gives such apps a chance to have correct CLC. The goal for the
future is to enforce verify_library_check by default.

Bug: 132357300
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: Iea3be0fc9d7775c52950848b5a3fd3b7fcd36c53
2021-03-22 10:49:31 +00:00
Ulya Trafimovich
0aba252e15 Reimplement verify_uses_libraries.sh in manifest_check.py.
Previously there were two different scripts that did similar things:
1) build/soong/scripts/manifest_check.py
2) build/make/core/verify_uses_libraries.sh

Both scripts extracted <uses-library> tags and `targetSdkVersion` from
the manifests of Java modules, but 1) worked for XML manifests, and 2)
worked for APKs. This CL reimplements the functionality from 2) in 1),
so that one script can handle both XML manifests and APKs.

Bug: 132357300

Test: lunch cf_x86_64_phone-userdebug && m && launch_cvd \
      adb wait-for-device && adb root && adb logcat \
      | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
      # empty grep output, no errors

Change-Id: Id1b66e4f3f30f307dba70cb111c7571762cb546a
2021-03-15 13:50:49 +00:00
Nicolas Geoffray
b4f9d5c837 Merge "Stop using '&' for class loader context." 2021-03-08 11:16:09 +00:00
Nicolas Geoffray
863f3b9fb4 Stop using '&' for class loader context.
Test: m
Bug: 132357300
Change-Id: I01a5008d5b84820f793c1c23fe2e4c4687cf4acf
2021-03-05 18:14:10 +00:00
Ulya Trafimovich
4a13acb07d Use compiler filter "verify" for dexpreopt w/o class loader context.
In cases when class loader context cannot be computed at build time and
verify_uses_libraries check is relaxed (RELAX_USES_LIBRARY_CHECK=true)
dexpreopt uses a special compiler filter that suppresses AOT-compilation
to native code and applies only those optimizations that do not require
class loader context. Previously the "extract" filter was used. Now ART
supports using "verify" filter in this configuration.

Bug: 132357300
Test: lunch cf_x86_64_phone-userdebug && m && launch_cvd \
      adb wait-for-device && adb root && adb logcat \
      | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
      # empty grep output, no errors
Change-Id: Id4933aa94ea96894278355283383c16d103e98c9
2021-03-02 12:32:23 +00:00
Ulya Trafimovich
8c35fcf4d5 Add non-fatal mode for verify_uses_libraries check.
The new mode is enabled with environment variable
RELAX_USES_LIBRARY_CHECK. If the variable is set to true, then a
verify_uses_libraries check failure does not fail the build, instead it
sets a special compiler filter "extract" for dexpreopt, which means that
the DEX file will be extracted, but it won't be compiled to native code.
Class loader context will be set to empty in this case (not &, as it is
going to be deprecated soon).

If the variable RELAX_USES_LIBRARY_CHECK is unset or set to something
other than "true", then the old behaviour of the verify_uses_libraries
check is preserved.

The intended use case for this flag is to have a smoother migration path
for the Java modules that need to add <uses-library> information in
the build files. The flag allows to quickly silence build errors. This
flag should be used with caution and only as a temporary measure, as it
masks real errors and affects performance.

verify_uses_libraries check is reworked so that it writes the error
message to a status file (which is used instead of the former timestamp
file). Currently the stored error message is not used, but it may be
used later to produce a warning. Dexpreopt command checks if the status
file exists and is nonempty; if that is the case, then compiler filter
is set to "extract".
Bug: 132357300

Test: Manually add some mismatch between the libraries in the Android.bp
      and Android.mk files for dexpreopted apps, build with
      RELAX_USES_LIBRARY_CHECK=true and obsserve that the build doesn't
      fail and they are compiled with compiler-filter "extract".
      Unset RELAX_USES_LIBRARY_CHECK and observe that the build fails.

Change-Id: Ibb5d993a25b1df1d2e70b7d5aafc6997f9d64e67
2021-02-22 15:41:03 +00:00
Colin Cross
f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Ulya Trafimovich
8cbc5d269b Rework class loader context implementation.
The old representation consisted of a list of libraries (UsesLibraries),
a list of optional libraries (OptionalUsesLibraries) and a mapping from
library name to its build/install paths (LibraryPaths). The separation
into lists and map was necessary because of special handling of
compatibility libraries, which is now unified with normal libraries.

The new representation is a mapping from target SDK version to a tree
structure ClassLoaderContext. Each node of the tree represents a library
and contains library name, build/install paths and a slice of
subcontexts for dependencies. The same library may occur in the tree
multiple times in case it is a dependency of multiple libraries. The
order in which libraries are added matters (the resulting tree shape may
be different).

Test results have to be updated, as the resulting <uses-library> list is
reodered (previously it was a sorted list of map keys, and now it is
formed by a depth-first preorder traversal of the class loader tree).

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: I11be8cd2967f004fd58753d7c5fb99fed179cd63
2020-11-03 15:15:46 +00:00
Ulya Trafimovich
6961267298 Add unit tests for class loader context.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I96a1e6cef86652e429b1678a655fc0b02f40d00c
2020-10-29 17:47:34 +00:00
Ulya Trafimovich
eb26886c85 Move class loader context definitions to a separate file.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I1e7e9db1654d0b835276be1cfa6a8eeffc5e96ee
2020-10-29 17:47:34 +00:00
Ulya Trafimovich
8130c482ab Move part of logic from construct_context.py to Soong.
construct_context.py is a script that runs at Ninja stage and constructs
class loader context arguments for dex2oat. Previously it accepted lists
of library paths and concatenated them into one class loader context
string. The script also handled the dependency of "android.hidl.manager"
on "android.hidl.base" that is not tracked by the build system and must
be handled in a special way.

Now that class loader context representation is going to change from
flat lists to trees, passing paths to individual libraries as script
arguments is no longer possible, because a list of paths cannot
represent a class loader context tree. Passing the trees in a serialized
form is also inconvenient, because the script would have to parse them,
which would complicate it a lot.

Therefore this patch ports all the concatenation and "android.hidl.base"
handling to Soong. It is not possible to port the remaining script to
Soong because Soong has no information about the targetSdkVersiion of
the dexpreopted apps (it is in the manifest and sometimes in an APK, and
has to be extracted at Ninja time).

Test: construct_context_test.py
Test: new subtests in TestUsesLibs
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: Icdb03cf00d1e27e4cff3844b89bfaec4de502dd7
2020-10-27 17:05:30 +00:00
Ulya Trafimovich
46b3d5bd05 Fix handling of "android.test.mock" in class loader context.
Mimick the way PackageManager handles it at runtime: do add it to class
loader context for apps with targetSdkVersion < 30, but only if
"android.test.runner" is used. Previously it was not added at all.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I4c06635277ab13e21069b9fa0b46eb6a2547dfdd
2020-10-27 17:05:30 +00:00
Ulya Trafimovich
24813e1d80 Unify handling of compat and normal libs in class loader contexts.
Also, add tests for compatibility libraries in class loader context.

This CL separates special-case handling of compatibility libraries into
a "fixup" step that is done after class loader context is constructed by
Soong. This allows to handle compatibility libraries and normal
libraries uniformly, which should enable further simplification of class
loader context representation (in subsequent CLs).

Currently the only "fixup" step is removal of libraries from conditional
class loader context if they already are in unconditional context. This
check cannot be done at the time when the libraries are added to
conditional context, because the full uncoditional context is not yet
known at that time. Previously construction of unconditional context was
delayed, now it is no longer delayed and handled in the same way as
unconditional context, and the "fixup" does the filtering.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: Ie71e9fb2d3d529b5317cd84e09ab3c853017c349
2020-10-27 17:05:30 +00:00
Ulya Trafimovich
a54d33be77 Fail the build if dexpreopt cannot find path to a <uses-library>.
This doesn't apply to compatibility libraries (those added to
conditional class loader context base on the targetSdkVersion in the
manifest), because Soong doesn't know if they are needed or not (the
build rule that parses the manifest runs at Ninja stage).

In the future we may need to relax this and instead of failing the build
to disable dexpreopt in such cases. But currently there are no use cases
that fail.

Also, this patch replaces hardcoded install path with a constant.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: Ic894bc17e8ffd8ff173111ea09366252001ef3e9
2020-10-06 14:04:49 +01:00
Anton Hansson
da4d9d967d Revert "Re-land "Move odexes of non-system apps into system_other""
This reverts commit 75480154e2.

Bug: 141707536
Test: Clean revert
Reason for revert: breaks some builds
Change-Id: If0cf736099ec12e35bcb190d0fc33daf7f1da98e
2020-09-15 09:29:11 +00:00
Anton Hansson
75480154e2 Re-land "Move odexes of non-system apps into system_other"
Previously, the patterns in SYSTEM_OTHER_ODEX_FILTER only matched
subpaths of /system. Broaden this to match /<p> as well as /system/<p>.

Because SYSTEM_OTHER_ODEX_FILTER includes system_ext and product apps,
the immediate effect of this is that the odex and vdex files of
apps on those partitions move to system_other for A/B devices.

This makes the product and system_ext partitions smaller, which
in turn makes OTAs smaller.

Bug: 141707536
Test: soong unit tests
Test: (adb shell dumpsys package com.android.stk &&
       adb shell dumpsys package com.android.quicksearchbox) | grep -i dexopt -A3;
      verify dexpreopt state is 'prebuilt' for both
Test: adb shell find /data/dalvik-cache -name '*QuickSearch*.vdex' | wc -l == 1
Test: adb shell find /data/dalvik-cache -name '*Stk*.vdex' | wc -l == 1
Change-Id: I0cf141014b9ddf32159cff47b0325d9c78b549f2
2020-09-11 14:45:51 +01:00
Ulya Trafimovich
fc24ad3d4e Propagate transitive SDK Java library dependencies to dexpreopt.
For some dependencies, like stubs, the SDK library may not be found at
build time (either because the implementation library is not among the
dependencies of the dexpreopted module, or because it's part of a
prebuilt, or because it's missing from the build altogether). In such
cases dexpreopt is useless, because dex2oat does not have access to the
full classpath (unless the &-classpath is used). Therefore do not
dexpreopt in such cases.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: If289088cfd103011ccb16165e95a97b30fd31b81
2020-08-27 11:47:15 +01:00
Ulya Trafimovich
b8063c6a86 Use common helper functions for getting sorted map keys.
Add a new helper SortedIntKeys similar to SortedStringKeys.

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I08a43ec2cae7d1a82531295aca1a0658e3a0dd6f
2020-08-20 12:37:03 +01:00
Ulya Trafimovich
249386ad69 Add structured representation for colon-separated jar lists.
With the addition of apexes and /system_ext some of the bootclasspath
and system server jars have moved from /system to the new locations.
This has been implemented by using lists of colon-separated strings
called "apex-jar pairs" (although "apex" was misleading as it could
refer to "platform" or "system_ext", not necessarily a real apex).

Using the colon-separated string representation is inconvenient, as it
requires splitting and reassembling the list components many times,
which harms performance and makes error handling difficult. Therefore
this patch refactors the colon-separated lists into a struct that
hides the implementation details.

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: Id248ce639a267076294f4d4d73971da2f2f77208
2020-08-05 09:59:30 +01:00
Ulya Trafimovich
5f364b63b8 Rewrite construct_context.sh in Python.
This allows to reuse SDK version comparison routine from other Python
scripts. With the addition of non-numeric versions comparison has become
more complex, and the deleted shell script did it incorrectly: it used
comparisons like `[[ "S" -lt 28 ]]` which results in "true" as strings
are converted to zero in numeric context. This resulted in adding legacy
libraries to class loader context of apps targeting recent SDK versions.
The error was masked because currently there is only one non-AOSP app
that uses the script (GoogleDialer), and it targets numeric SDK version.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I26e752e29b5453fd3e29d5dbfbe4d9df9c0a55b7
2020-07-01 09:44:23 +01:00
Ulya Trafimovich
c4dac26596 Refactor class loader context generation.
There are essentially three different cases: 1) system server dexpreopt,
2) the case when uses-library information is known, and 3) the case when
uses-library information is unknown and the &-classpath hack is used.
This patch reorganizes the code into three branches that correspond to
the above cases.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: Ife2d01a612e0a608ae1346d7007741498e938bc5
2020-06-30 12:40:54 +01:00
Ulya Trafimovich
6e82748bd6 Rename dexpreopt.config field to bring it in sync with Make.
Different names of the same field in Make and in Soong resulted in
the field being dropped when unmarshalling the JSON from Make to Soong.
As a result, the list of optional used libraries for Make modules was
always empty.

Test: lunch aosp_cf_x86_phone-userdebug && m
Test: Cherry-pick in internal master and observe that the logcat error
    "ClassLoaderContext shared library size mismatch" for GoogleDialer
    has disappeared:
    terminal-1 $ lunch cf_x86_phone-userdebug && m && launch_cvd
    terminal-2 $ adb wait-for-device && adb root \
        && adb logcat | grep 'size mismatch'
    (nothing after the change, one error message before the change)

Bug: 132357300
Change-Id: I57820646fa7c1d8af01b2cbc8a3521da450719c0
2020-06-12 14:47:50 +01:00
Ulya Trafimovich
d4bcea4e24 Refactor library path representation in dexpreopt.
This is a preliminary CL before fixing on-device paths to DEX jars.
It groups together the inormation about on-host build paths and
on-device install paths to library DEX jars.

This CL changes the structure of module dexpreopt.config files
generated by the build system. Aside of that, no functional changes.

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I059654be7670f2ba66248d9c49b9694a0591f9c1
2020-06-04 17:37:42 +01:00
Ulyana Trafimovich
eb3f9f1295 Merge "Error on suboptimal system server classpath order, unless explicitly allowed." 2020-06-04 08:56:04 +00:00
Ulya Trafimovich
cd3203fd5d Error on suboptimal system server classpath order, unless explicitly allowed.
The order is non-optimal if some jar X precedes its dependency Y. In
that case dexpreopt will be unable to resolve any rerefences from X to
Y. Raise an error unless the product sets variable
PRODUCT_BROKEN_SUBOPTIMAL_ORDER_OF_SYSTEM_SERVER_JARS to true in the
makefile. This is to prevent regressions in existing products that
currently have correct order.

Test: lunch cf_x86_phone-userdebug && m nothing
Test: lunch aosp_car_arm && m nothing

Bug: 140451054

Change-Id: Iafe8fe0e992deb628e38f2321d4601a4804a3c79
Merged-In: Iafe8fe0e992deb628e38f2321d4601a4804a3c79
(cherry picked from commit fb863c87c4)
2020-06-03 17:37:34 +01:00
Ulya Trafimovich
696c59dbfa Clean up class loader context generation.
Deduplicate code for conditional class loader contexts that depend
on the target SDK version in the manifest.

Test: lunch aosp_cf_phone-userdebug && m
Change-Id: I69d51b212bf502b2659d43f124ea3922f90e1881
2020-06-01 17:55:56 +01:00
Ulya Trafimovich
c9af538c90 Add android.test.base to class loader context for API version < 30.
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: Id2ece4b2a6317998e1dd177e853848cc3956f39f
2020-06-01 12:51:38 +01:00
Ulya Trafimovich
df00ddecb6 Don't use obsolete "org.apache.http.legacy.impl" in class loader context.
This library no longer exists on device and "org.apache.http.legacy"
should be used instead. The error lingered in the code for a while
because the code was not exercised at all. (The only app that sets
EnforceUsesLibraries is GoogleDialer (not in AOSP), and it specifies
"org.apache.http.legacy" explicitly as an optional used library.)

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I05fa115476315d35a8877fc42bd1c2042f72f2de
2020-05-29 15:00:30 +01:00
Ulya Trafimovich
8640ab9c26 Raise an error instead of panic in SplitApexJarPairs.
Test: aosp_walleye-userdebug boots
Change-Id: I9a1c4ec5c8862b1936820b3cee6db9a85015cb0e
2020-05-12 12:27:50 +01:00
Ulya Trafimovich
50c4a4b19f Use two-column format for PRODUCT_BOOT_JARS components.
The first component is the apex name, or a special name "platform"
if the boot jar is a platform jar rather than a part of some apex.
This is a prerequisite change for moving core-icu4j to a separate
com.android.i18n apex.

Old one-column format is still supported, but all unqualified
components of PRODUCT_BOOT_JARS get "platform:" prepended to them
after reading the product makefiles.

Test: aosp_walleye-userdebug boots
Bug: 138994281
Change-Id: Ic229159fdcdaf6182210a53b63850a389dd786fc
2020-04-30 14:42:32 +01:00
Ulya Trafimovich
6cf2c0c197 Fix host path for dexpreopted system server modules defined in Make.
Previously there was a discrepancy between the generated paths for
Soong modules that used "out/soong/system_server_jars" and Make modules
that used "out/system_server_jars". This happened because the default
output directory is $OUT/soong for the normal Soong config and jsut $OUT
for the reduced "null config" created by dexpreopt_gen.

As a result, class loader context for system server jars defined in Make
was referring to nonexistent jars, which caused non-fatal dex2oat errors
(the build was not broken, so the errors remained unnoticed):

  Failed to determine oat file name for dex location out/system_server_dexjars/[...]
  Could not open dex files from location: out/system_server_dexjars/[...]

Test: aosp_walleye-userdebug boots
Test: cherry-picked in master-arc-dev that has a Make system server jar
  arc-services, `lunch lunch cheets_x86-userdebug && m`, check that the
  generated dexpreopt.sh for arc-services contains paths starting with
  "out/soong/system_server_jars" rather than "out/system_server_jars".

Bug: 140712566
Change-Id: Ia7ea8ac383e32042c31d64971cdc8101ea3068cd
2020-04-27 09:46:02 +01:00
Ulya Trafimovich
55285bab77 Don't append build directory to the output path twice.
Soong already uses build directory as a part of the generated output
path. Besides, trying to append it second time breaks if the build
directory is an absolute path.

Test: frameworks/rs/build_rs.py  (uses absolute build directory path)
Test: walleye-userdebug boots

Bug: 152235239
Change-Id: Ie03c9e688013b9a3e6bc859c936d89c538ded76a
Merged-In: Ie03c9e688013b9a3e6bc859c936d89c538ded76a
(cherry picked from commit f798c3ec32)
2020-03-24 17:02:19 +00:00
Ulya Trafimovich
b358ebb759 Fix null pointer dereference in Soong on dexpreopt arc-services.
The null pointer dereference was caused by trying to get device name
from a config passed to dexpreopt. For Make modules, the config is a
stub that doesn't have a device name. The fix is to use build directory
instead of device name (it is provided both by the full config passed
to Soong modules and by the stub config passed to Make modules).

Test: walleye-userdebug boots.
Test: aosp_bertha_arm-userdebug builds and generates arc-services.odex.

Bug: 152193534
Change-Id: I7adc173646cae6eaee188c6672629a5a942aea06
Merged-In: I7adc173646cae6eaee188c6672629a5a942aea06
(cherry picked from commit 9790f1058b)
2020-03-23 17:13:07 +00:00
Ulya Trafimovich
dacc6c56f1 Simplify the construction of class loader contexts for system server jars.
This reworks CL https://r.android.com/1180134 as follows:

1) Do not reorder the list of system server jars passed from Make to
  Soong via the product variable PRODUCT_SYSTEM_SERVER_JARS. This means
  that for some products the order of jars on the system server
  classpath may be non-optimal: a jar X that depends on Y may be
  dexpreopted before Y, so that all references to the classes and
  methods from Y wil be unresolved.

  Unfortunately for such products, fixing the order is not a simple
  matter of rearranging their PRODUCT_SYSTEM_SERVER_JARS, because the
  conflicts may arise when the product-specific variable gets merged
  with the common variable.

2) As a consequence of 1), do not add artificial dependencies between
  system server jars: this is now impossible, as it would create
  circular dependencies for those products that have non-optimal order
  of jars.

3) Copy dex files for system server jars to a predefined build location.
  This is necessary because at the time when Soong generates class
  loader context for k-th jar, it needs to know the paths to jars 1 ..
  (k-1), and it might have not processed those jars yet (so Soong can't
  query the paths from the modules).

  This approach is similar to the way Soong handles bootclasspath jars.

4) Do not exclude from dexpreopting system server jars that are not
  Soong modules (those that are Make modules). The previous CL excluded
  them because Make modules do not have ModuleContext. But it turns out
  that ModuleContext is not necessary, as all the information is passed
  via the dexpreopt config.

Test: aosp_walleye-userdebug boots and there are no messages in the
  logcat regarding class loader context mismatch:

  $ adb logcat | grep 'mismatch'
  # empty

Test: Class loader contexts in the oat files for system server jars
  match expectations:

  $ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/com.android.location.provider.odex 2>/dev/null | grep '^classpath'
  classpath = PCL[]
  $ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/services.odex 2>/dev/null | grep '^classpath'
  classpath = PCL[/system/framework/com.android.location.provider.jar*1989208671]
  $ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/ethernet-service.odex 2>/dev/null | grep '^classpath'
  classpath = PCL[/system/framework/com.android.location.provider.jar*1989208671:/system/framework/services.jar*4040443083:/system/framework/services.jar!classes2.dex*2210087472]

Test: The phone boots and logcat has no scary messages related to
  class loader contexts:

  $ lunch aosp_walleye-userdebug && m
  $ adb reboot bootloader && fastboot flashall -w && adb wait-for-device
  $ adb root
  $ adb shell stop
  $ adb logcat -c
  $ adb shell setprop dalvik.vm.extra-opts -verbose:oat
  $ adb shell start
  $ adb logcat | egrep -io 'system_server: .*load.*/system/framework.*'
  system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 0
  system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 0
  system_server: Loading /system/framework/oat/arm64/services.odex with executable: 0
  system_server: Successfully loaded /system/framework/oat/arm64/services.odex with executable: 0
  system_server: Loading /system/framework/oat/arm64/ethernet-service.odex with executable: 0
  system_server: Successfully loaded /system/framework/oat/arm64/ethernet-service.odex with executable: 0
  system_server: Loading /system/framework/oat/arm64/wifi-service.odex with executable: 0
  system_server: Successfully loaded /system/framework/oat/arm64/wifi-service.odex with executable: 0
  system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
  system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
  system_server: Loading /system/framework/oat/arm64/services.odex with executable: 1
  system_server: Successfully loaded /system/framework/oat/arm64/services.odex with executable: 1
  system_server: Loading /system/framework/oat/arm64/ethernet-service.odex with executable: 1
  system_server: Successfully loaded /system/framework/oat/arm64/ethernet-service.odex with executable: 1
  system_server: Loading /system/framework/oat/arm64/wifi-service.odex with executable: 1
  system_server: Successfully loaded /system/framework/oat/arm64/wifi-service.odex with executable: 1
  system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
  system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1

Bug: 141785760
Bug: 140451054
Bug: 148944771
Bug: 147017252

Change-Id: I33c4087f8bfacd0ecb89877aa150b75360d06710
Merged-In: I33c4087f8bfacd0ecb89877aa150b75360d06710
(cherry picked from commit a4a83b0ef9)

Exempt-From-Owner-Approval: cherry-pick.
2020-03-23 12:27:31 +00:00