Commit graph

68 commits

Author SHA1 Message Date
Spandan Das
7eb9243fa7 Revert "R8/D8 should use sdk_version prop to determine API surface stability."
This reverts commit daa220ae8c.

Reason for revert: Suspected to be a cause for build failure (b/264446640). Will use go/abtd to verify

Change-Id: Ibfdf893f2426b1119923467027e0acedd08808c1
2023-01-04 22:14:59 +00:00
Spandan Das
daa220ae8c R8/D8 should use sdk_version prop to determine API surface stability.
`min_sdk_version` is used to represent the api_level of the device and
its type will eventually become android.ApiLevel. OTOH,
`sdk_version` property represents the API surface a module builds against
_and_ the version of that API surface. For
R8/D8, the additional `--android-platform-build` should be determined
using the sdk_version of the soong module and not min_sdk_version, since
min_sdk_version will not contain information about the api surface used
for compilation.

The unit test for `core_platform_app` in TestR8 were passing since
min_sdk_version was not set, and therefore it implicitly defaulted to
sdk_version.

Also created a custom struct to propagate params to the helper dex
functions

Test: In build/soong, go test ./java
Test: TH
Bug: 208456999
Change-Id: I08ac6f496444d603557e498c8a1794af665abc7a
2023-01-04 00:16:45 +00:00
Jared Duke
51b0a10229 Support testing for resource shrinking
Add a simple test and add necessary mock dependencies to allow future
resoure shrinking use with platform targets.

Test: m
Bug: 202959019
Change-Id: Id2dd44d52ce5ea62c06784caab0af6276248cb3f
2022-09-28 09:46:35 -07:00
Rico Wind
6008155960 Merge "Add soong support for resource shrinking" 2022-09-24 05:19:33 +00:00
Rico Wind
351bac996c Add soong support for resource shrinking
This adds a new flag to the optimize section to allow invoking the resource shrinker

Bug: 246217952
Test: m
Change-Id: I2e7851af1189db2a6adf6f9f9f444a1d7f3a8d60
2022-09-23 19:11:11 +02:00
Jared Duke
40d731a4c3 Provide platform build flag to R8/D8
Provide the `--android-platform-build` flag to R8/D8 for targets that do
not specify a stable min SDK. This includes all targets that do not
specificy an SDK version (e.g., services.jar, SystemUI.apk) and all
targets that specify core_platform (e.g., framework.jar).

Follow-up work will expand the set of cases that qualify for this flag,
but for now this captures some of the highest priority targets.

Bug: 232073181
Test: m + presubmit + validate verbose.log
Change-Id: Idad32c80490a101c049b7f937807a615564ca8ef
2022-09-21 11:01:32 -07:00
Remi NGUYEN VAN
bdad314f97 Make ignorewarnings optional in optimize
This allows targets to opt-in to running R8 without the -ignorewarnings
flag. Ignored warnings can cause code to be optimized incorrectly, so
this makes the build safer for such targets.

Ideally ignore_warnings should default to false, but all targets need to
be fixed for that to happen. As a first step, provide a mechanism for
individual targets to ensure they do not introduce regressions.

Bug: 229727645
Bug: 180878971
Bug: 226127213
Bug: 239990030
Test: m (runs dex_test)
Change-Id: Ic0eef29598c1ee47e958da8a5048d9696165a235
2022-08-04 13:23:47 +09:00
Jared Duke
63a3da94d1 Revert "Revert "Use D8 by default for android_test""
This reverts commit 4e445be558.

Reason for revert: GtsExoPlayerTestCases now uses multidex.

Test: m + manual builds for test_suites_x86_64_coverage
Change-Id: I608b3b79137b216f546446e1c4f89f4542572581
Bug: 192032291
2022-06-02 19:12:30 +00:00
Jared Duke
4e445be558 Revert "Use D8 by default for android_test"
This reverts commit 02edc10047.

Reason for revert: Breaks test_suites_x86_64_coverage

Bug: 233421462
Change-Id: I7b04d3fd7802be0f271ea3c29ef25e3d08ab1389
2022-05-23 04:00:44 +00:00
Jared Duke
02edc10047 Use D8 by default for android_test
android_test defaults to using R8, but with shrinking, optimization and
obfuscation disabled, eliminating most of the benefits of R8. Instead,
use D8 by default, improving build performance and avoiding any other
issues that may arise in test-specific code related to whole-program R8
execution. An initial audit shows that android_test targets that *do*
enable shrinking or optimization also explicitly opt in to R8.

A follow-up CL will do the same for android_test_helper_app, but that
requires some additional auditing of downstream targets.

Bug: 192032291
Test: m + presubmit
Change-Id: I5b14a0986dde210f241a77c3a93daacf9e53d667
2022-05-20 12:02:13 -07:00
Jared Duke
0cf7c9686d Limit JIT tier for D8
Standalone D8 invocations tend to be shorter-lived than R8 invocations
that involve deeper layers of bytecode optimization. Introduce a
separate set of JVM flags to pass to D8 invocations which limit the
runtime JIT tiering, reducing overall compile times and significantly
reducing total CPU consumption for D8 build workloads.

Representative incremental build times with this change:
 * real 6m1.388s
 * user 63m32.412s
and without:
 * real 6m30.624s
 * user 79m12.626s

Bug: 205303544
Test: time m
Change-Id: Ic568a4316325c6a4e0da41ffebbb07f0c5f7e8e3
2022-04-21 18:24:52 +00:00
Colin Cross
22e6a6f69a Collect deps from r8
Use the new --deps-file argument to collect a deps file from r8, which
will correctly track files included by proguard flags files.

Fixes: 221956283
Test: manual
Change-Id: I92341948311c2c994cc0ae032dc19e09b2e80550
2022-03-22 16:40:13 +00:00
Colin Cross
9bb9bfb349 Don't pass static libs to r8
r8 gets the static libs in the program jar, it shouldn't also get
them as library jars.  Keep a separate classpath for dexing that
includes libs but not static_libs.

Bug: 222468116
Test: m checkbuild
Test: TestD8
Test: TestR8
Change-Id: Icca3393f496cbcadcc633f3b156761e6c145f975
2022-03-18 20:48:27 +00:00
Wei Li
92cd54b29d Correct the comments on property exclude_kotlinc_generated_file, which defaults to false actually.
Bug: 204888276
Test: Treehugger

Change-Id: Ib0405395da9a8cf8dd7bb1983fcd13425fe69b34
2022-01-12 13:27:36 -08:00
Wei Li
9ab9437b40 Merge "Add new property "exclude_files_in_output" for excluding files from the output files of Java related modules." 2022-01-12 09:25:22 +00:00
Wei Li
1e73c6573e Add new property "exclude_files_in_output" for excluding files from the output files of Java related modules.
Bug: 204888276
Test: m service-permission
Change-Id: I9f6113834826358b0e3af22ed1dd63a43c255452
2022-01-11 14:20:16 -08:00
Ian Zerny
fc7df6124f Add mapping id to source file information for R8 full-mode builds.
Bug: 204293621
Test: atest --host r8retrace-check-retraced-stacktrace
Change-Id: I09703cfaab72350efe67217be9e2d37988218f4d
2022-01-10 08:56:19 +01:00
Colin Cross
5ea963e60a Add support for d8 --main-dex-rules
d8 supports --main-dex-rules to replace the old mainDexClasses
script.

Test: m checkbuild
Change-Id: Ieddcf4de50cc13509c06bbffe9f1314ee3dd23b6
2021-09-16 19:14:30 -07:00
Colin Cross
a79a52c7c4 Reland: Pass jars with resources to R8
R8 will parse proguard files out of resources in injars.  Use the jar
with resources instead of the classes jar so that R8 can see the
proguard files.

R8 fails when an input jar contains dex files, so filter out dex files
from the input jar before passing it to r8.

This relands Ibb870ee9c70470336f542a3b7542dab86716dbf8 with a fix to
move the temporary input jar outside the output directory.  Putting it
in the output directory interacted poorly with RBE in local comparison
mode, which incorrectly deleted the output directory.

Bug: 195558228
Test: m checkbuild
Change-Id: Id4b091c3bd72d4c2904883e8793ec4b9499e4b78
2021-08-05 23:16:38 -07:00
Colin Cross
a832a04db2 Revert "Time out and dump stacks from R8 processes after 30 minutes"
This reverts commit 29c294b2cf.

The deadlock in R8 was fixed, and the timeout causes problems when
RBE is enabled and the R8 action gets stuck in a queue for 30
minutes.

Test: checkbuild
Change-Id: Ia0c3826aea3288e9db3bdbd4955e1e16c0a38fe3
2021-08-05 17:04:06 -07:00
Colin Cross
1d73c7316c Revert "Pass jars with resources to R8"
This reverts commit 05ca7c8769.

Reason for revert: b/195644067

Flaky failure with "missing in.jar" error.

Change-Id: Id2d6452de1e43951bc58f0bfb8e20d5ad65ce830
2021-08-05 15:16:11 +00:00
Colin Cross
05ca7c8769 Pass jars with resources to R8
R8 will parse proguard files out of resources in injars.  Use the jar
with resources instead of the classes jar so that R8 can see the
proguard files.

R8 fails when an input jar contains dex files, so filter out dex files
from the input jar before passing it to r8.

Bug: 195558228
Test: m checkbuild
Change-Id: Ibb870ee9c70470336f542a3b7542dab86716dbf8
2021-08-04 14:57:45 -07:00
Colin Cross
29c294b2cf Time out and dump stacks from R8 processes after 30 minutes
R8 processes are sometimes hanging on the build servers.  Wrap R8 with
run_with_timeout to dump the stacks with jstack and kill the process
after 30 minutes.  Switch from running with the r8-compat-proguard
shell script to running the jar directly so that jstack gets the pid
of the java process.

Bug: 181095653
Test: m checkbuild
Test: m NetworkStackNextIntegrationTests
Test: m USE_RBE=true RBE_R8=true RBE_R8_EXEC_STRATEGY=remote NetworkStackNextIntegrationTests
Change-Id: If6996bd8eb39c7a8453d79e825004339c009ade2
2021-04-30 14:14:46 -07:00
Jiyong Park
54105c48f4 SdkSpec = Scope + ApiLevel
SdkSpec.Version was an int type. Now it becomes ApiLevel type which
is a better abstraction of the version (or api level).

Bug: 1655587
Test: m

Change-Id: I4d67b9b9eae45f653b6af4f5b73da9e091b3dfab
2021-04-05 09:32:06 +09:00
Jiyong Park
f1691d2a2c Move java.sdkSpec to the android package
... in preparation for making the handling of sdk versions consistent
across java and cc modules.

Bug: 175678607
Test: m
Change-Id: I598f0454bce9b7320621022115412fbe97403945
2021-04-03 08:25:12 +09:00
Colin Cross
77cdcfdeaf Move android package on top of remotexec
Remove the references to the android package in remotexec so that
the android package can reference the remoteexec package.  This
will allow RuleBuilder to integrate directly with remoteexec.

Bug: 182612695
Test: m checkbuild
Change-Id: I15be5ef126d8aacbd605518638f341daf6f31bb3
2021-03-18 16:17:34 -07:00
Christoffer Quist Adamsen
e850737ee4 Allow missing classes in R8 builds
Bug: 180878971
Bug: 72683872
Test: TARGET_PRODUCT=aosp_arm64 m -j core-oj
Change-Id: I663220567eb8f034d3efccc60228aef91c716e8e
2021-02-22 12:09:01 +00:00
Colin Cross
dcf71b299c Convert java.Dependency to JavaInfo provider
Export information about java dependencies through a Provider
instead of accessing the module directly.

Test: java_test.go
Test: no changes to build.ninja
Change-Id: Ifc5d566bf6f6ebc0ad399e948effaa1ef6a22876
2021-02-09 15:36:25 -08:00
Paul Duffin
612e610063 Switch hiddenapi to use OutputPath instead of ModuleOutPath
In order to allow the hiddenapi code which expects its inputs to be
ModuleOutPath to consume the output from the deapexer module type (used
by prebuilt_apex) this change converts the hiddenapi from using
ModuleOutPath to use OutputPath instead.

This is part of a general cleanup to remove ModuleOutPath that is
separated out in order to avoid having the hiddenapi work dependent
upon and possibly delayed by issues with that general cleanup.

Bug: 178361284
Bug: 179124768
Test: m droid
Change-Id: I890f775bf5e33528dbaa62b95fa4198185ff8bf8
2021-02-02 14:42:05 +00:00
Colin Cross
e00c0e795d Fix proguard_usage.zip with USE_RBE=true RBE_R8=true
proguard_usage.zip contained an empty unused.txt file when
the unused.txt file wasn't specified as an output of the r8
rule.

Test: m USE_RBE=true RBE_R8=true out/soong/.intermediates/frameworks/base/packages/Shell/Shell/android_common/proguard_usage.zip
Change-Id: Ie6a7c8b2581fe21d643e675143d6cc0e2ce80456
2020-09-17 18:56:43 +00:00
Treehugger Robot
b900066dab Merge "Add support for building with R8 full mode" 2020-09-10 22:41:42 +00:00
Christoffer Quist Adamsen
f2d7b16dd6 Add support for building with R8 full mode
Change-Id: Ibb6fd64e8901cb4311b29b3bbb266ec8fd1835c9
Test: make external/libtextclassifier
2020-09-10 21:24:57 +02:00
Ramy Medhat
16f23a4038 Check UseRBE is set before replacing any template with the RE version.
Test: presubmit
Change-Id: I6df58b4e700f0d231367af2710672d731d20a736
2020-09-03 01:29:49 -04:00
Colin Cross
cb6143a142 Capture list of unused methods when shrinking in R8
Use the -printusage flag in R8 to output a list of the unused
methods.  Some of the files can be large (2MB for DocumentsUI,
87MB for all of AOSP), so immediately zip them and remove the
originals.  The zipped files will be merged and disted.

Bug: 151857441
Test: m TARGET_BUILD_APPS=DocumentsUI dist
Change-Id: I780e84e80eba7fe4d4fa15fec0f461890afd900b
2020-08-19 11:19:14 -07:00
Liz Kammer
a7a64f3c5b Refactor java compileDex
We want to support a compile_dex property for java_import. This splits
dex-related properties into a dexer struct which can be embedded in
relevant modules.

Test: m
Test: soong tests
Bug: 160455085
Change-Id: If56a51dac43f630d49483a36db29cd50e9ccd529
2020-08-04 10:15:04 -07:00
Kousik Kumar
366afc5011 Add support for remote-execution / caching of jar/zip actions
Test: Ran a build with `RBE_JAR="true" RBE_JAR_EXEC_STRATEGY="remote"
RBE_ZIP="true" RBE_ZIP_EXEC_STRATEGY="remote" ... use_rbe m` and that
succeeded.

Bug: b/156765207
Change-Id: Ia17b532fbb282be414adf879914870082dd0841b
2020-05-28 11:56:48 -07:00
David Srbecky
e033cbae40 Allow the user to explicitly set the java Uncompress_dex property.
ART tests need to explicitly control the compression.

Bug: 157239179
Test: "atest ArtGtestsTarget" with http://go/aog/1302773
Change-Id: I4490aa0a61fa71f830feb6a46dffd8d20fccfc66
Merged-In: I4490aa0a61fa71f830feb6a46dffd8d20fccfc66
2020-05-21 19:59:25 +01:00
Ramy Medhat
1dcc27e47c Refactor RBE support for Javac/R8/D8 to use the remoteexec package.
Test: build with and without RBE_JAVAC/RBE_R8/RBE_D8 set.
Change-Id: I1607f8cfb4d2c3cbb3672152bbeb561d9968bc60
2020-04-28 11:37:20 -04:00
Ramy Medhat
5e0dbe4e3e Merge "Add support for experimentally enabling RBE support on specific rules." 2020-01-28 19:49:05 +00:00
Ramy Medhat
8ea054a81e Add support for experimentally enabling RBE support on specific rules.
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.

Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
2020-01-28 12:42:47 -05:00
Jiyong Park
6a927c4e6a Abstract sdk_version string using sdkSpec type
The value format that sdk_version (and min_sdk_version, etc.) can have
has consistently evolved and is quite complicated. Furthermore, with the
Mainline module effort, we are expected to have more sdk_versions like
'module-app-current', 'module-lib-current', etc.

The goal of this change is to abstract the various sdk versions, which
are currently represented in string and is parsed in various places,
into a type called sdkSpec, so that adding new sdk veresions becomes
easier than before.

The sdk_version string is now parsed in only one place 'SdkSpecFrom', in
which it is converted into the sdkSpec struct. The struct type provides
several methods that again converts sdkSpec into context-specific
information such as the effective version number, etc.

Bug: 146757305
Bug: 147879031
Test: m
Change-Id: I252f3706544f00ea71c61c23460f07561dd28ab0
2020-01-25 21:56:43 +09:00
Søren Gjesse
8e505e3f01 Remove passing -forceprocessing flag to R8
R8 ignores the flag

Test: m -j
Change-Id: Ie394dfc24bb5632717d23e9ab4ad03b6cc1feb19
2019-11-27 12:40:07 +01:00
Colin Cross
c2557d1d56 Fix turbine classpath arguments
We were passing "--classpath foo.jar --classpath bar.jar" to turbine,
it now wants (and may always have expected?)
"--classpath foo.jar bar.jar".

Test: m checkbuild
Change-Id: Ib7e0ca64a34e236110f7b785d6a0fb51ed75567e
2019-11-07 02:05:06 +00:00
Sasha Smundak
2057f82161 Fix handling optimize.enabled from java_defaults
Some module types (`android_test`, etc.) set `optimize.enabled` by
default. If such module happens to have `defaults` attribute which
clears `optimize.enabled`, the latter value is ignored.

Fixes: 129858282
Test: unit tests in java_test.go,
`atest CtsExtendedMockingTestCases` succeeds with aog/936802
reverted (that is, with cts/test/mocking converted to Android.bp)

Change-Id: Ib8e3a0ab0bd489d70ed07f626082aeae31c45e7c
2019-04-17 17:59:09 -07:00
Colin Cross
6db4d4e613 Optimize CPU time when running d8 and r8
Pass -XX:+TieredCompilation -XX:TieredStopAtLevel=1 to the JVM to
reduce the time spent JITing, which in local experiments reduced
the CPU time of R8 while increasing the wall time, and reduced the
total number of runnable threads.

Without the new flags:
$ time out/soong/host/linux-x86/bin/r8-compat-proguard -injars out/target/common/obj/APPS/Dialer_intermediates/classes.jar ...
real	0m30.090s
user	4m35.337s
sys	0m12.853s

With the new flags:
$ time out/soong/host/linux-x86/bin/r8-compat-proguard -JXX:+TieredCompilation -JXX:TieredStopAtLevel=1 -injars out/target/common/obj/APPS/Dialer_intermediates/classes.jar ...
real	0m35.674s
user	2m45.135s
sys	0m9.531s

The increase in wall time should be offset by the better parallelism from
the reduced CPU time.

Bug: 119270658
Test: m checkbuild
Change-Id: I8fc78b5ef1f8c6f51fd97a29461b96ae2b35cada
2019-03-01 15:28:19 +00:00
Sasha Smundak
d3cf4ee87b soong_zip after dex should not fail when no *.dex files were generated
This can happen if Android application has only a manifest file and no
java code.

Fixes: 124483155
Test: treehugger with Android.mk converted to Android.bp in
platforms/frameworks/base/tests/privapp-permnissions

Change-Id: I45f95680b62a7a4bda0e7ab156f3df24b0dad70b
2019-02-15 10:14:23 -08:00
Nicolas Geoffray
65fd8ba2fe Always do zipalign with uncompressDex.
Also preserve the alignment with hiddenapi.

Test: conscrypt apex has its conscrypt.jar zipaligned.
Change-Id: I43cc0eadb9548c3d8055e12c077fb7bbe1935fe6
2019-01-23 06:36:48 +00:00
Colin Cross
afbb1734f6 Pass annotation processors to kotlinc
Enable the kotlin-annotation-processing plugin and pass annotation
processors to it.

Bug: 122251693
Test: m checkbuild
Test: TestKapt in kotlin_test.go
Change-Id: I841df454beaaa7edd263eea714ca0d958a03c9de
2019-01-22 21:47:19 +00:00
Jaewoong Jung
1d6eb68731 Use debug mode when running R8 for eng builds.
Bug: 119601874
Test: Built for different variants.
Change-Id: I2250587eb3e457ca15265eb9f8359872e9b08363
2018-11-30 15:23:16 +00:00
Colin Cross
5a0dcd5acf Store dex files uncompressed and unstripped in privileged APKs
Privileged APKs need to store their dex files uncompressed so they
can be verified and mapped directly out of the APK.

Also track whether the module will be dexpreopted or not in order
to determine if the dex file should be stripped before signing.

Test: SystemUI.apk contains an uncompressed dex file
Change-Id: I4dca86c7f8778595882405b34adcf2a7bae03c67
2018-10-25 17:47:12 +00:00