Commit graph

55 commits

Author SHA1 Message Date
Wei Li
92cd54b29d Correct the comments on property exclude_kotlinc_generated_file, which defaults to false actually.
Bug: 204888276
Test: Treehugger

Change-Id: Ib0405395da9a8cf8dd7bb1983fcd13425fe69b34
2022-01-12 13:27:36 -08:00
Wei Li
9ab9437b40 Merge "Add new property "exclude_files_in_output" for excluding files from the output files of Java related modules." 2022-01-12 09:25:22 +00:00
Wei Li
1e73c6573e Add new property "exclude_files_in_output" for excluding files from the output files of Java related modules.
Bug: 204888276
Test: m service-permission
Change-Id: I9f6113834826358b0e3af22ed1dd63a43c255452
2022-01-11 14:20:16 -08:00
Ian Zerny
fc7df6124f Add mapping id to source file information for R8 full-mode builds.
Bug: 204293621
Test: atest --host r8retrace-check-retraced-stacktrace
Change-Id: I09703cfaab72350efe67217be9e2d37988218f4d
2022-01-10 08:56:19 +01:00
Colin Cross
5ea963e60a Add support for d8 --main-dex-rules
d8 supports --main-dex-rules to replace the old mainDexClasses
script.

Test: m checkbuild
Change-Id: Ieddcf4de50cc13509c06bbffe9f1314ee3dd23b6
2021-09-16 19:14:30 -07:00
Colin Cross
a79a52c7c4 Reland: Pass jars with resources to R8
R8 will parse proguard files out of resources in injars.  Use the jar
with resources instead of the classes jar so that R8 can see the
proguard files.

R8 fails when an input jar contains dex files, so filter out dex files
from the input jar before passing it to r8.

This relands Ibb870ee9c70470336f542a3b7542dab86716dbf8 with a fix to
move the temporary input jar outside the output directory.  Putting it
in the output directory interacted poorly with RBE in local comparison
mode, which incorrectly deleted the output directory.

Bug: 195558228
Test: m checkbuild
Change-Id: Id4b091c3bd72d4c2904883e8793ec4b9499e4b78
2021-08-05 23:16:38 -07:00
Colin Cross
a832a04db2 Revert "Time out and dump stacks from R8 processes after 30 minutes"
This reverts commit 29c294b2cf.

The deadlock in R8 was fixed, and the timeout causes problems when
RBE is enabled and the R8 action gets stuck in a queue for 30
minutes.

Test: checkbuild
Change-Id: Ia0c3826aea3288e9db3bdbd4955e1e16c0a38fe3
2021-08-05 17:04:06 -07:00
Colin Cross
1d73c7316c Revert "Pass jars with resources to R8"
This reverts commit 05ca7c8769.

Reason for revert: b/195644067

Flaky failure with "missing in.jar" error.

Change-Id: Id2d6452de1e43951bc58f0bfb8e20d5ad65ce830
2021-08-05 15:16:11 +00:00
Colin Cross
05ca7c8769 Pass jars with resources to R8
R8 will parse proguard files out of resources in injars.  Use the jar
with resources instead of the classes jar so that R8 can see the
proguard files.

R8 fails when an input jar contains dex files, so filter out dex files
from the input jar before passing it to r8.

Bug: 195558228
Test: m checkbuild
Change-Id: Ibb870ee9c70470336f542a3b7542dab86716dbf8
2021-08-04 14:57:45 -07:00
Colin Cross
29c294b2cf Time out and dump stacks from R8 processes after 30 minutes
R8 processes are sometimes hanging on the build servers.  Wrap R8 with
run_with_timeout to dump the stacks with jstack and kill the process
after 30 minutes.  Switch from running with the r8-compat-proguard
shell script to running the jar directly so that jstack gets the pid
of the java process.

Bug: 181095653
Test: m checkbuild
Test: m NetworkStackNextIntegrationTests
Test: m USE_RBE=true RBE_R8=true RBE_R8_EXEC_STRATEGY=remote NetworkStackNextIntegrationTests
Change-Id: If6996bd8eb39c7a8453d79e825004339c009ade2
2021-04-30 14:14:46 -07:00
Jiyong Park
54105c48f4 SdkSpec = Scope + ApiLevel
SdkSpec.Version was an int type. Now it becomes ApiLevel type which
is a better abstraction of the version (or api level).

Bug: 1655587
Test: m

Change-Id: I4d67b9b9eae45f653b6af4f5b73da9e091b3dfab
2021-04-05 09:32:06 +09:00
Jiyong Park
f1691d2a2c Move java.sdkSpec to the android package
... in preparation for making the handling of sdk versions consistent
across java and cc modules.

Bug: 175678607
Test: m
Change-Id: I598f0454bce9b7320621022115412fbe97403945
2021-04-03 08:25:12 +09:00
Colin Cross
77cdcfdeaf Move android package on top of remotexec
Remove the references to the android package in remotexec so that
the android package can reference the remoteexec package.  This
will allow RuleBuilder to integrate directly with remoteexec.

Bug: 182612695
Test: m checkbuild
Change-Id: I15be5ef126d8aacbd605518638f341daf6f31bb3
2021-03-18 16:17:34 -07:00
Christoffer Quist Adamsen
e850737ee4 Allow missing classes in R8 builds
Bug: 180878971
Bug: 72683872
Test: TARGET_PRODUCT=aosp_arm64 m -j core-oj
Change-Id: I663220567eb8f034d3efccc60228aef91c716e8e
2021-02-22 12:09:01 +00:00
Colin Cross
dcf71b299c Convert java.Dependency to JavaInfo provider
Export information about java dependencies through a Provider
instead of accessing the module directly.

Test: java_test.go
Test: no changes to build.ninja
Change-Id: Ifc5d566bf6f6ebc0ad399e948effaa1ef6a22876
2021-02-09 15:36:25 -08:00
Paul Duffin
612e610063 Switch hiddenapi to use OutputPath instead of ModuleOutPath
In order to allow the hiddenapi code which expects its inputs to be
ModuleOutPath to consume the output from the deapexer module type (used
by prebuilt_apex) this change converts the hiddenapi from using
ModuleOutPath to use OutputPath instead.

This is part of a general cleanup to remove ModuleOutPath that is
separated out in order to avoid having the hiddenapi work dependent
upon and possibly delayed by issues with that general cleanup.

Bug: 178361284
Bug: 179124768
Test: m droid
Change-Id: I890f775bf5e33528dbaa62b95fa4198185ff8bf8
2021-02-02 14:42:05 +00:00
Colin Cross
e00c0e795d Fix proguard_usage.zip with USE_RBE=true RBE_R8=true
proguard_usage.zip contained an empty unused.txt file when
the unused.txt file wasn't specified as an output of the r8
rule.

Test: m USE_RBE=true RBE_R8=true out/soong/.intermediates/frameworks/base/packages/Shell/Shell/android_common/proguard_usage.zip
Change-Id: Ie6a7c8b2581fe21d643e675143d6cc0e2ce80456
2020-09-17 18:56:43 +00:00
Treehugger Robot
b900066dab Merge "Add support for building with R8 full mode" 2020-09-10 22:41:42 +00:00
Christoffer Quist Adamsen
f2d7b16dd6 Add support for building with R8 full mode
Change-Id: Ibb6fd64e8901cb4311b29b3bbb266ec8fd1835c9
Test: make external/libtextclassifier
2020-09-10 21:24:57 +02:00
Ramy Medhat
16f23a4038 Check UseRBE is set before replacing any template with the RE version.
Test: presubmit
Change-Id: I6df58b4e700f0d231367af2710672d731d20a736
2020-09-03 01:29:49 -04:00
Colin Cross
cb6143a142 Capture list of unused methods when shrinking in R8
Use the -printusage flag in R8 to output a list of the unused
methods.  Some of the files can be large (2MB for DocumentsUI,
87MB for all of AOSP), so immediately zip them and remove the
originals.  The zipped files will be merged and disted.

Bug: 151857441
Test: m TARGET_BUILD_APPS=DocumentsUI dist
Change-Id: I780e84e80eba7fe4d4fa15fec0f461890afd900b
2020-08-19 11:19:14 -07:00
Liz Kammer
a7a64f3c5b Refactor java compileDex
We want to support a compile_dex property for java_import. This splits
dex-related properties into a dexer struct which can be embedded in
relevant modules.

Test: m
Test: soong tests
Bug: 160455085
Change-Id: If56a51dac43f630d49483a36db29cd50e9ccd529
2020-08-04 10:15:04 -07:00
Kousik Kumar
366afc5011 Add support for remote-execution / caching of jar/zip actions
Test: Ran a build with `RBE_JAR="true" RBE_JAR_EXEC_STRATEGY="remote"
RBE_ZIP="true" RBE_ZIP_EXEC_STRATEGY="remote" ... use_rbe m` and that
succeeded.

Bug: b/156765207
Change-Id: Ia17b532fbb282be414adf879914870082dd0841b
2020-05-28 11:56:48 -07:00
David Srbecky
e033cbae40 Allow the user to explicitly set the java Uncompress_dex property.
ART tests need to explicitly control the compression.

Bug: 157239179
Test: "atest ArtGtestsTarget" with http://go/aog/1302773
Change-Id: I4490aa0a61fa71f830feb6a46dffd8d20fccfc66
Merged-In: I4490aa0a61fa71f830feb6a46dffd8d20fccfc66
2020-05-21 19:59:25 +01:00
Ramy Medhat
1dcc27e47c Refactor RBE support for Javac/R8/D8 to use the remoteexec package.
Test: build with and without RBE_JAVAC/RBE_R8/RBE_D8 set.
Change-Id: I1607f8cfb4d2c3cbb3672152bbeb561d9968bc60
2020-04-28 11:37:20 -04:00
Ramy Medhat
5e0dbe4e3e Merge "Add support for experimentally enabling RBE support on specific rules." 2020-01-28 19:49:05 +00:00
Ramy Medhat
8ea054a81e Add support for experimentally enabling RBE support on specific rules.
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.

Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
2020-01-28 12:42:47 -05:00
Jiyong Park
6a927c4e6a Abstract sdk_version string using sdkSpec type
The value format that sdk_version (and min_sdk_version, etc.) can have
has consistently evolved and is quite complicated. Furthermore, with the
Mainline module effort, we are expected to have more sdk_versions like
'module-app-current', 'module-lib-current', etc.

The goal of this change is to abstract the various sdk versions, which
are currently represented in string and is parsed in various places,
into a type called sdkSpec, so that adding new sdk veresions becomes
easier than before.

The sdk_version string is now parsed in only one place 'SdkSpecFrom', in
which it is converted into the sdkSpec struct. The struct type provides
several methods that again converts sdkSpec into context-specific
information such as the effective version number, etc.

Bug: 146757305
Bug: 147879031
Test: m
Change-Id: I252f3706544f00ea71c61c23460f07561dd28ab0
2020-01-25 21:56:43 +09:00
Søren Gjesse
8e505e3f01 Remove passing -forceprocessing flag to R8
R8 ignores the flag

Test: m -j
Change-Id: Ie394dfc24bb5632717d23e9ab4ad03b6cc1feb19
2019-11-27 12:40:07 +01:00
Colin Cross
c2557d1d56 Fix turbine classpath arguments
We were passing "--classpath foo.jar --classpath bar.jar" to turbine,
it now wants (and may always have expected?)
"--classpath foo.jar bar.jar".

Test: m checkbuild
Change-Id: Ib7e0ca64a34e236110f7b785d6a0fb51ed75567e
2019-11-07 02:05:06 +00:00
Sasha Smundak
2057f82161 Fix handling optimize.enabled from java_defaults
Some module types (`android_test`, etc.) set `optimize.enabled` by
default. If such module happens to have `defaults` attribute which
clears `optimize.enabled`, the latter value is ignored.

Fixes: 129858282
Test: unit tests in java_test.go,
`atest CtsExtendedMockingTestCases` succeeds with aog/936802
reverted (that is, with cts/test/mocking converted to Android.bp)

Change-Id: Ib8e3a0ab0bd489d70ed07f626082aeae31c45e7c
2019-04-17 17:59:09 -07:00
Colin Cross
6db4d4e613 Optimize CPU time when running d8 and r8
Pass -XX:+TieredCompilation -XX:TieredStopAtLevel=1 to the JVM to
reduce the time spent JITing, which in local experiments reduced
the CPU time of R8 while increasing the wall time, and reduced the
total number of runnable threads.

Without the new flags:
$ time out/soong/host/linux-x86/bin/r8-compat-proguard -injars out/target/common/obj/APPS/Dialer_intermediates/classes.jar ...
real	0m30.090s
user	4m35.337s
sys	0m12.853s

With the new flags:
$ time out/soong/host/linux-x86/bin/r8-compat-proguard -JXX:+TieredCompilation -JXX:TieredStopAtLevel=1 -injars out/target/common/obj/APPS/Dialer_intermediates/classes.jar ...
real	0m35.674s
user	2m45.135s
sys	0m9.531s

The increase in wall time should be offset by the better parallelism from
the reduced CPU time.

Bug: 119270658
Test: m checkbuild
Change-Id: I8fc78b5ef1f8c6f51fd97a29461b96ae2b35cada
2019-03-01 15:28:19 +00:00
Sasha Smundak
d3cf4ee87b soong_zip after dex should not fail when no *.dex files were generated
This can happen if Android application has only a manifest file and no
java code.

Fixes: 124483155
Test: treehugger with Android.mk converted to Android.bp in
platforms/frameworks/base/tests/privapp-permnissions

Change-Id: I45f95680b62a7a4bda0e7ab156f3df24b0dad70b
2019-02-15 10:14:23 -08:00
Nicolas Geoffray
65fd8ba2fe Always do zipalign with uncompressDex.
Also preserve the alignment with hiddenapi.

Test: conscrypt apex has its conscrypt.jar zipaligned.
Change-Id: I43cc0eadb9548c3d8055e12c077fb7bbe1935fe6
2019-01-23 06:36:48 +00:00
Colin Cross
afbb1734f6 Pass annotation processors to kotlinc
Enable the kotlin-annotation-processing plugin and pass annotation
processors to it.

Bug: 122251693
Test: m checkbuild
Test: TestKapt in kotlin_test.go
Change-Id: I841df454beaaa7edd263eea714ca0d958a03c9de
2019-01-22 21:47:19 +00:00
Jaewoong Jung
1d6eb68731 Use debug mode when running R8 for eng builds.
Bug: 119601874
Test: Built for different variants.
Change-Id: I2250587eb3e457ca15265eb9f8359872e9b08363
2018-11-30 15:23:16 +00:00
Colin Cross
5a0dcd5acf Store dex files uncompressed and unstripped in privileged APKs
Privileged APKs need to store their dex files uncompressed so they
can be verified and mapped directly out of the APK.

Also track whether the module will be dexpreopted or not in order
to determine if the dex file should be stripped before signing.

Test: SystemUI.apk contains an uncompressed dex file
Change-Id: I4dca86c7f8778595882405b34adcf2a7bae03c67
2018-10-25 17:47:12 +00:00
Colin Cross
4b964c00a6 Fix instrumentation_for to match LOCAL_INSTRUMENTATION_FOR
The value from instrumentation_for should not go to
aapt2 link --rename-instrumentation-target-package, that should
be the equivalent of LOCAL_MANIFEST_INSTRUMENTATION_FOR.  That
property is never used in Make, so it is left unimplemented in
Soong.

Add the module listed in instrumentation_for as a shared library
for javac, but don't import its resources.

Bug: 117804211
Test: m checkbuild
Change-Id: I7f035dc0ecb964a3ca391ae1ca2b87cb0f6a7cec
2018-10-18 23:54:38 +00:00
Colin Cross
791399f1fd Use file glob for zipping classes*.dex files
Using a directory to zip the output of d8 and r8 was causing the
empty classes.dex.jar file in the output directory to get
zipped in.  Use a file glob instead so that it only picks up the
classes*.dex files even if classes.dex.jar already exists.

Test: m checkbuild
Change-Id: I1f3c1cbc23439fc52756aad027ab8cac10213d1d
2018-09-28 16:14:58 -07:00
Colin Cross
6dab9bd57a Add dependencies on d8 and r8 library arguments
d8 and r8 were missing dependencies on jars passed as library
arguments.  This normally worked because the input jar is likely
to also depend on the same jars.  When the classpath was added
to d8 it started to fail on libraries that have no code but
include other static libraries.

Bug: 116840360
Test: rm out/soong/.intermediates/external/nist-sip/nist-sip/android_common/turbine-combined/nist-sip.jar && m out/soong/.intermediates/frameworks/base/ext/android_common/dex/ext.jar
Change-Id: Ie642f4a343b949778d8ee6dcc4511f3fd7633be1
2018-09-28 15:15:08 +00:00
Colin Cross
ffb657ebc0 Pass classpath to d8
d8 requires the full classpath if --min-api requires default method
or try-with-resources desugaring.  Always pass the full classpath,
since it will already have been built to generate the input jar.

Bug: 116243313
Test: m checkbuild
Change-Id: Ib0c18873880697b7cf942e0548dd2c53ee5ad23c
2018-09-27 11:05:09 -07:00
Søren Gjesse
24f1702872 Don't include data resources when running R8
The change https://r8-review.googlesource.com/c/r8/+/26482 made R8
process data resources by default. This is not expected for the
Android Platform build

Test: m -j PRODUCT-aosp_x86-userdebug core-oj
Change-Id: Idb17e39dae524dacf4b258c9f27be86e3179667f
2018-09-14 15:20:42 +02:00
Søren Gjesse
0e84935ac4 Ensure that the R8 minification map (ProGuard dictionary) file exists
R8 no longer outputs an empty map file when minification
(obfuscation) is turned off.

Test: m -j PRODUCT-aosp_x86-eng
Change-Id: I317e703597368dc8c16751bcc8d0e43f985614fa
2018-08-27 12:56:16 +02:00
Colin Cross
331a1213b0 Don't pass resources to r8
R8 complains when it gets dex files in the input jar, but some tests
use dex files or dex jars as resources.  Keep resources separate
from classes until after r8 has been run.

Test: java_test.go
Test: m checkbuild
Change-Id: I1d9164d60d6b054ebb138648da07d80ee769177f
2018-08-16 17:03:56 -07:00
Colin Cross
3063b78ea5 Make :module provide the output file for java modules
Make :module on a java_library provide the output file, which is
normally the implementation jar.  For java_library modules with
installable: true or compile_dex: true this will be the dexjar
instead.  For android_app modules this will be the apk.

Bug: 80144045
Test: no change to out/soong/build.ninja
Change-Id: I739674aee60a38bfccb859369e4414b46f293d82
2018-08-16 16:13:58 -07:00
Colin Cross
bd1cef5618 Add proguard_flags_files to r8 command line
We were not passing the proguard_flags_files files as -include
arguments to R8.

Bug: 112462307
Test: m checkbuild
Change-Id: I8f3b10fc338aa4b15c0fb220ee2891efbb0b020f
2018-08-13 17:29:58 +00:00
Colin Cross
4c03f68763 Make merge_zips -stripFile use blueprint style globs
merge_zips -stripFile was only considering the name of the file and
ignoring the path.  Make it more useful by supporting blueprint style
globs.  The previous behavior can be recreated by prefixing with **/.

Bug: 111389216
Test: m checkbuild
Change-Id: I25760fe3f1f77704dd9da9d107d9a38a415d681f
2018-07-16 15:24:43 -07:00
Colin Cross
83bb3167f6 Add support for min_sdk_version
Add min_sdk_version properties and use it for aapt2
--min-sdk-version and --target-sdk-version flags.  Add an sdkContext
interface that any function that needs an sdk version can take in
order to get the values for the current module.

Bug: 110848854
Test: m checkbuild
Change-Id: Ic69f1f935d8b865ec77689350407df08bfac5925
2018-07-09 12:42:01 -07:00
Colin Cross
bafb897de7 Revert "Revert "Remove desugar support""
This reverts commit 7576916efc.

Reason for revert: includes art veridex fix this time

Change-Id: Ia8da00b5ec8ed3dc3ce2a2521f0a7b2b2b131960
2018-06-06 21:46:32 +00:00
Colin Cross
7576916efc Revert "Remove desugar support"
This reverts commit 000280bb0e.

Reason for revert: broke art veridex build

Change-Id: I9f7d7cdbe8e34671614687ac0b34c0338ef240e7
2018-06-06 21:42:39 +00:00