Commit graph

1083 commits

Author SHA1 Message Date
Tobias Thierer
f6bd495728 Break EXPERIMENTAL_USE_OPENJDK9 into {USE,TARGET}_OPENJDK9.
This CL topic breaks the single flag value EXPERIMENTAL_USE_OPENJDK9
into two flags USE_OPENJDK9 and TARGET_OPENJDK9 which are computed
independently in config.mk and config.go; this means that later
build logic no longer depends on / duplicates the semantics of an
unset EXPERIMENTAL_USE_OPENJDK9, because that later build logic
can use USE_OPENJDK9 and TARGET_OPENJDK9 instead. Thus, it becomes
slightly easier to change this default in future CLs (touching
only config.mk and config.go).

It also makes it more straightforward to convert .mk files
to .bp.

This CL also introduces a new allowed value "false" for
EXPERIMENTAL_USE_OPENJDK9. This fourth possible value currently
has the same semantics as a default/unset
EXPERIMENTAL_USE_OPENJDK9, but allows people to explicitly
switch back to the old semantics when the default changes.

Test: make showcommands core-oj (in environments with
      EXPERIMENTAL_USE_OPENJDK9 set to "", "1.8", and "true").
Bug: 38177295

Change-Id: Iaade9610b237722e5d175143808b8653b1e98996
2017-11-17 13:14:11 +00:00
Alan Leung
7d6144d34b Rename r8 to r8-compat-proguard
Test: USE_R8=false USE_D8=true m -j32 && USE_R8=true USE_D8=true m -j32

Change-Id: I2610340cf75f8db5920e33fac542f0dcf294ce29
2017-11-16 17:24:01 -08:00
Søren Gjesse
c66d297e0a Use prebuild R8 for USE_R8=true
Instead of building R8 from external/r8 use the prebuild jar in
prebuilds/r8.

Test: m -j USE_R8=true

Change-Id: I9680ae430cdbc2b3beb910873c8eed27354f6af1
2017-11-07 16:21:50 +01:00
Treehugger Robot
85e391b2a2 Merge changes I30a83721,I1094f213
* changes:
  Validate args passed by jacoco.mk to rm -rf
  Implement validate-paths-are-subdirs
2017-11-06 22:10:41 +00:00
Tobias Thierer
96257655c5 Merge "Skip Proguard for EXPERIMENTAL_USE_OPENJDK9=true" 2017-11-02 12:35:26 +00:00
Treehugger Robot
fba0a560de Merge "Revert "Revert "CFI compatibility with static executables and nested archives""" 2017-11-01 22:16:00 +00:00
Nicolas Geoffray
21a3aef7d7 Merge "Add a product variable for priv-apps dependencies." 2017-11-01 13:40:20 +00:00
Vishwath Mohan
85f72449ae Revert "Revert "CFI compatibility with static executables and nested archives""
This reverts commit 8350c4c540.

Reverting the revert so a proper fix can be applied.

Change-Id: I69f106dfd294198e03a62bcd88c8f18033410141
2017-11-01 09:21:20 +00:00
Tobias Thierer
89740664a5 Skip Proguard for EXPERIMENTAL_USE_OPENJDK9=true
Proguard doesn't support v53 class files. For now, this
CL simply replaces the Proguard step with a plain 'cp',
which is essentially a no-op.

To avoid apps exceeding the dex limit in this mode, this
CL also enables --multi-dex for those builds.

Test: Treehugger
Bug: 67673860
Change-Id: I17dc5d827360d606a0cac12e324f97e8dedbb630
2017-10-31 21:02:14 +00:00
Orion Hodson
8350c4c540 Revert "CFI compatibility with static executables and nested archives"
This reverts commit 3d3e1cf260.

Rationale: part of a group of commits that left aosp_x86_64 not
building. (See https://android-build.googleplex.com/builds/
submitted/4426589/aosp_x86_64-eng/latest/logs/build_error.log)

Bug: 30227045
Test: builds
Change-Id: Ie22590abe3d1cdccb8d141baf6480d49dedf8789
2017-10-31 17:41:16 +00:00
Vishwath Mohan
22b6abc96d Merge "CFI compatibility with static executables and nested archives" 2017-10-31 01:42:16 +00:00
Colin Cross
c7bd540a6e Export AAPT-related variables to soong
Test: m checkbuild
Change-Id: I1816df9b9cc7b51e228d6d847cefa2d51a524897
2017-10-30 17:31:44 -07:00
Vishwath Mohan
3d3e1cf260 CFI compatibility with static executables and nested archives
This CL makes the following changes:
(a) It disables diagnostics for CFI which requires the runtime ubsan
library (which isn't included in static executables).

(b) It applies the ar flags for CFI correctly for nested .a
archives.

(c) Applies the version script to export CFI shadow for non-static
binaries

(d) Doesn't apply cross-dso CFI for static executables

Bug: 30227045
Test: Static executables build correctly and do not complain about
missing symbols from the ubsan runtime library.
Test: Nested .a files correctly use the gold plugin.

Change-Id: Id8fe3c13f6b76565aafbf1266e95f50d1447a790
2017-10-27 03:26:27 -07:00
Jeff Gaston
3964db36da Implement validate-paths-are-subdirs
Test: m -j nothing # which runs unit tests
Bug: 68056327
Change-Id: I1094f2134af28695ea2376134ea463d34cd63676
2017-10-26 14:10:19 -07:00
Colin Cross
5ea99d420b Desugar default methods if required
Desugar default methods if requested by LOCAL_MIN_SDK_VERSION < 26.
Also remove --desugar_try_with_resources_if_needed=false, it is no
longer necessary if we always proguard against frameworks.jar.
Fixes d8 issues when it sees unexpected default methods that dx
was passing through.

This reapplies I81796b452d5a70a951e175727953d4295ca32c0c after
If30943efcaea44b2db1b38a6c1d558113324757f.

Test: m checkbuild
Change-Id: I3bafbb2d63de2a6b50bc8c7ff40259b54d24f128
2017-10-26 13:09:07 -07:00
Colin Cross
98caed6e7f Fix desugaring when LOCAL_SDK_VERSION is not set
When LOCAL_SDK_VERSION is not set, frameworks.jar is put
in the classpath for javac.  If it is also in the classpath
(as opposed to the bootclasspath) for desugar then desugar
tries to rewrite classes that implement interfaces with
default methods to contain calls to the generated companion
classes for those interfaces.  frameworks.jar will not
contain the companion classes, which causes proguard to
fail.

Move frameworks.jar to the bootclasspath, which matches more
closely with builds against the SDK stubs jars.

Also remove the static libraries from the classpath, they have
already been merged into the input jar.

Test: m checkbuild
Change-Id: If30943efcaea44b2db1b38a6c1d558113324757f
2017-10-26 13:09:07 -07:00
Colin Cross
4d85d55e09 Merge "Revert "Desugar default methods if required"" 2017-10-25 22:33:08 +00:00
Colin Cross
53aa02d2d2 Revert "Desugar default methods if required"
This reverts commit 6548d2a98d.

Reason for revert: broke apps build

Change-Id: Ibc503fa89b51e1f64f78cb6cee3e3ad7bbd8f2eb
2017-10-25 22:32:14 +00:00
Colin Cross
437bbed858 Merge changes I81796b45,Iaf585336
* changes:
  Desugar default methods if required
  Always raise proguard libraries
2017-10-25 21:44:18 +00:00
Colin Cross
6548d2a98d Desugar default methods if required
Desugar default methods if requested by LOCAL_MIN_SDK_VERSION < 26.
Also remove --desugar_try_with_resources_if_needed=false, it is no
longer necessary if we always proguard against frameworks.jar.
Fixes d8 issues when it sees unexpected default methods that dx
was passing through.

Test: m checkbuild
Change-Id: I81796b452d5a70a951e175727953d4295ca32c0c
2017-10-24 13:29:36 -07:00
Søren Gjesse
9047dce212 Merge "Added support for using R8 instead of Proguard + dx" 2017-10-24 06:39:09 +00:00
Tao Bao
d690ca5b8a Merge "Fix an issue in assert-max-image-size." 2017-10-23 22:43:23 +00:00
Søren Gjesse
0c1fe64ce8 Added support for using R8 instead of Proguard + dx
With this change building with USE_R8=true will run R8 instead of Proguard + dx
for all targets that have LOCAL_PROGUARD_ENABLED set. R8 (through the
compatproguard wrapper) will use the same options as Proguard for now.

Additional Proguard configurations for R8 can be added using LOCAL_R8_FLAG_FILES
on individual targets. This makes it possible to make configuration updates
without touching the Proguard + dx configuration

Test: m -j
Test: m -j USE_D8=true USE_R8=true
Test: m -j USE_D8=true USE_R8=false
Test: m -j USE_D8=true
Change-Id: I78e8a0fd17cc5cefedccffe95b93c94a60a727f1
2017-10-23 08:58:30 +02:00
Treehugger Robot
1cba05bb39 Merge "Slightly easier debugging of missing inputs to copy-file-to-target" 2017-10-19 22:52:20 +00:00
Colin Cross
58cd98dc47 Merge "Use system modules generated by soong when using javac -target 1.9" 2017-10-17 18:37:35 +00:00
Tao Bao
6730f9017e Fix an issue in assert-max-image-size.
When building the recovery image on marlin (m bootimage):
----- Making recovery image ------
Copying baseline ramdisk...
Modifying ramdisk contents...
/bin/bash: line 0: [: 0x02000000: integer expression expected
----- Made recovery image: out/target/product/marlin/boot.img --------

Because if -gt doesn't like hexadecimal numbers. Fix the issue by doing
a dummy arithmetic to get integer $$maxsize.

Test: `m bootimage` with aosp_marlin-userdebug. Also change the
      BOARD_BOOTIMAGE_PARTITION_SIZE closer to the actual size to
      trigger the different size check paths.
Change-Id: I75c7727664d7255b9c18f57ae38076ff90b8a957
2017-10-17 10:08:56 -07:00
Treehugger Robot
51e7a0cca4 Merge "Make SOONG_JAVAC_WRAPPER use JAVAC_WRAPPER to call javac." 2017-10-17 05:08:10 +00:00
Colin Cross
f960257c14 Use system modules generated by soong when using javac -target 1.9
Soong has support for building system modules.  Use the directories
produced by Soong with --system to replace -bootclasspath arguments
when using javac -target 1.9.

Since soong can't generate current SDK stubs yet, and no existing
SDK stubs need -target, only use -target 1.9 for modules that are
not compiling against the SDK.  That means in practice the only
system modules that will be used for now is the default one,
core-system-modules.

Bug: 63986449
Test: m -j EXPERIMENTAL_USE_OPENJDK9=true makes some progress
Change-Id: I350ef50aedf36fdd72458c23d4fe8a2edf1a9a02
2017-10-16 17:55:24 -07:00
Nan Zhang
ad818dcbea Support java compilation sharding for target side.
Add "LOCAL_JAVAC_SHARD_SIZE" to represent the number of java
source path entries in each shard.

Sharding is not allowed when "LOCAL_JAVAC_SHARD_SIZE" and
"LOCAL_JAR_PROCESSORS" are both enabled.

Limitation:
1. 0 <= LOCAL_JAVAC_SHARD_SIZE <= 8192
1. 0 < NUM(java_sources) <= 8192
2. 0 <= NUM(shards) <= 100

Performance Compare:
<Unsharded Build------------------------------------------------>
1. Build from clean state
rm -r -f out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/
&& time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

real	1m2.720s user	5m26.604s sys	0m39.552s

2. Incremental build
m out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar &&
touch frameworks/base/core/java/android/net/http/HttpResponseCache.java && time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

real	0m37.586s user	5m47.804s sys	0m50.388s

<Sharded Build-------------------------------------------------->
1. Build from clean state
rm -r -f out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/
&& time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

Javac Shard Size: 50----real	1m10.163s user	25m59.008s sys	1m58.460s
Javac Shard Size: 100---real	1m2.115s user	21m3.600s sys	1m15.964s
Javac Shard Size: 150---real	0m59.520s user	18m10.544s sys	1m12.628s
Javac Shard Size: 200---real	0m56.894s user	15m39.244s sys	1m11.608s
Javac Shard Size: 250---real	0m55.991s user	14m38.716s sys	1m2.292s
Javac Shard Size: 300---real	0m55.114s user	13m6.568s sys	1m8.200s
Javac Shard Size: 350---real	0m53.144s user	12m7.740s sys	1m3.836s
Javac Shard Size: 400---real	0m54.929s user	12m9.324s sys	1m4.340s
Javac Shard Size: 450---real	1m30.194s user	25m31.468s sys	1m52.416s
Javac Shard Size: 500---real	0m53.976s user	10m35.500s sys	0m55.160s

2. Incremental build
m out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar &&
touch frameworks/base/core/java/android/net/http/HttpResponseCache.java && time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

Javac Shard Size: 50-----real	0m16.322s user	1m8.648s sys	0m31.700s
Javac Shard Size: 100----real	0m16.163s user	1m22.932s sys	0m29.440s
Javac Shard Size: 150----real	0m16.611s user	1m37.828s sys	0m21.168s
Javac Shard Size: 200----real	0m16.936s user	1m49.248s sys	0m28.636s
Javac Shard Size: 250----real	0m17.509s user	1m54.944s sys	0m32.768s
Javac Shard Size: 300----real	0m18.868s user	1m54.088s sys	0m28.824s
Javac Shard Size: 350----real	0m17.629s user	1m54.108s sys	0m31.056s
Javac Shard Size: 400----real	0m18.658s user	2m7.712s sys	0m30.636s
Javac Shard Size: 450----real	0m18.874s user	2m8.808s sys	0m33.540s
Javac Shard Size: 500----real	0m19.432s user	2m24.400s sys	0m30.368s

time m nothing:
real 0m5.799s user 0m7.236s sys 0m3.068s

Test: m clean && m -j checkbuild
Bug: b/67424047
Change-Id: Id0766d2b7de7c4546d29bbc7f8a0dd0e4b9ad45b
2017-10-12 13:23:06 -07:00
Yoshisato Yanagisawa
42b808b18e Make SOONG_JAVAC_WRAPPER use JAVAC_WRAPPER to call javac.
JAVAC_WRAPPER seems not be used when SOONG_JAVAC_WRAPPER is used.
Since JAVAC_WRAPPER just wraps actual javac call, the result must
be the same.

Bug: 67723445
Change-Id: I944f9534587c00ffd1bc79806c534432467ba4a5
2017-10-12 16:04:20 +09:00
Dan Willemsen
2d081b35d4 Fix broken foreach loop
This wasn't initializing the STATS.MODULE_TYPE.* variables, leading to
106kB of stack usage in ckati for sdk_x86_64-eng. With this fixed, the
next highest stack usage is 35kB.

Bug: 36182021
Test: Run $(flavor) over all the variables
Test: Check stack usage with custom ckati patches
Test: build-sdk_x86_64.ninja is identical before/after this change
Change-Id: Id1d8ffb8415ad6613d0093d31be5472d42d300fd
2017-10-10 03:36:42 +00:00
Colin Cross
ea1629e788 Refactor removing classes.dex for dexpreopt
Combining $(call dexpreopt-remove-classes.dex) into a
$(call copy-one-file) rule is hacky and error prone.  Make a helper
dexpreopt-copy-jar function that takes LOCAL_DEX_PREOPT and copies
the jar, removing the dex files if necessary.

Test: only description message changes to out/build-${TARGET_PRODUCT}.ninja
Change-Id: Id2fa2d18571ca01bc81302c912b8de25f3dde7a1
2017-10-06 14:26:25 -07:00
Nicolas Geoffray
a95fbd1ed5 Add a product variable for priv-apps dependencies.
Those dependencies are jars/apks on which we want to
uncompress their dexs.

bug: 30972906
bug: 63920015

Test: sailfish build

Change-Id: Ic96ffe9dbe39abc1c28e7de134892d689207c9ca
2017-10-06 11:14:08 +01:00
Jiyong Park
03eb06150b Don't reserve size for bad blocks when checking image size
When checking *.img size with the partition size, the build system
reserves additional bits for spare bits and spare bad blocks.

However, for emmc and ufs, the space bits and blocks are entirely
managed by the underlying controller and thus not visible from outside
of the controller. In fact the check routine was made for legacy MTD
storages where raw flash blocks are directly exposed.

This makes the size checking a little bit conservative in modern devices.
Builds were failed even though the *.img can actually fit into the
partition. To handle this problem, the additional size is no longer
reserved when checking *.img size with the partition size.

This change also removes following build flags that are meaningful
only for devices having MTD storages:
BOARD_NAND_PAGE_SIZE
BOARD_NAND_SPARE_SIZE
Further use of them breaks the build

Bug: 35790399
Bug: 66399382
Test: build

Merged-In: I954bf261441b53844e75d05788866f1692a2ad43
Change-Id: I954bf261441b53844e75d05788866f1692a2ad43
2017-10-04 21:25:08 +09:00
Treehugger Robot
b039cf792f Merge "Delete jack support" 2017-10-03 01:17:25 +00:00
Dan Willemsen
f439c334da Fix incremental aapt builds
If a resource was moved / removed without any other timestamp updates,
ninja would not re-run aapt. To fix this, add a list of resource files
into the same command as aapt (echo ... >/dev/null). That way if the
list changes, ninja will recognize the command line changing and rerun
aapt.

Bug: 67315436
Test: mmma cts/tests/autofillservice
Test: mv cts/tests/autofillservice/res/layout/welcome_activity.xml \
      cts/tests/autofillservice/res/layout/welcome_activity_test.xml
Test: mmma cts/tests/autofillservice

Change-Id: Ia93ac4ff48910fed74a1f1539ce4e1f5a9db023f
2017-10-02 14:22:02 -07:00
Colin Cross
e673deb280 Delete jack support
Remove all support for running jack.

Bug: 65302138
Test: m -j checkbuild
Change-Id: I1ef87c88af3f2689f8b0dcf1a01c958b16159631
Merged-In: I1ef87c88af3f2689f8b0dcf1a01c958b16159631
(cherry picked from commit 5db5d31d73)
2017-10-02 11:15:19 -07:00
Dan Willemsen
ba9b0fbc05 Fix multiple output file generation for AAPT2
Make R.stamp actually part of the same rule that creates R.java.
Otherwise if the generated sources directory is removed, we don't
regenerate R.java / Manifest.java.

Also move the package-export.apk and proguard options file into the same
rule.

Test: m telephony-common; rm out/target/common/obj/*/*_intermediates/src; m telephony-common
Change-Id: I71f6fcaf20593ca23271d52d15537aaa9ef4cc77
2017-09-29 20:35:32 -07:00
Colin Cross
9deb331f20 Respect LOCAL_MIN_SDK_VERSION for dx
Follow Jack's behavior for now and pass LOCAL_MIN_SDK_VERSION to
dx.  Don't pass it to desugar for now, desugar with low
--min_sdk_version numbers enables features we haven't verified
yet, and existing apps won't be using the new language features
that require extra desugaring.

Test: examine classes.dex from an app with LOCAL_MIN_SDK_VERSION
Change-Id: Ic7d679b1b11fba98a33418a4f79d4762a91f1f9c
2017-09-28 21:40:35 -07:00
Colin Cross
7dc9043546 Allow disabling turbine
m -j TURBINE_ENABLED=false will build without turbine.  This is
primarily useful for A/B comparisions of turbine builds, but will
also be used to temporarily disable turbine for unbundled builds
until we have turbine prebuilts.

Bug: 64308460
Test: m -j TURBINE_ENABLED=false java
Test: m -j java
Change-Id: Ie48746c8bfc60c361be9634cb1805ca1c09aa1fe
2017-09-26 16:23:00 -07:00
Colin Cross
9b2e4c6950 Simplify the *-lib-* functions
Simplify the java-lib-* type functions, three layers of abstraction
is a bit much for a single line function, and isn't reducing enough
duplication to be worth the complexity.

Add app-lib-* functions and use them.

Test: m -j checkbuild
Change-Id: Ia47ebc95e4af2073396233416f14fdee7d856844
2017-09-26 16:23:00 -07:00
Colin Cross
b541aae17c Remove duplication of bootclasspath in LOCAL_JAVA_LIBRARIES
The bootclasspath libraries have direct dependencies now, so
remove them from LOCAL_JAVA_LIBRARIES so they don't also end
up in the classpath.

Test: m -j checkbuild
Test: m -j ANDROID_COMPILE_WITH_JACK=true java
Test: manually inspection of build-${TARGET_PRODUCT}.ninja
      only has expected changes
Test: no changes to system.img
Change-Id: Icbc8987468f237aa351708b33c3abe5ea3d1816e
2017-09-25 16:15:10 -07:00
Colin Cross
1d1e5ef61b Add dependencies on bootclasspath libraries
The libraries in the bootclasspath were getting their dependencies
because they were also in the default libraries list.  Make the
dependencies explicit in preparation for rearranging the default
libraries.

Test: m -j checkbuild
Change-Id: I3ea004714a31cd648ec2ef57ed3099bce70bc6b9
2017-09-25 16:14:42 -07:00
Miao Wang
28e78f4600 Correctly set the linker search path for libm and libc.
- ndk/r10 is removed, we need to use SOONG ndk stubs instead.
  - Use MIN_SUPPORTED_SDK_VERSION instead of hard coded version 9 for
    32 bit compilation.

Bug: 38342163
Test: mm
Test: make -j40 FORCE_BUILD_RS_COMPAT=true RSTest_Compat
Change-Id: I9549446cb7f8faee8842b06f096d035535dc7f3d
2017-09-22 14:57:02 -07:00
Nan Zhang
fda45fdbe1 Re-org the merge_zips logiic under Turbine rule.
Mac java build has flaky error for merge_zips. The error happend is
because both turbine.jar.premerged and static_java_libs are missing.

We allow one of them can be missing, but we don't allow both of them are
missing. So re-org the merge_zips logic to see if the input of Turbine is
empty or it is due to Turbine internal.

Bug: b/66441248
Test: m clean && m -j32
Change-Id: I3c4b15004fd67fef2b83eabc36841bc8c8ff6e4c
2017-09-21 16:51:18 -07:00
Nan Zhang
b88011768d Move private_manifest file insertion to jars merging phase.
By doing this, the change of private_manifest file won't trigger
Javac to re-run during re-build.

Bug: b/65455145
Test: m clean && m -j32
Change-Id: Ib90d56daf0be5f31b51e07b7714a933bc5cf4496
2017-09-20 14:56:48 -07:00
Nan Zhang
39bbfe217a Merge "Use merge_zips to merge jars during Java build in Make." 2017-09-20 19:48:27 +00:00
Nan Zhang
9bd540594d Use merge_zips to merge jars during Java build in Make.
1. Use merge_zips to merge jars without unzipping/rezipping static Java
libraries. And seperate merging process as a independent Make target so
that Javac can only depend on Turbine.

2. Remove duplicate vars: "full_java_lib_deps" in droiddoc.mk

3. Also use classes-header.jar for link_apk & link_instr

Bug: b/65455145
Test: m clean && m -j32

Change-Id: I7b0307664a6b86075c983284fce099bd3946318c
2017-09-19 18:49:33 -07:00
Nicolas Geoffray
b06c30b480 Revert "Revert "Make privileged apps always keep uncompressed dex files.""
bug: 30972906
bug: 63920015

Test: doing zipinfo on the produced apks after build.
Test: fugu build

This reverts commit f69441b76b.

Change-Id: I6b029f2044385f3e3fa332091a9244cec79d2474
2017-09-19 13:09:27 +01:00
Nicolas Geoffray
f69441b76b Revert "Make privileged apps always keep uncompressed dex files."
Broke Fugu build. Will work on a workaround for it.

bug: 30972906
bug: 63920015

This reverts commit 09dfe6739a.

Change-Id: I5a8b8de4ec98ae44808762ca1ba974ce89bebb05
2017-09-18 15:07:26 +00:00