Commit graph

71 commits

Author SHA1 Message Date
Colin Cross
3f99cd5933 Replace extract_srcjars.sh with zipsync
Use zipsync instead of extract_srcjars.sh.

Also fixes an issue where unzipping a resource zip would
ignore anything that wasn't a *.java file.

Bug: 73885582
Test: m checkbuild
Change-Id: I1c25f65afa196614879071c8655866febebac185
2018-03-15 16:40:05 -07:00
Colin Cross
11e2d55c0f Add LOCAL_SRCJARS support
Allow java modules to take jars containing .java files as an input.
For turbine, pass them directly to turbine.  For javac, extract them
to a directory and pass a list of the files to javac.

Bug: 73885582
Test: m checkbuild
Change-Id: Ib4351a5b4b165a55e5fb02ee2680466db368983a
2018-03-14 12:05:04 -07:00
Andreas Gampe
21a1c6d03f Build: Move LOCAL_ERROR_PRONE_FLAGS
Move application of module flags to host_java_library_common.mk
so that dalvik host builds can also have flags.

Bug: 72451126
Test: m javac-check RUN_ERROR_PRONE=true
Change-Id: Idbf43afd886536cc7ea674aa4487da3c22987850
2018-02-15 15:02:15 -08:00
Colin Cross
5b73ab34ed Disable duplicates warnings when merging jars
There are too many duplicates warnings, disable them until someone
is actually looking at them.

Bug: 69316739
Test: m checkbuild
Change-Id: Ifb67dbf7734b5f53941d110f578c4fe642e36005
2018-02-14 18:35:37 +00:00
Colin Cross
fdea893833 Replace build/tools with build/make/tools
Test: m checkbuild
Change-Id: I8db6c27a6d2372f142d10568fe92d6f2e9a59252
2017-12-06 16:12:40 -08:00
Jeff Gaston
f001a5097c Update a Turbine Jacoco check
Test: m EMMA_INSTRUMENT=true
Bug: 64308460
Change-Id: Idbfa58bdea178495e1f717043b908670fd4b15ed
2017-10-25 20:15:51 +00:00
Colin Cross
7dc9043546 Allow disabling turbine
m -j TURBINE_ENABLED=false will build without turbine.  This is
primarily useful for A/B comparisions of turbine builds, but will
also be used to temporarily disable turbine for unbundled builds
until we have turbine prebuilts.

Bug: 64308460
Test: m -j TURBINE_ENABLED=false java
Test: m -j java
Change-Id: Ie48746c8bfc60c361be9634cb1805ca1c09aa1fe
2017-09-26 16:23:00 -07:00
Colin Cross
1d1e5ef61b Add dependencies on bootclasspath libraries
The libraries in the bootclasspath were getting their dependencies
because they were also in the default libraries list.  Make the
dependencies explicit in preparation for rearranging the default
libraries.

Test: m -j checkbuild
Change-Id: I3ea004714a31cd648ec2ef57ed3099bce70bc6b9
2017-09-25 16:14:42 -07:00
Nan Zhang
b88011768d Move private_manifest file insertion to jars merging phase.
By doing this, the change of private_manifest file won't trigger
Javac to re-run during re-build.

Bug: b/65455145
Test: m clean && m -j32
Change-Id: Ib90d56daf0be5f31b51e07b7714a933bc5cf4496
2017-09-20 14:56:48 -07:00
Nan Zhang
9bd540594d Use merge_zips to merge jars during Java build in Make.
1. Use merge_zips to merge jars without unzipping/rezipping static Java
libraries. And seperate merging process as a independent Make target so
that Javac can only depend on Turbine.

2. Remove duplicate vars: "full_java_lib_deps" in droiddoc.mk

3. Also use classes-header.jar for link_apk & link_instr

Bug: b/65455145
Test: m clean && m -j32

Change-Id: I7b0307664a6b86075c983284fce099bd3946318c
2017-09-19 18:49:33 -07:00
Nan Zhang
b3ec534bd1 Revert "Revert "Add Turbine as part of java build toolchain in Make.""
This reverts commit 33dcd0c5eb.

Reason for revert: <stub.jar.toc needs classes.jar.toc which is no longer existed after Turbine was introduced. So use classes.jar to represent stub.jar.toc>

Bug: b/64308460
Test: m clean && time m -j32 ANDROID_COMPILE_WITH_JACK=true
Change-Id: If618993f1e0c789d93cc15664212603f994fd460
2017-08-31 14:56:54 -07:00
Colin Cross
9a594e05af Remove GLOBAL_JAVAC_DEBUG_FLAGS
GLOBAL_JAVAC_DEBUG_FLAGS is no longer set, -g is already included
in the javac command line.  Jack is on its way out, just use -g
directly there for now.

Test: m -j checkbuild
Change-Id: Ib99c248bbb8b65fa2cd2b55449a502fc20deffa8
2017-08-31 14:48:31 -07:00
Nan Zhang
33dcd0c5eb Revert "Add Turbine as part of java build toolchain in Make."
This reverts commit 0739155c86.

Reason for revert: <sdk_google_aw_x86-sdk (linux) build 4310016 in git_master is broken>

Change-Id: I525d48557bd0425917e61a85f596663ccc63c887
2017-08-31 19:56:59 +00:00
Nan Zhang
0739155c86 Add Turbine as part of java build toolchain in Make.
1. Bundle static_java_header_libs during turbine-classes.jar if
necessary. This can be improved if we have the ability to merge jars on
the fly.

2. Copy sdk/stubs-classes.jar to sdk/stubs-classes-header.jar, and Use
.KATI_RESTAT to cut un-necessary targets rebuilt.

3. Copy prebuilt-classes.jar to prebuilt-classes-header.jar since
Turbine doesn't apply to prebuilt.

4. Run jarjar after Turbine compilation, otherwise downstream targets
won't find corresponding symbols.

5. Change classes.jar in -classpath used in Desugar to
classes-header.jar.

6. Change legacy-libs in Proguard from classes.jar to
classes-header.jar.

7. Add .KATI_RESTAT for turbine-classes.jar, and remove IJAR.

8.boot.art is re-generated during the incremental rebuild since
javalib.jar got re-generated. Then boot.art re-triggered lots of
downstream targets to be rebuilt. So we add .KATI_RESTAT for javalib.jar
to prevent downstream to be rebuilt. The ziptime & commit-change-to-toc
operation for javalib.jar(small) didn't bring too much overhead when we build
from clean state.

Next step: specify static_java_header_libs in -classpath, and merge jars
at the end.

touch frameworks/base/core/java/com/google/android/util/Procedure.java &&
time m
time: 1m58s (794 -> 49 targets)

m clean && time m java
time: 9m34s

Bug: b/64308460

Test: m clean && m checkbuild

Change-Id: Iefcc234405b9f461b6882c06bba15e21fa783d28
2017-08-30 16:01:39 -07:00
Colin Cross
dfc45ec337 Separate java source list generation into its own rule
Move the java source list generation step into its own rule.  This
has a couple of advantages.  It consolidates the source list
generation so that it only has to run once for javac, jack, and
jack-check.  It also massively reduces the length of the javac
command line, so that error messages are significantly shorter,
and allows easily rerunning the failing build command because
the file list is still on disk.

The primary disadvantage is that javac error messages no longer
include the list of files passed to javac, but the list is
available earlier in the build long when the file list was written,
and is still available on disk.

Test: m -j javac-check
Change-Id: I9730b352b33a060e08221b61c11c617d23320d67
2017-08-29 16:50:11 -07:00
Colin Cross
128800f2c8 Sort files in directories passed to jar
jar -C <dir> . produces a jar containing files in filesystem order,
which can vary between builds.  Manually find and sort the list of
files, and convert them into a list of -C <dir> <file> pairs.

Fixes: 64634025
Test: m -j checkbuild
Test: m -j out/target/product/sailfish/system/framework/ext.jar, check
      that entries are sorted
Test: m -j out/target/product/generic_arm64/system/framework/ext.jar on mac
Change-Id: I7dced6acbe621a60cd49daf17872941485602732
2017-08-16 13:37:00 -07:00
Jeff Gaston
aaae43cd24 Update to latest jacoco usage
Bug: 36792868
Test: EMMA_INSTRUMENT_STATIC=true m -j

Change-Id: I419b543283b52be9a72f5c6b10e4cbea68782174
2017-07-14 13:04:05 -07:00
Colin Cross
0851df8c82 Provide make variables for java binaries
Replace java -> $(JAVA), etc. to prepare for moving java binaries
out of the path and using full paths to them.

Test: builds
Bug: 62956999
Change-Id: I5dfcd0d4e855ecde69cd127169e9545e59723d12
2017-07-01 01:03:14 +00:00
Colin Cross
1664b90d67 Normalize timestamps in host java libraries
Remove timestamps from jar files in host java libraries to get
consistent results between builds to help with build artifact
caching.

Bug: 38215808
Test: sha1sum desugar.jar && touch Desugar.java && m -j desugar && sha1sum desugar.jar
Change-Id: I1ecac9b2b80f673937086686540ab0736259df1b
2017-05-16 11:07:28 -07:00
Colin Cross
5f89212e24 Filter and colorize javac output V3
Add soong_javac_wrapper to all javac commands. soong_javac_wrapper
colorizes javac output, and hides noisy messages.

Previously attempted in I7e8fcd2e4e1ed3ff530a8ccfe931ceb7e411e0ad
and I449a61ed2c95b3e1afd51a31a11baab15d97e2f2.

Test: m -j ANDROID_COMPILE_WITH_JACK=false
Change-Id: I9d7a20d766fadab323a4cf95e5734741b3f578bb
2017-04-18 21:04:58 +00:00
Colin Cross
cde14987ba Support java annotation proccessors
The codebase is growing more uses of annotation processors,
through a mixture of LOCAL_ADDITIONAL_DEPENDENCIES and
LOCAL_JACK_FLAGS.  Add LOCAL_ANNOTATION_PROCESSORS and handle
the rest in the build system.

For now we also need LOCAL_ANNOTATION_PROCESSOR_CLASSES to
work around bad behavior in grok (b/25860419).

Test: m -j java
Change-Id: I465b021e65d25ed3ea517333a82ec4ebd63fc038
2017-04-11 09:56:09 -07:00
Colin Cross
5e58793731 Add support for LOCAL_ERROR_PRONE_FLAGS
This will allow individual modules to enable error prone checks
as needed so that the change to treat an error prone check as a
compiler error can be done atomically with the changes to fix
the issues that check reveals.

Tested with the following steps, cleaning up the targets in
between to ensure that it was rebuilt everytime.

Set LOCAL_ERROR_PRONE_FLAGS := -Xep:ClassCanBeStatic:WARN
compiler error in junit-params-test target. Built
javac-check-junit-params-test with RUN_ERROR_PRONE=true to make
sure that compilation failed.

Built the javac-check-junit-params-test without RUN_ERROR_PRONE
set to make sure that the flag is not included unless explicitly
requested.

Built the junit-params-test as normal to make sure it still
worked properly.

Bug: 31507496
Test: See above
Change-Id: I88b5800dacd0e30dd9dcfa5f42e89d1428957365
Merged-In: I88b5800dacd0e30dd9dcfa5f42e89d1428957365
(cherry picked from commit 3ca9298d99)
2017-04-07 17:46:40 -07:00
Colin Cross
941b682099 Rearrange java library outputs
Jack can silently handle using a java library as a static java
library by merging in the java library dex file.  This causes
problems when switching to javac, because dex doesn't support
dex merging with multidex enabled?

Make the output files consistent between java libraries and static
java libraries.  Java libraries will now produce:
classes-pre-proguard.jar: the classes before proguard processing
classes.jar: the final jar file containing classes before dexing
javalib.jar: a jar containing classes.dex

Static java libraries will eventually only produce
classes-pre-proguard.jar and classes.jar.  All inter-library
linking is done with classes.jar, so a java library can be
used as a static java library.

There are too many dependencies outside the build system that
expect javalib.jar to exist for static and host java libraries,
so for now continue to build a javalib.jar that is a copy of
classes.jar.

Test: m -j ANDROID_COMPILE_WITH_JACK=false java
Test: m -j java
Bug: 36901093
Change-Id: I6730e2d3ec38004874265b2a690442dec57b33f4
2017-04-05 18:04:37 -07:00
Colin Cross
0e53734035 Optimize out extra jar copies
Don't copy jar files for disabled stages.  Instead, set the name of
the output to the name of the input so the next stage will directly
pick up the output of the previous stage.

Test: m -j ANDROID_COMPILE_WITH_JACK=false java
Change-Id: Ib7268cbff7ea7ff2ad2caf994aa145e1b7a12b83
2017-03-30 19:35:15 -07:00
Colin Cross
fc8ead6609 Revert "Revert "Revert "Add soong_javac_filter to all javac commands"""
This reverts commit 440079b84b.

Adding the pipe interferes with error codes, which results
in an empty classes-full-debug.jar output file being left
after the failure.  The next build sees the output file with
an up-to-date timestamp and doesn't rerun the failing rule.

Bug: 36666657
Change-Id: I6658edb766d8ba3120f88e3d8d6eda6ea6c691da
2017-03-29 21:01:24 +00:00
Colin Cross
440079b84b Revert "Revert "Add soong_javac_filter to all javac commands""
This reverts commit 52d2253ff9.

Test: pipe the grok build output through soong_javac_filter
Change-Id: I449a61ed2c95b3e1afd51a31a11baab15d97e2f2
2017-02-19 19:56:51 -08:00
Colin Cross
52d2253ff9 Revert "Add soong_javac_filter to all javac commands"
This reverts commit a341bf0f86.

Breaks grok builds that replace javac with another tool:
reading standard input: bufio.Scanner: token too long

Change-Id: I825587c465b05c1fbddb7794dc1c84f5f2c883d1
2017-02-17 22:53:42 +00:00
Colin Cross
a341bf0f86 Add soong_javac_filter to all javac commands
soong_javac_filter colorizes javac output, and hides noisy messages.

Test: builds
Change-Id: I7e8fcd2e4e1ed3ff530a8ccfe931ceb7e411e0ad
2017-02-16 14:08:45 -08:00
Paul Duffin
32bd0adf9c Add targets for building individual modules using javac
This is needed in order to be able to run error prone (which is
a plugin replacement for javac) on a subset of the java code.

Bug: 31507496
Test: Build javac-check-core-tests and check the resulting JAR
Change-Id: I5a556920572fcdad7b39d67e3f00fe0e0f1f8f46
2016-09-19 13:38:51 +01:00
Dan Willemsen
3bf15e71d3 Record module type statistics
Creates a build_system_stats.txt build artifact that contains statistics
on how many BUILD_* modules are defined in a build. Also writes out
information about the Soong module types sent from the Soong build.

Merged-In: Iaf0c7062f542dc6942b5349854f3d49267cac4a5
Change-Id: Iaf0c7062f542dc6942b5349854f3d49267cac4a5
2016-07-29 19:58:35 +00:00
Shinichiro Hamaji
d50001e74a Add normalize_path.py to prerequisites
Bug: 27954979
Change-Id: I1e22eb4f42a8cb348ce1101e207e239bc5aa7976
2016-04-19 16:22:27 +09:00
Sebastien Hertz
8f0e52c3a6 Keep emma files for host rules
Keep coverage.em file name for host rules where Jack is not called.

(Partially reverts change 491c9ce48a)

Bug: 27822188
Change-Id: Ia9344e17ecbecfb9c335608740ab7dc5490c48d5
2016-03-30 09:39:18 +02:00
Sebastien Hertz
f7e06ef3b4 Merge "Rename emma coverage files" 2016-03-29 07:17:12 +00:00
Sebastien Hertz
491c9ce48a Rename emma coverage files
We no longer use emma with Jack. However, during the process of a
full build, we may run the emma task and overwrite the coverage file
generated by Jack with the one from emma because they use the same
filename.

This CL renames coverage files generated by emma so they no longer
break those created by Jack.

Bug: 27822188
Change-Id: Ie3fdc2b8f53deafbefbd6973e39f29d8c46f371e
2016-03-25 17:11:38 +01:00
Ying Wang
3ae7861252 Remove javac support in host dex rules.
- Remove javac support in host_dalvik_java_library.mk
- Simplify host_dalvik_static_java_library.mk by including
  host_dalvik_java_library.mk instead of host_java_library.mk.
- Take configure_local_jack out of host_java_library_common.mk because
  it's only needed by host dalvik java libraries.

Bug: 27400061
Change-Id: I4088b09092198118d832a759663bbaadff669ed5
2016-03-24 16:48:47 -07:00
Ying Wang
cddeb6c4f0 Add phony target javac-check.
javac-check covers all classes.jars that are compiled with javac
instead of jack. This target is required by tools that still need java
class files as input.

Bug: 26705771
Change-Id: I6896b9a6026a2a0dae6145d1ef5a839756aa3458
2016-03-24 11:12:16 -07:00
Neil Fuller
b97ce3bb61 Merge "Add JarJar support to BUILD_HOST_JAVA_LIBRARY" 2016-03-04 12:58:45 +00:00
Neil Fuller
432bdaa2f7 Add JarJar support to BUILD_HOST_JAVA_LIBRARY
This will allow us to convert icu4j-icudata / icu4j-icutzdata
targets to "host" targets rather than "target" targets.
Right now they are forcing Jack off just so they can use
javac + jarjar (even though the resulting file is used on
host).

Bug: 27218410
(cherry picked from commit 30b21ad579)

Change-Id: I4f16896606bcfdd00164078eeb142d05b29bb832
2016-03-04 12:02:40 +00:00
Dan Willemsen
1c6dc5b942 Remove USE_NINJA=false
And everything special-cased on that. Add a warning if USE_NINJA is
set to let users know that it no longer changes anything.

Change-Id: Ib8739151fe26ea6bf8f76b7ac2b8f4097dab0b47
2016-02-26 21:48:31 -08:00
Colin Cross
a4447e8c89 Don't add dependencies on Makefiles when using ninja
Ninja has an implicit dependency on the command being run, and kati will
regenerate the ninja manifest if any read makefile changes, so there is no
need to have dependencies on makefiles.
This won't catch all the cases where LOCAL_ADDITIONAL_DEPENDENCIES contains
a .mk file, because a few users of LOCAL_ADDITIONAL_DEPENDENCIES don't
include base_rules.mk, but it will fix the most common ones.

Bug: 23566977
Change-Id: I66de882421376303ab7233c8ce7274548f6b2199
2015-09-28 16:29:52 -07:00
Ying Wang
447d69678e Clean up Javac a little bit.
- Added GLOBAL_JAVAC_DEBUG_FLAGS and merge it to
  PRIVATE_JAVACFLAGS/PRIVATE_JACK_FLAGS to get rid of
  PRIVATE_JAVAC_DEBUG_FLAGS/PRIVATE_JACK_DEBUG_FLAGS.
- With Java rules out of base_rules.mk we can get rid
  of java_alternative_checked_module now.

Change-Id: I1a14716c785e3d49330a75044107662ce96a9307
2015-09-02 10:04:18 -07:00
Ying Wang
956dcccf76 Move Java stuff out of base_rules.mk
- Moved the target-only .aidl and .logtag rules to java.mk.
- Moved the .proto rules,  Java library dependency calculation,
  Java resource processing and Java-related
  target-specific and global variables setup to a new java_common.java,
  which is shared by both target and host Java modules.
- Minor cleanup for the moved code.

Bug: 23597357
Change-Id: Ic3eb72e26bb1a663e5739abe0a935f96b88bc352
2015-08-28 17:17:19 -07:00
Yohann Roussel
b518c3ec0b [DO NOT MERGE] Add support for RMTYPEDEF with Jack
Bug: 21099584

(cherry-picked from commit 5b2c6e26db)

Change-Id: I39aa1f24780c216e978ef743098ca7ac65b2962e
2015-06-02 12:25:51 +02:00
Yohann Roussel
f09e59eb52 [DO NOT MERGE] Compile using Jack.
This allows to compile dex targeted java sources using Jack and Jill.
Default is still to compile with the legacy toolchain. Default can be
switched to the new toolchain by setting environement variable:
export ANDROID_COMPILE_WITH_JACK=true
Toolchain can also be forced for one module by defining
LOCAL_JACK_ENABLED:=full # disabled, full, incremental
in the mk portion defining the module.

Jack execution environement can be controlled with:

Global variable ANDROID_JACK_VM allow to change the jvm executing Jack.
Global variable ANDROID_JACK_VM_ARGS allows to change default args given
to the jvm.
Global variable ANDROID_JACK_EXTRA_ARGS allows to define some default args
to give to Jack

LOCAL_JACK_VM_ARGS allows to override default args given to the jvm for
the module.
LOCAL_JACK_EXTRA_ARGS allows to override default args passed to Jack.

This includes cherry-picks of the following changes:
b4c49cba57
22c3fa6d73
138768c1bb
5dd3e1d312
83d5d04047
8bc90fd2d6
140274707e
0fbc9ff2a2
833b427d72
f9a27f45b4
2809666941
37822c443d
c6b44d43c3
d2a76c14bf
06744f60fc
95573d5036
b821391614
2794e7b582
801f2c44d0
c76d99dca1
f528e132d6
76a5e0bd1a
e25b3984ff Partially, only Jack related parts werekept
ec46a3b71f
abee3a9f41
77cbe10fd9
daf07db4cd
b6bfb5893a

Ie all Jack related changes untill
b6bfb5893a
except
a96cc59ab5 "Use Jack by default"

Change-Id: If9d47ef1c4fd1e6765ad2a47d816c1ad3cfab0e3
2015-05-19 18:31:05 +02:00
Dan Albert
954b5bdcf4 Make modules depend on their makefile.
This should obviate much of the need for cleanspecs, and also make it
unnecessary to continue adding LOCAL_ADDITIONAL_DEPENDENCIES for this
sort of thing all over the tree.

Change-Id: I97aa8fd280ae868a5f6364f8b7bf3c2fe235d6ce
2014-11-11 14:30:54 -08:00
Jeff Brown
4c4aa99ced Support more precise package exclusion from JAR files.
Allow LOCAL_JAR_PACKAGES to work on subpackages.  Previously it
only worked on top-level packages due to a bug in how the directories
were recursively deleted.

Add LOCAL_JAR_EXCLUDE_PACKAGES to allow specific subpackages to
be excluded.  This rule applies after LOCAL_JAR_PACKAGES has
selected which packages to include.  It may also be used independently.

Change-Id: Ibd0e495be1a20c84b59c9da132e92100ef3f8705
2014-05-23 18:41:19 -07:00
Ying Wang
1ff0836ce0 Emma support for host Java libraries.
To run emma on a host Java library, in the module definition use:
LOCAL_EMMA_INSTRUMENT := true
then run make with "EMMA_INSTRUMENT=true".

Bug: 12178695
Change-Id: Ie31582b94f5c45381f534912e810f88d21dde9b2
2013-12-30 09:17:06 -08:00
Narayan Kamath
0dd273a3f6 Rework the generation of host java libraries.
We currently have two types of host libraries, those
meant for the host VM (Sun, OpenJDK etc.) and those meant
for a host dalvik build. The former need to be compiled
against the host standard libraries and the latter need
to be compiled against libcore. This change introduces
two new build rules to complement the existing the existing
host rules.

BUILD_HOST_DALVIK_JAVA_LIBRARY : Build a java library for
a host build of dalvik. Bootclasspath will be set to a host
build of libcore.
BUILD_HOST_DALVIK_STATIC_JAVA_LIBRARY : Build a static java
library for a host build of dalvik. Bootclasspath will be set
to a host build of libcore.

This change also removes support for the LOCAL_BUILD_HOST_DEX
flag, which is now unnecessary.

bug: 8992787

Change-Id: I3569fff8eaa4d26d55fcc317bd98471f55d74c14
2013-11-07 09:39:05 +00:00
Chris Wren
3872ead7e6 am 67107450: am d4b1d52b: support protos in host java libraries
* commit '67107450fa0df245c6d1328c17a2af070aa537df':
  support protos in host java libraries
2013-10-02 12:36:27 -07:00
Chris Wren
d4b1d52bb2 support protos in host java libraries
Change-Id: Ia4499d439ea23ee4d6707cfb005b7230c0a2374f
2013-10-02 12:19:28 -07:00