Commit graph

162 commits

Author SHA1 Message Date
Colin Cross
3b706fde7f Shard kythe invocations when javac is sharded
The kythe pipeline has trouble injesting all 6000 files of
framework.jar at once.  Shared invocations of the kythe extractor
when javac is sharded to produce multiple smaller kzip files.

Bug: 140426870
Test: no change to build.ninja
Test: m out/soong/.intermediates/frameworks/base/framework/android_common/framework0.kzip
Test: TestSharding in java_test.go
Change-Id: I867db4ef5cb1e7f3ce8359a46aac2c00ed8a8912
2019-09-05 16:49:11 -07:00
Sasha Smundak
2a4549ec98 Support source code cross-referencing for C++ and Java
Use Kythe (https://kythe.io) to build cross reference for the Android
source code. ~generate the input for it during the build. This is done
on demand: if XREF_CORPUS environment variable is set, build emits a
Ninja rule to generate Kythe input for each compilation rule. It
also emits two consolidation rules (`xref_cxx` and `xref_java`),
that depend on all Kythe input generation rules for C++ and Java.

The value of the XREF_CORPUS environment variable is recorded in the
generated files and thus passed to Kythe. For the AOSP master branch it is
`android.googlesource.com/platform/superproject`, so the command to build
all input for Kythe on that branch is:
```
XREF_CORPUS=android.googlesource.com/platform/superproject m xref_cxx xref_java
```

Each Kythe input generation rule generates a single file with .kzip
extension. Individual .kzip files have a lot of common information, so
there will be a post-build consolidation step run to combine them.
The consolidated .kzip file is then passed to Kythe backend.

The tools to generate .kzip files are provided by Kythe (it calls them
'extractors'). We are going to build them in toolbuilding branches
(clang-tools and build-tools) and check them in as binaries into master
and other PDK branches:
For C++,  `prebuilts/clang-tools/linux-x86/bin/cxx_extractor`
for Java, `prebuilts/build-tools/common/framework/javac_extractor.jar`

Bug: 121267023
Test: 1) When XREF_CORPUS is set, build generates Ninja rules to create
.kzip files; 2) When XREF_CORPUS is set, building
`xref_cxx`/`xref_java` creates .kzip files; 3) Unless XREF_CORPUS is
set, build generates the same Ninja rules as before

Change-Id: If957b35d7abc82dbfbb3665980e7c34afe7c789e
2019-07-26 09:16:47 -07:00
Colin Cross
b77043e207 Add a systemModules utility type
Storing system modules in a classpath is clumsy, as there should
only ever be one system modules, and it needs to store both a
directory to pass as the argument and a set of generated files
to use as dependencies.  Store them in a separate systemModules
type instead.

Test: m checkbuild
Change-Id: I020556c736bd5091865bcca51dc0fb9e4db6b45b
2019-07-18 14:28:17 -07:00
Colin Cross
33961b54e6 Convert droidstubs to android.RuleBuilder
The droiddoc rules are ripe for converting to android.RuleBuilder,
they conditionally use many input files which is hard to track
with the standard blueprint.Rule style, as the argument and the
dependency have to be handled independently.  Start converting
to android.RuleBuilder by converting the droidstubs module.

Test: m docs
Test: m checkapi
Test: m updateapi
Change-Id: I08713e91149471e88a40115f69824cf5eaf88fb6
2019-07-18 14:28:14 -07:00
Dan Willemsen
ff60a73d89 Depend on all the files from system modules
Instead of just one of the files that we pass into javac.

Test: treehugger
Change-Id: I8478e88656487c9f667893d7c17839f0ea63c78f
2019-06-18 01:29:02 +00:00
Sasha Smundak
26c6d9e9fb Tune JVM, print full diagnostics on crash.
JVM occasionally (0.5% time) crashes on entering a method with SIGSEGV.
It might be due to excessive load on the host machine. Reduce the number
of the compiler threads to 6, and do not start all the GC threads
immediately. If JVM crashes, copy full diagnostics to stderr.

Bug: 132766811
Test: treehugger
Change-Id: Ib7bfd568f6d32456bf623cbb615ffc990c326a34
2019-06-11 13:30:13 -07:00
Jimmy Chen
e88eaa7db7 Revert "Revert "Revert "Print full diagnostics file when Java process crashes."""
This reverts commit b4be092855.

Reason for revert: break build

Change-Id: I11fc5eb5f635a5d99eba962f5817851629e50ab2
2019-06-05 06:47:11 +00:00
Alexander Smundak
b4be092855 Revert "Revert "Print full diagnostics file when Java process crashes.""
This reverts commit d71dc6b5e0.

Reason for revert: ag/974065 fixes the problem that caused the revert

Change-Id: I3ba9eecdab34602aaf19fa518520ac932c9b97af
2019-06-03 16:53:22 +00:00
Alexander Smundak
d71dc6b5e0 Revert "Print full diagnostics file when Java process crashes."
This reverts commit 9d6df2246c.

Reason for revert: This change set the value of JAVA make variable to a string containing "foo bar". When soong_ui is asked to evaluate such string, it drops the quotes, causing bash to treat 'foo' and 'bar' as separate command line arguments.

Change-Id: If4e3c4c950c305bb67a572de3d86119c2da81dc1
2019-05-31 03:15:55 +00:00
Sasha Smundak
9d6df2246c Print full diagnostics file when Java process crashes.
Bug: 132766811
Test: treehugger
Change-Id: I8cca2486f2194a8643007a53e4a695d2a18abbf8
2019-05-29 17:46:22 -07:00
Colin Cross
3047fa23da Build framework.aidl in Soong
Move the rules to build framework.aidl into Soong, and use it
when compiling aidl files with sdk_version: "current".

Also fixes incorrectly using the aidl includes exported by the
"framework" module when the proguardRaiseDep dependency was
added.

Bug: 130798034
Test: sdk_test.go
Change-Id: I126adf1d9e7b6acb528875ff62b974ba7ad9a337
2019-04-18 13:51:58 -07:00
Vladimir Marko
0975ee0de3 Check package restrictions for Java libs.
Test: m checkbuild; inspect verbose log.
Test: Manual - compile with unmet restrictions.
Bug: 122937705
Change-Id: I9360ae8b6d9ce016b7827be5e8ffc6eb521809b7
2019-04-08 14:02:14 +01:00
Colin Cross
cc0ce80ed5 Fix package path of android/soong/android pctx
android/soong/common was renamed to android/soong/android long
ago, but the pctx package path was still "android/soong/common".
This required all users of rules defined in android/soong/android
to import "android/soong/android" and then
pctx.Import("android/soong/common").

Test: m checkbuild
Change-Id: I20d096522760538f7cfc2bec3d4bfeba99b275d4
2019-04-04 03:59:34 +00:00
Colin Cross
19878da6a0 Move proto compilation to RuleBuilder
Using blueprint.Rule for protoc commands was causing code duplication
because there was no good way to run the same protoc for cc, java and
python but then run custom source packaging steps for java and python.
Move most of the code into a common function that returns a
RuleBuilder, and then let java and python add their own commands at
the end of the rule.

Bug: 70706119
Test: All Soong tests
Test: m checkbuild
Change-Id: Ic692136775d273bcc4f4de99620ab4878667c83a
2019-04-02 16:38:47 +00:00
Colin Cross
0b9f31fb08 Replace *Escape with *EscapeList
Follow the change to blueprint to make *Escape take and return a string
and add *EscapeList that take and return slices of strings.  Fix up
a few places that were unnecessarily converting a string to a slice
and back to a string.

Test: m nothing
Change-Id: I3fa87de175522205f36544ef76aa2f04aef1b936
2019-03-04 18:11:53 +00:00
Colin Cross
e4246abd7f Make manifest and APK agree on uncompressed native libs
Only put uncompressed native libs in an APK if the min_sdk_version
supports it (>= 23, Marshmallow), and set
android:extractNativeLibs="false" in the AndroidManifest.xml so
that the platform won't extract them anyways.

Bug: 117618214
Test: m checkbuild
Change-Id: I760017e48bf3c6b618aabde0982df45995765d48
2019-02-08 15:24:47 +00:00
Colin Cross
44c29a8fe1 Clean up srcjar directories after each rule
The srcjar directories just confuse people looking for generated
files, clean them up after each rule.

Test: m checkbuild
Change-Id: If712ce7d4922619fd8420ae8f89830fe7824114d
2019-01-25 00:53:05 +00:00
Colin Cross
7788c1283f Remove annotation_processors property
Now that there are no uses left of annotation_processors remove
the property.

If there are no annotation processor classes for javac for a module,
due to an empty "plugins" property or using kapt for annotation
processors, pass -proc:none to javac to ensure it does not try
to run any annotation processors found in the classpath.

Bug: 77284273
Test: plugin_test.go, kotlin_test.go
Change-Id: I3823d9fec8d3d07d2e49b1d97839f7fcbdd35647
2019-01-24 23:48:04 +00:00
Colin Cross
be9cdb8d64 Replace annotation_processors with plugins
Follow bazel's modules for annotation processors by introducing
a java_plugin module type that can contain extra metadata about
the annotation processor, the processor class and a flag to
specify if the annotation processor is compatible with the
turbine optimization.  Deprecate the annotation_processors
property, which took a list of java_library_host modules, in
favor of the plugins property, which takes a list of java_plugin
modules.  The annotation_processors property will be removed
once all uses have been replaced with plugins.

Bug: 77284273
Test: plugin_test.go
Test: m caliper
Change-Id: I37c1e80eba71ae2d6a06199fb102194a51994989
2019-01-23 20:17:51 +00:00
Colin Cross
afbb1734f6 Pass annotation processors to kotlinc
Enable the kotlin-annotation-processing plugin and pass annotation
processors to it.

Bug: 122251693
Test: m checkbuild
Test: TestKapt in kotlin_test.go
Change-Id: I841df454beaaa7edd263eea714ca0d958a03c9de
2019-01-22 21:47:19 +00:00
Colin Cross
21fc9bbe19 Move kotlin to it's own file
kapt is going to make kotlin a little more complicated, move the
rules and tests to their own files.

Bug: 122251693
Test: m checkbuild
Change-Id: Ieed78b97995ced210b710bd50c357514cc8e3bc6
2019-01-22 16:32:11 +00:00
Colin Cross
43f08db29e Dexpreopt soong modules inside soong
Port the dexpreopt logic from Make to the dexpreopt package in Soong,
and use it to dexpreopt Soong modules.  The same package is also
compiled into the dexpreopt_gen binary to generate dexpreopt scripts
for Make modules.

This relands Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb and
I462182638bd57b1367b5bfb0718e975c11ae66f7, along with multiple fixes
to depsfile generation in dexpreopt_gen that caused .odex files for
modules in defined make to be missing dependencies on boot.art, and
a fix to not dexpreopt and strip tests.

Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
      only expected changes to dexpreopt outputs on system_other
      (.vdex files for privileged Soong modules no longer incorrectly
      contain .dex contents).
Test: OUT_DIR=$PWD/out m
Test: NINJA_ARGS="-t deps out/target/product/sailfish/obj/APPS/Contacts_intermediates/dexpreopt.zip" m
Change-Id: I6bb2c971cee65d2338839753aa0d84939f335b1b
2018-12-15 19:07:54 -08:00
Colin Cross
4ff85ebeb7 Revert "Dexpreopt soong modules inside soong"
This reverts commit 29ff88741e.

Test: none
Bug: 119412419
2018-12-14 11:49:58 -08:00
Colin Cross
29ff88741e Dexpreopt soong modules inside soong
Port the dexpreopt logic from Make to the dexpreopt package in Soong,
and use it to dexpreopt Soong modules.  The same package is also
compiled into the dexpreopt_gen binary to generate dexpreopt scripts
for Make modules.

Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
      only expected changes to dexpreopt outputs on system_other
      (.vdex files for privileged Soong modules no longer incorrectly
      contain .dex contents).
Change-Id: Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb
2018-12-13 15:53:54 +00:00
Colin Cross
37a07b1784 Revert "Use protoc-gen-javalite for java lite protos"
This reverts commit 5a5aca0568.

Change-Id: Iadbc2ec7fbc45d888383e0e6f569cbfc5ef6a996
2018-11-04 17:23:48 -08:00
Colin Cross
5a5aca0568 Use protoc-gen-javalite for java lite protos
Protobuf 3.5.2 does not natively support lite protos, instead they
are generated by the protoc-gen-javalite plugin compiled from
external/protobuf-javalite.

Bug: 117607748
Test: m checkbuild
Change-Id: I95c2d873f19d4c00e9dc312d7fdbe98cae250a8b
2018-11-03 00:05:58 +00:00
Colin Cross
094054ab07 Support main_class property in java_binary modules
Add a main_class property that will be used to generate a manifest
containing a Main-Class entry.

Test: m checkbuild
Change-Id: I0a59bb2b93cad915afd82fba708fa0f7eda2fe7a
2018-10-18 14:55:07 -07:00
Colin Cross
a4f08813a3 Add support for JNI libraries to android_app modules
Make android_app modules a MultiTargets module, which means the
common variant will have a list of Targets that it needs to handle.
Collect JNI libraries for each Target, and package them into or
alongside the APK.

Bug: 80095087
Test: app_test.go
Change-Id: Iabd3921e1d4c4b4cfcc7e131a0b0d9ab83b0ebbb
2018-10-08 15:20:56 -07:00
Colin Cross
1d98ee23a3 soong_zip: support globs in -f and -D arguments
-f and -D arguments can now take globs in the Soong format.

Also update the use of soong_zip that jars resources to escape the
globs in the arguments, and then shell-escape them when writing to
the rsp file so the glob escape are not intepreted by ReadRespFile.

Also remove an unused argument to the buildAAR rule that could
have contained values that needed escaping.

Relands I7f20bb169dc01f952d2a7681ec6ee9c05737ed37 with a fix for
trailing "\n" in list files, which causes a call to pathtools.Glob("")
that returns "./", which could then get incorrectly translated to
"../../../" in the zip file. Also adds tests.

Test: m checkbuild
Test: zip_test.go
Change-Id: I54b8eef9231875e6042a32c9f8bcc5c2f779922a
2018-09-28 14:01:29 -07:00
Colin Cross
7f33b81ef6 Revert "soong_zip: support globs in -f and -D arguments"
This reverts commit 08e28abc4e.

Reason for revert: inserts extra ../../ entries in the zip file.

Bug: 116737386
Change-Id: I048f94889a66fe618058e37a5827d0cf6b1a6b68
2018-09-26 21:38:07 +00:00
Colin Cross
08e28abc4e soong_zip: support globs in -f and -D arguments
-f and -D arguments can now take globs in the Soong format.

Also update the use of soong_zip that jars resources to escape the
globs in the arguments, and then shell-escape them when writing to
the rsp file so the glob escape are not intepreted by ReadRespFile.

Also remove an unused argument to the buildAAR rule that could
have contained values that needed escaping.

Test: m checkbuild
Change-Id: I7f20bb169dc01f952d2a7681ec6ee9c05737ed37
2018-09-21 16:08:16 -07:00
Colin Cross
fe4bc36f87 Allow '$' in some paths
The icu resource directories contain filenames that have '$'
characters.

Allow paths returned by the Glob functions to contain '$', on the
assumption that real paths on disk are unlikely to contain strings
that are valid ninja variables.  Fix the Build rules to escape any
paths that are passed as Path arguments.  Fix the resource rules to
manually escape the paths that are passed as strings.

Test: m checkbuild
Change-Id: Ie631bc6d96259e592adb280491a365c0df7ed0e2
2018-09-13 18:27:50 +00:00
Colin Cross
0f40a7527a Move kotlinc-build.xml out of classes directory
kotlinc-build.xml was ending up in the final jar because it was
being created in the directory where the classes were written.
Rename the variable containing the classes directory from outDir
to classesDir to discourage putting other files in there, and
move kotlinc-build.xml up a directory.

Bug: 114524493
Test: m checkbuild
Change-Id: Id2e2af0a6543499bd25b8fbd642bc59b14bd241f
2018-09-08 21:07:30 +00:00
Nan Zhang
4c819fb590 Add jetifier support in Soong.
This is a blocker for droiddoc targets migration under vendor/.

Test: N/A
Bug: b/72552006
Change-Id: If85a0917c1ac0d88b9d5b488216a638b4dabd39f
2018-08-29 11:37:32 -07:00
Colin Cross
9b38aef182 Revert "Add support for renamed kotlin stdlib."
This reverts commit 66c0c4067f.

Bug: 112674805
Test: m checkbuild
Change-Id: I687888136b570f080b57af352af0130355acd216
2018-08-29 05:41:25 +00:00
Colin Cross
8144008360 Support patch_module in java modules
A few tests that have classes in the java.base module need to pass
--patch-module=java.base=<classpath> to javac.

Test: m checkbuild
Change-Id: I246bad92dcde976969b064aace5e2856e2bac971
2018-08-16 16:14:09 -07:00
Colin Cross
3063b78ea5 Make :module provide the output file for java modules
Make :module on a java_library provide the output file, which is
normally the implementation jar.  For java_library modules with
installable: true or compile_dex: true this will be the dexjar
instead.  For android_app modules this will be the apk.

Bug: 80144045
Test: no change to out/soong/build.ninja
Change-Id: I739674aee60a38bfccb859369e4414b46f293d82
2018-08-16 16:13:58 -07:00
Colin Cross
37f6d79c7e Add exclude_files and exclude_dirs properties to java_import
Prebuilt jars sometime contain files that we don't want.  In Make
we would delete everything in META-INF when importing jars, but
that caused problems when there were necessary files in there,
so we added LOCAL_DONT_DELETE_JAR_META_INF.

Soong does the opposite, keeping everything by default.  Add
properties to allow explicitly stripping unwanted files instead.

Bug: 111389216
Test: m checkbuild
Change-Id: I6d07f519ebc7d0e1bf0af93416bb569e3c2b1500
2018-07-12 16:59:59 -07:00
Sundong Ahn
24a099c3b8 Add condtion to avoid build error
If docs file which is result of javadoc is null, there is a build error
when building stubs file. To avoid build error, we don't build if srcJar
files is exist but they don't have java file list.

Bug:77577799
Test: make -j
Test: Marking @hide to all APIs in javax.obex library
      -> make -j -> check build result.
      result is success

Change-Id: I85439417a36425e89a8fa2ecbc66d1b2022bc63a
2018-06-29 10:51:48 +09:00
Colin Cross
6654810f37 Run ErrorProne in OpenJDK9
Use the OpenJDK9 javac to run ErrorProne by adding it to
-processorpath and using -Xplugin:ErrorProne.

Bug: 69485063
Test: m RUN_ERROR_PRONE=true javac-check
Change-Id: I0496006b71b70766ef16d57753cbcf037897799c
2018-06-20 21:03:16 -07:00
Colin Cross
6a77c9871c Use -processorpath for annotation processors
ErrorProne needs to be added to -processorpath, but once anything is
passed via -processorpath javac stops looking in the classpath for
annotation processors.  Move annotation processors to -processorpath
in preparation.

This reapplies Ife06b3c7432ab971f5605dcc18a558bcbd99a675 with fixes
to processorpath in errorprone rules.

Test: m java
Change-Id: I188136771386a2183fb31f4b70b50c95ac419ff8
2018-06-20 21:03:07 -07:00
Colin Cross
2f10997e6b Revert "Use -processorpath for annotation processors"
This reverts commit 6dd0bb3b77.

Reason for revert: broke RUN_ERROR_PRONE=true build

Change-Id: I23d066281e392c6e7df29c5cd4558c7cf731c1a5
2018-06-21 03:56:45 +00:00
Colin Cross
6dd0bb3b77 Use -processorpath for annotation processors
ErrorProne needs to be added to -processorpath, but once anything is
passed via -processorpath javac stops looking in the classpath for
annotation processors.  Move annotation processors to -processorpath
in preparation.

Test: m java
Change-Id: Ife06b3c7432ab971f5605dcc18a558bcbd99a675
2018-06-20 16:16:49 -07:00
Colin Cross
bafb897de7 Revert "Revert "Remove desugar support""
This reverts commit 7576916efc.

Reason for revert: includes art veridex fix this time

Change-Id: Ia8da00b5ec8ed3dc3ce2a2521f0a7b2b2b131960
2018-06-06 21:46:32 +00:00
Colin Cross
7576916efc Revert "Remove desugar support"
This reverts commit 000280bb0e.

Reason for revert: broke art veridex build

Change-Id: I9f7d7cdbe8e34671614687ac0b34c0338ef240e7
2018-06-06 21:42:39 +00:00
Colin Cross
000280bb0e Remove desugar support
Desugaring is always handled by d8 now.

Test: m checkbuild
Change-Id: I29e8f68e51b194a4090866acc63f19f0605e318d
2018-06-06 10:05:06 -07:00
Przemyslaw Szczepaniak
66c0c4067f Add support for renamed kotlin stdlib.
Added new CompilerProperty flag, rename_kotlin_stdlib, which
allow to build kotlin libraries/binaries that use platform internal
version of kotlin stdlib (com.android.kotlin.*). This way, app-provided
kotlin standard library won't collide with version used internaly.

+ remove kotlinc-build.xml after compilation so it won't end up in the
result jar file

+ remove *.kotlin_module and *.kotlin_bultin filesfrom result jar file.
These files are needed only by kotlin-reflect library and
need more work to support kotlin-stdlib renaming.

Bug: 73281388
Test: java_test.go
Change-Id: Iae7ccb919e2ae9853b3f30f3dd447ebd01a1bed0
2018-03-20 12:55:19 +00:00
Colin Cross
436b76564d Replace extract-srcjars.sh with zipsync tool
extract_srcjars.sh uses zipinfo and unzip, which fail with an
error on an empty zip file.  Instead of trying to hack around
this (which is hard to make guarantees for since they are
non-hermetic host tools), replace them with a go tool to unzip
a set of zip files into a directory.

Bug: 73885582
Test: m checkbuild
Change-Id: I151fed347ed5196726e36866ffc27bc831799afb
2018-03-15 16:42:43 -07:00
Colin Cross
924a0aa7d5 Update turbine command line flags
The latest version of turbine requires a -- after the javac
options, and uses multiple --bootclasspath arguments instead
of a javac-style colon separated list.

Test: m javac-check
Bug: 74339924
Change-Id: Ib7096294a43e9bd30abc36751299bfac7879f765
2018-03-13 22:22:49 -07:00
Colin Cross
6981f658ef Use turbine's native source jar support
Turbine doesn't need extract_srcjars.sh, just pass the source jars
with --source_jars.

Bug: 74339924
Test: m checkbuild
Change-Id: Ie7fe06f5a9f5e29287cf79c2d46b23a676a03d41
2018-03-07 15:14:50 -08:00
Dan Willemsen
ab9f4268c0 Add proto.canonical_path_from_root
Historically, we've always passed '-I .' as the first argument to
protoc, essentially treating all proto file package names as their full
path in the android source tree. This would make sense in a monorepo
world, but it makes less sense when we're pulling in external projects
with established package names.

So keep the same default (for now), but allow individual builds to opt
into using local paths as the default names with
'canonical_path_from_root: false'. A cleanup effort and/or large scale
change in the future could change the default to false.

As part of this, run protoc once per input proto file, since the flags
may need to change per-file. We'll also need this in order to specify
--dependency_out in the future.

Bug: 70704330
Test: aosp/master build-aosp_arm.ninja is identical
Test: aosp/master soong/build.ninja has expected changes
Test: m
Test: Build protobuf test
Change-Id: I9d6de9fd630326bbcced1c62a4a7e9546429b0ce
2018-02-22 16:48:35 -08:00
Nan Zhang
581fd21e91 Droiddoc Support in Soong
Support Droiddoc to Soong based on core/droiddoc.mk. The non-std doclet
based droiddoc compilation output is a "real" stubs.jar instead of a
directory of java files and a timestamp file.

The std doclet based javadoc compilation output is a "empty" stubs.jar
instead of a timestamp file.

The stubs.jar will be exported to
out/target/common/obj/JAVA_LIBRARIES/$(LOCAL_MODULE)_intermediates/classes.jar
and out/target/common/docs/$(LOCAL_MODULE)-stubs.jar

A $(LOCAL_MODULE).zip file will be generated also, and is exported to
out/target/common/docs/$(LOCAL_MODULE)-docs.zip if property: installable is not set
to false.

Bug: b/70351683
Test: unittest + convert libcore docs Android.mk to Soong manually.

Change-Id: I1cffddd138a5d9d445f86a3d4a3fd4de88a2bc0f
(cherry picked from commit 78188ec622cb1ee24171455867fc58ffab91562e)
2018-02-22 11:14:13 -08:00
Colin Cross
f7eac7a2ac Fix comment on how java rules handle unknown output files
Test: none
Change-Id: I985e7961c87dd145d6eb2bd94cb8bdc43becc360
2018-02-20 17:54:00 +00:00
Przemyslaw Szczepaniak
4b5fe9d1b4 Add rsp and srcjar support to kotlinc build rule
Rsp files are supported through helper script (gen-kotlin-build-file.sh)
that generates the kotlinc module/build xml file.

Since rsp files are supported, I've added ExtractSrcJarsCmd step
to handle srcjars extraction.

Minor reorderings to make sure that TransformKotlinToClasses
recives only .java and .kt files when called from Module.compile.

Bug: 73281388
Test: make -j hidl-doc
Change-Id: I5a40b914569018dc529903a7f2864a5aeae838e5
2018-02-14 20:47:17 +00:00
Nan Zhang
674dd932d8 Add Respfile support for soong_zip.
Sometime the size of our command line passed to soong_zip go program
exceeds the cmdline size limit. So add an RespFile support with "@" special
character prefix.

The args in the cmdline will be considered together with the
args in RespFile during soong_zip running.

Test: real tests in my local machine, and compare the
res/libphonenumber.jar before and after changes.

./cmd -o test.zip '""'-C -> [./cmd,-o,test.zip,""-C]
./cmd -o test.zip '-C -f -> [./cmd,-o,test.zip,-C -f]
./cmd -o test.zip '\"'-C -f -> [./cmd,-o,test.zip,\"-C -f]
./cmd -o test.zip '\\'-C -f -> [./cmd,-o,test.zip,\\-C -f]
./cmd -o test.zip '\a'-C -f -> [./cmd,-o,test.zip,\a-C -f]

./cmd -o test.zip \'-C -> [./cmd,-o,test.zip,'-C]
./cmd -o test.zip \\-C -> [./cmd,-o,test.zip,\-C]
./cmd -o test.zip \"-C -> [./cmd,-o,test.zip,"-C]

./cmd -o test.zip "'"-C -> [./cmd,-o,test.zip,'-C]
./cmd -o test.zip "\\"-C -f -> [./cmd,-o,test.zip,\a-C -f]
./cmd -o test.zip "\""-C -f -> [./cmd,-o,test.zip,"a-C -f]

Bug: b/72484223

Change-Id: I83c3630b70c8396c8e8a3f266244d868d754c4e8
2018-02-13 15:13:01 -08:00
Colin Cross
f91a08caca Disable duplicates warnings when merging jars
There are too many duplicates warnings, disable them until someone
is actually looking at them.

Bug: 69316739
Test: m checkbuild
Change-Id: I7b3315f999dff1c01b6df37ab5ca83f7b8a60bd9
2018-02-08 01:15:39 +00:00
Andreas Gampe
f3e5b558e7 Soong: support Errorprone javacflags
Add support for an errorprone block in Java modules that accepts
additional javacflags for the errorprone build.

Sample:

    errorprone: {
        javacflags: ["-Xep:EqualsNaN:WARN"],
    },

Bug: 72004718
Test: m RUN_ERROR_PRONE=true
Test: manual - add block with ERROR to a project
Test: androidmk_test
Change-Id: I502248fe76c26aa19102f413af72a7324c35b7f4
2018-01-23 20:30:01 -08:00
Colin Cross
66dbc0bc32 Add R8 support
Add support for R8 to optimize apps and java libraries.

Test: m checkbuild
Change-Id: I2afd5d7a84912d3ab613c32c599bd1ebe60562e0
2018-01-03 14:15:57 -08:00
Colin Cross
f0056cb2ed Refactor dexing
Move dexing support into java/dex.go, including the rules and logic
from builder.go and the function from java.go.

Test: no change to build.ninja
Change-Id: I098d2a9774e28079ba44791679a0db6f876fe3e6
2018-01-02 18:23:43 -08:00
Colin Cross
7b60cdd6e5 Strip module-info.class files when combining jars
Combining static jars from dependencies may bring in module-info.class
files, which don't make sense once multiple modules have been combined,
and sometimes confuse downstream tools like desugar.  Strip them out
like make does when combining jars.

Test: m checkbuild
Change-Id: I560c5acfcc6e1be9adf604c22cf200581f92f702
2017-12-21 14:29:56 -08:00
Alan Leung
c37c6343e2 Prep D8 default instead of CompatDX in build/soong
Bug: 69329508
Test: m checkbuild tests
Change-Id: Ibaa2d787c644cf25104af78305a6eed16028dcf0
2017-12-18 23:56:30 -08:00
Colin Cross
9bc4343bea Add missing dependencies on classpath jars for kotlinc
The kotlinc build rules were missing dependencies on the jars in
the classpath.

Test: TestKotlin in java_test.go
Change-Id: I56681785289f942b1070751b7dc5944e7a995b7f
2017-12-16 06:05:38 +00:00
Joe Onorato
09e94ab074 When compiling with the lite protobuf option, pass the option to aprotoc to force the lite runtime.
Test: make
Merged-In: I450f89d144d496a6ddfccc6a6a5a679a05809595
Change-Id: I450f89d144d496a6ddfccc6a6a5a679a05809595
2017-12-15 08:40:27 -08:00
Alan Leung
9f319118e1 Add USE_D8_DESUGAR option in build/soong
Add an option to use D8's desugar instead of the standalone version.

USE_D8_DESUGAR=true m would trigger DCHECK in dex2oat unless
https://android-review.googlesource.com/c/platform/art/+/562595 is
patched in as well.

Bug: 69329508
Test: m && USE_D8_DESUGAR=false m

Change-Id: I864d88e257a2ba0b7f19aa5cced537301950e963
2017-12-11 16:05:51 -08:00
Colin Cross
6510f91a1c Replace ModuleContext.AConfig() with Config()
AConfig() now duplicates Config().  Replace the uses of AConfig()
with Config().  Leave AConfig() for now until code in other
projects is cleaned up.

Test: m checkbuild
Change-Id: Ic88be643049d21dba45dbd1a65588ed94bf43bdc
2017-11-30 00:44:18 +00:00
Tobias Thierer
96427d63bc kotlinc: Always pass -jvm-target 1.8
AOSP's current version of kotlinc only supports -jvm-target
1.8 or 1.6. Previously, we were passing the javaVersion
(the value passed to javac -target) but that would fail if
javaVersion was eg. 1.9.

This CL changes the kotlinc invocation to always pass
-jvm-target 1.8, regardless of javaVersion.

Bug: 69160377
Test: Treehugger
Change-Id: Ib12a8c2afbe908319513861a5380e70bf09d0be0
2017-11-18 04:11:59 +00:00
Colin Cross
d5dbfb78a0 Fix proto flags in java
Compute the common proto flags and pass them to the protoc invocation
when generating java files.

Test: m checkbuild
Change-Id: I0d4c23ad001d01eab03b404545383f009214106d
2017-11-17 11:22:08 -08:00
Nan Zhang
61eaedbddf Support Javac sharding in Soong.
Test: m clean && m -j java and java_test.go

Change-Id: I110a0ff029448d3319aed2788d25d90a6c1a7fa0
2017-11-06 14:23:59 -08:00
Colin Cross
715d7110f7 Fix kotlin -classpath argument
kotlinc expects -classpath, not --classpath.

Also add a test that uses only .kt files to exercise the code when
there are no java sources.

Test: java_test.go
Change-Id: Ifa5a007b460b40ea2188d0907570fbdca6c48da7
2017-10-26 14:47:40 +00:00
Colin Cross
8eadbf0aaf Fix source jars
Source jars were not working as designed because javac will only
compile files from the -sourcepath if there are references to them
starting from files on the command line.  Switch to extracting
the source jars into a directory and passing a list of the files
to javac.

Test: m checkbuild
Change-Id: I9f7d824f8538d081b2f5ad64ae3cbfd0e96213af
2017-10-26 01:00:46 +00:00
Colin Cross
ae88703df5 Move ModuleContext.ModuleBuild to ModuleContext.Build
Now that android.ModuleContext does not include blueprint.ModuleContext
we can rename android.ModuleContext.ModuleBuild to
android.ModuleContext.Build without colliding with
blueprint.ModuleContext.Build.  Leave ModuleBuild as a wrapper around
Build for now to avoid having to update all the users outside
build/soong simultaneously.

Test: m checkbuild
Change-Id: I18eb8cc04faf002049a11d9aac97e9732ff5d638
2017-10-24 10:59:00 -07:00
Nan Zhang
ed19fc3e2c Support Turbine in Soong.
If sdk jars(android_stubs_current, etc) are compiled using soong java
modules, we have to filter them when running Java build with Turbine.

TODO: provide more unit-tests.

Test: m clean && m -j32; go test java_test

Change-Id: Iad7c241b0e8b0ca760950733f513124b56c84564
2017-10-20 16:26:18 -07:00
Colin Cross
1ee2317555 Move all jar intermediates to subdirs
Make all the jar intermediates called modulename.jar, and put each
in a subdir for the tool that generated it.  This will simplify
using the jars as inputs to genrules and resources.

Test: java_test.go
Change-Id: If00e16bd7df5a4ba24ebc4b68c9ccf2cfda49544
2017-10-19 20:45:11 +00:00
Colin Cross
a482065454 Use soong_javac_wrapper in soong javac compiles
Test: m checkbuild
Change-Id: I0f9ff0a719ac550a011c2878fc51595dc3427aed
2017-10-18 17:09:54 +00:00
Colin Cross
93e8595044 Initial kotlin support
Allow java libraries to specify .kt sources, precompile them with
kotlin, and then pass them in the classpath to javac.

Bug: 65219535
Test: java_test.go
Change-Id: Ife22b6ef82ced9ec26a9e5392b2dadacbb16546f
2017-10-17 16:25:22 -07:00
Colin Cross
59149b6df5 Use jars containg sources for java generators
srcFileLists was an ill-fated attempt to deal with generators that
produce a set of java sources that is not known ahead of time.
For example, the list of files produced by protoc depends on the
package statement in the .proto file.  srcFileLists put the list
of generated files into a file, which was then passed to javac
using the @file syntax.  This worked, but it was too easy to cause
missing dependencies, and will not work well in a future distributed
build environment.

Switch to putting generated sources into a jar, and then pass them
jar to javac using -sourcepath.

Test: m checkbuild
Change-Id: Iaab7a588a6c1239f7bf46e4f1b102b3ef517619b
2017-10-16 19:39:39 -07:00
Colin Cross
e9a275b440 Pass output file names into java.Transform* functions
Pass the output file name into the java.Transform* functions.
This consistently puts control of the filename into java.go,
which is often necessary to avoid collisions when the same
rule is used multiple times in a single module.  It also
has the side-effect of removing the poorly named "stem"
parameters.

Test: java_test.go
Change-Id: I7bc1d1f3bfae6f9d2c92870e6df381817817aab4
2017-10-16 19:39:39 -07:00
Colin Cross
1369cdb280 Initial support for converting jars to java9 system modules
Adds a java_system_modules module type that (when
EXPERIMENTAL_USE_OPENJDK9 is set to true) converts a list of
java library modules and prebuilt jars into system modules,
and plumbs the system modules through to the javac command
line.

Also exports the location of the system modules to make
variables, as well as the name of the default system module.

Test: TestClasspath in java_test.go, runs automatically as part of the build
Bug: 63986449
Change-Id: I27bd5d2010092422a27b69c91568e49010e02f40
2017-10-16 15:00:02 -07:00
Colin Cross
070879e69e Refactor errorprone support to reduce duplication
Use a helper function to set up errorprone and javac compiles.

Test: m -j checkbuild
Test: m -j RUN_ERROR_PRONE=true
Change-Id: Icef3a5e1b359487eea3c3306d3d5763dab912b38
2017-10-16 12:50:29 -07:00
Colin Cross
7db5d6332f Get dex jar resources from classpath jar
Dex jars were getting their resources from the res.jar files of
their transitive static dependencies.  This accidentally bypassed
jarjar on resources, since the jarjar pass only happened once the
resources jar was combined into the classpath jar.  Switch to
getting the resources out of the classpath jar by merging it
with the dex jar while skipping *.class.

Test: m -j checkbuild
Test: compare ext.jar to one generated by make
Change-Id: I5f6f3da738dcb0af56ab9a1bd7174ed5359de2b2
2017-10-04 17:19:43 -07:00
Colin Cross
6af17aa022 Add support for .proto files in java modules
Test: m -j checkbuild
Change-Id: Ia03429948baebff85164a91a34507866c97a08ef
2017-10-03 10:25:15 -07:00
Colin Cross
40a3671416 Remove jarSpec structure
It's not doing anything anymore, and the next patch will need more
complex jar arguments.  Just remove it.

Test: m -j checkbuild
Change-Id: I96d15995e86263ec04fd5c13ab0fd54d8b85c788
2017-09-29 14:02:22 -07:00
Colin Cross
6ade34f641 Initial device java support
First pass at java support for the device.  Adds desugar before
dx, and passes javalib.jar to make.

Test: m -j checkbuild
Change-Id: I3138d943bc4867a52c3fce8fbdb597773793988d
2017-09-20 13:20:45 -07:00
Colin Cross
635acc9446 Rearrange manifest file handling in merge_zips and soong_zip
Jar always puts default MANIFEST.MF files in if none was specified.
Copying that behavior in soong_zip causes problems with merge_zips,
because it ends up taking the default manifest from the classes.jar
instead of the user's manifest from res.jar.  We don't want the
user's manifest in the classes.jar, otherwise a change to the
manifest will cause all the class files to rebuild.  Instead,
move the manifest insertion to the final merge_zips stage.

Test: m -j checkbuild
Change-Id: Id6376961dbaf743c2fb92843f9bdf2e44b963be0
2017-09-20 13:20:45 -07:00
Colin Cross
3a5de657db Add missing space between errorprone flags
A space was missing between flags in the errorprone build rule.

Test: m -j RUN_ERROR_PRONE=true checkbuild
Change-Id: If6f48a700e6f19e318b6cc9734e1d8fd2289491f
2017-09-11 23:10:21 -07:00
Colin Cross
0a6e007e06 Make javac rules output a jar with soong_zip
Make javac rules output a jar file instead of a classes.list.
Combine the output jar, static jar dependencies, and resources
into the final jar using a separate rule.  For now, use a shell
command with unzip and soong_zip to create the final jar,
eventually it will be done with a zip2zip-style jar combiner.

Bug: 64691570
Test: java_test.go
Change-Id: Id8e6313e0097b78947d88e86e47b56ad08caca1a
2017-09-05 22:08:13 -07:00
Colin Cross
fee57cb33e Move error_prone config to external/error_prone
The error_prone config will likely need to be updated at the same
time as the error_prone prebuilts, so move the config into the same
project.  This requires jumping through some hoops because
external/error_prone may not be present in all manifests.

Bug: 64489631
Test: m -j RUN_ERROR_PRONE=true
Change-Id: I2c59df1148134ffedac0e00d32ac7082a9fb5330
2017-09-05 22:08:13 -07:00
Colin Cross
3203dde60b Pass javac flags to errorprone builds
Pass CommonJdkFlags to errorprone builds, which is required for
frameworks/base/layoutlib/create to work correctly.  Unfortunately,
javac and errorprone cannot use the same syntax for the heap size,
javac needs -J-Xmx and errorprone needs -Xmx, so split the heap size
flag out into a separate variable.

Test: m -j RUN_ERROR_PRONE=true javac-check
Change-Id: Icd7f8cf627534fb089ec83e462c060572dd2d20c
2017-08-28 19:27:36 -07:00
Colin Cross
c6bbef326f Add error-prone support
Add support for compiling java sources with the error-prone tool.

Test: m -j checkbuild
Change-Id: Ieb4ee0e05f8f34a52ed7bcf1c7cbacf1c9c4d0b5
2017-08-28 11:12:38 -07:00
Colin Cross
b852a58aa6 Convert soong java from soong_zip to jar
soong_zip produces jar files that are not compatible with java's
ZipInputStream.  Switch to jar using ugly sed scripts to munge
file lists into the alternating -C and file arguments required
by jar.

Bug: 64536066
Test: m -j checkbuild
Test: build/soong/scripts/jar_args.sh --test
Change-Id: Ifcc4bdab25e7d02342720eb246c673ff9a58bddb
2017-08-14 10:07:53 -07:00
Colin Cross
6416271a1f Bring java support closer to current version of make
Make the javac arguments match what is used by make, and export them
back to make.  A future change will switch make to use the the
exported ones.

This makes a dx.jar compiled with soong have identical class files
as one compiled with make.

Test: manual
Change-Id: Ia5196f1f42bc564e99de22e32e72fd2930e9fbae
2017-08-11 15:24:11 -07:00
Colin Cross
74d73e2bfb Rename java_prebuilt_library to java_import
And make it work like bazel's java_import, using a "jars" property
instead of "srcs", and allowing multiple jars to be listed.

Test: soong tests
Change-Id: Ida2ace6412bd77b4feb423646000a1401004e0ea
2017-08-11 15:24:11 -07:00
Colin Cross
eb54da6ebe Fix incorrect package for variable
java doesn't use a config package for its variables

Test: java_test.go
Change-Id: I84c396b794d6eef971b1ef174a6f6e529071c452
2017-07-14 08:24:36 -07:00
Yoshisato Yanagisawa
572324ac33 Add JAVAC_WRAPPER support
To run javac with AndroidGomaStaticRule, let me make JAVAC_WRAPPER
also show up to javac/builder.go.

Test: USE_GOMA=true JAVAC_WRAPPER=gomacc m -j32
Bug: b/62334576

Change-Id: I19e1e4315f71d8706800da810b5212f5964171a4
2017-06-14 01:04:50 +00:00
Colin Cross
67a5c132c5 Prettify soong ninja build descriptions
Descriptions currently look like:
[  0% 4/29328] cc out-soong/.intermediates/external/clang/lib/Sema/libclangSema/android_arm_armv7-a-neon_denver_static_core/obj/external/clang/lib/Sema/SemaCodeComplete.o

This is not very helpful - most of the characters are used to show the
output path, which contains useful information like target architecture,
but also contains most of the path to the source files twice, and less
useful information like the exact variant name used by soong.
Make the descriptions look like:
[  0% 3/29329] //external/clang/lib/Sema:libclangSema clang++ SemaTemplateInstantiate.cpp

This is //path/to/module:modulename tool relative/path/to/source/file

Test: builds, looks pretty
Change-Id: I3087aa7d4eb1860ef6239d77407b8b35445616d7
2017-05-10 11:04:16 -07:00
Colin Cross
9d45bb78c5 Enable goma in soong
When the UseGoma flag is set, put all rules except the C compilation
rule in an externally defined local_pool, which will have been created
by kati.  The gomacc wrapper will already be in the CC_WRAPPER
environment variable.

Bug: 31142427
Change-Id: I699d4edff2e302eee398dad8692ceb14721a628c
2016-08-31 09:48:08 -07:00
Dan Willemsen
47ec28f3b1 Rename soong_jar to soong_zip
This is a general purpose tool that happens to contain some jar specific
features.

Change-Id: I05f4654d4517c245ad7a3c15492e0d2368bbf64f
2016-08-11 01:30:16 -07:00
Dan Willemsen
c2aa4a9c52 Start using blueprint_go_binary
And install the tools into a more obvious location. soong_env is not
moved, since we need it to exist early, so that we can use it in
soong.bash in case there's a build failure.

Change-Id: I9bd1fa320d84d180b2cf3deb90782d380666f7a6
2016-05-31 18:31:56 -07:00
Colin Cross
635c3b0157 Rename common to android
Rename the "common" package to "android", because common is too
generic.  Also removes all android.Android naming stutter.

Ran:
gomvpkg -from 'android/soong/common' -to 'android/soong/android'
gorename -from '"android/soong/android".AndroidModuleContext' -to 'ModuleContext'
gorename -from '"android/soong/android".AndroidBaseContext' -to 'BaseContext'
gorename -from '"android/soong/android".AndroidModuleBase' -to 'ModuleBase'
gorename -from '"android/soong/android".AndroidBottomUpMutatorContext' -to 'BottomUpMutatorContext'
gorename -from '"android/soong/android".AndroidTopDownMutatorContext' -to 'TopDownMutatorContext'
gorename -from '"android/soong/android".AndroidModule' -to 'Module'

Change-Id: I3b23590b8ce7c8a1ea1139411d84a53163288da7
2016-05-18 15:37:25 -07:00
Dan Willemsen
34cc69e4bf Use Path instead of string for file paths
This centralizes verification and common operations, like converting the
path to a source file to the path for a built object.

It also embeds the configuration knowledge into the path, so that we can
remove "${SrcDir}/path" from the ninja file. When SrcDir is '.', that
leads to paths like './path' instead of just 'path' like make is doing,
causing differences in compiled binaries.

Change-Id: Ib4e8910a6e867ce1b7b420d927c04f1142a7589e
2015-12-09 14:29:12 -08:00