It looks like there is a bug in rewrapper/reproxy that makes it upload the
output file as an input if it still exists in the output tree. This causes
unnecessary cache misses on incremental builds, and can also cause the
action to fail remotely (depending on the service). By removing the output
file at the beginning of the action, we ensure that it can't be uploaded.
Test: ran against a remote execution server; actions no longer fail remotely
Change-Id: I98fcc0cf105553a9920b0e96965a438e052d7c05
The android.WriteFile rule takes careful escaping to produce the
right contents. Wrap it in an android.WriteFileRule that handles
the escaping.
Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
Soong and Make have no ways to figure out what splits will be outputted
from a given android_app_set, so it's impossible for them to provide
full PACKAGES.$(LOCAL_MODULE).CERTIFICATE entries, which are required to
build a final apkcerts.txt. This change makes extract_apks produce
apkcerts.txt files for each input modules instead. The Make-side
counterpart of this change merges all local apkcerts.txt into a final
one.
Fixes: 160119159
Test: main_test.go
Test: m apkcerts-list
Merged-In: I321e80fd636a955213761f56a3ac64bfe7f7f7c0
Change-Id: I321e80fd636a955213761f56a3ac64bfe7f7f7c0
This reverts commit 91206d9ed1.
Reason for revert: package-check.sh doesn't work on mac
Bug: 158081251
Bug: 157649935
Change-Id: I50af71d54d5f1f8b516bfcf1efbcf6217e89c83a
Previously, the package check was only run when building checkbuild or
the phony target created for a specific module. It would not run when
building a module that depended upon a library with the
permitted_packages property. That was because the package check was
only run when the package-check.stamp file was built and that was only
added to the phony and checkbuild targets.
Instead of touching a separate package-check.stamp file to indicate
that the check has been performed this change copies the input jar file
to the package check to a separate ../package-check/<jar> file which is
then treated as the output of the library and is the input for any dex
processing. So, any modules that depend on this library will
transitively depend on the output file produced by the package check
command and so will ensure that the package check is always run.
Test: Removed "android.net" from the permitted_packages for
"framework-tethering", build "framework-tethering" which
triggered and failed the package check and
"com.android.tethering" which did not.
Made this change.
Built "com.android.tethering" which triggered and then failed the
package check.
Removed change to "framework-tethering"
Bug: 157649935
Change-Id: Ib01aa09e13f80282218049270eb7a58ec5f9f605
Test: Ran a build with `RBE_JAR="true" RBE_JAR_EXEC_STRATEGY="remote"
RBE_ZIP="true" RBE_ZIP_EXEC_STRATEGY="remote" ... use_rbe m` and that
succeeded.
Bug: b/156765207
Change-Id: Ia17b532fbb282be414adf879914870082dd0841b
Test: Ran a sample turbine action with:
RBE_TURBINE_EXEC_STRATEGY="remote" RBE_TURBINE="true" ... use_rbe m out/soong/.intermediates/external/protobuf/libprotobuf-java-micro/android_common/turbine/libprotobuf-java-micro.jar
and checked in reproxy_log.txt that remote-metadata had status SUCCESS.
Bug: b/156765475
Change-Id: I85c334e6998c6dd2a68460b3580d50fffd900e6f
kapt claims to support a comma separated list of annotation
processors, but it errors if multiple annotation processors
are given. Surrounding the the list with {} does not error,
but it also doesn't even warn if the second element in the
list is garbage, so it may not be running the second
processor. Repeat the processor argument for each
annotation processor class instead.
Bug: 154736649
Test: TestKapt
Test: m checkbuild
Change-Id: I4c7c161dbf867d7fba1aaf16fd5e502647e3f682
Set a system property to enable Android specific transformer that drops
UnsupportedAppUsage annotation for repackaged classes to avoid adding
unneeded hidden api symbols on jarjar.
Bug: 146418363
Test: m, manual diff hiddenapi-flags.csv
Exempt-From-Owner-Approval: patch approved by owner
Change-Id: I10b49feb15257301574162b5d389cbc30277f672
goma can't handle the --system argument that Android must pass to
javac.
Bug: b/143658984
Test: treehugger
Change-Id: I207668d8baa4ab28160938732a38f696aeac3f44
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.
Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
We were passing "--classpath foo.jar --classpath bar.jar" to turbine,
it now wants (and may always have expected?)
"--classpath foo.jar bar.jar".
Test: m checkbuild
Change-Id: Ib7e0ca64a34e236110f7b785d6a0fb51ed75567e
Java language was set for 1.8 for anything building against the
current SDK because the stubs were not built in Soong, so the
system modules could not be built. The stubs have been built
in Soong since Iabd32b30954b3f4a6d9a779fde52a032b684807e, but
I5e870c34dd0ebc8ae3f888ec627da590c846a76f missed updating
this TODO.
Use 1.9 when building against the stubs, but continue using
1.8 for unbundled builds until we have prebuilt system modules.
Always use the core-current-stubs-system-modules to avoid
splitting android.* packages between the system modules and
the classpath, which would cause new classes in android.*
packages in classpath jars to be ignored. Add a new
java9Classpath field to sdkDep that will contain the stubs jar
when targeting Java language level 1.9, and plumb it through
to javac and turbine. Rename the modules field to bootclasspath.
Bug: 142896162
Test: m checkbuild
Change-Id: Icfd32d0a863b2303a997c7cf03cb3708aade4724
turbine supports taking system modules on the command line,
now that we target Java language level 1.9 and use system modules
by default switch turbine to match javac.
Test: m javac-check
Change-Id: Ieee07502151da0d5693bb8929213d495c039106b
Remove the hardcoded checks against "1.9" by making javaVersion
an enum and implementing javaVersion.usesJavaModules().
Test: TestClasspath
Change-Id: I559eeb1f45880bb8177269c6d977ee4dfbadce57
Android builds by default put artifacts into out/ subdirectory of
the source tree, causing the extractor to record their names as
relative. The indexer considers such files as sources, which is wrong.
Fortunately, the extractor can be fed a set of filename rewriting
rules (see build/tools/vnames.json).
Also, undo previous unsuccessful attempt use to absolute path for the
output directory to distinguish between source code and artifacts.
Bug: 141385476
Test: run the build, inspect compilation units of the kzip file
Change-Id: I89ec3aed8fd14f43ea6e0b226d54f643346f6125
The kythe pipeline has trouble injesting all 6000 files of
framework.jar at once. Shared invocations of the kythe extractor
when javac is sharded to produce multiple smaller kzip files.
Bug: 140426870
Test: no change to build.ninja
Test: m out/soong/.intermediates/frameworks/base/framework/android_common/framework0.kzip
Test: TestSharding in java_test.go
Change-Id: I867db4ef5cb1e7f3ce8359a46aac2c00ed8a8912
Use Kythe (https://kythe.io) to build cross reference for the Android
source code. ~generate the input for it during the build. This is done
on demand: if XREF_CORPUS environment variable is set, build emits a
Ninja rule to generate Kythe input for each compilation rule. It
also emits two consolidation rules (`xref_cxx` and `xref_java`),
that depend on all Kythe input generation rules for C++ and Java.
The value of the XREF_CORPUS environment variable is recorded in the
generated files and thus passed to Kythe. For the AOSP master branch it is
`android.googlesource.com/platform/superproject`, so the command to build
all input for Kythe on that branch is:
```
XREF_CORPUS=android.googlesource.com/platform/superproject m xref_cxx xref_java
```
Each Kythe input generation rule generates a single file with .kzip
extension. Individual .kzip files have a lot of common information, so
there will be a post-build consolidation step run to combine them.
The consolidated .kzip file is then passed to Kythe backend.
The tools to generate .kzip files are provided by Kythe (it calls them
'extractors'). We are going to build them in toolbuilding branches
(clang-tools and build-tools) and check them in as binaries into master
and other PDK branches:
For C++, `prebuilts/clang-tools/linux-x86/bin/cxx_extractor`
for Java, `prebuilts/build-tools/common/framework/javac_extractor.jar`
Bug: 121267023
Test: 1) When XREF_CORPUS is set, build generates Ninja rules to create
.kzip files; 2) When XREF_CORPUS is set, building
`xref_cxx`/`xref_java` creates .kzip files; 3) Unless XREF_CORPUS is
set, build generates the same Ninja rules as before
Change-Id: If957b35d7abc82dbfbb3665980e7c34afe7c789e
Storing system modules in a classpath is clumsy, as there should
only ever be one system modules, and it needs to store both a
directory to pass as the argument and a set of generated files
to use as dependencies. Store them in a separate systemModules
type instead.
Test: m checkbuild
Change-Id: I020556c736bd5091865bcca51dc0fb9e4db6b45b
The droiddoc rules are ripe for converting to android.RuleBuilder,
they conditionally use many input files which is hard to track
with the standard blueprint.Rule style, as the argument and the
dependency have to be handled independently. Start converting
to android.RuleBuilder by converting the droidstubs module.
Test: m docs
Test: m checkapi
Test: m updateapi
Change-Id: I08713e91149471e88a40115f69824cf5eaf88fb6
JVM occasionally (0.5% time) crashes on entering a method with SIGSEGV.
It might be due to excessive load on the host machine. Reduce the number
of the compiler threads to 6, and do not start all the GC threads
immediately. If JVM crashes, copy full diagnostics to stderr.
Bug: 132766811
Test: treehugger
Change-Id: Ib7bfd568f6d32456bf623cbb615ffc990c326a34
This reverts commit d71dc6b5e0.
Reason for revert: ag/974065 fixes the problem that caused the revert
Change-Id: I3ba9eecdab34602aaf19fa518520ac932c9b97af
This reverts commit 9d6df2246c.
Reason for revert: This change set the value of JAVA make variable to a string containing "foo bar". When soong_ui is asked to evaluate such string, it drops the quotes, causing bash to treat 'foo' and 'bar' as separate command line arguments.
Change-Id: If4e3c4c950c305bb67a572de3d86119c2da81dc1
Move the rules to build framework.aidl into Soong, and use it
when compiling aidl files with sdk_version: "current".
Also fixes incorrectly using the aidl includes exported by the
"framework" module when the proguardRaiseDep dependency was
added.
Bug: 130798034
Test: sdk_test.go
Change-Id: I126adf1d9e7b6acb528875ff62b974ba7ad9a337
android/soong/common was renamed to android/soong/android long
ago, but the pctx package path was still "android/soong/common".
This required all users of rules defined in android/soong/android
to import "android/soong/android" and then
pctx.Import("android/soong/common").
Test: m checkbuild
Change-Id: I20d096522760538f7cfc2bec3d4bfeba99b275d4
Using blueprint.Rule for protoc commands was causing code duplication
because there was no good way to run the same protoc for cc, java and
python but then run custom source packaging steps for java and python.
Move most of the code into a common function that returns a
RuleBuilder, and then let java and python add their own commands at
the end of the rule.
Bug: 70706119
Test: All Soong tests
Test: m checkbuild
Change-Id: Ic692136775d273bcc4f4de99620ab4878667c83a
Follow the change to blueprint to make *Escape take and return a string
and add *EscapeList that take and return slices of strings. Fix up
a few places that were unnecessarily converting a string to a slice
and back to a string.
Test: m nothing
Change-Id: I3fa87de175522205f36544ef76aa2f04aef1b936
Only put uncompressed native libs in an APK if the min_sdk_version
supports it (>= 23, Marshmallow), and set
android:extractNativeLibs="false" in the AndroidManifest.xml so
that the platform won't extract them anyways.
Bug: 117618214
Test: m checkbuild
Change-Id: I760017e48bf3c6b618aabde0982df45995765d48
The srcjar directories just confuse people looking for generated
files, clean them up after each rule.
Test: m checkbuild
Change-Id: If712ce7d4922619fd8420ae8f89830fe7824114d
Now that there are no uses left of annotation_processors remove
the property.
If there are no annotation processor classes for javac for a module,
due to an empty "plugins" property or using kapt for annotation
processors, pass -proc:none to javac to ensure it does not try
to run any annotation processors found in the classpath.
Bug: 77284273
Test: plugin_test.go, kotlin_test.go
Change-Id: I3823d9fec8d3d07d2e49b1d97839f7fcbdd35647
Follow bazel's modules for annotation processors by introducing
a java_plugin module type that can contain extra metadata about
the annotation processor, the processor class and a flag to
specify if the annotation processor is compatible with the
turbine optimization. Deprecate the annotation_processors
property, which took a list of java_library_host modules, in
favor of the plugins property, which takes a list of java_plugin
modules. The annotation_processors property will be removed
once all uses have been replaced with plugins.
Bug: 77284273
Test: plugin_test.go
Test: m caliper
Change-Id: I37c1e80eba71ae2d6a06199fb102194a51994989
Enable the kotlin-annotation-processing plugin and pass annotation
processors to it.
Bug: 122251693
Test: m checkbuild
Test: TestKapt in kotlin_test.go
Change-Id: I841df454beaaa7edd263eea714ca0d958a03c9de
kapt is going to make kotlin a little more complicated, move the
rules and tests to their own files.
Bug: 122251693
Test: m checkbuild
Change-Id: Ieed78b97995ced210b710bd50c357514cc8e3bc6
Port the dexpreopt logic from Make to the dexpreopt package in Soong,
and use it to dexpreopt Soong modules. The same package is also
compiled into the dexpreopt_gen binary to generate dexpreopt scripts
for Make modules.
This relands Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb and
I462182638bd57b1367b5bfb0718e975c11ae66f7, along with multiple fixes
to depsfile generation in dexpreopt_gen that caused .odex files for
modules in defined make to be missing dependencies on boot.art, and
a fix to not dexpreopt and strip tests.
Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
only expected changes to dexpreopt outputs on system_other
(.vdex files for privileged Soong modules no longer incorrectly
contain .dex contents).
Test: OUT_DIR=$PWD/out m
Test: NINJA_ARGS="-t deps out/target/product/sailfish/obj/APPS/Contacts_intermediates/dexpreopt.zip" m
Change-Id: I6bb2c971cee65d2338839753aa0d84939f335b1b
Port the dexpreopt logic from Make to the dexpreopt package in Soong,
and use it to dexpreopt Soong modules. The same package is also
compiled into the dexpreopt_gen binary to generate dexpreopt scripts
for Make modules.
Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
only expected changes to dexpreopt outputs on system_other
(.vdex files for privileged Soong modules no longer incorrectly
contain .dex contents).
Change-Id: Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb
