Commit graph

50240 commits

Author SHA1 Message Date
Isaac Chiou
a23d994022 Revert "Use kapt stubs for kotlin header jar for javac"
This reverts commit c6ef485370.

Reason for revert: Multiple systemui presubmits are broken due to this change

Bug: 228262695
Change-Id: I1efed519c49d8d0583815c7bf2b3801bf0ba12fa
2022-04-06 07:04:09 +00:00
Colin Cross
c6ef485370 Use kapt stubs for kotlin header jar for javac
When compling kotlin code with annotation processors we are generating
java stubs for the kotlin sources using kapt, and parsing them in
turbine when running the annotation processors.  Passing --output
to turbine will also compile the stubs into a header jar that can
be used as a dependency to javac, allowing the javac and kotlinc
rules to run in parallel.  The turbine-apt header jar can't be used
as the header jar for downstream modules as it doesn't contain the
kotlin metadata needed by kotlinc rules, so the kotlinc gen-jvm-abi
plugin output is still used for the module's final header jar.

Test: TestKapt
Bug: 222095735
Change-Id: I82d0900f3dc30f3e3ebd7cab0693dfe741d9b955
2022-04-04 16:42:46 -07:00
Colin Cross
b5db401599 Enable turbine for api generating annotation processors when using kotlin
Normally turbine is disabled when an api generating annotation processor
is in use because the annotation processors are run in javac, so any
generated api would not be present in the turbine header jar.  When
compiling with kotlin and annotation processors, the annotation processors
(including any that generate api) are run in an extra turbine invocation
to generate sources to make the results visible to kotlinc, and so can
also be compiled by the normal turbine invocation.  Force turbine enabled
when kotlin sources are present.

Test: m SystemUI
Bug: 222095735
Change-Id: I5fcfe67e46cb4a4e90388f088327488959b6571f
2022-04-04 16:42:46 -07:00
Colin Cross
220a9a1245 Enable kotlin's jvm-abi-gen plugin to generate header jars
Kotlin's jvm-abi-gen plugin can generate header jars similar to the
turbine output for java sources, which can be used to avoid recompiling
downstream modules when the ABI hasn't changed.  Unlike turbine, the
plugin runs as part of the main kotlinc invocation, so it doesn't allow
the downstream modules to start compiling any sooner.

A future possible optimization is to use turbine to compile the kapt
stubs, at least for the  kotlin+annotation processor modules that already
generate them, which would allow compiling downstream modules before
invoking kotlinc or javac, as well as invoking kotlinc and javac in
parallel with each other.

Bug: 222095735
Test: TestKotlin
Test: m SystemUI
Change-Id: Ib1bb2ecea47c851a108a26f9ed4f827f289d1321
2022-04-04 16:42:46 -07:00
Colin Cross
e52c2ac7fb Enable restat for sbox rules
Add support to sbox for only writing output files if they are changed
to support enabling restat.  Move the clearing of the output directory
into sbox, and when write if changed is enabled delete obsolete output
files without deleting output files that will be replaced.

Enable restat for local metalava rules.  Restat for metalava rules
run remotely is still blocked on b/183630617.

Bug: 222095735
Test: TestRuleBuilder_Build
Test: m SystemUI
Change-Id: If8fc47e711f4a6f43947ab69f17cccc8ba6ae61a
2022-04-04 16:42:46 -07:00
Sam Delmerico
3c25ab1ac3 Merge changes I8e4c3f37,I89a32bde
* changes:
  add plugins to javaLibraryAttributes
  convert java_import_host with bp2build; enable jetifier
2022-03-30 21:08:12 +00:00
Treehugger Robot
d93afba1e6 Merge changes from topic "no-srcs-no-deps"
* changes:
  enable conversion of all java_import modules
  emulate java_library static_deps with Bazel exports
2022-03-30 21:06:46 +00:00
Treehugger Robot
91dac89447 Merge "analyze_bcpf: Compute hidden_api package properties" 2022-03-30 20:00:43 +00:00
Cole Faust
71350eaff2 Merge "Always build mk2rbc/rbcrun" 2022-03-30 19:43:06 +00:00
Sam Delmerico
77267c7d9c add plugins to javaLibraryAttributes
Test: go test ./bp2build
Change-Id: I8e4c3f373c62c60ea5013cd8c585915a366c1f81
2022-03-30 19:19:13 +00:00
Sam Delmerico
5f83b491b4 convert java_import_host with bp2build; enable jetifier
Also, add java_imports that were blocked by jetifier binary.
b/221082840 was blocking the build of jetifier which these java_imports
depended on. That bug is now resolved, so we can unblock these targets.

Bug: 220168131
Test: b build //packages/apps/Car/libs/car-ui-lib/car-ui-androidx/...
Test: build/bazel/ci/bp2build.sh
Change-Id: I89a32bde30c133a44aeb7826afa11d4140433f5d
2022-03-30 19:19:13 +00:00
Sam Delmerico
a9b047a9da enable conversion of all java_import modules
Bug: 220168131
Test: b build '...'
Change-Id: Ib1eaf47b81ac19cb6e930e5f9f3fc2aebe02b0c1
2022-03-30 19:19:09 +00:00
Sam Delmerico
c016143602 emulate java_library static_deps with Bazel exports
In Soong, java_library can specify static_deps which are dependencies
that get aggregated into the final jar (akin to static linking). This is
useful because it allows dependencies higher up in the chain to compile
against the APIs exported by transitive dependencies. Bazel does not
support this functionality directly, but it can be emulated via the
exports attribute which makes any targets listed in the attribute public
to targets further up the chain.

Bug: 217236083
Bug: 219908977
Test: b build //external/error_prone:error_prone_core
Test: b build //external/bouncycastle:bouncycastle-host
Test: b build --platforms=//build/bazel/platforms:linux_x86
  //prebuilts/sdk/tools/jetifier/jetifier-standalone:jetifier
Change-Id: I2867e3f816de720a6f4bd9ff7a847d1b0c2da2d6
2022-03-30 18:58:38 +00:00
Paul Duffin
dd97fd2520 analyze_bcpf: Compute hidden_api package properties
Analyzes the signatures of the class members contained within the
bootclasspath_fragment to construct the split_packages,
single_packages and package_prefixes hidden_api properties that are
necessary to remove the internal implementation details from the
hidden API flags.

Bug: 202154151
Test: m analyze_bcpf && analyze_bcpf --bcpf art-bootclasspath-fragment
      m analyze_bcpf && analyze_bcpf --bcpf art-bootclasspath-fragment --fix
      m analyze_bcpf && analyze_bcpf --bcpf com.android.mediaprovider-bootclasspath-fragment --fix
      atest --host analyze_bcpf_test
Change-Id: I4a8e5a8bfee2a44775e714b9226cd4a7382e0123
2022-03-30 16:00:08 +01:00
Paul Duffin
26f19919ea analyze_bcpf: Add --fix option
Add a --fix option that will cause the script to automatically fix the
issues that it finds. It uses the bpmodify tool to add values to the
bootclasspath_fragment's hidden_api properties.

This adds analyze_bcpf to bp2buildModuleDoNotConvertList as
analyze_bcpf depends on bpmodify which is a blueprint_go_binary which
is not yet supported by bazel.

Bug: 202154151
Test: m analyze_bcpf && analyze_bcpf --bcpf art-bootclasspath-fragment
      m analyze_bcpf && analyze_bcpf --bcpf art-bootclasspath-fragment --fix
      atest --host analyze_bcpf_test

Change-Id: I5ee52419b4829474f6dbeb47f86ab2aeb22b1382
2022-03-30 16:00:08 +01:00
Paul Duffin
4dcf65951b Add analyze_bcpf.py script to help manage bootclasspath_fragments
The analyze_bcpf.py script can be used by bootclasspath_fragment owners
to help setup their fragment and also to debug issues that arise during
use.

It is run by specifying the name of the bootclasspath_fragment module
in the --bcpf option. If specified the --apex and --sdk options will
allow the tool to produce more helpful messages.

The script currently:
* Builds out/soong/hiddenapi/hiddenapi-stub-flags.txt and checks any
  inconsistencies between those flags and the flags generated for the
  bootclasspath_fragment.

* Builds out/soong/hiddenapi/hiddenapi-flags.csv and checks any
  inconsistencies between those flags and the flags generated for the
  bootclasspath_fragment. It will describe the changes that need to
  be made in order to fix any inconsistencies in the flags.

Bug: 202154151
Test: m analyze_bcpf && analyze_bcpf --bcpf art-bootclasspath-fragment
      atest --host analyze_bcpf_test
Change-Id: I824fd9d527d246dfe56725c073d89a0b0e72d25a
2022-03-30 12:18:56 +01:00
Colin Cross
0f2a718ce4 Merge changes from topic "musl_clang_runtimes"
* changes:
  Enable sanitizers for musl
  Switch to libunwind to remove dependency on libgcc for musl
2022-03-30 03:54:38 +00:00
Treehugger Robot
0cfe1f43f7 Merge "android_system_image only packages "system" items" 2022-03-30 02:58:58 +00:00
Cole Faust
d9932ad053 Always build mk2rbc/rbcrun
Currently, mk2rbc/rbcrun aren't built in certain phases
of the build, which causes the Starlark product configuration
to fail when using get_build_var if a regular build hasn't
been run before.

Bug: 226677850
Test: Manually
Change-Id: If85608eef63be25fddc4d916c82247c13eb0195f
2022-03-29 14:55:17 -07:00
Colin Cross
ed12a04a3e Enable sanitizers for musl
Now that we provide runtimes built for musl, enable the sanitizers
when targeting musl.

Bug: 215802826
Test: m USE_HOST_MUSL=true host-native
Change-Id: Id17513ee305274874c31e9c99ce4faeff4a1c057
2022-03-29 13:35:17 -07:00
Colin Cross
441898c285 Switch to libunwind to remove dependency on libgcc for musl
The libgcc dependency is only used for _Unwind* symbols, remove
it now that libc_musl.so embeds libunwind.a.

Bug: 215802826
Test: m USE_HOST_MUSL=true host-native
Change-Id: I1ad4c1435fc0090c6df2d50d542d2203dfb94d7b
2022-03-29 13:35:17 -07:00
Rob Seymour
9e5cc8a674 Add pre-built attribute to host module definition for fake snapshot.
The host-snapshot provides host tools via prebuilt modules.
Having 2 prebuilts of the same module in a source tree leads to a build
time error.   Add new attribute when building the fake snapshot to
indicate that the host tool contains a prebuilt version.

When installing the host snapshot the user can opt not to include
modules that have a prebuilt version.

Bug: 225890931
Test: m HOST_FAKE_SNAPSHOT_ENABLE=true host-fake-snapshot dist
Change-Id: I044a92a280536f9c5ec93dcb277a5e5568cc4e42
2022-03-29 16:08:43 +00:00
Ram Muthiah
ac38c0bb93 Merge changes from topic "refactor-packagingbase"
* changes:
  Move GatherPackagingSpecs out of CopyDepsToZip
  Keep 'partition' in PackagingSpec
2022-03-29 13:45:22 +00:00
Ivan Lozano
72cbf5888b Merge "rust: rust_proc_macro host snapshot support." 2022-03-29 13:29:24 +00:00
Jooyung Han
0fbbc2b0d4 android_system_image only packages "system" items
android_system_image filers packaging items installed outside "system"
partition.

Some packaging items install related items to different partitions but
putting them altogether to android_system_image doesn't make sense.
(android_system_image is suppposed to be "system" partition)

To be specific, this filters out "apex" partition items.  "apex"
partition is used by APEX installation to install APEX contents to paths
similar to activated paths on device so that symbol lookup works well
with APEX contents.

Bug: 225121718
Test: atest MicrodroidHostTestCases
Test: debugfs <intermediate>/microdroid.img -R 'ls system'
  shows no "com.android.runtime"
Change-Id: Ibc3d85ead2fda99e231132ce8ab9ccf1cc9317b7
2022-03-29 07:56:02 +09:00
Treehugger Robot
4242115d59 Merge "Add values to json module graph" 2022-03-28 20:25:43 +00:00
Colin Cross
edd46702e1 Merge "Use turbine instead of kapt for kotlin annotation processors" 2022-03-28 18:07:14 +00:00
Colin Cross
e311abcd44 Merge "Don't use ziptime or --temp_dir for turbine" 2022-03-28 18:00:49 +00:00
Jooyung Han
76c255f45a Merge "Revert "Add a tool to list contents of .img file"" 2022-03-28 09:48:10 +00:00
Jooyung Han
1df464a90b Revert "Add a tool to list contents of .img file"
Revert submission 2041990-microdroid-contents

Reason for revert: b/227144320 (broken build)
Reverted Changes:
Ie2f471d39:Add the golden list of microdroid contents
I7c4fca184:Add a tool to list contents of .img file

Change-Id: If2b5d6cb1325cb17ef2bdc4f13ded003ab2dd6d7
2022-03-28 09:44:33 +00:00
Jooyung Han
a883428293 Move GatherPackagingSpecs out of CopyDepsToZip
This gives a PackageModule a chance to filter/customize the contents of
resulting package.

Bug: 225121718
Test: m (no changes)
Change-Id: I45505e8234dff42201dc40d4f038e7b08eea89f0
2022-03-28 14:29:14 +09:00
Jooyung Han
99c5fe6a38 Keep 'partition' in PackagingSpec
Add a property 'partition' to PackagingSpec and fill it when
PackagingSpec is created.

This will enable PackageModule to filter PackgingSpec based on
partition. (Will be used in a follow-up change)

Bug: 225121718
Test: m (nothing changes)
Change-Id: I7657edd380e215df66bae57d8b07dd4181841fdf
2022-03-28 14:29:14 +09:00
Treehugger Robot
23ce5b4ca2 Merge "Add a tool to list contents of .img file" 2022-03-28 05:06:55 +00:00
Treehugger Robot
6876d5aa5f Merge "Dedupe libraries in both static and whole static" 2022-03-26 00:32:01 +00:00
Treehugger Robot
1c2d87ba83 Merge "Add metrics collection to soong_build" 2022-03-26 00:21:21 +00:00
Liz Kammer
898e076d6f Add values to json module graph
Increases size of json module graph but it still remains smaller than
action information.

Test: m json-module-graph
Change-Id: I215e92afb0f5d3d688e5e590517392d0cc534455
2022-03-25 18:30:49 -04:00
Chris Parsons
715b08fd82 Add metrics collection to soong_build
Metrics are added to a materialized protobuf soong_build_metrics.pb

Recorded events have scoped names and recorded runtimes.
For example, mixedbuilds includes the following events:

mixedbuilds.prepare
mixedbuilds.analyze.write_files
mixedbuild     (this is a total of all mixedbuild events)

Test: Manually verified materialized protos for mixedbuild, bp2build,
and legacy build.

Change-Id: I2e72d4502c0373bee0279bb1320b2e4dd586129c
2022-03-25 17:19:44 -04:00
Mitch Phillips
c6ea944e18 Merge "[HWASan] Enable zero-initialization." 2022-03-25 20:53:02 +00:00
Colin Cross
f61766e987 Use turbine instead of kapt for kotlin annotation processors
Follow Bazel by using turbine instead of kapt to run annotation
processors.  This still requires using kapt to generate java stubs
of kotlin soruces, then uses turbine to run annotation processors
on the java stubs and any java sources to generate sources and
resources, and passes the annotation processor generated sources
to kotlinc and javac.

Bug: 225013372
Test: m checkbuild
Test: TestKapt
Change-Id: I9c6fc496a9fba64658bb062538bc5f7b9478b07a
2022-03-25 10:28:50 -07:00
Colin Cross
411647e8f4 Don't use ziptime or --temp_dir for turbine
Turbine sets dates to 2010 and doesn't need ziptime any more, and
ignores the --temp_dir flag.

Bug: 225013372
Test: m checkbuild
Change-Id: I6122f27d873ca671050e4dc0d65641bb52e90e99
2022-03-25 10:26:47 -07:00
Spandan Das
183ed1044c Merge "Do not propagate strict_updatability linting to conscrypt and framework-media java_sdk_libraries" 2022-03-25 15:05:06 +00:00
Trevor Radcliffe
dadc037ef3 Merge "Allow appending artifact in dist with product name" 2022-03-25 14:34:23 +00:00
Ivan Lozano
872d579a22 rust: rust_proc_macro host snapshot support.
Adds support for capturing rust_proc_macros as part of the host
snapshot. Proc macros target the host and can be thought of as compiler
plugins. Because of this, they don't have vendor image variants and
can't be easily captured as part of the vendor snapshot. Instead we
capture them as part of the host snapshot.

This adds a rust_prebuilt_proc_macro module type.

Bug: 204304380
Test: m HOST_FAKE_SNAPSHOT_ENABLE=true host-fake-snapshot dist
Test: python3 development/vendor_snapshot/update.py --image=host
  --install-dir=vendor/vendor_name/ 31 --local out/dist
Test: Checked Android.bp for rust_prebuilt_proc_macro modules.
Change-Id: I4a8c4d9c41b7ca361b5b97d3f74973918c2a5fe3
2022-03-25 09:25:07 -04:00
Yi Kong
846a25912d Merge "Update to clang-r450784" 2022-03-25 05:49:48 +00:00
Yi Kong
75ce5d3320 Update to clang-r450784
Test: presubmit
Bug: 219872355
Change-Id: I3cd5f42d887b07929906e08d7d9f1a677b8a4403
2022-03-25 04:30:53 +00:00
Spandan Das
2cf278e75b Do not propagate strict_updatability linting to conscrypt
and framework-media java_sdk_libraries

Tmp fix for
https://android-build.googleplex.com/builds/quarterdeck?branch=aosp-master&target=errorprone&lkgb=8352129&lkbb=8353967&fkbb=8352538

Test: m
out/soong/.intermediates/external/conscrypt/conscrypt.module.public.api.stubs.system/android_common/lint/lint-baseline.xml
Test: (internal CI target) https://android-build.googleplex.com/builds/abtd/run/L67700000953708519
Bug: 182349282
Bug: 226637002
Bug: 226608044

Change-Id: I17e540877200f59bfd51e69e57d8bcca6cf3850f
2022-03-25 02:31:20 +00:00
Jooyung Han
b3a3a2893b Add a tool to list contents of .img file
list_image <img> lists the contents of the file.

For example:
  $ m microdroid
  $ list_image <path to microdroid.img>
  ./
  ./apex/
  ./bin
  ..

Bug: 195425111
Bug: 225121718
Test: list_image .img
Change-Id: I7c4fca184751ba20066fd25fa4c366a955dbabf6
2022-03-25 02:44:36 +09:00
Spandan Das
ad40fffa64 Merge changes I4c373b29,I9ccda6fc,I1b390b0e,I30151217
* changes:
  Do not propagate strict updatability linting to libcore/
  Create allowlist to skip strict updatability lint check
  Propagate strict_updatability_linting to transitive deps of updatable apexes
  Export LintDepSetsIntf from java package
2022-03-24 16:38:20 +00:00
Trevor Radcliffe
90727f4576 Allow appending artifact in dist with product name
Some modules need to have their artifacts copied to dist with the
product name appended. This CL enables that functionality in a
boolean Soong property called append_artifact_with_product.

Fixes: 224561567
Test: Unit tests and build relevant target/modules
Change-Id: I4b824d4001679cebf0a9059be2d090d33a310933
2022-03-24 14:46:18 +00:00
Andrei-Valentin Onea
b47e89f2d1 Merge "Revert "Prevent unspecified values in soong_config_string_variable"" 2022-03-24 14:11:18 +00:00