Commit graph

66362 commits

Author SHA1 Message Date
Spandan Das
06f3c9bb42 Merge changes from topic "go_bp2build" into main
* changes:
  Respect package boundaries in bp2build conversion of go modules
  Create a temporary denylist for go binaries used in mixed builds
  Partial bp2build conversion of blueprint_go_binary
  Partial bp2build conversion of bootstratp_go_package
2023-07-13 16:16:00 +00:00
Alix
b1e5c6a69a Bp2build for errorprone modules that manually enabled/disabled it
Test: go test ./bp2build
Change-Id: Ie60c0959ee9ae8ce86c11a8e85a0bc7592f63df8
2023-07-13 15:50:18 +00:00
Ayush Budhwani
62efa1d650 Add unconverted_reasons_from_heuristics fields to the Module message
A heuristics is implemented in the build/bazel/scripts/bp2build_progress/bp2build_progress.py script to determine the reason for why a module is not converted.

Test: b build //build/bazel/scripts/bp2build_progress:bp2build_progress

Change-Id: I9d4eaf309dbf26bbb6de18e1af0d9cdc8fe09e94
2023-07-13 13:31:09 +00:00
Igor Foox
ae2e23a4ba Merge "Remove openwrt modules from genrule denylist." into main 2023-07-13 13:29:13 +00:00
Cole Faust
bbe2cc6191 Cleanup genrule_sandbox_test.py
In preparation for adding the ability to check all genrules in the tree.

Bug: 290816499
Test: ./build/soong/tests/genrule_sandbox_test.py extensions_db.pb
Change-Id: I6454be7e88a876a2ab1105c04f5ecda03ecd7771
2023-07-12 18:01:32 -07:00
Spandan Das
0a8a27500e Respect package boundaries in bp2build conversion of go modules
bp2build's codegen context does not implement
BazelPathConversionContext. To reuse the utility function
transformPackagePaths, update its signature

(Also make deps of go_library unique to make the conversion resilient)

Test: go test ./bp2build
Change-Id: I126b1057d2b26bc6c7d3be2780f1b62d28323cf0
2023-07-13 00:22:26 +00:00
Spandan Das
69afa98fbd Create a temporary denylist for go binaries used in mixed builds
This allows us to rollout building _some_ go targets using rules_go
without affecting mixed builds.

Test: Presubmit
Bug: 284483729
Change-Id: I0ccb4c9b90614369147a380f44f7ae372ef9396e
2023-07-13 00:22:26 +00:00
Colin Cross
654f176379 Merge "Use depsets for transitive manifests and assets" into main 2023-07-12 21:49:13 +00:00
Vadim Spivak
7a2e48bd36 Merge "AIDEGen: Collect apt generated sources" into main 2023-07-12 21:33:59 +00:00
Alix
c0dac52975 Make errorprone a configurable attribute for bazel conversion
Change-Id: Icf2265e9f712c3255321456e977928163696dc22
Test: ./bp22build tests in child commit
2023-07-12 18:18:15 +00:00
Fabián Cañas
3875e1c582 Merge "Make clang debug level configurable" into main 2023-07-12 17:58:33 +00:00
Vadim Spivak
3c496f00fa AIDEGen: Collect apt generated sources
The Java compiler may generate sources when processing annotations.
These sources are currently not included in the IDE project model, which
results in unresolved symbols when using Hilt and other annotation
processors. These sources are not passed to the Java compiler for
compilation, so they cannot be handled like other Java module sources.

This change archives the generated sources and appends them to the
existing srcjar list in module_bp_java_deps.json.

Test: go test .
Test: Verified that module_bp_java_deps.json contained srcjar entry
Test:   with path/to/android_common/javac/anno.srcjar.
Test: Verified that IntelliJ can resolve generated symbols after
Test:   invoking aidegen.
Change-Id: I6a3011f9140fd25f785178baee0cd6074ad94600
2023-07-12 17:34:39 +00:00
Treehugger Robot
105005bfd4 Merge "Remove metalava --no-banner argument" into main 2023-07-12 17:08:42 +00:00
Trevor Radcliffe
d71487173e Update name of blocklist feature in bp2build
Bug: 290908194
Test: Forrest
Change-Id: I0576fb644edb6b6043df9cf01b508ee887f946f1
2023-07-12 13:45:10 +00:00
Mårten Kongstad
bd2624442b Remove metalava --no-banner argument
metalava no longer prints an ASCII banner, and has removed its
--no-banner argument. Update all call sites accordingly.

Test: presubmit
Bug: 286023667
Change-Id: I61cf5f1125dfbd93496e6a59eb1ea62886b32046
2023-07-12 14:07:01 +02:00
Spandan Das
de623294fe Partial bp2build conversion of blueprint_go_binary
This module type does not implement android.Module, and therefore we
cannot write a conventional bp2build converter for this module type.
Instead this has been special-cased inside bp2build/build_conversion.go.

There is one major deviation between Soong and Bazel's
go_binary/go_library. Soong
collects the deps in the transitve closure and puts them on compile/link
paths. Bazel OTOH, requires the direct imports to be listed in deps of the binary
explicitly (AFAIK). Since bp2build cannot determine the list of direct
imports from the list of transitive deps, put all the transitive deps in
`deps`

Test: unit tests
Test: TH
Bug: 284483729
Change-Id: I004aaf8607fef1697a0d9e7d018ad657b67778ac
2023-07-11 22:03:48 +00:00
Spandan Das
ea2abba3a9 Partial bp2build conversion of bootstratp_go_package
This module type does not implement android.Module, and therefore we
cannot write a conventional bp2build converter for this module type.
Instead this has been special-cased inside bp2build/build_conversion.go.

Because of the above, we also do not have access to useful functions
available in the ctx object of ConvertWithBp2build. This includes
1. Finding the package (directory) of a dep. This requires getting a
   handle of the underlying module from its name (string). To solve, we
   do a pre-visit to collect this information. This did not increase the
   wall time. On my machine, `m bp2build --skip-soong-tests` takes ~14s
   before and after this CL
2. Converting srcs to labels. This requires glob and package boundary
   resolution. This CL introduces a partial implementation for this
   function. (glob patterns are not used in go tools)

For (1), I considered creating a `ModuleFromName` on
`blueprint.Context` instead of a pre-run, but this increased the time to ~27s.

Test: unit tests
Test: TH
Bug: 284483729

Change-Id: Ifeb029103d14947352556dba295591dd7038b090
2023-07-11 22:03:26 +00:00
Liz Kammer
e3c11d0aad Merge "Ensure data is passed to all generation actions" into main 2023-07-11 21:05:32 +00:00
Jiakai Zhang
b898c112a2 Merge changes I0918f2fa,I3fc6ff91,I7adc97cb,I674a5fa1,I22c45cbf into main
* changes:
  Change the profile path on host.
  Extract duplicate code to common helper functions.
  Fix dumpOatRules.
  Remove Modules() from BootclasspathFragmentApexContentInfo.
  Fix some tests for dexpreopt and remove unnecessary tests.
2023-07-11 20:59:07 +00:00
Colin Cross
120b55a7ca Merge "Add tests for code that will be affected by resource processor feature" into main 2023-07-11 20:17:05 +00:00
Wei Li
1ab0564e65 Merge "Add integration test for m build SBOM of APEXs/APKs." into main 2023-07-11 18:25:39 +00:00
Treehugger Robot
71f5a63267 Merge "Turn on Global ThinLTO by default" into main 2023-07-11 17:42:12 +00:00
Treehugger Robot
da7973112a Merge "Experiment with not using ninja weights" into main 2023-07-11 17:42:06 +00:00
Jiakai Zhang
3444e561dc Change the profile path on host.
This is to decouple profile generation from image configs. A boot image
profile is either for a mainline module or for the platform, which is
orthogonal to boot images.

Bug: 290583827
Test: m nothing
Change-Id: I0918f2fa945a2af1839f1a08ecede331c64d0317
2023-07-11 17:59:24 +01:00
Jiakai Zhang
b1639db53f Extract duplicate code to common helper functions.
Also, fall back to using a default name for the dexpreopt directory if
we are not building for Android.

Bug: 290583827
Test: m nothing
Change-Id: I3fc6ff9142a2dcdf995796f75891b242fe2848d0
2023-07-11 17:59:05 +01:00
Colin Cross
4c90f99cb3 Use depsets for transitive manifests and assets
Instead of rolling assets and manifests up through each static lib,
provide them as a DepSet.  This will help with the next patch, which
needs to pass all the transitive manifests and R.txt files together.

Test: app_test.go
Change-Id: Id8b3aa2bed3771e82ab6bde192c9b43baa38b54c
2023-07-11 09:54:59 -07:00
Jiakai Zhang
da47d3c6f3 Fix dumpOatRules.
This method generates a build rule that runs oatdump for debugging
purposes.

- Pass "--runtime-arg -Xgc:CMC" to oatdump if UFFD GC is enabled.
- Build the phony name from the image name to avoid conflicts when using
  this method for multiple boot images.

Bug: 290583827
Test: m dump-oat-boot
Change-Id: I7adc97cb85b571486b78f173fb80ff7da2c4bfe4
2023-07-11 16:27:01 +01:00
Jiakai Zhang
f7f782c4f1 Remove Modules() from BootclasspathFragmentApexContentInfo.
The Modules() method was for the ART boot image in the ART APEX. It's no
longer needed because we don't install the ART boot image in the ART
APEX anymore.

Bug: 290583827
Test: m nothing
Change-Id: I674a5fa1ed18908413a64129df5947c7d26e638e
2023-07-11 15:29:17 +01:00
Jiakai Zhang
b69e89559f Fix some tests for dexpreopt and remove unnecessary tests.
This CL is to prepare for the changes in http://r.android.com/2652081.
See the description of that CL for the reasons.

Some tests were set up in a fragile way and were easy to be broken when
the implementation changes. This CL is a pure test change that fixes
them.

This CL also removes TestNoUpdatableJarsInBootImage. That test checks
what should go to the ART boot image and what should go to the
platform one, which does not apply today because boot images are not
associated with mainline modules anymore: a boot image may take jars
from multiple mainline modules and the platform, and a mainline module
can contribute to multiple boot images. In practice, we have ART jars in
the platform boot image, and we are going to add core-icu4j and
consrypt to the ART boot image, which is now for testing only.

Bug: 290583827
Test: m nothing
Change-Id: I22c45cbf6f853b030b68edb51197854e9c53a02e
2023-07-11 15:29:12 +01:00
Romain Jobredeaux
aa634fdbdb Merge "Support target and max sdk version in app bp2build." into main 2023-07-11 13:34:29 +00:00
Liz Kammer
796921d211 Ensure data is passed to all generation actions
Test: go test genrule tests
Test: build/soong/tests/genrule_sandbox_test.py \
      framework-cppstream-protos framework-javastream-protos
Change-Id: I16fa939280c7ff0f230a0a8cf18af5e55f16eb03
2023-07-11 08:27:51 -04:00
Yi Kong
add6375a10 Turn on Global ThinLTO by default
Build C/C++ targets with ThinLTO with "--lto-O0" by default. This takes
advantage of better dead code elimination and CFG simplification due to
ThinLTO's global view of the program, but do not enable the costly LTO
optimizations. This also makes builds faster because we can avoid doing
codegen for the dead code.

Code size for panther-userdebug:
                   /system/bin  /system/lib  /system/lib64
Original           57648        86264        181700
Global ThinLTO     55764        84916        175728
                   -3.27%       -1.56%       -3.29%

Build time for aosp_arm64-userdebug:
                   clang time   linker time
Original           56993.87s    1712.36s
Global ThinLTO     52839.18s    3114.93s

saving 4.68% build time in total.

Bug: 169004486
Test: presubmit
Change-Id: Idb0f4675ca4750b8b12b24f4679579419d3448c8
2023-07-11 21:26:52 +09:00
Aditya Kumar
9488eb82d4 Merge "gsimple-template-names" into main 2023-07-11 05:54:11 +00:00
Treehugger Robot
5af242249d Merge "Copy imported APKs to output filename" into main 2023-07-11 04:52:46 +00:00
Igor Foox
24739676c7 Remove openwrt modules from genrule denylist.
With the other change in this topic,
`OpenwrtControlServerProto_h` and
`OpenwrtControlServerProto_cc` both successfully build with sandboxing
on.

Test: build/soong/tests/genrule_sandbox_test.py --show-diff OpenwrtControlServerProto_h OpenwrtControlServerProto_cc
Test: GENRULE_SANDBOXING=true m OpenwrtControlServerProto_h OpenwrtControlServerProto_cc
Test: GENRULE_SANDBOXING=true m droid
Change-Id: Id56824ed935e1d16d2333e5a1aeac248cdfcaeb6
2023-07-10 23:15:43 -04:00
Igor Foox
921a465919 Merge "Update genrule denylist to make m droid build." into main 2023-07-10 22:51:44 +00:00
Wei Li
59df0ee442 Add integration test for m build SBOM of APEXs/APKs.
Bug: 266726655
Test: build/soong/tests/sbom_test.sh
Change-Id: Id8002042f607f8e158f561ca47e0a84a8b8927e2
2023-07-10 15:50:38 -07:00
Colin Cross
5780d57a71 Copy imported APKs to output filename
android_app_import and android_test_import modules with processed: true
and skip_preprocessed_apk_checks are used directly from the source tree.
If the source file name doesn t match the module name and the file is
used as test data it can result in the wrong filename being used.  Copy
the source file to an output file with the correct name first.

Bug: 290376750
Test: m CtsAppSecurityHostTestCases
Change-Id: I4a6dd1c5e48db7085ea41035def31f0844948a46
2023-07-10 22:18:01 +00:00
Fabián Cañas
bc10544dec Make clang debug level configurable
The -g flag is controllable with CLANG_DEFAULT_DEBUG_LEVEL from -g0 to
-g3. The default remains -g

Test: Build with CLANG_DEFAULT_DEBUG_LEVEL=debug_level_1

Change-Id: I913d3a0cb028484f9496a7e0a2298852f9b699cd
2023-07-10 20:49:22 +00:00
Igor Foox
04ff5f90ae Update genrule denylist to make m droid build.
Test: lunch aosp_cf_x86_phone && GENRULE_SANDBOXING=true m droid
Change-Id: I9092df9a407ffe9a7f1c12f18fb0ce1c5fa027a1
2023-07-10 14:58:51 -04:00
AdityaK
423e4ce9e3 gsimple-template-names
Based on: https://chromium-review.googlesource.com/c/chromium/src/+/3988987

```
Savings are almost 330M on 11G

du -s out.diff/target/product/vsoc_arm64/symbols
10751692        out.diff/target/product/vsoc_arm64/symbols
du -s out/target/product/vsoc_arm64/symbols
11086508        out/target/product/vsoc_arm64/symbols
```

Change-Id: I49a5c4cadc77cb0d97e680e95c4efa7156a42f13
2023-07-10 17:36:22 +00:00
Romain Jobredeaux
496660d38a Support target and max sdk version in app bp2build.
Test: Unit tests + CI
Bug: 285304294
Bug: 280465047
Change-Id: I9bb1b6f9729ad783a43837d77dcf2d02b50ca77e
2023-07-10 11:50:00 -04:00
Elliott Hughes
ff96fe1589 Merge "Remove -fstrict-aliasing from arm32." into main 2023-07-10 14:37:39 +00:00
Liz Kammer
b492843c3f Handle static binary repetition of system deps
Test: b build `bmod toybox-static`
Test: go test bp2build tests
Change-Id: Id728f6fd08832a4fc153f0ff5282cdfb6b19c2f6
2023-07-10 10:28:02 -04:00
Treehugger Robot
ab8ed75b6b Merge "kzip build: pipe error message to stderr." into main 2023-07-10 11:45:09 +00:00
Jingwen Chen
ce08efd395 kzip build: pipe error message to stderr.
This was going to stdout so it wasn't showing up on build_error.log.

Test: n/a
Change-Id: I58ac08961b089d06720a307ece3398594b3456de
2023-07-10 08:18:58 +00:00
Treehugger Robot
b2ace8d72d Merge "Implement OutputFileProducer for android_app_import and android_test_import" into main 2023-07-08 00:36:53 +00:00
Liz Kammer
8822ff5193 Experiment with not using ninja weights
Test: CI
Change-Id: Id611059dba35aaaebc7cf787381669c4294ceb53
2023-07-07 18:31:39 -04:00
Treehugger Robot
00042a4875 Merge "rust: Allow any host tool for custom_bindgen" into main 2023-07-07 22:10:25 +00:00
Elliott Hughes
9bbe96068d Remove -fstrict-aliasing from arm32.
Our global default is -fno-strict-aliasing (including for 32-bit x86),
so let's avoid a source of arm32-only bugs.

Bug: http://b/31600139
Test: treehugger
Change-Id: Iaf3069200b2969718593387dc5b756c574a5d68a
2023-07-07 19:45:24 +00:00