Commit graph

64693 commits

Author SHA1 Message Date
Jason Wu
2520f5e36b Add re_analysis environment varibles fields to soong_metrics
Test: Tested by following steps
1.m nothing: field is empty
2.USE_RBE=false m nothing: field log cc_wrapper and rbe_wrapper
3.USE_RBE=false m nothing: field is empty
Bug: 281922291

Change-Id: I1bbb324752b9a2dea1ff2c9df5817559d4cec3a6
2023-06-06 16:11:39 -04:00
Jiakai Zhang
5324cc84ad Merge "Move CLC construction to Ninja phase." 2023-05-30 14:58:15 +00:00
Jiakai Zhang
a449678996 Move CLC construction to Ninja phase.
Before this change, dexpreopt was often broken with optional libraries.
This was because the CLC construction was done in Soong at an early
stage, where we don't have sufficient information to determine whether
an optional library is installed or not.

For example, the "Settings" package uses an optional library called
"androidx.window.extensions". On some devices, the library is installed,
but on some other devices, it's not. Soong always adds the library to
the CLC, meaning the CLC is wrong for devices which don't have the
library. This change fixes the problem. See the tests below.

After this change, the CLC construction is done by a Python script
invoked at a very late stage. It uses product_packages.txt, which is
generated by Make, to determine whether an optional library is
installed or not, and filter out libraries that are not installed.

Note that optional libraries are still added as dependencies by Soong.
This is because dependencies have to be added at an early stage. This
means what dex2oat eventually uses will be a subset of the dependencies,
which is fine.

Bug: 282877248
Test: m
Test: atest construct_context_test
Test: -
  1. lunch aosp_cf_x86_64_phone-userdebug && m
  2. Check the .invocation file of the "Settings" package (defined in
     .bp file)
  3. See androidx.window.extensions
Test: -
  1. lunch aosp_redfin-userdebug && m
  2. Check the .invocation file of the "Settings" package (defined in
     .bp file)
  3. Don't see androidx.window.extensions
Test: Check the .invocation file of the "Dialer" package (defined in
  .mk file)
Test: -
  1. Build a Pixel 5 system image and flash it to a Pixel 5 device.
  2. adb shell pm art dump
  3. See "reason=prebuilt" instead of "reason=vdex".
     (https://diff.googleplex.com/#key=fB6Ls9q2QGSN, before: left,
     after: right)

Change-Id: Ia112bd7c2328373e68db6bffb74bf34030f683d8
2023-05-30 15:46:38 +01:00
Saagar Jha
204beb184a Merge "Replace prodcertstatus with gcertstatus" 2023-05-30 11:12:43 +00:00
Kiyoung Kim
5500fc89c1 Merge "Rename build.ninja with product name" 2023-05-30 07:17:08 +00:00
Jingwen Chen
30d48351be mk2rbc: temporarily bump matching paths limit to 155.
It was 150, but an internal change bumped up the usage to 153. Temporarily increase it to fix build break.

Bug: b/284854738

Change-Id: I23478f4c5333c5ef1ad344aadafe5803685c5d92
Test: abtd 
Test: treehugger
2023-05-29 16:01:34 +00:00
Alix Espino
e5641c51b9 Merge "bp2build java_resources that only contain a filegroup" 2023-05-26 13:59:42 +00:00
Treehugger Robot
8321c1a8fb Merge "Use hashed subdir for soong_config modules" 2023-05-26 12:50:42 +00:00
Ivan Lozano
979dee31e3 Merge "rust: Support sanitizers in rust_ffi modules" 2023-05-26 12:41:23 +00:00
Inseob Kim
81b00a8db7 Use hashed subdir for soong_config modules
This is to differentiate soong intermediate directories for soong config
modules. This will help incremental build across different
devices.

Test result of building panther, building cheetah, and building panther
again:

Before this change
- build time: 02:57
- # of tasks: 31044

After this change
- build time: 01:48
- # of tasks: 1694

Build time includes build.ninja generating time (which is more than 1
minute), so the overriden artifacts become far fewer.

And "NINJA_ARGS='-n -d explain' m" only gave 4 "command line changed"
nodes.

Bug: 279362051
Test: see above
Change-Id: I4891cbe823ae21628465e5c6eb26a4837ccdd202
2023-05-26 18:07:05 +09:00
Anton Hansson
0c6ae56eea Merge "ensure that privapp_allowlist is installed before android_app" 2023-05-26 07:17:37 +00:00
Wei Li
40c224656e Merge "Exclude diffs caught by sbom_test." 2023-05-26 05:42:09 +00:00
Jingwen Chen
d261085999 Merge "Add jihoonkang and mrziwang to OWNERS" 2023-05-26 02:04:55 +00:00
Saagar Jha
f513b23ac4 Replace prodcertstatus with gcertstatus
Prodaccess is apparently deprecated now.

Test: Build with/without credentials and check that it works

Change-Id: Iaadaebe84fe9531b2d7d5f0d265751462bb827d8
2023-05-26 00:23:50 +00:00
Spandan Das
85b7932892 Add jihoonkang and mrziwang to OWNERS
Test: N/A
Change-Id: I39145a7115e8e3bda4fb4cb92561742089ad2414
2023-05-26 00:10:25 +00:00
Wei Li
704146094a Exclude diffs caught by sbom_test.
Bug: 284279034
Test: build/soong/tests/sbom_test.sh
Change-Id: I152c2e2c55b6547b2dddf7b1b2524681c9af952e
2023-05-25 16:29:20 -07:00
Cory Barker
70b2292646 Merge "Add both classes jar and dexed jar to java_fuzz module packaged output for dist as we move towards instrumentation at runtime" 2023-05-25 23:26:28 +00:00
Jihoon Kang
de68c07ce5 Merge "Modify error emit condition for checking api files in java_api_library" 2023-05-25 21:09:40 +00:00
Treehugger Robot
0cf043fd48 Merge "Open metrics visibility to script subpackages" 2023-05-25 18:58:37 +00:00
Sam Delmerico
b1daccdc97 ensure that privapp_allowlist is installed before android_app
AndroidMk assumes that the app is the last file installed, and it uses
this assumption to populate the LOCAL_SOONG_INSTALLED_MODULE entry. This
CL moves the privapp_allowlist installation to before the app
installation to respect this assumption.

Bug: 242509786
Test: go test
Test: OUT_DIR=out.ref m nothing &&
  cp aosp/2562351 && OUT_DIR=out.change m nothing &&
  GOWORK=$PWD/build/bazel/mkcompare/go.work \
  go run android/bazel/mkcompare/cmd -json \
  <(sed -e "s/out\.ref/out/g" out.ref/soong/Android-aosp_cheetah.mk) \
  <(sed -e "s/out\.change/out/g" out.change/soong/Android-aosp_cheetah.mk)
  && verify manually that the only diffs are related to the removal of
  the prebuilt_etc module.
Change-Id: I95ec27070f575e79fb976de68493a219717ed89a
2023-05-25 18:54:06 +00:00
Treehugger Robot
51f09423fb Merge "build: Fix bp2build panic issue" 2023-05-25 14:47:52 +00:00
Jihoon Kang
160634ca3d Modify error emit condition for checking api files in java_api_library
java_api_library currently emits error based when the
java_api_contribution module has an empty api file or when the module
does not have any api files. This may be problematic in partial
manifests where allow missing dependency is set, thus add this condition
for error emission.

Test: go test ./java
Bug: 284212875
Change-Id: I10b4baa5723e7a53d931823b3029ea20ac731911
2023-05-25 05:28:29 +00:00
Kiyoung Kim
a37d9baa96 Rename build.ninja with product name
Current build.ninja does not contain any product name, while other ninja
files (such as combined ninja) do. This change adds product name to the
build.ninja so it can be separated over multiple lunch targets

Bug: 277029044
Test: build succeeded and checked if out/soong/build.ninja has been
renamed

Change-Id: I16dc71f829fd76f01b98da0d509a8e0ef6f62fa9
2023-05-25 10:09:45 +09:00
Cory Barker
c29c37a2c0 Add both classes jar and dexed jar to java_fuzz module packaged output for dist as we move towards instrumentation at runtime
Test: Built locally and tested pipeline
Bug: 283993246
Change-Id: I69eb98d2b7966d77ac8819348f7b99a7b18f40b6
2023-05-24 22:53:41 +00:00
Ivan Lozano
7f9d7cbb75 rust: Support sanitizers in rust_ffi modules
Don't automatically assume that HWASAN or ASAN is enabled if Fuzzer is
enabled.

Bug: 178365482
Test: m bluetooth_stack_with_facade
Test: SANITIZE_TARGET=fuzzer m bluetooth_stack_with_facade
Test: SANITIZE_TARGET=address m bluetooth_stack_with_facade
Test: SANITIZE_TARGET=hwaddress m bluetooth_stack_with_facade
Test: SANITIZE_TARGET="fuzzer address" m bluetooth_stack_with_facade
Test: SANITIZE_TARGET="fuzzer hwaddress" m bluetooth_stack_with_facade
Change-Id: Ief8c0f899837c5889a8035782616025f1b0d54e7
2023-05-24 20:15:52 +00:00
Treehugger Robot
1a608b0b0a Merge "don't require package_name for non-override android_apps" 2023-05-24 18:47:33 +00:00
Treehugger Robot
873691ae99 Merge "simplify m clean" 2023-05-24 18:36:39 +00:00
Alix
b29a3cd2aa bp2build java_resources that only contain a filegroup
supports filegroup that specifies path property

Bug: 280860624
Test: built libauto_value_plugin
Change-Id: I9ed0b13e055beb92ba8090f6b5e88b9873c9ce61
2023-05-24 18:21:15 +00:00
Chris Parsons
24e9eec800 Open metrics visibility to script subpackages
Test: N/A
Change-Id: Iee750369efa9138d0a3ada74549dde239f87396f
2023-05-24 18:07:18 +00:00
Sam Delmerico
15809f8101 don't require package_name for non-override android_apps
For a non-override android_app, we can assume that the privapp_allowlist
already contains the correct package_name, and so we don't need to
overwrite it in this case.

Bug: 242509786
Test: go test
Merged-In: I0f137e34cae3478dc8b9178d138121ff1d936f07
Change-Id: I0f137e34cae3478dc8b9178d138121ff1d936f07
2023-05-24 11:09:31 -04:00
liulvping
4d11d10ce2 build: Fix bp2build panic issue
panic in top down mutator "bp2build_conversion_bp2build"
for module "foo".
reflect: call of reflect.Value.NumField on interface Value

Test: go test ./bp2build
Change-Id: I9ee460ca38a37c6a6c1a0787159fa5f38b852b55
2023-05-24 06:38:58 +00:00
Treehugger Robot
cd4e6a609e Merge "Sandbox inputs to aidl rule in cc" 2023-05-23 23:40:35 +00:00
Kun Niu
50533df497 Merge "data_native_bins is only available in java_test_host, add java_test_host properties to java_defaults to make it's available in java_defaults as well." 2023-05-23 22:35:40 +00:00
Treehugger Robot
ea2cafe21e Merge "Run "plugins" singleton in parallel with the rest." 2023-05-23 22:01:34 +00:00
Treehugger Robot
0d149f4a4f Merge "Remove GetPythonBinary request type" 2023-05-23 21:42:31 +00:00
Usta Shrestha
6e1aa7830c simplify m clean
`rm -rf out` used to fail because bazel would not set write permission on some files. Now it's been fixed and thus there is little reason to prefer `m clean` (also users are accustomed to `rm -rf out`)

Bug: NA
Test: verified `m clean` works;interestingly soong_metrics file is created at the end
Change-Id: I000d16508613045811fc7792e5798f7c150dcc05
2023-05-23 17:34:15 -04:00
Vinh Tran
c1910c9e07 Merge "Support sandboxing genrule" 2023-05-23 21:29:27 +00:00
Treehugger Robot
8d3df9beeb Merge "Increase R8 max heap size to 4GB" 2023-05-23 21:21:53 +00:00
Vinh Tran
095819530a Sandbox inputs to aidl rule in cc
Bug: 279960133
Test: go test
Test: Remove hdrs prop from IDropBoxManagerService_aidl && run  BUILD_BROKEN_DISABLE_BAZEL=true m libservices && Expect an error from aidl
Change-Id: Ifdb260d8e2da9a5767f1e212393de4134b210616
2023-05-23 15:53:10 -04:00
Chris Parsons
6b49888a17 Remove GetPythonBinary request type
This has been dead code for over a year.

Fixes: 232976601
Test: m nothing
Change-Id: Ida305fc8fa61d8ac83d74ea2cf007073a0019cec
2023-05-23 17:57:38 +00:00
LaMont Jones
9b44c2a725 Run "plugins" singleton in parallel with the rest.
This completes the changes in aosp/q/topic:"parallel-singletons".

(cherry picked from commit 30bf7fe32d)

Bug: 281536768
Test: manual, presubmits
Change-Id: I6529e957fbcca6c18441f823a00061db7556e303
2023-05-23 17:57:00 +00:00
Kun Niu
bd0fd20bdc data_native_bins is only available in java_test_host, add java_test_host properties to java_defaults to make it's available in java_defaults as well.
Test: local build
Bug: 279622634
Change-Id: Iceaf960d587544ae934424a773a6ae3b17fb443e
2023-05-23 17:54:16 +00:00
Jared Duke
3e65f2a53c Increase R8 max heap size to 4GB
The size was increased for D8 in an earlier change, but is more
critical for R8. Match the D8 configuration.

Bug: 241220562,283888124
Test: m
Change-Id: I3b98bf6098f2196c87a97f6c8f7308d9299170d8
2023-05-23 17:32:30 +00:00
Treehugger Robot
1befe407a1 Merge "Revert "Revert "Restrict plugins to an existing allowlist""" 2023-05-23 17:08:46 +00:00
Treehugger Robot
04a1809d2a Merge "Handle missing dependencies in mixed builds" 2023-05-23 16:07:01 +00:00
Liz Kammer
c13f785061 Handle missing dependencies in mixed builds
Bazel will fail on queries with missing dependencies. Instead, we check
for missing dependencies in mixed builds and we fall back to Soong when
we are aware of missing dependencies in a module.

Test: go test soong tests
Change-Id: I4f83752704970d8b43650d5b55ff35799c7bc625
2023-05-23 12:14:02 +00:00
Jihoon Kang
4aed3703dd Merge changes Idc77d018,I7dff0094
* changes:
  Enable java_api_library modules in TxtStubLibraries.bp
  Add missing dependency in create_mock_soong
2023-05-23 11:26:53 +00:00
Treehugger Robot
634692f131 Merge changes from topic "b/278060169"
* changes:
  Unset BUILD_NUMBER/HOSTNAME in kati.go
  BUILD_NUMBER and BUILD_HOSTNAME doesn't affect kati regeneration
2023-05-23 08:56:48 +00:00
Jihoon Kang
b34bb06780 Enable java_api_library modules in TxtStubLibraries.bp
Since the stubs generation from text file feature has been merged,
enable the java_api_library modules in TxtStubLibraries.bp file.

Test: m --build-from-text-stub
Change-Id: Idc77d01882f136c4e255fb4284f9dc058fafe44d
2023-05-23 07:32:04 +00:00
Jihoon Kang
f530696762 Add missing dependency in create_mock_soong
This change adds missing dependency of Soong modules as symlink
directory in create_mock_soong so that a correct ninja graph can be
created when invoking androidmk_test.sh.

Test: build/soong/test/run_integration_tests.sh
Bug: 283905797
Change-Id: I7dff0094fdc23b7c438c917dbc00a5143a4dc5ae
2023-05-23 06:57:38 +00:00