Commit graph

455 commits

Author SHA1 Message Date
Greg Kaiser
13596d20af Merge "ndk: Hard-code the TARGET_RELEASE" into main 2023-11-10 19:59:07 +00:00
Greg Kaiser
a0b356f6b0 ndk: Hard-code the TARGET_RELEASE
Trunk stable builds require a TARGET_RELEASE to be set, but this
variable doesn't have a real meaning for NDK builds.  So we
hard-core it here so individual build setups don't need to
configure it.

Bug: 309900087
Test: Treehugger
Change-Id: I17a2e463db0b561aea57417bc2b469070ba2062d
2023-11-09 12:43:38 -07:00
Jooyung Han
eb421ee86e Deprecate 'contributions' field from linkerconfig
Linkerconfig's contribution feature was introduced in Android T, but
it's not used actively (probably, never used?). Let's deprecate it. Note
that its runtime behavior is kept to support any potential uses.

Bug: 309716069
Test: linkerconfig_diff_test
Test: Add "contributions" field to an apex, build it.
      It should fail with an error message.
Change-Id: Icb91919142de14696219308674f1b3ac3e2d029b
2023-11-08 13:38:36 +09:00
Almaz Mingaleev
773ec2744a Add java.util.random to the allowed package list.
This is part of OpenJDK17 update.

Bug: 282678719
Bug: 270674727
Bug: 308103782

Test: m
Change-Id: I9be31ef257ee94bd731220ba31b74ebfb0562fd3
2023-10-31 16:05:52 +00:00
Eric Rahm
eba99f4ab1 Decompress debug sections when creating debug info
`create_minidebuginfo` can fail when processing objects with compressed
debuginfo. To work around this we can use `llvm-objcopy` to decompress
the debug sections prior to processing.

Test: m
Bug: 305277519
Change-Id: Ia149a81f01b0be974f395732c713fb23d92ac22f
2023-10-20 16:01:10 +00:00
Matías Hernández
77833b7502 Revert "Decompress debug sections when creating debug info"
Revert submission 2788316

Reason for revert: DroidMonitor-triggered revert due to breakage https://android-build.corp.google.com/builds/quarterdeck?branch=git_main&target=panther-next-userdebug&lkgb=10981485&lkbb=10981536&fkbb=10981536, bug b/306625613

Reverted changes: /q/submissionid:2788316

Change-Id: Idebeff995fa2cd6dd23cd4a2675c9f3c7f8307d9
BUG: 306625613
2023-10-20 13:07:14 +00:00
Eric Rahm
6f4c50ea9d Decompress debug sections when creating debug info
`create_minidebuginfo` can fail when processing objects with compressed
debuginfo. To work around this we can use `llvm-objcopy` to decompress
the debug sections prior to processing.

Test: m
Bug: 305277519
Change-Id: I0b3c1f08dfbcb7f8ebea2607ed3e22d901e63ecd
2023-10-19 22:24:58 +00:00
Colin Cross
c00fa152f7 Fix manifest_check.py for generated manifests that have no application tag
Make generates manifests for modules that are missing AndroidManifest.xml
files in order to paper over differences between aapt and aapt2.  These
manifests don't have an <application> tag. which was tripping up an
unexercised error path in manifest_check.py.

Change manifest_check,py's extract_uses_libs_xml to consider manifests
with no application tag as having empty uses_library and
optional_uses_library lists.  Also fix up a few nearby pylint warnings.

Fixes: 303554426
Test: m ConnectivityMonitorRobotests
Change-Id: I3815a1c86e24d7b3dea7f4ea26c34d3af182ded1
2023-10-06 13:24:38 -07:00
Kiyoung Kim
3df5f50e1a Do not allow a module exists both in provideLibs and requireLibs
There was an issue that a library installed in the system with stub is
manually marked as requireLibs, so it created link to the self (system)
namespace from linkerconfig. This change checks from conv_linker_config
if there is any common module in provideLibs and requireLibs before
write result to output path.

Bug: 298333253
Test: Build failed when the common module is not removed.
Change-Id: I855dfc3484bb6c1fec24d498703c4a2a805913c0
2023-10-05 10:27:43 +09:00
Colin Cross
004bd3f526 Revert "Split Rust crate builds into compile and link phases."
This reverts commit e7c71c344d.
The intermediates created by the separated rust compile steps are very
large, on the order of 60GB.  This is more than CI can support for now,
revert the split into compile and link phases.  This comes at the expense
of incremental build time, especially when modifying C/C++ sources that
are dependencies of rust modules.

Bug: 293349612
Test: builds
Change-Id: I81169e7d0727330c8de5e7688dcdc87fe7b8d3b5
2023-10-02 22:15:55 -07:00
Cole Faust
3835509f95 Merge "Error out if skip_preprocessed_apk_checks is set when it's not necessary" into main 2023-09-13 22:38:33 +00:00
Sebastian Pickl
3d2b08fe0d Merge "Revert "Change symlink_forest to use relative symlinks."" into main 2023-09-12 18:53:47 +00:00
Sebastian Pickl
90355f79bf Revert "Change symlink_forest to use relative symlinks."
Revert submission 2673616-relativesymlinks-fix

Reason for revert: this breaks tests verified with go/abtd https://android-build.googleplex.com/builds/abtd/run/L36000000963001181

Bug: 300122962

Reverted changes: /q/submissionid:2673616-relativesymlinks-fix

Change-Id: I5a97c4fbe4df5727c0604a07137093d0f00c7776
2023-09-12 18:51:39 +00:00
Mark Dacek
4eabebec69 Merge "Change symlink_forest to use relative symlinks." into main 2023-09-12 15:47:42 +00:00
MarkDacek
b98b3a429f Change symlink_forest to use relative symlinks.
Also add script to remove the Bazel output base.
This will assist with supporting movable checkouts alongside
mixed builds.

Bug: 259191764
Test: m && (move topic and prepare_moved_top.sh) && m
Test: m && prepare_moved_top.sh && m
Test: build/soong/tests/relative_symlinks_test.sh

Change-Id: I0f53da8d99f752fad496cf3ac61b01f001b7296d
2023-09-12 04:02:23 +00:00
Jooyung Han
4bc102672a apex: invoke conv_linker_config validate as validation
`conv_linker_config validate` command is used to validate the linker
configuration embedded in APEX to detect common mistakes.

For example, when used in APEX, linker configuration can't set
provideLibs/requireLibs. For APEX, there are
provideSharedLibs/requireSharedLibs in APEX manifest for that purpose.

One might make mistake by setting provideLibs in linker config.
Now, when these unsupported properties are set, there'll be build-time
error like:

 // set provideLibs key in com.android.art's linker config.
 $ m com.android.art
   ...image.apex/etc/linker.config.pb: provideLibs is set. Use provideSharedLibs in apex_manifest

Bug: 264341796
Test: m com.android.art (see above)
Change-Id: Ibaf7322616ad333569e6d721680f3d72243402a2
2023-09-08 15:40:12 +09:00
Cole Faust
51d7bfd9a1 Revert^2 "Only allow setting presigned without preprocessed on targetSdk < 30"
This reverts commit d293e28f52.

Reason for revert: The underlying issue was fixed in ag/24685010

Change-Id: I06810d37dba37aa12f9a1e14b0749f1e1eb41136
2023-09-07 05:31:32 +00:00
Karl Shaffer
d293e28f52 Revert "Only allow setting presigned without preprocessed on targetSdk < 30"
This reverts commit 6158528e15.

Reason for revert: DroidMonitor-triggered revert due to breakage https://android-build.corp.google.com/builds/quarterdeck?branch=git_udc-d1-dev-plus-aosp&target=aosp_bramble-trunk_staging-userdebug&lkgb=10771573&lkbb=10771600&fkbb=10771587, bug https://buganizer.corp.google.com/issues/299369971

BUG: 299369971

Change-Id: I6bf6eb5c0fb9e30197e145121adc7ed58871526f
2023-09-07 00:51:09 +00:00
Cole Faust
9c5c09f0ad Error out if skip_preprocessed_apk_checks is set when it's not necessary
So that we don't erroneously add skip_preprocessed_apk_checks
everywhere.

Bug: 185811447
Test: Presubmits
Change-Id: Icb1c6163d170ca4181c5c6a814b51fda4777746f
2023-09-06 16:11:44 -07:00
Cole Faust
6158528e15 Only allow setting presigned without preprocessed on targetSdk < 30
When targetSdk is >= 30, the system verifies that you use a valid
signature V2+ certificate. Uncompressing ndk/dex files or aligning
the zip file will break a signature V2, so these apks should really
just set preprocessed: true.

Fixes: 185811447
Test: Presubmits
Change-Id: Id89c42bcd5b5daa6eda1716bff4023423298036b
2023-08-29 11:18:44 -07:00
A. Cody Schuffelen
f29ca58e88 Darwin/Mac OS host rust compilation fixes
- Don't pass `--as-needed` to the linker on Mac OS which is unsupported
  there
- Use `--force_load` rather than `--Wl,--whole-archive` on Mac OS
- Scan `rustc`'s linker arguments for `-dylib` and `-dynamiclib`, which
  it can use instead of `-shared` on Mac OS:
  7bd81ee190/compiler/rustc_codegen_ssa/src/back/linker.rs (L319)

Test: m libhalf serde_derive
Bug: 291164566
Change-Id: Iecd6c2532fa31c9476834f49b109de98cbd2dccf
2023-08-03 16:26:44 -07:00
Alyssa Ketpreechasawat
ac6c853ee6 Fix the number of required parameters in deapexer script.
Regarding aosp/2624311, `blkid_path` was removed from the command, so the number of parameters should be reduced by 1.

Test: MODULE_BUILD_FROM_SOURCE=false m (in internal-main branch)
Bug: 279858383
Change-Id: Iad494b757e2a4152a6867098c3eb7419c3bba71b
2023-07-14 17:26:13 +00:00
Jooyung Han
6c762092ee Remove --blkid_path argument
deapexer doesn't need it.

Bug: 279858383
Test: presubmit
Test: m MODULE_BUILD_FROM_SOURCE=false droid
Change-Id: Ie8311c74682f366cf6aeebb669500d01675cd0bf
2023-06-14 15:23:03 +09:00
Jiakai Zhang
a449678996 Move CLC construction to Ninja phase.
Before this change, dexpreopt was often broken with optional libraries.
This was because the CLC construction was done in Soong at an early
stage, where we don't have sufficient information to determine whether
an optional library is installed or not.

For example, the "Settings" package uses an optional library called
"androidx.window.extensions". On some devices, the library is installed,
but on some other devices, it's not. Soong always adds the library to
the CLC, meaning the CLC is wrong for devices which don't have the
library. This change fixes the problem. See the tests below.

After this change, the CLC construction is done by a Python script
invoked at a very late stage. It uses product_packages.txt, which is
generated by Make, to determine whether an optional library is
installed or not, and filter out libraries that are not installed.

Note that optional libraries are still added as dependencies by Soong.
This is because dependencies have to be added at an early stage. This
means what dex2oat eventually uses will be a subset of the dependencies,
which is fine.

Bug: 282877248
Test: m
Test: atest construct_context_test
Test: -
  1. lunch aosp_cf_x86_64_phone-userdebug && m
  2. Check the .invocation file of the "Settings" package (defined in
     .bp file)
  3. See androidx.window.extensions
Test: -
  1. lunch aosp_redfin-userdebug && m
  2. Check the .invocation file of the "Settings" package (defined in
     .bp file)
  3. Don't see androidx.window.extensions
Test: Check the .invocation file of the "Dialer" package (defined in
  .mk file)
Test: -
  1. Build a Pixel 5 system image and flash it to a Pixel 5 device.
  2. adb shell pm art dump
  3. See "reason=prebuilt" instead of "reason=vdex".
     (https://diff.googleplex.com/#key=fB6Ls9q2QGSN, before: left,
     after: right)

Change-Id: Ia112bd7c2328373e68db6bffb74bf34030f683d8
2023-05-30 15:46:38 +01:00
Cole Faust
8f065e6c5d Delete rbc-run script
It was inlined into the usage in the other cl in this topic.

Bug: 280685526
Test: ./build/bazel/ci/rbc_dashboard.py --quick aosp_arm64-userdebug
Change-Id: I68af9186e17a996baf4676d97cdc9895abcb7e06
2023-05-04 18:02:22 -07:00
Almaz Mingaleev
cb97a76ad4 Merge "Add jdk.internal.access to the allowed package list." 2023-05-04 18:01:13 +00:00
Almaz Mingaleev
2429776555 Add jdk.internal.access to the allowed package list.
This is part of OpenJDK17 update.

Bug: 270674727
Test: m
Change-Id: I0708ee286cd7c3c6eef593784affaeffbdf0a76c
2023-05-03 07:21:27 +00:00
Andrei Onea
580f7b0094 Add script for modifying privapp permission allowlists
This script will be used to change the package that is being allowed to
use privileged permissions.

Test: python scripts/modify_permissions_allowlist.py
Bug: 242509786
Change-Id: I1d7cb3c906ad39aef42ad4394953251db40d1bf1
2023-05-02 15:10:16 -04:00
Treehugger Robot
f5cd477841 Merge "Split Rust crate builds into compile and link phases." 2023-04-13 22:45:02 +00:00
Peter Collingbourne
e7c71c344d Split Rust crate builds into compile and link phases.
Teach rustc to generate object files instead of a linked executable
for binary and shared library crates. This lets us skip the lengthy
Rust compilation phase when only the C++ dependencies of these crates
have changed.

This works using a Python script that replaces the linker invoked by
rustc and converts the linker command line into a rspfile that can be
used during the link phase. The script makes certain assumptions about
how rustc invokes the linker, but since we control the version of Rust
we use, we can update the script if those assumptions ever break. I've
asked the Rust developers about adding an official way to do this so
hopefully we won't need to rely on this hack in the future.

The rustc rule for Kythe has been altered to pass linker=true, somewhat
consistent with the main rustc rule, which now doesn't invoke the actual
linker either. `m xref_rust` still succeeds, but hopefully we can find
someone who knows more about the rest of the Kythe pipeline who can let
us know whether it should still work. Otherwise, let's use this CL as
an opportunity to find out whether the rest of the pipeline is happy
with linker=true, since otherwise we will find it hard to maintain the
Kythe rule in the future.

Change-Id: Ifb9c8689c1bd78c0afcf6fdfd111140d20377536
2023-04-12 15:51:41 -07:00
Jooyung Han
4b4f03fd82 Make conv_linker_config visible to linkerconfig
linkerconfig_diff_test uses conv_linker_config.

Bug: 276890767
Test: atest --host linkerconfig_diff_test
Change-Id: I2f669b83f24b9dd563fc034738fe4ba83f95fd49
2023-04-12 17:20:47 +09:00
Elliott Hughes
7d8f39e127 build-ndk-prebuilts.sh: disable bazel.
```
ERROR: /buildbot/src/android/master/out/soong/workspace/external/libcxx/BUILD.bazel:30:18: While resolving toolchains for target //external/libcxx:libc++: com.google.devtools.build.lib.packages.NoSuchTargetException: no such target '@soong_injection//product_config_platforms/products/ndk-eng:ndk-eng_android_riscv64': target 'ndk-eng_android_riscv64' not declared in package 'product_config_platforms/products/ndk-eng' defined by /buildbot/src/android/master/out/bazel/output/external/soong_injection/product_config_platforms/products/ndk-eng/BUILD (Tip: use `query "@soong_injection//product_config_platforms/products/ndk-eng:*"` to see all the targets in that package)
```

Bug: https://github.com/google/android-riscv64/issues/10
Bug: http://b/274509603
Bug: http://b/262192655
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I4b6633ab5aab3f01953b628d78ac1d79f2a4ee3c
2023-03-21 00:42:35 +00:00
Elliott Hughes
65a97364ba Merge "Generate NDK stubs for riscv64 too." 2023-03-20 21:10:53 +00:00
Elliott Hughes
f7d3109972 Generate NDK stubs for riscv64 too.
These will only be provisional, but there are enough different groups
that want *something* to make a start on porting that it's probably
time...

Ignore missing dependencies in build-ndk-prebuilts.sh for now, because
various pieces (that the NDK doesn't care about) are still missing.

Bug: http://b/273792258
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I39202a953a73dc449a10a887d497d09079c43402
2023-03-17 18:49:18 +00:00
Romain Jobredeaux
6d8716c176 Create a sh_binary target for jars-to-module-info
This will enable its bp2build conversion, and is needed for java_system_modules support in Bazel

Change-Id: I4f3ff5e36c8cd7f78efbb42f641efb2f76a8b71d
Bug: 215230098
2023-03-17 14:58:08 +00:00
Jooyung Han
fba49bbf56 Add tests for conv_linker_config
to cover the recent features.
- conv_linker_config proto with empty input
- conv_linker_config proto with existing output
- conv_linker_config proto with --append
- conv_linker_config proto with --force

Bug: n/a
Test: conv_linker_config_test
Change-Id: I0de79b6e05c2608e0e2f30dfbf04d8289672f362
2023-03-09 07:46:25 +00:00
Jooyung Han
f6fd4c2823 Fix conv_linker_config with multiple input json files
`conv_linker_config proto -s` should work with multiple json input
files, but ParseDict() overwrites list fields (e.g. provideLibs), not
appending elements.

Also added a test.

Bug: 264330513
Test: conv_linker_config_test
Change-Id: Idc482f941201f15e5fc276c0ffc0dfeaa09d0cc2
2023-03-09 07:46:00 +00:00
Jooyung Han
3397b6a2aa conv_linker_config handling existing output file
`proto` sub command now fails when the output file exists.
This is to avoid accidental overwrite.

To handle the case properly, it support --force and --append.

This is to support amending /{partition}/etc/linker.config.pb in the
build process.

Bug: 244531518
Test: manual testing
Change-Id: I0af8c83015e485f2c7533221cae8caf6143403c8
2023-03-07 21:18:17 +00:00
Treehugger Robot
6bc5e2cefa Merge "conv_linker_config proto works with empty input" 2023-03-06 18:36:10 +00:00
Jooyung Han
b531beecd5 conv_linker_config proto works with empty input
PRODUCT_VENDOR_LINKER_CONFIG_FRAGMENTS lists json files for vendor
linker config. It's annoying to handle the case of empty list.

`proto` subcommand now works for empty input. This is useful to generate
the empty linker config.

Bug: 244531518
Test: conv_linker_config proto --source --output output.pb
Change-Id: Iec6de67a979814a818730c393d9a4a7ca5d2eebe
2023-03-04 08:28:40 +09:00
Sorin Basca
93bca1585e Merge changes from topic "java17-record-support"
* changes:
  Allow java.lang.runtime in boot image
  Emit record annotation
2023-03-03 15:12:28 +00:00
Joe Onorato
442b6fc701 Remove OWNERS access for non-build team people to core build system
Test: treehugger
Change-Id: I0c318ddd7d78110327bbd849ed9aa3973c86de5d
2023-02-28 19:29:47 -08:00
Victor Chang
7c05d2d185 Allowlist java.lang.constant in libcore
Bug: 270028670
Test: m droid
Change-Id: I7612998fd518442defd5f9f00115e6a16e261b18
2023-02-20 17:27:18 +00:00
Sorin Basca
4025390468 Allow java.lang.runtime in boot image
Test: m
Change-Id: Icb208bb235f9c6ac97492c54c32a3174982756ae
2023-02-10 07:31:51 +00:00
Seth Moore
c6f4b532f3 Allow android tests to specify a mainline package name
Tests that run against mainline modules generally need to be built
twice: once for aosp (unsigned) and once for internal (signed).
The override_android_test rule is really useful for generating a
signed version of the unsigned test.

However, mainline tests need to use a MainlineTestModuleController
to only run tests on targets with the expected module. Since unsigned
and signed mainline modules have different package names, this means
the test config needs to be updated when overriding a test.

Add a new "mainline_package_name" argument so that android test rules
can update the targeted mainline package. Without this, the only way
to create a mainline package test is to instead use android_test, which
results in a lot of copy-pasta.

Test: added tests to soong
Test: manual, using a locally-modified RkpdAppUnitTest
Test: test_config_fixer_test.py
Change-Id: Idaffd63f225719a2bfda41018fda630b17db0080
2023-02-08 12:38:50 -08:00
Almaz Mingaleev
8b04d82ff1 Add jdk.internal.util.jar in the allowed package list.
This is part of OpenJDK17 update. sun.misc.JarIndex was moved to
jdk.internal.util.jar.

Bug: 260718199
Test: m
Change-Id: I7107f980d6afe500acff88164b1e6cb04cded896
2023-02-03 11:42:46 +00:00
Paul Duffin
0f386bc12b Allow obfuscated classes on bootclasspath
Previously, the signature_trie python library would reject classes
which started with a lower case character which in turn caused the
verify_overlaps tool to fail. That meant that it was impossible to
add obfuscated classes (which commonly used lower case characters
for their name) from being used on the bootclasspath.

This change removes that restriction and the accompanying test.

Bug: 265833521
Test: TH and partner testing
Change-Id: I70710484e427f64d79fb30301f3413f3b67b27e7
2023-01-18 12:59:23 +00:00
Jooyung Han
014ccd4ef0 conv_linker_config proto supports multiple input json files
conv_linker_config proto -s input1.json:input2.json generates a merged
proto output file.

Bug: 264330513
Test: conv_linker_config proto -s input1.json:input2.json
Change-Id: I4806fc606d115c42a1c745fba67960ce6db8c8d7
2023-01-09 16:26:53 +09:00
Sasha Smundak
648e32c341 Run ckati step of the Android build tracing given variables
Test: `m nothing && build/soong/scripts/run-ckati.sh FOO BAR`, see FOO
and BAR assignments in out/ckati.trace

Change-Id: Ife8bb6d716bfde5ee9f75b85b1e5738ab2238dce
2023-01-04 10:08:43 -08:00
Paul Duffin
e17c31611a Fix deapexer usage in prebuilt_apex/apex_set
A previous change to the deapexer script to check that blkid_path and
fsckerofs_path were set broke the scripts/unpack-prebuilt-apex.sh
scripts use of deapexer. This change fixes it by passing the paths
through to the script which then passes it on to deapexer.

Bug: 255963179
Bug: 240288941
Bug: 259958590
Test: m MODULE_BUILD_FROM_SOURCE=false droid
Change-Id: Ief7f36219b7fe1cf7104c08163e10d8e39f181d0
2022-12-12 17:40:26 +00:00