To build system.img in Soong, we need all artifacts including
build.prop. This fully migrates buildinfo.prop file into Soong as a
first step to build build.prop on Soong.
This fixes an error caused by an incorrect path to build thumbprint
file.
Bug: 322090587
Test: compare build.prop before and after
Test: build multiple times and see build.prop isn't rebuilt
Change-Id: Id4fa830009538856c30825ff47268b11fa6cb5d6
To build system.img in Soong, we need all artifacts including
build.prop. This fully migrates buildinfo.prop file into Soong as a
first step to build build.prop on Soong.
Bug: 322090587
Test: compare build.prop before and after
Test: build multiple times and see build.prop isn't rebuilt
Change-Id: Icaa7e1fdab2a8c169ac00949d3aaf6c8212a1872
If `include_make_built_files` is set to the name of a partition, the
make-built files from that partition will be incorperated into this
soong module. This is to ease the transition to soong built filesystems.
If any files are present in both the soong-built file list and the
make-built one, the soong ones will be preferred.
Bug: 329146343
Test: go test
Change-Id: I456b283e1189116e699ed75357cc056f5d217688
New soong module type `TestModuleConfig`
This module type allows to write:
test_module_config {
name: "DerivedFrameworks1",
base: "FrameworksServicesTests",
include_filters: ["com.android.server.accessibility.FingerprintGestureDispatcherTest"],
}
This goal is to put the test options (include_fitler in this case) in
Android.bp rather than TEST_MAPPING to be able to name a set of options
and to allow the CI to more precise about which part of a TestModule
might be failing.
In general, it is recommended to split up tests rather than use this
approach, but that isn't always feasible.
More details on project here: https://docs.google.com/document/d/1MpA4BS6mTr-1D-K9MVmzdHkSufAvIkkM4gPEUglNodk/edit?tab=t.0
Migration plan here: https://docs.google.com/document/d/1x3w-BvKO3E-GbY8YOSBuzJC6-pkLkuhfjSyZ1hJQBYM/edit?tab=t.0
Some details of final output:
* All the [data] files from Derived are the same as base.
* The apk names is changed to be the Module name
* The test options are added to .config file
* test-file-name in the .config file (for base.apk) is changed to be
new apk.
* module-info.json is hand inspected between base and derived.
* I added a .manifest file so future tools know what base is, if they
need it.
We are working a mechanism to use a CAS so the zip of the testcases
won't explode. (however the trident disk image will get bigger)
% tree -ls out/target/product/shiba/testcases/DerivedFrameworks1 out/target/product/shiba/testcases/FrameworksServicesTests
[ 4096] out/target/product/shiba/testcases/DerivedFrameworks1
├── [ 4096] arm64
│ └── [ 72346281] DerivedFrameworks1.apk
├── [ 20887] BstatsTestApp.apk
├── [ 4096] data
│ └── [ 851] broken_shortcut.xml
├── [ 3157] DerivedFrameworks1.config
├── [ 12695] JobTestApp.apk
├── [ 8599] MediaButtonReceiverHolderTestHelperApp.apk
├── [ 16791] SimpleServiceTestApp1.apk
├── [ 16791] SimpleServiceTestApp2.apk
├── [ 16791] SimpleServiceTestApp3.apk
├── [ 1017540] SuspendTestApp.apk
└── [ 36] test_module_config.manifest
[ 4096] out/target/product/shiba/testcases/FrameworksServicesTests
├── [ 4096] arm64
│ └── [ 72346281] FrameworksServicesTests.apk
├── [ 20887] BstatsTestApp.apk
├── [ 4096] data
│ └── [ 851] broken_shortcut.xml
├── [ 2866] FrameworksServicesTests.config
├── [ 12695] JobTestApp.apk
├── [ 8599] MediaButtonReceiverHolderTestHelperApp.apk
├── [ 16791] SimpleServiceTestApp1.apk
├── [ 16791] SimpleServiceTestApp2.apk
├── [ 16791] SimpleServiceTestApp3.apk
└── [ 1017540] SuspendTestApp.apk
Fixes: 314148134
Test: atest DerivedFrameworks1 DerivedFrameworks2
Test: m blueprint_tests
Change-Id: Ib73a3404557e0bd583b065f0cf2fd55fba9ccdbe
How it works:
1. build/make/core/Makefile generates a txt file with the kernel
version, which is taken from an explicit BOARD_KERNEL_VERSION value,
or extracted from the kernel image on the source tree, or extracted
from the kernel image extracted from the prebuilt boot.img.
The file is saved at
$ANDROID_PRODUCT_OUT/obj/PACKAGING/check_vintf_all_intermediates/kernel_version.txt.
2. If PRODUCT_ENABLE_UFFD_GC is "default", meaning the GC type needs to
be determined by the kernel version, build/make/core/Makefile copies
kernel_version.txt to
out/soong/dexpreopt/kernel_version_for_uffd_gc.txt.
3. build/soong/dexpreopt/config.go writes the the UFFD GC flag to
out/soong/dexpreopt/uffd_gc_flag.txt. The flag is determined by an
explicit PRODUCT_ENABLE_UFFD_GC value or by contruct_uffd_gc_flag.py,
which reads kernel_version_for_uffd_gc.txt and determines the flag
accordingly.
4. dex2oat takes the UFFD GC flag from uffd_gc_flag.txt.
5. post_process_props.py mangles ro.dalvik.vm.enable_uffd_gc based on
the same logic.
Bug: 321751629
Bug: 319554951
Bug: 318763448
Bug: 319648491
Test: m --no-skip-soong-tests nothing
Test: atest uffd_gc_utils_test
Test: Build with `OVERRIDE_ENABLE_UFFD_GC=default m` for device with no
UFFD support -
1. Check the existence of `-Xgc:CMC` in
out/soong/dexpreopt_arm64/dex_bootjars/android/system/framework/arm64/boot.invocation
(dex2oat invocation for a boot image)
2. Check the existence of `-Xgc:CMC` in
out/soong/.intermediates/packages/apps/Settings/Settings/android_common/dexpreopt/Settings/oat/arm64/package.invocation
(dex2oat invocation for an app defined in .bp)
3. Check the existence of `-Xgc:CMC` in
$ANDROID_PRODUCT_OUT/obj/APPS/Dialer_intermediates/oat/arm64/package.invocation
(dex2oat invocation for an app defined in .mk)
4. Check the value of ro.dalvik.vm.enable_uffd_gc in
$ANDROID_PRODUCT_OUT/product/etc/build.prop
Test: Build with `OVERRIDE_ENABLE_UFFD_GC=default m` for device with
UFFD support, and do the steps above.
Test: Build with `OVERRIDE_ENABLE_UFFD_GC=true m`, and do the steps
above.
Test: Build with `OVERRIDE_ENABLE_UFFD_GC=false m`, and do the steps
above.
Change-Id: I035ad32233e49e2a30ce11f6c7c318a648511ef8
Setting CC and CXX to point to our clang prebuilts on mac fails to build with:
_cgo_export.c:3:10: fatal error: 'stdlib.h' file not found
Skip setting CC and CXX on mac to go back to using the host linker.
Change-Id: I9bab4cda961874c381eacc34397cc67f62fd4730
Test: treehugger
* changes:
Add script to run Soong tests with go tools
Disable TestVariantSingletonModule when go test -short is used
Fix data race in propagateRROEnforcementMutator
Fix data race in finder_test.go
Fix data race in dex_bootjars
Fix race CommonGlobalCflags when running tests in parallel.
Fix data race in snapshot singletons when running parallel tests
Add go test -race -short to run-soong-tests-with-go-tools.sh to make
the tests run in presubmit with the race detector enabled.
Test: presubmit
Change-Id: I3162a46ebef965879915c0b534821d922392f1f8
Move the logic that runs Soong tests with go tools from
prebuilts/build-tools/build-prebuilts.sh to
build/soong/scripts/run-soong-tests-with-go-tools.sh.
Test: build/soong/scripts/run-soong-tests-with-go-tools.sh
Change-Id: Ice6532a1bdc53291d791af0c4001406fa47c9537
Throwing a RuntimeError leads to a stack trace in the build failure
error message, which is not useful and makes the error harder to
understand.
Bug: 320698986
Test: Presubmits
Change-Id: I0b79185fff7b9c8d4085b23936b80b3d026a4384
`keep-flagged-apis` module generates a txt file that
contains annotations based on aconfig flags that are passed to metalava
when generating "exportable" or "runtime" stubs.
`keep-flagged-apis.sh` already exists under
`packages/modules/SdkExtensions` directory, but modifying the script
(and the bp module definition) will break the `udc-mainline-prod` branch
in downstream. Therefore, the modified script and the module definition
is defined here, and will be used in the long term.
Test: m keep-flagged-apis && manual
Bug: 315485740
Change-Id: I4ab8d4ea2c6c8ddba6d55b2b25e8c8d3b7c8fa07
Trunk stable builds require a TARGET_RELEASE to be set, but this
variable doesn't have a real meaning for NDK builds. So we
hard-core it here so individual build setups don't need to
configure it.
Bug: 309900087
Test: Treehugger
Change-Id: I17a2e463db0b561aea57417bc2b469070ba2062d
Linkerconfig's contribution feature was introduced in Android T, but
it's not used actively (probably, never used?). Let's deprecate it. Note
that its runtime behavior is kept to support any potential uses.
Bug: 309716069
Test: linkerconfig_diff_test
Test: Add "contributions" field to an apex, build it.
It should fail with an error message.
Change-Id: Icb91919142de14696219308674f1b3ac3e2d029b
`create_minidebuginfo` can fail when processing objects with compressed
debuginfo. To work around this we can use `llvm-objcopy` to decompress
the debug sections prior to processing.
Test: m
Bug: 305277519
Change-Id: Ia149a81f01b0be974f395732c713fb23d92ac22f
`create_minidebuginfo` can fail when processing objects with compressed
debuginfo. To work around this we can use `llvm-objcopy` to decompress
the debug sections prior to processing.
Test: m
Bug: 305277519
Change-Id: I0b3c1f08dfbcb7f8ebea2607ed3e22d901e63ecd
Make generates manifests for modules that are missing AndroidManifest.xml
files in order to paper over differences between aapt and aapt2. These
manifests don't have an <application> tag. which was tripping up an
unexercised error path in manifest_check.py.
Change manifest_check,py's extract_uses_libs_xml to consider manifests
with no application tag as having empty uses_library and
optional_uses_library lists. Also fix up a few nearby pylint warnings.
Fixes: 303554426
Test: m ConnectivityMonitorRobotests
Change-Id: I3815a1c86e24d7b3dea7f4ea26c34d3af182ded1
There was an issue that a library installed in the system with stub is
manually marked as requireLibs, so it created link to the self (system)
namespace from linkerconfig. This change checks from conv_linker_config
if there is any common module in provideLibs and requireLibs before
write result to output path.
Bug: 298333253
Test: Build failed when the common module is not removed.
Change-Id: I855dfc3484bb6c1fec24d498703c4a2a805913c0
This reverts commit e7c71c344d.
The intermediates created by the separated rust compile steps are very
large, on the order of 60GB. This is more than CI can support for now,
revert the split into compile and link phases. This comes at the expense
of incremental build time, especially when modifying C/C++ sources that
are dependencies of rust modules.
Bug: 293349612
Test: builds
Change-Id: I81169e7d0727330c8de5e7688dcdc87fe7b8d3b5
Also add script to remove the Bazel output base.
This will assist with supporting movable checkouts alongside
mixed builds.
Bug: 259191764
Test: m && (move topic and prepare_moved_top.sh) && m
Test: m && prepare_moved_top.sh && m
Test: build/soong/tests/relative_symlinks_test.sh
Change-Id: I0f53da8d99f752fad496cf3ac61b01f001b7296d
`conv_linker_config validate` command is used to validate the linker
configuration embedded in APEX to detect common mistakes.
For example, when used in APEX, linker configuration can't set
provideLibs/requireLibs. For APEX, there are
provideSharedLibs/requireSharedLibs in APEX manifest for that purpose.
One might make mistake by setting provideLibs in linker config.
Now, when these unsupported properties are set, there'll be build-time
error like:
// set provideLibs key in com.android.art's linker config.
$ m com.android.art
...image.apex/etc/linker.config.pb: provideLibs is set. Use provideSharedLibs in apex_manifest
Bug: 264341796
Test: m com.android.art (see above)
Change-Id: Ibaf7322616ad333569e6d721680f3d72243402a2
So that we don't erroneously add skip_preprocessed_apk_checks
everywhere.
Bug: 185811447
Test: Presubmits
Change-Id: Icb1c6163d170ca4181c5c6a814b51fda4777746f
When targetSdk is >= 30, the system verifies that you use a valid
signature V2+ certificate. Uncompressing ndk/dex files or aligning
the zip file will break a signature V2, so these apks should really
just set preprocessed: true.
Fixes: 185811447
Test: Presubmits
Change-Id: Id89c42bcd5b5daa6eda1716bff4023423298036b
- Don't pass `--as-needed` to the linker on Mac OS which is unsupported
there
- Use `--force_load` rather than `--Wl,--whole-archive` on Mac OS
- Scan `rustc`'s linker arguments for `-dylib` and `-dynamiclib`, which
it can use instead of `-shared` on Mac OS:
7bd81ee190/compiler/rustc_codegen_ssa/src/back/linker.rs (L319)
Test: m libhalf serde_derive
Bug: 291164566
Change-Id: Iecd6c2532fa31c9476834f49b109de98cbd2dccf
Regarding aosp/2624311, `blkid_path` was removed from the command, so the number of parameters should be reduced by 1.
Test: MODULE_BUILD_FROM_SOURCE=false m (in internal-main branch)
Bug: 279858383
Change-Id: Iad494b757e2a4152a6867098c3eb7419c3bba71b
deapexer doesn't need it.
Bug: 279858383
Test: presubmit
Test: m MODULE_BUILD_FROM_SOURCE=false droid
Change-Id: Ie8311c74682f366cf6aeebb669500d01675cd0bf
Before this change, dexpreopt was often broken with optional libraries.
This was because the CLC construction was done in Soong at an early
stage, where we don't have sufficient information to determine whether
an optional library is installed or not.
For example, the "Settings" package uses an optional library called
"androidx.window.extensions". On some devices, the library is installed,
but on some other devices, it's not. Soong always adds the library to
the CLC, meaning the CLC is wrong for devices which don't have the
library. This change fixes the problem. See the tests below.
After this change, the CLC construction is done by a Python script
invoked at a very late stage. It uses product_packages.txt, which is
generated by Make, to determine whether an optional library is
installed or not, and filter out libraries that are not installed.
Note that optional libraries are still added as dependencies by Soong.
This is because dependencies have to be added at an early stage. This
means what dex2oat eventually uses will be a subset of the dependencies,
which is fine.
Bug: 282877248
Test: m
Test: atest construct_context_test
Test: -
1. lunch aosp_cf_x86_64_phone-userdebug && m
2. Check the .invocation file of the "Settings" package (defined in
.bp file)
3. See androidx.window.extensions
Test: -
1. lunch aosp_redfin-userdebug && m
2. Check the .invocation file of the "Settings" package (defined in
.bp file)
3. Don't see androidx.window.extensions
Test: Check the .invocation file of the "Dialer" package (defined in
.mk file)
Test: -
1. Build a Pixel 5 system image and flash it to a Pixel 5 device.
2. adb shell pm art dump
3. See "reason=prebuilt" instead of "reason=vdex".
(https://diff.googleplex.com/#key=fB6Ls9q2QGSN, before: left,
after: right)
Change-Id: Ia112bd7c2328373e68db6bffb74bf34030f683d8
It was inlined into the usage in the other cl in this topic.
Bug: 280685526
Test: ./build/bazel/ci/rbc_dashboard.py --quick aosp_arm64-userdebug
Change-Id: I68af9186e17a996baf4676d97cdc9895abcb7e06
This script will be used to change the package that is being allowed to
use privileged permissions.
Test: python scripts/modify_permissions_allowlist.py
Bug: 242509786
Change-Id: I1d7cb3c906ad39aef42ad4394953251db40d1bf1
Teach rustc to generate object files instead of a linked executable
for binary and shared library crates. This lets us skip the lengthy
Rust compilation phase when only the C++ dependencies of these crates
have changed.
This works using a Python script that replaces the linker invoked by
rustc and converts the linker command line into a rspfile that can be
used during the link phase. The script makes certain assumptions about
how rustc invokes the linker, but since we control the version of Rust
we use, we can update the script if those assumptions ever break. I've
asked the Rust developers about adding an official way to do this so
hopefully we won't need to rely on this hack in the future.
The rustc rule for Kythe has been altered to pass linker=true, somewhat
consistent with the main rustc rule, which now doesn't invoke the actual
linker either. `m xref_rust` still succeeds, but hopefully we can find
someone who knows more about the rest of the Kythe pipeline who can let
us know whether it should still work. Otherwise, let's use this CL as
an opportunity to find out whether the rest of the pipeline is happy
with linker=true, since otherwise we will find it hard to maintain the
Kythe rule in the future.
Change-Id: Ifb9c8689c1bd78c0afcf6fdfd111140d20377536
```
ERROR: /buildbot/src/android/master/out/soong/workspace/external/libcxx/BUILD.bazel:30:18: While resolving toolchains for target //external/libcxx:libc++: com.google.devtools.build.lib.packages.NoSuchTargetException: no such target '@soong_injection//product_config_platforms/products/ndk-eng:ndk-eng_android_riscv64': target 'ndk-eng_android_riscv64' not declared in package 'product_config_platforms/products/ndk-eng' defined by /buildbot/src/android/master/out/bazel/output/external/soong_injection/product_config_platforms/products/ndk-eng/BUILD (Tip: use `query "@soong_injection//product_config_platforms/products/ndk-eng:*"` to see all the targets in that package)
```
Bug: https://github.com/google/android-riscv64/issues/10
Bug: http://b/274509603
Bug: http://b/262192655
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I4b6633ab5aab3f01953b628d78ac1d79f2a4ee3c
These will only be provisional, but there are enough different groups
that want *something* to make a start on porting that it's probably
time...
Ignore missing dependencies in build-ndk-prebuilts.sh for now, because
various pieces (that the NDK doesn't care about) are still missing.
Bug: http://b/273792258
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I39202a953a73dc449a10a887d497d09079c43402
This will enable its bp2build conversion, and is needed for java_system_modules support in Bazel
Change-Id: I4f3ff5e36c8cd7f78efbb42f641efb2f76a8b71d
Bug: 215230098
to cover the recent features.
- conv_linker_config proto with empty input
- conv_linker_config proto with existing output
- conv_linker_config proto with --append
- conv_linker_config proto with --force
Bug: n/a
Test: conv_linker_config_test
Change-Id: I0de79b6e05c2608e0e2f30dfbf04d8289672f362
