Adds a new --key_field option to merge_csv.py which specifies the name
of the field that should be used to sort the input. If specified it
causes that field to be the first in each row and performs the merge
operation of a merge sort on the input files. That assumes that each
input file is already sorted into the same order.
Modifies the rules that use merge_csv.py to pass in:
--key_field signature
to sort the rows by signature.
Bug: 180387396
Test: Verified that hiddenapi files (both aggregated ones and for the
individual modules) are not affected by this change other than
changing the order.
Change-Id: Idcd5f0fea373b520b604889e1c280f21ed495660
Adds the hiddenapi_additional_annotations to allow a library to list
the libraries that provided additional hiddenapi related annotations
for a library.
Modifies merge_csv.py so it can process multiple zip files at the same
time and uses that to merge the embedded .uau files from a module and
those it depends upon.
Bug: 180102243
Test: m droid
Verified that hiddenapi files (both aggregated ones and for the
individual modules) are not affected by this change.
Change-Id: I796520021c7357398a9e2a09f1029e4a578b05b3
This is needed first for ART to enable statsd metrics reporting, but
this will be needed by more modules as they become unbundled.
Test: build-mainline-modules.sh
Bug: 178500083
Change-Id: I5af0b8db7a24c86ffd3f34732ac747a4fc6a7a55
Also, creates a python_binary_host module for generate_hiddenapi_lists
and uses that when constructing the build rule rather than using the
file directly.
Bug: 177317659
Test: m droid
Verified that hiddenapi files (both aggregated ones and for the
individual modules) are not affected by this change.
Change-Id: Ia11bb203ce5a74740d35f1b7e86716e15aad336e
Running find on the entire module directory can lead to flaky failures
when a directory is modified by another rule while find is traversing
it. Pass a list of libraries in the module into the script instead
of finding them on disk.
Fixes: 178509901
Test: TARGET_BUILD_APPS=com.android.runtime dist
Change-Id: Ie4bf288f0e3d99cbbed5ca8770687a9bbe5377e4
When we start switching vanilla platform builds to use ART prebuilts,
this script should still build from sources.
Test: TH
Bug: 172480615
Change-Id: I04cb61b484855d22f22fd9b82f6abb30eb5665c3
Revert submission 1535006-bap
Reason for revert: Looks like it breaks the build as aapt2 depends on the build_number.txt this script was generating. Error log below.
Reverted Changes:
I0f6b34310:Delete build-aml-prebuilts.sh
I4eae4153a:Replace use of build-aml-prebuilts
Change-Id: I7f5f1b3047f3d71eb8ac7bec95c5916425d5d8f2
FAILED: ninja: 'out/aml/soong/build_number.txt', needed by 'out/aml/soong/.intermediates/frameworks/base/tools/aapt2/aapt2/linux_glibc_x86_64/unstripped/aapt2', missing and no known rule to make it
This reverts commit 556b2ad77e.
Original change-id: Ib1b2f0dd2f9ae85b1545c6cc5bb4c5bbdfac1c15
Reason for revert: Add excutable permission to the gen_ndk_backedby_apex.sh.
Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only
Change-Id: Ib587ba200cd7f2f61d478452b43c329d72b2de06
Add gen_ndk_usedby_apex.sh script to generate NDK API list used by Mainlain modules when modules get build.
Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only
Change-Id: Ib1b2f0dd2f9ae85b1545c6cc5bb4c5bbdfac1c15
Dexpreopt and boot jars package check all require access to dex
implementation jars created for java_library and java_sdk_library. They
were available when building from source but not when building from
prebuilts, even though they are embedded within the .apex files that
are referenced from prebuilt_apex.
This changes adds support to prebuilt_apex to export the dex
implementation jars and updates java_import to use those exported dex
implementation jars.
In a source build dexpreopt/boot jars package check access the apex (or
platform) specific variant of a java_library, e.g. core-oj, from which
it retrieves the dex implementation jar path.
After this change in a prebuilt build dexpreopt/boot jars package check
behave in the same way except in this case they retrieve the dex
implementation jar path from the apex (or platform) specific variant of
the java_import, e.g. core-oj.
The work to export files from a `.apex` file for use by other modules
is performed by a new `deapexer` module type. It is not used directly
in an `Android.bp` file but instead is created implicitly by
`prebuilt_apex`,
In order to do that this contains the following changes:
* Adds a new `dexapexer` module type to handle the exporting of files
from the `.apex` file.
* Adds an exported_java_libs property to prebuilt_apex to specify the
set of libraries whose dex implementation jars need exporting.
* Creates apex specific variants of the libraries listed in the
exported_java_libs property.
* Adds the set of exported files to the ApexInfo to make them available
to the apex specific variants.
* Prevents the prebuilt_apex variants from being merged together as
they will not be compatible.
* Modifies java_import to use the exported file for variants of a
prebuilt_apex.
* Adds a ninja rule to unpack (using deapexer) the contents of the
prebuilt_apex's apex file, verify that the required files are present
and make them available as outputs for other rules to use.
* Some minor refactorings to support these changes.
* Adds tests to cover prebuilt only, prebuilt with source preferred,
and prebuilt preferred with source.
Test: m nothing
Bug: 171061220
Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
Its usecase can now be supported natively by soong_ui via the
--skip-kati flag + the mainline_sdk product. Migrate
build-mainline-modules.sh to use this new approach, and delete the
script.
Bug: 174315599
Test: build/soong/scripts/build-mainline-modules.sh
Change-Id: I0f6b34310df51ddf060b2912d43487f4d37dd01b
manifest_fixer.py sets a lib's targetSdkVersion to '15' if it is
not explicitly set. But ManifestMerger will add dangerous
permissions CALL_LOG_READ/CALL_LOG_WRITE if the following
conditions are met:
- The app's targetSdkVersion is >16
- A linked lib's targetSdkVersion is <16
- The app has CONTACTS_READ/CONTACTS_WRITE permissions
This condition is triggered for some (OEM) app builds (of
Settings). Update manifest_fixer.py to use '16' as its default
targetSdkVersion so that manifest_merge doesn't add these
dangerous permissions.
Change-Id: Id5f41f7da98c190c8d145fceba0085cf473f4c7a
Without this Soong may run into inconsistencies due to stale prebuilts.
This amends https://r.android.com/1529658.
Test: TH build of NDK
Bug: 175918603
Bug: 172480615
Change-Id: I14680e4547d7d5f560d31ef61b71af8d159c4c53
Should fix build errors on mainline_modules on aosp-master. This script
still does not propagate all options to aapt2 correctly, but that will
be fixed by removing the script altogether (b/174315599).
Test: build/soong/scripts/build-aml-prebuilts.sh art-module-sdk \
art-module-test-exports conscrypt-module-host-exports \
conscrypt-module-sdk conscrypt-module-test-exports \
i18n-module-host-exports i18n-module-sdk \
i18n-module-test-exports platform-mainline-sdk \
platform-mainline-test-exports runtime-module-host-exports \
runtime-module-sdk
(the failing command in
build/soong/scripts/build-mainline-modules.sh)
Bug: 175386777
Change-Id: I81aa20b1abd177f0d2886ab9b04831437da72f2a
In platform builds missing libraries are defined with make variable
INTERNAL_PLATFORM_MISSING_USES_LIBRARIES via soong_config.mk. When Soong
is invoked with "--skip-make" parameter,
INTERNAL_PLATFORM_MISSING_USES_LIBRARIES is not picked up. As a result
the build fails because of missing dependencies.
This CL uses get_build_var to get the value of
INTERNAL_PLATFORM_MISSING_USES_LIBRARIES, then converts it to JSON list.
In the future would be better to migrate both scripts to use --skip-kati
introduced in https://r.android.com/1512613.
Bug: 175286760
Bug: 132357300
Test: forrest build for target "ndk" branch "aosp-master".
Test: Patch build-aml-prebuilts.sh and build-ndk-prebuilts.sh to dump
their soong.variables config, observe that MissingUsesLibraries is
coherent with INTERNAL_PLATFORM_MISSING_USES_LIBRARIES. Start running
each script and observe that they don't fail early due to missing
dependencies.
Change-Id: I5dbc1d3ea7a64de1e8be6332003acc940c2e6a76
Necessary only when platform start using i18n prebuilts, but this helps
CI testing.
Test: build/soong/scripts/build-mainline-modules.sh
Bug: 172480615
Change-Id: Ic24f934ad0b4b7ac7c38d791f80df33315b936e9
Add 'append' command for conv_linker_config so any item can be appended
to linker.config.pb file.
Bug: 174646355
Test: buildbot_build.sh succeeded
Change-Id: Ic3f12ee5e5c0e7ca7c27d4a8cff347e93061eb5f
Add gen_ndk_usedby_apex.sh script to generate NDK API list used by Mainlain modules when modules get build.
Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only
Forrest run: android-build.googleplex.com/builds/forrest/run/L97800000742565481
Change-Id: I39e5aa7c74eff46aba06808642b2bd67ae45ba1e
List of stub libraries from system (and bionic APEX) is currently
managed in static file. This change generates the list at build time and
adds to the system config so linkerconfig can use it.
Bug: 172889962
Test: Build cuttlefish and confirmed list is generated in
/system/etc/linker.config.pb
Change-Id: Ie0400e9d1098b0de19c6a7c25f261384aadde0a3
The xml output from dexdump is more stable than the text output so this
change switches to use that.
Bug: 171479578
Test: m check-boot-jars - for failing and passing cases
Change-Id: Ie347b1b685951524d5ae09db280248c61b4228ee
Necessary for some build steps. aapt2 is one example, although it still
has problems because part of its build logic is in make.
This isn't strictly required for any supported build target at the
moment, but the error when it fails can be confusing so let's add it
anyway.
Test: build/soong/scripts/build-aml-prebuilts.sh art-module-sdk \
art-module-test-exports conscrypt-module-sdk \
conscrypt-module-test-exports conscrypt-module-host-exports \
runtime-module-sdk runtime-module-host-exports \
i18n-module-test-exports i18n-module-sdk platform-mainline-sdk \
platform-mainline-test-exports
Bug: 172548608
Change-Id: I88526a85f288287c3a138f8cbb6fd04ad28c3795
The switch to use dex jars instead of class jars means that a boot jar
that is defined by a dex_import module will now be checked against the
package_allowed_list.txt so it is possible that it will detect
previously unreported problems.
Test: m check-boot-jars - for failing and passing cases
Bug: 171479578
Bug: 125517186
Change-Id: Ie614898dade0fb43c9418d7afb9138169db6f097
Filter out lines starts with "//" from json file to allow simple
comments on the contents. Original json format does not support
comments, but this reduces readability compared to txt file or other
formats. This change allows simple comments on the linker.config.json to
give more information on the contents.
Test: parse succeeded with commented contents
Change-Id: I1c734bf9a054f81f57aa2aea1038d0041297acf1
Adds a singleton that traverses the module variants finding the ones
that are in the list (updatable and non-updatable) of boot jars and
add a ninja rule to ensure that they only contain packages from an
allowed list.
Replaces a hack that ignored any prebuilt boot jars supplied as dex
file with an equivalent one to ensure that they are still ignored.
A follow up change that switches to checking dex jars will allow the
hack to be removed.
The boot jars check can be strict or lax. If strict then all the boot
jars listed in the configuration must be found, otherwise it will only
check the ones it finds. It is strict by default unless
TARGET_BUILD_UNBUNDLED=true or ALLOW_MISSING_DEPENDENCIES=true.
Moves the script and data file from build/make.
Test: m check-boot-jars - for failing and passing cases
SKIP_BOOT_JARS_CHECK=true - no check-boot-jars target created
ALLOW_MISSING_DEPENDENCIES=true - not strict
TARGET_BUILD_UNBUNDLED=true - not strict
verified manually that apart from path differences the same
files (same check sum) were checked in both old make checks and
the new Soong ones
EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true m check-boot-jars
Bug: 171479578
Change-Id: I9d81d6650ba64fc0d48d2dab4ba5a3ba8dd03dec
construct_context.py is a script that runs at Ninja stage and constructs
class loader context arguments for dex2oat. Previously it accepted lists
of library paths and concatenated them into one class loader context
string. The script also handled the dependency of "android.hidl.manager"
on "android.hidl.base" that is not tracked by the build system and must
be handled in a special way.
Now that class loader context representation is going to change from
flat lists to trees, passing paths to individual libraries as script
arguments is no longer possible, because a list of paths cannot
represent a class loader context tree. Passing the trees in a serialized
form is also inconvenient, because the script would have to parse them,
which would complicate it a lot.
Therefore this patch ports all the concatenation and "android.hidl.base"
handling to Soong. It is not possible to port the remaining script to
Soong because Soong has no information about the targetSdkVersiion of
the dexpreopted apps (it is in the manifest and sometimes in an APK, and
has to be extracted at Ninja time).
Test: construct_context_test.py
Test: new subtests in TestUsesLibs
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: Icdb03cf00d1e27e4cff3844b89bfaec4de502dd7
Handle new filetype 'linker_config' which is configuration for
linkerconfig in json type and convert into protobuf in build time.
Bug: 169634881
Test: Build succeeded and cuttlefish boot succeeded
Change-Id: I56555fc738e6d6600d15a191a24f79a2ee747f52
Instead of tracking per module and per module variant, track allowed
list of dependecies for all modules combined. This avoids issues with
different products and different downstream branches having different
build graphs.
To compare allowed_deps.txt vs head, run:
:; m -j out/soong/apex/depsinfo/new-allowed-deps.txt.check
To update source allowed_deps.txt, run:
:; build/soong/scripts/update-apex-allowed-deps.sh
Bug: 149622332
Test: m
Change-Id: Ic518fbd9ebfe1b46aaf9a58df731780a7e5a676b
Merged-In: Ic518fbd9ebfe1b46aaf9a58df731780a7e5a676b
(cherry picked from commit 453555083b)
(cherry picked from commit e5207cd9a6)
Revert submission 1312796-apex-allowed-deps
Reason for revert: b/161974327
Reverted Changes:
I52a4be72e:Add a check for apex/allowed_deps.txt to droidcore...
I56771ba3f:Track allowed transitive deps in any updatable mod...
Change-Id: I3be0a1b0dd824dafeadb485daca8e58b81a3ec5c
