Host tools are in out/host/linux-x86/bin now instead of
out/soong/host/linux-x86/bin.
Bug: 204136549
Test: forrest
Change-Id: I293accdde5ee4d1d21a471e12ba66e169f9410fc
Mk2rbc takes a .mk file and produces a .rbc file under the
same path, but with a different stem. For example:
build/test/foo.mk becomes out/build/test/foo.rbc
This makes it difficult to see everything that mk2rbc has
generated in the out folder. Move the generated files to
out/rbc instead, so they have a common stem that is separate
from the rest of the build outputs.
Fixes: 202249430
Test: m RBC_PRODUCT_CONFIG=1 nothing
Change-Id: If3edba3feef9c2d3631244d533b997ef0b8b4e8b
This was broken in aosp/1878498; ccross@ submitted aosp/1875758 between
writing that change and submitting it, which changed the paths to mk2rbc
and rbcrun, thereby breaking Starlark product configuration.
Test: Presubmits.
Change-Id: I185489589997e6bdc815a54a8f6f328c18f813d0
It's better not to have state hidden in dotfiles (or rather,
dotdirectories) if we can help.
It's questionable whether the "linux-x86" path segment makes sense since
soong_build only ever runs on one operating system, but I didn't want to
rock the boat now.
Drive-by fixed some quoting fixes in rbc-run. Notably, I didn't wrap
`$@` into double quotes because I don't know whether the lack of double
quotes was intended or not.
Also drive-by fixed the fact that "out/soong" was added twice to the
directory name of bpglob. This turned out not to be a problem because
bpglob doesn't need to be explicitly built: if it's needed, it's
declared as an input of the glob files so it'll be built automatically
(at the cost of the first "null build" not actually being a null build)
Test: Presubmits.
Change-Id: I710d8d16cd8212059a0ca1ee95378505303eed83
Insert ANSI escape codes in the error text. Colors are red, blue, green
and some text is in bold --- these are all bright enough on either dark
or light background.
Also, add a link to the online documentation.
Bug: 132357300
Test: manually mangle one of Android.bp files to get a manifest_check
error and observe that it is colorful.
Change-Id: I2af2aa0415d0eb0eabc88dc5504198e11bfb91b6
This is needed for adding jdk.internal.HotSpotIntrinsicCandidate as a no-op
Bug: 202495224
Test: m
Change-Id: Ib97a7fe955d920aca93630d1d5b04fedff1af960
--skip-make is deprecated because of its odd behavior around the
out/.soong.kati_enabled file. Replace it with --skip-config --soong-only
instead, which will unambigiously skip running kati.
Bug: 204136549
Test: build/soong/tests/run_integration_tests.sh
Test: build/soong/scripts/build-mainline-modules.sh
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I629ee7542687aaafd68eea50346b0cf53eafe1ef
For instance, this will match `clang-dev` when using a local build of
llvm-toolchain.
Test: run script
Change-Id: I3f2a0a6adc360f0a299def79535ce139353aece2
Previously, the signature-patterns.csv file (which was output in an sdk
snapshot) included implementation details, e.g. the names of classes
that are not part of any API, including the hidden API. This change
allows a bootclasspath_fragment module owner to customize the generated
signature patterns file to remove as many implementation details as
possible from the sdk snapshot. That means that implementation only
changes would not require that sdk snapshots be updated in sync with
the corresponding APEX.
Module owners can remove implementation in the following ways:
1) Limit the number of packages that are treated as split packages by
explicitly specifying the split packages in the split_packages
property. A split package is one whose classes are provided by
separate bootclasspath_fragment modules and so the signature
patterns has to include every class in that package provided by a
specific bootclasspath_fragment module, including implementation
classes, instead of just listing the package.
The default is to treat all packages as being split, so if no
split_packages is specified then it defaults to ["*"] which matches
all packages. Assuming that no package was split unless specifically
stated would require that all the modules that had split packages
explicitly list them before this change could be submitted as
without them this change would break the build.
Once all existing modules with split packages have been updated to
include them then the default may be changed.
2) Adding package prefixes for any hierarchy of packages that are
owned solely by that bootclasspath_fragment. This removes the need
to list the packages in that hierarchy, including implementation
specific packages.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: If2f90cfc41b6fff9fa4ac8b9d4973110938b9392
Needed for the board configuration.
rbc-run will also now pass any extra arguments it receives
to rbcrun. (The real program, not the script)
Bug: 201700692
Test: rbc_product_config.py aosp_arm64-userdebug
Change-Id: Ic9869f693f1d28bafee59050b6ae7b1fba14dbdd
The behaviour of shell parameter expansion on linux and macOS is
different:
- on linux "\/" in the replaced string is interpreted as simply "/"
- on macOS it is interpreted as the full string "\/"
For example, "android.package.example" would be changed to:
- "android/package/example" on linux
- "android\/package\/example" on macOS
The character / is not a special character for pattern matching in bash,
so it doesn't need to be escaped. Hence using / instead of \/ in the
replaced string works on both linux and macOS.
Change-Id: Id6a5cf32afc53d5ffd989c0ac8aa0b9e0fcbaf82
Fixes: 201947033
As part of the migration to OpenJDK 11, sun.reflect has moved to
jdk.internal.reflect. For now we'll have both packages available
as new code uses the jdk.internal version.
Test: m
Bug: 188889082
Bug: 190791083
Change-Id: Ib03c9c8c346daebd7d97387817b506095f5ec693
Updating OpenJDK 11 means that there is a dependency on
jdk.internal.misc, which is renamed from sun.misc. The new
package is added to the allowed list.
Bug: 190470684
Bug: 190791083
Test: m
Change-Id: I174df7bad5c5ec4286c35998e16c26bcd0737612
https://android-review.googlesource.com/c/platform/build/soong/+/1820496/1/scripts/manifest_check.py#361
caused a regression wrt to statusfile creation. Specifically it inverted
the logic used to create the status file. This file is used in
dexpreopt, thereby affecting all odex files
Test: In internal git, m <an_odex_file>. File size is restored to
pre-regression value
Bug: 199921980
Bug: 200329228
Change-Id: If1e61ac8e2b95455dacc6e7b9431c8f6e1f339b3
Test: Ran the python version of the script and the python3 version
Test: and verified the output is the same. This is for all three
Test: outputs (html, text, xml).
Test: Builds without an error.
Change-Id: I344504a4351c14e00caeb109c5507a60fb6be05a
1. Run pyformat scripts/<script>.py -s 4 --force_quote_type
none -i to fix formatting
2. Annotate #pylint: disable, where straightforward fix is not available
Test: m manifest_check_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/scripts/manifest_check.py
build/soong/scripts/manifest_check_test.py
Bug: 195738175
Change-Id: I9af498c4abd6ac9f8b9df4f93cbdd4424eacff8e
This is part of merging upstream changes from OpenJDK 11.28. This CL
adds jdk.internal.math to package_allowed_list.txt
Bug: 199067710
Test: atest CtsLibcoreOjTestCases
Change-Id: I343bed53ce6b52a6b373a75828b3234073b9cba8
1. Run pyformat scripts/check_boot_jars/check_boot_jars.py -s 4
--force_quote_type none -i to fix formatting.
3. use pylint: disable=<X> where fixes are not obvious
Test: m check_boot_jars
Test: pylint --rcfile tools/repohooks/tools/pylintrc
scripts/check_boot_jars/check_boot_jars.py
Bug: 195738175
Change-Id: I0f86d95e0d174c9f87a2872dedaa65cd3e814246
There is nothing in this script that is not python3, so
make it python3. Without this, a system without python will
fail when trying to run gdbclient.py.
Test: Ran gdbclient.py on a system without a python executable.
Change-Id: I9f8418932628569c445ee01bc6f9ead8aed7989f
1. Run pyformat scripts/hiddenapi -s 4
--force_quote_type none -i to fix formatting.
2. rename restricted variable names (e.g. variable name "input" has been
changed to "csvfile")
3. use pylint: disable=<X> where fixes are not obvious
Test: m merge_csv signature_patterns signature_patterns_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc <file1>
<file1_test>
Bug: 195738175
Change-Id: I800a208f9c0ee1d32e68e4b20fd5933b3ab92c0e
1. Run pyformat scripts/hiddenapi/generate_hiddenapi_lists.py -s 4
--force_quote_type none -i to fix formatting. Some double quotes change
to single quotes since pyformat enforces consistent quotes
2. change map/filter to list comprehension to fix pylint's bad-builtin
warning
3. use pylint: disable=<X> where fixes are not obvious
Test: m generate_hiddenapi_lists_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc <file1>
<file1_test>
Bug: 195738175
Change-Id: I15e71d8d81f0e3fa66d84e6e62eb1848963136e6
1. Run pyformat scripts/conv_linker_config.py -s 4 --force_quote_type none -i to fix formatting
2. Annotate #pylint: disable=import-error for linker_config_pb2 since it
will be provided by soong
Test: m conv_linker_config
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/scripts/conv_linker_config.py
Bug: 195738175
Change-Id: I791576cf65cb053f68c804f8ec5c2fc22976fdb4
Fetching artifacts from the build server doesn't go well with large
numbers of files in nested directories.
Produce an additional rustdoc.zip artifact to make this easier.
This change also prevents ${DIST} from receiving the unpacked rustdoc
site. This can be changed if b/188822051 is fixed.
Bug: 162741284
Test: m rustdoc; check resulting zip contents manually
Change-Id: I80b6a8fa6e274d2d8c3419d8734251afd4d7dba7
Previously, the sdk snapshot would include all the entries from the
stub-flags.csv and all-flags.csv modular files generated by a single
bootclasspath_fragment. That included implementation details, i.e.
class members that are not part of a stable API or the hidden API which
meant that the sdk snapshots were implementation dependent.
This change removes the implementation details from the modular flag
files, i.e. those entries that are only marked as "blocked". When
comparing the files against the corresponding subset of the monolithic
files it assumes that any entries missing from the modular flag files
are blocked.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I6b67b2253cf029d6830b58a06ebb0c8fcaa0dd71
Previously, the signature-patterns.csv file included a lot of
implementation details, e.g. the signatures of dex members or inner
classes that are not part of any API, including the hidden API.
This change will remove all member signatures and inner class names
from the file and replace them with just the outermost qualified class
name.
That will still leave some implementation details, e.g. the names of
implementation only classes and packages.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I9de6a2a6129e875e19f7ded5fae578cbdb584660
Previously, a simple map from dex signature to the flags was used to
store the monolithic hidden API flags. This change replaces that with a
trie that uses packages and class names extracted from the signature
to map from the signature to the flags.
The signature is broken down into pieces consisting of package names,
class names and the remaining member signature. They are used in turn
to navigate through nodes in the trie to a Leaf that contains the flags
associated with the signature.
The flags can be retrieved from the trie in a number of ways:
* Using the complete signature to retrieve the flags for a single
member.
* Using a class name to retrieve the flags for all the members in a
class and any nested classes.
* Using a package wildcard to retrieve the flags for all the members in
the classes in that package.
* Using a recursive package wildcard to retrieve the flags for all the
members in the classes in that package and nested packages.
This will allow a bootclasspath_fragment to select a subset of the
monolithic flags to compare against itself without specifying the
signatures of every member of that set.
Before creating the trie an attempt was made to compute the subset by
iterating over all the signatures in the monolithic flags and matching
against a regular expression created from the patterns but that was too
slow. It took minutes to run whereas using the simple map or the trie
only takes a few seconds.
Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: I36f5e319d3e7d62dd34305de1eec990a93cb3a89
Previously, the signatures used to select the subset of the monolithic
flags were simply the signatures read from the modular flags file. This
change moves the creation of the signature list into a separate script
that outputs the signatures to a file and then passes the path through
Soong from the bootclasspath_fragment modules that create it to the
platform_bootclasspath module that uses it to compare the modular
flags against the monolithic flags.
Currently, the signatures are the full signatures but follow up changes
will replace them with patterns (hence the name) that avoids having to
include implementation details in the hidden API flags that are output
as part of a bootclasspath_fragment's snapshot.
This change moves the stub flags related code next to the all flags
related code as they are treated in a similar way.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I2855bf6d05c91b8a09591664185750361c7e644f
Previously, the compare_signature_flags compared the full set of
monolithic flags against the modular flags and just ignored any
monolithic flags for a signature that was not present in the modular
flags. It essentially performed two tasks, identifying the subset of
the monolithic flags against which the modular flags should be compared
and then doing the actual comparison.
This change separates those two tasks, by first creating a dict that is
the subset of the monolithic flags against which the modular flags will
be compared and then do the comparison.
At the moment the subset is simply the set of monolithic flags
associated with the signatures that are present in the modular flags.
Follow up changes will provide other ways to select the subset.
Bug: 194063708
Test: atest --host verify_overlaps_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: Icb547f071b814dd955f31255b0d35597f7ba4aa1
Switches symbols to use monolithic... and modular... to try and clarify
the purpose of those symbols.
Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: Ifc646edebe3d43b51dc3528fa883718399cb5254
Refactor verify_overlaps to make it testable and add tests for the
comparison. It does make one significant change in behavior which is to
read each of the files produced by a bootclasspath_fragment into a dict
before comparison, rather than reading and comparing them a row at a
time. That allows it to reuse the code to read a CSV into a dict.
Bug: 194063708
Test: atest --host verify_overlaps_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: Ib70ac87fe089fc25e3bef18f367d4939bfc0cb8d
Major operations:
* Fix the go_package entries to be consistent, as the new tool
complains when it doesn't have a '/'.
* Regenerate with the new protoc-gen-go tool.
* github.com/golang/protobuf -> google.golang.org/protobuf
* proto.[Un]MarshalText -> prototext.[Un]Marshal
Change-Id: Ie1147bd2457fafb66ba555461b3bf14f0561a25d
atest generate_hiddenapi_lists_test would not work without setting
verbosity=2.
Bug: 194063708
Test: atest --host generate_hiddenapi_lists_test
Change-Id: Ic076ea85d58917921081ae33e72f06011c4ca49e
Normally Soong does that automatically when it handles module names
specified in Android.bp properties. However not all <uses-library>
entries in the manifest correspond to real modules: some of the
optional libraries may be missing at build time. Therefor this script
accepts raw module names as spelled in Android.bp/Amdroid.mk and trims
the optional namespace part manually.
Bug: 193891722
Test: manifest_check_test.py
Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
$ adb wait-for-device && \
adb root && \
adb logcat | \
grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C1
# empty output, no errors
Change-Id: I184fb8a2dc26f49e90fb597ebcd6a58c00705206
Bug: http://b/187231324
This script parses soong/cc/config/global.go to return clang prebuilts
version. (It may get the version from a json config file in the
future.)
Currently it's used in development/scripts/symbol.py and is the
recommended way to get clang prebuilts version number in the future.
Test: get_clang_version_test.py
Change-Id: Id92a74fd523a5270999bb4d6454525bf2f2f0c04
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-stub-flags.txt
- check that an error is reported if a modular stub-flags.csv file,
i.e. one created by a fragment is not a subset of the monolithic
file, e.g. because a signature in the modular file has different
flags than it does in the monolithic or is not present there.
Change-Id: I46ebb495cb093a5e3abe7571c49933c845318549
Previously, if the --header property was not specified then merge_csv
would use a header constructed by sorting all the fields in the input
files. That required that any use of merge_csv which did not already
have headers in the required order would have to explicitly specify the
headers. That made it harder to use merge_csv as a generic tool as each
invocation needed to be aware of what headers were exported in the
output.
This change causes merge_csv to simply use the headers in the order in
which they are encountered in the input files. That removes the need to
specify the --header option when generating the index files.
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-index.csv out/soong/hiddenapi/hiddenapi-unsupported.csv
- make sure that they are not changed by this change.
Change-Id: I420b7d07aea85af6372cd7580a8be5e2cc82a513
Use the --soong-only flag instead to create it the proper way with the
config make step. This fixes build problems arising from the incomplete
soong.variables settings.
This partially relands https://r.android.com/1535202 so that we'll use
TARGET_PRODUCT=mainline_sdk in module SDK builds everywhere.
build-aml-prebuilts.sh isn't removed completely because we still need
the special case to create build_number.txt.
Test: art/build/build-art-module.sh --skip-apex && \
packages/modules/ArtPrebuilt/update-art-module-prebuilts.py \
--local-dist out/dist
then check that the created prebuilt dex2oatd64 runs with the repro
in b/190194345
Test: build/soong/scripts/build-mainline-modules.sh
then
prebuilts/runtime/mainline/update.py --local-dist .../out/dist
in a master-art tree and check that art/build/build-art-module.sh
works there
Bug: 190194345
Bug: 174315599
Change-Id: I86327b082b61c292f18c0a6b315d9add08002832
This script expects to build uncompressed APEX and use them as
prebuilts.
Bug: 189926000
Test: build_mainline_modules.sh
Change-Id: If0f62c2baa3db3c079b2913fcdc3dafd66394011
This makes sure that where there is overlap between the hidden API
flags generated for a module and the monolithic flags that they are
identical. That ensures that the modular hidden API flags will be
compatible with previous releases that relied on the monolithic flags.
Bug: 179354495
Test: m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
m out/soong/hiddenapi/hiddenapi-flags.csv
- Create some inconsistencies between the above two files.
m out/soong/hiddenapi/hiddenapi-flags.csv.valid
Change-Id: Iaf9e23cef63e221608955d89dc8d496bcc70c86e
The classpaths.proto file is needed at build time to generate appropriate
classpath configs for the modules. Unbundled ART branches do not include
packages/modules/SdkExtenstion in their manifest; so put the proto in a
project that should be visible to all unbundled modules.
Bug: 180105615
Test: m nothing
Change-Id: Ie5be1822fbee92f280332622a9317894cc98d7ac
The behaviour is semantically identical, however,
the tool additionally sorts the symbols by address,
compresses frame unwind information more efficiently,
and improves random-accessibility for lazy decompression.
Overall, the changes balance and the output size is same,
however, libunwindstack can access the data much faster
while using less memory (due to the lazy decompression).
It will also enable further improvements in the future.
Bug: 110133331
Test: ART unwinding tests, run prefetto on the device.
Change-Id: Id48f9fe67fb67fcf2b90cc3b217b71bb8f5147ca
Previously, if generate_hiddenapi_lists.py was invoked without any
custom flag files, e.g. hiddenapi-unsupported.txt, then it would fail.
This change allows it to be used for generating the hiddenapi lists
for modules that do not have any custom flag files,
e.g. framework-sdkextensions
Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
unchanged by this change
Change-Id: I455d453024c9f06ed59cbc1e9838234f8b7c7317
Add a function to lint-project-xml to scan the given lint baseline file
for disallowed baseline lint issues.
Test: Manual
Bug: 182349282
Change-Id: I0db32bec0da24487b2f2b3f6704629f56ae76f56
'merge' command can merge multiple configurations.
It just delegates protobuf's msg.MergeFrom(other_msg).
usage: conv_linker_config merge -o out.pb -i a.pb -i b.pb ...
out.pb will have all keys from multiple inputs.
Bug: 181093750
Test: m
Test: MicrodroidTestCase
Change-Id: Ibf715aa0bcc1e5c82c85a9af8fe7dca8d6ab68ad
This is unused and doesn't work with our current API review process
anyway (un-drafting an API won't be flagged for API council review).
Test: treehugger
Bug: None
Change-Id: I6d8fcc9885b82dac5ada7772d9e3fb9101524ece
The proto is used at build time to define classpath fragments to be read
and parsed at runtime by `derive_classpath` service; in order to define
and set BOOTCLASSPATH, DEX2OATCLASSPATH, and SYSTEMSERVERCLASSPATH
environ variables.
The tool is used for debugging and generation of binary protos.
Bug: 180105615
Test: m && launch_cvd with a follow up changes
Change-Id: I7e8b084e5f63e9aaad11da3221b909818e69e235
Bug: 132357300
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Test: temporarily patch Gallery2 to fail the check, observe the error
Change-Id: Id67e359188396f68dcecd8b3f4d1bc26271af56d
This no longer needs to be a prebuilt since stats-log-api-gen can be
built from proto_logging without extra dependencies.
Test: Treehugger
Bug: 170149255
Change-Id: I6925c2526cee1cbff3ba78bedd7f6c2f58dfff0f
Update OWNERS file under build/soong/scripts to include coverage team as owner of related scripts.
Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only
Change-Id: I6150d5bda0d59828a985cd2ab5dd0aa38a7987c3
Instead, return "any" SDK version 10000, which will cause dexpreopt to
not add any compatibility libraries to class loader context. This is as
good guess as any, and the build system already uses "any" version if
there is not manifest or APK from which to extract targetSdkVersion.
This fixes broken build on cf_x86_auto-userdebug.
Bug: 132357300
Bug: 183943146
Test: lunch cf_x86_auto-userdebug \
&& m out/target/product/vsoc_x86/obj/APPS/playback_intermediates/dexpreopt.zip
Change-Id: I5ee75639ece716c3a9800ddea346fe7c73da7c3a
Update the gen_ndk_usedby_apex.sh script to generate NDK API list with
library name.
Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only
Change-Id: I799e03ad7d81a4184229f658fd6c8824b9ba6e1f
There are modules that would have passed verify_uses_library check, but
don't run it for some reason (the check gets enabled either with an
explicit setting, or if the module has nonempty <ues-library> lists in
the build properties). Previously all such modules were assumed to have
empty CLC, which is not always true. In particular, compatibility
libraries are ignored, which affected e.g. Calendar and messaging apps.
This CL gives such apps a chance to have correct CLC. The goal for the
future is to enforce verify_library_check by default.
Bug: 132357300
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: Iea3be0fc9d7775c52950848b5a3fd3b7fcd36c53
The section is stripped from binaries (but preserved in /symbols).
However, is it also unintentionally copied into the minidebuginfo
compressed section, which is included in the final shipped binary.
(This does not affect symtab symbols, which are handled as usual)
Removing it from mini-debug-info shrinks some rust binaries by
up to 10x, saving >10MB on the system partition overall (AOSP).
Test: extract minidebuginfo and check it manually with readelf
Change-Id: I462d81455626bb2b5244e1bf3d5f8b71646f3401
With clang r377782, llvm binutils implements all the necessary
funtionaility for strip.sh. We can finally get rid of all the fallback
GNU binutils usage.
Test: m
Bug: 141010852
Bug: 135627985
Change-Id: I110f6028dab7f599decf59a5cb1b927b35e11857
Java modules that are defined in makefiles are not processed in
topological order, so it is necessary to communicate information from
dependencies via dexpreopt.config files. This has already been done in
make/core/dex_preopt_config_merger.py, and now manifest_check.py also
needs to get library names from their dexpreopt.config files.
This is to accommodate Java libraries which name differs from their
modules name. Soong properties `uses_libs`/`optional_uses_libs` and
makefile vars `LOCAL_USES_LIBRARIES`/`LOCAL_OPTIONAL_USES_LIBRARIES`
contain module names, not library names, so it is necessary to translate
them when comparing against library names in the manifest.
Bug: 132357300
Test: lunch cf_x86_64_phone-userdebug && m && launch_cvd \
adb wait-for-device && adb root && adb logcat \
| grep -E 'ClassLoaderContext [a-z ]+ mismatch'
# empty grep output, no errors
Change-Id: I7cfb17d6d74e37fc9f37d977f15f0d711bab5c8e
Previously there were two different scripts that did similar things:
1) build/soong/scripts/manifest_check.py
2) build/make/core/verify_uses_libraries.sh
Both scripts extracted <uses-library> tags and `targetSdkVersion` from
the manifests of Java modules, but 1) worked for XML manifests, and 2)
worked for APKs. This CL reimplements the functionality from 2) in 1),
so that one script can handle both XML manifests and APKs.
Bug: 132357300
Test: lunch cf_x86_64_phone-userdebug && m && launch_cvd \
adb wait-for-device && adb root && adb logcat \
| grep -E 'ClassLoaderContext [a-z ]+ mismatch'
# empty grep output, no errors
Change-Id: Id1b66e4f3f30f307dba70cb111c7571762cb546a
The new mode is enabled with environment variable
RELAX_USES_LIBRARY_CHECK. If the variable is set to true, then a
verify_uses_libraries check failure does not fail the build, instead it
sets a special compiler filter "extract" for dexpreopt, which means that
the DEX file will be extracted, but it won't be compiled to native code.
Class loader context will be set to empty in this case (not &, as it is
going to be deprecated soon).
If the variable RELAX_USES_LIBRARY_CHECK is unset or set to something
other than "true", then the old behaviour of the verify_uses_libraries
check is preserved.
The intended use case for this flag is to have a smoother migration path
for the Java modules that need to add <uses-library> information in
the build files. The flag allows to quickly silence build errors. This
flag should be used with caution and only as a temporary measure, as it
masks real errors and affects performance.
verify_uses_libraries check is reworked so that it writes the error
message to a status file (which is used instead of the former timestamp
file). Currently the stored error message is not used, but it may be
used later to produce a warning. Dexpreopt command checks if the status
file exists and is nonempty; if that is the case, then compiler filter
is set to "extract".
Bug: 132357300
Test: Manually add some mismatch between the libraries in the Android.bp
and Android.mk files for dexpreopted apps, build with
RELAX_USES_LIBRARY_CHECK=true and obsserve that the build doesn't
fail and they are compiled with compiler-filter "extract".
Unset RELAX_USES_LIBRARY_CHECK and observe that the build fails.
Change-Id: Ibb5d993a25b1df1d2e70b7d5aafc6997f9d64e67
DO NOT MERGE / RESTRICT AUTOMERGE and its cousin DO NOT MERGE
ANYWHERE are almost always misused. Make it an error to upload
commits with those merge strategies.
Bug: 156896646
Test: repo upload -n (with DNM and RM commits)
Change-Id: Iae3540fa35dbd7b87a14b53a20be0cd23a206f3a
This script will be used to merge the csv outputs from bloaty into a
single proto file.
Bug: 172339795
Test: atest --host bloaty_merger_test
Change-Id: I9a3c12d7b0c4b5e347fec04e6758bcc1fbc8ab40
Adds a new --key_field option to merge_csv.py which specifies the name
of the field that should be used to sort the input. If specified it
causes that field to be the first in each row and performs the merge
operation of a merge sort on the input files. That assumes that each
input file is already sorted into the same order.
Modifies the rules that use merge_csv.py to pass in:
--key_field signature
to sort the rows by signature.
Bug: 180387396
Test: Verified that hiddenapi files (both aggregated ones and for the
individual modules) are not affected by this change other than
changing the order.
Change-Id: Idcd5f0fea373b520b604889e1c280f21ed495660
Adds the hiddenapi_additional_annotations to allow a library to list
the libraries that provided additional hiddenapi related annotations
for a library.
Modifies merge_csv.py so it can process multiple zip files at the same
time and uses that to merge the embedded .uau files from a module and
those it depends upon.
Bug: 180102243
Test: m droid
Verified that hiddenapi files (both aggregated ones and for the
individual modules) are not affected by this change.
Change-Id: I796520021c7357398a9e2a09f1029e4a578b05b3
This is needed first for ART to enable statsd metrics reporting, but
this will be needed by more modules as they become unbundled.
Test: build-mainline-modules.sh
Bug: 178500083
Change-Id: I5af0b8db7a24c86ffd3f34732ac747a4fc6a7a55
Also, creates a python_binary_host module for generate_hiddenapi_lists
and uses that when constructing the build rule rather than using the
file directly.
Bug: 177317659
Test: m droid
Verified that hiddenapi files (both aggregated ones and for the
individual modules) are not affected by this change.
Change-Id: Ia11bb203ce5a74740d35f1b7e86716e15aad336e
Running find on the entire module directory can lead to flaky failures
when a directory is modified by another rule while find is traversing
it. Pass a list of libraries in the module into the script instead
of finding them on disk.
Fixes: 178509901
Test: TARGET_BUILD_APPS=com.android.runtime dist
Change-Id: Ie4bf288f0e3d99cbbed5ca8770687a9bbe5377e4
This allows the compression to be done on multiple threads
and more importantly, it allows libunwindstack to decompress
only the needed blocks, which reduces unwind time and memory.
Bug: 110133331
Test: verify lazy decompression in libunwindstack
Change-Id: Ia3f597928edcbc85acdf05cb5c83ac7fced39148
When we start switching vanilla platform builds to use ART prebuilts,
this script should still build from sources.
Test: TH
Bug: 172480615
Change-Id: I04cb61b484855d22f22fd9b82f6abb30eb5665c3
Revert submission 1535006-bap
Reason for revert: Looks like it breaks the build as aapt2 depends on the build_number.txt this script was generating. Error log below.
Reverted Changes:
I0f6b34310:Delete build-aml-prebuilts.sh
I4eae4153a:Replace use of build-aml-prebuilts
Change-Id: I7f5f1b3047f3d71eb8ac7bec95c5916425d5d8f2
FAILED: ninja: 'out/aml/soong/build_number.txt', needed by 'out/aml/soong/.intermediates/frameworks/base/tools/aapt2/aapt2/linux_glibc_x86_64/unstripped/aapt2', missing and no known rule to make it
This reverts commit 556b2ad77e.
Original change-id: Ib1b2f0dd2f9ae85b1545c6cc5bb4c5bbdfac1c15
Reason for revert: Add excutable permission to the gen_ndk_backedby_apex.sh.
Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only
Change-Id: Ib587ba200cd7f2f61d478452b43c329d72b2de06
Add gen_ndk_usedby_apex.sh script to generate NDK API list used by Mainlain modules when modules get build.
Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only
Change-Id: Ib1b2f0dd2f9ae85b1545c6cc5bb4c5bbdfac1c15
Dexpreopt and boot jars package check all require access to dex
implementation jars created for java_library and java_sdk_library. They
were available when building from source but not when building from
prebuilts, even though they are embedded within the .apex files that
are referenced from prebuilt_apex.
This changes adds support to prebuilt_apex to export the dex
implementation jars and updates java_import to use those exported dex
implementation jars.
In a source build dexpreopt/boot jars package check access the apex (or
platform) specific variant of a java_library, e.g. core-oj, from which
it retrieves the dex implementation jar path.
After this change in a prebuilt build dexpreopt/boot jars package check
behave in the same way except in this case they retrieve the dex
implementation jar path from the apex (or platform) specific variant of
the java_import, e.g. core-oj.
The work to export files from a `.apex` file for use by other modules
is performed by a new `deapexer` module type. It is not used directly
in an `Android.bp` file but instead is created implicitly by
`prebuilt_apex`,
In order to do that this contains the following changes:
* Adds a new `dexapexer` module type to handle the exporting of files
from the `.apex` file.
* Adds an exported_java_libs property to prebuilt_apex to specify the
set of libraries whose dex implementation jars need exporting.
* Creates apex specific variants of the libraries listed in the
exported_java_libs property.
* Adds the set of exported files to the ApexInfo to make them available
to the apex specific variants.
* Prevents the prebuilt_apex variants from being merged together as
they will not be compatible.
* Modifies java_import to use the exported file for variants of a
prebuilt_apex.
* Adds a ninja rule to unpack (using deapexer) the contents of the
prebuilt_apex's apex file, verify that the required files are present
and make them available as outputs for other rules to use.
* Some minor refactorings to support these changes.
* Adds tests to cover prebuilt only, prebuilt with source preferred,
and prebuilt preferred with source.
Test: m nothing
Bug: 171061220
Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
Its usecase can now be supported natively by soong_ui via the
--skip-kati flag + the mainline_sdk product. Migrate
build-mainline-modules.sh to use this new approach, and delete the
script.
Bug: 174315599
Test: build/soong/scripts/build-mainline-modules.sh
Change-Id: I0f6b34310df51ddf060b2912d43487f4d37dd01b
manifest_fixer.py sets a lib's targetSdkVersion to '15' if it is
not explicitly set. But ManifestMerger will add dangerous
permissions CALL_LOG_READ/CALL_LOG_WRITE if the following
conditions are met:
- The app's targetSdkVersion is >16
- A linked lib's targetSdkVersion is <16
- The app has CONTACTS_READ/CONTACTS_WRITE permissions
This condition is triggered for some (OEM) app builds (of
Settings). Update manifest_fixer.py to use '16' as its default
targetSdkVersion so that manifest_merge doesn't add these
dangerous permissions.
Change-Id: Id5f41f7da98c190c8d145fceba0085cf473f4c7a
Without this Soong may run into inconsistencies due to stale prebuilts.
This amends https://r.android.com/1529658.
Test: TH build of NDK
Bug: 175918603
Bug: 172480615
Change-Id: I14680e4547d7d5f560d31ef61b71af8d159c4c53
Should fix build errors on mainline_modules on aosp-master. This script
still does not propagate all options to aapt2 correctly, but that will
be fixed by removing the script altogether (b/174315599).
Test: build/soong/scripts/build-aml-prebuilts.sh art-module-sdk \
art-module-test-exports conscrypt-module-host-exports \
conscrypt-module-sdk conscrypt-module-test-exports \
i18n-module-host-exports i18n-module-sdk \
i18n-module-test-exports platform-mainline-sdk \
platform-mainline-test-exports runtime-module-host-exports \
runtime-module-sdk
(the failing command in
build/soong/scripts/build-mainline-modules.sh)
Bug: 175386777
Change-Id: I81aa20b1abd177f0d2886ab9b04831437da72f2a
