https://android-review.googlesource.com/c/platform/build/soong/+/1820496/1/scripts/manifest_check.py#361
caused a regression wrt to statusfile creation. Specifically it inverted
the logic used to create the status file. This file is used in
dexpreopt, thereby affecting all odex files
Test: In internal git, m <an_odex_file>. File size is restored to
pre-regression value
Bug: 199921980
Bug: 200329228
Change-Id: If1e61ac8e2b95455dacc6e7b9431c8f6e1f339b3
Test: Ran the python version of the script and the python3 version
Test: and verified the output is the same. This is for all three
Test: outputs (html, text, xml).
Test: Builds without an error.
Change-Id: I344504a4351c14e00caeb109c5507a60fb6be05a
1. Run pyformat scripts/<script>.py -s 4 --force_quote_type
none -i to fix formatting
2. Annotate #pylint: disable, where straightforward fix is not available
Test: m manifest_check_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/scripts/manifest_check.py
build/soong/scripts/manifest_check_test.py
Bug: 195738175
Change-Id: I9af498c4abd6ac9f8b9df4f93cbdd4424eacff8e
This is part of merging upstream changes from OpenJDK 11.28. This CL
adds jdk.internal.math to package_allowed_list.txt
Bug: 199067710
Test: atest CtsLibcoreOjTestCases
Change-Id: I343bed53ce6b52a6b373a75828b3234073b9cba8
1. Run pyformat scripts/check_boot_jars/check_boot_jars.py -s 4
--force_quote_type none -i to fix formatting.
3. use pylint: disable=<X> where fixes are not obvious
Test: m check_boot_jars
Test: pylint --rcfile tools/repohooks/tools/pylintrc
scripts/check_boot_jars/check_boot_jars.py
Bug: 195738175
Change-Id: I0f86d95e0d174c9f87a2872dedaa65cd3e814246
There is nothing in this script that is not python3, so
make it python3. Without this, a system without python will
fail when trying to run gdbclient.py.
Test: Ran gdbclient.py on a system without a python executable.
Change-Id: I9f8418932628569c445ee01bc6f9ead8aed7989f
1. Run pyformat scripts/hiddenapi -s 4
--force_quote_type none -i to fix formatting.
2. rename restricted variable names (e.g. variable name "input" has been
changed to "csvfile")
3. use pylint: disable=<X> where fixes are not obvious
Test: m merge_csv signature_patterns signature_patterns_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc <file1>
<file1_test>
Bug: 195738175
Change-Id: I800a208f9c0ee1d32e68e4b20fd5933b3ab92c0e
1. Run pyformat scripts/hiddenapi/generate_hiddenapi_lists.py -s 4
--force_quote_type none -i to fix formatting. Some double quotes change
to single quotes since pyformat enforces consistent quotes
2. change map/filter to list comprehension to fix pylint's bad-builtin
warning
3. use pylint: disable=<X> where fixes are not obvious
Test: m generate_hiddenapi_lists_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc <file1>
<file1_test>
Bug: 195738175
Change-Id: I15e71d8d81f0e3fa66d84e6e62eb1848963136e6
1. Run pyformat scripts/conv_linker_config.py -s 4 --force_quote_type none -i to fix formatting
2. Annotate #pylint: disable=import-error for linker_config_pb2 since it
will be provided by soong
Test: m conv_linker_config
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/scripts/conv_linker_config.py
Bug: 195738175
Change-Id: I791576cf65cb053f68c804f8ec5c2fc22976fdb4
Fetching artifacts from the build server doesn't go well with large
numbers of files in nested directories.
Produce an additional rustdoc.zip artifact to make this easier.
This change also prevents ${DIST} from receiving the unpacked rustdoc
site. This can be changed if b/188822051 is fixed.
Bug: 162741284
Test: m rustdoc; check resulting zip contents manually
Change-Id: I80b6a8fa6e274d2d8c3419d8734251afd4d7dba7
Previously, the sdk snapshot would include all the entries from the
stub-flags.csv and all-flags.csv modular files generated by a single
bootclasspath_fragment. That included implementation details, i.e.
class members that are not part of a stable API or the hidden API which
meant that the sdk snapshots were implementation dependent.
This change removes the implementation details from the modular flag
files, i.e. those entries that are only marked as "blocked". When
comparing the files against the corresponding subset of the monolithic
files it assumes that any entries missing from the modular flag files
are blocked.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I6b67b2253cf029d6830b58a06ebb0c8fcaa0dd71
Previously, the signature-patterns.csv file included a lot of
implementation details, e.g. the signatures of dex members or inner
classes that are not part of any API, including the hidden API.
This change will remove all member signatures and inner class names
from the file and replace them with just the outermost qualified class
name.
That will still leave some implementation details, e.g. the names of
implementation only classes and packages.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I9de6a2a6129e875e19f7ded5fae578cbdb584660
Previously, a simple map from dex signature to the flags was used to
store the monolithic hidden API flags. This change replaces that with a
trie that uses packages and class names extracted from the signature
to map from the signature to the flags.
The signature is broken down into pieces consisting of package names,
class names and the remaining member signature. They are used in turn
to navigate through nodes in the trie to a Leaf that contains the flags
associated with the signature.
The flags can be retrieved from the trie in a number of ways:
* Using the complete signature to retrieve the flags for a single
member.
* Using a class name to retrieve the flags for all the members in a
class and any nested classes.
* Using a package wildcard to retrieve the flags for all the members in
the classes in that package.
* Using a recursive package wildcard to retrieve the flags for all the
members in the classes in that package and nested packages.
This will allow a bootclasspath_fragment to select a subset of the
monolithic flags to compare against itself without specifying the
signatures of every member of that set.
Before creating the trie an attempt was made to compute the subset by
iterating over all the signatures in the monolithic flags and matching
against a regular expression created from the patterns but that was too
slow. It took minutes to run whereas using the simple map or the trie
only takes a few seconds.
Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: I36f5e319d3e7d62dd34305de1eec990a93cb3a89
Previously, the signatures used to select the subset of the monolithic
flags were simply the signatures read from the modular flags file. This
change moves the creation of the signature list into a separate script
that outputs the signatures to a file and then passes the path through
Soong from the bootclasspath_fragment modules that create it to the
platform_bootclasspath module that uses it to compare the modular
flags against the monolithic flags.
Currently, the signatures are the full signatures but follow up changes
will replace them with patterns (hence the name) that avoids having to
include implementation details in the hidden API flags that are output
as part of a bootclasspath_fragment's snapshot.
This change moves the stub flags related code next to the all flags
related code as they are treated in a similar way.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I2855bf6d05c91b8a09591664185750361c7e644f
Previously, the compare_signature_flags compared the full set of
monolithic flags against the modular flags and just ignored any
monolithic flags for a signature that was not present in the modular
flags. It essentially performed two tasks, identifying the subset of
the monolithic flags against which the modular flags should be compared
and then doing the actual comparison.
This change separates those two tasks, by first creating a dict that is
the subset of the monolithic flags against which the modular flags will
be compared and then do the comparison.
At the moment the subset is simply the set of monolithic flags
associated with the signatures that are present in the modular flags.
Follow up changes will provide other ways to select the subset.
Bug: 194063708
Test: atest --host verify_overlaps_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: Icb547f071b814dd955f31255b0d35597f7ba4aa1
Switches symbols to use monolithic... and modular... to try and clarify
the purpose of those symbols.
Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: Ifc646edebe3d43b51dc3528fa883718399cb5254
Refactor verify_overlaps to make it testable and add tests for the
comparison. It does make one significant change in behavior which is to
read each of the files produced by a bootclasspath_fragment into a dict
before comparison, rather than reading and comparing them a row at a
time. That allows it to reuse the code to read a CSV into a dict.
Bug: 194063708
Test: atest --host verify_overlaps_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: Ib70ac87fe089fc25e3bef18f367d4939bfc0cb8d
Major operations:
* Fix the go_package entries to be consistent, as the new tool
complains when it doesn't have a '/'.
* Regenerate with the new protoc-gen-go tool.
* github.com/golang/protobuf -> google.golang.org/protobuf
* proto.[Un]MarshalText -> prototext.[Un]Marshal
Change-Id: Ie1147bd2457fafb66ba555461b3bf14f0561a25d
atest generate_hiddenapi_lists_test would not work without setting
verbosity=2.
Bug: 194063708
Test: atest --host generate_hiddenapi_lists_test
Change-Id: Ic076ea85d58917921081ae33e72f06011c4ca49e
Normally Soong does that automatically when it handles module names
specified in Android.bp properties. However not all <uses-library>
entries in the manifest correspond to real modules: some of the
optional libraries may be missing at build time. Therefor this script
accepts raw module names as spelled in Android.bp/Amdroid.mk and trims
the optional namespace part manually.
Bug: 193891722
Test: manifest_check_test.py
Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
$ adb wait-for-device && \
adb root && \
adb logcat | \
grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C1
# empty output, no errors
Change-Id: I184fb8a2dc26f49e90fb597ebcd6a58c00705206
Bug: http://b/187231324
This script parses soong/cc/config/global.go to return clang prebuilts
version. (It may get the version from a json config file in the
future.)
Currently it's used in development/scripts/symbol.py and is the
recommended way to get clang prebuilts version number in the future.
Test: get_clang_version_test.py
Change-Id: Id92a74fd523a5270999bb4d6454525bf2f2f0c04
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-stub-flags.txt
- check that an error is reported if a modular stub-flags.csv file,
i.e. one created by a fragment is not a subset of the monolithic
file, e.g. because a signature in the modular file has different
flags than it does in the monolithic or is not present there.
Change-Id: I46ebb495cb093a5e3abe7571c49933c845318549
Previously, if the --header property was not specified then merge_csv
would use a header constructed by sorting all the fields in the input
files. That required that any use of merge_csv which did not already
have headers in the required order would have to explicitly specify the
headers. That made it harder to use merge_csv as a generic tool as each
invocation needed to be aware of what headers were exported in the
output.
This change causes merge_csv to simply use the headers in the order in
which they are encountered in the input files. That removes the need to
specify the --header option when generating the index files.
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-index.csv out/soong/hiddenapi/hiddenapi-unsupported.csv
- make sure that they are not changed by this change.
Change-Id: I420b7d07aea85af6372cd7580a8be5e2cc82a513
Use the --soong-only flag instead to create it the proper way with the
config make step. This fixes build problems arising from the incomplete
soong.variables settings.
This partially relands https://r.android.com/1535202 so that we'll use
TARGET_PRODUCT=mainline_sdk in module SDK builds everywhere.
build-aml-prebuilts.sh isn't removed completely because we still need
the special case to create build_number.txt.
Test: art/build/build-art-module.sh --skip-apex && \
packages/modules/ArtPrebuilt/update-art-module-prebuilts.py \
--local-dist out/dist
then check that the created prebuilt dex2oatd64 runs with the repro
in b/190194345
Test: build/soong/scripts/build-mainline-modules.sh
then
prebuilts/runtime/mainline/update.py --local-dist .../out/dist
in a master-art tree and check that art/build/build-art-module.sh
works there
Bug: 190194345
Bug: 174315599
Change-Id: I86327b082b61c292f18c0a6b315d9add08002832
This script expects to build uncompressed APEX and use them as
prebuilts.
Bug: 189926000
Test: build_mainline_modules.sh
Change-Id: If0f62c2baa3db3c079b2913fcdc3dafd66394011
This makes sure that where there is overlap between the hidden API
flags generated for a module and the monolithic flags that they are
identical. That ensures that the modular hidden API flags will be
compatible with previous releases that relied on the monolithic flags.
Bug: 179354495
Test: m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
m out/soong/hiddenapi/hiddenapi-flags.csv
- Create some inconsistencies between the above two files.
m out/soong/hiddenapi/hiddenapi-flags.csv.valid
Change-Id: Iaf9e23cef63e221608955d89dc8d496bcc70c86e
