Previously, the signature-patterns.csv file (which was output in an sdk
snapshot) included implementation details, e.g. the names of classes
that are not part of any API, including the hidden API. This change
allows a bootclasspath_fragment module owner to customize the generated
signature patterns file to remove as many implementation details as
possible from the sdk snapshot. That means that implementation only
changes would not require that sdk snapshots be updated in sync with
the corresponding APEX.
Module owners can remove implementation in the following ways:
1) Limit the number of packages that are treated as split packages by
explicitly specifying the split packages in the split_packages
property. A split package is one whose classes are provided by
separate bootclasspath_fragment modules and so the signature
patterns has to include every class in that package provided by a
specific bootclasspath_fragment module, including implementation
classes, instead of just listing the package.
The default is to treat all packages as being split, so if no
split_packages is specified then it defaults to ["*"] which matches
all packages. Assuming that no package was split unless specifically
stated would require that all the modules that had split packages
explicitly list them before this change could be submitted as
without them this change would break the build.
Once all existing modules with split packages have been updated to
include them then the default may be changed.
2) Adding package prefixes for any hierarchy of packages that are
owned solely by that bootclasspath_fragment. This removes the need
to list the packages in that hierarchy, including implementation
specific packages.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: If2f90cfc41b6fff9fa4ac8b9d4973110938b9392
Needed for the board configuration.
rbc-run will also now pass any extra arguments it receives
to rbcrun. (The real program, not the script)
Bug: 201700692
Test: rbc_product_config.py aosp_arm64-userdebug
Change-Id: Ic9869f693f1d28bafee59050b6ae7b1fba14dbdd
The behaviour of shell parameter expansion on linux and macOS is
different:
- on linux "\/" in the replaced string is interpreted as simply "/"
- on macOS it is interpreted as the full string "\/"
For example, "android.package.example" would be changed to:
- "android/package/example" on linux
- "android\/package\/example" on macOS
The character / is not a special character for pattern matching in bash,
so it doesn't need to be escaped. Hence using / instead of \/ in the
replaced string works on both linux and macOS.
Change-Id: Id6a5cf32afc53d5ffd989c0ac8aa0b9e0fcbaf82
Fixes: 201947033
As part of the migration to OpenJDK 11, sun.reflect has moved to
jdk.internal.reflect. For now we'll have both packages available
as new code uses the jdk.internal version.
Test: m
Bug: 188889082
Bug: 190791083
Change-Id: Ib03c9c8c346daebd7d97387817b506095f5ec693
Updating OpenJDK 11 means that there is a dependency on
jdk.internal.misc, which is renamed from sun.misc. The new
package is added to the allowed list.
Bug: 190470684
Bug: 190791083
Test: m
Change-Id: I174df7bad5c5ec4286c35998e16c26bcd0737612
https://android-review.googlesource.com/c/platform/build/soong/+/1820496/1/scripts/manifest_check.py#361
caused a regression wrt to statusfile creation. Specifically it inverted
the logic used to create the status file. This file is used in
dexpreopt, thereby affecting all odex files
Test: In internal git, m <an_odex_file>. File size is restored to
pre-regression value
Bug: 199921980
Bug: 200329228
Change-Id: If1e61ac8e2b95455dacc6e7b9431c8f6e1f339b3
Test: Ran the python version of the script and the python3 version
Test: and verified the output is the same. This is for all three
Test: outputs (html, text, xml).
Test: Builds without an error.
Change-Id: I344504a4351c14e00caeb109c5507a60fb6be05a
1. Run pyformat scripts/<script>.py -s 4 --force_quote_type
none -i to fix formatting
2. Annotate #pylint: disable, where straightforward fix is not available
Test: m manifest_check_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/scripts/manifest_check.py
build/soong/scripts/manifest_check_test.py
Bug: 195738175
Change-Id: I9af498c4abd6ac9f8b9df4f93cbdd4424eacff8e
This is part of merging upstream changes from OpenJDK 11.28. This CL
adds jdk.internal.math to package_allowed_list.txt
Bug: 199067710
Test: atest CtsLibcoreOjTestCases
Change-Id: I343bed53ce6b52a6b373a75828b3234073b9cba8
1. Run pyformat scripts/check_boot_jars/check_boot_jars.py -s 4
--force_quote_type none -i to fix formatting.
3. use pylint: disable=<X> where fixes are not obvious
Test: m check_boot_jars
Test: pylint --rcfile tools/repohooks/tools/pylintrc
scripts/check_boot_jars/check_boot_jars.py
Bug: 195738175
Change-Id: I0f86d95e0d174c9f87a2872dedaa65cd3e814246
There is nothing in this script that is not python3, so
make it python3. Without this, a system without python will
fail when trying to run gdbclient.py.
Test: Ran gdbclient.py on a system without a python executable.
Change-Id: I9f8418932628569c445ee01bc6f9ead8aed7989f
1. Run pyformat scripts/hiddenapi -s 4
--force_quote_type none -i to fix formatting.
2. rename restricted variable names (e.g. variable name "input" has been
changed to "csvfile")
3. use pylint: disable=<X> where fixes are not obvious
Test: m merge_csv signature_patterns signature_patterns_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc <file1>
<file1_test>
Bug: 195738175
Change-Id: I800a208f9c0ee1d32e68e4b20fd5933b3ab92c0e
1. Run pyformat scripts/hiddenapi/generate_hiddenapi_lists.py -s 4
--force_quote_type none -i to fix formatting. Some double quotes change
to single quotes since pyformat enforces consistent quotes
2. change map/filter to list comprehension to fix pylint's bad-builtin
warning
3. use pylint: disable=<X> where fixes are not obvious
Test: m generate_hiddenapi_lists_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc <file1>
<file1_test>
Bug: 195738175
Change-Id: I15e71d8d81f0e3fa66d84e6e62eb1848963136e6
1. Run pyformat scripts/conv_linker_config.py -s 4 --force_quote_type none -i to fix formatting
2. Annotate #pylint: disable=import-error for linker_config_pb2 since it
will be provided by soong
Test: m conv_linker_config
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/scripts/conv_linker_config.py
Bug: 195738175
Change-Id: I791576cf65cb053f68c804f8ec5c2fc22976fdb4
Fetching artifacts from the build server doesn't go well with large
numbers of files in nested directories.
Produce an additional rustdoc.zip artifact to make this easier.
This change also prevents ${DIST} from receiving the unpacked rustdoc
site. This can be changed if b/188822051 is fixed.
Bug: 162741284
Test: m rustdoc; check resulting zip contents manually
Change-Id: I80b6a8fa6e274d2d8c3419d8734251afd4d7dba7
Previously, the sdk snapshot would include all the entries from the
stub-flags.csv and all-flags.csv modular files generated by a single
bootclasspath_fragment. That included implementation details, i.e.
class members that are not part of a stable API or the hidden API which
meant that the sdk snapshots were implementation dependent.
This change removes the implementation details from the modular flag
files, i.e. those entries that are only marked as "blocked". When
comparing the files against the corresponding subset of the monolithic
files it assumes that any entries missing from the modular flag files
are blocked.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I6b67b2253cf029d6830b58a06ebb0c8fcaa0dd71
Previously, the signature-patterns.csv file included a lot of
implementation details, e.g. the signatures of dex members or inner
classes that are not part of any API, including the hidden API.
This change will remove all member signatures and inner class names
from the file and replace them with just the outermost qualified class
name.
That will still leave some implementation details, e.g. the names of
implementation only classes and packages.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I9de6a2a6129e875e19f7ded5fae578cbdb584660
Previously, a simple map from dex signature to the flags was used to
store the monolithic hidden API flags. This change replaces that with a
trie that uses packages and class names extracted from the signature
to map from the signature to the flags.
The signature is broken down into pieces consisting of package names,
class names and the remaining member signature. They are used in turn
to navigate through nodes in the trie to a Leaf that contains the flags
associated with the signature.
The flags can be retrieved from the trie in a number of ways:
* Using the complete signature to retrieve the flags for a single
member.
* Using a class name to retrieve the flags for all the members in a
class and any nested classes.
* Using a package wildcard to retrieve the flags for all the members in
the classes in that package.
* Using a recursive package wildcard to retrieve the flags for all the
members in the classes in that package and nested packages.
This will allow a bootclasspath_fragment to select a subset of the
monolithic flags to compare against itself without specifying the
signatures of every member of that set.
Before creating the trie an attempt was made to compute the subset by
iterating over all the signatures in the monolithic flags and matching
against a regular expression created from the patterns but that was too
slow. It took minutes to run whereas using the simple map or the trie
only takes a few seconds.
Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: I36f5e319d3e7d62dd34305de1eec990a93cb3a89
Previously, the signatures used to select the subset of the monolithic
flags were simply the signatures read from the modular flags file. This
change moves the creation of the signature list into a separate script
that outputs the signatures to a file and then passes the path through
Soong from the bootclasspath_fragment modules that create it to the
platform_bootclasspath module that uses it to compare the modular
flags against the monolithic flags.
Currently, the signatures are the full signatures but follow up changes
will replace them with patterns (hence the name) that avoids having to
include implementation details in the hidden API flags that are output
as part of a bootclasspath_fragment's snapshot.
This change moves the stub flags related code next to the all flags
related code as they are treated in a similar way.
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: I2855bf6d05c91b8a09591664185750361c7e644f
Previously, the compare_signature_flags compared the full set of
monolithic flags against the modular flags and just ignored any
monolithic flags for a signature that was not present in the modular
flags. It essentially performed two tasks, identifying the subset of
the monolithic flags against which the modular flags should be compared
and then doing the actual comparison.
This change separates those two tasks, by first creating a dict that is
the subset of the monolithic flags against which the modular flags will
be compared and then do the comparison.
At the moment the subset is simply the set of monolithic flags
associated with the signatures that are present in the modular flags.
Follow up changes will provide other ways to select the subset.
Bug: 194063708
Test: atest --host verify_overlaps_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: Icb547f071b814dd955f31255b0d35597f7ba4aa1
Switches symbols to use monolithic... and modular... to try and clarify
the purpose of those symbols.
Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: Ifc646edebe3d43b51dc3528fa883718399cb5254
Refactor verify_overlaps to make it testable and add tests for the
comparison. It does make one significant change in behavior which is to
read each of the files produced by a bootclasspath_fragment into a dict
before comparison, rather than reading and comparing them a row at a
time. That allows it to reuse the code to read a CSV into a dict.
Bug: 194063708
Test: atest --host verify_overlaps_test
m out/soong/hiddenapi/hiddenapi-flags.csv
- manually change files to cause difference in flags to check
that it detects the differences.
Change-Id: Ib70ac87fe089fc25e3bef18f367d4939bfc0cb8d
Major operations:
* Fix the go_package entries to be consistent, as the new tool
complains when it doesn't have a '/'.
* Regenerate with the new protoc-gen-go tool.
* github.com/golang/protobuf -> google.golang.org/protobuf
* proto.[Un]MarshalText -> prototext.[Un]Marshal
Change-Id: Ie1147bd2457fafb66ba555461b3bf14f0561a25d
atest generate_hiddenapi_lists_test would not work without setting
verbosity=2.
Bug: 194063708
Test: atest --host generate_hiddenapi_lists_test
Change-Id: Ic076ea85d58917921081ae33e72f06011c4ca49e
Normally Soong does that automatically when it handles module names
specified in Android.bp properties. However not all <uses-library>
entries in the manifest correspond to real modules: some of the
optional libraries may be missing at build time. Therefor this script
accepts raw module names as spelled in Android.bp/Amdroid.mk and trims
the optional namespace part manually.
Bug: 193891722
Test: manifest_check_test.py
Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
$ adb wait-for-device && \
adb root && \
adb logcat | \
grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C1
# empty output, no errors
Change-Id: I184fb8a2dc26f49e90fb597ebcd6a58c00705206
Bug: http://b/187231324
This script parses soong/cc/config/global.go to return clang prebuilts
version. (It may get the version from a json config file in the
future.)
Currently it's used in development/scripts/symbol.py and is the
recommended way to get clang prebuilts version number in the future.
Test: get_clang_version_test.py
Change-Id: Id92a74fd523a5270999bb4d6454525bf2f2f0c04
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-stub-flags.txt
- check that an error is reported if a modular stub-flags.csv file,
i.e. one created by a fragment is not a subset of the monolithic
file, e.g. because a signature in the modular file has different
flags than it does in the monolithic or is not present there.
Change-Id: I46ebb495cb093a5e3abe7571c49933c845318549
Previously, if the --header property was not specified then merge_csv
would use a header constructed by sorting all the fields in the input
files. That required that any use of merge_csv which did not already
have headers in the required order would have to explicitly specify the
headers. That made it harder to use merge_csv as a generic tool as each
invocation needed to be aware of what headers were exported in the
output.
This change causes merge_csv to simply use the headers in the order in
which they are encountered in the input files. That removes the need to
specify the --header option when generating the index files.
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-index.csv out/soong/hiddenapi/hiddenapi-unsupported.csv
- make sure that they are not changed by this change.
Change-Id: I420b7d07aea85af6372cd7580a8be5e2cc82a513
