Commit graph

68 commits

Author SHA1 Message Date
Paul Duffin
2c36f24082 Sort hiddenapi monolithic files by signature
Adds a new --key_field option to merge_csv.py which specifies the name
of the field that should be used to sort the input. If specified it
causes that field to be the first in each row and performs the merge
operation of a merge sort on the input files. That assumes that each
input file is already sorted into the same order.

Modifies the rules that use merge_csv.py to pass in:
    --key_field signature
to sort the rows by signature.

Bug: 180387396
Test: Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change other than
      changing the order.
Change-Id: Idcd5f0fea373b520b604889e1c280f21ed495660
2021-02-16 17:38:08 +00:00
Paul Duffin
82b3fcf123 Remove duplicates in monolithic hidden API files
Previously, multiple APEX variants of some boot jars were being
processed by hiddenapi to extract information which resulted in
duplicate entries in the monolithic hidden API files.

This change applies the same filter that was previously used to ensure
that the hiddenapi-flags.csv file did not include any duplicates to all
sources of hidden API information.

Bug: 180102243
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change other than
      removing some duplicates entries.
Change-Id: I9ffc8586d5d6efea4e3440be2dfd5424790665c8
2021-02-16 13:28:26 +00:00
Colin Cross
dcf71b299c Convert java.Dependency to JavaInfo provider
Export information about java dependencies through a Provider
instead of accessing the module directly.

Test: java_test.go
Test: no changes to build.ninja
Change-Id: Ifc5d566bf6f6ebc0ad399e948effaa1ef6a22876
2021-02-09 15:36:25 -08:00
Paul Duffin
fdada68320 Move hiddenapi tools used by build/soong from frameworks/base
Also, creates a python_binary_host module for generate_hiddenapi_lists
and uses that when constructing the build rule rather than using the
file directly.

Bug: 177317659
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
Change-Id: Ia11bb203ce5a74740d35f1b7e86716e15aad336e
2021-02-09 10:46:36 +00:00
Paul Duffin
01289a2407 Add test for hiddenAPI index file generation
The index file and the metadata file both currently include duplicate
entries due to them including both the <x> and <x>.impl libraries
created by java_sdk_library in their inputs, plus including both source
and prebuilt versions of the same named module. This change adds a test
to illustrate that behavior. A follow up change will correct the problem
and update the test accordingly.

This change only adds a test for the index file because the metadata
file depends on files from frameworks/base which makes it difficult to
test. Bug 177317659 will fix that and allow the metadata file
generation to be tested too.

Bug: 178361284
Test: m nothing
Change-Id: I33921d7267c9f4bb42726343d73f8a396d536aaa
2021-02-07 10:58:24 +00:00
Paul Duffin
dd63d6d7bd Improve module filtering in hiddenapi stubFlagsRule()
Previously, it ignored any module that was not a platform apex variant
however that required every module that was referenced from the boot
jars to have a platform variant which is not the case when building
from prebuilts.

This change switches it to explicitly check that the variant is for
either the appropriate apex or the platform depending on what is
configured in the BootJars or UpdatableBootJars.

It partially duplicates some logic from the getBootImageJar() function.
It intentionally does not refactor the getBootImageJar() to allow for
reuse because coupling the hiddenapi and dexpreopt logic would make
refactoring either of them more difficult. Any duplicated code will be
deduped once they have both been refactored.

Bug: 178361284
Test: m droid
Change-Id: I4b4e0dc8ee40c1ba1713d7ef72df13d175e84af6
2021-02-05 13:33:25 +00:00
Paul Duffin
ff774a04ad Add documentation for the different files created by hiddenapi
Test: m nothing
Bug: 178361284
Change-Id: Id55646b9d4b7bc1acdb4ed6a6dd4456b746eb54c
2021-01-29 13:19:51 +00:00
Bill Peckham
bae4749de2 Enable prebuilt hiddenapi CSV files.
By enabling these hiddenapi CSV files to be prebuilt, it
becomes possible to create a split build that supports
the hiddenapi encode dex step, but doesn't contain all
of the java sources needed to generate the CSV files.

Bug: 175048716
Test: m nothing
Test: new TestHiddenAPISingletonWithPrebuiltCsvFile
Test: local build without prebuilt hiddenapi
Test: local build with prebuilt hiddenapi
Change-Id: I805ea3ec05860d2df211a3985ec025bf36f0d775
2021-01-19 10:07:45 -08:00
Mathew Inwood
c1be2f8105 Move temp blocklist APIs to max-sdk-r list.
These APIs were temporarily blocked but are now being moved to the
max-sdk-r for final release. Tag these APIs as "lo-prio" since we
believe that they are unused.

Bug: 173499988
Test: m
Change-Id: If9c6be963faa75df77cf3cc7761b384324c8cd3e
2021-01-13 15:49:17 +00:00
Paul Duffin
7f48eeff59 Improve error messages for missing dependencies
Adds some additional information into the paths that are created when
modules are missing in Soong but SOONG_ALLOW_MISSING_DEPENDENCIES=true.

Test: try and build platform against art prebuilts
Bug: 171061220
Change-Id: Ifbcc0af5bdbd15409758a3b6f216cf9b3b5dba31
2020-12-04 16:21:57 +00:00
Colin Cross
f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Mathew Inwood
17a8cd9634 Add 'lo-prio' tag to temp blocklist.
This is a temporary measure to allow these APIs to be identified by
downstream tooling. Before S is finished, these APIs will be moved
to the max-target-R list anyway.

Adding the lo-prio flag should have no semantic impact on these APIs
but does allow them to be distinguished from the regular blocklist
APIs.

Test: m
Bug: 174455522
Change-Id: I031dcecbdef34a4f93a0fec1a92d95f769918486
2020-11-30 11:47:30 +00:00
Mathew Inwood
6ba603c8a1 Merge "Explicitly tag max-target-o APIs as "lo-prio"." 2020-11-23 09:45:48 +00:00
Mathew Inwood
1ef4ba906a Explicitly tag max-target-o APIs as "lo-prio".
This allows the presubmit checker to explicitly allow certain changes to
these APIs, rather than special casing tha max-target-o list. This in turn
will allow other APIs to be tagged as lo-prio in future to get the same
treatment.

Test: m out/soong/hiddenapi/hiddenapi-flags.csv
Bug: 172993934
Change-Id: Ic7cc0243b2b018c96d420a3f266c4b3a6958c32f
2020-11-11 11:01:07 +00:00
Mathew Inwood
82d5b6726b Merge "Temporarily move APIs to blocked list." 2020-11-11 09:13:48 +00:00
Mathew Inwood
59093e3769 Temporarily move APIs to blocked list.
Bug: 170729553
Test: m

Change-Id: I62db78941785fd1bd336cfe81c8e9763e08a0eb0
2020-11-10 13:08:58 +00:00
Anton Hansson
30a4c9d665 Remove dead code
I removed the last use of this method in r.android.com/1450117.

Bug: 169395887
Test: m nothing
Change-Id: I3bbb600d92dac8d9c3dec0eee14c45cdfd320ce0
2020-11-05 10:49:45 +00:00
Mathew Inwood
a44e8c515d Tag removed APIs as such in hiddenapi_flags.csv.
Refactor flags in generate_hiddenapi_lists.py" the invocation to use the
new more flexible schema. Add "--tag removed" to removed.txt APIs so
that we can identify those APIs in the final output.

Test: m -j out/soong/hiddenapi/hiddenapi-flags.csv
Bug: 171300342
Change-Id: I38f2ec7c6e2ff7e5bdd9fe2aeb771d5153a2dc99
2020-10-21 15:52:15 +01:00
Anton Hansson
53781d55bc Merge "Make hiddenapi flag generation use new artifact" 2020-10-12 16:08:11 +00:00
Anton Hansson
b3cbd61846 Make hiddenapi flag generation use new artifact
Use the output of new genrule combined-removed-dex instead of the
removedDexApi output from various metalava runs when generating the
hiddenapi-flags.csv file.

There are some minor difference in the two combined-removed-dex files,
but these diffs do not amount to any diffs in the generated
hiddenapi-flags.csv file. See the full set of diffs here:
https://paste.googleplex.com/6632343525654528

Bug: 158465496
Test: diff hiddenapi-flags.csv before and after
Change-Id: I082d18fc2b8a57ea8c5941d5c955a7970ab9d860
2020-10-07 12:28:23 +01:00
Colin Cross
56a8321c21 Remove global state from apex modules
A global variant was used to store the global mapping between
modules and APEXes.  Replace it with storing pointers to APEX
contents inside each module so that they can query the contents
of any APEXes they belong to.

Bug: 146393795
Test: all Soong tests
Test: single line change to build.ninja host install dependency ordering
Test: no Android-${TARGET_PRODUCT}.mk, make_vars-${TARGET_PRODUCT}.mk or late-${TARGET_PRODUCT}.mk
Change-Id: Id2d7b73ea27f8c3b41d30820bdd86b65c539bfa4
2020-10-06 13:39:57 -07:00
Anton Hansson
a2adc37fc9 Make hiddenapi use prebuilt stubs when it should
Bug: 160455085
Test: build_unbundled_mainline_module.sh (with disabled source-stubs)
Test: hiddenapi_singleton_test.go
Change-Id: Id93c974351b3f8efdf8e4efe4192d3809f4fcaa5
2020-09-18 08:06:58 -07:00
Liz Kammer
88d593d942 Merge "Add hidden_api for java_import" 2020-09-14 13:57:09 +00:00
Liz Kammer
5ca3a6293e Add hidden_api for java_import
Test: go java tests
Test: m
Bug: 160455085
Change-Id: Ib6e826e32ca73ceea0799b26145ad06b1e62a1bf
2020-09-10 07:56:03 -07:00
Xin Li
65cb5c69ea Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)
Bug: 166295507
Merged-In: Ifca7b65f4e27bf14cdc30f72f790b0de90130bae
Change-Id: I3a39be5f0b8736de4822c6a14072c78d4e4ad89d
2020-08-29 01:17:45 -07:00
Andrei Onea
47841973b2 Rename hiddenapi csv artifact
Use a more inclusive terminology (go/hiddenapi-list-renaming).

Bug: 162844296
Test: m
Change-Id: I9feea8f9d8e88977d949dfe4dcbd7e72bf77a451
2020-08-10 17:23:52 +01:00
Jeongik Cha
6df337255c Merge "Introduce AlwaysUsePrebuiltSdks" 2020-08-10 00:51:56 +00:00
Jeongik Cha
816a23a50d Introduce AlwaysUsePrebuiltSdks
Instead of UnbundledBuild, use AlwaysUsePrebuiltSdks
to determine if java modules needs to be built against prebuilt sdks.
And rename UnbundledBuildUsePrebuiltSdks to AlwaysUsePrebuiltSdks to
express its behavior more correctly.(It can be orthgonal to "Unbundled")

Bug: 160390776
Test: TARGET_BUILD_UNBUNDLED_IMAGE=true m vendorimage

Change-Id: I0be7265c1959d8774c295372cd7a9250169f6df9
2020-08-07 12:15:52 +09:00
Andrei Onea
ca79081bde Rename hiddenapi manual lists
Use new names for lists from go/hiddenapi-list-renaming
Bug: 162827370
Test: m

Change-Id: I9522f32faa46d1d49aefad0921ba6d9c427d8779
2020-08-04 15:34:35 +01:00
Aleksei Kalinov
f0f5cdc1cc Update language to comply with Android's inclusive language guidance
See https://source.android.com/setup/contribute/respectful-code for
reference.

Bug: 161896447
Test: ran `m`.
Change-Id: If5ec29482911c281bc1c4bf2cfc84b2f05c2240d
2020-07-29 15:49:05 +00:00
Pete Gillin
811814282d Merge "Split the core/platform API into stable and legacy versions." am: d7db79cc8b am: 5dbf9e09f2
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1322704

Change-Id: I731da4ed59432a67bdc3df124a8939bcf316aa0c
2020-06-09 16:37:21 +00:00
TreeHugger Robot
a4cf51ff72 Merge "Include car project in grey list removed apis" into rvc-dev am: 50cea1a76e
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/11746426

Change-Id: Ie39738a277df3af9bdae6c59ad39b4bf07b16287
2020-06-08 18:13:03 +00:00
Pete Gillin
1f41dbff64 Split the core/platform API into stable and legacy versions.
For now, everything outside libcore still uses the legacy version.

Test: treehugger
Bug: 157640067
Change-Id: If5234e9ee533ff537926801a0af045d36b1caf01
2020-06-08 10:53:51 +01:00
Eric Jeong
9791efa3e8 Include car project in grey list removed apis
- Car framework has removed system APIs.
- These APIs should be tracked at run-time.

Bug: 154832144
Test: 1) android.signature.cts.api.AnnotationTest#testAnnotation
      2) diff $OUT/soong/hiddenapi/hiddenapi-flags.csv before and after this CL

Change-Id: I9813df4b514da41e82d45e6c0bc92643916fe0d8
2020-06-05 09:07:27 -07:00
Ulyana Trafimovich
5539e7b568 Revert^2 "Rename DexJar interface method to DexJarBuildPath."
This reverts commit b0dc851ff4.

Reason for revert: relanding original change. Build failures were
  caused by a race with another CL: https://r.android.com/1320920

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: Ic9016582dae7773b4d4f84a63425f1ef7a5d061f
2020-06-04 17:20:38 +01:00
Ulyana Trafimovich
b0dc851ff4 Revert "Rename DexJar interface method to DexJarBuildPath."
This reverts commit 562c240185.

Reason for revert: breaks `lunch full-eng && m checkbuild`.

Change-Id: Id7c7d6240d98afaf8edd49b6c96cd05534b784cc
2020-06-04 10:37:36 +00:00
Ulya Trafimovich
562c240185 Rename DexJar interface method to DexJarBuildPath.
This is a prerequisite change before adding DexJarInstallPath.

Test: lunch aosp_cf_phone-userdebug && m
Change-Id: I033e08b8bb06c0a844a6bbbfcdc48ce33e9c95cf
2020-06-03 11:40:45 +01:00
Artur Satayev
a8ec55990f Use modular removed-dex.txt files for greylisting.
Use droidstubs for public and system stubs to provide a list of @removed APIs. As these APIs are not present in the stubs, they are not whitelisted / greylised automatically. Keep them on greylist manually.

Bug: 143864733
Test: diff out/soong/hiddenapi/hiddenapi-flags.csv
Change-Id: I4c8e6899fadfdfd1da82f6f453cc92e71aa9b78c
Merged-In: I4c8e6899fadfdfd1da82f6f453cc92e71aa9b78c
Exempt-From-Owner-Approval: clean cherry-pick
(cherry picked from commit c7fb5c9964)
2020-03-31 14:46:39 +00:00
Artur Satayev
c7fb5c9964 Use modular removed-dex.txt files for greylisting.
Use droidstubs for public and system stubs to provide a list of @removed APIs. As these APIs are not present in the stubs, they are not whitelisted / greylised automatically. Keep them on greylist manually.

Bug: 143864733
Test: diff out/soong/hiddenapi/hiddenapi-flags.csv
Change-Id: I4c8e6899fadfdfd1da82f6f453cc92e71aa9b78c
2020-03-30 17:25:13 +01:00
Artur Satayev
8a950790ee Merge CSV files generated by UnsupportedAppUsageProcessor.
Flow:
1. Annotation processor generates a CSV file per class as a CLASS_OUTPUT resource.
2. hiddenapi.go extracts individual .csv files and merges them into an index.csv file per module.
3. hiddenapi_singleton.go merges individual index.csv files into a combined .csv file.

In a follow up hiddenapi-index.csv would replace unsupportedappusage_index.csv

Bug: 145132366
Change-Id: I87d92f9c8d4b1cc1df526fc576ee3c2101116b58
Merged-In: I87d92f9c8d4b1cc1df526fc576ee3c2101116b58
Test: diff unsupportedappusage_index.csv hiddenapi-index.csv
Exempt-From-Owner-Approval: cp from r.android.com/1239709
2020-03-21 13:01:17 +00:00
Artur Satayev
b5df8a04ad Merge CSV files generated by UnsupportedAppUsageProcessor.
Flow:
1. Annotation processor generates a CSV file per class as a CLASS_OUTPUT resource.
2. hiddenapi.go extracts individual .csv files and merges them into an index.csv file per module.
3. hiddenapi_singleton.go merges individual index.csv files into a combined .csv file.

In a follow up hiddenapi-index.csv would replace unsupportedappusage_index.csv

Bug: 145132366
Change-Id: I87d92f9c8d4b1cc1df526fc576ee3c2101116b58
Test: diff unsupportedappusage_index.csv hiddenapi-index.csv
2020-02-20 20:21:49 +00:00
Artur Satayev
79fac05b45 Use flag names with merge_csv.py.
Bug: 145132366
Test: diff -w out/soong/hiddenapi/hiddenapi-greylist.csv
Change-Id: If63c8ae5eda48e49bf58a338bf08f0f2d4fea3cf
2020-02-19 13:24:04 +00:00
Artur Satayev
b01dd44266 Use merge_csv python_binary.
Instead of using .py file directly.

Bug: 145132366
Test: diff -w out/soong/hiddenapi/hiddenapi-greylist.csv
Change-Id: Ib79019234b4ae05486e831b30d27bd1be3bbba67
2020-01-20 17:59:02 +00:00
Jiyong Park
4ed468c1af Revert submission 1194828-revert-1191937-art_apex_available-DWXQGTKMAR
Reason for revert: relanding with fix
Reverted Changes:
Ic4119368c:Revert submission 1191937-art_apex_available
Ia084976bb:Revert submission 1191937-art_apex_available
Iada86226d:Revert submission 1191937-art_apex_available
Ic76735eac:Revert submission 1191937-art_apex_available
I1eb30e355:Revert submission 1191937-art_apex_available
Icaf95d260:Revert submission 1191937-art_apex_available
Ie8bace4be:Revert submission 1191937-art_apex_available
I8961702cf:Revert submission 1191937-art_apex_available
I39316f9ef:Revert submission 1191937-art_apex_available
I522a7e83b:Revert submission 1191937-art_apex_available
I8b9424976:Revert submission 1191937-art_apex_available
I48b998629:Revert submission 1191937-art_apex_available

Change-Id: I95cef82fa5dfaba5b7044cf274ce59ab954b8f2b
2019-12-19 02:11:10 +00:00
Joseph Murphy
4752219db3 Revert submission 1191937-art_apex_available
Original Commit Message:
"""
Rename modules that are APEX-only

The renamed modules are only available for APEXes, but not for the
platform. Use the <module_name>.<apex_name> syntax to correctly install
the APEX variant of the modules.
"""

Reason for revert: Build Cop - Breaks about 15 AOSP targets, with high confidence due to these changes being the only non-robot changes in those builds.

Reverted Changes:
I190ce2d10:Use apex_available property
I990e0a67e:Use apex_available property
I0d1295683:Revert "Avoid duplicated classes for boot dex jars...
I5fb725403:Find the jar libraries in APEX from the correct pa...
I322b1efcc:Rename modules that are APEX-only
Ifa2bd0f8f:Use apex_available property
Iac6533177:Use apex_available property
Ie999602c6:Use apex_available property
I2a3d73397:Use apex_available property
Ic91bcbb9a:Use apex_available property
Ia6c324eed:Use apex_available property
I964d0125c:Use apex_available property

Change-Id: I48b998629b0676cd7353e6844fd87585e0e42989
2019-12-19 01:06:24 +00:00
Jiyong Park
a92fbb3b72 Revert "Avoid duplicated classes for boot dex jars"
This reverts commit 2812df4edb.

This is no longer required as we have
I6463ebc59cf7fd861b812999d7a79c387bbb3335.

hostdex files are available even when the module doesn't have
"//apex_available:platform"

Bug: 128708192
Test: m
Change-Id: I0d1295683222bc9cdbb82630e61ab3421c717daf
2019-12-17 20:35:42 +09:00
Jiyong Park
2812df4edb Avoid duplicated classes for boot dex jars
When a boot classpath java library is directly or indirectly included in
APEXes, multiple variant of the library exist. When running the
hiddenapi tool, we need to eliminate the duplication, otherwise the tool
will complain.

Previously, we simply choose the platform variant of the java library
among the multiple variants. However, if the java library is marked not
available for the platform (i.e. "//apex_available:platform" is not in
the apex_available property), the platform variant does not exist and
thus it is not fed into the hiddenapi tool, which causes missing
references in the tool.

To solve the problem, the platform variant is selected only for the java
libs that are available for the platform. For those libs that are not
available for the platform, we choose one variant of it using a
heuristic; skip testing APEXes and choose com.android.art.release among
the com.android.art.* variants.

Bug: 128708192
Test: m

Change-Id: I33bf297eb3029696ae3504a011486210708fb2c2
2019-12-16 15:19:13 +09:00
Martin Stjernholm
7260d066ea Move/dedupe some host path functions in package_ctx.go.
These functions don't need a PackageContext object.

Test: m
Bug: 145934348
Change-Id: Ia1da2c76cbad292f9ca79617199b2b5b6b265568
2019-12-09 22:04:21 +00:00
Artur Satayev
c5b4f99104 Pass hiddenapi-greylist-max-q.txt to greylist generator.
Bug: 137350495
Test: m, manual inspection of out/soong/hiddenapi/hiddenapi-flags.csv
Change-Id: I67227307ec08ab56a385980eceacaab1b985eb49
Merged-In: I67227307ec08ab56a385980eceacaab1b985eb49
2019-11-20 10:46:16 +00:00
Jiyong Park
7f7766d5ee Revert "Revert "Split Java libraries per apex""
This reverts commit f0f7ca8335.

Bug: 138182343
Test: apex_test added
Change-Id: I5c8a0935ac0f456137c8656815e220b0e7848a5d
2019-07-26 14:19:38 +09:00