Commit graph

93 commits

Author SHA1 Message Date
Colin Cross
313aa5475f Convert OtherModuleProvider to generic providers API
Convert all of the callers of OtherModuleProvider/OtherModuleHasProvider
to use the type-safe android.OtherModuleProvider API.

Bug: 316410648
Test: builds
Change-Id: Id77f514d68761a262d9ea830a601dbed804bbbe5
2023-12-14 16:12:22 -08:00
Pratyush
faec4db56f Disable Hidden API Checks for ENG Builds
For more discussions/details, please check internal CL ag/24145146

Bug: 289409213
BUg: 285976182
Test: Look for hiddenapi calls in build trace
Change-Id: Ia2780ee419b8da1418ba1c7a7d732712b7c2a322
2023-07-30 13:57:10 +00:00
Jiakai Zhang
b1639db53f Extract duplicate code to common helper functions.
Also, fall back to using a default name for the dexpreopt directory if
we are not building for Android.

Bug: 290583827
Test: m nothing
Change-Id: I3fc6ff9142a2dcdf995796f75891b242fe2848d0
2023-07-11 17:59:05 +01:00
LaMont Jones
0c10e4dcc0 Parallelize singleton execution
Bug: 281536768
Test: manual, presubmits
Change-Id: I57fdc76ba6b277e88e196b506af87127a530fd37
2023-05-19 20:31:32 +00:00
Martin Stjernholm
be10503d3a Drop "prebuilt_" prefixes from names registered in ApexInfo.InApexXxx.
Neither InApexVariants nor InApexModules should have them. This allows
us to get rid of InApexVariantByBaseName as well.

Test: m nothing
Test: m nothing SOONG_CONFIG_art_module_source_build=false
Bug: 180325915
Change-Id: Icbe4e025ce1a4c8dd258ff95d326ca2f27905188
2021-05-26 17:05:17 +01:00
Jiyong Park
ab50b0795a Rename InApexes -> InApexVariants
.. in preparation for the upcoming change. This change doesn't alter any
behavior.

InApexes is a misleading name. People expects that it has the list of
soong module names of the APEXes that a module is part of. So, for
example, `core-oj` is a part of both `com.android.art` and
`com.google.android.art`. However, in reality, that's not true. The
field has `com.android.art` only. This is because the two APEXes
(android and Google) have the same apex name which is `com.android.art`.
That apex name is used in various places like the `apex_available` and
allows us to keep using the same name regardless of whether the APEX is
overridden or not.

However, this is causing problems in some cases where the exact list of
soong module names is required. The upcoming change will add a new field
to handle the case and the new field actually will get the name
'InApexes'. So, the existing field is renamed to a less misleading name
`InApexVariants`.

Bug: 180325915
Test: m nothing

Change-Id: I0c73361b452eddb812acd5ebef5dcedaab382436
2021-05-18 16:05:14 +09:00
Paul Duffin
b6f53c064e Refactor special handling of hidden API encoding for master-art
Instead of encoding the hidden API with an empty set of flags when the
monolithic flags are not available this simply disables encoding
altogether which should have the same behavior at runtime.

This change also removes the unused flags field in hiddenAPISingleton
which was set but never read.

Bug: 179354495
Test: m nothing
Change-Id: I32d5825e5271829993dd4e5be4d4ee1b22fa7b22
2021-05-14 15:57:10 +01:00
Paul Duffin
7ebebfd5f8 Move copying of dex files from dexpreopt_bootjars singleton
The art dex files are copied in the bootclasspath_fragment and the
non-updatable and updatable dex files are copied in the
platform_bootclasspath.

Bug: 177892522
Test: m nothing
Change-Id: I5d3d533d1a7a9f8e7ae20c12eb33029a898a2cd6
2021-05-12 21:51:19 +01:00
Paul Duffin
d6894ca4b9 Split findAndCopyBootJars into separate find and copy functions
The main difference between the dexpreopt_bootjars singleton and the
platform_bootclasspath singleton module is the way they find the
modules to use. The former searches all modules, the latter adds
dependencies on the modules that they need. This change separates the
finding of the modules from the copying of the boot jars for those
modules to make it easier to move the remaining functionality to
platform_bootclasspath.

This temporarily creates a singleton specific copy of the hidden API
function isModuleInConfiguredList() to select the modules in place of
the logic in the getBootJar() method. There is a slight loss of context
information from the error messages but as these methods will be
removed once the boot image creation has moved this is not an issue.

While switching the isModuleInConfiguredListForSingleton() to use the
SingletonContext the error message was fixed to include the name of
the module with the issue.

Bug: 177892522
Test: m nothing
Change-Id: Iaea906da95d9da5301fb964fc593890f2216d336
2021-04-30 12:10:54 +01:00
Paul Duffin
12d29b7786 Move generation of hidden API make vars to platform_bootclasspath
Bug: 179354495
Test: rm out/soong/make_vars*
      m nothing
      grep INTERNAL_PLATFORM_HIDDENAPI_FLAGS out/soong/make_vars*
      - make sure it is still out/soong/hiddenapi/hiddenapi-flags.csv
Change-Id: I86e56512a9a2091f446bad25294d41ea1f4341ee
2021-04-29 15:11:35 +01:00
Paul Duffin
110b0add9e Treat "apex" system_ext the same as platform
Change https://r.android.com/1672245 added support to treat
system_ext:foo in the boot jars configuration the same as platform:foo,
at least for dexpreopt_bootjars.go's getBootJar.

This change replicates that mechanism in platform_bootclasspath and as
that now handles hidden API processing also made a similar change in
the isModuleInConfiguredList function.

Bug: 177892522
Bug: 154976937
Test: m nothing
Change-Id: I105f4fbaa3b0355b013b7c5618d218d888faefb6
2021-04-29 10:35:10 +01:00
Paul Duffin
74431d57c6 Move monolithic stub flags generation to platform_bootclasspath
As part of that this change:
* Moves code that will be common to platform_bootclasspath and
  bootclasspath_fragment from hiddenapi_singleton.go into
  hiddenapi_modular.go.
* Fixes the tests in hiddenapi_singleton_test.go but intentionally
  does not rename them or move them into a more appropriate place so
  as to make it easier to see the differences. A TODO has been added
  and these will be cleaned up in a follow up change.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I680e4dab2e6bdf4a655fa9f255c195175904667e
2021-04-22 14:23:28 +01:00
Paul Duffin
3e7fcc3fe7 Extract common stub flags code
The stubFlagsRule does three separate tasks:
1. It computes the set of modules that provide the stubs.
2. It scans all the modules to find the stub modules and retrieves the
   paths to their dex files.
3. It constructs the ninja rule.

Of those three tasks, 1 and 3 will be same for the
platform_bootclasspath. Instead of searching all the modules for the
ones that provide the stubs it will simply add dependencies onto the
stub modules and retrieve the dex file paths from them.

This change extracts tasks 1 and 3 into separate methods for reuse. It
also parameterizes the generation of the ninja rule.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I893845dbddc4b001dfd44d0e0b1c8a31b7f3f89f
2021-04-22 14:23:08 +01:00
Paul Duffin
d3c1513df9 Add tempPathForRestat to improve consistency
Previously, there were two approaches used to construct the temporary
path needed by the commitChangeForRestat method and neither was
suitable for general use. One was:
    android.PathForOutput(ctx, outputPath.Rel()+".tmp")
The other was:
    combinedAidl.ReplaceExtension(ctx, "aidl.tmp")

The first approach would not work if the supplied path had been created
by PathForModuleOut() or similar as it would drop the module directory.
That could lead to the same path being used for multiple rules.

The second approach would not work for files with a different
extension.

The tempPathForRestat combines the two approaches so it can be used
generally with any WritablePath.

Bug: 179354495
Test: verified that the ninja rules that used these files were not
      changed by these changes.
Change-Id: I4439dea0a823512c281eeb1366522fb49dceb4e3
2021-04-21 23:26:57 +01:00
Paul Duffin
85dee5d8fb Move hidden API metadata file rule to platform_bootclasspath
This change moves the monolithic hidden API index file creation rule
from the hiddenAPISingleton into the platform_bootclasspath.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ib25be3618e30a83dc2929a18062eb58eefdabefb
2021-04-14 18:54:55 +01:00
Paul Duffin
22c743170d Move handling of prebuilt hiddenapi-index.csv to hiddenapi singleton
This change moves the handling of the prebuilt hiddenapi-index.csv file
from the hiddenAPIIndexSingleton and removes that as it no longer
needed.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Iab1b39a57f28caa855c48538aa2230795f2c3271
2021-04-14 18:54:45 +01:00
Paul Duffin
00b2bfdea5 Move hidden API index file rule to platform_bootclasspath
This change moves the monolithic hidden API index file creation rule
from the hiddenAPIIndexSingleton into the platform_bootclasspath. It
also moves the corresponding test from java/hiddenapi_singleton_test.go
to java/platform_bootclasspath_test.go.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ia295d0f7ae9b51ea816f16921aa42339ed91704e
2021-04-14 18:54:29 +01:00
Paul Duffin
082afbb656 Remove unused hiddenapi_flags module type
Bug: 177892522
Test: m nothing
Change-Id: I604a86a25bbc722ec718b0b0272d0481218d3cd9
2021-04-13 00:23:55 +01:00
Paul Duffin
6a766453fd Export monolithic hidden API files from platform_bootclasspath
Makes the monolithic hidden API files accessible from the
platform_bootclasspath so they can be output to the dist build target
and used by other modules, e.g. by doing something like this:
  java_resources: [
    ":platform-bootclasspath{hiddenapi-flags.csv}",
  ],

It makes the paths relative to the out/soong/hiddenapi directory rather
than the out/soong directory to make them easier to use in the
java_resources property without changing the structure of the APK.
Without that attempting to use them in a java_resources property will
result in them being copied to a hiddenapi/ within the APK instead of
being used at the top level as existing APKs like
CtsHiddenApiBlocklistTestApiTestCases expect.

Bug: 177892522
Test: m nothing
Change-Id: I829412fc7d25411e0c2e0713d0d219a18f4af2ee
2021-04-13 00:23:55 +01:00
Paul Duffin
702210b804 Move generation of global hidden API flags to platform_bootclasspath
This change moves the generation of the global hidden API flags from
the singleton to the platform_bootclasspath module. It involves:
1. Moving the ruleToGenerateHiddenApiFlags to hiddenapi_modular.go.
2. Adding HiddenAPIAugmentationProperties to be used by the
   platform_bootclasspath type.
3. Moving the file paths into the platform-bootclasspath module
   definition in frameworks/base/boot/Android.bp.

The flagsRule is kept as a placeholder for now. The emptyFlagsRule is
also kept so that builds continue to work even when the frameworks/base
repository is not present.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Idf4dd414a016831bfe04a01f93234c1c33819881
2021-04-12 11:04:24 +01:00
Paul Duffin
c6bb7cf8d7 Extract ruleToGenerateHiddenApiFlags
Extracts the code for creating the rule that creates the monolithic
hidden API flags file which is encoded into dex implementation jars.
This refactoring is in preparation for moving the functionality from
the hiddenapi_singleton into the platform_bootclasspath.

A follow up change will move the method into the new
hiddenapi_modular.go alongside the hiddenAPIAugmentationInfo as they
will both be used to perform hidden API processing for a
bootclasspath_fragment.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: I2729afa80cdfd2d1d4717365001648453d65632f
2021-04-12 11:04:09 +01:00
Paul Duffin
5bdc83c20a Merge "Update build to new location of hiddenapi config files" 2021-04-07 15:52:06 +00:00
Paul Duffin
b5933f64e4 Remove hidden API special handling of android.test.base
The android.test.base module should never be on the bootclasspath.

Bug: 184331423
Test: m droid
Change-Id: Ibb963b4ce2d915d0205fb1b5a7ca9159a5c99b3f
2021-04-07 10:06:31 +01:00
Paul Duffin
aa73ae5b59 Update build to new location of hiddenapi config files
Bug: 179354495
Test: m nothing
Change-Id: I563237d46f35ae1d2b99ffaa93f77dfcef5a2b6e
2021-04-06 13:08:26 +01:00
Paul Duffin
175947f654 Convert hiddenapi tests to use test fixtures
These tests rely on files provided by javaMockFS() so have been
converted to test fixtures to allow them to remove that dependency
which will allow javaMockFS() to be removed.

Bug: 182638834
Test: m nothing
Change-Id: Ifd4069a74fcf67e555f998ddbc4de3fde26b2aae
2021-03-16 14:25:21 +00:00
Paul Duffin
2c36f24082 Sort hiddenapi monolithic files by signature
Adds a new --key_field option to merge_csv.py which specifies the name
of the field that should be used to sort the input. If specified it
causes that field to be the first in each row and performs the merge
operation of a merge sort on the input files. That assumes that each
input file is already sorted into the same order.

Modifies the rules that use merge_csv.py to pass in:
    --key_field signature
to sort the rows by signature.

Bug: 180387396
Test: Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change other than
      changing the order.
Change-Id: Idcd5f0fea373b520b604889e1c280f21ed495660
2021-02-16 17:38:08 +00:00
Paul Duffin
82b3fcf123 Remove duplicates in monolithic hidden API files
Previously, multiple APEX variants of some boot jars were being
processed by hiddenapi to extract information which resulted in
duplicate entries in the monolithic hidden API files.

This change applies the same filter that was previously used to ensure
that the hiddenapi-flags.csv file did not include any duplicates to all
sources of hidden API information.

Bug: 180102243
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change other than
      removing some duplicates entries.
Change-Id: I9ffc8586d5d6efea4e3440be2dfd5424790665c8
2021-02-16 13:28:26 +00:00
Colin Cross
dcf71b299c Convert java.Dependency to JavaInfo provider
Export information about java dependencies through a Provider
instead of accessing the module directly.

Test: java_test.go
Test: no changes to build.ninja
Change-Id: Ifc5d566bf6f6ebc0ad399e948effaa1ef6a22876
2021-02-09 15:36:25 -08:00
Paul Duffin
fdada68320 Move hiddenapi tools used by build/soong from frameworks/base
Also, creates a python_binary_host module for generate_hiddenapi_lists
and uses that when constructing the build rule rather than using the
file directly.

Bug: 177317659
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
Change-Id: Ia11bb203ce5a74740d35f1b7e86716e15aad336e
2021-02-09 10:46:36 +00:00
Paul Duffin
01289a2407 Add test for hiddenAPI index file generation
The index file and the metadata file both currently include duplicate
entries due to them including both the <x> and <x>.impl libraries
created by java_sdk_library in their inputs, plus including both source
and prebuilt versions of the same named module. This change adds a test
to illustrate that behavior. A follow up change will correct the problem
and update the test accordingly.

This change only adds a test for the index file because the metadata
file depends on files from frameworks/base which makes it difficult to
test. Bug 177317659 will fix that and allow the metadata file
generation to be tested too.

Bug: 178361284
Test: m nothing
Change-Id: I33921d7267c9f4bb42726343d73f8a396d536aaa
2021-02-07 10:58:24 +00:00
Paul Duffin
dd63d6d7bd Improve module filtering in hiddenapi stubFlagsRule()
Previously, it ignored any module that was not a platform apex variant
however that required every module that was referenced from the boot
jars to have a platform variant which is not the case when building
from prebuilts.

This change switches it to explicitly check that the variant is for
either the appropriate apex or the platform depending on what is
configured in the BootJars or UpdatableBootJars.

It partially duplicates some logic from the getBootImageJar() function.
It intentionally does not refactor the getBootImageJar() to allow for
reuse because coupling the hiddenapi and dexpreopt logic would make
refactoring either of them more difficult. Any duplicated code will be
deduped once they have both been refactored.

Bug: 178361284
Test: m droid
Change-Id: I4b4e0dc8ee40c1ba1713d7ef72df13d175e84af6
2021-02-05 13:33:25 +00:00
Paul Duffin
ff774a04ad Add documentation for the different files created by hiddenapi
Test: m nothing
Bug: 178361284
Change-Id: Id55646b9d4b7bc1acdb4ed6a6dd4456b746eb54c
2021-01-29 13:19:51 +00:00
Bill Peckham
bae4749de2 Enable prebuilt hiddenapi CSV files.
By enabling these hiddenapi CSV files to be prebuilt, it
becomes possible to create a split build that supports
the hiddenapi encode dex step, but doesn't contain all
of the java sources needed to generate the CSV files.

Bug: 175048716
Test: m nothing
Test: new TestHiddenAPISingletonWithPrebuiltCsvFile
Test: local build without prebuilt hiddenapi
Test: local build with prebuilt hiddenapi
Change-Id: I805ea3ec05860d2df211a3985ec025bf36f0d775
2021-01-19 10:07:45 -08:00
Mathew Inwood
c1be2f8105 Move temp blocklist APIs to max-sdk-r list.
These APIs were temporarily blocked but are now being moved to the
max-sdk-r for final release. Tag these APIs as "lo-prio" since we
believe that they are unused.

Bug: 173499988
Test: m
Change-Id: If9c6be963faa75df77cf3cc7761b384324c8cd3e
2021-01-13 15:49:17 +00:00
Paul Duffin
7f48eeff59 Improve error messages for missing dependencies
Adds some additional information into the paths that are created when
modules are missing in Soong but SOONG_ALLOW_MISSING_DEPENDENCIES=true.

Test: try and build platform against art prebuilts
Bug: 171061220
Change-Id: Ifbcc0af5bdbd15409758a3b6f216cf9b3b5dba31
2020-12-04 16:21:57 +00:00
Colin Cross
f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Mathew Inwood
17a8cd9634 Add 'lo-prio' tag to temp blocklist.
This is a temporary measure to allow these APIs to be identified by
downstream tooling. Before S is finished, these APIs will be moved
to the max-target-R list anyway.

Adding the lo-prio flag should have no semantic impact on these APIs
but does allow them to be distinguished from the regular blocklist
APIs.

Test: m
Bug: 174455522
Change-Id: I031dcecbdef34a4f93a0fec1a92d95f769918486
2020-11-30 11:47:30 +00:00
Mathew Inwood
6ba603c8a1 Merge "Explicitly tag max-target-o APIs as "lo-prio"." 2020-11-23 09:45:48 +00:00
Mathew Inwood
1ef4ba906a Explicitly tag max-target-o APIs as "lo-prio".
This allows the presubmit checker to explicitly allow certain changes to
these APIs, rather than special casing tha max-target-o list. This in turn
will allow other APIs to be tagged as lo-prio in future to get the same
treatment.

Test: m out/soong/hiddenapi/hiddenapi-flags.csv
Bug: 172993934
Change-Id: Ic7cc0243b2b018c96d420a3f266c4b3a6958c32f
2020-11-11 11:01:07 +00:00
Mathew Inwood
82d5b6726b Merge "Temporarily move APIs to blocked list." 2020-11-11 09:13:48 +00:00
Mathew Inwood
59093e3769 Temporarily move APIs to blocked list.
Bug: 170729553
Test: m

Change-Id: I62db78941785fd1bd336cfe81c8e9763e08a0eb0
2020-11-10 13:08:58 +00:00
Anton Hansson
30a4c9d665 Remove dead code
I removed the last use of this method in r.android.com/1450117.

Bug: 169395887
Test: m nothing
Change-Id: I3bbb600d92dac8d9c3dec0eee14c45cdfd320ce0
2020-11-05 10:49:45 +00:00
Mathew Inwood
a44e8c515d Tag removed APIs as such in hiddenapi_flags.csv.
Refactor flags in generate_hiddenapi_lists.py" the invocation to use the
new more flexible schema. Add "--tag removed" to removed.txt APIs so
that we can identify those APIs in the final output.

Test: m -j out/soong/hiddenapi/hiddenapi-flags.csv
Bug: 171300342
Change-Id: I38f2ec7c6e2ff7e5bdd9fe2aeb771d5153a2dc99
2020-10-21 15:52:15 +01:00
Anton Hansson
53781d55bc Merge "Make hiddenapi flag generation use new artifact" 2020-10-12 16:08:11 +00:00
Anton Hansson
b3cbd61846 Make hiddenapi flag generation use new artifact
Use the output of new genrule combined-removed-dex instead of the
removedDexApi output from various metalava runs when generating the
hiddenapi-flags.csv file.

There are some minor difference in the two combined-removed-dex files,
but these diffs do not amount to any diffs in the generated
hiddenapi-flags.csv file. See the full set of diffs here:
https://paste.googleplex.com/6632343525654528

Bug: 158465496
Test: diff hiddenapi-flags.csv before and after
Change-Id: I082d18fc2b8a57ea8c5941d5c955a7970ab9d860
2020-10-07 12:28:23 +01:00
Colin Cross
56a8321c21 Remove global state from apex modules
A global variant was used to store the global mapping between
modules and APEXes.  Replace it with storing pointers to APEX
contents inside each module so that they can query the contents
of any APEXes they belong to.

Bug: 146393795
Test: all Soong tests
Test: single line change to build.ninja host install dependency ordering
Test: no Android-${TARGET_PRODUCT}.mk, make_vars-${TARGET_PRODUCT}.mk or late-${TARGET_PRODUCT}.mk
Change-Id: Id2d7b73ea27f8c3b41d30820bdd86b65c539bfa4
2020-10-06 13:39:57 -07:00
Anton Hansson
a2adc37fc9 Make hiddenapi use prebuilt stubs when it should
Bug: 160455085
Test: build_unbundled_mainline_module.sh (with disabled source-stubs)
Test: hiddenapi_singleton_test.go
Change-Id: Id93c974351b3f8efdf8e4efe4192d3809f4fcaa5
2020-09-18 08:06:58 -07:00
Liz Kammer
88d593d942 Merge "Add hidden_api for java_import" 2020-09-14 13:57:09 +00:00
Liz Kammer
5ca3a6293e Add hidden_api for java_import
Test: go java tests
Test: m
Bug: 160455085
Change-Id: Ib6e826e32ca73ceea0799b26145ad06b1e62a1bf
2020-09-10 07:56:03 -07:00
Xin Li
65cb5c69ea Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)
Bug: 166295507
Merged-In: Ifca7b65f4e27bf14cdc30f72f790b0de90130bae
Change-Id: I3a39be5f0b8736de4822c6a14072c78d4e4ad89d
2020-08-29 01:17:45 -07:00