Commit graph

91 commits

Author SHA1 Message Date
Cole Faust
021bf3d973 Reapply "Make the enabled property configurable"
Previously, I had changed some loadhook-appended property structs
to use selects instead of the "target" property struct. This seems
to not be exactly equivalent because "target" properties are merged
with the regular properties later, at the time the arch mutator runs.

With this reapplication, leave those target property structs alone
to avoid breakages, but I'll have to look into what the issue is
with them later.

This reverts commit ed5276f082.

Ignore-AOSP-First: This cl needs to be in a topic with internal-only projects, will cherrypick to aosp after.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: If355d24506e3f117d27b21442a6c02bca3402dc7
2024-05-02 10:56:36 -07:00
Priyanka Advani
ed5276f082 Revert "Make the enabled property configurable"
Revert submission 27162921-configurable_enabled_property

Reason for revert: Droid-monitor created revert due to Build breakage in b/338253720. Will be verifying through ABTD before submission.

Reverted changes: /q/submissionid:27162921-configurable_enabled_property

Change-Id: I2d144f9d297373a13a1190b173d10c966181ad84
2024-05-01 20:02:41 +00:00
Cole Faust
0e0d749062 Make the enabled property configurable
This allows using select statements with it.

Ignore-AOSP-First: This needs to be in a topic with changes in interal-only repositories, I'll cherrypick to aosp after.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I6f3efaaa3d82505e38a91ee4ba0e18e404360191
2024-04-30 10:56:12 -07:00
Jihoon Kang
6d6d067c37 Remove extractEncodedDexJarsFromModules
The usages of this function have been removed with
https://r.android.com/2816477 and https://r.android.com/2816578, and
this function is currently unused.

Test: Presubmits
Change-Id: Ic1ae71ade16b930b777fa251febc4d071c0dd65e
2024-04-17 18:19:14 +00:00
Cole Faust
22e8abcaa6 Make hiddenAPIFlagFileCategory an int
hiddenAPIFlagFileCategory used to contain function pointers, and is
used in a provider. Providers must be serializable for incremental
soong support, so the function pointers must be removed.

Refactor hiddenAPIFlagFileCategory into a simple int.

Bug: 322069292
Test: m nothing --no-skip-soong-tests
Change-Id: I2bd50fa1b59979f30869b405314cbef16ee345f1
2024-01-23 18:01:47 -08:00
Treehugger Robot
193bb8c317 Merge "Add better error handling to bootDexJar function" into main 2024-01-18 20:38:13 +00:00
Spandan Das
3a3920129d Add better error handling to bootDexJar function
aosp/2876754 missed adding error handling to one of the places which
consumes the dex jar deapexed from prebuilt apexes. This CL ensures that
when we have multiple prebuilt apexes _without_ flagging, the hiddenapi
processing code emits a more descriptive error "Multiple deapxers..."
rather than a cryptic error "<module> does not provide a dex jar"

Test: m nothing --no-skip-soong-tests

Change-Id: I80849fa7cca17d724ac53c94bb353b169175ee28
2024-01-17 20:59:43 +00:00
Jihoon Kang
bd093457e2 Enable hiddenapi check for exportable stubs
This change modifies the dependencies of the hiddenapi to always depend
on the exportable stubs, instead of the currently utilized everything
stubs.

To support this, the full api surface exportable stubs are defined in a
separate change at the `frameworks/base` project. Note that the full api
surface exportable stubs are only used for the hiddenapi purpose, and
`sdk_version` continues to utilize the currently existing everything
stubs.

Currently, this feature is hidden behind the build flag
"RELEASE_HIDDEN_API_EXPORTABLE_STUBS". This feature will be fully
enabled once metalava fully supports handling of the flagged apis.

Test: ENABLE_HIDDENAPI_FLAGS=true m
Bug: 317426356
Change-Id: I109b7cd27b20ceffcdf1766ab8106b0c276be2b3
2024-01-12 01:03:10 +00:00
Spandan Das
59a4a2b8d2 Replace panic with ModuleErrorf
This is a followup cleanup for aosp/2876754 and replaces panic with
ctx.ModuleErrorf. The latter creates a more expressive build error.

Implementation details
- export moduleErrorf interface from build/soong/android. This minimal
  interface will be used as a parameter for `DexJarBuildPath`
- Add ModuleErrorf to the function signature of DexJarBuildPath. This
  parameter only gets used for Import and SdkLibraryImport structs.
  These two can have duplicate deapexer definitions, and ModuleErrorf
  will be used to report that error
- Create a minimal implementation of `ModuleErrorf` in tests of java and
  apex

Test: m nothing --no-skip-soong-tests
Change-Id: I0febec651f40c3f04deb957e64133c94b80fbd78
2024-01-09 22:53:52 +00:00
Spandan Das
64c9e0ce6e Use the correct bootjars for hiddneapi when multiple prebuilts exist
hiddenapi processing require boot jars from apexes to determine the full
set of methods available at runtime.
When building with prebuilts, this comes via
java_import/java_sdk_library_import, which acts as a hook for
prebuilt_apex/apex_set. If we have multiple apexes in the tree, this
hook becomes 1:many. This CL prepares platform_bootclasspath to select the right
deapexerd .jar files when mutliple prebuilts exist.

Implementation details
- Create a dependency edge from platform_bootclasspath to
  all_apex_contributions (DepsMutator)
- For every boot jar, query all_apex_contributions to get the path to
  dexjar file (GenerateAndroidBuildActions)

Some other important details
- This CL does not drop the old mechanism to get the dex file (i.e. by
creating a dep on java_library). Once all mainline
modules have been flagged using apex_contributions, the old mechanism
will be dropped
- This CL has a functional change when building with source apexes. At
  ToT, the unecoded hiddenapi dex jar is used for package check and
  generating the monolithic stub file. After this change, the hiddenapi
  encoded file will be used for these operations.
  This should be fine since the
  package and dex signature do not change across the encoded and
  unencoded dex file. In fact, we already have a split today. When
  building with prebuilts, we use the encoded dex file. When building
  with source, we use the unecoded dex file.

Test: Added a unit test
Test: Manual testing in internal described below
- lunch cf_x86_64_phone-next-userdebug
- flagged com.google.android.adservices using apex_contributions
- aninja -t commands out/soong/hiddenapi/hiddenapi-stubs-flags.txt # no
  diff before and after
Bug: 308790777

Change-Id: I72c70f0ae1b587679203ea254c9c12a48e7aa782
2024-01-02 17:56:14 +00:00
Spandan Das
5be6333692 Use the correct bootjars when multiple prebuilt apexes exist
hiddenapi and dexpreopt require boot and system server jars from apexes.
When building with prebuilts, this comes via
java_import/java_sdk_library_import, which acts as a hook for
prebuilt_apex/apex_set. If we have multiple apexes in the tree, this
hook becomes 1:many. This CL prepares dex_bootjars to select the right
deapexerd .jar files when mutliple prebuilts exist.

Implementation details
- Update prebuilt module types (prebuilt_apex/apex_set) and source
  apexes to set a map of
  library name to dex jar path on host.
- dex_bootjars will access the path of the .dex jar on host via the
  provider. These then
  copied/installed to the right locations.

This CL does not drop the old mechanism to get the dex file (i.e. by
creating a dep on java_library). Once all mainline
modules have been flagged using apex_contributions, the old mechanism
will be dropped

Bug: 308790457
Test: git_master-art-host:art-gtest https://android-build.corp.google.com/builds/abtd/run/L21500030000926533
Test: git_main:art_standalone_dexpreopt_tests https://android-build.corp.google.com/builds/abtd/run/L99000030000891212
Test: Added a unit test that checks that the right .jar is selected
when multiple prebuilts exists

Change-Id: I6ef94135b9303a35135810930af4b641df13a583
2023-12-20 16:43:01 +00:00
Colin Cross
313aa5475f Convert OtherModuleProvider to generic providers API
Convert all of the callers of OtherModuleProvider/OtherModuleHasProvider
to use the type-safe android.OtherModuleProvider API.

Bug: 316410648
Test: builds
Change-Id: Id77f514d68761a262d9ea830a601dbed804bbbe5
2023-12-14 16:12:22 -08:00
Colin Cross
bc7d76cca2 Convert NewProvider/NewMutatorProvider to generic providers API
Convert all of the callers to NewProvider and NewMutatorProvider
to use a generic type parameter instead of an example object.

Bug: 316410648
Test: builds
Change-Id: Ic9cdafc87336e26730d3fd596df05de0e7267542
2023-12-14 16:12:20 -08:00
Alyssa Ketpreechasawat
7daf2786b6 Disable verify_overlaps test instead of hiddenapi check.
Disabling hiddenapi check entirely can be dangerous and it might change the
accessibility of @hide Api too. We should only disable verify_overlaps
test which is required for module_sdk and apexes mismatch.

Test: atest CtsInlineMockingTestCases
Bug: 308187068
Bug: 307480026
Change-Id: I44f48e85d802b3a8f2711bc2047bdab721369f08
2023-11-06 09:49:16 +00:00
Jihoon Kang
244d42a91b Utilize module lib and test api superset module in hiddenapi
Hiddenapi takes a single widest api scope stub dex jar as an input, as
the tool does not support handling duplicate classes passed as inputs.
A problem regarding this is that the test and module lib api surfaces do
not strictly have a subset/superset relationship, unlike other api
surfaces.

This has not become a problem for stubs generated from source
files as the stubs contain all methods in the source files, but became a
problem for stubs genereated from text files as the stubs only contain
the methods that are essential for compilation of the stubs and its
reverse dependencies, and there were cases where the hiddenapi flags are
not properly propagated to the subclasses.

To resolve this problem, a java_api_library module that provides the
union of the test and the module lib api surfaces was introcudes. Since
hiddenapi_modular currently defines the module lib api surface to be a
wider api scope over the test api scope, the new module can be passed as
input to hiddenapi over the module lib non updatable stub module to
resolve the problem.

Test: enable hiddenapi for from-text stub build && ENABLE_HIDDENAPI_FLAGS=true m --build-from-text-stub
Bug: 191644675
Bug: 275570206
Change-Id: I9a230ec5082c52ed866f29b0748814f2cf10279b
2023-10-11 17:06:23 +00:00
Mårten Kongstad
bd2624442b Remove metalava --no-banner argument
metalava no longer prints an ASCII banner, and has removed its
--no-banner argument. Update all call sites accordingly.

Test: presubmit
Bug: 286023667
Change-Id: I61cf5f1125dfbd93496e6a59eb1ea62886b32046
2023-07-12 14:07:01 +02:00
Jihoon Kang
6c0df88809 Cleanup remaining android.JavaApiLibraryName() references
Since the name of the java_library generated from sdk_library per api
scope does not depend on the build configuration anymore, all
dependency switching "magic" via android.JavaApiLibraryName() can be
removed.

This change also removes from-text-build-specific test cases, as those
test cases depend on build configurations.

Test: m nothing && m nothing --build-from-text-stub
Bug: 287340610
Change-Id: I3bac35259e0cbaa16432a46cb2b128951c9bc075
2023-06-15 19:42:34 +00:00
Jihoon Kang
91c839566e Cleanup android.JavaApiLibraryName references
This change cleans up references to android.JavaApiLibraryName() so that
rdeps link against "android_*_stubs_current" instead of
"android_*_stubs_current.from-text".

Bug: 284995489
Test: m
Change-Id: Ibdf33b15981317c43e468dbc1657229d54bbd226
2023-06-05 21:45:01 +00:00
Jihoon Kang
7e9e20e389 Fix bootclasspath_fragment build failure during from-text stub build
This change enables bootclasspath_fragment module to be correctly built
during from-text stub build by android-non-updatable module to be
interpreted as its java_api_library equivalent.

Test: go test ./java
Bug: 279460171
Change-Id: Ia6b60edcb60782977cc59855f1cf5d3a46e9f682
2023-05-22 07:13:25 +00:00
Jihoon Kang
b507831b71 Utilize from text core platform api surface jar in build
Use JavaApiLibraryName function to redirect the usage
of core platform api stubs from .txt files based on config.

Test: m --build-from-text-stub
Change-Id: I926a0a455fed301ba4ff9dfa509d4dbbbd076029
2023-04-06 18:34:32 +00:00
Spandan Das
8c9ae7ed67 Update min_sdk_version from SdkSpec to ApiLevel
This relands aosp/2457063. The original change broke T and U since those
branches still contain soong modules of type (kind+level). Those soong
modules have been cleaned up now

Test: Used go/abtd to test T and U branches with this change

Bug: 208456999
Change-Id: I0ef7933c055f88cb512a02108f1173e51156ef1c
2023-03-22 20:15:45 +00:00
Spandan Das
626a8ad934 Cleanup hardcoded references to android_*stubs_current
These hardcoded refs will need to be updated when we start using .txt
stub equivalent (single-tree/multi-tree). Instead of strewing this logic
all over the codebase, create a helper function that contains the
replacement logic. All other places should call this helper function
instead of calculating the name of .txt equivalent soong module by
itself.

(Will do a similar cleanup in build/make)

Test: no change in ninja file

Change-Id: I6bf999eb4aeaba6ac2a44b9016bae4ec8c79ce19
2023-03-20 20:50:03 +00:00
Cole Faust
18994c73f1 Replace SortedStringKeys with SortedKeys
Now that we have generics.

Bug: 193460475
Test: presubmits
Change-Id: I1594fd8feb505175d5c09c03ef397e5ffd5b09cb
2023-02-28 16:51:32 -08:00
Paul Duffin
458a15bd8a Remove IsModuleInVersionedSdk
Previously, as all versioned sdk snapshots have been removed from
Android.bp files this method would always return false. This change
effectively replaces all calls to it with false, and then optimizes
away any unused code.

Bug: 260237150
Test: m nothing
Change-Id: I1b717ee8345e807bd888451f6e7e3c3a0d391ee2
2022-12-12 17:33:05 +00:00
Paul Duffin
1938dba8b6 Exclude unsupported libraries from sdk snapshot
When an sdk snapshot is targeted at release X then it cannot include
bootclasspath fragment libraries which are not present in that build as
otherwise it causes build failures. It should also not include any
unsupported libraries, i.e. libraries that cannot work on that release.

This change causes sdk snapshot to exclude libraries that have a
    min_sdk_version > target build release

It also ensures that hidden API flags do not include any information
from excluded libraries.

Bug: 240406019
Test: BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
      # Ran the previous command with and without this change to make
      # sure that this change excludes framework-connectivity-t library from the
      # tethering sdk snapshot for S, including from the hidden API flag files.
Change-Id: I57969b85a12e9e5a3fc76c055b260cec5d5f7d7f
2022-09-14 16:47:18 +01:00
Paul Duffin
887efdd779 Separate hidden API flag info for sdk from platform_bootclasspath
Previously, the sdk snapshot and platform_bootclasspath both used the
HiddenAPIInfo/Provider to retrieve the hidden API related information
they needed. This change separates them by adding a new
HiddenAPIInfoForSdk/Provider that is used to provide the information
needed by the sdk snapshot. This allows a follow up change to generate
different information for the sdk and for the platform depending on the
target build release of the sdk snapshot.

Bug: 240406019
Test: packages/modules/common/build/mainline_modules_sdks.sh
      # Ran the previous command with and without this change to make
      # sure that this change does not change the sdk snapshot
      # contents.
Change-Id: I30cea6cf8dae1be467b7063430d6197e5baa684b
2022-09-14 16:37:12 +01:00
Paul Duffin
af70518189 Separate hidden API flag generation from encoding
Previously, the rules for encoding hidden API flags into dex files were
generated in the hiddenAPIRulesForBootclasspathFragment alongside the
rules for creating the hidden API flag files.

This change separates the encoding from the generation of the flags.
This will allow a follow up change to generate API flags for the sdk
snapshot separately without affecting the encoded dex files.

Bug: 240406019
Test: m nothing
Change-Id: I3ab3ff5780e7b2e322665d3ea15f1438f2e403bd
2022-09-14 16:07:28 +01:00
Paul Duffin
3f1ae0b55a Add hidden API properties to java_sdk_library modules
Previously, hidden API properties were only allowed on
bootclasspath_fragment and platform_bootclasspath module types. This
change allows them to be specified on java_sdk_library modules too. It
involves the following changes:

1. Add the properties to the java.Module.
2. Populate and provide a HiddenAPIPropertyInfo struct from
   java_sdk_library modules.
3. Modify bootclasspath_fragment to merge information gathered from its
   content libraries as if it was specified on the fragment itself.

Bug: 240406019
Test: m nothing
      packages/modules/common/build/mainline_modules_sdks.sh
      # Ran the previous command with and without this change to make
      # sure that this change does not change the sdk snapshot
      # contents.
Change-Id: I64eb71c2039ddc14cf380689d0cec7ec221f5b88
2022-08-19 16:45:38 +00:00
Paul Duffin
9b61abbd84 Push Hidden_api inside HiddenAPIFlagFileProperties
All usages of HiddenAPIFlagFileProperties use it as the type of a
Hidden_api field. This change pushes the field inside the struct and
just embeds the struct inside the structs that need it making it
simpler and more consistent with how the HiddenApiPackageProperties
struct is used.

This is extracted as a separate change as while the change is simple
it does affect a lot of lines. Keeping it separate makes it easier to
review the changes.

Bug: 240406019
Test: packages/modules/common/build/mainline_modules_sdks.sh
      # Ran the previous command with and without this change to make
      # sure that this change does not change the sdk snapshot
      # contents.
Change-Id: I664453029574ef68dc5712f7bf137a1a6e78e864
2022-08-19 13:29:47 +00:00
Paul Duffin
1e9e9388d8 Refactor handling of hidden API property provided information
In preparation for adding hidden API properties to individual
java_sdk_library modules this change refactors the handling of that
information.

1. Renames HiddenApiPackageProperties to HiddenAPIPackageProperties
   to be consistent with other similar structs.

2. Creates a new HiddenAPIPropertyInfo struct to be used to provide
   property derived information from one module to another. It includes
   both flag files as well as the information about which packages are
   used.

3. The HiddenAPIPropertyInfo is embedded within the existing
   HiddenAPIFlagInput in place of the existing flag files field.

4. The fields in the HiddenAPIPropertyInfo struct are populated from
   the HiddenAPI*Properties structs.

5. Access to package information is retrieved from the info struct
   instead of directly from the properties. That means that no changes
   will be needed to this code when info from other modules is merged
   into the struct in a follow up change.

Bug: 240406019
Test: packages/modules/common/build/mainline_modules_sdks.sh
      # Ran the previous command with and without this change to make
      # sure that this change does not change the sdk snapshot
      # contents.
Change-Id: I773279a4b621bcc3c40e3bfe193f2c7b0caeccd6
2022-08-19 13:28:47 +00:00
Paul Duffin
09817d66de hiddenapi: Prevent libraries for Q/R from include S+ flags.
The Q and R runtimes can handle Q/R flags but not S flags. So, this
change verifies that any library that can run on Q/R
(min_sdk_version <= R) by adding --max-hiddenapi-level=max-target-r
to the "hiddenapi encode" command. That will cause a failure if any
S+ flags are found in the flags to encode.

Bug: 172453495
Test: m droid && launch_cvd
      Cherry pick changes in https://r.android.com/q/topic:max-target-s
      Add @UnsupportedAppUsage maxTargetSdk=S in classes in framework-permission (for r/q)
      and framework-permission-s (nominally for S+). I had to incresed the min_sdk_version
      in the latter to 31 (S) as it was still set at 30 (R).
Change-Id: Ie0f68482603adc7b4e3d7a5c81bf203d81a84a9e
2022-04-28 18:34:48 +01:00
Paul Duffin
bd88c882f6 Remove implementation details from stub flags in sdk snapshot
Previously, the build applied the same filtering to remove
implementation details from the sdk snapshot's stub-flags.csv file as
it did for its all-flags.csv, i.e. removing the signatures that only
had a "blocked" flag. Unfortunately, that had no effect on the stub
flags as the implementation signatures had no flags, not a single
blocked flag. That meant that the sdk snapshot's
filtered-stub-flags.csv file contained a lot of implementation details.

This change removes signatures from stub-flags.csv that have no flags
which removes all implementation details from the sdk snapshot.

Bug: 194063708
Test: atest --host verify_overlaps_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      m art-module-sdk
      # Check contents of its filtered-stub-flags.csv file
Change-Id: I30edc77348fad118ea732e787ae8e206c8841f84
2022-04-12 17:44:15 +01:00
Paul Duffin
0c12b78ae8 Use named options for verify_overlaps
Previously, verify_overlaps used positional arguments, the first was
the monolithi flags and the rest were the module flag pairs (filtered
flags file and signature patterns file). This change makes them use
named options to make the purpose of the arguments clearer on the
command line.

Bug: 194063708
Test: atest --host verify_overlaps_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
Change-Id: Ife0af0016eb0f91416e8330d5d98cb53c97d68a4
2022-04-08 10:56:21 +01:00
Paul Duffin
846beb7f54 Make bootclasspath_fragment hidden API package checks exhaustive
Previously, the bootclasspath_fragment's hidden_api.split_packages and
hidden_api.package_prefixes properties did not specify an exhaustive
set of packages that were provided by the fragment. They excluded
packages which were either not split or which could not be used as a
package prefix because it would match sub-packages provided by other
bootclasspath modules.

This change adds the hidden_api.single_packages list to specify those
additional packages and then uses that information to verify that any
bootclasspath_fragment that specifies at least one of split_packages,
single_packages or package_prefixes properties only contains classes
from a package that matches one of those properties. That will
prevent a module from accidentally including unexpected classes, such
as might happen when statically including a common utility library.

It also adds coverage specific versions of the properties as additional
packages are added to the art-bootclasspath-fragment when building
coverage builds.

Bug: 194063708
Test: atest signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      m EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true out/soong/hiddenapi/hiddenapi-flags.csv
      # Breaks without corresponding change to add android.system to
      # the art-bootclasspath-fragment.
      /usr/bin/pylint --rcfile $ANDROID_BUILD_TOP/tools/repohooks/tools/pylintrc scripts/hiddenapi/signature_patterns*.py
      pyformat -s 4 --force_quote_type single -i scripts/hiddenapi/signature_patterns*.py
Change-Id: Iddf6c59cd4dc8c36dde7943a9840ccef5794b320
2022-03-18 10:54:51 +00:00
Paul Duffin
1e18e98b14 Allow implementation details to be excluded from signature patterns file
Previously, the signature-patterns.csv file (which was output in an sdk
snapshot) included implementation details, e.g. the names of classes
that are not part of any API, including the hidden API. This change
allows a bootclasspath_fragment module owner to customize the generated
signature patterns file to remove as many implementation details as
possible from the sdk snapshot. That means that implementation only
changes would not require that sdk snapshots be updated in sync with
the corresponding APEX.

Module owners can remove implementation in the following ways:
1) Limit the number of packages that are treated as split packages by
   explicitly specifying the split packages in the split_packages
   property. A split package is one whose classes are provided by
   separate bootclasspath_fragment modules and so the signature
   patterns has to include every class in that package provided by a
   specific bootclasspath_fragment module, including implementation
   classes, instead of just listing the package.

   The default is to treat all packages as being split, so if no
   split_packages is specified then it defaults to ["*"] which matches
   all packages. Assuming that no package was split unless specifically
   stated would require that all the modules that had split packages
   explicitly list them before this change could be submitted as
   without them this change would break the build.

   Once all existing modules with split packages have been updated to
   include them then the default may be changed.

2) Adding package prefixes for any hierarchy of packages that are
   owned solely by that bootclasspath_fragment. This removes the need
   to list the packages in that hierarchy, including implementation
   specific packages.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: If2f90cfc41b6fff9fa4ac8b9d4973110938b9392
2021-10-05 15:47:05 +01:00
Paul Duffin
191be3a186 Retry: Separate hidden API flags needed in sdk snapshots for S and T
Previous change that was reverted: https://r.android.com/1835222
An additional test was added that revealed a bug in the previous change
which has been fixed here.

Previously, the behavior of the stub_flags and all_flags properties
was different between S and T. In S they contained paths for the
complete set of stub flags and all the encoded flags. However, in T
they contained filtered sets of flags which if used in S would prevent
build checks from detecting possible inconsistencies. Also, a new
signature_patterns property was added in T that is not supported in S.

This change creates separate properties/files for T and reverts the
behavior of the properties/files that were added in S back to how they
behaved in S. The new properties are called filtered_stub_flags and
filtered_flags.

The S and T properties are tagged with the appropriate
supported_build_releases tag to ensure that they are only output when
specifically targeted.

Bug: 197842263
Test: m nothing
Change-Id: I1ce0a3d6623dabf73e32af1a7457b9b444fc3b7c
2021-10-05 15:27:42 +01:00
Salmax Chang
8194d61ae0 Revert "Separate hidden API flags needed in sdk snapshots for S and T"
This reverts commit ba68c93630.

Reason for revert: suspect to cause build break in b/201948713

Change-Id: Ide3693ac1f8e24bd2f7c12857361885cb6b787ef
2021-10-04 05:28:58 +00:00
Paul Duffin
ba68c93630 Separate hidden API flags needed in sdk snapshots for S and T
Previously, the behavior of the stub_flags and all_flags properties
was different between S and T. In S they contained paths for the
complete set of stub flags and all the encoded flags. However, in T
they contained filtered sets of flags which if used in S would prevent
build checks from detecting possible inconsistencies. Also, a new
signature_patterns property was added in T that is not supported in S.

This change creates separate properties/files for T and reverts the
behavior of the properties/files that were added in S back to how they
behaved in S. The new properties are called filtered_stub_flags and
filtered_flags.

The S and T properties are tagged with the appropriate
supported_build_releases tag to ensure that they are only output when
specifically targeted.

Bug: 197842263
Test: m nothing
Change-Id: Iec8b9c539796c507245b69c0aed980fde6d8694f
2021-10-01 17:28:13 +01:00
Martin Stjernholm
8be1e6db16 Propagate the dex jar path as an OptionalPath which is either valid or
invalid with a message.

This will allow propagating any error from the deapexer module for
prebuilt APEXes to the location where the dex jars get used. It's only
at those points that we can raise errors about not being able to
extract files from the deapexer modules if they are invalid, and this
way we avoid encoding knowledge there about why they may be invalid.

To keep the refactoring limited it intentionally does not change any of
the existing logic for when dexJarFiles are set or not (non-nil vs nil
prior to this change), although there may be opportunity to use this
for more conditions when dex jars aren't available.

The refactoring is also not extended to
dexpreopt.ClassLoaderContextMap.

Test: m nothing
Bug: 192006406
Change-Id: I68986dccd9a9b3fee4d24caa1947ea17a36caedc
2021-09-23 17:19:55 +01:00
Martin Stjernholm
531d8fdc2a Remove workaround for modules with missing BCP fragments, as they are
present now.

#codehealth

Test: presubmits
Bug: 192868581
Change-Id: I8370525849ea589af0bf4f8d3fd081f3004516a1
2021-09-20 23:47:24 +01:00
Paul Duffin
f7b3d0d317 Rename SdkMemberTypeDependencyTag to SdkMemberDependencyTag
Bug: 195754365
Test: m nothing
Change-Id: Id4ef6f8be54c60a1f269d7e7c46a8dcb715fcca4
2021-09-14 17:26:14 +01:00
Paul Duffin
280bae6d20 Filter blocked entries from modular flag files
Previously, the sdk snapshot would include all the entries from the
stub-flags.csv and all-flags.csv modular files generated by a single
bootclasspath_fragment. That included implementation details, i.e.
class members that are not part of a stable API or the hidden API which
meant that the sdk snapshots were implementation dependent.

This change removes the implementation details from the modular flag
files, i.e. those entries that are only marked as "blocked". When
comparing the files against the corresponding subset of the monolithic
files it assumes that any entries missing from the modular flag files
are blocked.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I6b67b2253cf029d6830b58a06ebb0c8fcaa0dd71
2021-08-11 17:29:47 +01:00
Paul Duffin
67b9d61ac2 Separate creation of signature patterns from overlap checking
Previously, the signatures used to select the subset of the monolithic
flags were simply the signatures read from the modular flags file. This
change moves the creation of the signature list into a separate script
that outputs the signatures to a file and then passes the path through
Soong from the bootclasspath_fragment modules that create it to the
platform_bootclasspath module that uses it to compare the modular
flags against the monolithic flags.

Currently, the signatures are the full signatures but follow up changes
will replace them with patterns (hence the name) that avoids having to
include implementation details in the hidden API flags that are output
as part of a bootclasspath_fragment's snapshot.

This change moves the stub flags related code next to the all flags
related code as they are treated in a similar way.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I2855bf6d05c91b8a09591664185750361c7e644f
2021-08-06 13:40:34 +01:00
Paul Duffin
6e9bc01b97 Remove unnecessary dedup logic
The CreateClasspathElements function makes sure that each standalone
library and bootclasspath_fragment on the platform bootclasspath
appear once and once only and each file extracted is unique to the
module so there is no chance that any duplicate files will be used.

Bug: 194063708
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - check that this does not change the contents.
Change-Id: I60c02baa4d179293ee0f305375afaa3a8c98e112
2021-07-20 22:02:44 +01:00
Paul Duffin
da286f4615 Make bootclasspath_fragments always perform hidden API processing
Previously, bootclasspath_fragment modules would only perform hidden
API processing if they provided some stub libraries and fragments. That
was needed because the bootclasspath_fragment modules were added before
Soong supported hidden API processing on all the different modules and
before they all provided the necessary information that hidden API
processing required.

This change stops hidden API being conditional as it is no longer
required as it has been enabled on all existing bootclasspath_fragment
modules.

Bug: 179354495
Test: m nothing
Change-Id: I0cbf11986adff1f2f967b96f86e6bfe0e9b8b1ef
2021-07-16 15:09:59 +01:00
Paul Duffin
4b51e27046 Merge "Defer error reporting of missing prebuilt dex jar files" 2021-07-01 10:55:16 +00:00
Paul Duffin
3f0290ef79 Support using java_sdk_library components in stub_libs
Previously, if a bootclasspath_fragment had both a java_sdk_library
module and one of its components in stub_libs properties they would be
treated as separate modules instead of simply different APIs from the
same module. That would result in them both providing stub dex jars to
"hiddenapi list" which would fail because it found duplicate
definitions of the same class.

e.g. Specifying something like this:
    api: {
        stub_libs: [
            "art.module.public.api",
        ],
    },
    core_platform_api: {
        stub_libs: [
            "art.module.public.api.stubs.module_lib",
        ],
    },

would cause "hiddenapi list" to fail because it would have been passed
paths to two dex jars (actually the same dex jar but that does not
matter) each of which defined the same class, e.g. java.lang.Object.

This change treats the "art.module.public.api.stubs.module_lib" and
"art.module.public.api" modules as being the same for the purposes of
hidden API processing.

Bug: 192446466
Test: m nothing
Change-Id: I9de96337f64f26e24cff040d4bbed9eecc67b1ed
2021-06-30 19:36:52 +01:00
Paul Duffin
ef083c9556 Defer error reporting of missing prebuilt dex jar files
Unless the prebuilt dex jar files are explicitly required to build
system images or dex preopting defer reporting of any missing dex boot
jars from Soong to Ninja.

This will prevent builds that contain prebuilt sdks, containing prebuilt
bootclasspath_fragment modules but without a corresponding apex_set or
prebuilt_apex module from failing even when it is not building against
the prebuilt sdk.

Bug: 179354495
Test: m nothing
Change-Id: Ibde3bf840a7413785cd32bd6cea1c322f90c59af
2021-06-29 13:41:03 +01:00
Paul Duffin
280a31aac3 Calculate widest stub dex jars per module
Previously, the stub dex jars for each HiddenAPIScope was created by
merging the stub dex jars provided by each module for that scope. Then
the widest stub dex jars were chosen. So, if module A provided public,
system and test stub dex jars and module B provided only public then
the stub dex jars for each scope would be:
* public -> A,B
* system -> A
* test -> A

So, the widest API scope for which there are stub dex jars is "test"
and so the widest stub dex jars would just come from module A and not
module B. So, when "hiddenapi list" is run for module C which depends
on modules A and B it only gets given stub dex jars for module A which
means that it cannot resolve all the types that C may use which can
lead to incorrect flags being generated.

This change does not merge the stub dex jars from each module together
and instead keeps them separate by module. The widest stub dex jars
list is constructed by asking each module in turn for their widest stub
dex jars. e.g. Given the above example we would have:

Module A:
* public
* system
* test <- widest

Module B:
* public <- widest

So, the widest stub dex jars will be A's test and B's public stub dex
jars.

Bug: 179354495
Test: m out/soong/hiddenapi-flags.csv
      - make sure that this does not change the file.
Change-Id: Ib137825ebffe94b2bf220732bae6077f7b7ac6db
2021-06-28 10:56:43 +01:00
Paul Duffin
d2b1e0ca92 Add test to show issues with widest stub dex jars
The widest stub dex jars should include the widest stub dex jars
provided by each module. So, if module A has public, system and test
and module B has only public then the widest stub dex jars should
include module A's test and module B's public stub dex jars. Instead,
they just include module A's test.

That behaviour is needed so that when the "hiddenapi list" tool is run
against a module C that it is passed stub dex jars from both module A
and module B so that any references to the types provided by those APIs
can be resolved.

A follow up change will fix this issue.

Bug: 179354495
Test: m nothing
Change-Id: Ibd31964e8d2a33fa92fbd0b800c9fe054ee359c7
2021-06-27 22:46:14 +01:00