Commit graph

49 commits

Author SHA1 Message Date
Paul Duffin
67b9d61ac2 Separate creation of signature patterns from overlap checking
Previously, the signatures used to select the subset of the monolithic
flags were simply the signatures read from the modular flags file. This
change moves the creation of the signature list into a separate script
that outputs the signatures to a file and then passes the path through
Soong from the bootclasspath_fragment modules that create it to the
platform_bootclasspath module that uses it to compare the modular
flags against the monolithic flags.

Currently, the signatures are the full signatures but follow up changes
will replace them with patterns (hence the name) that avoids having to
include implementation details in the hidden API flags that are output
as part of a bootclasspath_fragment's snapshot.

This change moves the stub flags related code next to the all flags
related code as they are treated in a similar way.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I2855bf6d05c91b8a09591664185750361c7e644f
2021-08-06 13:40:34 +01:00
Paul Duffin
6e9bc01b97 Remove unnecessary dedup logic
The CreateClasspathElements function makes sure that each standalone
library and bootclasspath_fragment on the platform bootclasspath
appear once and once only and each file extracted is unique to the
module so there is no chance that any duplicate files will be used.

Bug: 194063708
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - check that this does not change the contents.
Change-Id: I60c02baa4d179293ee0f305375afaa3a8c98e112
2021-07-20 22:02:44 +01:00
Paul Duffin
da286f4615 Make bootclasspath_fragments always perform hidden API processing
Previously, bootclasspath_fragment modules would only perform hidden
API processing if they provided some stub libraries and fragments. That
was needed because the bootclasspath_fragment modules were added before
Soong supported hidden API processing on all the different modules and
before they all provided the necessary information that hidden API
processing required.

This change stops hidden API being conditional as it is no longer
required as it has been enabled on all existing bootclasspath_fragment
modules.

Bug: 179354495
Test: m nothing
Change-Id: I0cbf11986adff1f2f967b96f86e6bfe0e9b8b1ef
2021-07-16 15:09:59 +01:00
Paul Duffin
4b51e27046 Merge "Defer error reporting of missing prebuilt dex jar files" 2021-07-01 10:55:16 +00:00
Paul Duffin
3f0290ef79 Support using java_sdk_library components in stub_libs
Previously, if a bootclasspath_fragment had both a java_sdk_library
module and one of its components in stub_libs properties they would be
treated as separate modules instead of simply different APIs from the
same module. That would result in them both providing stub dex jars to
"hiddenapi list" which would fail because it found duplicate
definitions of the same class.

e.g. Specifying something like this:
    api: {
        stub_libs: [
            "art.module.public.api",
        ],
    },
    core_platform_api: {
        stub_libs: [
            "art.module.public.api.stubs.module_lib",
        ],
    },

would cause "hiddenapi list" to fail because it would have been passed
paths to two dex jars (actually the same dex jar but that does not
matter) each of which defined the same class, e.g. java.lang.Object.

This change treats the "art.module.public.api.stubs.module_lib" and
"art.module.public.api" modules as being the same for the purposes of
hidden API processing.

Bug: 192446466
Test: m nothing
Change-Id: I9de96337f64f26e24cff040d4bbed9eecc67b1ed
2021-06-30 19:36:52 +01:00
Paul Duffin
ef083c9556 Defer error reporting of missing prebuilt dex jar files
Unless the prebuilt dex jar files are explicitly required to build
system images or dex preopting defer reporting of any missing dex boot
jars from Soong to Ninja.

This will prevent builds that contain prebuilt sdks, containing prebuilt
bootclasspath_fragment modules but without a corresponding apex_set or
prebuilt_apex module from failing even when it is not building against
the prebuilt sdk.

Bug: 179354495
Test: m nothing
Change-Id: Ibde3bf840a7413785cd32bd6cea1c322f90c59af
2021-06-29 13:41:03 +01:00
Paul Duffin
280a31aac3 Calculate widest stub dex jars per module
Previously, the stub dex jars for each HiddenAPIScope was created by
merging the stub dex jars provided by each module for that scope. Then
the widest stub dex jars were chosen. So, if module A provided public,
system and test stub dex jars and module B provided only public then
the stub dex jars for each scope would be:
* public -> A,B
* system -> A
* test -> A

So, the widest API scope for which there are stub dex jars is "test"
and so the widest stub dex jars would just come from module A and not
module B. So, when "hiddenapi list" is run for module C which depends
on modules A and B it only gets given stub dex jars for module A which
means that it cannot resolve all the types that C may use which can
lead to incorrect flags being generated.

This change does not merge the stub dex jars from each module together
and instead keeps them separate by module. The widest stub dex jars
list is constructed by asking each module in turn for their widest stub
dex jars. e.g. Given the above example we would have:

Module A:
* public
* system
* test <- widest

Module B:
* public <- widest

So, the widest stub dex jars will be A's test and B's public stub dex
jars.

Bug: 179354495
Test: m out/soong/hiddenapi-flags.csv
      - make sure that this does not change the file.
Change-Id: Ib137825ebffe94b2bf220732bae6077f7b7ac6db
2021-06-28 10:56:43 +01:00
Paul Duffin
d2b1e0ca92 Add test to show issues with widest stub dex jars
The widest stub dex jars should include the widest stub dex jars
provided by each module. So, if module A has public, system and test
and module B has only public then the widest stub dex jars should
include module A's test and module B's public stub dex jars. Instead,
they just include module A's test.

That behaviour is needed so that when the "hiddenapi list" tool is run
against a module C that it is passed stub dex jars from both module A
and module B so that any references to the types provided by those APIs
can be resolved.

A follow up change will fix this issue.

Bug: 179354495
Test: m nothing
Change-Id: Ibd31964e8d2a33fa92fbd0b800c9fe054ee359c7
2021-06-27 22:46:14 +01:00
Paul Duffin
156b5d3b61 Allow hiddenapi list to ignore missing classes/members
When generating the stub-flags.csv for a bootclasspath_fragment the
hiddenapi list tool is not given a complete set of all classes and
members. This change causes it to ignore them by passing the new
--fragment option to it.

This does not risk changing the flags as the stub-flags.csv files
created with the --fragment option are compared with the monolithic
out/soong/hiddenapi/hiddenapi-stub-flags.txt file which is not run
with this option to ensure that they match.

Bug: 179354495
Test: m out/soong/hiddenapi-stub-flags.csv
      - make sure that this does not change the file.
Change-Id: I890c7374c445759cade4d685f51e81261b7ccea2
2021-06-27 22:46:14 +01:00
Paul Duffin
99926a2a77 Merge "Support hidden API processing for modules that use platform APIs" 2021-06-25 18:25:39 +00:00
Paul Duffin
5cca7c44e5 Support hidden API processing for modules that use platform APIs
Previously, hidden API processing could only be done by those
bootclasspath_fragment modules that either did not depend on any other
fragments (e.g. art-bootclasspath-fragment) or only depended on APIs
provided by other fragments (e.g. i18n-bootclasspath-fragment). That
meant that modules like com.android.os.statsd-bootclasspath-fragment
that depended on APIs provided by parts of the platform which are not
yet part of another bootclasspath_fragment could not perform hidden
API processing.

This change adds support for a bootclasspath_fragment to specify the
additional stubs needed to perform hidden API processing. It adds a new
additional_stubs property that can be used to specify the additional
stub libraries.

Most bootclasspath_fragments that need to use the property will need
access to the APIs provided by the android-non-updatable.* libraries.
Rather than have each fragment explicitly specify the correct module
for each scope it treats "android-non-updatable" as if it was a
java_sdk_library that can provide different jars for each scope.
Soong will handle mapping that to the correct android-non-updatable.*
module.

Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv \
        out/soong/hiddenapi/hiddenapi-index.csv \
        out/soong/hiddenapi/hiddenapi-stub-flags.txt \
        out/soong/hiddenapi/hiddenapi-unsupported.csv
      - make sure that this change does not change the contents.
      m TARGET_BUILD_APPS=Calendar nothing
Change-Id: Ia8b79830ed0e6d42100de03d76b0c51b7f6c8ade
2021-06-25 16:20:19 +01:00
Paul Duffin
2e88097152 Verify the modular stub flags are subsets of the monolithic stub flags
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-stub-flags.txt
      - check that an error is reported if a modular stub-flags.csv file,
        i.e. one created by a fragment is not a subset of the monolithic
        file, e.g. because a signature in the modular file has different
        flags than it does in the monolithic or is not present there.
Change-Id: I46ebb495cb093a5e3abe7571c49933c845318549
2021-06-25 14:49:10 +01:00
Paul Duffin
4539a37a61 Make ruleToGenerateHiddenAPIStubFlagsFile build rule
Previously, the func created a rule and returned it for the caller to
create with the appropriate name and description. This change passes
the name and description into the func and causes it to create the rule
itself. The func is also renamed to make it more consistent with the
other similar rules.

Bug: 179354495
Test: m nothing
Change-Id: I2a4455daa8a6090ed5568994b255848d063e1ab2
2021-06-25 14:49:07 +01:00
Paul Duffin
b51db2ed8e Add ModuleLibHiddenAPIScope for hidden API processing
Although the hidden API runtime does not support a module-lib API flag
the hidden API processing does need to used them as they are needed by
the "hiddenapi list" tool for any bootclasspath_fragment module whose
contents builds against an sdk_version of "module_current". Without it
the "hiddenapi list" tool could fail to resolve references to classes
provided by other modules which would break the build.

Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - make sure that this change has no effect on the generated flags.
Change-Id: I3ecb80fdaeba0e66d1ee25cb57152ab546d9bfe0
2021-06-24 22:04:09 +01:00
Paul Duffin
31fad800a7 Add HiddenAPIScope to replace use of SdkKind
Previously, the hidden API processing used SdkKind to identify the API
scopes, e.g. public, system, etc. that are of interest for hidden API
processing. Unfortunately, there is a mismatch between the SdkKind and
what hidden API processing needs. e.g. SdkKind includes values that are
not used by hidden API processing and hidden API processing needs
additional API scope specific information not provided by SdkKind. The
apiScope struct used in sdk_library.go is also not a suitable
representation for similar reasons.

This change adds the HiddenAPIScope (following a similar approach as
apiScope) that rectifies that and uses it as a replacement for SdkKind
in most parts of the hidden API processing. The SdkKind is still used
for retrieving information from java_sdk_library[_import] modules.

Follow up changes will extend the HiddenAPIScope with more information.

Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - make sure that this change has no effect on the generated flags.
Change-Id: I97968f58535121652852b8d25217aa288afd2bfd
2021-06-24 21:56:27 +01:00
Paul Duffin
d061d40eb6 Fix monolithic hidden API processing with prebuilts
Prebuilt modules do not provide classesJars containing annotations.
Previously, the monolithic hidden API processing just used classesJars
from all the modules that provided them so when building against
prebuilts would have fewer classesJars than when building against
sources and so would produce different hidden API flags.

This change will generate the monolithic files from both classesJars
and files previously generated from hidden API processing. A fragment
that has performed hidden API processing will contribute its generated
files whereas standalone libraries and fragments which have not
performed hidden API processing will contribute classesJars.

Bug: 177892522
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      m SOONG_CONFIG_art_module_source_build=false out/soong/hiddenapi/hiddenapi-flags.csv
      - verify that the files are identical whether built from
        source or prebuilts.
Change-Id: I06f3c7df49626bec21a452bc9abf1bb9e7545e5c
2021-06-20 19:09:09 +01:00
Paul Duffin
7f8721618c Exclude jacocoagent from boot jars package check
This replicates the previous behavior when the boot jars package check
was performed by a singleton. When the singleton was removed and the
check was performed by the platform-bootclasspath the logic to exclude
jacocoagent from the list was lost. This change replaces it.

Bug: 191377804
Test: m EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true check-boot-jars
      - fails without this change passes with this change.
Change-Id: If9eae254045bef9a0c0963213721fc12a9f463cb
2021-06-17 19:33:24 +01:00
Paul Duffin
e521881bd4 Combine hidden API encoding with flag generation
Previously, the rules to perform hidden API encoding were generated
separately to the rules to perform hidden API flag generation. This
change combines them within the (renamed) produceHiddenAPIOutput()
method and makes the paths to the encoded dex files an output of the
generateHiddenAPIBuildActions method alongside the paths to the
generated flag files.

As encoded dex jars are now an output of the produceHiddenAPIOutput()
method which is implemented for both prebuilts and source
bootclasspath_fragment modules that necessitated the prebuilt also
providing paths to encoded dex files. That in turn required updates to
some of the tests to provide dex files from prebuilt_apex modules.

Similarly, as the produceHiddenAPIOutput() method may not be called
for some bootclasspath_fragment modules as they do not yet provide all
the information needed to perform hidden API encoding then it is
necessary to extract the encoded dex files produced by the modules
themselves. That also required a few changes to tests that did not
previously provide dex files.

Bug: 177892522
Test: m com.android.art
      - check that this change does not change the contents of the apex
        file, i.e. is byte-for-byte identical.
Change-Id: I60996a34d06ed1c87ed244ab3509621999ad86ec
2021-06-17 10:20:33 +01:00
Paul Duffin
dd5993f6d4 Avoid passing around []hiddenAPIModule
Previously, an []android.Module was converted to an []hiddenAPIModule
that was then used to retrieve boot dex jars. That was ok when
obtaining the dex jar files from source modules for
bootclasspath_fragment but does not work well for other use cases as
it would require doing that conversion in multiple places.

This change pushes the use of hiddenAPIModule down to the methods that
retrieve information from it which makes the methods more flexible and
easier to reuse.

Bug: 177892522
Test: m nothing
Change-Id: Ib84aaf03d8f5a63b48232036fe4589646fc23352
2021-06-14 17:23:52 +01:00
Paul Duffin
5f148ca7cf Make copyBootJarsToPredefinedLocations simpler and less fragile
Previously, copyBootJarsToPredefinedLocations relied on all its
parameters having the same length and the same order. That made it
quite fragile as changes to one of the parameters without corresponding
changes to the other would cause failures. It also combined the
retrieval of the boot dex jars from the modules, handling of missing
boot dex jar files and the generation of the rules to copy the files.

This change separates the retrieval of boot dex jars and handling of
missing files from the copying of those files while at the same time
making the function less fragile by replacing the three ordered
parameters with two maps that shared common keys.

Bug: 179354495
Test: m nothing
Change-Id: Idbcd24a7e8af89f7895a20aeddc58502dcbaad03
2021-06-10 14:28:10 +01:00
Paul Duffin
524c82c01a Export hidden api related types and fields
This will export some hidden api related types and fields so they can
be used from outside the java package. This is needed to allow a follow
up change to move the TestPlatformBootclasspath_Fragments from the java
to the apex package.

Bug: 179354495
Test: m nothing
Change-Id: Ib69eea9d79cc83b8e3fc29919a29f071e1ec17b5
2021-06-10 12:36:43 +01:00
Paul Duffin
32cf58a8fc Support removed API members in modular hidden API processing
Previously, the hidden API flags generated for a bootclasspath_fragment
did not include removed API members. That was because it did not supply
a file containing the dex signatures of the removed API members.

The monolithic hidden API processing uses combined-removed-dex which is
the output of a genrule that takes as input the *removed.txt files from
all the APIs and uses metalava to construct the dex signatures file.
This change does the equivalent for the *removed.txt files for the APIs
provided by a bootclasspath_fragment and then passes them to the rule
that generates the final all-flags.csv.

Bug: 179354495
Test: - Update packages/modules/RuntimeI18N to enable hidden API
        processing.
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - Before this change that fails as the flags generated for the
        i18n-bootclasspath-fragment differ from the monolithic flags.
	After this change that passes.
      - Verify that this change does not change any of the monolithic
        hidden API files.
      m com.android.i18n
      - Verify that the apex file before and after this change are byte
        for byte identical.
Change-Id: I6a21edb8a5231666e3f35b2c99a8687f36dd98fd
2021-05-24 15:05:21 +01:00
Paul Duffin
f1b358cb57 Support hidden API processing for fragments with dependencies
Previously, a bootclasspath_fragment that depended on classes provided
by another bootclasspath_fragment did not support hidden API processing
as it would not supply information about those dependencies.

This change adds support for that as follows. Each fragment:
1. Exports the transitive sets of stub dex jars for each of the public,
   system, test and core_platform APIs (where relevant).
2. Adds dependencies onto its dependent fragments.
3. Retrieves the API stubs dex jars from its dependent fragments and
   passes them to the "hiddenapi list" tool which will use them to
   resolve dependencies but will not output them to the generated
   flags.

Once the flags are generated the existing encoding functionality
encodes the flags into the dex files of the bootclasspath_fragment's
content modules which are then packaged into the apex.

Bug: 179354495
Test: m com.android.sdkext
      - verify that this does not change the contents of the apex files
Change-Id: I3e82a6dbb437f1e417e5d7e25aeb212e378603d0
2021-05-24 14:56:10 +01:00
Paul Duffin
18cf19745e Remove bootclasspathApiInfo
Previously, the bootclasspathApiInfo was only used for tests and follow
up changes will need to provide the stub dex jars. This change moves
the stubJarsByKind into HiddenAPIInfo and removes bootclasspathApiInfo
and the corresponding provider.

Bug: 179354495
Test: m nothing
Change-Id: I5459c56de561c053ed671dc9d5cb3ee4820c0ee8
2021-05-24 13:32:56 +01:00
Paul Duffin
6237092391 Fix hidden API flags in com.android.i18n
Change 70cfdff3da changed the hidden API
flags in com.android.i18n as it stopped the i18n-bootclasspath-fragment
from making the hidden API flag files available for use by
platform-bootclasspath.

This change fixes that by exporting the flag files even if hidden API
flag generation is skipped.

Bug: 179354495
Test: m com.android.i18 out/soong/hiddenapi/hiddenapi-flags.csv
      - make sure that the flags in
        packages/modules/RuntimeI18n/apex/hiddenapi/hiddenapi-max-target-o-low-priority.txt
        are reflected in the core-icu4j dex files in the apex.
Change-Id: I9b5c7c74bd996ab447bc0e0452da5fd49191a35d
2021-05-24 13:32:55 +01:00
Paul Duffin
af99afa919 Rename hiddenAPIFlagFileInfo to HiddenAPIInfo
This reflects that it has expanded from its initial purpose to include
more than just flag files. It is exported for use in tests in other
packages.

Bug: 179354495
Test: m nothing
Change-Id: I9f780b20e18ce3a774e4aa04a276463070a64c34
2021-05-24 13:32:55 +01:00
Paul Duffin
1352f7c471 Separate input to flag generation from hiddenAPIFlagFileInfo
Encapsulating the information needed by hidden API processing in a
struct makes it easy to add additional information in future and allows
the code to populate that struct from various different sources to be
grouped together.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: I53805737dff36a3ae87aca5aad51cf46ae1361fe
2021-05-24 13:32:55 +01:00
Paul Duffin
1e6f5c4e63 Separate output of flag generation from hiddenAPIFlagFileInfo
HiddenAPIFlagOutput encapsulates the paths to the files produced by the
hidden API flag generation of a single bootclasspath_fragment. It is
returned from hidden API flag generation and is embedded within the
hiddenAPIFlagFileInfo so they can be passed to other modules.

Unlike the fields it replaces in hiddenAPIFlagFileInfo the fields in
HiddenAPIFlagOutput are of type Path not Paths which makes it easier to
use.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: I7373ba1001cac3a75eb54a23e62fa52f5013ee7f
2021-05-24 13:32:55 +01:00
Paul Duffin
438eb57a27 Separate monolithic hidden API processing from hiddenAPIFlagFileInfo
The hiddenAPIFlagFileInfo was being used for both the input and output
of bootclasspath_fragment and platform_bootclasspath and also to pass
information around to various hidden API rule methods. Supporting
multiple different uses in this way made it hard to reason about.

This change creates a separate structure for use by the
platform_bootclasspath. Follow up changes will split out other
functionality into separate types.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: Ia5c5f65ae5645486c42819c669a8601588217f88
2021-05-24 13:32:55 +01:00
Paul Duffin
7487a7abb6 Workaround to make AlwaysUsePrebuiltSdks() work with platform_bootclasspath
The AlwaysUsePrebuiltSdks() causes all java_sdk_library_import modules
to be preferred over the source, i.e. as if they had prefer: true set.
That interacts badly with the work that is being done to integrate the
bootclasspath_fragment/platform_bootclasspath modules into the build.

It would work fine once that integration has been completed but in the
interim it causes problems. e.g. it does not cause a problem in AOSP
because those java_sdk_library_import modules that are affected have
already been integrated into the build properly.

Unfortunately, internally that is not the case because there are
java_sdk_library/java_sdk_library_import modules that still need to
be updated.

Before the java_sdk_library_import can be safely preferred each
java_sdk_library/java_sdk_library_import module that contributes to the
bootclasspath must:
* Be in the contents of matching bootclasspath_fragment and
  prebuilt_bootclasspath_fragment modules.
* Have an apex and one of a prebuilt_apex/apex_set that contains the
  dex implementation jar and lists the prebuilt_bootclasspath_fragment
  name in its exported_bootclasspath_fragments property.

Safely preferred in this context means that the whole build will
continue to work rather than the current situation which is that only
some of the build will work and some will fail if an attempt is
actually made to build it.

Unfortunately, many java_sdk_library_import modules are missing:
* The prebuilt_bootclasspath_fragment.
* The exported_bootclasspath_fragments property on the
  prebuilt_apex/apex_set that contains them.

Together these cause the following symptoms:
1. The java_sdk_library_import does not have a dex implementation jar.
2. The java_sdk_library_import does not have a myapex variant.

These workarounds will avoid Soong reporting build failures. However,
the build will still fail if an attempt is made to build anything
produced by the platform-bootclasspath, e.g. hidden API processing or
a system image.

Bug: 188505921
Bug: 179354495
Test: m TARGET_BUILD_APPS=Calendar
Change-Id: I3226e21cd6a7f9e4d6bbe94e54129ac5e1d4c679
Merged-In: I3226e21cd6a7f9e4d6bbe94e54129ac5e1d4c679
2021-05-19 11:42:20 +00:00
Paul Duffin
21f62ef867 Merge "Generate monolithic hidden API files direct from class jars" 2021-05-14 18:52:11 +00:00
Paul Duffin
2bc8b3a646 Merge "Dedup hidden API rule generation" 2021-05-14 18:51:10 +00:00
Paul Duffin
537ea3d04c Generate monolithic hidden API files direct from class jars
Previously, the monolithic hidden API files, e.g. hiddenapi-index.csv
file, were generated in two steps. First, each module created its own
files using the information in its class jars. Then, the monolithic
files were created by merging those module specific files into a larger
file.

This change switches to generating the monolithic files directly from
the class jar files and bypassing the intermediate files.

In order to ensure that this change did not change the monolithic files
it is necessary for the hiddenapi-metadata.csv to go through a
reformatting step. Hopefully, this will be able to be removed in a
follow up change.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I5a78e747516014b7c0f402a4b4431b14be6a84b2
2021-05-14 15:57:04 +01:00
Paul Duffin
850e61f234 Dedup hidden API rule generation
Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I54b677c97712a7573ff5eed017c1b2198b80bc51
2021-05-14 14:53:19 +01:00
Paul Duffin
098c878838 Make hidden API handle jacoco-stubs consistently
Previously, when code coverage was enabled the monolithic hidden API
processing would add jacoco-stubs to public APIs only as that would
make them accessible for everyone. However, the
art-bootclasspath-fragment added jacoco-stubs to public, system and
test APIs as that was the simplest way of handling it and while the
extract APIs were unnecessary they would not change behavior.

Unfortunately, that lead to a difference in the flags generated which
caused the check that verifies flags are consistent between the
monolithic and modular files to fail.

This change adds jacoco-stubs to system and test APIs for the
monolithic hidden API processing to ensure consistency.

Bug: 179354495
Bug: 188143639
Test: m EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true out/soong/hiddenapi/hiddenapi-flags.csv
      - ran before making the change to reproduce the problem and after
        to verify the fix.
Change-Id: Icbd15f0ece871a8c44d9e4a73fd0f7acc3760bba
2021-05-14 10:19:46 +00:00
Paul Duffin
34827d4c0e Use java_sdk_library in bootclasspath_fragment contents as stubs
A java_sdk_library specified in the bootclasspath_fragment contents
must be providing APIs for the bootclasspath_fragment and so must be
treated as if they were specified in the stub_libs. This avoids having
to specify them in both contents and stub_libs.

Bug: 179354495
Test: m nothing
Change-Id: I535065ee1a79b439e2676f35e06a75d4626adcaf
2021-05-14 01:49:19 +01:00
Paul Duffin
dfa1083fee Validate monolithic and modular hidden API flags are consistent
This makes sure that where there is overlap between the hidden API
flags generated for a module and the monolithic flags that they are
identical. That ensures that the modular hidden API flags will be
compatible with previous releases that relied on the monolithic flags.

Bug: 179354495
Test: m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - Create some inconsistencies between the above two files.
      m out/soong/hiddenapi/hiddenapi-flags.csv.valid
Change-Id: Iaf9e23cef63e221608955d89dc8d496bcc70c86e
2021-05-14 01:49:19 +01:00
Paul Duffin
2fef136885 Generate hidden API flags for a bootclasspath_fragment
This change adds support for generating the hidden API flags for the
contents of a bootclasspath_fragment. Currently, it will only work for
the art-bootclasspath-fragment as it has no support for creating
dependencies between bootclasspath_fragment modules which will be
needed for handling any other bootclasspath_fragment.

The hidden API flag generation added by this change is completely
separate to the normal hidden API processing and is not as yet encoded
in dex jars so will have no effect on the runtime.

The generated files are provided for use by other modules and copied
into the sdk snapshot. That is needed to allow the build to verify that
the hidden API flags generated by the individual bootclasspath_fragment
modules are consistent with the flags generated for the whole
bootclasspath, whether building from source or prebuilts.

Bug: 179354495
Test: m art-module-sdk
      m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - test that the former file is a subset of the latter and that
        where they overlap they are identical.
Change-Id: Ie27303e2960953db1b7abe95510e3bca4411b09a
2021-05-14 01:48:51 +01:00
Paul Duffin
1093158faf Add support for specifying api provided by bootclasspath_fragment
The hidden API processing needs access to dex stub jars for the API
scopes provided by the bootclasspath_fragment so that it can mark the
corresponding dex members as being part of that API. This change adds
the ability to specify the modules that provide those jars, resolve the
modules to dex jar stubs and make them available for other modules and
tests to use.

While the stubs are only needed for hidden API at the moment these were
not added to the hidden_api properties section as there are other parts
of the build that may need to access the stubs in future, e.g. api
fingerprint and aidl generation.

The stubs properties can be affected by coverage as when coverage is
enabled the jacoco-stubs needs adding to the list, just like how
jacocoagent is added to the contents.

Bug: 177892522
Test: m nothing
Change-Id: I31097d1ca45c84adeba4cbb38f693698cb289e99
2021-04-28 21:59:46 +01:00
Paul Duffin
976b0e5bbb Automatically export bootclasspath_fragment stubs from sdk
This change makes the tag that is used to add dependencies onto stubs
from the bootclaspath_fragment module cause the dependency to be
automatically exported from the sdk. If the dependency is a
java_sdk_library then it will be as if it was added to the
java_sdk_libs property, otherwise it will be as if it was added to the
java_header_libs property instead.

Bug: 177892522
Test: m nothing
Change-Id: I97010c4342dd39415dba81a4395db7831410436e
2021-04-27 23:27:34 +01:00
Paul Duffin
74431d57c6 Move monolithic stub flags generation to platform_bootclasspath
As part of that this change:
* Moves code that will be common to platform_bootclasspath and
  bootclasspath_fragment from hiddenapi_singleton.go into
  hiddenapi_modular.go.
* Fixes the tests in hiddenapi_singleton_test.go but intentionally
  does not rename them or move them into a more appropriate place so
  as to make it easier to see the differences. A TODO has been added
  and these will be cleaned up in a follow up change.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I680e4dab2e6bdf4a655fa9f255c195175904667e
2021-04-22 14:23:28 +01:00
Paul Duffin
3e7fcc3fe7 Extract common stub flags code
The stubFlagsRule does three separate tasks:
1. It computes the set of modules that provide the stubs.
2. It scans all the modules to find the stub modules and retrieves the
   paths to their dex files.
3. It constructs the ninja rule.

Of those three tasks, 1 and 3 will be same for the
platform_bootclasspath. Instead of searching all the modules for the
ones that provide the stubs it will simply add dependencies onto the
stub modules and retrieve the dex file paths from them.

This change extracts tasks 1 and 3 into separate methods for reuse. It
also parameterizes the generation of the ninja rule.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I893845dbddc4b001dfd44d0e0b1c8a31b7f3f89f
2021-04-22 14:23:08 +01:00
Paul Duffin
d3c1513df9 Add tempPathForRestat to improve consistency
Previously, there were two approaches used to construct the temporary
path needed by the commitChangeForRestat method and neither was
suitable for general use. One was:
    android.PathForOutput(ctx, outputPath.Rel()+".tmp")
The other was:
    combinedAidl.ReplaceExtension(ctx, "aidl.tmp")

The first approach would not work if the supplied path had been created
by PathForModuleOut() or similar as it would drop the module directory.
That could lead to the same path being used for multiple rules.

The second approach would not work for files with a different
extension.

The tempPathForRestat combines the two approaches so it can be used
generally with any WritablePath.

Bug: 179354495
Test: verified that the ninja rules that used these files were not
      changed by these changes.
Change-Id: I4439dea0a823512c281eeb1366522fb49dceb4e3
2021-04-21 23:26:57 +01:00
Paul Duffin
cc17bfe7de Rename propertyAccessor to propertyValueReader
Bug: 179354495
Test: m nothing
Change-Id: I956e93fdbe43e09b5460337d2eb72084440f14be
2021-04-21 10:14:02 +01:00
Paul Duffin
9b381ef2b8 platform_bootclasspath: aggregate hidden API flag files from fragments
Aggregates hidden API flag files from the bootclasspath_fragments which
will allow the hidden API flag files in frameworks/base/boot/hiddenapi
to be modularized and moved to the appropriate repo.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
      also verified that changes to the fragment provided files do
      affect the monolithic files.
Change-Id: Ifce14c9ef24c58c7ab1085475d85b61cfbfefecd
2021-04-16 00:29:53 +01:00
Paul Duffin
4616977948 Rename hidden API types ..Augmentation.. to ..FlagFile..
Augmentation was too abstract this makes it clearer.

Bug: 177892522
Test: m nothinge
Change-Id: I60ad005e68d9e15b01bcb46bc6a9b7f84d8bfd43
2021-04-15 10:45:39 +01:00
Paul Duffin
e3dc6608cb Generalize hiddenAPIAugmentationInfo to make it easier to use
Previously, each category of flag file had its own property in the
hiddenAPIAugmentationInfo struct that required a lot of repetition
to use. This change moves the flag file specific handling into a
new hiddenAPIFlagFileCategory struct which allows use of the
hiddenAPIAugmentationInfo struct to be parameterized.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: I4413134c0c9382139bef3813f847e453f426692c
2021-04-15 10:45:39 +01:00
Paul Duffin
702210b804 Move generation of global hidden API flags to platform_bootclasspath
This change moves the generation of the global hidden API flags from
the singleton to the platform_bootclasspath module. It involves:
1. Moving the ruleToGenerateHiddenApiFlags to hiddenapi_modular.go.
2. Adding HiddenAPIAugmentationProperties to be used by the
   platform_bootclasspath type.
3. Moving the file paths into the platform-bootclasspath module
   definition in frameworks/base/boot/Android.bp.

The flagsRule is kept as a placeholder for now. The emptyFlagsRule is
also kept so that builds continue to work even when the frameworks/base
repository is not present.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Idf4dd414a016831bfe04a01f93234c1c33819881
2021-04-12 11:04:24 +01:00
Paul Duffin
c6bb7cf8d7 Extract ruleToGenerateHiddenApiFlags
Extracts the code for creating the rule that creates the monolithic
hidden API flags file which is encoded into dex implementation jars.
This refactoring is in preparation for moving the functionality from
the hiddenapi_singleton into the platform_bootclasspath.

A follow up change will move the method into the new
hiddenapi_modular.go alongside the hiddenAPIAugmentationInfo as they
will both be used to perform hidden API processing for a
bootclasspath_fragment.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: I2729afa80cdfd2d1d4717365001648453d65632f
2021-04-12 11:04:09 +01:00