Commit graph

1602 commits

Author SHA1 Message Date
Rupert Shuttleworth
a9d76dd423 Add partial bp2build support for APEX targets (second try).
Test: Added unit test
Change-Id: Icbf0475251aac5d1f13bd35a18f90b6c70f3ae29
2021-07-02 07:36:07 -04:00
Weng Su
7ce730b90c Revert "Add partial bp2build support for APEX targets."
Revert submission 1733232-roboleaf-apex

Reason for revert: b/192523357 Broken build 7512065 on aosp-master on sdk_x86_64-sdk
Reverted Changes:
Ie82c263f6:Add a stub rule for building APEX bundles.
I93e2a25fa:Add partial bp2build support for APEX targets.

Change-Id: I8e836fe7f466e02f3108680f9653412299e929e0
2021-07-01 06:51:11 +00:00
Rupert Shuttleworth
22b606b5f0 Add partial bp2build support for APEX targets.
Test: Added unit test

Change-Id: I93e2a25fa43eec1ffb8fb30c346a9b523195a0fd
2021-06-30 22:20:23 -04:00
Paul Duffin
46ab291f67 Ignore license and similar dependencies of prebuilt_apex/apex_set
Previously, when determining which dependencies, direct or transitive,
of a prebuilt_apex/apex_set required APEX variants the code assumed
that all dependencies implemented ApexModule. While that is true for
the modules that can be explicitly mentioned in the exported...
properties it is not true for all of them. e.g. A
prebuilt_apex/apex_set can depend on license modules which do not
implement ApexModule.

This change simply ignores any module that does not implement
ApexModule.

Bug: 179354495
Test: m nothing
Merged-In: Iead6f9d1085d169335b88ceadcce2d8cc042254d
Change-Id: Iead6f9d1085d169335b88ceadcce2d8cc042254d
(cherry picked from commit fee8cf36e3)
2021-06-30 00:08:49 +01:00
Paul Duffin
280a31aac3 Calculate widest stub dex jars per module
Previously, the stub dex jars for each HiddenAPIScope was created by
merging the stub dex jars provided by each module for that scope. Then
the widest stub dex jars were chosen. So, if module A provided public,
system and test stub dex jars and module B provided only public then
the stub dex jars for each scope would be:
* public -> A,B
* system -> A
* test -> A

So, the widest API scope for which there are stub dex jars is "test"
and so the widest stub dex jars would just come from module A and not
module B. So, when "hiddenapi list" is run for module C which depends
on modules A and B it only gets given stub dex jars for module A which
means that it cannot resolve all the types that C may use which can
lead to incorrect flags being generated.

This change does not merge the stub dex jars from each module together
and instead keeps them separate by module. The widest stub dex jars
list is constructed by asking each module in turn for their widest stub
dex jars. e.g. Given the above example we would have:

Module A:
* public
* system
* test <- widest

Module B:
* public <- widest

So, the widest stub dex jars will be A's test and B's public stub dex
jars.

Bug: 179354495
Test: m out/soong/hiddenapi-flags.csv
      - make sure that this does not change the file.
Change-Id: Ib137825ebffe94b2bf220732bae6077f7b7ac6db
2021-06-28 10:56:43 +01:00
Treehugger Robot
f46e95899b Merge "Fix inconsistencies in the apex names used for the apex variations." 2021-06-25 22:50:00 +00:00
Paul Duffin
99926a2a77 Merge "Support hidden API processing for modules that use platform APIs" 2021-06-25 18:25:39 +00:00
Jiyong Park
56a39ad6d9 Merge changes from topic "stub_from_rust"
* changes:
  Add platform_apis property to APEX module type
  Rust module in APEX uses stub libraries across APEX boundaries
2021-06-25 17:52:43 +00:00
Martin Stjernholm
bfffae7bec Fix inconsistencies in the apex names used for the apex variations.
Add an apex_name property to prebuilt APEX modules to allow specifying
the "runtime" name of the APEX, i.e. the one it gets mounted as in /apex/,
which is also the one used for the apex variations.

Introduce a callback to retrieve that name consistently for all APEX
modules (apex, override_apex, prebuilt_apex, and apex_set), and update
some apex mutator code paths to use it.

For APEX source modules (apex and override_apex), it's necessary to add
a new field in apexBundle, since the name property gets overridden for
the override variant that override_apex creates.

Test: m nothing
Bug: 191269918
Change-Id: If8612639bffdf91cbcab3387b0603bf5dffef1f5
2021-06-25 16:41:05 +01:00
Paul Duffin
5cca7c44e5 Support hidden API processing for modules that use platform APIs
Previously, hidden API processing could only be done by those
bootclasspath_fragment modules that either did not depend on any other
fragments (e.g. art-bootclasspath-fragment) or only depended on APIs
provided by other fragments (e.g. i18n-bootclasspath-fragment). That
meant that modules like com.android.os.statsd-bootclasspath-fragment
that depended on APIs provided by parts of the platform which are not
yet part of another bootclasspath_fragment could not perform hidden
API processing.

This change adds support for a bootclasspath_fragment to specify the
additional stubs needed to perform hidden API processing. It adds a new
additional_stubs property that can be used to specify the additional
stub libraries.

Most bootclasspath_fragments that need to use the property will need
access to the APIs provided by the android-non-updatable.* libraries.
Rather than have each fragment explicitly specify the correct module
for each scope it treats "android-non-updatable" as if it was a
java_sdk_library that can provide different jars for each scope.
Soong will handle mapping that to the correct android-non-updatable.*
module.

Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv \
        out/soong/hiddenapi/hiddenapi-index.csv \
        out/soong/hiddenapi/hiddenapi-stub-flags.txt \
        out/soong/hiddenapi/hiddenapi-unsupported.csv
      - make sure that this change does not change the contents.
      m TARGET_BUILD_APPS=Calendar nothing
Change-Id: Ia8b79830ed0e6d42100de03d76b0c51b7f6c8ade
2021-06-25 16:20:19 +01:00
Paul Duffin
b51db2ed8e Add ModuleLibHiddenAPIScope for hidden API processing
Although the hidden API runtime does not support a module-lib API flag
the hidden API processing does need to used them as they are needed by
the "hiddenapi list" tool for any bootclasspath_fragment module whose
contents builds against an sdk_version of "module_current". Without it
the "hiddenapi list" tool could fail to resolve references to classes
provided by other modules which would break the build.

Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - make sure that this change has no effect on the generated flags.
Change-Id: I3ecb80fdaeba0e66d1ee25cb57152ab546d9bfe0
2021-06-24 22:04:09 +01:00
Paul Duffin
48b6741510 Add test for modular hiddenapi list rule
Adds a test to establish a baseline for a follow up change that adds
module_lib support to hidden API processing.

Bug: 179354495
Test: m nothing
Change-Id: I5630075825c674b2e4f38bd459c7b061c01fc362
2021-06-24 22:04:05 +01:00
Paul Duffin
31fad800a7 Add HiddenAPIScope to replace use of SdkKind
Previously, the hidden API processing used SdkKind to identify the API
scopes, e.g. public, system, etc. that are of interest for hidden API
processing. Unfortunately, there is a mismatch between the SdkKind and
what hidden API processing needs. e.g. SdkKind includes values that are
not used by hidden API processing and hidden API processing needs
additional API scope specific information not provided by SdkKind. The
apiScope struct used in sdk_library.go is also not a suitable
representation for similar reasons.

This change adds the HiddenAPIScope (following a similar approach as
apiScope) that rectifies that and uses it as a replacement for SdkKind
in most parts of the hidden API processing. The SdkKind is still used
for retrieving information from java_sdk_library[_import] modules.

Follow up changes will extend the HiddenAPIScope with more information.

Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - make sure that this change has no effect on the generated flags.
Change-Id: I97968f58535121652852b8d25217aa288afd2bfd
2021-06-24 21:56:27 +01:00
Jiyong Park
9477c26f6c Add platform_apis property to APEX module type
The property is used to allow non-updatable APEXes to use platform APIs
(e.g. symbols marked as "# platform-only").

Bug: 191637950
Test: m com.android.virt com.android.compos
Merged-In: Id2410b4e38a78ec2146a42298840954381a7c472
Change-Id: Id2410b4e38a78ec2146a42298840954381a7c472
(cherry picked from commit fb63625a7f)
2021-06-24 11:28:33 +09:00
Jiyong Park
7d55b61783 Rust module in APEX uses stub libraries across APEX boundaries
This change fixes a bug that rust module in APEX has access to private
symbols of a native library even when the native library is outside of
the APEX.

To fix this, the stub selection logic in the cc package is exctacted as
a function ChooseStubOrImpl and is used also in the rust package.

Bug: 190767845
Test: m
Merged-In: I5c4cbdd5d27f257ab329d9dadbcd87d41a87f46a
Change-Id: I5c4cbdd5d27f257ab329d9dadbcd87d41a87f46a
(cherry picked from commit 3b5f88e091)
2021-06-24 11:27:41 +09:00
satayev
17c4507dec Merge "VNDK APEX: skips VNDK-Ext" 2021-06-23 19:52:56 +00:00
Jooyung Han
91913e3e07 Merge "fix: comparison between min_sdk_version" 2021-06-23 17:02:02 +00:00
Jooyung Han
7d6e79bc63 VNDK APEX: skips VNDK-Ext
In legacy VNDK mode (DeviceVndkVesion=""), the check was missing. We
don't want VNDK-Ext in VNDK APEX in any case.

Bug: 191770320
Test: m nothing
Change-Id: I08f36a4c1696bf82d25d629841e172d2716a366d
2021-06-24 01:56:03 +09:00
satayev
a580b5ef0f Merge "Append platform classpath proto configs with missing apex jars." 2021-06-23 09:51:39 +00:00
Jooyung Han
4c4da0688c fix: comparison between min_sdk_version
Added test demonstrates the error case. The bug was that java modules'
ShouldSupportSdkVersion() converts min_sdk_version into effective
version before comparison, which discards "preview" version into
"future" version. That caused a weird case like rejecting with error
"should support "S"" even when it is set to "S".

Bug: n/a
Test: m nothing
Change-Id: Ieb53f0097b2969a8535778b1286dbfa0d4895ad4
2021-06-23 10:45:28 +09:00
satayev
b30905019f Append platform classpath proto configs with missing apex jars.
Any apex classpath fragment that doesn't generate its own classpaths
proto, must still propagate it's boot jars for the platform classpath
fragment to include for complete CLASSPATH variables on device.

Bug: 191127295
Test: atest CtsClasspathsTestCases derive_classpath_test
Change-Id: I93687f69006741fcd66eb6e04891a4b4bbcc3b47
2021-06-22 20:03:54 +01:00
Paul Duffin
a1a2baf8a7 Merge "Fix monolithic hidden API processing with prebuilts" 2021-06-22 11:44:11 +00:00
Paul Duffin
07cdd37ae5 Merge "Make CheckHiddenAPIRuleInputs more reusable" 2021-06-21 16:16:12 +00:00
Paul Duffin
9d31361873 Merge "Use classpath elements in platform_bootclasspath" 2021-06-21 16:16:00 +00:00
Treehugger Robot
348f20be0e Merge "AIDEGen: Collect dependencies info from APEX module." 2021-06-21 07:17:28 +00:00
Paul Duffin
d061d40eb6 Fix monolithic hidden API processing with prebuilts
Prebuilt modules do not provide classesJars containing annotations.
Previously, the monolithic hidden API processing just used classesJars
from all the modules that provided them so when building against
prebuilts would have fewer classesJars than when building against
sources and so would produce different hidden API flags.

This change will generate the monolithic files from both classesJars
and files previously generated from hidden API processing. A fragment
that has performed hidden API processing will contribute its generated
files whereas standalone libraries and fragments which have not
performed hidden API processing will contribute classesJars.

Bug: 177892522
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      m SOONG_CONFIG_art_module_source_build=false out/soong/hiddenapi/hiddenapi-flags.csv
      - verify that the files are identical whether built from
        source or prebuilts.
Change-Id: I06f3c7df49626bec21a452bc9abf1bb9e7545e5c
2021-06-20 19:09:09 +01:00
Paul Duffin
51d7da2c4e Make CheckHiddenAPIRuleInputs more reusable
Adds a message parameter and allows leading spaces in the expected file
string to allow them to be nicely indented.

Bug: 177892522
Test: m nothing
Change-Id: I33df26610738c48879fa0b8250dc377dd04bb07d
2021-06-20 19:09:09 +01:00
Paul Duffin
89f570ac44 Use classpath elements in platform_bootclasspath
Use classpath elements in newMonolithicHiddenAPIInfo. That means the
method can collate information from both fragments and libraries rather
than just fragments. So, this change moves the collation of the
classesJars into the method.

Bug: 177892522
Test: m out/soong/hiddenapi/hiddenapi-flags.csv out/soong/hiddenapi/hiddenapi-index.csv
      - make sure that this change does not affect the contents.
Change-Id: I7c2a229fab60d02bd211438735a8d7303ed83386
2021-06-20 19:09:09 +01:00
Paul Duffin
0d7f2d30b5 Merge "Simplify deapexer support" 2021-06-20 18:08:41 +00:00
Paul Duffin
6b592bdcdc Merge "Remove javalib special case in createDeapexerModuleIfNeeded" 2021-06-20 18:08:29 +00:00
Paul Duffin
e6aefedeba Merge "Add ClasspathElement support" 2021-06-18 18:13:06 +00:00
Paul Duffin
e91f1537e2 Merge "Extract createEntriesForApexFile" 2021-06-18 17:11:29 +00:00
Nikita Ioffe
f932613c3f Merge "Set default value of generate_hashtree property to true" 2021-06-18 13:14:22 +00:00
Paul Duffin
e245b61aa2 Add ClasspathElement support
The configuration of the bootclasspath, e.g. PRODUCT_BOOT_JARS,
includes libraries that are standalone and libraries that are part of
an APEX. All libraries that are part of an APEX must also be part of a
bootclasspath_fragment. Currently, the libraries and fragments are
handled separately but that is limiting as it does not make it easy to
process all the libraries while treating those from fragments
differently to standalone libraries. That is needed for hidden API
processing as it needs to use classesJars from standalone libraries
but pre-generated flag files from fragments.

This change adds support for ClasspathElements which is represents the
classpath as a whole while differentiating between standalone libraries
and fragments.

Bug: 177892522
Test: m nothing
Change-Id: I7a4adc67164a46079eb8ec0a17fc755044b4949d
2021-06-18 11:46:52 +01:00
satayev
f45966f35b Merge "Add classpath fragment property to skip proto generation." 2021-06-18 10:17:50 +00:00
Paul Duffin
155c17779c Extract createEntriesForApexFile
While this seems like a simple refactoring it does actually fix an
issue. Previously, when the ExtraEntries func was being defined inline
within the for ... loop it used the loop variable "fi". Unfortunately,
that meant that the func created on each iteration ended up using the
value for "fi" that was set on the last iteration.

Extracting the creation of the entry into a separate method means that
each func created gets its own "fi" variable with the correct values.

Bug: 177892522
Test: m SOONG_CONFIG_art_module_source_build=false nothing
      - without this change it reported duplicate rules for apache-xml.jar
        which is the last entry in the art-bootclasspath-fragment. With
        this change it works fine.
Change-Id: Ia063b513f758e1bbe73407c0441b72453f412acf
2021-06-18 07:43:32 +01:00
Paul Duffin
5466a3699c Export boot image files from prebuilt_apex/apex_set
Previously, the prebuilt art-bootclasspath-fragment did not provide any
boot image files. That meant it was impossible to build any module that
requires access to those files from prebuilts, e.g. any module that
needs to be dexpreopt-ed.

This change enables that module to retrieve those files from the
prebuilt_apex.

Bug: 177892522
Bug: 189298093
Test: m nothing
      m droid SOONG_CONFIG_art_module_source_build=false SKIP_BOOT_JARS_CHECK=true
      - the previous command does not work but this change does fix one
        of the issues reported.
Change-Id: I1d4d9545172d79282918130df6b9aa55471bffc1
2021-06-17 23:05:43 +01:00
Paul Duffin
ce918b0278 Copy boot dex jars from prebuilt art-bootclasspath-fragment if preferred
Previously, the boot dex jars were only copied to the predefined
locations used by the build from the source art-bootclasspath-fragment
if it was preferred, otherwise no files were copied. That caused build
failures when attempting to build with ART prebuilts. This change
copies the files from prebuilts too.

Bug: 177892522
Bug: 189298093
Test: m nothing
      m droid SOONG_CONFIG_art_module_source_build=false SKIP_BOOT_JARS_CHECK=true
      - the previous command does not work but this change does fix one
        of the issues reported.
Change-Id: I35b37355170546daf6ecac2134d1ca9a73d0e3bc
2021-06-17 23:05:43 +01:00
Paul Duffin
85fa344d09 Copy implementation and header jars to make when possible
Fixes test issues caused by a jar containing a classes.dex file was
copied to a place that was expected a jar containing class files.

Bug: 191360201
Test: - in master-art
      find out/ -name \*core-icu4j\* | xargs rm -fr
      art/tools/buildbot-build.sh --target --installclean
      art/tools/bootjars.sh --path --core
      - examine file output from previous script for core-icu4j,
        before this change it contained classes.dex file after this
        change it contained *.class files.
Change-Id: I05ecf652535ec097125fb76d6ca80fee6587f98d
2021-06-17 20:33:54 +01:00
Paul Duffin
b4bbf2ca10 Simplify deapexer support
Uses the apex relative path to the file as the identifier that is used
to obtain the path to the corresponding file extracted from the apex.
That is instead of a special constructed string id.

Bug: 177892522
Test: m nothing
Change-Id: I5dc77c8fb272bac289b8891d1eac801e541af1f5
2021-06-17 15:59:07 +01:00
Paul Duffin
fef5500a76 Remove javalib special case in createDeapexerModuleIfNeeded
Removes the special case in createDeapexerModuleIfNeeded for handling
java libraries and just get the Import and SdkLibraryImport module
types to implement RequiredFilesFromPrebuiltApex instead.

Bug: 177892522
Test: m nothing
Change-Id: I5cc341b5b4168b8eb196f72273a00d498de6856f
2021-06-17 14:56:05 +01:00
satayev
b98371cc13 Add classpath fragment property to skip proto generation.
This must always be true for updatable apexes, but is not necessary
for non-updatable apexes like com.android.i18n.

In a follow up this will be used to figure out whether apex boot jars
should be bundled into platform_bootclasspath.

Bug: 191127295
Test: atest CtsClasspathsTestCases derive_classpath_test
Change-Id: Ib7dc5b057cb24955222fb97f3ff9da079f30ed77
2021-06-17 12:59:33 +01:00
Paul Duffin
e521881bd4 Combine hidden API encoding with flag generation
Previously, the rules to perform hidden API encoding were generated
separately to the rules to perform hidden API flag generation. This
change combines them within the (renamed) produceHiddenAPIOutput()
method and makes the paths to the encoded dex files an output of the
generateHiddenAPIBuildActions method alongside the paths to the
generated flag files.

As encoded dex jars are now an output of the produceHiddenAPIOutput()
method which is implemented for both prebuilts and source
bootclasspath_fragment modules that necessitated the prebuilt also
providing paths to encoded dex files. That in turn required updates to
some of the tests to provide dex files from prebuilt_apex modules.

Similarly, as the produceHiddenAPIOutput() method may not be called
for some bootclasspath_fragment modules as they do not yet provide all
the information needed to perform hidden API encoding then it is
necessary to extract the encoded dex files produced by the modules
themselves. That also required a few changes to tests that did not
previously provide dex files.

Bug: 177892522
Test: m com.android.art
      - check that this change does not change the contents of the apex
        file, i.e. is byte-for-byte identical.
Change-Id: I60996a34d06ed1c87ed244ab3509621999ad86ec
2021-06-17 10:20:33 +01:00
Paul Duffin
a35f8db127 Make apex variants of prebuilt apex exported modules available in make
Previously, the apex variants of modules exported from prebuilt_apex
and apex_set modules were not exported to make. Neither by the modules
themselves or by the prebuilt apex module. The master-art build relied
on the platform variants of the conscrypt and core-icu4j libraries
being exported to make so that they could be used by vogar.

Unfortunately, a change to export the prebuilt_bootclasspath_fragment
modules that contain the conscrypt and core-icu4j prebuilt libraries
from the corresponding prebuilt_apex prevented the platform variants
of those libraries from being exported at all which broke the
master-art builds as it only has prebuilts of those modules. It did not
break the aosp/master build as that has source modules.

The difference between the two builds is that the apex module type
makes its contents available in make but the prebuilt_apex/apex_set
module types do not.

This change causes the prebuilt_apex/apex_set module types to behave
more like the apex module type by making its exported libraries
available in make.

Test: m droid
      - in aosp/master
      art/tools/buildbot-build.sh --target --installclean
      - in master-art and aosp/master
Change-Id: I57537d17d4920d37d896ed491c6aaa15302cb25d
2021-06-16 19:02:58 +01:00
Paul Duffin
6717d88f46 Create APEX variant for prebuilt_apex/apex_set
Previously, the prebuilt_apex/apex_set did not have an apex specific
variant created which meant that they depended upon the platform
variant of the modules it depended upon. This change creates an
apex variant for them just as is done for the apex module type which
causes it to depend upon the apex specific variant of the modules it
depends upon.

Test: m droid
Bug: 179354495
Change-Id: I7d6f3609c267b3e90b90b9befe7d76f351a0c2bd
2021-06-16 18:54:19 +01:00
Nikita Ioffe
e261ae64e2 Set default value of generate_hashtree property to true
Test: presubmit
Test: m
Bug: 190621617
Change-Id: I546730f168632c5d5d41c686853bde93a8ba7911
2021-06-16 18:15:03 +01:00
Paul Duffin
a9c8110470 Dedup apex/Prebuilt and apex/ApexSet
Moves common properties, fields and method into the prebuiltCommon
type.

One slight change is that this change causes the Prebuilt module to use
the outputApex when generating the AndroidMkEntries instead of the
inputApex that it used before. They are the exact same files as the
inputApex is simply copied to the outputApex. The only impact will be
on build time as some rules now depend on the cp rule being run when
they did not before and the base name of the file may be different.

Test: m droid
      - check generated out/soong/Android...mk file before and after
        to make sure the only difference is due to switching from the
	inputApex to outputApex.
Bug: 179354495
Change-Id: I8437af00d6bb7d0d339f25b3b02cd1cf67d6938a
Merged-In: I8437af00d6bb7d0d339f25b3b02cd1cf67d6938a
2021-06-16 10:35:48 +01:00
bralee
b0c1f0c45a AIDEGen: Collect dependencies info from APEX module.
Add apex dependencies.

Bug: 188004865
Test: 1. aidegen frameworks/base/apex/appsearch frameworks/base/apex/appsearch/testing   frameworks/base/services/tests/servicestests   frameworks/base/core/tests/coretests   cts/tests/appsearch   external/icing   vendor/google_testing/integration/tests/scenarios
      2. The dependency graph shows it is workable to provide the result to let IDE's [Project structure] to include the service-appsearch as source directory.
Change-Id: Ifd1548f0a86c1c73f8279e8ecea67756aeb1281e
2021-06-15 08:35:13 +08:00
Nikita Ioffe
5868ba016d Merge "Rename test_only_no_hashtree to generate_hashtree" 2021-06-14 20:16:21 +00:00
Nikita Ioffe
9d9960f240 Rename test_only_no_hashtree to generate_hashtree
In order not to introduce any changes in the system image default value
of generate_hashtree is currently set to false. Once all the APEXes that
shouldn't have hashtree enable (i.e. non-updatable ones) explicitly set
`generate_hashtree: false` default value will be switched to true.

Test: checked that are no diffs between ${OUT}/system/apex without this
  change and with this change.
Test: m
Bug: 190621617
Change-Id: I5a3144f8f6ea7dee25678af43e2b07a8c85daa9e
Merged-In: I5a3144f8f6ea7dee25678af43e2b07a8c85daa9e
(cherry picked from commit da6dc31389)
2021-06-10 16:54:53 +01:00