Commit graph

3947 commits

Author SHA1 Message Date
Aurimas Liutikas
24a987fefb Update flags passed to kotlinc
- Remove -Xuse-ir as it is default in Kotlin 1.5
- Add -Xsam-conversions=class for KT-46512

Change-Id: Ie351bb9ad5ec51ef7b7346d0076868d04cfe9811
2021-05-17 17:47:13 +00:00
Paul Duffin
1bbd0626f2 Make uncompressDex available for use through hiddenAPIModule
In order for the bootclasspath_fragment to perform dex encoding on its
contents it needs to know whether the dex file is uncompressed or not.
This change makes that information available by passing it to
initHiddenAPI, storing it in hiddenAPI struct and providing access
through the hiddenAPIModule.

Bug: 179354495
Test: m droid
Change-Id: I913416b4836766de194203fd8ed5124b61dfa3dd
2021-05-17 18:38:06 +01:00
Paul Duffin
45897dd663 Remove hiddenAPI.primary field
Now that the individual modules no longer participate in the generation
of the monolithic files it is no longer necessary to select a single
primary module to provide the information they need.

Bug: 179354495
Test: m droid
Change-Id: If09796de710927e3e3f2ccecad0b57ca5fce5dc9
2021-05-17 18:38:06 +01:00
Paul Duffin
74d18d1d6f Merge initHiddenAPI and hiddenAPIUpdatePaths
These two methods did very similar jobs and merging them together
simplifies the behavior.

Bug: 179354495
Test: m droid
Change-Id: Ibe1a23d54105e6a0e5693079cd8743679301fc85
2021-05-17 18:38:06 +01:00
Paul Duffin
66cdbf07ef Remove configurationName from java library and hidden API
The configurationName was intended to separate the name of the module
from the name used in configuration (such as BootJars) so that the
child implementation library of a java_sdk_library on the bootclasspath
would have hidden API encoding performed on it just as for the main
java_library embedded within the java_sdk_library.

While that did use to work it no longer does as the test added in the
preceding change proves. It is not surprising that this regression does
not appear to have caused any issues as the the child implementation
library is only a build time artifact and not used at runtime.

In future the only modules that will require hidden API encoding are
those that are part of a bootclasspath module so there is no point in
maintaining this capability.

Bug: 179354495
Test: m droid
Change-Id: Ief8136fa9e98600cdd8d36108ec22edc2ebd7c69
2021-05-17 18:38:06 +01:00
Paul Duffin
0d586581d1 Add a test for hidden API encoding of java_sdk_library
Fill a hole in the testing of hidden API encoding. Some comments in the
code indicate that the child implementation java_library of a
java_sdk_library should have hidden API flags encoded in its dex jar
just like the embedded java_library. However, this test proves that it
is not working. This will be fixed in a follow up change.

Bug: 179354495
Test: m nothing
Change-Id: Ia581a17f1e48dff252d17f16bf76adf039f46b60
2021-05-17 18:38:06 +01:00
Paul Duffin
70cfdff3da Add temporary restriction on hidden API processing
Hidden API processing of a bootclasspath_fragment requires the fragment
provides information, such as dependencies on other fragments and flag
files to override the default flags. Failing to do so will cause hidden
API generation to either fail or to generate different flags to that
generated by the hidden API processing done by platform_bootclasspath
which will cause the build to fail.

Previously, this was handled by only performing hidden API processing
for those modules that provide stub libs and relied on there only being
bootclasspath_fragments defined for ART (which already supports hidden
API processing), and Conscrypt and I18n neither of which provide stubs.
Unfortunately, that can no longer be relied upon due to a couple of
recent changes:
1. A java_sdk_library in a bootclasspath_fragment's content property is
   automatically treated a stub library. That avoids duplication for
   most bootclasspath_fragments that provide the implementation and
   stub libraries through the same java_sdk_library. It does not affect
   either ART, conscrypt or i18n as they all define the implementation
   separately to the stubs.

2. bootclasspath_fragment modules have been defined for a number of
   android modules as they are needed for reasons other than hidden API
   processing.

In combination this meant that rules to perform hidden API processing
were being created but they were not currently being used which is
good because they fail.

However, adding the fragment to the platform-bootclasspath will cause
those rules to be used as the platform_bootclasspath module performs a
consistency check on the hidden API flags generated by each of the
fragments to ensure that they are consistent with those it generates
itself.

The dynamic bootclasspath work will need to add fragments to the
platform_bootclasspath and without this change that would be blocked
until all fragments had been switched to generating hidden API flags
properly.

This change adds a new fragments property to the bootclasspath and
disables hidden API processing for everything other than ART and tests
if the fragments property or stub libs is empty.

Bug: 179354495
Test: - Before making this change.
      - Add com.android.os.statds-bootclasspath-fragment to platform-bootclasspath.fragments
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - hidden API processing fails in statsd as it cannot find java.lang.Object.
      - After making this change, run the above command again and it should pass.

Change-Id: Ifbb362f8fcfb2c06595fbd5ae39421b536e329ef
2021-05-17 07:39:10 +01:00
Treehugger Robot
718ac9276b Merge "Add usesTargetFiles option in dexpreopt_gen" 2021-05-14 23:39:49 +00:00
Paul Duffin
fdd9743609 Merge "Refactor special handling of hidden API encoding for master-art" 2021-05-14 18:53:43 +00:00
Paul Duffin
b7c78731d4 Merge "Stop generating unnecessary hidden API rules" 2021-05-14 18:53:08 +00:00
Paul Duffin
21f62ef867 Merge "Generate monolithic hidden API files direct from class jars" 2021-05-14 18:52:11 +00:00
Paul Duffin
2bc8b3a646 Merge "Dedup hidden API rule generation" 2021-05-14 18:51:10 +00:00
Paul Duffin
b0f20d134f Merge "Allow shared libraries on bootclasspath" 2021-05-14 17:03:03 +00:00
Jeongik Cha
4dda75e73e Add usesTargetFiles option in dexpreopt_gen
For running dex2oat on the target_files, the paths should be use the
device install path instead of the path starting with $(OUT).
So add usesTargetFiles option and basePath option which indicates
extracted path. With those options, the path is replaced with
$(basePath)/$(device path)

And also, add DexPreoptImageDeviceLocations in the config which refers
to the boot image path(without arch) on the device. Because
DexPreoptImage related device path was missing.

Bug: 158843648
Test: dexpreopt_gen -usesTargetFiles -basePath (extract path) and then
check if paths in the generated shell script are based on on-device
path.

Change-Id: I9667fadbf3b7c6f770e0d1bcbee5d67c1ecd8a3d
2021-05-15 00:29:49 +09:00
Paul Duffin
79fd3d728c Allow shared libraries on bootclasspath
A previous change treated this as an error in order to try and detect
issues with misconfigured java_sdk_library modules and/or
bootclasspath_fragment modules but unfortunately this is not always an
error, e.g. when migrating a library that was a shared library to the
bootclasspath.

This change stops treating that as an error.

Bug: 179354495
Test: m nothing
Change-Id: I4a833ab5f4caf86c6cd340090fc65d2c2f141512
2021-05-14 16:14:17 +01:00
Paul Duffin
b6f53c064e Refactor special handling of hidden API encoding for master-art
Instead of encoding the hidden API with an empty set of flags when the
monolithic flags are not available this simply disables encoding
altogether which should have the same behavior at runtime.

This change also removes the unused flags field in hiddenAPISingleton
which was set but never read.

Bug: 179354495
Test: m nothing
Change-Id: I32d5825e5271829993dd4e5be4d4ee1b22fa7b22
2021-05-14 15:57:10 +01:00
Paul Duffin
afaa47c74a Stop generating unnecessary hidden API rules
The rules to extract hidden API information from an individual module
in order to create module specific files are no longer necessary as
the monolithic files are created directly from the module's class jars
and not the module specific files.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I573ac17f3ea5da5a2a7e4f08718160dacca71c0c
2021-05-14 15:57:10 +01:00
Paul Duffin
537ea3d04c Generate monolithic hidden API files direct from class jars
Previously, the monolithic hidden API files, e.g. hiddenapi-index.csv
file, were generated in two steps. First, each module created its own
files using the information in its class jars. Then, the monolithic
files were created by merging those module specific files into a larger
file.

This change switches to generating the monolithic files directly from
the class jar files and bypassing the intermediate files.

In order to ensure that this change did not change the monolithic files
it is necessary for the hiddenapi-metadata.csv to go through a
reformatting step. Hopefully, this will be able to be removed in a
follow up change.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I5a78e747516014b7c0f402a4b4431b14be6a84b2
2021-05-14 15:57:04 +01:00
Paul Duffin
850e61f234 Dedup hidden API rule generation
Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I54b677c97712a7573ff5eed017c1b2198b80bc51
2021-05-14 14:53:19 +01:00
Paul Duffin
098c878838 Make hidden API handle jacoco-stubs consistently
Previously, when code coverage was enabled the monolithic hidden API
processing would add jacoco-stubs to public APIs only as that would
make them accessible for everyone. However, the
art-bootclasspath-fragment added jacoco-stubs to public, system and
test APIs as that was the simplest way of handling it and while the
extract APIs were unnecessary they would not change behavior.

Unfortunately, that lead to a difference in the flags generated which
caused the check that verifies flags are consistent between the
monolithic and modular files to fail.

This change adds jacoco-stubs to system and test APIs for the
monolithic hidden API processing to ensure consistency.

Bug: 179354495
Bug: 188143639
Test: m EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true out/soong/hiddenapi/hiddenapi-flags.csv
      - ran before making the change to reproduce the problem and after
        to verify the fix.
Change-Id: Icbd15f0ece871a8c44d9e4a73fd0f7acc3760bba
2021-05-14 10:19:46 +00:00
Paul Duffin
34827d4c0e Use java_sdk_library in bootclasspath_fragment contents as stubs
A java_sdk_library specified in the bootclasspath_fragment contents
must be providing APIs for the bootclasspath_fragment and so must be
treated as if they were specified in the stub_libs. This avoids having
to specify them in both contents and stub_libs.

Bug: 179354495
Test: m nothing
Change-Id: I535065ee1a79b439e2676f35e06a75d4626adcaf
2021-05-14 01:49:19 +01:00
Paul Duffin
dfa1083fee Validate monolithic and modular hidden API flags are consistent
This makes sure that where there is overlap between the hidden API
flags generated for a module and the monolithic flags that they are
identical. That ensures that the modular hidden API flags will be
compatible with previous releases that relied on the monolithic flags.

Bug: 179354495
Test: m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - Create some inconsistencies between the above two files.
      m out/soong/hiddenapi/hiddenapi-flags.csv.valid
Change-Id: Iaf9e23cef63e221608955d89dc8d496bcc70c86e
2021-05-14 01:49:19 +01:00
Paul Duffin
2fef136885 Generate hidden API flags for a bootclasspath_fragment
This change adds support for generating the hidden API flags for the
contents of a bootclasspath_fragment. Currently, it will only work for
the art-bootclasspath-fragment as it has no support for creating
dependencies between bootclasspath_fragment modules which will be
needed for handling any other bootclasspath_fragment.

The hidden API flag generation added by this change is completely
separate to the normal hidden API processing and is not as yet encoded
in dex jars so will have no effect on the runtime.

The generated files are provided for use by other modules and copied
into the sdk snapshot. That is needed to allow the build to verify that
the hidden API flags generated by the individual bootclasspath_fragment
modules are consistent with the flags generated for the whole
bootclasspath, whether building from source or prebuilts.

Bug: 179354495
Test: m art-module-sdk
      m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - test that the former file is a subset of the latter and that
        where they overlap they are identical.
Change-Id: Ie27303e2960953db1b7abe95510e3bca4411b09a
2021-05-14 01:48:51 +01:00
Paul Duffin
001e606e5c Populate hiddenAPI structure even when not active
The modular hidden API processing needs access to the classesJarPaths
and bootDexJarPath fields of the hiddenAPI structure even if the
modules are not themselves considered to be active participants in the
existing hidden API processing.

This change moves the initialization of those fields to before inactive
modules are ignored.

Bug: 179354495
Test: m art-module-sdk out/soong/hiddenapi/hiddenapi-flags.csv
Change-Id: I06f96d39d0b413295d3e2af50453ebe7e4d3e9c8
2021-05-14 01:45:42 +01:00
Paul Duffin
f4600f6e6a Disallow shared libraries in bootclasspath_fragment contents
Bug: 177892522
Test: m nothing
Change-Id: I78c8ef8664ec1eb0fe3456a2de2cb956162ca0da
2021-05-14 00:39:24 +01:00
Yuntao Xu
0589914135 Merge "syntax check to jacoco filters for any build" 2021-05-13 23:11:48 +00:00
Paul Duffin
a10bd3c127 Add java_sdk_library in bootclasspath_fragment contents to sdk
A java_library specified in a bootclasspath_fragment's contents
property will be automatically added to the sdk containing that
bootclasspath_fragment. Previously, if that was attempted with a
java_sdk_library it would be added to the sdk as if it was a normal
java_boot_libs which would prevent the sdk from containing the
API specific artifact such as current.txt files and stub libraries
and sources.

This change fixes that and adds a java_sdk_library as a java_sdk_libs
module.

Bug: 177892522
Test: m nothing
Change-Id: Ided57b846ce5b8940c7e898c786fd77602582ea2
2021-05-13 21:25:42 +01:00
Yuntao Xu
5b009aedf9 syntax check to jacoco filters for any build
Perform syntax check to jacoco filters for any build.

Bug: 183622051
Test: Manual
Test: TreeHugger
Change-Id: Iff90d7b45bdad8706c43c7e16d98270af688d20b
2021-05-13 12:42:24 -07:00
Paul Duffin
4bbf45faef Merge "Build boot images in bootclasspath_fragment/platform_bootclasspath" 2021-05-13 16:32:23 +00:00
Treehugger Robot
820397deb2 Merge "Add support for SOONG_SDK_SNAPSHOT_USE_SRCJAR" 2021-05-13 16:00:40 +00:00
Paul Duffin
f356e5a533 Merge "Separate storing config for MakeVars from building boot image" 2021-05-13 14:01:36 +00:00
Paul Duffin
cfd24da1f4 Merge "Move copying of dex files from dexpreopt_bootjars singleton" 2021-05-13 10:53:51 +00:00
Paul Duffin
22628d50f3 Add support for SOONG_SDK_SNAPSHOT_USE_SRCJAR
Test: m SOONG_SDK_SNAPSHOT_USE_SRCJAR=true ipsec-module-sdk
      - check generated snapshot.zip file.
Change-Id: I02991e2a60d7784984b308cff2c47ee809d61f01
2021-05-12 23:13:22 +01:00
Paul Duffin
2fc82ad044 Build boot images in bootclasspath_fragment/platform_bootclasspath
Moves the building of boot images from the dexpreopt_bootjars singleton
to the bootclasspath_fragment and platform_bootclasspath.

The art boot image is generated by the art-bootclasspath-fragment
module and the framework boot image by the platform-bootclasspath
module.

This does temporarly duplicate the generation of an identical boot
profile for each image. As part of the work to modularize the boot
image profile each image will have its own custom default boot profile.

Bug: 177892522
Bug: 186455808
Test: m droid and TreeHugger
Change-Id: I23cf05ec7648749b21c7cf6fcba282b46649a981
2021-05-12 21:51:27 +01:00
Paul Duffin
47932935e9 Separate storing config for MakeVars from building boot image
Previously, the initialization of the dexpreoptBootJars fields used to
store information needed in its MakeVars method was interleaved with
the calls to buildBootImage(). In fact those fields were initialized
from the *bootImageConfig returned by buildBootImage(). However, the
method simply returned the *bootImageConfig value that was passed in.

Separating the initialization from the calls to buildBootImage() allows
the functionality to be moved to platform_bootclasspath separately.

Bug: 177892522
Test: m nothing
Change-Id: I8ba460f60be553516de5f186e14377bb0ec1b2cd
2021-05-12 21:51:27 +01:00
Paul Duffin
7ebebfd5f8 Move copying of dex files from dexpreopt_bootjars singleton
The art dex files are copied in the bootclasspath_fragment and the
non-updatable and updatable dex files are copied in the
platform_bootclasspath.

Bug: 177892522
Test: m nothing
Change-Id: I5d3d533d1a7a9f8e7ae20c12eb33029a898a2cd6
2021-05-12 21:51:19 +01:00
Ivan Lozano
d7586b6526 Refactor vendor snapshot to use LinkableInterface.
Refactors the vendor snapshot support to use the LinkableInterface
so that support can be extended to Rust. This CL does not add
vendor snapshot support for Rust; that is left for a follow-on CL.

Bug: 184042776
Test: m nothing
Change-Id: Id0c4970ca00053484a52677d182153cbc454c301
2021-05-12 14:01:10 -04:00
Jaewoong Jung
ccbbeb6965 Merge "Strict updatability linting against dependencies." 2021-05-11 15:24:06 +00:00
satayev
3db3547d1f Generate empty classpaths.proto for bootclasspath_fragment.go.
- Adds all required details for bootclasspath_fragment to implement
  classpath_fragment.
- Keeps the actual boot jars in platform-bootclasspath to begin with.
- Makes sure to put the file in apex/etc/classpath on device. Note that
  for platform versions of classpath fragment AndroidMkEntries perform
  the installation, while for APEXes it must be plumbed via apex.go.

Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: I6101ebdf5b8bcbe95c0b7ce21f3f67a2685aef50
2021-05-11 10:34:47 +01:00
Jaewoong Jung
476b9d6d17 Strict updatability linting against dependencies.
Propagate strict_updatability_linting to transitive dependencies using a
top-down mutator.

Test: lint_test.go
Bug: 182349282
Change-Id: Ifc9e58f1a597e3c7725ee49b4027afb6f42f45cb
2021-05-10 15:30:00 -07:00
Treehugger Robot
564fce4578 Merge "Rename fields in dexpreopt config" 2021-05-08 01:47:39 +00:00
satayev
ee7e359131 Merge "Declare ConfiguredJarList in specific fragment implementations." 2021-05-07 19:41:08 +00:00
Treehugger Robot
3b755d4560 Merge "Split SYSTEMSERVERCLASSPATH entries from platform_bootclasspath." 2021-05-07 18:34:52 +00:00
satayev
013485bd83 Declare ConfiguredJarList in specific fragment implementations.
Each specific classpath_fragment module knows what jars must be part
of the corresponding classpaths.proto config.

Note that bootclasspath_fragment does not implement classpath_fragment
yet, thus all boot jars and all system server jars go into corresponding
platform classpaths.

Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: I6a8c7b0a5d17d62e790a441b8e2c5c1a816e7f30
2021-05-07 16:31:07 +01:00
Paul Duffin
225e4dbae4 Merge "Install updatable-bcp-packages.txt and boot-image.bprof" 2021-05-07 14:14:20 +00:00
Paul Duffin
320055eb8c Merge "Stop exporting java_sdk_library libs in the snapshot" 2021-05-07 13:34:33 +00:00
Jeongik Cha
a596909342 Rename fields in dexpreopt config
Add 'host' into the name of fields regarding path on the host side to
distinguish between paths on the device(which will be added in the
following commit), and paths on the host.

Bug: 158843648
Test: build and flash, and then
  adb wait-for-device \
    && adb -s $S root \
    && adb -s $S logcat \
    | grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C 1
Change-Id: Ib2645ed51591ba2f4b726c115b401ad2bd6675da
2021-05-07 21:40:23 +09:00
Paul Duffin
dbb490359a Merge "Make all SdkMemberTypes support transitive member deps" 2021-05-07 12:31:15 +00:00
satayev
95e9c5bbfa Split SYSTEMSERVERCLASSPATH entries from platform_bootclasspath.
Boot jars are different to system server jars at build level, due to
added complexity of dexpreopt. As a logical separation add a new
classpath fragment type and split existing classpaths.proto generation
into relevant pieces.

Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: I88bec09fc920166ffd0092faef980754ddeb6593
2021-05-07 13:24:16 +01:00
Paul Duffin
820ed78954 Merge "Add missing calls to InitSdkAwareModule" 2021-05-07 12:15:16 +00:00
Paul Duffin
be007d1800 Install updatable-bcp-packages.txt and boot-image.bprof
Previous refactorings of the code for creating these two files caused
them to be created before the boot.prof file instead of afterwards. As
a result even though they were appended to the list of files to install
they were discarded when the boot.prof file set the list rather than
appended to it. That worked before the refactoring because the
boot.prof file set the list first before the others were appended to
it.

This change appends the boot.prof file to the list making the order in
which they are added irrelevant.

Bug: 187494247
Test: m nothing
      - check DEXPREOPT_IMAGE_PROFILE_BUILT_INSTALLED in generated
        make_vars file to ensure it includes all 3 files.
Change-Id: Idb94531daf61b1b047c72eb5c67a57d3fdced05c
2021-05-07 13:07:34 +01:00
Paul Duffin
b17d044628 Transitively add APEX variants for contents of prebuilt_apex/apex_set
This is part of the work needed to allow the exported_java_libs
property to be replaced by exported_bootclasspath_fragments.

Bug: 187266082
Test: m nothing
      m SOONG_CONFIG_art_module_source_build=false nothing
Change-Id: I2c1d70a390200b93163f9799719290c9d55a041c
2021-05-06 23:43:43 +01:00
Paul Duffin
e7c94a689b Stop exporting java_sdk_library libs in the snapshot
The libs property in java_sdk_library is not safe to export as it can
contain libraries that contain implementation specific classes. No
snapshot appears to need this capability but if it does become
necessary then either it will need to use the existing stub_only_libs
property or a new one will need to be added.

Bug: 157884619
Test: m nothing
Change-Id: I60a069177dbee4070d311b23d25f4eb3c5e7ea13
2021-05-06 23:22:15 +01:00
Paul Duffin
2d3da31d41 Make all SdkMemberTypes support transitive member deps
Previously, only those SdkMemberTypes which had specific need to
automatically add some of their dependencies as sdk members would cause
the sdk to visit their transitive dependencies. However, as any module
can have dependencies on license modules and license modules need to be
included in the sdk then it needs to visit transitive dependencies of
all members.

So, this change removes the support for allowing an SdkMemberType to
control whether its transitive dependencies are visited and just visits
them all.

This does not have any effect on sdk snapshots as in order for a
dependency to be added to an sdk it needs to be added with a tag that
implements SdkMemberTypeDependencyTag and the only tags that implement
this are used by SdkMemberTypes that had enabled transitive members.

Bug: 181569894
Test: m art-module-sdk art-module-host-exports art-module-test-exports
      - verify that this change has no effect on the generated snapshots
Change-Id: If0293af0237aa7e39335e5b8383a41c023ff5853
2021-05-06 23:13:06 +01:00
Paul Duffin
b6b89a4074 Add missing calls to InitSdkAwareModule
A follow up change will add methods to SdkBase which requires its
module field to have been initialized.

Bug: 181569894
Test: m nothing
Change-Id: I9b02f260ad3f82316cc7ab3b5717b7e81090b0d8
2021-05-06 23:13:06 +01:00
satayev
128ce2ff73 Rename classpath_fragment.go methods for better readability.
Bug: 180105615
Test: m nothing
Change-Id: Ic663c22e5b7cbab487dc1fe99805e08843c3213d
2021-05-06 13:27:28 +01:00
satayev
63bc30fc12 Merge "Make SystemServerJars ConfiguredJarList." 2021-05-06 11:32:39 +00:00
satayev
9a6f87ebd6 Make SystemServerJars ConfiguredJarList.
Consistent with plumbing of boot jars. SystemServerJars only support /system/framework/ jars.

Bug: 180105615, 155630745
Test: m && launch_cvd
Change-Id: I58b005b7c4103c8e250090e995b1d9b2f9ed4a76
2021-05-05 14:24:02 +01:00
Anton Hansson
93e8f79577 Merge "Move stub providers to droidstubs.go" 2021-05-05 13:17:37 +00:00
Anton Hansson
5260932812 Move stub providers to droidstubs.go
They fit better there than in droiddoc.go.

Test: m nothing
Change-Id: I44588f1df2094f14881f920246bd19ffc6e1615c
2021-05-05 10:36:05 +01:00
Paul Duffin
4977540bcb Support dex_import on platform_bootclasspath
Maintain compatibility with previous behavior by ignoring dex_import
during hidden API processing.

Bug: 179354495
Test: m nothing
Change-Id: I976b02129bf981b7b61dce233567d6f89e04f92d
2021-05-04 14:07:23 +01:00
Colin Cross
29c294b2cf Time out and dump stacks from R8 processes after 30 minutes
R8 processes are sometimes hanging on the build servers.  Wrap R8 with
run_with_timeout to dump the stacks with jstack and kill the process
after 30 minutes.  Switch from running with the r8-compat-proguard
shell script to running the jar directly so that jstack gets the pid
of the java process.

Bug: 181095653
Test: m checkbuild
Test: m NetworkStackNextIntegrationTests
Test: m USE_RBE=true RBE_R8=true RBE_R8_EXEC_STRATEGY=remote NetworkStackNextIntegrationTests
Change-Id: If6996bd8eb39c7a8453d79e825004339c009ade2
2021-04-30 14:14:46 -07:00
Paul Duffin
21695e2ce9 Merge "Split findAndCopyBootJars into separate find and copy functions" 2021-04-30 16:09:56 +00:00
Paul Duffin
fdf4050440 Merge "Fix build failure when building unbundled apps (second try)" 2021-04-30 14:16:32 +00:00
Treehugger Robot
3f9af76391 Merge "Move configuration checks from getBootImageJar" 2021-04-30 12:56:56 +00:00
Paul Duffin
d504c3ac83 Fix build failure when building unbundled apps (second try)
The previous attempt, which simply skipped the hidden API processing
altogether when unbundled builds were enabled failed when attempting to
build module snapshots as while they enabled an unbundled build they
actually need the hidden API processing to be performed.

This change just checks whether missing dependencies are allowed and if
so it fakes up any missing files so that the build will only fail if
they are not present AND they are used.

Bug: 186695448
Bug: 185828824
Test: tapas Calendar
      m -j60

Change-Id: Ie13fed05af0aba51f45f6791fce944d0e4285037
2021-04-30 13:42:44 +01:00
Andrei-Valentin Onea
f794e8269d Merge "Exclude RequiresApi from generated stubs" 2021-04-30 12:31:07 +00:00
Paul Duffin
d6894ca4b9 Split findAndCopyBootJars into separate find and copy functions
The main difference between the dexpreopt_bootjars singleton and the
platform_bootclasspath singleton module is the way they find the
modules to use. The former searches all modules, the latter adds
dependencies on the modules that they need. This change separates the
finding of the modules from the copying of the boot jars for those
modules to make it easier to move the remaining functionality to
platform_bootclasspath.

This temporarily creates a singleton specific copy of the hidden API
function isModuleInConfiguredList() to select the modules in place of
the logic in the getBootJar() method. There is a slight loss of context
information from the error messages but as these methods will be
removed once the boot image creation has moved this is not an issue.

While switching the isModuleInConfiguredListForSingleton() to use the
SingletonContext the error message was fixed to include the name of
the module with the issue.

Bug: 177892522
Test: m nothing
Change-Id: Iaea906da95d9da5301fb964fc593890f2216d336
2021-04-30 12:10:54 +01:00
Paul Duffin
f23bc472b0 Move configuration checks from getBootImageJar
The getBootImageJar function will be removed once the boot image
creation has been moved to the platform_bootclasspath and
bootclasspath_fragment module types. However, the consistency checks
that it performs are still useful so this change moves them out
first.

The ART boot image related checks are now performed in the
bootclasspath_fragment module type. A previous change accidentally
disabled the checks when the contents property was not empty which has
been fixed. Also, the error messages have been tweaked to make it clear
that the art-bootclasspath-fragment is now the source of truth as to
its contents not the configuration.

The framework boot image related checks are now performed in the
platform_bootclasspath module type.

Initially, this change included an extra check to make sure that
UpdatableBootJars comes from updatable APEXes but that broke because
framework-wifi and framework-tethering are not currently marked as
updatable in AOSP.

Bug: 177892522
Test: m nothing
Change-Id: I80fb600fa2c7cec4566b3461c6a33c4c6f0743f4
2021-04-30 12:06:28 +01:00
Treehugger Robot
1468cc4a10 Merge "Revert "Fix build failure when building unbundled apps"" 2021-04-30 01:52:57 +00:00
Vishnu Nair
0dbd02a3ac Revert "Fix build failure when building unbundled apps"
This reverts commit c027119e73.

Reason for revert: b/186797512
Test: vendor/google/build/build_mainline_modules.sh -j80

Change-Id: I2bb062cce09ac6717702c4f6b110acbb2887adec
2021-04-30 00:24:07 +00:00
Paul Duffin
5983d856c1 Merge "Fix build failure when building unbundled apps" 2021-04-29 20:19:54 +00:00
Paul Duffin
c027119e73 Fix build failure when building unbundled apps
Bug: 186695448
Bug: 185828824
Test: tapas Calendar
      m -j60
Change-Id: I1c5365f6d2afb2f2d159e6f6ed004647ec6d2427
2021-04-29 19:50:40 +01:00
Andrei Onea
4985e518f8 Exclude RequiresApi from generated stubs
This annotation is required for linting, but is not useful in stubs.

Bug: 185579441
Test: cherry-pick http://ag/14098354 and regenerate stubs
Change-Id: I7e4da99a8843b9c709f3b59f125f43cbd66bda4e
2021-04-29 17:00:46 +00:00
Paul Duffin
12d29b7786 Move generation of hidden API make vars to platform_bootclasspath
Bug: 179354495
Test: rm out/soong/make_vars*
      m nothing
      grep INTERNAL_PLATFORM_HIDDENAPI_FLAGS out/soong/make_vars*
      - make sure it is still out/soong/hiddenapi/hiddenapi-flags.csv
Change-Id: I86e56512a9a2091f446bad25294d41ea1f4341ee
2021-04-29 15:11:35 +01:00
Paul Duffin
e3ecce67c1 Make platform_bootclasspath a singleton module
This is needed in order to allow it to implement MakeVars so it can
create make variables just like the dexpreopt_bootjars and hiddenapi
singletons currently do.

Bug: 177892522
Bug: 179354495
Test: m nothing
Change-Id: Ida5bf8abeecde531e1f6430151650065445804ac
2021-04-29 13:37:16 +01:00
Paul Duffin
f7a5592d38 Move dumpOatRules to platform_bootclasspath
Bug: 177892522
Test: m oat-dump-boot
      - test output to make sure that this change does not change the
        generated files, at least no more than no changes do as the
	output from this rule is not deterministic. See b/186459873.

Change-Id: Ib2b4203d9bb1fd0ee9443aee4e58b54b38b491cf
2021-04-29 11:50:48 +01:00
Paul Duffin
4c094428a7 Move generateUpdatableBcpPackagesRule to platform_bootclasspath
Changes generateUpdatableBcpPackagesRule to use ModuleContext instead
of SingletonContext and moves the call from dexpreoptBootJar's
GenerateSingletonBuildActions method to platform_bootclasspath's
GenerateAndroidBuildActions.

Bug: 177892522
Test: lunch art_module_arm64
      m out/soong/module_arm64/dex_bootjars/updatable-bcp-packages.txt
      - make sure it is not affected by this change
Change-Id: I9741ae1eb9573cafe12dd7a17dc1d8449b224dc8
2021-04-29 11:50:48 +01:00
Paul Duffin
01b463bc06 Differentiate between art, non-updatable and updatable modules
Bug: 177892522
Test: lunch art_module_arm64
      m out/soong/module_arm64/dex_bootjars/updatable-bcp-packages.txt
      - make sure it is not affected by this change
Change-Id: I3d35f1664cbbaba5df3f5809859dd9815a26d05d
2021-04-29 11:50:42 +01:00
Paul Duffin
9bacf56564 Extract logic to gather deps added for <apex>:<module> pairs
Trades having to traverse a module's direct dependencies multiple times
for reusable code. While at the minute the amount of duplicated code is
small (checking a tag and appending to a list) follow up changes will
modify the gatherApexModulePairDepsWithTag module to improve error
reporting greatly increasing the benefit of deduping.

The cost of traversing a module's direct dependencies is very small as
there will be only a very few dependencies.

Bug: 177892522
Test: m nothing
Change-Id: I16389102abd8038e1bfa52b63f4153bdf92ff553
2021-04-29 11:34:59 +01:00
Paul Duffin
110b0add9e Treat "apex" system_ext the same as platform
Change https://r.android.com/1672245 added support to treat
system_ext:foo in the boot jars configuration the same as platform:foo,
at least for dexpreopt_bootjars.go's getBootJar.

This change replicates that mechanism in platform_bootclasspath and as
that now handles hidden API processing also made a similar change in
the isModuleInConfiguredList function.

Bug: 177892522
Bug: 154976937
Test: m nothing
Change-Id: I105f4fbaa3b0355b013b7c5618d218d888faefb6
2021-04-29 10:35:10 +01:00
Paul Duffin
895c7140a9 Add stub_libs properties to bootclasspath_fragment snapshot
Bug: 177892522
Test: m nothing
Change-Id: I7644122c4abed8e8bc2cc7b58f408bee3bdc91f6
2021-04-28 22:02:00 +01:00
Paul Duffin
1093158faf Add support for specifying api provided by bootclasspath_fragment
The hidden API processing needs access to dex stub jars for the API
scopes provided by the bootclasspath_fragment so that it can mark the
corresponding dex members as being part of that API. This change adds
the ability to specify the modules that provide those jars, resolve the
modules to dex jar stubs and make them available for other modules and
tests to use.

While the stubs are only needed for hidden API at the moment these were
not added to the hidden_api properties section as there are other parts
of the build that may need to access the stubs in future, e.g. api
fingerprint and aidl generation.

The stubs properties can be affected by coverage as when coverage is
enabled the jacoco-stubs needs adding to the list, just like how
jacocoagent is added to the contents.

Bug: 177892522
Test: m nothing
Change-Id: I31097d1ca45c84adeba4cbb38f693698cb289e99
2021-04-28 21:59:46 +01:00
Andrei-Valentin Onea
b75031e918 Merge "Replace source based system server stubs" 2021-04-28 15:06:22 +00:00
Paul Duffin
8d817b70e6 Merge "Extract IsModuleInVersionedSdk" 2021-04-28 12:58:43 +00:00
Paul Duffin
44d1594081 Extract generateUpdatableBcpPackagesRule from updatableBcpPackagesRule
This simplifies the process of moving the updatableBcpPackagesRule to
the platform_bootclasspath by separating the gathering of the list of
updatable modules (which differs between the singleton and the
platform_bootclasspath module) from the gathering of the permitted
packages list and generation of the rule which are generally common.

Bug: 177892522
Test: lunch art_module_arm64
      m out/soong/module_arm64/dex_bootjars/updatable-bcp-packages.txt
      - make sure it is not affected by this change
Change-Id: I3cb64310f618059758a32cfe00d3745d52388e49
2021-04-28 09:03:46 +01:00
Paul Duffin
2ac45f019b Remove special handling of frameworks/base/config/boot-profile.txt
Previously, the method was run in repo manifests that did not include
the frameworks/base repository so it had to handle the file being
missing. However, now that this is being called from
platform_bootclasspath which is only defined in frameworks/base that
is no longer required.

Bug: 177892522
Test: m droid
Change-Id: I77fa5a204d1426a6be24a6f0b48e18f48f3dd908
2021-04-28 09:03:46 +01:00
Paul Duffin
ad19d3858e Move bootFrameworkProfileRule to platform_bootclasspath
Changes bootFrameworkProfileRule to use ModuleContext instead of
SingletonContext and moves the call from dexpreoptBootJar's
GenerateSingletonBuildActions method to platform_bootclasspath's
GenerateAndroidBuildActions.

Changing the context also allows the code to switch from
bootFrameworkProfileRule to GetGlobalSoongConfig.

Also extracts the shouldBuildBootImages function so it can be used by
platform_bootclasspath to preserve the existing behavior.

Bug: 177892522
Test: m droid
Change-Id: I30d3ca10be7f84348ad3aa9cc984dd15b8f6f4e9
2021-04-28 09:03:46 +01:00
Paul Duffin
0c2e08312c Extract IsModuleInVersionedSdk
Bug: 177892522
Test: m nothing
Change-Id: I348d380a0d29abdd6a84420be52cc1b674f2a7ab
2021-04-28 00:39:52 +01:00
Paul Duffin
976b0e5bbb Automatically export bootclasspath_fragment stubs from sdk
This change makes the tag that is used to add dependencies onto stubs
from the bootclaspath_fragment module cause the dependency to be
automatically exported from the sdk. If the dependency is a
java_sdk_library then it will be as if it was added to the
java_sdk_libs property, otherwise it will be as if it was added to the
java_header_libs property instead.

Bug: 177892522
Test: m nothing
Change-Id: I97010c4342dd39415dba81a4395db7831410436e
2021-04-27 23:27:34 +01:00
Paul Duffin
eee466e439 Pass module to SdkMemberTypeDependencyTag.SdkMemberType
This change allows an SdkMemberTypeDependencyTag to select the
SdkMemberType to use to add a dependency based on the module.

Bug: 177892522
Test: m nothing
Change-Id: I2d6e51b615636dc7cd41da65808ef851dd8ae6ac
2021-04-27 23:23:20 +01:00
Paul Duffin
5861242a78 Extract default image specific logic from buildBootImage
Previously, buildBootImage had to be called for the default boot image
config before the art boot image config as it created some rules which
were specific to default boot image. The creation of these rules was
encapsulated in Config().Once(..) to ensure that they were only ever
created once.

This change extracts that functionality out of buildBootImage to the
calling method which means that they are only called once and so no
longer need protecting against being called again.

Bug: 177892522
Test: m droid
Change-Id: I464477de1a08df15e577873a9accf7db2bc088d1
2021-04-27 15:56:47 +01:00
Paul Duffin
57e2e7d199 Simplify missing deps handling in buildBootImage
Previously, the buildBootImage function maintained a list of
missingDeps that was returned from findAndCopyBootJars and was passed
to various other methods that created ninja rules that depended on the
files created by findAndCopyBootJars. The purpose of that list was
basically to record that findAndCopyBootJars could not find some files
so that the subsequent ninja rules could be marked as being in error.

This changes avoids having to explicitly propagate the missing deps to
each rule by instead having findAndCopyBootJars generate an error rule
instead of a Cp rule for any missing file. That means that any attempt
to build any rule that depends on the missing file will fail when the
missing file is created.

Bug: 177892522
Test: Add an unknown module to PRODUCT_BOOT_JARS
      lunch art_module_arm64
      - Run each of the following targets and made sure it failed with a
        message indicating that the unknown module was missing.
      m SOONG_ALLOW_MISSING_DEPENDENCIES=true out/soong/module_arm64/dex_bootjars/boot.prof
      m SOONG_ALLOW_MISSING_DEPENDENCIES=true out/soong/generic_arm64/dex_bootjars/boot.bprof
      m SOONG_ALLOW_MISSING_DEPENDENCIES=true out/soong/generic_arm64/dex_bootjars/updatable-bcp-packages.txt
Change-Id: Ib7196c128419fbffdfeb61c1c5d47a3b08d1e132
2021-04-27 15:56:47 +01:00
Paul Duffin
81667c8c11 Remove panics from reportMissingVariationDependency
The code that has been removed triggers in many cases when it should
not, e.g. an unknown module on the bootclasspath. The code was
originally added to try and ensure that if the attempt to add useful
error messages failed it would help debug the issue. However, it ended
up just creating more failures.

If this does become an issue then this will be revisited.

Bug: 177892522
Test: Add an unknown module to PRODUCT_BOOT_JARS
      m SOONG_ALLOW_MISSING_DEPENDENCIES=true
Change-Id: I08dbd4b29d301f3f1f78a0fb396687062b47deb8
2021-04-27 15:56:47 +01:00
Treehugger Robot
bdae1e5765 Merge "Drop DexPreoptImages field in dexpreopt config" 2021-04-27 01:53:45 +00:00
Paul Duffin
2dc665bb48 bootclasspath_fragment: Always output contents property in snapshot
Bug: 177892522
Test: m art-module-sdk
      - check generated snapshot contains contents property
Change-Id: I122dedba6600a199bfd83b01988da36ddfd09d63
2021-04-26 21:21:58 +01:00
Paul Duffin
e95b53a55d Automatically add bootclasspath_fragment contents to sdk
Previously, both a bootclasspath_fragment and its contents had to be
explicitly added to the sdk. This change means that adding a
bootclasspath_fragment to and sdk will automatically add its contents
as if they were added using java_boot_libs.

Bug: 177892522
Test: m nothing
Change-Id: I8f7e70649f272c9a109d4606571a2d12c44b7904
2021-04-26 21:21:58 +01:00
Paul Duffin
ba6afd0dba Allow contents and image_name to be specified together
Previously, only one of the contents or image_name properties could be
specified at once which meant that there was no way to create a
prebuilt which lists its fixed contents while at the same time allowing
it to check that that the contents matched what the build configuration
required.

e.g. a prebuilt_bootclasspath_fragment that had image_name: "art",
could not list its contents and also check that those contents matched
the ART_APEX_JARS which the build configuration required.

This change allows contents and image_name to be specified together and
adds a check to make sure that the contents are consistent with the
configuration appropriate to the image_name. The check is only
performed for modules that are active so that a
prebuilt_bootclasspath_fragment which was created without coverage
enabled (the default) would not cause a build failure in a coverage
build unless it was preferred.

Bug: 177892522
Test: m nothing
Change-Id: Ie601f29f707b3f6030fa7d252afa2c4826cc9f8e
2021-04-26 21:21:58 +01:00
Paul Duffin
2da0424b19 Extract java SdkMemberType instances as vars
This will allow them to be used from elsewhere in a follow up change.

Bug: 177892522
Test: m nothing
Change-Id: I60e94e148af5b1810aea7f724cba090b49ae758e
2021-04-26 21:21:58 +01:00
Paul Duffin
c7d1644b0b Add coverage specific properties to bootclasspath_fragment
This allows a bootclasspath_fragment (specifically the
art-bootclasspath-fragment) to specify additional contents to be
appended when coverage is enabled.

The art-bootclasspath-fragment will use this to add jacocoagent to its
contents to ensure that it is always consistent with the configuration.

Bug: 177892522
Test: m nothing
Change-Id: I50d05fe5e0e9b8c14bdf3dfd63bba0ac97e31d48
2021-04-26 17:22:28 +01:00