Commit graph

3947 commits

Author SHA1 Message Date
Colin Cross
986b69aa51 Support dist_group property instead of owner for setting sdk dist subdirectory
Reusing the owner property is confusing, especially when the property is
required on every java_sdk_library module.  Create a new dist_group property
to use instead.

Bug: 186723288
Test: TestJavaSdkLibraryDist
Change-Id: I9e62c703a95d6b63cafa60bffb1b37ba85388593
2021-06-01 13:40:44 -07:00
Colin Cross
30c491b885 Add test for java_sdk_library dist properties
Add tests that cover the owner, core_lib and dist_stem properties.

Bug: 186723288
Test: TestJavaSdkLibraryDist
Change-Id: I4c2ae2a23dcd9a668c46ed5cc23b1d7dcfae798a
2021-06-01 13:40:17 -07:00
Colin Cross
1d2b6b3b2e Move java_sdk_library tests to sdk_library_test.go
Seperate the java_sdk_library tests into their own file.

Bug: 186723288
Test: go test ./java/...
Change-Id: I899c2946cb2234dc595a4281e64bbb239b89bda8
2021-06-01 13:40:17 -07:00
Colin Cross
c33e5216f1 Treat java libraries in classpath fragments as directly in apex
Coverage is applied to java libraries that are directly in an apex.
Mark java libraries that are in an apex through a bootclasspath_fragment
or a systemserverclasspath_fragment as directly in the apex by
implementing CopyDirectlyInAnyApexTag on the dependency tags used for
their contents.

Bug: 183759446
Test: TestApexJavaCoverage
Change-Id: I0116f5f415083b5194000988cb257454ef115200
2021-05-26 10:07:44 -07:00
satayev
70975ac815 Merge "Rename generate proto config file to match classpath type." 2021-05-25 14:17:04 +00:00
satayev
8a7bbb5b79 Merge "Populate individual systemserverclasspath_fragments' proto configs." 2021-05-25 11:38:40 +00:00
Treehugger Robot
c2d8fa0515 Merge "Clarify how metalava @SuppressLint works" 2021-05-25 00:21:39 +00:00
Aurimas Liutikas
b23b745e19 Clarify how metalava @SuppressLint works
Bug: 186157121
Change-Id: Ia3316b80dcf16e7fb2a41983732e0be8d0657497
Test: None
2021-05-24 18:01:17 +00:00
Paul Duffin
8018e50ddb Tighten bootclasspath_fragment property validation
Previously, due to legacy reasons, the property validation did not
require a contents property and allowed the image_name to be either
"art" or "boot". Those reasons no longer apply and so this change
requires a contents property and only allows the image_name to be set
to "art" if specified.

Bug: 177892522
Test: m nothing
Change-Id: I8855d6e5365ef0b55490e90e7b6c0081cf070ee5
2021-05-24 16:22:47 +01:00
Paul Duffin
58e0e769c9 Only export boot image files to APEX if they are actually created
Previously, the generation of the rules to create the boot image files
was separate from the code to provide those files to the APEX which
meant that it was possible for the APEX to try and use files that had
no rules to create them.

This changes avoids that by only exporting the files once the rules
have been created.

This necessitated a few changes to tests that were relying on the
previous behavior. Including removing completely the test that used an
image_name: "boot" as that is no longer a valid configuration name as
its functionality has been replaced by platform_bootclasspath.

A follow up change will make the validation of the properties of
bootclasspath_fragment stricter to prevent "boot" being used as the
image_name.

Bug: 177892522
Bug: 188680624
Test: lunch qemu_trusty_arm64-userdebug
      m droid dist
      - verify that before this change it fails and after this change
        it works.
      m com.android.art
      - verify that this change does not change the APEX contents with
        either qemu_trusty_arm64-userdebug or aosp_arm64-userdebug
Change-Id: I0497a151eb0731cbe6a1a7e7bbbb1e4dda75898f
2021-05-24 16:22:43 +01:00
Paul Duffin
32cf58a8fc Support removed API members in modular hidden API processing
Previously, the hidden API flags generated for a bootclasspath_fragment
did not include removed API members. That was because it did not supply
a file containing the dex signatures of the removed API members.

The monolithic hidden API processing uses combined-removed-dex which is
the output of a genrule that takes as input the *removed.txt files from
all the APIs and uses metalava to construct the dex signatures file.
This change does the equivalent for the *removed.txt files for the APIs
provided by a bootclasspath_fragment and then passes them to the rule
that generates the final all-flags.csv.

Bug: 179354495
Test: - Update packages/modules/RuntimeI18N to enable hidden API
        processing.
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - Before this change that fails as the flags generated for the
        i18n-bootclasspath-fragment differ from the monolithic flags.
	After this change that passes.
      - Verify that this change does not change any of the monolithic
        hidden API files.
      m com.android.i18n
      - Verify that the apex file before and after this change are byte
        for byte identical.
Change-Id: I6a21edb8a5231666e3f35b2c99a8687f36dd98fd
2021-05-24 15:05:21 +01:00
Paul Duffin
f1b358cb57 Support hidden API processing for fragments with dependencies
Previously, a bootclasspath_fragment that depended on classes provided
by another bootclasspath_fragment did not support hidden API processing
as it would not supply information about those dependencies.

This change adds support for that as follows. Each fragment:
1. Exports the transitive sets of stub dex jars for each of the public,
   system, test and core_platform APIs (where relevant).
2. Adds dependencies onto its dependent fragments.
3. Retrieves the API stubs dex jars from its dependent fragments and
   passes them to the "hiddenapi list" tool which will use them to
   resolve dependencies but will not output them to the generated
   flags.

Once the flags are generated the existing encoding functionality
encodes the flags into the dex files of the bootclasspath_fragment's
content modules which are then packaged into the apex.

Bug: 179354495
Test: m com.android.sdkext
      - verify that this does not change the contents of the apex files
Change-Id: I3e82a6dbb437f1e417e5d7e25aeb212e378603d0
2021-05-24 14:56:10 +01:00
satayev
703c67a0a9 Populate individual systemserverclasspath_fragments' proto configs.
Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: Ic0ae897e9baca75dcc022d84656496563645e74e
2021-05-24 13:41:20 +01:00
Paul Duffin
18cf19745e Remove bootclasspathApiInfo
Previously, the bootclasspathApiInfo was only used for tests and follow
up changes will need to provide the stub dex jars. This change moves
the stubJarsByKind into HiddenAPIInfo and removes bootclasspathApiInfo
and the corresponding provider.

Bug: 179354495
Test: m nothing
Change-Id: I5459c56de561c053ed671dc9d5cb3ee4820c0ee8
2021-05-24 13:32:56 +01:00
Paul Duffin
6237092391 Fix hidden API flags in com.android.i18n
Change 70cfdff3da changed the hidden API
flags in com.android.i18n as it stopped the i18n-bootclasspath-fragment
from making the hidden API flag files available for use by
platform-bootclasspath.

This change fixes that by exporting the flag files even if hidden API
flag generation is skipped.

Bug: 179354495
Test: m com.android.i18 out/soong/hiddenapi/hiddenapi-flags.csv
      - make sure that the flags in
        packages/modules/RuntimeI18n/apex/hiddenapi/hiddenapi-max-target-o-low-priority.txt
        are reflected in the core-icu4j dex files in the apex.
Change-Id: I9b5c7c74bd996ab447bc0e0452da5fd49191a35d
2021-05-24 13:32:55 +01:00
Paul Duffin
af99afa919 Rename hiddenAPIFlagFileInfo to HiddenAPIInfo
This reflects that it has expanded from its initial purpose to include
more than just flag files. It is exported for use in tests in other
packages.

Bug: 179354495
Test: m nothing
Change-Id: I9f780b20e18ce3a774e4aa04a276463070a64c34
2021-05-24 13:32:55 +01:00
Paul Duffin
1352f7c471 Separate input to flag generation from hiddenAPIFlagFileInfo
Encapsulating the information needed by hidden API processing in a
struct makes it easy to add additional information in future and allows
the code to populate that struct from various different sources to be
grouped together.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: I53805737dff36a3ae87aca5aad51cf46ae1361fe
2021-05-24 13:32:55 +01:00
Paul Duffin
1e6f5c4e63 Separate output of flag generation from hiddenAPIFlagFileInfo
HiddenAPIFlagOutput encapsulates the paths to the files produced by the
hidden API flag generation of a single bootclasspath_fragment. It is
returned from hidden API flag generation and is embedded within the
hiddenAPIFlagFileInfo so they can be passed to other modules.

Unlike the fields it replaces in hiddenAPIFlagFileInfo the fields in
HiddenAPIFlagOutput are of type Path not Paths which makes it easier to
use.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: I7373ba1001cac3a75eb54a23e62fa52f5013ee7f
2021-05-24 13:32:55 +01:00
Paul Duffin
438eb57a27 Separate monolithic hidden API processing from hiddenAPIFlagFileInfo
The hiddenAPIFlagFileInfo was being used for both the input and output
of bootclasspath_fragment and platform_bootclasspath and also to pass
information around to various hidden API rule methods. Supporting
multiple different uses in this way made it hard to reason about.

This change creates a separate structure for use by the
platform_bootclasspath. Follow up changes will split out other
functionality into separate types.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: Ia5c5f65ae5645486c42819c669a8601588217f88
2021-05-24 13:32:55 +01:00
satayev
227e745e56 Rename generate proto config file to match classpath type.
There is no need to leak soong module names.

Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: I04f4e181d2f42b9d71641980a2c7c4e8cbc8e426
2021-05-24 12:52:01 +01:00
Paul Duffin
54c98f5b4a Perform hidden API encoding in bootclasspath_fragment
Previously, the apex content info was populated with hidden API encoded
dex jars retrieved directly from the java module. This change retrieves
the unencoded dex jars from the java module, encodes them and then
stores the result in the apex content info.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: Ib1b6eb8b62ac50e03b9e0d07c877ca70bb6f6d25
2021-05-23 11:21:38 +01:00
Paul Duffin
0916595b1c Refactor hiddenAPIEncodeDex for use by bootclasspath_fragment
Previously, if the method was called multiple times by the same module
the resulting build rules would all use the same temporary directory
which meant that if run in parallel they would conflict with each
other. This change fixes that by providing a jar specific temporary
directory so it can be used by bootclasspath_fragment to encode its
content modules.

Also, cleans up, simplifies and improves the documentation.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: I0ebe61abc8e16111c6e8a822eb96c57846b98461
2021-05-23 11:21:38 +01:00
Spandan Das
fc1fbc428a Merge "Adding relative_install_path field to android_app_import" 2021-05-21 16:15:38 +00:00
Colin Cross
90463fe8cb Merge "Allow missing java_sdk_library files in AllowMissingDependencies builds" 2021-05-21 15:07:49 +00:00
Colin Cross
18f840c45e Allow missing java_sdk_library files in AllowMissingDependencies builds
java_sdk_library modules in builds with AllowMissingDependencies may
be missing defaults modules that cause them to look for api files that
they normally wouldn't need.  Move the error to runtime so it doesn't
block the build unless the branch tries to build that module.

Test: prebuilts/build-tools/build-prebuilts.sh in aosp-build-tools
Change-Id: I279b0cd8493779f972c0ac02235967c10b35a5a0
2021-05-20 18:00:37 -07:00
Spandan Das
d1fac64807 Adding relative_install_path field to android_app_import
Test: Ran go test ./java from build/soong
Bug: 149873762
Change-Id: I4abc1fb9e2386c7319b446b84aebaabacc882fd6
2021-05-20 23:10:26 +00:00
satayev
8fab6f86af Populate individual classpath_fragments' classpaths.proto configs.
To avoid duplicates on *CLASSPATH environ variables at runtime, remove
split entries from platform-*classpath, i.e. all updatable jars that
have their own classpath fragments should not appear in the
platform-*classpath's classpaths.proto config.

Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: Id2759ab8e106cc183e695bf3509a6ab60ab0ef2a
Merged-In: Id2759ab8e106cc183e695bf3509a6ab60ab0ef2a
2021-05-20 20:06:03 +01:00
Martin Stjernholm
ecf2dc4614 Merge "Skip TestDex2oatToolDeps on Darwin." 2021-05-20 16:52:30 +00:00
Martin Stjernholm
1b784a7caa Skip TestDex2oatToolDeps on Darwin.
This fixes https://r.android.com/1711292.

Test: m nothing
Bug: 188647117
Bug: 145934348
Bug: 172480615
Change-Id: I0c80b546a814d799562f374148eae5ca23b0e1f8
2021-05-20 15:53:00 +01:00
Paul Duffin
bf4e6ec8a7 Merge "Store dex jar paths in bootclasspath_fragment's apex content info" 2021-05-20 12:46:48 +00:00
Paul Duffin
94e594c799 Merge "Move hidden API encoding after resource merging" 2021-05-20 12:45:42 +00:00
Jiyong Park
6fae608181 Merge changes I4e7a7ac5,I0c73361b
* changes:
  Record the actual APEXes that a module is part of.
  Rename InApexes -> InApexVariants
2021-05-20 09:59:59 +00:00
Paul Duffin
40aa4363cb Merge "Workaround to make AlwaysUsePrebuiltSdks() work with platform_bootclasspath" 2021-05-20 06:58:34 +00:00
Paul Duffin
1a8010a241 Store dex jar paths in bootclasspath_fragment's apex content info
Previously, the DexBootJarPathForContentModule(module) simply called
directly through to the module to retrieve the dex jar path. This
change changes it so the bootclasspath_fragment retrieves the dex
jars from the module and stores them in the info structure for this
method to retrieve directly.

This makes it easier for the bootclasspath_fragment to stop retrieving
hidden API encoded dex jars from the module and perform the encoding
itself.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: Ic79dea080f10f4017f1a75d6d1fb5a3bfe04c2ce
2021-05-19 16:17:30 +01:00
Paul Duffin
4de9450433 Move hidden API encoding after resource merging
Previously, the hidden API encoding was done before resource merging.
However, hidden API modularization requires that the encoding be done
by the bootclasspath_fragment/platform_bootclasspath modules which will
be after the resource merging. Therefore, this change moves the hidden
API encoding after to match the future behavior.

It also moves the initHiddenAPI() method call after resource merging
too and passes it the result of the resource merging so it is available
for the bootclasspath modules via bootDexJar().

Although the resource merging was not always done when it was done it
would reorder the entries in the generated jar to match java ordering,
which puts the MANIFEST.MF first. This change preserves that behavior
by adding -j to the call to MergeZipCmds. This does mean that jars
which did not require resource merging now have a different order but
as both orders work that is not a significant change.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that apart from the ordering change in the jars that this does
        not change the contents of the apex files
Change-Id: If74baad5659301ca6ca9c0f6484374420dda8c34
2021-05-19 13:48:25 +01:00
Martin Stjernholm
b81b29c2f6 Merge changes Ibc673303,I70317eb8
* changes:
  Don't fail if the target module is disabled in dex2oat tool dependencies.
  Use oatdump rather than oatdumpd for boot jar boot.*.oatdump.txt files.
2021-05-19 11:47:07 +00:00
Paul Duffin
7487a7abb6 Workaround to make AlwaysUsePrebuiltSdks() work with platform_bootclasspath
The AlwaysUsePrebuiltSdks() causes all java_sdk_library_import modules
to be preferred over the source, i.e. as if they had prefer: true set.
That interacts badly with the work that is being done to integrate the
bootclasspath_fragment/platform_bootclasspath modules into the build.

It would work fine once that integration has been completed but in the
interim it causes problems. e.g. it does not cause a problem in AOSP
because those java_sdk_library_import modules that are affected have
already been integrated into the build properly.

Unfortunately, internally that is not the case because there are
java_sdk_library/java_sdk_library_import modules that still need to
be updated.

Before the java_sdk_library_import can be safely preferred each
java_sdk_library/java_sdk_library_import module that contributes to the
bootclasspath must:
* Be in the contents of matching bootclasspath_fragment and
  prebuilt_bootclasspath_fragment modules.
* Have an apex and one of a prebuilt_apex/apex_set that contains the
  dex implementation jar and lists the prebuilt_bootclasspath_fragment
  name in its exported_bootclasspath_fragments property.

Safely preferred in this context means that the whole build will
continue to work rather than the current situation which is that only
some of the build will work and some will fail if an attempt is
actually made to build it.

Unfortunately, many java_sdk_library_import modules are missing:
* The prebuilt_bootclasspath_fragment.
* The exported_bootclasspath_fragments property on the
  prebuilt_apex/apex_set that contains them.

Together these cause the following symptoms:
1. The java_sdk_library_import does not have a dex implementation jar.
2. The java_sdk_library_import does not have a myapex variant.

These workarounds will avoid Soong reporting build failures. However,
the build will still fail if an attempt is made to build anything
produced by the platform-bootclasspath, e.g. hidden API processing or
a system image.

Bug: 188505921
Bug: 179354495
Test: m TARGET_BUILD_APPS=Calendar
Change-Id: I3226e21cd6a7f9e4d6bbe94e54129ac5e1d4c679
Merged-In: I3226e21cd6a7f9e4d6bbe94e54129ac5e1d4c679
2021-05-19 11:42:20 +00:00
Treehugger Robot
c798d27e2c Merge "Use absolute paths in classpaths.proto." 2021-05-19 09:14:05 +00:00
Martin Stjernholm
0e4ccebb16 Don't fail if the target module is disabled in dex2oat tool
dependencies.

dexpreopt.RegisterToolDeps runs late after prebuilt dependencies have
been resolved, and there's special code in dex2oatPathFromDep to
resolve the prebuilt from the source module. However, if the source
module is disabled then the dependencies check in validateAndroidModule
will complain, so we need to disable that check in this particular
situation.

Also add a comment to explain why dexpreopt.RegisterToolDeps needs to
run so late.

Test: m nothing
Bug: 145934348
Bug: 172480615
Change-Id: Ibc673303d0336768fa23261a2068e91a08f46a30
2021-05-18 21:58:38 +01:00
Martin Stjernholm
0780372ae8 Use oatdump rather than oatdumpd for boot jar boot.*.oatdump.txt files.
oatdumpd isn't available as a prebuilt.

Test: m SOONG_CONFIG_art_module_source_build=false droid
Bug: 172480615
Change-Id: I70317eb8f253272d629f23063cbe265d556caad3
2021-05-18 21:57:28 +01:00
Treehugger Robot
fe027e0ec9 Merge "Update flags passed to kotlinc" 2021-05-18 20:54:39 +00:00
satayev
7d22657c45 Use absolute paths in classpaths.proto.
The intention before was to use relative paths to a partition where a
config is defined. However, jars in /system_ext partition are planned to
be declared in /system's classpaths.proto config.

Bug: 180105615
Test: derive_classpath_test, CtsClasspathsTestCases
Change-Id: Icc3e1a903c34187cfcd67a3ae7bc3dd746445c03
2021-05-18 19:37:32 +01:00
Jiyong Park
78349b56a0 Record the actual APEXes that a module is part of.
Consider this case:

apex {
    name: "com.android.foo",
    native_libs: ["foo"],
}

override_apex {
    name: "com.mycompany.android.foo",
    base: "com.android.foo",
}

cc_library {
    name: "foo",
}

There are two APEXes defined: "com.android.foo" and
"com.mycompany.android.foo" which is a copy of "com.android.foo" with
some properties overridden (e.g. signing keys).

The module "foo" is mutated into two variants by the apex mutator: the
platform variant and the apex variant. The former has the variation name
"" and the later has "apex<min_api_ver>" which usually is "apex10000".

Internally, the apex variant has an alias "com.android.foo".

ApexInfo.InApexVariants() returns only "com.android.foo" when called for
the module "foo".

We can see that the information that "foo" is also part of
"com.mycompany.android.foo" is completely lost. This is causing problem
when we compare the apex membership by their "soong module name", not
the "apex name". In the example above, the two modules have different
soone module names, but have the same apex name: "com.android.foo".

To fix that, this CL introduces a new field `InApexes` to the `ApexInfo`
struct. It has the actual name of the APEXes that the module is part of.
With the example above, `InApexes` is ["com.android.foo",
"com.mycompany.android.foo"].

Bug: 180325915
Test: m nothing
Test: m nothing on non-AOSP targets with ag/13740887 applied.

Change-Id: I4e7a7ac5495d2e622ba92a4358ed967e066c6c2e
2021-05-18 16:05:29 +09:00
Jiyong Park
ab50b0795a Rename InApexes -> InApexVariants
.. in preparation for the upcoming change. This change doesn't alter any
behavior.

InApexes is a misleading name. People expects that it has the list of
soong module names of the APEXes that a module is part of. So, for
example, `core-oj` is a part of both `com.android.art` and
`com.google.android.art`. However, in reality, that's not true. The
field has `com.android.art` only. This is because the two APEXes
(android and Google) have the same apex name which is `com.android.art`.
That apex name is used in various places like the `apex_available` and
allows us to keep using the same name regardless of whether the APEX is
overridden or not.

However, this is causing problems in some cases where the exact list of
soong module names is required. The upcoming change will add a new field
to handle the case and the new field actually will get the name
'InApexes'. So, the existing field is renamed to a less misleading name
`InApexVariants`.

Bug: 180325915
Test: m nothing

Change-Id: I0c73361b452eddb812acd5ebef5dcedaab382436
2021-05-18 16:05:14 +09:00
satayev
f1d8819d7d Merge changes I8e8e8b01,Ifb1f54d5,I9986e64f
* changes:
  Add systemserverclasspath_fragments property to apex.
  Add "contents" property to systemserverclasspath_fragment.
  Move classpaths.proto related info into a separate provider.
2021-05-17 22:40:08 +00:00
satayev
3d3a2218a6 Merge "Add no-op systemserverclasspath_fragment module." 2021-05-17 22:37:29 +00:00
satayev
333a1732b1 Add systemserverclasspath_fragments property to apex.
- All contents of the fragment are added as java_lib dependencies.
- Generated classpaths.proto is added into etc as required.

Bug: 180105615
Test: m nothing
Change-Id: I8e8e8b019c4ca2909182f205a47deffa946de6da
2021-05-17 21:46:08 +01:00
satayev
9366a053da Add "contents" property to systemserverclasspath_fragment.
Similar to bcp_fragment's contents, this property lists all java library
contributions made by this fragment.

Bug: 180105615
Test: m nothing
Change-Id: Ifb1f54d5db290fffaa31933d15207014bb72d2fb
2021-05-17 21:46:08 +01:00
satayev
14e49130bb Move classpaths.proto related info into a separate provider.
The new info struct can be easily shared with systemserverclasspath
fragments.

Bug: 180105615
Test: m nothing
Change-Id: I9986e64fdf19f4168da63c156de3dc9bcafac8d8
2021-05-17 21:46:04 +01:00
satayev
aa86bac2b0 Add no-op systemserverclasspath_fragment module.
This would allow to start introducing these modules for apexes that
contribute to SYSTEMSERVERCLASSPATH.

In follow up, it will be evolved:
- platform_systemserverclasspath would have "fragments" property to
  list all individual systemserverclasspath_fragments;
- systemserverclasspath_fragment would have "contents" property to list
  contibuting java libs;
- systemserverclasspath_fragment would generate non-empty
  classpaths.proto config within individual apexes.

Bug: 180105615
Test: m nothing
Change-Id: Ibaaa3fae5f1eab9a41ceecc1214a53be6bbc8ba6
2021-05-17 20:53:59 +01:00