Commit graph

63 commits

Author SHA1 Message Date
Colin Cross
045ed0f684 Merge changes I71a83e3a,I66101c0c,Ie387c8c4,Iea742e75
* changes:
  Strengthen metalava sandbox support using sbox
  Move metalava's output files into a subdirectory
  Fix lint warnings in droidstubs.go
  Split droidstubs out of droiddoc.go
2021-03-26 15:14:53 +00:00
Colin Cross
cb77f75aae Move metalava's output files into a subdirectory
Move all of the output files of the metalava rule into a single
subdirectory to make it compatible with sandboxing in sbox.

Test: TestDroidstubs
Change-Id: I66101c0c224dee702c8175e61c12cc9cd1aa8b93
2021-03-25 11:11:36 -07:00
Paul Duffin
db462dd987 Disallow non-existent paths in sdk package
Test behavior was changed a while ago so that tests by default ignore
non-existent source paths (unless they explicitly check for/rely on
them). Prior to that CheckSnapshot() could detect when files were
missing from the snapshot but it no longer can.

This change disallows non-existent source files in all the sdk tests
which means that they are disallowed when processing the snapshots as
they use the same preparers as were used to process the sources.

This caused a test failure which has been temporarily ignored and has
a TODO and bug associated with it.

Bug: 183184375
Test: m nothing
Change-Id: I969d8515d20ef5ae515f2b5f93d8ed4e4f8ede75
2021-03-25 12:53:22 +00:00
Paul Duffin
1822a0a371 Improve sdk snapshot testing
The sdk produces snapshots that are expected to be unpacked in one of
the Android repos. Often that can lead to issues due to conflicts
between the source and prebuilts. This change attempts to avoid those
conflicts by testing the different ways that those files can be used.

With the existing test to cover adding the snapshot
This change will cause the sdk tests to check the following:
1) Snapshot on its own (already done).
2) Snapshot plus original source where the original source is
   preferred.
3) Snapshot plus original source where the snapshot is preferred.

It also adds the ability for tests to provide their own custom checkers
to verify the result of each of the previous tests.

This change reveals a number of bugs already present. Rather than
attempt to fix them this change adds the ability to specify error
handlers for the two cases that mix source and snapshot to allow those
errors to be temporarily ignored while allowing the majority of the
tests to benefit from this improvement. Each of those failures has a
TODO and bug associated with it.

Bug: 183184375
Test: m nothing
Change-Id: I105233195074dbe7a6422b6dfc5486e74398ea15
2021-03-25 12:50:18 +00:00
Paul Duffin
ac94726ef2 Remove testSdkWithJava
Bug: 181070625
Test: m nothing
Change-Id: I9ef906a386cc87f69b166ec88e0b6c7388c3d06a
2021-03-24 15:47:28 +00:00
Paul Duffin
9ec86b14e7 Convert java_sdk_test.go tests to fixtures
Bug: 182638834
Test: m nothing
Change-Id: I6a1fabdd2c8385e5fbaef6985047f9d0bdceb209
2021-03-24 15:47:28 +00:00
Paul Duffin
d14cbc1e69 Add prepareForSdkTestWithJava
Bug: 182638834
Test: m nothing
Change-Id: I86cb08a640c7e39c1eaaf54926699f32a32ccc9b
2021-03-21 11:01:13 +00:00
Paul Duffin
36474d322b Stop sdk package depending on testing.T being embedded in TestResult
This change is in preparation for removing testing.T from TestResult.

Bug: 181070625
Test: m nothing
Change-Id: I67535aff0d894e6e3d8456b75540f340af853355
2021-03-13 15:59:33 +00:00
Paul Duffin
4a2a29ce6a Support test fixtures in sdk package
Bug: 181070625
Test: m nothing
Change-Id: Ifc96992e54c1b1d89a82b88ab27e555ae267a51e
2021-03-11 17:25:29 +00:00
Paul Duffin
981b94b460 Switch CheckSnapshot from a testSdkResult method to a function
This will allow the testSdkResult to be replaced with the TestResult
when switching sdk package to use test fixtures.

Bug: 181070625
Test: m nothing
Change-Id: Ieca63f4c189f5e804102aeb4073289ea03143b6e
2021-03-11 13:39:24 +00:00
Paul Duffin
8306f25679 Make testSdkResult compatible with android.TestResult
This change makes it easier to switch the sdk package over to using the
new fixture mechanism by removing inconsistencies between the
testSdkResult and TestResult structures.

Bug: 181070625
Test: m nothing
Change-Id: Ic4c06e08ea5060fd09123f2ca65580e18b4d2ef6
2021-03-11 13:38:56 +00:00
Mathew Inwood
f8dcf5ead2 Make apex.updatable default to true.
Update tests accordingly and add a new test case for this.

Bug: 180375550
Test: Treehugger
Change-Id: I835e189f4dae1e4bc79dce7bc59b7b9c7bd19fd9
2021-03-03 10:28:26 +00:00
Paul Duffin
22ff0aaf51 Export implementation class jars for java_boot_libs
Hiddenapi processing currently requires access to the class
implementation jars for libraries on the bootclasspath which means that
they need to be provided as part of the prebuilts. This change modifies
the java_boot_libs property on the sdk to make those files available.

Modularization of the hiddenapi processing will hopefully remove the
need for these to be exported so this should be temporary.

Bug: 178361284
Test: m art-module-sdk
      check generated snapshot zip contains implementation jars
Change-Id: I9e94662dddb0ddb85a477ae6d27e533085147e88
2021-02-05 13:35:25 +00:00
Paul Duffin
db170e4a92 Add java_boot_libs to sdk
The build has some implicit dependencies (via the boot jars
configuration) on a number of modules, e.g. core-oj, apache-xml, that
are part of the java boot class path and which are provided by mainline
modules (e.g. art, conscrypt, runtime-i18n) but which are not otherwise
used outside those mainline modules.

As they are not needed outside the mainline modules adding them to
the sdk/module-exports as either java_libs, or java_header_libs would
end up exporting more information than was strictly necessary. This
change adds the java_boot_libs property to allow those modules to be
exported as part of the sdk/module_exports without exposing any
unnecessary information.

Some points to note:
* The java_import has to have a valid file for the src property
  otherwise it will be disabled.
* The src property is supposed to reference a jar file but the
  java_boot_libs property will make it reference an empty file (not
  an empty jar) so that any attempt to use that file as a jar, e.g.
  compiling against it, will cause a build failure.
* The name of the file passed to the src property should make it
  clear that the file is not intended to be used.
* The test makes sure that only the jar file is copied to the
  snapshot.

Test: m nothing
Bug: 171061220
Change-Id: I175331e4c8e3874ab70a67cdc2f76ed1576e41eb
2021-01-15 18:14:10 +00:00
Anton Hansson
dff2c78a20 Add attribute to disable last-api compat tracking
Setting this to true by default is dangerous as it can mask bugs. Create
a dedicated attribute for java_sdk_library to enable this behavior
instead. The default will be flipped in a future CL when all the current
offenders have been fixed.

Fix all the tests to have the right API files.

Bug: 176092454
Test: m nothing
Change-Id: Ieab94bcb74abf8d018365a56fb447fe3dbd46957
2020-12-21 17:25:30 +00:00
Paul Duffin
ab5ac8f169 Revert "Use glob for java_sdk_library_import stub_srcs"
This reverts commit 7f97957ded.

Reason for revert: breaks sdk snapshots b/173508731
Bug: 173508731
Test: Ran prebuilts/runtime/update.py and then m nothing
      Before revert it failed
      After revert it worked

Change-Id: I9c081681fac589e37788a0d592435e3224011c58
2020-11-19 12:03:51 +00:00
Treehugger Robot
0a829f2a0a Merge "Always set apex_available in SDK snapshots." 2020-11-17 01:30:46 +00:00
Paul Duffin
a3cb6cf2e5 Remove support for droidstubs in sdk/module_exports
The droidstubs support in sdk/module_exports was a temporary measure
to work around the fact that some APIs were being defined by direct use
of droidstubs instead of java_sdk_library.

This change removes that support as those APIs have all been switched
from droidstubs to use java_sdk_library so droidstubs support is no
longer needed.

Bug: 168301990
Test: m nothing
Change-Id: I3517bed29b030438a0423a6cb8c248992a988222
2020-11-10 13:55:17 +00:00
Paul Duffin
7f97957ded Use glob for java_sdk_library_import stub_srcs
Some java_sdk_library modules do not have any stubs for some API
scopes. That results in an empty ".srcjar" being created for them which
ends up not creating a directory for the stubs sources when that
snapshot is unzipped. Previously, that would cause a build failure as
the generated java_sdk_library_import module used the directory, which
did not exist, in its stub_srcs property.

This change switches the stubs_srcs property to use a glob pattern of
"**/*.java" relative to the directory instead of using the directory
directly. When the directory does not exist the glob pattern is
resolved to an empty set of paths and does not break the build.

Bug: 172811712
Test: Add sdkextensions-sdk sdk module (local patch)
      m sdk-extensions (local patch)
      unpack generated snapshot
      m nothing - results in build failure due to missing directory
      Make this change and repeat above except this time the build works.
Change-Id: I691ffbfdc01ba89bbcaf647dcbb7dfebc3c8aec2
2020-11-09 17:42:52 +00:00
Martin Stjernholm
1e04109c6e Always set apex_available in SDK snapshots.
Also fix //apex_available:platform that could get lost in the generated
apex_available properties. It only happened if a module didn't specify
an apex_available setting and then got extra entries from
apex.BaselineApexAvailable.

Test: Build ART SDK with libnativeloader_lazy in the SDK, and check
  that the generated prebuilt module has //apex_available:platform
  (before https://r.android.com/1480559).
Change-Id: I9df20aee63455932a7e558993f0f26769cb02792
2020-11-04 18:44:14 +00:00
Colin Cross
405af07859 Revert "Make lots of tests run in parallel"
This reverts commit 323dc60712.

Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests

Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
2020-10-09 18:34:24 -07:00
Colin Cross
323dc60712 Make lots of tests run in parallel
Putting t.Parallel() in each test makes them run in parallel.
Additional t.Parallel() could be added to each subtest, although
that requires making a local copy of the loop variable for
table driven tests.

Test: m checkbuild
Change-Id: I5d9869ead441093f4d7c5757f2447385333a95a4
2020-10-06 15:12:22 -07:00
Paul Duffin
d99d997238 Explicitly specify visibility in sdk/module_exports snapshot
This change ensures that each prebuilt in a snapshot explicitly
specifies its visibility even when that visibility is the current
default. This is done for two reasons:
1. It simplifies a follow up change that adds visibility rules to an
   existing set of rules.
2. It ensures that the snapshots are independent of the current Soong
   default visibility.

The latter is important because we intend to switch from modules being
visible to everyone by default (i.e. //visibility:public) to only being
visible to modules in the same package (i.e. //visibility:private). By
making the snapshots of modules that do not specify any visibility
explicitly specify that they are "//visibility:public" it ensures that
the snapshots will not need to be changed when the default changes.

Bug: 168301990
Test: m nothing
Change-Id: Ia034f4a1e5124c17f46d73b0e9a6c5f2a251038e
2020-10-01 18:20:13 +01:00
Paul Duffin
a2ae7e0358 Allow java_sdk_library to specify doctags_files
When generating Javadoc the processor needs to be given information
about the doctags that are present in the source. This change allows
that information to be managed with the java_sdk_library that generates
the stubs source from which the Javadoc is generated.

Bug: 168301990
Test: Built offline-sdk-docs with and without the change and
      diffed them. The only difference was the timestamp.js
      file.
Change-Id: I4adbeb0781bc2191461fec856ffa90ea185e7434
2020-09-28 18:35:57 +01:00
Paul Duffin
ee9ad5d526 Remove framework-modules naming scheme
The naming_scheme property was not removed as it may be useful for
future when migrating to java_sdk_library.

Bug: 168301990
Test: m nothing
Change-Id: Ie97dd60355a207f1312a2dd910f1fb25b46fd737
2020-09-11 16:09:15 +01:00
Colin Cross
aede88c1c7 Reland: Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-19 10:21:17 -07:00
Paul Duffin
a37eca27c8 Avoid creating APEX variant for sdk member
Previously, an APEX variant was created for a module that was a member
of an SDK just in case it had to be replaced with an APEX requested
snapshotted version of that member. That was necessary because that was
the only way to have APEX specific replacements.

Since then a new method called ReplaceDependenciesIf() has been added
which provides fine grained control over which dependencies are
replaced. This change uses that new method to handle the replacements
which allows the APEX variants to be removed.

Bug: 161928524
Test: m nothing
Change-Id: If3869dd6753dc182b099af566b20fbc9c9c6eff7
2020-07-24 11:10:10 +01:00
Pete Gillin
84c3807b99 Enforce the legacy core/platform API restriction.
This change silently decides whether modules which depend on either
sdkPrivate or sdkCorePlatform get the legacy or the stable version of
the core/platform API, based on whether the module's name is on a
hard-coded list or not.

Test: m java
Test: make a target from the list when its entry is commented out, which correctly fails
Bug: 157640067
Change-Id: I15e5a6c2f07e73718803501d705de0d7ab9bec90
Merged-In: I15e5a6c2f07e73718803501d705de0d7ab9bec90
Merged-In: Iaa97ddaa015e8079fcb3426585c5101c7ec9e22a
(cherry picked from commit c0f4373106)
2020-07-23 09:06:52 +01:00
Paul Duffin
15f34ef811 Generate module lib API and scope together
Previously, due to limitations in metalava the module lib API and stubs
source were generated separately. That limitation has since been
removed so this change optimizes the generating code.

Changes:
* Add an annotation field to apiScope to contain the annotation that
  identifies the scope.
* Rename droidstubArgs to extraArgs to better reflect that they are not
  passed to droidstubs.
* Rename droidstubsArgsForGeneratingStubsSource to droidstubsArgs.
* Remove droidstubsArgsForGeneratingApi and
  createStubsSourceAndApiTogether fields along with code that supports
  generating stubs source and the API separately.
* Fix the module lib test.

Test: m checkapi
Bug: 161364853
Change-Id: I821886bb68645addf5ae9c96ed2f4f3649151d1a
2020-07-20 20:51:25 +01:00
Martin Stjernholm
3ff2e661bd Disable all SDK snapshot generation tests on mac.
We frequently miss putting in the per-test skip condition, and since
mac isn't tested in presubmit this leads to build breakages and time
consuming reverts. To avoid that this blanket disables all the SDK
tests on mac. It's not used there and we got test coverage in many
linux-based builds.

Bug: 145598135
Bug: 161315642
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I2aea92fef2c0f8c2742396fe36610501dc5a6f0f
2020-07-15 16:58:05 +01:00
Paul Duffin
cee7e66b07 Ensure that sdk/module_exports depends on source members
Previously, preferring a prebuilt of an sdk/module_exports's member
would cause the sdk/module_exports to depend on the prebuilt instead
of the source and cause problems with the build.

This chance prevents the dependency from an sdk/module_exports to its
members from being replaced with prebuilts.

Bug: 160785918
Test: m nothing
Change-Id: Iee4bcd438c11929e30fb5766701b05a0e89956d9
2020-07-09 17:51:06 +01:00
Paul Duffin
4e77284ab4 Include resources in sdk snapshot of java library impl
Bug: 159607838
Test: m nothing
Change-Id: Ie7ddbffc3957f832abfadb27292c436e8e5dcbf3
2020-06-24 13:04:15 +01:00
Paul Duffin
0c5bae5e83 java_sdk_library: Add system-server scope
Bug: 155164730
Test: m nothing
Merged-In: I49a2dab5c064b05f16691a3fae65f2b4ffc53bfd
Change-Id: I49a2dab5c064b05f16691a3fae65f2b4ffc53bfd
(cherry picked from commit 5a757b1ebb)
2020-06-15 15:02:19 +01:00
Pete Gillin
1f41dbff64 Split the core/platform API into stable and legacy versions.
For now, everything outside libcore still uses the legacy version.

Test: treehugger
Bug: 157640067
Change-Id: If5234e9ee533ff537926801a0af045d36b1caf01
2020-06-08 10:53:51 +01:00
Paul Duffin
3dbf9fd892 Copy removed.txt file to the snapshot correctly
Previously, the code copied the current.txt file to both the
current_api and removed_api properties. This change copies the
removed.txt file to the removed_api property instead.

Bug: 157980685
Test: m nothing
Change-Id: Iad34e91051da43222d22c240c16f50887c43d73e
2020-06-02 14:40:28 +01:00
Paul Duffin
d7eb1c2a76 java_sdk_library: Propagate shared_library to snapshot
Passes the shared_library property through to the snapshot. It does not
optimize away the default value in order to make it easier to invert
the default value in future. The current default value was only chosen
for convenience because most existing usages were already treated as
shared libraries. It would be safer if modules had to opt in to be used
as shared libraries.

Bug: 155164730
Test: m nothing
Change-Id: I33c7323f2389b44ed49cebe517ae2cce349117f1
2020-05-28 17:39:25 +01:00
Paul Duffin
6b836ba566 Correct module_lib scope name
The module_lib scope should be called module-lib in order to pick up
the latest filegroup. Without it the API lint does not use a baseline
and so reports issues with released and unchangeable APIs.

It is also needed for the correct dist path.

Test: m update-api
Bug: 155164730
Change-Id: I7dbafd7164d5da600ca45c39a7f93a5a40027a1f
2020-05-14 00:53:50 +01:00
Paul Duffin
f7a6433bab java_sdk_library: Propagate naming_scheme to snapshot
Bug: 155164730
Test: m nothing
Change-Id: Iec7c52d2102b6294540090f3c1e32bd9d8cc7da8
2020-05-13 20:40:53 +01:00
Paul Duffin
87a05a389d java_sdk_library: Add sdk_version for API scopes
Allow the sdk_version against which the stubs for a scope are compiled
to be specified in the module on a per scope basis.

Bug: 155164730
Test: m nothing
Change-Id: I5881e5ee7c2169c30f544882344a60a602dae917
2020-05-13 13:09:23 +01:00
Paul Duffin
0ff08bdb07 java_sdk_library: Allow separate api/stubs source modules
The API file for a scope represents the differences between the API
provided by that scope and that provided by the scope that it extends.
On the other hand the stubs source for a scope represents the union of
the API provided by the scope and the scope it extends (all the way
back to public).

Unfortunately, while metalava supports this behavior for scopes that
extend public (e.g. system and test) it does not support this behavior
for scopes that extend others, e.g. module_lib which extends system.
This is because it always assumes that the baseline for the API file
is 'public' and so has no way to defined other baselines.

This change works around that by having separate droidstubs modules to
generate the API and stubs sources for scopes that require different
arguments to generate the API and stubs sources.

Test: m checkapi
Bug: 155164730
Change-Id: Iea7d59852d7aeb503120acf3c44e08eb0d9d07b9
2020-05-13 08:52:53 +01:00
Paul Duffin
780c5f4715 java_sdk_library: Preserve sdk_version: "none" in snapshot
Test: m nothing
Bug: 156354511
Change-Id: Icf7c20dc109a773430768c1a2d91c674551dbc1b
2020-05-12 15:52:55 +01:00
Paul Duffin
4911a89181 java_sdk_library: Specify visibility of stubs modules
Adds two new properties to specify the visibility of the stubs modules
(source and library) created by the java_sdk_library. Excludes
visibility property from being inherited when creating the module so it
can be properly specified by copying across the relevant property.

Test: m checkapi
Bug: 155164730
Change-Id: Iffdd9f191ff0d74646356ac577560cc38efdd790
2020-05-08 18:03:31 +01:00
Paul Duffin
8f265b9ab9 java_sdk_library: Add support for module_lib API surface
Adds api scope for the module_lib and adds that to the list of all
all available scopes. The scope is generated if and only if the
api_surfaces property contains "module_lib".

No other changes are needed as the generation of the APIs is completely
driven by the allApiScopes array and the information in its contained
apiScope structures.

Test: m checkapi
Bug: 155164730
Change-Id: I7769af6823badca8715a270f86cf53b4e954b7df
2020-05-08 07:51:00 +01:00
Paul Duffin
3375e35d99 java_sdk_library: Control API surfaces generated
Adds a per scope set of properties that allows explicit control over
the API surfaces generated.

Previously, the term active was used to determine whether it was
generated but that was a little abstract and unclear so has been
replaced by generated.

Test: m nothing
Bug: 155164730
Change-Id: I7539d89618b61f6b9d1a4b60cc3f9614b157f0d9
2020-05-08 07:51:00 +01:00
Paul Duffin
260bd316bf Fix definition of android_stubs_current in sdk tests
This change fixes an issue with the definition of the
android_stubs_current (and similar) modules in the sdk tests. They were
incorrectly defined as java_sdk_library_import and not java_import.

Test: m nothing
Bug: 155164730
Change-Id: I4e9bfebdd2ff0a374e12284ccdf5ee5518968969
2020-04-29 18:59:56 +01:00
Paul Duffin
1fd005d5b3 Export API files as part of the java_sdk_library
Bug: 153443117
Test: m nothing

Change-Id: I9d6f5b91a7cc25019e2eb9e3c138f0874d2831de
2020-04-09 01:33:46 +01:00
Paul Duffin
3d1248ceb6 Export stub sources as part of the java_sdk_library
Minor refactoring of checkMergeZip(string) -> checkMergeZips(...string)
to allow testing of multiple merge zips.

Bug: 153443117
Test: m nothing
Change-Id: I8db00f611ced15f8476ba16f2834a72e8c913596
2020-04-09 01:32:38 +01:00
Paul Duffin
dd46f71493 Support java_sdk_library as member of sdk
Bug: 153443117
Test: m nothing
Change-Id: I9d8089b2555038e3f10ad5939a6a7b01839c67ea
2020-04-09 01:32:38 +01:00
Paul Duffin
6534770793 Stop requiring apex_available on java_library members of sdks
Previously, adding java_library to an sdk required that the names of
any APEXes that transitively compiled against it were added to its
apex_available property. This change removes that requirement.

Also corrects the dependency path in the TestApexAvailable_IndirectDep
error which previously passed through "shared from static" static
dependency tags even though those are explicitly NOT followed when
checking apex_available settings.

Bug: 152878661
Test: m droid
Change-Id: I995ed38956c1bc210b09494812de012fed9f9232
2020-04-08 08:40:44 +01:00
Paul Duffin
07ef3cb1fd Output properties before sets in snapshot module
This ensures a consistent output irrespective of whether property sets
are created before or after the properties are added. This provides a
little more flexibility in the creation code which allows that to be
simplfied.

Also switches from using reflection to a type switch.

Bug: 142918168
Test: m nothing
Change-Id: Ia025bfc751f1217d1658de6fb8e15091ea0ea9ff
2020-03-13 17:25:23 +00:00