Commit graph

484 commits

Author SHA1 Message Date
Treehugger Robot
3078f3bddc Merge "Add SOONG_SDK_SNAPSHOT_VERSION support" 2021-05-24 17:58:17 +00:00
Paul Duffin
8018e50ddb Tighten bootclasspath_fragment property validation
Previously, due to legacy reasons, the property validation did not
require a contents property and allowed the image_name to be either
"art" or "boot". Those reasons no longer apply and so this change
requires a contents property and only allows the image_name to be set
to "art" if specified.

Bug: 177892522
Test: m nothing
Change-Id: I8855d6e5365ef0b55490e90e7b6c0081cf070ee5
2021-05-24 16:22:47 +01:00
Paul Duffin
2fef136885 Generate hidden API flags for a bootclasspath_fragment
This change adds support for generating the hidden API flags for the
contents of a bootclasspath_fragment. Currently, it will only work for
the art-bootclasspath-fragment as it has no support for creating
dependencies between bootclasspath_fragment modules which will be
needed for handling any other bootclasspath_fragment.

The hidden API flag generation added by this change is completely
separate to the normal hidden API processing and is not as yet encoded
in dex jars so will have no effect on the runtime.

The generated files are provided for use by other modules and copied
into the sdk snapshot. That is needed to allow the build to verify that
the hidden API flags generated by the individual bootclasspath_fragment
modules are consistent with the flags generated for the whole
bootclasspath, whether building from source or prebuilts.

Bug: 179354495
Test: m art-module-sdk
      m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - test that the former file is a subset of the latter and that
        where they overlap they are identical.
Change-Id: Ie27303e2960953db1b7abe95510e3bca4411b09a
2021-05-14 01:48:51 +01:00
Paul Duffin
f4600f6e6a Disallow shared libraries in bootclasspath_fragment contents
Bug: 177892522
Test: m nothing
Change-Id: I78c8ef8664ec1eb0fe3456a2de2cb956162ca0da
2021-05-14 00:39:24 +01:00
Paul Duffin
a10bd3c127 Add java_sdk_library in bootclasspath_fragment contents to sdk
A java_library specified in a bootclasspath_fragment's contents
property will be automatically added to the sdk containing that
bootclasspath_fragment. Previously, if that was attempted with a
java_sdk_library it would be added to the sdk as if it was a normal
java_boot_libs which would prevent the sdk from containing the
API specific artifact such as current.txt files and stub libraries
and sources.

This change fixes that and adds a java_sdk_library as a java_sdk_libs
module.

Bug: 177892522
Test: m nothing
Change-Id: Ided57b846ce5b8940c7e898c786fd77602582ea2
2021-05-13 21:25:42 +01:00
Treehugger Robot
820397deb2 Merge "Add support for SOONG_SDK_SNAPSHOT_USE_SRCJAR" 2021-05-13 16:00:40 +00:00
Paul Duffin
475daafb1b Make hidden API bootclasspath_fragment test more realistic
Adds the bootclasspath_fragment to an apex and also adds an sdk library
to the stub_libs property.

Bug: 179354495
Test: m nothing
Change-Id: I2a5fffe915eb62923cc60732352d4d050dec4398
2021-05-13 12:40:08 +01:00
Paul Duffin
22628d50f3 Add support for SOONG_SDK_SNAPSHOT_USE_SRCJAR
Test: m SOONG_SDK_SNAPSHOT_USE_SRCJAR=true ipsec-module-sdk
      - check generated snapshot.zip file.
Change-Id: I02991e2a60d7784984b308cff2c47ee809d61f01
2021-05-12 23:13:22 +01:00
Paul Duffin
43f7bf0efd Add SOONG_SDK_SNAPSHOT_VERSION support
SOONG_SDK_SNAPSHOT_VERSION=current will generate unversioned and
versioned prebuilts and a versioned snapshot module. This is the
default behavior. The zip file containing the generated snapshot will
be <sdk name>-current.zip.

SOONG_SDK_SNAPSHOT_VERSION=unversioned will generate unversioned
prebuilts only and the zip file containing the generated snapshot will
be <sdk name>.zip.

SOONG_SDK_SNAPSHOT_VERSION=<number> will generate versioned prebuilts
and a versioned snapshot module only. The zip file containing the
generated snapshot will be <sdk name>-<number>.zip.

Bug: 157884619
Test: m nothing
      m SOONG_SDK_SNAPSHOT_VERSION=current art-module-sdk
      - check that the generated Android.bp file has not changed
        from the default.
      m SOONG_SDK_SNAPSHOT_VERSION=none art-module-sdk
      - check that the generated Android.bp file does not contain
        versioned modules.
      m SOONG_SDK_SNAPSHOT_VERSION=2 art-module-sdk
      - check that the generated Android.bp file only contains
        version 2 of each module.
Change-Id: I087e9d7d3ad110508a3d6a39bca50cbb46b3ce82
2021-05-12 22:14:32 +01:00
Paul Duffin
6213170b3e Add package module to sdk snapshot that contains licenses
This ensures that the LSC license tool does not attempt to add a
package module to an sdk snapshot Android.bp.

Bug: 181569894
Test: m nothing
Change-Id: I37b66e0df56c5b9ec255f66cb4f2a066ea96a738
2021-05-11 08:38:30 +01:00
Paul Duffin
b0bb376efa Add license modules to the sdk
Adds initial support for adding license modules to the sdk, along with
any referenced license text files. There is a number of minor
improvements to be made but the core of the support is there and it
works for ART sdks.

Basically, this change will automatically add license modules
referenced from any sdk member module as an internal sdk member. An
internal module has an sdk snapshot specific name that will not
conflict with the source module and which cannot be referenced from
outside the sdk snapshot.

Bug: 181569894
Test: m art-module-sdk art-module-host-exports art-module-test-exports
      - diff output before and after this change
        made sure that every prebuilt had a licenses field
        and that the license modules were added
Change-Id: I0c5ccabf58f4ef487e42ef8e61a5b2a74c0e81af
2021-05-11 08:28:49 +01:00
Paul Duffin
e96108d797 Add baseline test for license with sdk
This change adds a test that includes license modules that are used by
modules which are part of an sdk but which does not yet copy the
license module into the snapshot. It includes the refactoring changes
needed to allow license modules to be used in an sdk test and provides
a baseline against which future changes can be compared.

Bug: 181569894
Test: m nothing
Change-Id: I60722f43cc9cc8375d97f46eb4c281e6c38987cd
2021-05-11 08:24:59 +01:00
Paul Duffin
0df49686b3 Add support for name-less modules and property comments
Bug: 181569894
Test: m nothing
Change-Id: Ia4da1d2a55a924db82ae999da455adedbaca47c0
2021-05-11 01:00:12 +01:00
Paul Duffin
1308205638 Allow an sdk member type not to provide a bp property
Bug: 181569894
Test: m nothing
Change-Id: I7e98f14cb377683457fba32fd05d6c614f78ffeb
2021-05-11 01:00:06 +01:00
Paul Duffin
0d4ed0ac56 Allow sdk snapshot to contain source module type
Bug: 181569894
Test: m nothing
Change-Id: I30513dd22e2719099a1a1ef63cacc98336d02e9f
2021-05-11 00:59:56 +01:00
Paul Duffin
83ad956ec4 Move setting of prefer to createMemberSnapshot
Test: m nothing
Change-Id: Iab1c048fe74a5e474f68ce4acc6c1b41381368ca
2021-05-11 00:05:53 +01:00
Paul Duffin
57f8359fe3 Derive deapexer module properties from transitive dependencies
Previously, the deapexer module properties were simply passed through
from the prebuilt_apex/apex_set modules. This change derives the
properties from the transitive dependencies of the modules so that it
can automatically include the contents of a bootclasspath_fragment
module without having to duplicate them in the exported_java_libs
property.

Part of this change involves moving the addition of dependencies onto
the prebuilt apex's contents from DepsMutator to ComponentDepsMutator
so that they can be visited in the createDeapexerModule() function. The
ComponentDepsMutator runs before prebuilts without matching sources are
renamed to match the source name which simplifies the process of adding
dependencies directly onto the prebuilts.

The deapexerDeps method was renamed as the dependencies are added both
for use by deapexer and also to create APEX variants for access by
platform_bootclasspath so the name was confusing.

Bug: 187266082
Test: m nothing
      m SOONG_CONFIG_art_module_source_build=false nothing
      - this fails but not because of missing APEX variants
Change-Id: Icb4f883e7d2b63739e5ff0dc6edb8a906d80189b
2021-05-10 11:51:20 +01:00
Paul Duffin
64fb526cd0 Add SOONG_SDK_SNAPSHOT_PREFER support
By default the generated snapshot has prefer: false. Building it with
SOONG_SDK_SNAPSHOT_PREFER=true will force it to generate prefer: true.

Bug: 157884619
Test: m nothing
      m SOONG_SDK_SNAPSHOT_PREFER=true art-module-sdk
      - check the generated Android.bp file to make sure it contains
        prefer: true
Change-Id: Ied297b32d9bd4822a140fa99016f38e234c50f64
2021-05-07 14:54:35 +01:00
Paul Duffin
2d3da31d41 Make all SdkMemberTypes support transitive member deps
Previously, only those SdkMemberTypes which had specific need to
automatically add some of their dependencies as sdk members would cause
the sdk to visit their transitive dependencies. However, as any module
can have dependencies on license modules and license modules need to be
included in the sdk then it needs to visit transitive dependencies of
all members.

So, this change removes the support for allowing an SdkMemberType to
control whether its transitive dependencies are visited and just visits
them all.

This does not have any effect on sdk snapshots as in order for a
dependency to be added to an sdk it needs to be added with a tag that
implements SdkMemberTypeDependencyTag and the only tags that implement
this are used by SdkMemberTypes that had enabled transitive members.

Bug: 181569894
Test: m art-module-sdk art-module-host-exports art-module-test-exports
      - verify that this change has no effect on the generated snapshots
Change-Id: If0293af0237aa7e39335e5b8383a41c023ff5853
2021-05-06 23:13:06 +01:00
Paul Duffin
b9e7a3ca7a Make licenseModule SdkAware
Making licenseModule SdkAware caused two breakages in the build. The
breakages were both caused by having an SdkAware module that was
depended upon by a versioned sdk snapshot but which was not itself
versioned and so did not have the member_name property set.

That occured because some default licenses have been added to the
packages containing prebuilts, e.g. prebuilts_runtime_license in
prebuilts/runtime/Android.bp. They apply to both the versioned and
unversioned members.

Once license support has been added to the sdk most of those will be
removed and replaced with properly versioned license modules. However,
in the meantime it is necessary to support that.

This change avoids the issue by checking to see whether the module is
itself versioned before relying on the member_name property. It also
improves the error message when a panic is recovered to make it easier
to identify where it originates.

Bug: 181569894
Test: m nothing
Change-Id: I0e7da2e0c4a30a6f814c2faab821b185aaed2135
2021-05-06 23:13:06 +01:00
Paul Duffin
525a590565 Make sdk tests more realistic
The tests use <sdk>_<module>_<version> as the format for a versioned
sdk member name but the format should be <sdk>_<module>@<version>. This
change corrects it and also fixes a similar issue in an error message.

Bug: 181569894
Test: m nothing
Change-Id: I8be0db4bcd0b6f4d6fbdf9e402ef7257fae8e18b
2021-05-06 23:13:06 +01:00
Mathew Inwood
7e554ec2de Merge "Ensure current.zip is put in the right place." 2021-05-06 15:21:04 +00:00
Mathew Inwood
60770e2250 Ensure current.zip is put in the right place.
Using the InstallFile as the snapshot ensure that the artifact
in out/soong/mainline-sdks/ is built when building the sdk
target.

Test: m ipsec-module-sdk
Change-Id: I45ce6001dbae3a7a9c4cf50f8d7d5d67f94dbcb3
2021-05-06 13:16:27 +00:00
Paul Duffin
62035b5991 Add baseline test for sdk snapshot env variables
Bug: 157884619
Test: m nothing
Change-Id: I56c14ddfec1c33b60fe70f56e4d59b90639657fe
2021-05-06 10:50:00 +01:00
Paul Duffin
f23bc472b0 Move configuration checks from getBootImageJar
The getBootImageJar function will be removed once the boot image
creation has been moved to the platform_bootclasspath and
bootclasspath_fragment module types. However, the consistency checks
that it performs are still useful so this change moves them out
first.

The ART boot image related checks are now performed in the
bootclasspath_fragment module type. A previous change accidentally
disabled the checks when the contents property was not empty which has
been fixed. Also, the error messages have been tweaked to make it clear
that the art-bootclasspath-fragment is now the source of truth as to
its contents not the configuration.

The framework boot image related checks are now performed in the
platform_bootclasspath module type.

Initially, this change included an extra check to make sure that
UpdatableBootJars comes from updatable APEXes but that broke because
framework-wifi and framework-tethering are not currently marked as
updatable in AOSP.

Bug: 177892522
Test: m nothing
Change-Id: I80fb600fa2c7cec4566b3461c6a33c4c6f0743f4
2021-04-30 12:06:28 +01:00
Paul Duffin
895c7140a9 Add stub_libs properties to bootclasspath_fragment snapshot
Bug: 177892522
Test: m nothing
Change-Id: I7644122c4abed8e8bc2cc7b58f408bee3bdc91f6
2021-04-28 22:02:00 +01:00
Paul Duffin
2ac45f019b Remove special handling of frameworks/base/config/boot-profile.txt
Previously, the method was run in repo manifests that did not include
the frameworks/base repository so it had to handle the file being
missing. However, now that this is being called from
platform_bootclasspath which is only defined in frameworks/base that
is no longer required.

Bug: 177892522
Test: m droid
Change-Id: I77fa5a204d1426a6be24a6f0b48e18f48f3dd908
2021-04-28 09:03:46 +01:00
Paul Duffin
eee466e439 Pass module to SdkMemberTypeDependencyTag.SdkMemberType
This change allows an SdkMemberTypeDependencyTag to select the
SdkMemberType to use to add a dependency based on the module.

Bug: 177892522
Test: m nothing
Change-Id: I2d6e51b615636dc7cd41da65808ef851dd8ae6ac
2021-04-27 23:23:20 +01:00
Paul Duffin
2dc665bb48 bootclasspath_fragment: Always output contents property in snapshot
Bug: 177892522
Test: m art-module-sdk
      - check generated snapshot contains contents property
Change-Id: I122dedba6600a199bfd83b01988da36ddfd09d63
2021-04-26 21:21:58 +01:00
Paul Duffin
e95b53a55d Automatically add bootclasspath_fragment contents to sdk
Previously, both a bootclasspath_fragment and its contents had to be
explicitly added to the sdk. This change means that adding a
bootclasspath_fragment to and sdk will automatically add its contents
as if they were added using java_boot_libs.

Bug: 177892522
Test: m nothing
Change-Id: I8f7e70649f272c9a109d4606571a2d12c44b7904
2021-04-26 21:21:58 +01:00
Paul Duffin
cd06467124 Generate snapshot member list properties from dependencies
Previously, the snapshot member list properties were simply copied from
the sdk variants. That worked because the snapshot member list
properties must include all exported (i.e. visible outside the sdk
snaphot) members and only members explicitly specified in the sdk
member list properties were exported. However, a preceding change
allowed members that were transitively added (and so not explicitly
specified on the sdk) to be exported.

This change ensures that those exported members are added to the
snapshot member properties which means that they will now work properly
with the sdk version mutators.

This does not add tests for this because no existing SdkMemberType
exports its transitive dependencies and there is currently no way to
register an SdkMemberType for testing only. A following change will
exercise this functionality.

Bug: 186290299
Test: m nothing
Change-Id: Iadbdd86f6dcdd183fa99f647222a43d412271501
2021-04-24 22:33:04 +01:00
Paul Duffin
a720811c38 Allow exporting of sdk members to be done per tag
Previously, every module added to an sdk directly through one of the
SdkMemberType specific properties, e.g. java_libs, was exported and
every module added automatically via a transitive dependencies was not
exported. This change allows that behavior to be customized per tag.

Bug: 186290299
Test: m art-module-sdk
      - verify that this change does not affect its contents.
Change-Id: I563b5bcd823e61c23cdb706cfcbb13337963d550
2021-04-24 22:30:26 +01:00
Paul Duffin
2d1bb89fe3 Extract the os specific snapshot module properties handling
Previously, the code was inlined in the addSnapshotModule() method
which made it difficult to follow, did not have clear separation
between the different pieces and duplicated functionality. This
change combines the dynamic and static properties together and then
separates the processing into collation, optimization and property
set updating methods.

Bug: 186290299
Test: m nothing
Change-Id: I311eebca2337cb67634e63d71da12305f4370c07
2021-04-24 22:30:18 +01:00
Paul Duffin
fbe470e7d3 Dedup snapshot module property set initialization
Previously, the code to add the compile_multilib property was
duplicated. This change moves the logic into the existing
addMemberPropertiesToPropertySet method, renames it to reflect new
behavior and renames and promotes the variantProperties struct from the
addSnapshotModule() method to the package level scope.

Bug: 186290299
Test: m nothing
Change-Id: I832a7fd89a3cf95467a1ff0807902c9be34c50b9
2021-04-24 16:50:47 +01:00
Paul Duffin
7b0259ff62 Make order of properties in snapshot module consistent
Previously the compile_multilib property was added after the member
list properties at the top level but before them in the target.os
level. This change makes it always be added before them.

Bug: 186290299
Test: m nothing
Change-Id: I01fe272933fd11f7f730cc379f60fb2ce70f2114
2021-04-24 16:50:47 +01:00
Paul Duffin
cc3132e496 Clarify behavior around handling of sdk members
Renames some types and adds some documentation to try and clarify how
the sdk handles the members.

Bug: 186290299
Test: m nothing
Change-Id: I4038276f5f4c473a96fe06aedc43050c90416877
2021-04-24 16:49:26 +01:00
Paul Duffin
2182726990 Rename sdkMemberRef to sdkMemberVariantDep
Previously, there were two types sdkMemberRef and sdkMember that seemed
as though the former was referring to the latter but that was not the
case. The former was actually information about the dependency the sdk
had on a specific member variant. The latter is all the information
about a specific member, including all its variants.

This change attempts to clarify that relationship, both through a
better type name and through some improved documentation.

Bug: 186290299
Test: m nothing
Change-Id: I1dcc0ddf3333da9e797bb23ae945d04729839014
2021-04-24 16:48:16 +01:00
Paul Duffin
26197a65da Extract addSnapshotModule from buildSnapshot
Bug: 186290299
Test: m nothing
Change-Id: I297d63366d71f692572e8e40f1e510b6e41859e6
2021-04-24 16:48:16 +01:00
Paul Duffin
023dba0a3f Add exported_bootclasspath_fragments to prebuilt_apex/apex_set
This is needed to allow a prebuilt_bootclasspath_fragment to be used
interchangeably with a bootclasspath_fragment in the
platform_bootclasspath module.

The platform_bootclasspath module depends on APEX specific variants of
bootclasspath_fragment modules. That works because the
bootclasspath_fragment modules are part of an apex and so have an APEX
specific variant which the platform_bootclasspath can specify.

Using a prebuilt_bootclasspath_fragment in place of a
bootclasspath_fragment requires that the prebuilt also has an APEX
specific variant.

Specifying exported_bootclasspath_fragments on a prebuilt_apex/apex_set
will cause it to create an APEX variant for the named module whcih will
allow it to be selected by the platform_bootclasspath module.

Bug: 186034565
Bug: 177892522
Test: m nothing
Change-Id: I7ddacc6498ec3a4a9f26c5f78b7f9a033e494d78
2021-04-23 12:12:37 +01:00
Paul Duffin
7c95555d79 bootclasspath_fragment: Add hidden API flag files to snapshot
These are needed at the moment to ensure the hidden API processing
generates the same set of flags whether it is being generated from
source or prebuilts. Soon these will be needed to ensure that the
hidden API flags generated for the individual modules are compatible
with previous releases.

Bug: 179354495
Test: m art-module-sdk and check snapshot zip contains the files
      and the generated Android.bp references them.
Change-Id: I9a3334cc48faa381bbbcbbb59479db719042796a
2021-04-21 23:54:22 +01:00
Paul Duffin
a57835e8e5 bootclasspath_fragment: Add contents to snapshot
Bug: 177892522
Test: m nothing
Change-Id: I54fe0537b758a0e3dacd34b139ef3eb21b8841fd
2021-04-21 23:54:22 +01:00
Paul Duffin
0b28a8d356 Replace boot image with bootclasspath fragment in sdk package
Bug: 177892522
Test: m nothing
Change-Id: I67589bccbde426de2caf513bd643d9484df86c1e
2021-04-21 23:54:17 +01:00
Treehugger Robot
1cd2357dd3 Merge "Do not output two trailing newlines in generated bp files." 2021-04-20 23:41:00 +00:00
Martin Stjernholm
ee9b24e6d2 Do not output two trailing newlines in generated bp files.
Test: m platform-mainline-sdk
  Check Android.bp in the generated platform-mainline-sdk-current.zip.
Bug: 156286550
Change-Id: Ifb99288f3b85c3b986270d48e2dc7d9e15fa7536
2021-04-20 15:54:21 +01:00
Paul Duffin
1267d875b6 java_sdk_library: Make dex stub jars available for hiddenapi
The hidden API processing needs access to dex jars for the API stubs in
order to determine which dex members are part of an API surface. The
dex stubs used for the monolithic file are provided by normal
java_library modules for legacy reasons. However, the APEXes that
contribute to the bootclasspath, and so need to perform hidden API
processing, typically provide stubs created by a java_sdk_library.

This change adds support to java_sdk_library/_import to make the dex
stub jars available when requested, that involves:
1. Adding compile_dex property to java_sdk_library_import and
   propagating it down the the java_import modules for the stubs. That
   is already handled for java_sdk_library.
2. Propagating the java_sdk_library compile_dex property to the
   java_sdk_library_import in the generated snapshot.
3. Refactoring and wiring to make the dex stubs jar available to other
   parts of Soong.

Bug: 179354495
Test: m nothing
Change-Id: I5895d4f2ba0b684870862b9429b2364865e4afc6
2021-04-16 18:48:20 +01:00
Paul Duffin
e1381887ae Migrate sdk tests away from checkAndroidBpContents
Replaces a single call to checkAndroidBpContents(...) with separate
calls to check the versioned and unversioned Android.bp files.

Test: m nothing
Bug: 179354495
Change-Id: I270bf73909958d97b2f298e8d7f6d10a1f75ae71
2021-04-16 18:48:20 +01:00
Jingwen Chen
2f6a21e957 Export OsTypeList through a function.
This is a cleanup CL to make the OsTypeList accessor consistent with
ArchTypeList by creating a function that creates a copy of the
underlying slice.

Test: TH
Change-Id: I346c5c3937bbdb8a373e4d7135b896a5274cc1f1
2021-04-05 07:35:06 +00:00
Treehugger Robot
25c47a43a5 Merge ""current" is implicitly added to stubs.versions" 2021-04-01 04:53:55 +00:00
Jiyong Park
d4a3a137ed "current" is implicitly added to stubs.versions
So far, when a library `libfoo` has `stubs.versions: ["10", "11"]`, then
`shared_libs: ["libfoo"]` is linked to the version 11 of the stub.

This requires the author of `libfoo` to manually update the property
whenever a new version is introduced. Otherwise, clients are not able
to use the newly added APIs because the latest stub is for an old
version.

This change eliminates the need for manual updating. "current" version
is always implicitly added to `stubs.versions`. It is added even when
nothing is set on the property, if `stubs.symbol_file` is set. i.e.

```
cc_library {
    name: "libfoo",
    stubs: {
        symbol_file: "libfoo.map.txt",
	// no versions: [...] needed
    },
}

cc_library {
    name: "a_client",
    shared_libs: ["libfoo"],
    apex_available: ["myapex"],
    min_sdk_version: "29",
}

apex {
    name: "myapex",
    native_shared_libraries: ["a_client"],
    min_sdk_version: "29",
}
```

`a_client` links to the "current" stub of `libfoo` that has all symbols
shown in the map file.

Note that, above doesn't mean that the client has unlimited access to
APIs that are introduced even after the min_sdk_version of the client
(29 in this example). The use of such APIs still has to be guarded with
`__builtin_available` check.

Bug: N/A
Test: m
Change-Id: I70bb1600c18e74d36c6b24c3569d2149f02aaf96
2021-04-01 09:58:53 +09:00
Paul Duffin
55e740e9a6 Remove varargs from RunTest(t *testing.T)
Use GroupFixturePreparers instead.

Bug: 182885307
Test: m nothing
Change-Id: Iaedb0ddc9d6a704f4d41363e705f3025a1291dc8
2021-03-31 16:03:59 +01:00
Colin Cross
045ed0f684 Merge changes I71a83e3a,I66101c0c,Ie387c8c4,Iea742e75
* changes:
  Strengthen metalava sandbox support using sbox
  Move metalava's output files into a subdirectory
  Fix lint warnings in droidstubs.go
  Split droidstubs out of droiddoc.go
2021-03-26 15:14:53 +00:00
Colin Cross
cb77f75aae Move metalava's output files into a subdirectory
Move all of the output files of the metalava rule into a single
subdirectory to make it compatible with sandboxing in sbox.

Test: TestDroidstubs
Change-Id: I66101c0c224dee702c8175e61c12cc9cd1aa8b93
2021-03-25 11:11:36 -07:00
Paul Duffin
bb9ff5108b Filter duplicate modules in platform_compat_config_singleton
Previously, unpacking a snapshot containing a
prebuilt_platform_compat_config into a source build would cause build
failure because of duplicate ids because the singleton would collate
ids from both prebuilts (versioned and unversioned) and source.

This change filters out versioned prebuilts and only uses prebuilts
that are preferred and source modules that have not been replaced by a
prebuilt.

Bug: 182402754
Test: m nothing
Change-Id: Idacbb34444e5156370df70bf88c6e8a7e2d67890
2021-03-25 12:53:25 +00:00
Paul Duffin
db462dd987 Disallow non-existent paths in sdk package
Test behavior was changed a while ago so that tests by default ignore
non-existent source paths (unless they explicitly check for/rely on
them). Prior to that CheckSnapshot() could detect when files were
missing from the snapshot but it no longer can.

This change disallows non-existent source files in all the sdk tests
which means that they are disallowed when processing the snapshots as
they use the same preparers as were used to process the sources.

This caused a test failure which has been temporarily ignored and has
a TODO and bug associated with it.

Bug: 183184375
Test: m nothing
Change-Id: I969d8515d20ef5ae515f2b5f93d8ed4e4f8ede75
2021-03-25 12:53:22 +00:00
Paul Duffin
1822a0a371 Improve sdk snapshot testing
The sdk produces snapshots that are expected to be unpacked in one of
the Android repos. Often that can lead to issues due to conflicts
between the source and prebuilts. This change attempts to avoid those
conflicts by testing the different ways that those files can be used.

With the existing test to cover adding the snapshot
This change will cause the sdk tests to check the following:
1) Snapshot on its own (already done).
2) Snapshot plus original source where the original source is
   preferred.
3) Snapshot plus original source where the snapshot is preferred.

It also adds the ability for tests to provide their own custom checkers
to verify the result of each of the previous tests.

This change reveals a number of bugs already present. Rather than
attempt to fix them this change adds the ability to specify error
handlers for the two cases that mix source and snapshot to allow those
errors to be temporarily ignored while allowing the majority of the
tests to benefit from this improvement. Each of those failures has a
TODO and bug associated with it.

Bug: 183184375
Test: m nothing
Change-Id: I105233195074dbe7a6422b6dfc5486e74398ea15
2021-03-25 12:50:18 +00:00
Paul Duffin
76e5c8a37f Convert test that disallows non existent paths to use fixtures
This change needed to add some additional files to the registered
files for PrepareForTestWithJavaDefaultModules because otherwise they
would fail when "TestAllowNonExistentPaths = false". Those files were
being added by the TestJavaLintRequiresCustomLintFileToExist (albeit in
some cases in different locations to that required by the default
modules but as the files are needed by the modules defined in
PrepareForTestWithJavaDefaultModules they should be defined in it.

A couple of other places also provided some files so moving them into
PrepareForTestWithJavaDefaultModules caused some conflicts which needed
to be resolved.

Bug: 183184375
Test: m nothing
Change-Id: I76ce9f1673c1c1c4000635b76b8377d582224bf1
2021-03-24 22:08:05 +00:00
Paul Duffin
ac94726ef2 Remove testSdkWithJava
Bug: 181070625
Test: m nothing
Change-Id: I9ef906a386cc87f69b166ec88e0b6c7388c3d06a
2021-03-24 15:47:28 +00:00
Paul Duffin
9ec86b14e7 Convert java_sdk_test.go tests to fixtures
Bug: 182638834
Test: m nothing
Change-Id: I6a1fabdd2c8385e5fbaef6985047f9d0bdceb209
2021-03-24 15:47:28 +00:00
Paul Duffin
001b2342f7 Add platform_compat_config to sdk
Bug: 182402754
Test: m nothing
Change-Id: Ife3f4f64fc116d62eb7c3cc10c50e00f19d1d81c
2021-03-22 19:13:40 +00:00
Paul Duffin
04b4a19fe6 Add TestBasicSdkWithBootImage
Added to reproduce the conditions that lead to the error reported in
the bug so they can be fixed. There were a number of issues that were
fixed in previous changes and this test verifies that they have been
fixed.

Bug: 182992071
Test: m nothing
Change-Id: I2197899b284a99973e698db314b15812f602b141
2021-03-22 19:13:40 +00:00
Paul Duffin
4a1d451405 Convert boot_image_sdk_test.go to fixtures
Bug: 182638834
Test: m nothing
Change-Id: I8c9cb399b15d04ec8465a3b3bb4b43561aef46fd
2021-03-22 19:13:22 +00:00
Paul Duffin
cf3ee2f87e Allow sdk package tests to optimize their test setup
Extracts the setup for apex from prepareForSdkTest to allow it to be
reused without using all of it. That will allow tests to optimize their
test setup.

Bug: 182638834
Test: m nothing
Change-Id: I2056103b15c2737a616ee29ff890c6af0722e6d2
2021-03-22 18:31:53 +00:00
Paul Duffin
c93c98e315 Use test fixtures in CheckSnapshot()
Using the preparer(s) that were used to run the test to verify the
integrity of the generated snapshot ensures that it will be verified in
the same environment as the snapshot was generated. This ensures that
as sdk tests are migrated to use fixtures that are optimized for each
test that they will benefit from those optimizations when checking the
snapshot.

Bug: 183184375
Test: m nothing
Change-Id: I62b383f9a1d9a77d1cabb101d9d1e4a976170fe3
2021-03-22 18:31:53 +00:00
Paul Duffin
89648f98fa Remove usages of FixtureFactory from misc packages
These packages have already been migrated to use per test build
directory so have no need for a FixtureFactory.

Bug: 183235980
Test: m nothing
Change-Id: I667d1d992caaf0f615de91f89efdae11c44986c2
2021-03-22 18:31:53 +00:00
Paul Duffin
6bef6fee3c Merge "Ensure that DepIsInSameApex is not called for ExcludeFromApexContentsTag" 2021-03-22 18:30:10 +00:00
Paul Duffin
c737facd2b Merge "Replace AssertPanic with AssertPanicMessageContains" 2021-03-22 08:58:28 +00:00
Paul Duffin
4c3e8e2d67 Ensure that DepIsInSameApex is not called for ExcludeFromApexContentsTag
The ExcludeFromApexContentsTag marker interface was added to avoid
every implementation of DepIsInSameApex() from having to deal with the
special tags, like PrebuiltDepTag. Unfortunately, when adding that
not all calls to DepIsInSameApex() were protected which meant that the
BootImageModule, which panics if it doesn't recognize a tag, was
causing failures. This change documents the need and improves the
consistency.

A follow up change will add a test for this.

Bug: 182992071
Test: m nothing
Change-Id: If0bf9a7447ebf7a0bb0c88e91951a7220d4af45c
2021-03-22 08:43:55 +00:00
Paul Duffin
d14cbc1e69 Add prepareForSdkTestWithJava
Bug: 182638834
Test: m nothing
Change-Id: I86cb08a640c7e39c1eaaf54926699f32a32ccc9b
2021-03-21 11:01:13 +00:00
Paul Duffin
9f4b3bbb7c Replace AssertPanic with AssertPanicMessageContains
Bug: 182885307
Test: m nothing
Change-Id: Idffa314285c90080796cc3df391de9c314eaa422
2021-03-20 12:08:51 +00:00
Paul Duffin
ebcb37bab1 Merge "Migrate sdk package to a per test build directory" 2021-03-18 17:05:11 +00:00
Paul Duffin
573989d821 Prevent ApexInfoMutator from creating unnecessary variants
Adds the AlwaysRequireApexVariantTag interface to enable
ApexInfoMutator to differentiate between a tag that is excluded from
apex contents but still requires an apex variant and a tag that is
excluded from apex contents and does not require an apex variant.

That is needed to support the sdkMemberVersionedDepTag which excludes
the target from being added to the APEX but requires an APEX variant.
A more detailed explanation is in the comments.

The AlwaysRequireApexVariant() method follows the pattern used in
ReplaceSourceWithPrebuilt of having a method that returns a bool to
trigger the behavior and not say ExcludeFromApexContentsTag that simply
relies on the tag implementing an interface to trigger. That is because
the former is more flexible and allows a tag type to parameterize the
behavior if necessary.

The tags that this will exclude from creating an apex variant are:
* PrebuiltDepTag - by the time the apex variant has been created any
  preferred prebuilts will have replaced the sources so there is no
  need to create an APEX variant if the only dependency path from the
  APEX to the prebuilt is via this tag.
* hiddenApiAnnotationsDependencyTag - the target of which is a purely
  build time artifect and MUST NEVER end up in the APEX.

It will also stop calling DepIsInSameApex for any dependency created
by the sdkMemberVersionedDepTag. Which will fix the issue reported in
the bug.

Bug: 182992071
Test: m nothing
Change-Id: I9569e488d6446ca45d3ea8f32a9b74524eb865df
2021-03-18 09:05:28 +00:00
Paul Duffin
abbf63d650 Migrate sdk package to a per test build directory
Bug: 182885307
Test: m nothing
Change-Id: I051f2572ce5e94e2a3a66cf7663797178b7d1506
2021-03-18 01:48:25 +00:00
Paul Duffin
36474d322b Stop sdk package depending on testing.T being embedded in TestResult
This change is in preparation for removing testing.T from TestResult.

Bug: 181070625
Test: m nothing
Change-Id: I67535aff0d894e6e3d8456b75540f340af853355
2021-03-13 15:59:33 +00:00
Paul Duffin
ebddef39dd Fix build failure in boot_image_sdk_test.go
Test: m nothing
Change-Id: I1f386fe4119ac9c3efc8d046142d02862af5fd99
2021-03-12 08:20:27 +00:00
Paul Duffin
32a8aadfaa Merge "Add prebuilt_boot_image and add boot_images to sdk" 2021-03-12 08:06:45 +00:00
Paul Duffin
c10ee77ea6 Merge changes Ifc96992e,Ic76523ba
* changes:
  Support test fixtures in sdk package
  Add apexFixtureFactory to apex package
2021-03-12 08:01:31 +00:00
Paul Duffin
fe9a9e3f7d Add missing // to clarify comment
Bug: 181070625
Test: m nothing
Change-Id: Ia22a1be7a05b3a9efd34f2bf70216eb046396be0
2021-03-11 17:41:01 +00:00
Paul Duffin
4a2a29ce6a Support test fixtures in sdk package
Bug: 181070625
Test: m nothing
Change-Id: Ifc96992e54c1b1d89a82b88ab27e555ae267a51e
2021-03-11 17:25:29 +00:00
Paul Duffin
981b94b460 Switch CheckSnapshot from a testSdkResult method to a function
This will allow the testSdkResult to be replaced with the TestResult
when switching sdk package to use test fixtures.

Bug: 181070625
Test: m nothing
Change-Id: Ieca63f4c189f5e804102aeb4073289ea03143b6e
2021-03-11 13:39:24 +00:00
Paul Duffin
8306f25679 Make testSdkResult compatible with android.TestResult
This change makes it easier to switch the sdk package over to using the
new fixture mechanism by removing inconsistencies between the
testSdkResult and TestResult structures.

Bug: 181070625
Test: m nothing
Change-Id: Ic4c06e08ea5060fd09123f2ca65580e18b4d2ef6
2021-03-11 13:38:56 +00:00
Paul Duffin
f7f65dafb4 Add prebuilt_boot_image and add boot_images to sdk
Bug: 177892522
Test: m nothing
Change-Id: I640359acd6840507f32d7034c97d4d1c7ff591e9
2021-03-11 07:24:26 +00:00
Paul Duffin
3b2636fdf3 Merge "Remove duplicate sdk.TestHelper" 2021-03-10 19:59:56 +00:00
Paul Duffin
a3cb2b396f Remove duplicate sdk.TestHelper
As part of the work on the new fixture mechanism some of the TestHelper
functionality was moved into the android/fixture.go package. This moves
the rest and removes the now duplicated TestHelper from the sdk
package.

Also removed some unnecessary & operators.

Bug: 181070625
Test: m nothing
Change-Id: Ia09a5d05e4fab3a4e28cf44b2d947a33541e3925
2021-03-10 14:07:20 +00:00
Paul Duffin
8edc99c803 Simplify TestSdkInstall
Test: m nothing
Change-Id: Iff59c8c331402b8d53323072213579ed9956cd90
2021-03-09 23:03:48 +00:00
Paul Duffin
6d9108f047 Extract sdk registration code into function for reuse
Test: m nothing
Change-Id: I0d0e36324808831deb9a32f07ca3696125703873
2021-03-09 23:03:45 +00:00
Paul Duffin
d6ceb8600c Clean up cc.RegisterRequiredBuildComponentsForTest()
Changes this function so it only registers components from the cc
package by pushing the call to genrule.RegisterGenruleBuildComponents()
down into those packages whose tests need it.

This will make it easier to migrate cc package tests to test fixtures
as the RegisterRequiredBuildComponentsForTest() no longer overlaps with
preparers from the genrule packages.

Bug: 181070625
Test: m nothing
Change-Id: Ic00c7e480dc738d7a88d038aca6ab95a1502a24a
2021-03-05 18:20:33 +00:00
Mathew Inwood
f8dcf5ead2 Make apex.updatable default to true.
Update tests accordingly and add a new test case for this.

Bug: 180375550
Test: Treehugger
Change-Id: I835e189f4dae1e4bc79dce7bc59b7b9c7bd19fd9
2021-03-03 10:28:26 +00:00
Paul Duffin
7a7d067c74 Generated headers may not be arch specific
Previously, it was assumed that generated headers must be arch specific
and so prevented the fields referencing the paths to those headers from
being automatically optimized by the sdk generation code. That is not
always the case, e.g. with headers generated from protos so this change
allows those fields to be optimized.

Bug: 180427921
Test: m nothing
Change-Id: Id2af419d58ae3c30ea6d9e87f71e33a9ff6ba13b
2021-02-22 18:23:22 +00:00
Paul Duffin
42dd4e6cd6 Fix the snapshot handling of generated headers
Previously, the snapshot handling code did not preserve the directory
structure of generated include directories and instead just copied the
headers into the same module specific directory and added that single
directory to the export_include_dirs (or similar) property.

That had a couple of issues:
* The include directory was repeated in the ..._include_dirs property.
* It did not work when the include directories overlapped.

In the latter case it had a couple of issues:
* Code which compiled fine against the source would not compile against
  the prebuilt.
* Header files were duplicated in the output.

e.g. assume the following generated header file structure:
  foo/
      foo.h
  bar/
      bar.h
      baz/
          baz.h

When the sdk snapshot was passed include directories of "foo", "bar" and
headers of "foo/foo.h", "bar/bar.h", "bar/baz/baz.h" it would generate a
snapshot with the structure:
  include_gen/
      foo.h
      bar.h
      baz/
         baz.h

And:
  export_include_dirs: ["include_gen", "include_gen"]

However, when the include directories overlapped and include directories
of "foo", "bar" and "bar/baz" were passed in the directory structure
would be the same and the export_include_dirs would contain 3 usages of
"include_gen".

That meant that source code which used the following would build
against the source (because it would find "baz.h" in the "bar/baz"
include directory) but would fail when built against the prebuilts
because the "include_gen" directory did not contain "baz.h":
    #include "baz.h"

This change preserves the input directory structure for generated files
in a similar way to how it does it for source files. So, the snapshot
structure looks something like this:

  include_gen/
      foo/
          foo.h
      bar/
          bar.h
          baz/
              baz.h

And:
  export_include_dirs: [
    "include_gen/foo",
    "include_gen/bar",
    "include_gen/bar/baz",
  ],

Bug: 180427921
Test: m nothing
Change-Id: Id69eef8cf5eecd033841d3b7cd0c044a697ce404
2021-02-22 18:23:22 +00:00
Paul Duffin
86b02a7962 Differentiate usages of word "include" in cc_sdk_test.go
A number of tests in cc_sdk_test.go use "include" for the input include
directory which results in the output containing "include/include"
(because the snapshot code uses "include" as the directory into which
native headers will be output). This change switches the inputs from
"include.." to "myinclude.." to differentiate between the two sources.

Bug: 180427921
Test: m nothing

Change-Id: Iba1e6d94647c74b31307b18254c03580e64c91cf
2021-02-22 18:23:22 +00:00
Paul Duffin
a43f927cd6 Add sdk test for incorrect handling of generated headers
Bug: 180427921
Test: m nothing
Change-Id: I3d7ff2625eb5c6f5ed5e094da2a5ce6acc0c987c
2021-02-22 18:23:22 +00:00
Paul Duffin
75b902a964 Separate versioned/unversioned testing in sdk/cc_sdk_test.go
Uses new capability added in previous change to separate the testing of
the versioned and unversioned snapshots in sdk/cc_sdk_test.go. Any test
that generated an _snapshot module that simply listed the modules and
did not check a new type only tests the unversioned output to reduce
the duplication and cost of changing the tests.

Bug: 180479010
Test: m nothing
Change-Id: If11f82b3dd8ec79110b3a2f5adf193b6464000ab
2021-02-22 18:23:22 +00:00
Paul Duffin
d075907d29 Support testing versioned/unversioned sdk Android.bp files separately
Previously, the only way to test the Android.bp file generated by the
sdk module was to test both the unversioned and versioned parts
together. This change allows them to be tested separately.

Bug: 180479010
Test: m nothing
Change-Id: I3d695bcfbff030a6da393283ab30ec0979fb2826
2021-02-22 18:23:22 +00:00
Treehugger Robot
b68036f44b Merge "Add LOCAL_LICENSE_KINDS to build/soong" 2021-02-10 07:18:29 +00:00
Bob Badour
02040de891 Add LOCAL_LICENSE_KINDS to build/soong
Added SPDX-license-identifier-Apache-2.0 to:
  Android.bp
  android/Android.bp
  android/soongconfig/Android.bp
  androidmk/Android.bp
  apex/Android.bp
  bazel/Android.bp
  bp2build/Android.bp
  bpf/Android.bp
  bpfix/Android.bp
  cc/Android.bp
  cc/config/Android.bp
  cc/libbuildversion/Android.bp
  cc/libbuildversion/tests/Android.bp
  cc/ndk_api_coverage_parser/Android.bp
  cc/ndkstubgen/Android.bp
  cc/symbolfile/Android.bp
  cmd/dep_fixer/Android.bp
  cmd/diff_target_files/Android.bp
  cmd/extract_apks/Android.bp
  cmd/extract_jar_packages/Android.bp
  cmd/extract_linker/Android.bp
  cmd/fileslist/Android.bp
  cmd/host_bionic_inject/Android.bp
  cmd/javac_wrapper/Android.bp
  cmd/merge_zips/Android.bp
  cmd/multiproduct_kati/Android.bp
  cmd/path_interposer/Android.bp
  cmd/pom2bp/Android.bp
  cmd/pom2mk/Android.bp
  cmd/sbox/Android.bp
  cmd/soong_build/Android.bp
  cmd/soong_env/Android.bp
  cmd/soong_ui/Android.bp
  cmd/zip2zip/Android.bp
  cmd/zipsync/Android.bp
  cuj/Android.bp
  dexpreopt/Android.bp
  dexpreopt/dexpreopt_gen/Android.bp
  env/Android.bp
  etc/Android.bp
  filesystem/Android.bp
  finder/Android.bp
  finder/cmd/Android.bp
  genrule/Android.bp
  jar/Android.bp
  java/Android.bp
  java/config/Android.bp
  kernel/Android.bp
  linkerconfig/Android.bp
  linkerconfig/proto/Android.bp
  makedeps/Android.bp
  partner/Android.bp
  phony/Android.bp
  python/Android.bp
  python/tests/Android.bp
  remoteexec/Android.bp
  rust/Android.bp
  rust/config/Android.bp
  scripts/Android.bp
  sdk/Android.bp
  sh/Android.bp
  shared/Android.bp
  symbol_inject/Android.bp
  symbol_inject/cmd/Android.bp
  sysprop/Android.bp
  tradefed/Android.bp
  ui/build/Android.bp
  ui/logger/Android.bp
  ui/metrics/Android.bp
  ui/metrics/proc/Android.bp
  ui/status/Android.bp
  ui/terminal/Android.bp
  ui/tracer/Android.bp
  xml/Android.bp
  zip/Android.bp
  zip/cmd/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD to:
  finder/fs/Android.bp
  third_party/zip/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work

Change-Id: Ia47ca14f16b8c9f84f9d533a07e5b00e2c04e8d4
2021-02-06 04:23:21 +00:00
Paul Duffin
22ff0aaf51 Export implementation class jars for java_boot_libs
Hiddenapi processing currently requires access to the class
implementation jars for libraries on the bootclasspath which means that
they need to be provided as part of the prebuilts. This change modifies
the java_boot_libs property on the sdk to make those files available.

Modularization of the hiddenapi processing will hopefully remove the
need for these to be exported so this should be temporary.

Bug: 178361284
Test: m art-module-sdk
      check generated snapshot zip contains implementation jars
Change-Id: I9e94662dddb0ddb85a477ae6d27e533085147e88
2021-02-05 13:35:25 +00:00
Paul Duffin
c059c8c9a0 Add java.RegisterRequiredBuildComponentsForTest function
Insulate tests that exercise code in the java package from having to
register the build components provided by the java package by providing
a single function that registers them all. This follows the pattern
currently used in the cc and rust packages.

This change is in preparation for switching the dex_bootjars singleton
from a singleton, which does not require a module definition in order
to be instantiated, to a singleton module which does. That will require
adding a module definition into java.GatherRequiredDepsForTest() and
this change ensures that the required components will have been
registered in every test.

Bug: 177892522
Test: m nothing
Change-Id: I6475db8240894947dd07c89a940a3e4f201aa598
2021-01-21 11:47:32 +00:00
Paul Duffin
db170e4a92 Add java_boot_libs to sdk
The build has some implicit dependencies (via the boot jars
configuration) on a number of modules, e.g. core-oj, apache-xml, that
are part of the java boot class path and which are provided by mainline
modules (e.g. art, conscrypt, runtime-i18n) but which are not otherwise
used outside those mainline modules.

As they are not needed outside the mainline modules adding them to
the sdk/module-exports as either java_libs, or java_header_libs would
end up exporting more information than was strictly necessary. This
change adds the java_boot_libs property to allow those modules to be
exported as part of the sdk/module_exports without exposing any
unnecessary information.

Some points to note:
* The java_import has to have a valid file for the src property
  otherwise it will be disabled.
* The src property is supposed to reference a jar file but the
  java_boot_libs property will make it reference an empty file (not
  an empty jar) so that any attempt to use that file as a jar, e.g.
  compiling against it, will cause a build failure.
* The name of the file passed to the src property should make it
  clear that the file is not intended to be used.
* The test makes sure that only the jar file is copied to the
  snapshot.

Test: m nothing
Bug: 171061220
Change-Id: I175331e4c8e3874ab70a67cdc2f76ed1576e41eb
2021-01-15 18:14:10 +00:00
Anton Hansson
dff2c78a20 Add attribute to disable last-api compat tracking
Setting this to true by default is dangerous as it can mask bugs. Create
a dedicated attribute for java_sdk_library to enable this behavior
instead. The default will be flipped in a future CL when all the current
offenders have been fixed.

Fix all the tests to have the right API files.

Bug: 176092454
Test: m nothing
Change-Id: Ieab94bcb74abf8d018365a56fb447fe3dbd46957
2020-12-21 17:25:30 +00:00
Jaewoong Jung
02b11a6035 Remove AndroidMkExtraFootersFunc entries param.
I added in case anyone needs to access AndroidMkEntries to generate
footer lines, but nobody uses it, and it only confuses people.

Test: m nothing, TreeHugger
Change-Id: Ic8a450e3c306d9228c1fdec212c7441bd6aaee03
2020-12-07 10:23:54 -08:00
Colin Cross
f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Jingwen Chen
cda22c9bb9 Refactor 'in_make' to mean Kati is not skipped.
In Nougat and before, Make wrote a marker file to indicate that
soong_build was invoked from Make to change certain behaviors of Soong
at build time.
https://cs.android.com/android/platform/superproject/+/android-7.1.2_r36:build/core/soong.mk;l=70-73;drc=ae18638b0406ad107b0882a02a13cdd8b92f2a4e

Things have changed, and now soong_build is invoked from soong_ui, which
supports a --skip-make configuration flag:
https://cs.android.com/android/platform/superproject/+/master:build/soong/ui/build/build.go;l=31-33;drc=680387bf1d3ce7cbc77f535be7c42cec411b1687

Thus, the various remnants of 'EmbeddedInMake' and 'inMake'
configuration are misleading, since soong_build is no longer invoked
from Make. This CL refactors all instances to actually mean that
Kati is enabled (not skipped with --skip-make), and will run after
soong_build finishes, so Kati-specific behavior like the AndroidMk
singleton should run.

Test: TH presubmit

Change-Id: I576ab8e54f99f5c8ddf9feaf9a828019b279e266
2020-11-23 00:29:18 -05:00
Paul Duffin
ab5ac8f169 Revert "Use glob for java_sdk_library_import stub_srcs"
This reverts commit 7f97957ded.

Reason for revert: breaks sdk snapshots b/173508731
Bug: 173508731
Test: Ran prebuilts/runtime/update.py and then m nothing
      Before revert it failed
      After revert it worked

Change-Id: I9c081681fac589e37788a0d592435e3224011c58
2020-11-19 12:03:51 +00:00
Treehugger Robot
0a829f2a0a Merge "Always set apex_available in SDK snapshots." 2020-11-17 01:30:46 +00:00
Colin Cross
ae8600b507 Pass Config to NewTestContext instead of ctx.Register
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.

Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
2020-11-12 10:07:49 -08:00
Paul Duffin
a3cb6cf2e5 Remove support for droidstubs in sdk/module_exports
The droidstubs support in sdk/module_exports was a temporary measure
to work around the fact that some APIs were being defined by direct use
of droidstubs instead of java_sdk_library.

This change removes that support as those APIs have all been switched
from droidstubs to use java_sdk_library so droidstubs support is no
longer needed.

Bug: 168301990
Test: m nothing
Change-Id: I3517bed29b030438a0423a6cb8c248992a988222
2020-11-10 13:55:17 +00:00
Paul Duffin
7f97957ded Use glob for java_sdk_library_import stub_srcs
Some java_sdk_library modules do not have any stubs for some API
scopes. That results in an empty ".srcjar" being created for them which
ends up not creating a directory for the stubs sources when that
snapshot is unzipped. Previously, that would cause a build failure as
the generated java_sdk_library_import module used the directory, which
did not exist, in its stub_srcs property.

This change switches the stubs_srcs property to use a glob pattern of
"**/*.java" relative to the directory instead of using the directory
directly. When the directory does not exist the glob pattern is
resolved to an empty set of paths and does not break the build.

Bug: 172811712
Test: Add sdkextensions-sdk sdk module (local patch)
      m sdk-extensions (local patch)
      unpack generated snapshot
      m nothing - results in build failure due to missing directory
      Make this change and repeat above except this time the build works.
Change-Id: I691ffbfdc01ba89bbcaf647dcbb7dfebc3c8aec2
2020-11-09 17:42:52 +00:00
Martin Stjernholm
1e04109c6e Always set apex_available in SDK snapshots.
Also fix //apex_available:platform that could get lost in the generated
apex_available properties. It only happened if a module didn't specify
an apex_available setting and then got extra entries from
apex.BaselineApexAvailable.

Test: Build ART SDK with libnativeloader_lazy in the SDK, and check
  that the generated prebuilt module has //apex_available:platform
  (before https://r.android.com/1480559).
Change-Id: I9df20aee63455932a7e558993f0f26769cb02792
2020-11-04 18:44:14 +00:00
Martin Stjernholm
59e0c7a98f Disable sanitizer flags and output files for sanitizer-enabled SDK members.
Test: build/soong/soong_ui.bash --make-mode \
      TARGET_PRODUCT=aosp_blueline_hwasan \
      TARGET_BUILD_VARIANT=userdebug nothing
Test: build/soong/scripts/build-aml-prebuilts.sh \
      runtime-module-{sdk,host-exports} \
      platform-mainline-{sdk,test-exports}
  Check that generated Android.bp files don't differ
Bug: 171754462
Change-Id: Ieef40ff202ceda2d9e613a5d9295f5c9d8160626
2020-10-29 00:26:24 +00:00
Jiyong Park
8fe14e6a49 Enable sdk and sdk members in os_arch granularity
This amends Idad7ef138cdbcbd209d390bf6c10ca8365d4619f. With the change,
when there is a member that returns IsHostOsDependent() == true,
the sdk having the member and the member itself are disable for host and
only the os that the member supports is explicitly enabled.

However, that change will cause a problem when we add the support for
the linux_bionic_arm64 target. The target is not enabled when building
sdk snapshots. The only linux_bionic target that is enabled is
'linux_bionic_x86_64'. However, since the granularity is os which is
linux_bionic, the snapshot is generated as follows.

cc_prebuilt_binary {
    target: {
        host: {
	    enabled: false,
	},
	linux_bionic: {
            enabled: true,
        },
	linux_bionic_x86_64: {
            srcs: ["x86_64/bin/..."],
	},
	// no srcs for linux_bionic_arm64
    },
}

Above is a problem for linux_bionic_arm64 target because the target is
enabled (via linux_bionic.enabled: true), but srcs is not provided.

To fix the problem, the enabling of a target is done in a target
(os_arch) granularity, rather than os granularity. For example, above
now becomes ...

cc_prebuilt_binary {
    target: {
        host: {
	    enabled: false,
	},
	linux_bionic_x86_64: {
            enabled: true,
            srcs: ["x86_64/bin/..."],
	},
    },
}

Only the targets that the snapshot actually can provide srcs are enabled
and the rest of the host targets are disabled.

Bug: 159685774
Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh
runtime-module-host-exports

Change-Id: Ibca48c40f6dc4628b5f4bfa4ceb68ebe0973cc81
2020-10-23 13:05:13 +09:00
Colin Cross
405af07859 Revert "Make lots of tests run in parallel"
This reverts commit 323dc60712.

Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests

Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
2020-10-09 18:34:24 -07:00
Colin Cross
323dc60712 Make lots of tests run in parallel
Putting t.Parallel() in each test makes them run in parallel.
Additional t.Parallel() could be added to each subtest, although
that requires making a local copy of the loop variable for
table driven tests.

Test: m checkbuild
Change-Id: I5d9869ead441093f4d7c5757f2447385333a95a4
2020-10-06 15:12:22 -07:00
Paul Duffin
157f40f056 Add prebuilt_visibility property
Some mainline modules provide an `sdk` and one or more set of
module_exports. When the snapshots are unpacked into separate prebuilts
directories (one per snapshot) then any dependencies from a member of
one snapshot to a member of another may require the latter to have
additional visibility rules.

Previously, these rules had to be added to each source module. This
change allows additional visibility rules to be specified on the
sdk/module_exports that are added to all the prebuilts in its snapshot.

Bug: 155921753
Bug: 168301990
Test: m nothing
Change-Id: Ia3eacb5749981f04770ae9872a8013e43c5c6ef1
2020-10-02 10:59:41 +01:00
Paul Duffin
d99d997238 Explicitly specify visibility in sdk/module_exports snapshot
This change ensures that each prebuilt in a snapshot explicitly
specifies its visibility even when that visibility is the current
default. This is done for two reasons:
1. It simplifies a follow up change that adds visibility rules to an
   existing set of rules.
2. It ensures that the snapshots are independent of the current Soong
   default visibility.

The latter is important because we intend to switch from modules being
visible to everyone by default (i.e. //visibility:public) to only being
visible to modules in the same package (i.e. //visibility:private). By
making the snapshots of modules that do not specify any visibility
explicitly specify that they are "//visibility:public" it ensures that
the snapshots will not need to be changed when the default changes.

Bug: 168301990
Test: m nothing
Change-Id: Ia034f4a1e5124c17f46d73b0e9a6c5f2a251038e
2020-10-01 18:20:13 +01:00
Martin Stjernholm
b7c77bd889 Merge "Retain all stubs versions in the snapshot." 2020-09-29 13:59:05 +00:00
Paul Duffin
41b0876c10 Merge "Allow java_sdk_library to specify doctags_files" 2020-09-29 13:55:53 +00:00
Martin Stjernholm
dc6f2b06be Merge "Propagate all sanitizer flags in SDK snapshots." 2020-09-29 10:43:08 +00:00
Martin Stjernholm
618b671bf4 Retain all stubs versions in the snapshot.
This is a quick fix to avoid build failures due to prebuilts lacking
stubs versions that the sources have, which causes dependencies on
older version stubs fail on master-art-host where the sources of e.g.
Bionic are no longer available.

More work is required to make the version handling of stubs DTRT - a
TODO comment outlines it.

Test: m nothing
Test: `m runtime-module-sdk`, check that the generated Android.bp
  has the full stubs.versions list like the sources.
Bug: 169373910
Change-Id: I0ebfaf94f9d45a0e58d9785c40e7cea626f0ab83
2020-09-28 22:32:02 +01:00
Paul Duffin
a2ae7e0358 Allow java_sdk_library to specify doctags_files
When generating Javadoc the processor needs to be given information
about the doctags that are present in the source. This change allows
that information to be managed with the java_sdk_library that generates
the stubs source from which the Javadoc is generated.

Bug: 168301990
Test: Built offline-sdk-docs with and without the change and
      diffed them. The only difference was the timestamp.js
      file.
Change-Id: I4adbeb0781bc2191461fec856ffa90ea185e7434
2020-09-28 18:35:57 +01:00
Martin Stjernholm
79bee057a3 Merge "Handle property structs and BpPropertySets as values to AddProperty." 2020-09-28 10:06:04 +00:00
Martin Stjernholm
b024957791 Propagate all sanitizer flags in SDK snapshots.
liblog snapshot needs to sanitizer.address=false to avoid cycle in asan
builds. Adding that separately in library_sdk_member.go would start to
feel like whack-a-mole, so the snapshot generation is instead extended
to handle nested property structs.

This uses the BpPropertySet.AddProperty extension in
https://r.android.com/1423510, and common value optimisation now
recurses into non-anonymous structs, instead of comparing them as a
whole.

Test: m nothing
Test: `m SANITIZE_TARGET=address nothing` with prebuilts/runtime
  present in the manifest and a fresh snapshot made with this
Bug: 151303681
Change-Id: I472554117a488e6c800045cb2ed59377778571a4
2020-09-25 00:33:04 +01:00
Martin Stjernholm
191c25f589 Handle property structs and BpPropertySets as values to AddProperty.
Both will create a nested property set, that may be merged with an
existing one.

Test: m nothing
Bug: 151303681
Change-Id: I30696ba3eb8960ca6fa54c9ee2cf6229ab9f5da9
2020-09-25 00:33:04 +01:00
Jiyong Park
1613e5541f HostCross is an attribute of a Target, not OsType
A host target is considered as being cross-compiled when the target
can't run natively on the build machine. For example, linux_glibc/x86_64
is a non-cross target on a standard x86/Linux machine, but is a cross
host on Mac. Previously, whether cross or not was a static attribute of
an OsType. For example, Windows was always considered as cross host,
while linux_bionic was not. This becomes a problem when we support more
host targets like linux_bionic/arm64 which should be cross-host on
standard x86/Linux machines.

This change removes HostCross from the OsClass type and instead adds a
property HostCross to the Target type. When a target is being added, it
is initialized to true when the target can't run natively on the current
build machine.

Bug: 168086242
Test: m
Change-Id: Ic37c8db918873ddf324c86b12b5412952b0f2be2
2020-09-24 14:14:31 +09:00
Paul Duffin
d6abaa71f8 Propagate recovery_available property through to sdk snapshot
Needed for jni_headers which is marked as recovery_available because libchrome
is also marked and that library is used by others that are the recovery
partition. There may be other dependency paths that require jni_headers to be
recovery_available that was just the first one found.

Bug: 168301990
Test: m nothing
Change-Id: I916ecfe7700b9c153b29115e849da148fc7e7a76
2020-09-11 16:09:28 +01:00
Paul Duffin
ee9ad5d526 Remove framework-modules naming scheme
The naming_scheme property was not removed as it may be useful for
future when migrating to java_sdk_library.

Bug: 168301990
Test: m nothing
Change-Id: Ie97dd60355a207f1312a2dd910f1fb25b46fd737
2020-09-11 16:09:15 +01:00
Paul Duffin
d1edbd4eed Propagate vendor_available property through to sdk snapshot
Test: m nothing
Change-Id: I68dcba111d290ae8a84c2bc023e1a4e0eaa43408
2020-09-03 18:49:35 +01:00
Treehugger Robot
4c2e4f381a Merge "Support ninja rsp files in soong_zip" 2020-08-24 18:40:56 +00:00
Martin Stjernholm
fbb486fc7a Propagate sanitize.never flag in SDK snapshots.
This is necessary to not get dependencies on libclang_rt sanitizer libs
for the CRT objects which lead to cyclic dependencies in sanitizer
builds.

Test: `lunch aosp_blueline_hwasan-userdebug && m nothing`
  in a tree that has the prebuilts created and unzipped from
  `build/soong/scripts/build-aml-prebuilts.sh runtime-module-{sdk,host-exports}`
Bug: 151303681
Change-Id: I3f848a084280bdc3ade4b74df03e981d8cc61222
2020-08-21 18:43:51 +01:00
Colin Cross
053fca10c9 Support ninja rsp files in soong_zip
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.

Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
2020-08-19 21:18:56 +00:00
Colin Cross
aede88c1c7 Reland: Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-19 10:21:17 -07:00
Pete Bentley
0c7b26e0e2 Revert "Deduplicate APEX variants that would build identically"
This reverts commit d6b2525b00.

Reason for revert: <Breaks tests - confirmed by Forrest: b/165188843>

Change-Id: I7cb68cb87522415004390c0672dc774e0067b122
2020-08-18 13:44:59 +00:00
Colin Cross
d6b2525b00 Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I0529837476a253c32b3dfb98dcccf107427c742c
2020-08-17 15:18:31 -07:00
Martin Stjernholm
caa47d7eed Disable all host OS variants except those explicitly included.
Necessary to avoid problems with implicitly enabled host OS'es, e.g.
linux_glibc getting enabled when we only supply a linux_bionic linker
in runtime-module-host-exports. That will then cause a non-functional
prebuilt to take precedence over source if the prebuilt is preferred.

We don't do this for device since we so far only support a single
device OS (android).

This introduces the notion that SDK member types can be host OS
dependent or not. That way java members with host prebuilts don't get
restricted to a specific host OS.

Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh runtime-module-host-exports
  Check that the generated Android.bp correctly disables the bionic
  linker prebuilt for linux_glibc.
Test: art/build/apex/runtests.sh
  on master-art with an updated runtime SDK snapshot
Test: art/tools/buildbot-build.sh {--host,--target}
  on master-art with an updated runtime SDK snapshot
Bug: 160349757
Change-Id: Idad7ef138cdbcbd209d390bf6c10ca8365d4619f
2020-08-03 21:06:50 +01:00
Martin Stjernholm
b4bfd148eb Merge "Set compile_multilib for SDK snapshots for device." 2020-08-03 17:55:25 +00:00
Paul Duffin
021f4e525f Fix prebuilt mutator ordering in tests
Previously, the prebuilt mutators were added by the
cc.RegisterRequiredBuildComponentsForTest() function as a convenience
but unfortunately it lead to some of the mutators being in a different
order in the tests than in the normal build.

This change:
* Extracts the RegisterPrebuiltMutators() call from
  cc.RegisterRequiredBuildComponentsForTest()
* Makes sure that the prebuilt mutators are registered before the
  visibility gatherer and enforcer mutators.

Bug: 162505935
Test: m nothing
Change-Id: I7d959b558200b502f0a5e4653c41ea01414e142a
2020-07-31 16:12:01 +01:00
Paul Duffin
a37eca27c8 Avoid creating APEX variant for sdk member
Previously, an APEX variant was created for a module that was a member
of an SDK just in case it had to be replaced with an APEX requested
snapshotted version of that member. That was necessary because that was
the only way to have APEX specific replacements.

Since then a new method called ReplaceDependenciesIf() has been added
which provides fine grained control over which dependencies are
replaced. This change uses that new method to handle the replacements
which allows the APEX variants to be removed.

Bug: 161928524
Test: m nothing
Change-Id: If3869dd6753dc182b099af566b20fbc9c9c6eff7
2020-07-24 11:10:10 +01:00
Pete Gillin
84c3807b99 Enforce the legacy core/platform API restriction.
This change silently decides whether modules which depend on either
sdkPrivate or sdkCorePlatform get the legacy or the stable version of
the core/platform API, based on whether the module's name is on a
hard-coded list or not.

Test: m java
Test: make a target from the list when its entry is commented out, which correctly fails
Bug: 157640067
Change-Id: I15e5a6c2f07e73718803501d705de0d7ab9bec90
Merged-In: I15e5a6c2f07e73718803501d705de0d7ab9bec90
Merged-In: Iaa97ddaa015e8079fcb3426585c5101c7ec9e22a
(cherry picked from commit c0f4373106)
2020-07-23 09:06:52 +01:00
Treehugger Robot
0bcda81dd5 Merge "Generate module lib API and scope together" 2020-07-21 04:21:07 +00:00
Paul Duffin
15f34ef811 Generate module lib API and scope together
Previously, due to limitations in metalava the module lib API and stubs
source were generated separately. That limitation has since been
removed so this change optimizes the generating code.

Changes:
* Add an annotation field to apiScope to contain the annotation that
  identifies the scope.
* Rename droidstubArgs to extraArgs to better reflect that they are not
  passed to droidstubs.
* Rename droidstubsArgsForGeneratingStubsSource to droidstubsArgs.
* Remove droidstubsArgsForGeneratingApi and
  createStubsSourceAndApiTogether fields along with code that supports
  generating stubs source and the API separately.
* Fix the module lib test.

Test: m checkapi
Bug: 161364853
Change-Id: I821886bb68645addf5ae9c96ed2f4f3649151d1a
2020-07-20 20:51:25 +01:00
Martin Stjernholm
4cfa2c692b Set compile_multilib for SDK snapshots for device.
Previously compile_multilib was ignored for device and defaulted to
"both", which could lead to an internal inconsistency in the snapshot
if a member was 32- or 64-only.

This also implements common property optimisation for compile_multilib,
by adding a struct that can be extended to other properties in the
future.

Test: m nothing
Bug: 151303681
Change-Id: I3e96222858e5e755173aca03508a23c431977e2a
2020-07-17 14:32:15 +01:00
Martin Stjernholm
89238f4703 Add compile_multilib properties to all native prebuilts (reland).
There was a special case in cc_prebuilt_binary only, which resulted in
prebuilt libraries getting both 32 and 64 bit variants even when their
sources only had one of them, and the other variant would be defunct
since it wouldn't get any prebuilt artefact.

This moves the handling of compile_multilib completely to the common
update code, so that SDK members don't need to deal with it. It doesn't
take SDK member defaults into account, which means a bit more
boilerplate in the snapshots, but it's simpler and less error prone
(different SDK member types have different defaults).

This relands https://r.android.com/1359962 after disabling the SDK
tests on mac once and for all.

Bug: 151303681
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I05f6603b2ac9b8676b25c3e297165ca23284e9cc
2020-07-17 14:32:15 +01:00
Martin Stjernholm
7feceb2354 Add support for SDK tests with LinuxBionic OS type.
Break apart test helpers a bit to make it possible to enable
LinuxBionic in a single test, and add LinuxBionic support to
cc.GatherRequiredDepsForTest.

Test: m nothing
Bug: 160349757
Change-Id: Iace1024c964cee2308c130c945daef9e46c18c66
2020-07-16 21:55:15 +01:00
Martin Stjernholm
3ff2e661bd Disable all SDK snapshot generation tests on mac.
We frequently miss putting in the per-test skip condition, and since
mac isn't tested in presubmit this leads to build breakages and time
consuming reverts. To avoid that this blanket disables all the SDK
tests on mac. It's not used there and we got test coverage in many
linux-based builds.

Bug: 145598135
Bug: 161315642
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I2aea92fef2c0f8c2742396fe36610501dc5a6f0f
2020-07-15 16:58:05 +01:00
Jakub Pawlowski
ca7b2c3eb7 Merge "Revert "Add compile_multilib properties to all native prebuilts."" 2020-07-15 10:33:41 +00:00
Charles Chen
29936ae5a7 Revert "Add compile_multilib properties to all native prebuilts."
This reverts commit 1e9c2677fd.

Reason for revert: Break build

Bug: 161315642
Change-Id: Ibf3d5e88412327e8f08d08272b75cd3df34a71e5
2020-07-15 04:01:45 +00:00
Treehugger Robot
6da69732e4 Merge "Add compile_multilib properties to all native prebuilts." 2020-07-15 02:51:21 +00:00
Treehugger Robot
9d98ac15bb Merge "Fix string concatenation error" 2020-07-12 23:08:40 +00:00
Martin Stjernholm
1e9c2677fd Add compile_multilib properties to all native prebuilts.
There was a special case in cc_prebuilt_binary only, which resulted in
prebuilt libraries getting both 32 and 64 bit variants even when their
sources only had one of them, and the other variant would be defunct
since it wouldn't get any prebuilt artefact.

This moves the handling of compile_multilib completely to the common
update code, so that SDK members don't need to deal with it. It doesn't
take SDK member defaults into account, which means a bit more
boilerplate in the snapshots, but it's simpler and less error prone
(different SDK member types have different defaults).

Bug: 151303681
Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh art-module-host-exports
  Check that the generated Android.bp passes Soong.
Change-Id: Ib73444c6788ee1c78480bdb103aa2b8ae8f2c63c
2020-07-11 00:38:24 +01:00
Colin Cross
afa6a77c64 Fix string concatenation error
Fix an error caught by golint that prevents using go test.  string(int)
produces a rune, not a string representation of the int.  Use fmt.Errorf
instead to format the string.

Test: go test
Change-Id: I4c2e1e30a0f1b3dc24afd123c38c4e41c2abb47a
2020-07-10 16:32:49 -07:00
Paul Duffin
ed62b9cc2a Differentiate sdk snapshot include dir by os type
Test: m nothing
Change-Id: Ide032aa64bb1fee69969c5e007bee6643d475309
2020-07-10 12:18:03 +01:00
Paul Duffin
cee7e66b07 Ensure that sdk/module_exports depends on source members
Previously, preferring a prebuilt of an sdk/module_exports's member
would cause the sdk/module_exports to depend on the prebuilt instead
of the source and cause problems with the build.

This chance prevents the dependency from an sdk/module_exports to its
members from being replaced with prebuilts.

Bug: 160785918
Test: m nothing
Change-Id: Iee4bcd438c11929e30fb5766701b05a0e89956d9
2020-07-09 17:51:06 +01:00
Treehugger Robot
2c79c871d9 Merge changes Iefcf4fbe,I450d4769,I1ffc7e7b
* changes:
  Prepend default to allow overriding in the bp file.
  Do not follow SDK member dependencies for APEX payloads.
  Do not follow prebuiltDependencyTags in APEX payload walks.
2020-07-09 12:32:31 +00:00
Paul Duffin
44f1d8404b Stop java_sdk_library_import from depending on source modules
Previously, java_sdk_library_import added the dependencies on its child
components in the deps mutator after prebuilts without a matching
source module are renamed to the source module name. That meant that
the java_sdk_library_import has to use the source module name and ended
up depending on the source module unless it was preferred.

This change adds a new component deps mutator that runs before the
PrebuiltRenameMutator so that the java_sdk_library_import can add
dependencies onto the prebuilt modules. It also updates an affected
test.

Bug: 159902351
Test: m nothing
Change-Id: I3576c4873302743e51aff547ea1497bef6d748ac
2020-07-08 16:46:16 +01:00
Martin Stjernholm
26ab8e80e4 Prepend default to allow overriding in the bp file.
Bug: 143948100
Bug: 151303681
Test: m nothing
Change-Id: Iefcf4fbe9d2971ce267242185165f0c38f68db30
2020-07-07 18:14:26 +01:00
Martin Stjernholm
cc77601d10 Do not follow SDK member dependencies for APEX payloads.
Test: m nothing
Test: `m` with prebuilts/runtime in the manifest (along with other
  fixes)
Bug: 151303681
Change-Id: I450d476975c7ab4434228b8c4baf3af192142211
2020-07-07 18:14:26 +01:00
Jingwen Chen
40fd90ae52 Support multiple dists per Android.bp module, and dist output selection.
This CL adds "dists" to the base property struct to support multiple
dist file configurations, and generic tag support to dist tagged outputs
of modules.

Fixes: b/152834186
Test: soong tests and `m sdk dist`

Change-Id: I80c86bc9b7b09e671f640a4480c45d438bdd9a2a
Signed-off-by: Jingwen Chen <jingwen@google.com>
2020-06-25 12:42:07 +00:00
Paul Duffin
4e77284ab4 Include resources in sdk snapshot of java library impl
Bug: 159607838
Test: m nothing
Change-Id: Ie7ddbffc3957f832abfadb27292c436e8e5dcbf3
2020-06-24 13:04:15 +01:00
Martin Stjernholm
47ed352227 Propagate unique_host_soname to cc_library prebuilts.
Test: m nothing
Test: Check on go/Android.bp that cc_prebuilt_library supports
  unique_host_soname
Bug: 158743135
Change-Id: Ie02a5cae057fb9092c226d8c5b7f63e1f66ad066
2020-06-17 23:06:43 +01:00
Paul Duffin
0c5bae5e83 java_sdk_library: Add system-server scope
Bug: 155164730
Test: m nothing
Merged-In: I49a2dab5c064b05f16691a3fae65f2b4ffc53bfd
Change-Id: I49a2dab5c064b05f16691a3fae65f2b4ffc53bfd
(cherry picked from commit 5a757b1ebb)
2020-06-15 15:02:19 +01:00
Colin Cross
440e0d0542 Use inclusive language in build/soong
Test: m checkbuild
Change-Id: Id07890b7cbc2397291a658ca00e86b43c743aafc
2020-06-11 15:33:16 -07:00
Pete Gillin
1f41dbff64 Split the core/platform API into stable and legacy versions.
For now, everything outside libcore still uses the legacy version.

Test: treehugger
Bug: 157640067
Change-Id: If5234e9ee533ff537926801a0af045d36b1caf01
2020-06-08 10:53:51 +01:00
Jaewoong Jung
264700345d Merge "Soong package structure refactoring" 2020-06-02 16:30:50 +00:00
Paul Duffin
3dbf9fd892 Copy removed.txt file to the snapshot correctly
Previously, the code copied the current.txt file to both the
current_api and removed_api properties. This change copies the
removed.txt file to the removed_api property instead.

Bug: 157980685
Test: m nothing
Change-Id: Iad34e91051da43222d22c240c16f50887c43d73e
2020-06-02 14:40:28 +01:00
Jaewoong Jung
4b79e98a6e Soong package structure refactoring
Give prebuilt_etc and sh_binary their own packages and split the
gigantic main Android.bp up to small, per-package ones.

Test: m nothing, TreeHugger
Bug: 156980228
Change-Id: I7b00cd344b9f16861f1ff39edf0029f016b853d0
2020-06-01 13:44:48 -07:00
Martin Stjernholm
a6a224586b Merge "Propagate flags necessary for a prebuilt of the Bionic linker." 2020-06-01 09:55:10 +00:00
Martin Stjernholm
7130fabd20 Propagate flags necessary for a prebuilt of the Bionic linker.
The flags are necessary to avoid implicit dependencies on crt libs,
libc, and the linker (recursively). The reason cc_prebuilt_binary react
to these flags is that they can affect the install rules.

Test: m nothing
Test: art/test/testrunner/run_build_test_target.py art-linux-bionic-x64
  on a master-art branch using snapshot built from runtime-module-sdk
  and runtime-module-host-exports
Bug: 152255951
Change-Id: I6b2fe92d105d1f446fffd00bd3267a416f75efb7
2020-05-28 22:58:01 +01:00
Paul Duffin
d7eb1c2a76 java_sdk_library: Propagate shared_library to snapshot
Passes the shared_library property through to the snapshot. It does not
optimize away the default value in order to make it easier to invert
the default value in future. The current default value was only chosen
for convenience because most existing usages were already treated as
shared libraries. It would be safer if modules had to opt in to be used
as shared libraries.

Bug: 155164730
Test: m nothing
Change-Id: I33c7323f2389b44ed49cebe517ae2cce349117f1
2020-05-28 17:39:25 +01:00
Paul Duffin
6b3ca917b0 Merge "Syntax check generated Android.bp snapshot" 2020-05-19 07:53:27 +00:00
Treehugger Robot
e3fa145692 Merge "Fix special case when appending a visibility rule to //visibility:private." 2020-05-15 14:08:24 +00:00
Martin Stjernholm
08f72a9709 Merge changes I404f6e5c,I0d2d86f0
* changes:
  Add explicit visibility on the self package in EffectiveVisibilityRules.
  Don't print :__pkg__ for package rules, since that's the default.
2020-05-14 21:49:05 +00:00
Martin Stjernholm
64aeaaddef Fix special case when appending a visibility rule to //visibility:private.
Test: m nothing
Bug: 151303681
Change-Id: Ibe7e953219e4660c653063e87cb08407a2cb513d
2020-05-14 16:52:24 +01:00
Martin Stjernholm
0641d189d0 Add explicit visibility on the self package in EffectiveVisibilityRules.
Modules in a package is implicitly visible within the same package.
When making effective visibility rules which work from another location
that visibility needs to be mentioned explicitly. In practice this
allows prebuilts to remain visible to other modules in the source
package when the prebuilts are preferred.

Test: m nothing
Test: m conscrypt-module-sdk, check that the snapshot Soong modules give
  visibility to //external/conscrypt
Bug: 151303681
Change-Id: I404f6e5ca7021974a8c7be5e4d6b4982c050b8a5
2020-05-14 16:50:59 +01:00
Martin Stjernholm
01407c525a Don't print :__pkg__ for package rules, since that's the default.
To reduce clutter a little.

Test: m nothing
Change-Id: I0d2d86f00c3d38e603d6f58c4b1dd9a4170bf819
2020-05-14 16:50:59 +01:00
Paul Duffin
6b836ba566 Correct module_lib scope name
The module_lib scope should be called module-lib in order to pick up
the latest filegroup. Without it the API lint does not use a baseline
and so reports issues with released and unchangeable APIs.

It is also needed for the correct dist path.

Test: m update-api
Bug: 155164730
Change-Id: I7dbafd7164d5da600ca45c39a7f93a5a40027a1f
2020-05-14 00:53:50 +01:00
Paul Duffin
f7a6433bab java_sdk_library: Propagate naming_scheme to snapshot
Bug: 155164730
Test: m nothing
Change-Id: Iec7c52d2102b6294540090f3c1e32bd9d8cc7da8
2020-05-13 20:40:53 +01:00
Paul Duffin
87a05a389d java_sdk_library: Add sdk_version for API scopes
Allow the sdk_version against which the stubs for a scope are compiled
to be specified in the module on a per scope basis.

Bug: 155164730
Test: m nothing
Change-Id: I5881e5ee7c2169c30f544882344a60a602dae917
2020-05-13 13:09:23 +01:00
Paul Duffin
0ff08bdb07 java_sdk_library: Allow separate api/stubs source modules
The API file for a scope represents the differences between the API
provided by that scope and that provided by the scope that it extends.
On the other hand the stubs source for a scope represents the union of
the API provided by the scope and the scope it extends (all the way
back to public).

Unfortunately, while metalava supports this behavior for scopes that
extend public (e.g. system and test) it does not support this behavior
for scopes that extend others, e.g. module_lib which extends system.
This is because it always assumes that the baseline for the API file
is 'public' and so has no way to defined other baselines.

This change works around that by having separate droidstubs modules to
generate the API and stubs sources for scopes that require different
arguments to generate the API and stubs sources.

Test: m checkapi
Bug: 155164730
Change-Id: Iea7d59852d7aeb503120acf3c44e08eb0d9d07b9
2020-05-13 08:52:53 +01:00
Paul Duffin
780c5f4715 java_sdk_library: Preserve sdk_version: "none" in snapshot
Test: m nothing
Bug: 156354511
Change-Id: Icf7c20dc109a773430768c1a2d91c674551dbc1b
2020-05-12 15:52:55 +01:00
Paul Duffin
f88d8e032f Syntax check generated Android.bp snapshot
Checks the syntax of the Android.bp file generated for the sdk
snapshot. While this will not detect all possible problems with the
generated Android.bp file it will detect some and it is far better to
detect those issues during generation than when it is unpacked and
used.

Bug: 155628860
Test: m nothing
Change-Id: Ieec86a7a49fa2e3bd8b9f83aca540114232a3476
2020-05-12 09:52:41 +01:00
Paul Duffin
1110827b3f Defer \n quoting generatedContents until creating the ninja rule
Previously, when writing to generatedContents \n characters were
quoted (replaced with \\n) so as to allow them to be preserved through
ninja/rsp/bash and were unquoted (replaced \\n with \n) just before
redirecting to the output file. That meant that any code which wanted
to access the contents for other purposes, e.g. testing had to unquote
\\n.

This change moves the quoting to be part of the code that generates the
ninja rule which simplifies any other code that has to access the
contents.

Without quoting the generated Android.bp files are not formatted
properly, are all on one line and completely unreadable.

Bug: 156286550
Test: m art-module-sdk and check generated Android.bp file to make sure
      it is properly formatted.
Change-Id: I768c3b96ed08a3daf251730e2a10d9d72338c49a
2020-05-11 22:59:25 +01:00
Paul Duffin
9f10bbf4cf Merge "java_sdk_library: Specify visibility of stubs modules" 2020-05-10 18:31:56 +00:00
Paul Duffin
7a1f7f32ab Retry: Fix snapshot of a host/device cc_library with stubs
Change since last attempt: Disable test that breaks on darwin.

Adds a test that fails with unknown property android.stubs.versions
and then fixes that by marking the field from which that property is
created with 'ignored-on-host' and implemented the isHostVariant on
*osTypeSpecificInfo.

Bug: 155628860
Test: m nothing
Change-Id: Ia4e744c9e799d5adaf8a2f761516f568ec363ad4
2020-05-08 22:32:08 +01:00
Paul Duffin
c459f89fb4 Retry: Adds support for 'ignored-on-host'
Adds a filter mechanism that can exclude property values from being
included in the common value extraction. That is needed to prevent the
snapshot mechanism from generating invalid output for properties that
are ignored on host (and have their values cleared) and which are not
tagged with `android:"arch_variant"`.

Changes:
* Updates the documentation of SdkMemberType to explain what effect
  the 'ignored-on-host' tag has.
* Adds some tests for this new mechanism.

Bug: 155628860
Test: m nothing
Change-Id: Ibafdb6e921ba5abe505bd8a91ca5a1d9c9b5d0cb
2020-05-08 22:32:02 +01:00
Paul Duffin
864e1b45db Retry: Detect invalid arch specific properties in snapshot
Previously, the snapshot code did not know whether a specific property
could be arch specific or not and assumed that they all were which
meant that it could generate snapshots containing arch specific values
for properties that are not arch specific and so would fail when
unpacked.

This change requires arch specific fields in SdkMemberProperties to be
tagged as such using `android:"arch_variant"` (just as in module input
property structures). Any property without that must have properties
that are common across all variants.

Bug: 155628860
Test: m nothing
Change-Id: I3df60f0b53ba02ec2c55a80c7da058eac5909d26
2020-05-08 22:31:56 +01:00
Greg Kaiser
a71fe150fa Merge changes from topic "revert-1302576-NGVCIVIIJO"
* changes:
  Revert "Detect invalid arch specific properties in snapshot"
  Revert "Adds support for 'ignored-on-host'"
  Revert "Fix snapshot of a host/device cc_library with stubs"
2020-05-08 19:18:40 +00:00
Greg Kaiser
e509447c93 Revert "Detect invalid arch specific properties in snapshot"
Revert submission 1302576

Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...

Change-Id: Id7eba0bdde5c579e10e9b42d94a7cfab5f34995f
2020-05-08 19:12:34 +00:00
Greg Kaiser
e08e03fa96 Revert "Adds support for 'ignored-on-host'"
Revert submission 1302576

Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...

Change-Id: I2a7ac0ef0232177eefc26542c11dc675d6f4cab2
2020-05-08 19:12:34 +00:00
Greg Kaiser
ad913d56ac Revert "Fix snapshot of a host/device cc_library with stubs"
Revert submission 1302576

Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...

Change-Id: Ibccce5286605bb71c6be3b3550ba86d8b7e24fa7
2020-05-08 19:12:34 +00:00
Paul Duffin
4911a89181 java_sdk_library: Specify visibility of stubs modules
Adds two new properties to specify the visibility of the stubs modules
(source and library) created by the java_sdk_library. Excludes
visibility property from being inherited when creating the module so it
can be properly specified by copying across the relevant property.

Test: m checkapi
Bug: 155164730
Change-Id: Iffdd9f191ff0d74646356ac577560cc38efdd790
2020-05-08 18:03:31 +01:00
Paul Duffin
8f265b9ab9 java_sdk_library: Add support for module_lib API surface
Adds api scope for the module_lib and adds that to the list of all
all available scopes. The scope is generated if and only if the
api_surfaces property contains "module_lib".

No other changes are needed as the generation of the APIs is completely
driven by the allApiScopes array and the information in its contained
apiScope structures.

Test: m checkapi
Bug: 155164730
Change-Id: I7769af6823badca8715a270f86cf53b4e954b7df
2020-05-08 07:51:00 +01:00
Paul Duffin
3375e35d99 java_sdk_library: Control API surfaces generated
Adds a per scope set of properties that allows explicit control over
the API surfaces generated.

Previously, the term active was used to determine whether it was
generated but that was a little abstract and unclear so has been
replaced by generated.

Test: m nothing
Bug: 155164730
Change-Id: I7539d89618b61f6b9d1a4b60cc3f9614b157f0d9
2020-05-08 07:51:00 +01:00
Paul Duffin
2af52380be Fix snapshot of a host/device cc_library with stubs
Adds a test that fails with unknown property android.stubs.versions
and then fixes that by marking the field from which that property is
created with 'ignored-on-host' and implemented the isHostVariant on
*osTypeSpecificInfo.

Bug: 155628860
Test: m nothing
Change-Id: I167b47a1374f541aa09d7e045972d740f1d9009c
2020-05-07 18:35:11 +01:00
Paul Duffin
12f67bcf42 Adds support for 'ignored-on-host'
Adds a filter mechanism that can exclude property values from being
included in the common value extraction. That is needed to prevent the
snapshot mechanism from generating invalid output for properties that
are ignored on host (and have their values cleared) and which are not
tagged with `android:"arch_variant"`.

Changes:
* Updates the documentation of SdkMemberType to explain what effect
  the 'ignored-on-host' tag has.
* Adds some tests for this new mechanism.

Bug: 155628860
Test: m nothing
Change-Id: I7ebd333079619dba546bc8c4911d567e0287b676
2020-05-07 18:35:11 +01:00
Paul Duffin
ee20962378 Detect invalid arch specific properties in snapshot
Previously, the snapshot code did not know whether a specific property
could be arch specific or not and assumed that they all were which
meant that it could generate snapshots containing arch specific values
for properties that are not arch specific and so would fail when
unpacked.

This change requires arch specific fields in SdkMemberProperties to be
tagged as such using `android:"arch_variant"` (just as in module input
property structures). Any property without that must have properties
that are common across all variants.

Bug: 155628860
Test: m nothing
Change-Id: Ifc8116e11d987cfe7aec2eeaa964f3bbf36b5dc2
2020-05-07 18:35:11 +01:00
Paul Duffin
4b8b79394f Allow extractCommonProperties to return an error
Refactoring in preparation for follow up changes.

Also:
* Adds a new AssertErrorMessageEquals() helper method.
* Improved error reporting in the accessor and added name to
  extractorProperty to ensure meaningful errors are reported.
* Added String() string method to propertiesContainer.
* Reports errors using the field name as the errors are not really
  fixable by developers and it is more meaningful to the build team.

Bug: 155628860
Test: m nothing
Change-Id: I5c5b8436bcbc39e4e7cd35df2577b2dac53e702a
2020-05-06 18:40:41 +01:00
Paul Duffin
1d6c0df597 Correct order of parameters to AssertDeepEquals
The expected/actual were around the wrong way.

Bug: 155628860
Test: m nothing
Change-Id: I98b575b8b85dcbfd2075e77689f0329aa2eadbf0
2020-05-06 18:40:41 +01:00
Paul Duffin
b28369a982 Encapsulate getter into an extractorProperty structure
Refactors the existing code to wrap a getter with a structure in order
to simplify adding additional per-property information. Adds an
emptyValue field to contain the empty value appropriate for the field
type.

Bug: 155628860
Test: m nothing
Change-Id: I7e169bd9b6e51b88e35bdf8673a5056337f778c1
2020-05-06 18:40:41 +01:00
Paul Duffin
f34f6d8538 Encapsulate properties to be optimized in a container
This allows additional metadata to be associated with a set of
properties that will be used in a follow up change to filter
properties that are ignored in some variants (e.g. host variants).

Bug: 155628860
Test: m nothing
Change-Id: Ie83c1ffbb71fd5d7a08d350571dad3e3209b1431
2020-05-06 18:40:41 +01:00
Paul Duffin
260bd316bf Fix definition of android_stubs_current in sdk tests
This change fixes an issue with the definition of the
android_stubs_current (and similar) modules in the sdk tests. They were
incorrectly defined as java_sdk_library_import and not java_import.

Test: m nothing
Bug: 155164730
Change-Id: I4e9bfebdd2ff0a374e12284ccdf5ee5518968969
2020-04-29 18:59:56 +01:00
Martin Stjernholm
cc330d665a Do not propagate dynamic dependencies for stub libs.
Test: m nothing
Test: Build an SDK snapshot including libc, check that the generated
  cc_prebuilt_library_shared for it is a stub that doesn't depend on
  ld-android.
Bug: 152255951
Bug: 154310674
Change-Id: Ie5ec02aebdf00b75756a6eda40db821488d30065
2020-04-27 18:43:40 +00:00