Commit graph

248 commits

Author SHA1 Message Date
Greg Kaiser
e08e03fa96 Revert "Adds support for 'ignored-on-host'"
Revert submission 1302576

Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...

Change-Id: I2a7ac0ef0232177eefc26542c11dc675d6f4cab2
2020-05-08 19:12:34 +00:00
Greg Kaiser
ad913d56ac Revert "Fix snapshot of a host/device cc_library with stubs"
Revert submission 1302576

Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...

Change-Id: Ibccce5286605bb71c6be3b3550ba86d8b7e24fa7
2020-05-08 19:12:34 +00:00
Paul Duffin
4911a89181 java_sdk_library: Specify visibility of stubs modules
Adds two new properties to specify the visibility of the stubs modules
(source and library) created by the java_sdk_library. Excludes
visibility property from being inherited when creating the module so it
can be properly specified by copying across the relevant property.

Test: m checkapi
Bug: 155164730
Change-Id: Iffdd9f191ff0d74646356ac577560cc38efdd790
2020-05-08 18:03:31 +01:00
Paul Duffin
8f265b9ab9 java_sdk_library: Add support for module_lib API surface
Adds api scope for the module_lib and adds that to the list of all
all available scopes. The scope is generated if and only if the
api_surfaces property contains "module_lib".

No other changes are needed as the generation of the APIs is completely
driven by the allApiScopes array and the information in its contained
apiScope structures.

Test: m checkapi
Bug: 155164730
Change-Id: I7769af6823badca8715a270f86cf53b4e954b7df
2020-05-08 07:51:00 +01:00
Paul Duffin
3375e35d99 java_sdk_library: Control API surfaces generated
Adds a per scope set of properties that allows explicit control over
the API surfaces generated.

Previously, the term active was used to determine whether it was
generated but that was a little abstract and unclear so has been
replaced by generated.

Test: m nothing
Bug: 155164730
Change-Id: I7539d89618b61f6b9d1a4b60cc3f9614b157f0d9
2020-05-08 07:51:00 +01:00
Paul Duffin
2af52380be Fix snapshot of a host/device cc_library with stubs
Adds a test that fails with unknown property android.stubs.versions
and then fixes that by marking the field from which that property is
created with 'ignored-on-host' and implemented the isHostVariant on
*osTypeSpecificInfo.

Bug: 155628860
Test: m nothing
Change-Id: I167b47a1374f541aa09d7e045972d740f1d9009c
2020-05-07 18:35:11 +01:00
Paul Duffin
12f67bcf42 Adds support for 'ignored-on-host'
Adds a filter mechanism that can exclude property values from being
included in the common value extraction. That is needed to prevent the
snapshot mechanism from generating invalid output for properties that
are ignored on host (and have their values cleared) and which are not
tagged with `android:"arch_variant"`.

Changes:
* Updates the documentation of SdkMemberType to explain what effect
  the 'ignored-on-host' tag has.
* Adds some tests for this new mechanism.

Bug: 155628860
Test: m nothing
Change-Id: I7ebd333079619dba546bc8c4911d567e0287b676
2020-05-07 18:35:11 +01:00
Paul Duffin
ee20962378 Detect invalid arch specific properties in snapshot
Previously, the snapshot code did not know whether a specific property
could be arch specific or not and assumed that they all were which
meant that it could generate snapshots containing arch specific values
for properties that are not arch specific and so would fail when
unpacked.

This change requires arch specific fields in SdkMemberProperties to be
tagged as such using `android:"arch_variant"` (just as in module input
property structures). Any property without that must have properties
that are common across all variants.

Bug: 155628860
Test: m nothing
Change-Id: Ifc8116e11d987cfe7aec2eeaa964f3bbf36b5dc2
2020-05-07 18:35:11 +01:00
Paul Duffin
4b8b79394f Allow extractCommonProperties to return an error
Refactoring in preparation for follow up changes.

Also:
* Adds a new AssertErrorMessageEquals() helper method.
* Improved error reporting in the accessor and added name to
  extractorProperty to ensure meaningful errors are reported.
* Added String() string method to propertiesContainer.
* Reports errors using the field name as the errors are not really
  fixable by developers and it is more meaningful to the build team.

Bug: 155628860
Test: m nothing
Change-Id: I5c5b8436bcbc39e4e7cd35df2577b2dac53e702a
2020-05-06 18:40:41 +01:00
Paul Duffin
1d6c0df597 Correct order of parameters to AssertDeepEquals
The expected/actual were around the wrong way.

Bug: 155628860
Test: m nothing
Change-Id: I98b575b8b85dcbfd2075e77689f0329aa2eadbf0
2020-05-06 18:40:41 +01:00
Paul Duffin
b28369a982 Encapsulate getter into an extractorProperty structure
Refactors the existing code to wrap a getter with a structure in order
to simplify adding additional per-property information. Adds an
emptyValue field to contain the empty value appropriate for the field
type.

Bug: 155628860
Test: m nothing
Change-Id: I7e169bd9b6e51b88e35bdf8673a5056337f778c1
2020-05-06 18:40:41 +01:00
Paul Duffin
f34f6d8538 Encapsulate properties to be optimized in a container
This allows additional metadata to be associated with a set of
properties that will be used in a follow up change to filter
properties that are ignored in some variants (e.g. host variants).

Bug: 155628860
Test: m nothing
Change-Id: Ie83c1ffbb71fd5d7a08d350571dad3e3209b1431
2020-05-06 18:40:41 +01:00
Paul Duffin
260bd316bf Fix definition of android_stubs_current in sdk tests
This change fixes an issue with the definition of the
android_stubs_current (and similar) modules in the sdk tests. They were
incorrectly defined as java_sdk_library_import and not java_import.

Test: m nothing
Bug: 155164730
Change-Id: I4e9bfebdd2ff0a374e12284ccdf5ee5518968969
2020-04-29 18:59:56 +01:00
Martin Stjernholm
cc330d665a Do not propagate dynamic dependencies for stub libs.
Test: m nothing
Test: Build an SDK snapshot including libc, check that the generated
  cc_prebuilt_library_shared for it is a stub that doesn't depend on
  ld-android.
Bug: 152255951
Bug: 154310674
Change-Id: Ie5ec02aebdf00b75756a6eda40db821488d30065
2020-04-27 18:43:40 +00:00
Paul Duffin
1fd005d5b3 Export API files as part of the java_sdk_library
Bug: 153443117
Test: m nothing

Change-Id: I9d6f5b91a7cc25019e2eb9e3c138f0874d2831de
2020-04-09 01:33:46 +01:00
Paul Duffin
3d1248ceb6 Export stub sources as part of the java_sdk_library
Minor refactoring of checkMergeZip(string) -> checkMergeZips(...string)
to allow testing of multiple merge zips.

Bug: 153443117
Test: m nothing
Change-Id: I8db00f611ced15f8476ba16f2834a72e8c913596
2020-04-09 01:32:38 +01:00
Paul Duffin
dd46f71493 Support java_sdk_library as member of sdk
Bug: 153443117
Test: m nothing
Change-Id: I9d8089b2555038e3f10ad5939a6a7b01839c67ea
2020-04-09 01:32:38 +01:00
Paul Duffin
6534770793 Stop requiring apex_available on java_library members of sdks
Previously, adding java_library to an sdk required that the names of
any APEXes that transitively compiled against it were added to its
apex_available property. This change removes that requirement.

Also corrects the dependency path in the TestApexAvailable_IndirectDep
error which previously passed through "shared from static" static
dependency tags even though those are explicitly NOT followed when
checking apex_available settings.

Bug: 152878661
Test: m droid
Change-Id: I995ed38956c1bc210b09494812de012fed9f9232
2020-04-08 08:40:44 +01:00
Paul Duffin
2010598a51 Merge "Extract DepIsInSameApex and RequiredSdks interfaces" 2020-04-08 06:55:03 +00:00
Paul Duffin
923e8a5e9e Extract DepIsInSameApex and RequiredSdks interfaces
The DepIsInSameApex() and RequiredSdks() methods were defined in a few
places to avoid having to depend on the whole ApexModule/SdkAware
interfaces directly. However, that has a couple of issues:
1) It duplicates functionality making it difficult to change, changes
   to the definitions outside the main interfaces do not cause compile
   time failures, instead they result in a runtime change in behavior
   which can be difficult to debug.
2) IDE navigation (specifically in Intellij) does not detect that the
   duplicate definitions can resolve to the definitions in the main
   interface.

This change extracts the methods into their own interfaces and reuses
those interfaces instead of duplicating the methods to fix both of
these issues.

Bug: 152878661
Test: m nothing
Change-Id: I0cfdf342a14eb0bfb82b1bd17e0633d81c7facfb
2020-04-07 15:26:06 +01:00
Martin Stjernholm
bdd0dffc1b Do not propagate the stub symbol file to prebuilts.
This partially reverts https://r.android.com/1278193. The propagated
symbol files contain all versions, even though only one is applicable
in a given SDK snapshot.

It's uncertain what repercussions this might have, but one is that if
we were to update a snapshot for a fixed version then it might change
because the symbol file contains new versions that aren't applicable.
Since the symbol file isn't actually needed at this point it's better
to wait with this step until the use cases for it are more clear.

Test: m nothing
Test: Create an SDK snapshot with Bionic libs, drop it into a
  master-art tree without bionic/ in it, build ART APEXes, and check
  that the Soong phase completes (specifically that the stubs are
  detected even without symbol files).
Bug: 152481980
Change-Id: Ic79f89bc6d11d0b6552fa20791f5680ff9a40c0d
2020-04-06 23:03:53 +01:00
Martin Stjernholm
c5dd4f7c1f Propagate stubs to the SDK for libraries that have them.
Necessary to make the APEX build logic treat the libraries as API
boundaries rather than dependencies to bundle.

The .so files in the snapshots are the compiled stub libraries in this
case. They are strictly speaking redundant since they can be generated
from the .map.txt files in the snapshots, but doing that would require
extending the cc_prebuilt_library(_shared) module types with a full
compiler pass etc, and that would break a lot of assumptions in the cc
package.

Test: m nothing
Test: Create an SDK snapshot with Bionic libs, drop it into a
  master-art tree without bionic/ in it, build ART APEXes, and check
  that the Soong phase completes (specifically no errors about various
  APEX libs requiring libc that is not available to them).
Bug: 152481980
Change-Id: I31b928e6261198b6dd6f6b17196e714f07b64172
2020-04-02 22:39:12 +01:00
Martin Stjernholm
66a06945aa Don't run new test on Darwin that lacks snapshot support.
Test: m nothing
Bug: 152514829
Bug: 152255951
Change-Id: Ib14af7caa408e252242e6933f7281aab3888789c
2020-03-26 17:39:30 +00:00
Martin Stjernholm
10566a035f Propagate empty vs unspecified system_shared_libs correctly.
Necessary to get correct prebuilts for many Bionic libs.

Cleaned up numerious "system_shared_libs: []" from test fixtures, since
they otherwise would need correction in the expected results, and it is
better to have a single test focused on testing system_shared_libs
propagation.

Test: m nothing
Bug: 152255951
Change-Id: If2e8a5296223e6281d833312660e8e9e4cd184c0
2020-03-25 23:19:37 +00:00
Paul Duffin
7b5f1a616b Use reflect.Zero(type) to get value to clear field
Previously, the common value extraction code used an empty structure
to get the value to use to clear a field whose value is common. This
change removed the structure and used reflect.Zero(..) to get the
value instead.

Bug: 142935992
Test: m nothing
Change-Id: Ibd5103dacb86e7754a786356c0d15ffbde7f98bf
2020-03-23 08:48:03 +00:00
Paul Duffin
6a7e953e62 Sdk snapshot set compile_multilib per OsType
Previously, when an sdk snapshot only supported a single os type the
compile_multilib was set based on the multilib usages by the members
of that variant. After the change to support multiple os types per
snapshot the multilib setting was based on the multilib usages across
all the members of all sdk variants. That meant that if one os type
used only "64" and the other used "both" then they would both be
treated as "both" leading to missing variants when the snapshot was
unpacked.

This change tracks the multilib usages per os type and adds a property
for each one.

It intentionally changes a couple of tests:
1) Either by adding compile_multilib that is missing.
2) By targeting it at a specific os type instead of host.

The latter change is important to prevent the snapshot from being
used on a host platform (which will match the host target section)
but which is a different os type to the ones supported by the
snapshot.

Bug: 142935992
Test: m nothing
Change-Id: I883919b644292c3d019db223bb4fd5c11b39591f
2020-03-23 08:48:03 +00:00
Paul Duffin
495ffb9ff9 Remove old SdkMemberType API for creating snapshot modules
Migrates system modules and droid stubs over to use the new API for
creating the snapshot modules and removes the old API.

Test: m nothing
Change-Id: Ia825767f1f7ee77f68cfe00f53e09e6f6bfa027f
2020-03-23 08:48:03 +00:00
Paul Duffin
a551a1c2f9 Improve consistency of handling java snapshot properties
Previously, java snapshot properties (java_library and java_test)
relied on the properties not being optimized when there was a single os
type and instead being added directly to the common os type properties.
However, that means that the behavior is inconsistent for other member
types depending on whether there was one os type or not.

This change updates the java sdk member handling to support
optimization. This involved:
1) Adding AidlIncludeDirs field to librarySdkMemberProperties to
   specify the aidl include dirs instead of extracting that from the
   library field.

2) Renaming jarToExport to JarToExport (in both
   library/testSdkMemberProperties)to allow it to be optimized.

3) Adding MemberType() and Name() methods to SdkMemberPropertiesContext
   to avoid having to store the former in the properties struct and
   retrieve the latter from the library/test fields.

4) Removing the now unused library/test fields from the properties
   structures.

5) Separating the processing of the jar/test config in
   AddToPropertySet(...) as they may be optimized separately.

6) Ditto for the jar/aidl include dirs.

7) While doing this work I noticed that although the contents of the
   aidl include dirs are copied into the snapshot the java_import does
   not make use of them. Raised bug 151933053 and added TODO to track
   that work.

Bug: 142935992
Test: m nothing
Change-Id: Iba9799e111ca5672b2133568163d8c49837ba9cd
2020-03-23 08:48:03 +00:00
Paul Duffin
3a4eb50829 Make new module creation API more flexible
Previously passing additional information to the implementations of
AddPrebuiltModule() or the SdkMemberProperties interface would have
required making changes to the API. This change added an
SdkMemberContext object into which additional information can easily
be added without requiring changes to existing implementations.

The BuildSnapshot() method was not modified because it is deprecated
and will be removed in a follow up change.

It also switches the API from passing variants as android.SdkAware to
android.Module. That is for a couple of reasons:
1) SdkAware is designed for managing the relationship between the
   module and the SDK, not for generating the output snapshot. As such
   there is nothing in SdkAware that is needed for generating the
   output snapshot.
2) Accepting android.Module instead makes it easier to use the
   underlying code for generating the snapshot module as well as the
   individual member modules.

This is in preparation for a number of improvements and bug fixes in
both the snapshot creation code and implementations to address found
while trying to built the platform against ART prebuilts.

Bug: 151937654
Test: m nothing
Change-Id: Iac10f1200c0f283aa35402167eec8f9aeb65a38e
2020-03-23 08:47:38 +00:00
Paul Duffin
9c3760eb14 Add abstraction for tracking compile multilib usages
Ensures consistent tracking of multilib usages.

Bug: 142935992
Test: m nothing
Change-Id: I82ba4a53a21a0823503cb98f7edfde449b48b440
2020-03-20 16:55:24 +00:00
Paul Duffin
9b76c0b65a Add support for using cc_library in sdk/module_exports
Added link type support in module creation code as cc_prebuilt_library
has to provide both shared and static libraries.

Had to add some calls to FirstUniquePaths() around the include dirs
settings in library_sdk_member as otherwise the shared variant contains
duplicate include dirs (presumably because it gets one set from its
dependency on the static variant and one set of its own). That
difference in include dirs causes problems in the generated
cc_prebuilt_library.

Test: m nothing
Bug: 142918168
Change-Id: Ie7f23fc2341c83c7814cc98e3970df4f5d4c8423
2020-03-20 16:55:24 +00:00
Martin Stjernholm
cd07bce437 Add SDK member support for cc_object.
Test: m nothing
Test: Add
    sdk {
        name: "runtime-module-sdk",
        native_shared_libs: [
            "libc",
            "libdl",
            "libm",
            "ld-android",
        ],
        native_objects: [
            "crtbegin_dynamic",
            "crtbegin_static",
            "crtend_android",
        ],
    }
  to bionic/apex/Android.bp. Then:
    build/soong/scripts/build-aml-prebuilts.sh runtime-module-sdk
  Take the generated runtime-module-sdk-current.zip and unzip into a
  master-art tree without bionic/, edit the generated Android.bp to
  extend cc_prebuilt_* modules with:
    nocrt: true,
    stl: "none",
    system_shared_libs: [],
    apex_available: ["//apex_available:anyapex"],
    recovery_available: true,
    vendor_available: true,
    ramdisk_available: true,
  Then "m com.android.art.debug". This passes Soong but fails in the
  build step because more members are required.
Bug: 148934017
Change-Id: I2ab8f6aadb1440b325697cae4a8ed761c62d15d2
2020-03-20 00:16:14 +00:00
Paul Duffin
00e4680d4d Extract the osTypeSpecificInfo code from module creation loop
Extract the functionality to create an osTypeSpecificInfo struct,
to optimize the properties, and add its properties to a property set
into methods of the *osTypeSpecificInfo struct.

This change is in preparation for adding support for link type which
is another dimension within arch type which itself sits within os type.

Test: m nothing
Bug: 142918168
Change-Id: I025ee90e1461f7389bf4a9d056b281453068cf87
2020-03-18 11:18:38 +00:00
Paul Duffin
fc8dd239ec Extract archTypeSpecificInfo code from module creation loop
Extract the functionality to create an archTypeSpecificInfo struct and
to add its properties to a property set into methods of the
*archTypeSpecificInfo struct.

Test: m nothing
Bug: 142918168
Change-Id: I2a9e0327b61bce7ad7699cd75de17aa0e5f1ebbb
2020-03-18 11:17:24 +00:00
Paul Duffin
b44b33a621 Clean up the main module creation loop
This change reorganizes the code within the main module creation loop
in preparation for delegating the work to separate types. It has been
split out into its own change to make it easier to review by keeping
the functional changes localized.

Renames the osTypeSpecificInfo archTypes field to archInfos as the
latter is more accurate.

Cleans up the arch variants handling:
1) Groups them by arch type to make them easier to process.
2) Fails fast when there is not exactly one variant per arch type as
   otherwise it results in a confusing failure later on.
3) Removes the commonArch flags and instead uses the fact that
   osInfo.archInfos is empty when the common architecture variant is
   available.

Cleans up the arch type specific property set handling.
1) Adds new archPropertySet variable to allow the choice of where the
   arch specific properties are added to be made alongside the choice
   of where the os specific properties are to be added.
2) Removes unnecessary check for commonArch around the loop to add
   properties from archInfos as the archInfos will be empty when the
   common architecture is present.

A number of other changes to make it easier to extract the code into
their own methods.

Test: m nothing
Bug: 142918168
Change-Id: I16a5fa79efff0d08c22916449eb46a5bd910723a
2020-03-18 11:02:24 +00:00
Paul Duffin
843f09723b Merge "Output properties before sets in snapshot module" 2020-03-16 11:04:33 +00:00
Paul Duffin
2c73260636 Merge "Remove SdkMemberType.FinalizeModule" 2020-03-16 11:04:26 +00:00
Paul Duffin
07ef3cb1fd Output properties before sets in snapshot module
This ensures a consistent output irrespective of whether property sets
are created before or after the properties are added. This provides a
little more flexibility in the creation code which allows that to be
simplfied.

Also switches from using reflection to a type switch.

Bug: 142918168
Test: m nothing
Change-Id: Ia025bfc751f1217d1658de6fb8e15091ea0ea9ff
2020-03-13 17:25:23 +00:00
Paul Duffin
0174d8d2c9 Remove SdkMemberType.FinalizeModule
This was only being used to set the "stl" property for cc library sdk
member type and so that functionality was moved to AddPrebuiltModule()
and FinalizeModule was removed.

Required a few test changes to move the property to the correct
position in the generated module.

Bug: 142918168
Test: m nothing
Change-Id: If6400189833d4ff3285e7a7adf63a9b509e2a03b
2020-03-13 17:25:16 +00:00
Paul Duffin
7f7bb3f46d Follow up a review comment that was missed
Bug: 142935992
Test: m nothing
Change-Id: I45b3b7dd3bde3042d67f647c188d61070864f3cc
2020-03-13 14:20:42 +00:00
Paul Duffin
13f0271478 Copy shared_libs and system_shared_libs to module snapshot
This change ensures that the runtime dependencies between a
binary/shared library are correctly specified in the snapshot so that
the build can ensure that shared libraries are built before the targets
that use them.

It adds support for differentiating between references that are
required to refer to another sdk member (required) and those that may
refer to either an sdk member or a non-sdk member (optional). The
latter is used for shared library references as the libraries used by
an sdk member may be provided from outside the sdk. e.g. liblog is not
part of the ART module but is used by some members of the ART sdk.

Bug: 142935992
Test: m nothing
Change-Id: Ia8509ffe79b208c23beba1880fe9c8a92b732685
2020-03-13 11:14:07 +00:00
Paul Duffin
b07fa51196 Support extracting common values from embedded structures
This change also added support for excluding properties from common
value extraction by using a struct tag of `sdk:"keep"` That was needed
to prevent the fields in SdkMemberPropertiesBase from having their
values cleared.

The purpose of this change is to make it easier to share functionality
across sdk member types.

Bug: 142935992
Test: m nothing
Change-Id: Ie5160a8f854056920e411801ca20721eab7c8578
2020-03-13 10:47:14 +00:00
Paul Duffin
c097e36172 Refactor common value extraction
This change pulled the common value extraction functionality out into
its own structure that can be used to extract common values from
multiple sets of properties.

Precursor to follow up changes that will allow properties to contain
embedded structures making it easier to share functionality across
sdk member types.

Bug: 142935992
Test: m nothing
Change-Id: Ic902ed61e40ced7a2c2fa4b5f793f532c6fc0034
2020-03-13 08:46:54 +00:00
Paul Duffin
7d74e7bea3 Copy white listed apex available settings into snapshot
Makes sure that the module snapshots do not rely on the white list
of apex available settings so that when those lists are removed it is
not necessary to update any snapshots.

Bug: 142935992
Test: m nothing
Change-Id: Iedcff7dfc2646a4da77258d16e06657dd2f411f9
2020-03-11 18:31:45 +00:00
Paul Duffin
0cb37b9ce1 Disable installation for sdk snapshot versioned prebuilts
The sdk snapshot creates two prebuilts for each member one that is
versioned and one that is not. If they are both installed then they
lead to duplicate rules in make for creating the same installed file.

This change adds an installable property to cc modules that will
prevent the installation of the file and then adds installable: false
on the versioned prebuilt for cc modules.

Bug: 142935992
Test: m nothing
Change-Id: I4cb294c2b0c8a3f411eea569775835d9e41726d6
2020-03-11 18:31:45 +00:00
Jooyung Han
4c8e3509ee Merge "Fix apex_available" 2020-03-10 23:53:41 +00:00
Jooyung Han
5e9013be22 Fix apex_available
Checking apex_available was missing some corner cases.
For example, the deps of share deps of cc_library modules are missed
while those from cc_library_shared are correctly tracked.

This was due to..

* calling DepIsInSameApex in WalkDeps: both work fine separately, but
when they are used together, it fails to work. It's due to how WalkDeps
works. (We might fix this bug too risky since it is used very widely)
* incorrect receiver for DepIsInSameApex in apex_deps mutator: receiver
is supposed to be parent, but child was used before. Interestingly lots
of deps are within the same group of module types(cc to cc, java to
java), it has worked. (note that receiver's DepIsInSameApex
implementation can be different).

This change fixes them by..

* walkPayloadDeps is now relying on ApexVariation, which is calculated
correctly by TopDown apex_deps mutator.
* use correct receiver for DepIsInSameApex in apex_deps mutator, which
requires for java.SdkLibrary to override the method and for
java.Library/Import to use passed dep instead of receiver to check its
membership of sdk.

Bug: 151071238
Test: build/boot
Change-Id: I0569ef4bb8e79635e4d97a89f421a8d8b7d26456
2020-03-10 23:52:01 +00:00
Paul Duffin
865171ed40 Allow sdk members to vary by os type
Adds support for specifying separate members to an sdk/module_exports
for different os types, e.g. separate ones for android and host.

Adds 'android:"arch_variant"' tag to the dynamically generated fields
for the sdk member types.

Merges the exported members from all variants together.

Determines the device/host_supported flags of the member snapshots by
whether the OsClasses used by their variants rather than the sdk's
host/device supported properties as they may be different.

Bug: 150451422
Test: m nothing
Change-Id: I41fbbcd8723aafd54826aad9b78eced9f66ef51c
2020-03-10 10:39:24 +00:00
Paul Duffin
a04c107bfa Add support for multiple os types
Updates the member snapshot creation code to support multiple os types.
It basically sorts the variants by os type, then applies the code to
optimize the arch properties and then it optimizes the properties that
are common across architectures and extracts any properties that are
common across os types.

The java and cc member types needed to be modified to make the location
of the generated files within the snapshot os type dependent when there
is more than one os type. That was done by adding an OsPrefix() method
to the SdkMemberPropertiesBase which returns the os prefix to use when
there is > 1 os type and otherwise returns an empty string.

Added three tests, one for cc shared libraries, one for cc binary and
one for java header libraries.

Bug: 150451422
Test: m nothing
Change-Id: I08f5fbdd7852b06c9a9a2f1cfdc364338a3d5bac
2020-03-09 15:54:31 +00:00
Paul Duffin
71754bf640 Merge "Copy sdk_version to cc library snapshots" 2020-03-09 14:15:54 +00:00
Treehugger Robot
a377311b90 Merge "Refactor java_library/java_test snapshot processing" 2020-03-09 13:14:32 +00:00
Paul Duffin
0c394f3919 Copy sdk_version to cc library snapshots
Bug: 142935992
Test: m nothing
Change-Id: Ib77c548b1971c841749c87115108af8b092bb809
2020-03-09 11:10:04 +00:00
Paul Duffin
b3c763711a Merge "Refactor snapshot module creation" 2020-03-09 11:02:29 +00:00
Paul Duffin
8c3fec4c37 Enable androidmk processing in sdk testing
Previously, while sdk tests would pass the sdk code would often fail
in androidmk processing. This change makes the tests more realistic
and will catch the errors earlier.

Bug: 142935992
Test: m nothing
Change-Id: Ifd0b2d7cf24e941c919f6b6e0beb2403a67d4308
2020-03-06 09:45:54 +00:00
Paul Duffin
14eb4679a6 Refactor java_library/java_test snapshot processing
Adds support for handling the common arch to the sdk module type
snapshot generation code and then refactors the java_library
and java_test snapshot processing to take advantage of that.

Bug: 150451422
Test: m nothing
Change-Id: If746f375f1c4288ab6b67c7d216593b70258b043
2020-03-05 13:00:26 +00:00
Paul Duffin
88f2fbe92e Refactor snapshot module creation
Generalize the processing of arch specific properties to reduce
duplication in snapshot module creation and simplify addition of
support for handling multiple os types.

Supporting multiple os types with the current method for building
snapshot modules would require every affected module type to add
support for it. Rather than duplicate multiple os type handling code
across those module types this work generalizes the process cc modules
use for handling arch types as it can be used as a basis for handling
multiple os types. Migrating module types over to this new process
will insulate them from having to handle multiple os types.
OB
SdkMemberType changes:
* BuildSnapshot is deprecated in favour of the new AddPrebuiltModule()
  method.
* Additional methods, CreateVariantPropertiesStruct() and
  FinalizeModule() are added.
* A new interface SdkMemberProperties, is defined that handles
  extracting information from the variant (prior to common value
  optimization) and adding properties to a property set.

The sdk module type uses these new methods and types to delegate the
member type specific processing to the relevant member types while
handling the behavior that is common across all members types, e.g.
extracting common values across multiple architectures. A future change
will leverage this processing to add support for multiple os types.

This change also refactors the cc module processing to use the new
process.

Bug: 150451422
Test: m nothing
Change-Id: If6ab2498407b17f50391d062cd9afc01b5e01af4
2020-03-05 13:00:26 +00:00
Nicolas Geoffray
8f20563514 Merge "Add a nice install paths for module SDKs and exports." 2020-03-05 09:25:33 +00:00
Paul Duffin
befa4b91d7 Copy apex_available properties to snapshot modules
Bug: 142935992
Test: m nothing
Change-Id: I1cdaae5b4e13a89dc46541dacd34ef5a44735b6a
2020-03-05 08:29:02 +00:00
Nicolas Geoffray
1228e9c0ba Add a nice install paths for module SDKs and exports.
Following how NDK also creates its own install path.

Bug: 142935992
Test: cc_sdk_test.go, java_sdk_test.go
Change-Id: I98a3656903f37f6d7c90e6cf609431b2461a6161
2020-03-04 14:20:46 +00:00
Paul Duffin
e1ddcc9d5a Only check copy rules into the snapshot directory
Test: m nothing
Change-Id: I6d9ab5a9f003193c5791a4ee6d3ee35730ddf96c
2020-03-03 16:02:26 +00:00
Paul Duffin
1356d8c0f3 Add CommonOS variant for sdk
Adds a CommonOS variant for sdk that depends on the os specific
variants and is used to generate a single sdk for multiple OsTypes,
e.g. host linux and android.

At the minute the member types only support a single OsType but the
basic mechanism for managing the CommonOS variant and collating the
variants across all of them is there.

The only visible effect of this change is that the location of the
generated snapshot is changed, it is no longer os specific and instead
is in the same location irrespective of which os it is built for.

A lot of tests needed to be changed to specify "common_os" as the
variant type instead of the specific os type. As that is the same across
all tests it is hard coded in CheckSnapshot method.

Test: m nothing
Bug: 150451422
Change-Id: If36be39b06d6910453649f7c288c2d34f688b2f4
2020-03-02 19:31:26 +00:00
Paul Duffin
109c2add59 Fix bug in error reporting when adding duplicate properties
Bug: 150451422
Test: m nothing
Change-Id: Ia557992d71537aca0757866e9114b30bdf52bb6b
2020-03-02 16:29:11 +00:00
Paul Duffin
cc1b3da17a Avoid invoking sdk member to add empty list of dependencies
Simplifies debugging sdk membership code when it is only called if an
appropriate member is present.

Test: m nothing
Change-Id: I32bc93e0484c829bdbd6b050822249feb6404f89
2020-02-28 11:21:06 +00:00
Paul Duffin
08798aae0b Add windows to the list of available OS's in sdk tests
Makes the tests more realistic as they cover what happens when an sdk
OS specific variant is disabled (Windows is disabled by default). This
will allow issues around disabled variants to be detected early during
Soong bootstrapping.

Test: m nothing
Change-Id: I7ec47dbca52e38750166d755daf706aadea12d15
2020-02-28 11:21:06 +00:00
Paul Duffin
91756d2498 Add support for cc_library_headers in sdk/module_exports
Bug: 148933848
Test: m nothing
Change-Id: Ife6ee0f736238727a11b4421532eaeb29d46c1b7
2020-02-24 14:49:22 +00:00
Paul Duffin
a78f3a7360 Prune any empty property sets from the modules before transforming
Ensures that the generated prebuilt modules do not contain empty
property sets.

Bug: 148933848
Test: m nothing
Change-Id: I99d5da1c4e3a72836e81e021ef043df53888e66b
2020-02-24 14:49:22 +00:00
Paul Duffin
180a006a3b Add support for transforming a property set after its contents
This change adds support for transforming a property set after its
contents. This allows a transform to recursively prune empty property
sets that were created for a module.

The transformPropertySet method was renamed to
transformPropertySetBeforeContents and a new
transformPropertySetAfterContents method was added.

Bug: 148933848
Test: m nothing
Change-Id: Ia198d47e042b98c69406db4bc12859869816a387
2020-02-24 14:49:22 +00:00
Paul Duffin
047fdcac0f Fix issues with bp transformation
Returning nil from transformPropertySet in order to remove it did
not work because it ends up comparing as (*bpPropertySet, nil) and
not an untyped nil which causes the test against nil to fail.

This change adds tests to check that returning nil will delete a
property/property set from the containing property set and fixes the
code so that it passes the tests. It extracts common code to transform
a property set and its contents as well as code for creating new
property sets.

Bug: 148933848
Test: m nothing
Change-Id: I35dc3c39c76e701821891622615c09b094cf697f
2020-02-24 14:49:22 +00:00
Paul Duffin
13ad94fce2 Allow compile_multilib to be specified on module exports
This is needed to allow the art-host-module-exports to restrict itself
to just managing the linux 64 bit version of the host tools as that is
the only variant that is currently supported by all host tools. This
greatly simplifies that process and allows us to make progress on the
unbundling.

Bug: 142935992
Test: m nothing
Change-Id: I62d016d97c2df73e5feecf912638f477fedd97c9
2020-02-21 09:01:08 +00:00
Paul Duffin
583bf7ebf7 Do not add dependencies from disabled sdk variants to its members
If an sdk variant is disabled (e.g. say windows) then it should not
add dependencies on corresponding sdk member variants as if those
variants have not been created then the build breaks unnecessarily.

This is needed to make progress on unbundling art which provides at
least one host tool (dex2oat) that is not supported on all platforms
and some (hiddenapi) that only produce 64 bit versions.
Initially, the art prebuilts will only support building 64 bit
versions of linux host tools.

Bug: 142935992
Test: m art-module-sdk art-module-host-exports art-module-test-exports
Change-Id: Idfbb40bb3cabc6eb731a2b38f045ed14f0a713cc
2020-02-20 21:51:52 +00:00
Treehugger Robot
53fe467c67 Merge changes from topic "soong_tests_ndk"
* changes:
  Make apex use cc.GatherRequiredDepsForTests
  Move NDK test modules into cc/testing.go
2020-02-20 20:39:16 +00:00
Paul Duffin
44885e29d6 Simplify visibility rules that include //visibility:public
While it is invalid to mix //visibility:public with other rules in the
visibility property in a .bp file tt was possible, by overriding
defaults, to have //visibility:public mixed in with other rules in the
effective visibility rules. That caused problems when those effective
rules were used in an sdk snapshot.

This change replaces any set of rules that include //visibility:public
with just the //visibility:public rule. That simplifies those rules,
making them cheaper to process and ensures that the effective rules are
valid in the visibility property.

Adding test support required some refactoring of the
effectiveVisibilityRules(BaseModuleContext, ...) and underlying methods
to take a Config instead of BaseModuleContext as the tests do not have
access to BaseModuleContext.

Bug: 142935992
Test: m nothing - new tests failed without change, work with it
      Add dex2oat to art-module-host-exports, build it and check the
      generated Android.bp file in the snapshot to ensure the
	  visibility property for the dex2oat prebuilt does not mix
	  //visibility:public with other rules.
Change-Id: I08e7f0dcb40838d426fe88fedf69eae27b77473c
2020-02-20 11:41:17 +00:00
Colin Cross
f9aabd719a Make apex use cc.GatherRequiredDepsForTests
Remove the duplicated native modules from apex_test.go.

Test: all soong tests
Change-Id: Ib88af058d23cf37446d7a4bb571edfb1e0880854
2020-02-19 19:07:11 -08:00
Colin Cross
f28329de73 Move NDK test modules into cc/testing.go
Move the NDK modules defined in app_test.go into cc/testing.go in
preparation for using sdk_version: "current" in more tests.

Test: all soong tests
Change-Id: I80bc31225fb4562fce42781219cefbbd6affae06
2020-02-19 18:59:15 -08:00
Paul Duffin
25ce04bd3a Add support for cc_binary in module_exports
Bug: 142920869
Test: m nothing
Change-Id: Ib7e16cb323806782c2186e73e13642e4485fc149
2020-02-18 16:17:35 +00:00
Paul Duffin
7291095d82 Differentiate between exported and internal sdk members
Internal sdk members are used by an sdk member but not exported by the
sdk. The exported sdk members are those listed explicitly in one of the
sdk member list properties, e.g. java_header_libs.

The prebuilts of an internal sdk member use a unique name so that they
do not clash with the source module. The use of the module internally
is an implementation detail that must not have any effect outside the
snapshot. Having the same name as the source module could cause it to
override the source module, hence why it needs a unique name.

Similarly, they are marked as private so as to prevent their accidental
use from outside the snapshot.

Bug: 142940300
Test: m nothing
Change-Id: Id5364b410be0592f65666afb3e40e9d3f020251c
2020-02-07 14:03:03 +00:00
Paul Duffin
7b81f5e9d7 Add java_system_modules to sdk/module_exports
Adds an SdkMemberType implementation for java_system_modules. It
specifies that java_system_modules can be used with sdk as well as
module_exports, and also that the libs property should be included
as transitive members in the sdk.

It also adds support for treating appropriate tagged properties in
the snapshot prebuilts module as references to sdk members so that
they are correctly transformed when creating the versioned modules.

Bug: 142940300
Test: m nothing
Change-Id: Ic10b5a6d5b92b6018334fe876f06feaf79cc55e9
2020-02-07 14:03:03 +00:00
Paul Duffin
f4ae4f1390 Add support for transitive sdk members
Allow an sdk member type to treat some of its dependencies as being
members of the sdk.

Needed for the java_system_modules type whose libs property are an
implementation detail of the system module and so should not be
explicitly listed in the sdk module but still have to be included in
the sdk snapshot.

Bug: 142940300
Test: m nothing
Change-Id: I90f37dae269ef64a6fe9debd0bbaf29a64dd74d8
2020-02-07 14:03:03 +00:00
Jiyong Park
0f80c1848a apex_available tracks static dependencies
This change fixes a bug that apex_available is not enforced for static
dependencies. For example, a module with 'apex_available:
["//apex_available:platform"]' was able to be statically linked to any
APEX. This was happening because the check was done on the modules that
are actually installed to an APEX. Static dependencies of the modules
were not counted as they are not installed to the APEX as files.

Fixing this bug by doing the check by traversing the tree in the method
checkApexAvailability.

This change includes a few number of related changes:

1) DepIsInSameApex implementation for cc.Module was changed as well.
Previuosly, it returned false only when the dependency is actually a
stub variant of a lib. Now, it returns false when the dependency has one
or more stub variants. To understand why, we need to recall that when
there is a dependency to a lib having stubs, we actually create two
dependencies: to the non-stub variant and to the stub variant during the
DepsMutator phase. And later in the build action generation phase, we
choose one of them depending on the context. Also recall that an APEX
variant is created only when DepIsInSameApex returns true. Given these,
with the previous implementatin of DepIsInSameApex, we did create apex
variants of the non-stub variant of the dependency, while not creating
the apex variant for the stub variant. This is not right; we needlessly
created the apex variant. The extra apex variant has caused no harm so
far, but since the apex_available check became more correct, it actually
breaks the build. To fix the issue, we stop creating the APEX variant
both for non-stub and stub variants.

2) platform variant is created regardless of the apex_available value.
This is required for the case when a library X that provides stub is in
an APEX A and is configured to be available only for A. In that case,
libs in other APEX can't use the stub library since the stub library is
mutated only for apex A. By creating the platform variant for the stub
library, it can be used from outside as the default dependency variation
is set to the platform variant when creating the APEX variations.

3) The ApexAvailableWhitelist is added with the dependencies that were
revealed with this change.

Exempt-From-Owner-Approval: cherry-pick from internal

Bug: 147671264
Test: m

Merged-In: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e
(cherry picked from commit fa89944c79)
Change-Id: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e
2020-02-06 14:45:08 +09:00
Paul Duffin
f8539922d4 Make sdkMemberDependencyTag usable outside sdk package
Moves the struct to android/sdk.go and abstracts it behind a factory
method and interface. That allows it to be used outside the sdk
package.

This change is in preparation for adding support for module types that
have transitive sdk members.

Bug: 142940300
Test: m nothing
Change-Id: I71e5e0adf839b28a3a0952f82637637887f02688
2020-01-30 11:45:47 +00:00
Paul Duffin
5b511a200e Allow an arbitrary tag to be associated with a bp property
This is preparation for enhancing the versioning tranformer to support
applying per property transformations. Specifically, to allow
properties to reference other libraries within the sdk.

Bug: 142940300
Test: m nothing
Change-Id: I99cdff4b407763ed395ff358d8110a63c6cf5589
2020-01-30 11:45:47 +00:00
Paul Duffin
e6c0d845fd Rewrite code for creating versioned modules as a transformation
Moves the code for transforming an unversioned module into a
versioned module into a transformer.

This is preparation for enhancing the versioning tranformer to support
applying per property transformations. Specifically, to allow
properties to reference other libraries within the sdk.

Bug: 142940300
Test: m nothing
Change-Id: Ia6b5cff8e1b5cd7232e031769d9fc0019e815fcb
2020-01-30 11:45:47 +00:00
Paul Duffin
b4d9c1f0ef Add support for transforming a module
Adds support for transforming a bpModule using a bpTransformer
instance. Rewrites the deepCopy code as a transformer.

This is preparation for allowing the update process to perform
per property transformations when transforming an unversioned
prebuilt module to a versioned prebuilt module for use by a
versioned snapshot.

Bug: 142940300
Test: m nothing
Change-Id: I0f438509df23bbb8a5862541b6fbfb1d5fbc06f6
2020-01-30 11:45:47 +00:00
Paul Duffin
cc72e981c7 Use deep copy when copying bpPropertySets
Previously, only a shallow copy was made so the copy ends up sharing
some contents with the original. That was a potential source of bugs
as the copy was being made in order to be mutated.

This change switches to a deep copy; renaming the methods from
copy -> deepCopy to clarify the intent.

Makes the bpPropertySet member of bpModule a *bpPropertySet to avoid
unnecessary copying of bpPropertySet.

Bug: 142940300
Test: m nothing
Change-Id: I3f2eaa9fffab4e61d5a7cec81aa42fee9fdfec44
2020-01-20 17:12:58 +00:00
Treehugger Robot
7839d8fbed Merge "Exclude sdkMemberVersionedDepTag from visibility enforcement" 2020-01-16 20:46:16 +00:00
Anton Hansson
eec79ebe03 Default apex_available to //apex_available:platform
This means everything that goes into apexes need to be
explicitly labeled to be available for apex.

Whitelist the current offenders. This list should be
trimmed down.

Bug: 147364041
Test: m
Test: multiproduct_kati -only-soong
Exempt-From-Owner-Approval: cherry-pick from internal
Change-Id: I837299c6a15d46f8a5ba544b613776b1cc27d7b8
Merged-In: I837299c6a15d46f8a5ba544b613776b1cc27d7b8
(cherry picked from commit 93488cbb10d4882845abb732f8e53714f0982031)
2020-01-16 09:14:03 +00:00
Paul Duffin
fe1492223f Exclude sdkMemberVersionedDepTag from visibility enforcement
The implicit dependency from an unversioned sdk member to the snapshot
versions should be excluded from visibility enforcement as it provides
no benefit and would require snapshot members to be visible to their
original source modules which would complicate visibility handling.

Also, corrects a spelling mistake in sdkMemberVersionedDepTag.

Bug: 142940300
Test: m nothing
Change-Id: Ib7ea6a3d3904d042dca0ea12f6b9196d40de970b
2020-01-15 11:17:57 +00:00
Paul Duffin
c132742c96 Dedup package build components registration
Bug: 146540677
Test: m nothing
Change-Id: Iff2d7063b7f06313e9068c61a5627229463c98dd
2020-01-15 11:17:57 +00:00
Paul Duffin
1b82e6a108 Add support for java_test in sdk
Adds java_test_import module type for use by the sdk snapshot and
adds java_tests property to the sdk and sdk_snapshot module type.

This is needed for the conscrypt test sdk.

Bug: 143678475
Test: m nothing
Change-Id: Ied4c56c978dac2f92a9b3bc34b3235d7eeba2fd3
2019-12-31 15:08:59 +00:00
Paul Duffin
e602918294 Restrict SdkMemberTypes that can be used with sdk/sdk_snapshot
By default SdkMemberTypes are only supported on module_exports module
type. Support for sdk module type has to be explicitly specified.

The java_header_libs, native_shared_libs and stubs_sources are
supported on sdk. The latter is required to provide the stubs source
for an API specified in java_header_libs as they should be kept in
sync.

Bug: 146341462
Test: m nothing
Change-Id: I19b9e60792780a797458d4a9e489506602b13144
2019-12-31 15:08:59 +00:00
Paul Duffin
8150da6e9d Added module_exports/_snapshot as alias for sdk/_snapshot
Bug: 146341462
Test: m nothing
Change-Id: I27e1ef494a2b0874074aa43614612189b17e7860
2019-12-30 18:03:30 +00:00
Paul Duffin
e2bc36f12c Merge "Generate .srcjar for prebuilt_stubs_sources" 2019-12-23 13:32:08 +00:00
Paul Duffin
a0843f6a6f Differentiate between cc library and binary
The cc library and cc binary (and other cc module types) are all
instances of cc.Module. So, to differentiate between them and make
sure that only appropriate library instances can be added to
native_shared/static_libs this adds a special sdkMemberTypes field to
Module which if set specifies the SdkMemberTypes the module supports.
If it is not set then the module type cannot be used in the sdk at all.

Corrects an issue with one of the tests where a prebuilt cc
library was added to the sdk instead of a source cc library.

Adds a new test to ensure that cc_library_(shared|static)_host module
types work with the sdk as well and another test to ensure that
cc_library can be used as either.

Bug: 142918168
Test: m checkbuild
Change-Id: I359cdbdd15328ca571f276d2b6ce9a229ebb2c86
2019-12-20 15:45:17 +00:00
Paul Duffin
6c26dc7392 Dedup cc library module type registration
Test: m checkbuild
Bug: 146540677
Change-Id: I6685338550f587212e70f3eba8d4342b66f566e2
2019-12-20 15:22:52 +00:00
Paul Duffin
9b478b0831 Generate .srcjar for prebuilt_stubs_sources
Changes prebuilt_stubs_sources to generate a .srcjar from its input
instead of just exposing the srcs it is given. This ensures that it can
be used as a drop in replacement for a droidstubs module.

Updates the test for prebuilt_stubs_sources to be more representative
of the actual use made of it by sdk snapshot which outputs a directory
not a glob pattern. Added some documentation of the
prebuilts_stubs_sources srcs property to make it clear that it is
supposed to be a set of directories.

Extracts common code from sdk/testing.go for normalizing path/paths
for testing.

Bug: 143678475
Test: m conscrypt-module-sdk conscrypt-module-host-sdk conscrypt-module-test-sdk
      unzip those in place of external/conscrypt
	  build core-current-stubs-source which expects it to provide a .srcjar.
Change-Id: I8204a022557a9b0b45e19eac79ecba98ff95213d
2019-12-20 08:13:45 +00:00
Paul Duffin
d686791c16 Improve cc tests by adding sanitizer mutators
This fixes an issue that arises when deduping cc build component
registration code.

The sanitize_runtime_deps and sanitize_runtime post deps mutators were
not previously added when running cc tests. That meant the tests were
not actually testing the same behavior as at runtime.

Adding the mutators breaks the TestFuzzTarget test as the mutator adds
libclang_rt.ubsan_standalone-aarch64-android as a dependency of libc++
and the former is not available.

This fixes the test by adding the missing dependency as a cc prebuilt
shared library.

Test: m checkbuild
Bug: 146540677
Change-Id: Ie13c7e6fcefef7d9cb1cc5364be3dc563ce40de5
2019-12-19 19:19:40 +00:00
Paul Duffin
0c4979bbdf Dedup prebuilt mutator registration
Test: m checkbuild
Bug: 146540677
Change-Id: I9f8a11adf61fe6f352ec1eabc7c5bac332348ba2
2019-12-19 19:19:40 +00:00
Paul Duffin
77980a8bb9 Dedup registration for cc default test config
The cc.GatherRequiredDepsForTest() method returns some default module
definitions that are required when using cc module types like
cc_library. Previously, the registration of the module types and
mutators needed to process those default definitions was duplicated
in the test config initialization.

This change removes that duplicated code and replaces it with calls
to cc.RegisterRequiredBuildComponentsForTest(ctx) which registers all
the required build components.

Test: m checkbuild
Bug: 146540677
Change-Id: I80b6913c5691ff164ce9d308b9e1da24940f2d42
2019-12-19 19:19:40 +00:00
Paul Duffin
59986b23cc Dedup cc prebuilts module type registration
Test: m checkbuild
Bug: 146540677
Change-Id: I7e9440a075ef9c683729ed83e0033ab529fe4ac0
2019-12-19 19:19:40 +00:00
Paul Duffin
884363e782 Dedup droiddoc module type registration
Some tests were changed as they were using the wrong name for the
droiddoc_exported_dir module type.

Test: m checkbuild
Bug: 146540677
Change-Id: If57404760d2df3757c93e237696359355cf0c96e
2019-12-19 19:19:40 +00:00
Paul Duffin
f9b1da0fcb Dedup registration code for module types and singletons
The registration of module types and singletons is duplicated between
init() functions that register them for use in the build runtime and
test context creation code that registers them for testing.

This is a proof of concept for a mechanism that will allow the code
to be shared. It defines a RegistrationContext interface that is
implemented by both the TestContext and the new initRegistrationContext
type. An instance of the the latter is available through the
InitRegistrationContext variable.

The intent is that the registration of the module types and singleton
types will be extracted from the init() function into a separate
function that takes a RegistrationContext parameter. That method is
called from init() passing in the InitRegistrationContext and from a
test passing in the TestContext. Something like this:

  func init() {
    RegisterBuildComponents(android.InitRegistrationContext)
  }

  func RegisterBuildComponents(ctx android.RegistrationContext) {
    ctx.RegisterModuleType(....)
    ....
  }

A test would do something like this:

  ctx := android.NewTestContext()
  RegisterBuildComponents(ctx)

Test: m nothing
Change-Id: I97173cabb6d6cf7ce98fdb5f73418438b1997b35
2019-12-19 10:09:53 +00:00
Colin Cross
98be1bb00f Move filesystem into Config
The filesystem object was available through ModuleContext.Fs(), but
gives too much access to the filesystem without enforicing correct
dependencies.  In order to support sandboxing the soong_build
process move the filesystem into the Config.  The next change will
make it private.

Bug: 146437378
Test: all Soong tests
Change-Id: I5d3ae9108f120fd335b21efd612aefa078378813
2019-12-18 08:19:10 -08:00
Treehugger Robot
bd0624304e Merge "Use empty string for core image variant" 2019-12-16 20:54:32 +00:00
Colin Cross
7113d20774 Use empty string for core image variant
Use the empty string for the core image variant so that modules
added to imageMutator do not change their build directory.

Bug: 142286466
Test: m checkbuild
Change-Id: Ida4534d9a4d6176236aaa480fed359ce27acfaa1
Merged-In: Ida4534d9a4d6176236aaa480fed359ce27acfaa1
(cherry picked from commit 72d685ee7f45e5393be44ae4159edf083ac918de)
2019-12-13 20:44:36 -08:00
Paul Duffin
255f18e584 Decouple addition of new sdk member types from sdk code
Previously, adding a new SdkMemberType would require adding a new
sdkMemberListProperty instance to the sdkMemberListProperties as well
as adding a new property into the sdkProperties struct. They are
potential sources of conflict and couple the sdk code with all the
packages that add members to it. This change switched to a
registration model that allows each package to register its sdk
member types decoupling them from the sdk code.

Adds an SdkPropertyName() method to SdkMemberType that specifies the
name of the property to use in the sdk/sdk_snapshot. Also provides
an SdkMemberTypeBase struct to be used by providers of SdkMemberType
implementations.

SdkMemberType instances are registered using the
RegisterSdkMemberType() func which sorts the registered instances
by their SdkPropertyName() to ensure the behavior is consistent and
not affected by order of registration.

When creating a new sdk module a dynamicSdkMemberTypes instance is
created that contains the following:

* A properties struct is created dynamically that contains a field for
  each registered SdkMemberType, corresponding to that type's
  SdkPropertyName().

* A list of sdkMemberListProperty instances is also created, one for
  each registered SdkMemberType.

The dynamicSdkMemberTypes instance is cached using a key that uniquely
identifies the set of registered types just in case new types are
registered after one has been created, e.g. by tests.

Bug: 142918168
Test: m checkbuild
Change-Id: I4bf2bf56a2a49025aa41454048bc1e8ccc6baca2
2019-12-13 20:07:33 +00:00
Paul Duffin
57b9e1da75 Improve handling of generated include dirs
The exported include dirs includes both source and generated
directories (e.g. containing aidl generated headers). The latter are
always arch specific so if they are present they make all the include
directories arch specific.

This change separates the source and generated include dirs so that
the source include dirs (which are probably not arch specific) can be
optimized separately from the arch specific generated include dirs.

The FilterPathList() func was refactored to extract the more general
FilterPathListPredicate() func.

A number of tests needed to be updated to reflect the more optimal
snapshot creation.

Bug: 142918168
Test: m checkbuild
Change-Id: Id1a23d35a45b250ae2168834f9c2a65c86a5fd77
2019-12-13 09:59:48 +00:00
Paul Duffin
a7cd8c8344 Generalize common property extraction
Previously, code that attempted to optimize the generated .bp rules
treated the properties structure as a single entity. So, a single arch
specific value would cause all properties to be treated as arch
specific. Also, that code was specific to one structure type.

This generalizes the optimization to work with any properties structure
which will be helpful for other multi-variant module types. It also
treats each property separately.

The hasArchSpecificFlags field has been removed from nativeLibInfo and
a commonProperties field has been added instead into which the common
values will be found. File path creation that conditionally prefixed a
path with archType has been replaced with general code that relies on
archType being "" for common properties and filepath.Join(..) ignoring
empty string components.

The common and arch variant properties are always processed. The first
within the context of the .bp module's property set and the latter
within an arch specific property set. There are always some properties
that are arch specific, e.g. outputFile, so there is no need to worry
about an empty arch property set being created.

The archSpecificNativeLibInfo type was renamed nativeLibInfoProperties
as it may not be arch specific.

The printExportedDirCopyCommandsForNativeLibs variable was renamed to
addExportedDirCopyCommandsForNativeLibs as it no longer does any
printing.

Bug: 142918168
Test: m checkbuild
Change-Id: Iad45913299c37fd76fe03ed0ca68bdc68ed76431
2019-12-13 09:59:48 +00:00
Paul Duffin
9ab556fd8e Added support for using static libraries in sdk snapshot
Parameterized the cc.librarySdkMemberType to allow it to support
both static and shared libraries. Created two instances, one for shared
and one for static libraries. A follow up change will add support for
libraries that can be both.

Added *librarySdkMemberType to nativeMemberInfo as information from
there is needed when generating the snapshot.

Made organizeVariants() func a method of *librarySdkMemberType so that
it can initialize the new field. Moved it to be with all the other
methods of that type.

Added host and device tests for the new module type.

Bug: 142918168
Test: m nothing
Change-Id: I00b1e8424b9d81f7d15edc4883971d10668ec2cc
2019-12-12 11:37:39 +00:00
Paul Duffin
c62a5107f8 Discard duplicate operations to copy files to snapshot
Header include directories are copied into the snapshot separately for
each cc library that exports them. However, the same include
directories can be exported by multiple libraries which caused ninja
error because two separate rules (albeit identical) were defined to
create the same files.

This avoids the duplicate ninja rules by detecting and discarding
duplicate copies, i.e. where the source and destination are the same.
It will also report an error if two or more different source files are
copied to the same destination.

Bug: 142918168
Test: m nothing
      added test and verified it produced two identical copy rules
      fixed code and verified duplicate copy rules had been eliminated

Change-Id: I39e37405035bee5093f96e03248e9e29ed30962c
2019-12-12 11:37:39 +00:00
Paul Duffin
fa02872b26 Sort sdkProperties by package and then name
Bug: 143678475
Test: m nothing
Change-Id: Ib610b799e7d86b413f4321239535a1caf4775ed9
2019-12-12 11:31:59 +00:00
Jiyong Park
0b0e1b9804 AndroidMkEntries() returns multiple AndroidMkEntries structs
AndroidMkEntries now returns multiple AndroidMkEntires so that a module
can emit multiple Make modules if needed.

Bug: 128708192
Test: m

Change-Id: I56b6f76d22943b80329951c5acb80a1b932441ad
2019-12-11 17:25:27 +09:00
Paul Duffin
ce482dce02 Exclude META-INF/ from repackaged source jar in snapshot
Prevents the META-INF/MANIFEST.MF file from the source jar from being
copied to the snapshot zip file.

Bug: 143678475
Test: m conscrypt-module-sdk
      manually check contents of generated zip

Change-Id: I6eca1435dfc25b562e49de46b049fa81cf8daf90
2019-12-11 08:13:07 +00:00
Paul Duffin
593b3c9fb0 Ensure prebuilt modules have same visibility as source modules
Exports visibility and package mutator registration functions so they
can be used in sdk testing. Updates sdk test to support visibility and
package modules.

Adds EffectiveVisibility(...)[]string function to make the effective
visibility rules available to sdk snapshot creation.

Extracts compositeRule.Strings() []string from compositeRule.String()
method so that it can be used by above func.

Adds visibility property to sdk snapshot and prebuilt modules along
with a test to ensure it works properly.

Adds dir parameter to CheckSnapshot so that it can check the snapshot
generated for a non-root package. That is required in order to ensure
that visibility of :__subpackages__ on a source module in package
<pkg> is resolved to an effective visibility of
//<pkg>:__subpackages__ on its corresponding prebuilt.

Test: m conscrypt-module-sdk
Bug: 143678475
Change-Id: Icaacac5b9c04726d28e6fec93e49715ac45df7f4
2019-12-09 13:32:28 +00:00
Paul Duffin
9d8d609fcd Pass SdkMember to AddPrebuiltModule instead of the name
This is needed for a follow up change that makes sure that the
prebuilt modules have the same visibility as the source modules.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I9461c8c094ab19ee9ececb5e5fd50565789f2fa2
2019-12-09 13:28:36 +00:00
Treehugger Robot
8cda6d93de Merge "Support header and implementation jars in sdk" 2019-12-07 08:52:09 +00:00
Colin Cross
ae6c5207cc Move ImageMutator after archMutator
Move the ImageMutator to be registered just after the archMutator
in preparation for moving it between osMutator and archMutator.
Requries updating variants in a few tests that now run the
ImageMutator.

Bug: 142286466
Test: no change to build.ninja
Test: all soong tests
Change-Id: Ia9d2a7bc0e225bedec3c9a83ea04f471a931bf47
2019-12-06 12:37:14 -08:00
Paul Duffin
a0dbf43a82 Support header and implementation jars in sdk
Add a new java_header_libs property that exports header jars (as
java_libs currently does) and switch java_libs to export implementation
jars instead.

Refactors implementation of the existing library sdk member type so
both properties can be supported from common code as they differ only
on the jar being exported.

Bug: 143678475
Test: m nothing
Change-Id: I04642122f72d083bbdfd3290624f957b71ee8875
2019-12-06 16:13:58 +00:00
Paul Duffin
d835daaade Separate out module type specific test config
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I71433a7c89dfae4c27537f1f66bd677cf9fba0b3
2019-12-06 16:13:58 +00:00
Paul Duffin
a6e737b078 Organize sdk member properties
Grouping in alphabetical order by package and then by name within the
package should minimize conflicts when making changes.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: Ia7dbcd41ce8b8dd8675a90b1b6868fcaeaf72ee4
2019-12-06 12:16:59 +00:00
Paul Duffin
a80fdec00d Extract the cc and java sdk related tests out into their own file
Makes it easier for multiple people to work on different areas without
conflicts.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I93140450c0b049946a9d0432225bccff82f5a873
2019-12-06 12:16:59 +00:00
Paul Duffin
c3c5d5e351 Improve testing of sdk snapshot generation
Adds TestHelper to provide general test helper functionality for use by
any test.

Adds testSdkResult, composed with TestHelper that encapsulates the
result of processing and sdk {..} definition and provides specialized
support for testing the build rules.

Dedups the analysis of the sdk build rules, and improves it to extract
more information, and in different forms. That is represented by the
snapshotBuildInfo struct.

Adds a CheckSnapshot() method which checks the snapshot for a specified
sdk version. It takes a list of functions that can each perform a check
on a specific facet of the supplied snapshotBuildInfo.

Methods are provided for tests to use to check the following facets:
* Generated Android.bp contents.
* Copy rules
* Merge zip inputs

This approach makes it possible for each test to customize what is
being checked without either duplicating functionality, causing a
proliferation of specialized forms of the CheckSnapshot method for
different types of tests or adding arguments for every possible check
that any test would need which would lead to lots of churn to existing
tests when new arguments are added.

The main testing improvement is for CheckSnapshot() to actually try and
load the Android.bp that is generated. In order to do that it was
necessary to create a mock filesystem populated with information from
the build rules, i.e. the destination files from every Cp command as
well as the destination directory from every repackage zip command.

That helps detect a number of sources of errors:
* Failing to copy a file/directory that is mentioned in the generated
  Android.bp file.
* Invalid properties.
* Invalid format of the .bp file.
* Integrity issues within the .bp file.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I4d3fe18f86698186d18e7e8b32d2e319183f7f0c
2019-12-06 12:16:59 +00:00
Paul Duffin
82d90438be Separate sdk testing infrastructure from sdk tests
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: Ib00870ddefc4c1dd9e42ca594e6ebe8e24c42e05
2019-12-06 12:16:59 +00:00
Paul Duffin
1387957727 Parameterize the sdk member processing
Extracts the type specific functionality into the SdkMemberType
interface which has to be implemented by each module type that can
be added as a member of the sdk. It provides functionality to add
the required dependencies for the module type, check to see if a
resolved module is the correct instance and build the snapshot.

The latter was previously part of SdkAware but was moved because
it has to be able to process multiple SdkAware variants so delegating
it to a single instance did not make sense.

The custom code for handling each member type specific property,
e.g. java_libs, has been replaced with common code that processes
a list of sdkMemberListProperty struct which associates the
property (name and getter) with the SdkMemberType and a special
DependencyTag which is passed to the SdkMemberType when it has to add
dependencies.

The DependencyTag contains a reference to the appropriate
sdkMemberListProperty which allows the resolved dependencies to be
grouped by type.

Previously, the dependency collection methods would ignore a module if
it was an unsupported type because they did not have a way of
determining which property it was initially listed in. That meant it
was possible to add say a droidstubs module to the java_libs property
(and because they had the same variants) it would work as if it was
added to the stubs_sources property. Or alternatively, a module of an
unsupported type could be added to any property and it would just be
ignored.

However, the DependencyTag provides information about which property
a resolved module was referenced in and so it can detect when the
resolved module is of the wrong type and report an error. That check
identified a bug in one of the tests where the sdk referenced a
java_import module (which is not allowed in an sdk) instead of a
java_library module (which is allowed). That test was fixed as part
of this.

A list of sdkMemberListProperty structs defines the member properties
supported by the sdk and are processed in order to ensure consistent
behaviour.

The resolved dependencies are grouped by type and each group is then
processed in defined order. Within each type dependencies are grouped
by name and encapsulated behind an SdkMember interface which includes
the name and the list of variants.

The Droidstubs and java.Library types can only support one variant and
will fail if given more.

The processing for the native_shared_libs property has been moved into
the cc/library.go file so the sdk package code should now have no type
specific information in it apart from what is if the list of
sdkMemberListProperty structs.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I10203594d33dbf53441f655aff124f9ab3538d87
2019-12-06 12:16:59 +00:00
Paul Duffin
375058f67d Use static build rules in snapshot generation
It is easier to extract information out of static build rules than it
is out of custom build rules built using the builder as the former
provides access to the in/out and args separate from the rule whereas
the latter only provides access to in/out.

Also, cleans up some warnings that appear in Intellij.

There is a lot of duplication in the testing code. That will be
resolved in a follow up change.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I973bc0c90b0affd84487f1b222dd3e6c22c07ec0
2019-12-06 12:16:53 +00:00
Paul Duffin
46fc58ee8d Disable TestHostSnapshot test for non-linux build
Building sdk snapshots on anything other than linux glibc is not
supported at the moment.

Bug: 145598135
Test: m nothing
Change-Id: I34155252545e9f37c5361ed94a27a87024fcece2
2019-12-04 10:15:40 +00:00
Paul Duffin
e44358fa8f Add support for host sdk
Adds HostSupported() method to ModuleBase for use by sdk. Adds
host_supported/device_supported to all prebuilt modules and the
sdk snapshot (where necessary).

Adds TestHostSnapshot to verify the behavior is correct.

Bug: 143678475
Test: m nothing
Change-Id: I8b4d097e46d5804f62cb8f651a83069299a3e639
2019-12-02 12:59:38 +00:00
Paul Duffin
b645ec8e34 Add model to represent generated snapshot .bp file
Having each module type generate the contents of the snapshot's .bp
file results in lots of duplicated code. This adds an intermediate
model for use by the module types and then generates the .bp file
contents from that.

This not only removes the duplicated formatting code but it also
allows consistent handling of shared properties such as name further
reducing duplication. It also makes it possible to duplicate the
versioned and unversioned prebuilt modules from the same model.

Extracts generatedContents from generatedFile to allow the contents
to be populated without creating an output file, for testing.

Cleans up unused code.

Bug: 143678475
Test: m nothing
Change-Id: If21b84db0ef3fdfb5dc11ea0973ce6cb73603ea3
2019-12-02 12:59:38 +00:00
Paul Duffin
01178ed7d1 Remove unused methods from sdk/sdk_test.go
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: Ic011fd63b226beb455793e2df08114992b858698
2019-11-29 20:35:06 +00:00
Paul Duffin
66905ed6cf Simplify sdk_snapshot module member list generation
When generating the list of members the names of the resolved
modules are used in the case of java_libs and stubs_source whereas
the name stored in the nativeLibInfo is used for native_shared_libs.

Those names are identical to the names provided in the corresponding
sdk property so this switches to use those instead as it is more
consistent and simplifies future refactorings.

Bug: 143678475
Test: m nothing
Change-Id: I9c08a8175fa9a37bb0cd89a361ef4c0f613b0dab
2019-11-28 21:02:12 +00:00
Paul Duffin
7264c69892 Remove unused frozenVersions() method
Bug: 143678475
Test: m nothing
Change-Id: Ia5d417cfccfcd90b02a95b5b806b211b97e340db
2019-11-28 21:02:12 +00:00
Paul Duffin
b1a8813520 Dedup code for generating prebuilt_stubs_sources in snapshot
Removes duplicated code for generating a versioned and non-versioned
form of the prebuilt_stubs_sources module in the snapshot.

Bug: 143678475
Test: m nothing

Change-Id: Ic431b6fcb6f3c7c85f1a04cb6f087424aaf0cffa
2019-11-26 22:30:27 +00:00
Paul Duffin
ac37c503e0 Check the contents of an SDK snapshot's generated Android.bp
Test: m nothing
Bug: 143678475
Change-Id: I407d83c79d6b1ad8082e560726f0bfa7cacab3f0
2019-11-26 22:30:19 +00:00
Paul Duffin
9010250a16 Generate unversioned cc_prebuilt_library_shared
Without this the generated snapshot build file will not load.

Bug: 143678475
Test: m nothing and check output for dummy sdk
Change-Id: I252ed01280aeceb4ff40b2d9de2bc2bf88858ea9
2019-11-26 16:19:36 +00:00
Colin Cross
09ef474b6f Merge changes I0dcc9c7b,I9bc40642
* changes:
  Move cc.imageMutator into the android package
  Make CreateVariations return []android.Module
2019-11-25 22:30:17 +00:00
Colin Cross
4b49b768a2 Make TestContext.RegisterModuleType take an android.ModuleFactory
Avoid having to pass ModuleFactoryAdaptor to every call to
RegisterModuleType in a test by wrapping RegisterModuleType.

Test: all soong tests
Change-Id: If8847d16487de0479cc3020b728256922b3cadba
2019-11-25 10:59:44 -08:00
Paul Duffin
9154718303 Adds droidstubs support to sdk module
Adds stubs_sources property to sdk and unzips the droidstubs srcjar
into the snapshot directory.

Adds an UnzipToSnapshot method to the SnapshotBuilder which creates
a rule that uses zip2zip to repackage the supplied zip content into a
temporary zip file that matches what the required snapshot structure.
e.g. if the supplied zip contains foo/Foo.java and that needs to be in
the snapshot directory java/foo/stubs then it will create a zip that
contains java/foo/stubs/foo/Foo.java.

The temporary zip that is the output of that rule is added to the
zipsToMerge field for merging later.

If the zipsToMerge is empty then the snapshot zip is created as
before. Otherwise, a temporary zip file is created. That is then
merged with the other zip files in zipsToMerge to create the final
snapshot zip.

Adds prebuilt_stubs_sources for use by the generated .bp module.

Bug: 143678475
Test: added conscrypt sdk module and attempted to build it
Change-Id: Ie274263af3a08e36a73c61c0dbf0c341fd6967e2
2019-11-22 20:52:59 +00:00
Paul Duffin
504b46160f Simplify building an SDK snapshot from the command line
Adds a phony target for each sdk module that builds the snapshot zip.

Test: built an sdk module from command line and checked the zip was created.
Bug: 143678475
Change-Id: I4599332443b8da9adea0a16f00f569ffbd421602
2019-11-22 14:52:29 +00:00
Colin Cross
7228ecd5e3 Move cc.imageMutator into the android package
Prepare for making the image mutator available to all modules and
moving it between the os and arch mutators by moving it into the
android package and using an interface implemented by the module
types to control it.

Bug: 142286466
Test: No unexpected changes to out/soong/build.ninja
Change-Id: I0dcc9c7b5ec80edffade340c367f6ae4da34151b
2019-11-20 15:21:32 -08:00
Paul Duffin
0e0cf1dca5 Refactor sdk update mechanism
Creates a SnapshotBuilder and GeneratedSnapshotFile interfaces to allow
the java library snapshot work to be moved into the java package.

Test: m -j60 checkbuild
Change-Id: I857167616026149d5e85885621b53876b419ba9b
2019-11-14 17:49:02 +00:00
Jiyong Park
232e785b98 SDK snapshot is dist'ed
`m module_sdk dist` produces snapshots of all SDKs in the source tree.
A snapshot is a zip file consists of Android.bp, exported headers,
exported AIDL files, stubs for native libs and jars. The zip file is
expected to be downloaded from the build server and extracted to a
directory (which probably will be
/prebuilts/module_sdks/<module_name>/current).

Bug: 138182343
Test: m (sdk_test.go updated)

Change-Id: Idbe4bc24795fe08f26fc1cf7497028f9d162053a
2019-11-07 12:24:48 +09:00
Jiyong Park
100f3fd118 sdk modules are by default compile_multilib: "both"
Bug: 143948100
Test: m (sdk_test.go amended)
Change-Id: I7df1b96af49a6569012e44eeb3c0722fac63fa51
2019-11-06 16:34:15 +09:00
Jiyong Park
73c54ee7fe native shared libs in an SDK can be snapshotted
The snapshot script can now handle native shared libs in an SDK.

Bug: 138182343
Test: create following sdk module:
sdk {
    name: "mysdk",
    native_shared_libs: ["libc", "libdl"],
}
, then execute `m mysdk` and execute the update_prebuilt-1.sh as
prompted. Following directories are generated under the directory where
mysdk is defined at:

1
├── aidl
├── Android.bp
├── arm64
│   ├── include
│   ├── include_gen
│   └── lib
│       ├── libc.so
│       └── libdl.so
├── include
│   └── bionic
│       └── libc
│           └── include
│               ├── alloca.h
│               ├── android
│               │   ├── api-level.h
<omitted>

Change-Id: Ia1dcc5564c1cd17c6ccf441d06d5995af55db9ee
2019-10-29 12:27:35 +09:00
Treehugger Robot
c5bba642f1 Merge "Prohibit dependencies outside of uses_sdks" 2019-10-18 00:30:18 +00:00
Colin Cross
ad4a597c79 Merge "Add method to determine variations from a Target" 2019-10-17 18:52:20 +00:00
Jiyong Park
a7bc8ad0b9 Prohibit dependencies outside of uses_sdks
When an APEX is built with uses_sdks, any depedndency from the APEX to
the outside of the APEX should be from the SDKs that the APEX is built
against.

Bug: 138182343
Test: m

Change-Id: I1c2ffe8d28ccf648d928ea59652c2d0070bf10eb
2019-10-17 11:19:53 +09:00
Colin Cross
0f7d2ef3ac Add method to determine variations from a Target
The arch variants are hardcoded in every module type.  Refactor
them out into a Target.Variations() method in preparation for
splitting the arch mutator into two, which will require using
different variations.

Test: m checkbuild
Change-Id: I28ef7cd5168095ac888fe77f04e27f9ad81978c0
2019-10-16 14:52:30 -07:00
Jiyong Park
9b409bcd10 Create scripts to update and freeze a module SDK
`m <sdk_name>` generates two scripts each of which is use to update the
current snapshot of the SDK and to freeze ToT as a new version,
respectively. Executing the scripts will copy necessary files (stub
libraries, AIDL files, etc.) along with Android.bp into the ./<apiver>
directory under the directory where the sdk is defined.

This change also introduces a new module type 'sdk_snapshot' that
represents a snapshot of an SDK. It will be auto-generated by the above
scripts, so developers are not expected to write this manually.

The module type 'sdk' is now used to simply specify the list of modules
that an SDK has.

Finally, this change changes the version separator from '#' to '@'
because '#' confuses Make.

Bug: 138182343
Test: m

Change-Id: Ifcbc3a39a2f6ad5b4f4b200ba55a1ce3281498cf
2019-10-15 18:49:58 +09:00
Jiyong Park
d1063c1586 Introduce module type 'sdk'
This change introduces a new module type named 'sdk'. It is a logical
group of prebuilt modules that together provide a context (e.g. APIs)
in which Mainline modules (such as APEXes) are built.

A prebuilt module (e.g. java_import) can join an sdk by adding it to the
sdk module as shown below:

sdk {
    name: "mysdk#20",
    java_libs: ["myjavalib_mysdk_20"],
}

java_import {
    name: "myjavalib_mysdk_20",
    srcs: ["myjavalib-v20.jar"],
    sdk_member_name: "myjavalib",
}

sdk {
    name: "mysdk#21",
    java_libs: ["myjavalib_mysdk_21"],
}

java_import {
    name: "myjavalib_mysdk_21",
    srcs: ["myjavalib-v21.jar"],
    sdk_member_name: "myjavalib",
}

java_library {
    name: "myjavalib",
    srcs: ["**/*/*.java"],
}

An APEX can specify the SDK(s) that it wants to build with via the new
'uses_sdks' property.

apex {
    name: "myapex",
    java_libs: ["libX", "libY"],
    uses_sdks: ["mysdk#20"],
}

With this, libX, libY, and their transitive dependencies are all built
with the version 20 of myjavalib (the first java_import module) instead
of the other one (which is for version 21) and java_library having the
same name (which is for ToT).

Bug: 138182343
Test: m (sdk_test.go added)
Change-Id: I7e14c524a7d6a0d9f575fb20822080f39818c01e
2019-09-22 08:21:27 +09:00