Previously, adding a new SdkMemberType would require adding a new
sdkMemberListProperty instance to the sdkMemberListProperties as well
as adding a new property into the sdkProperties struct. They are
potential sources of conflict and couple the sdk code with all the
packages that add members to it. This change switched to a
registration model that allows each package to register its sdk
member types decoupling them from the sdk code.
Adds an SdkPropertyName() method to SdkMemberType that specifies the
name of the property to use in the sdk/sdk_snapshot. Also provides
an SdkMemberTypeBase struct to be used by providers of SdkMemberType
implementations.
SdkMemberType instances are registered using the
RegisterSdkMemberType() func which sorts the registered instances
by their SdkPropertyName() to ensure the behavior is consistent and
not affected by order of registration.
When creating a new sdk module a dynamicSdkMemberTypes instance is
created that contains the following:
* A properties struct is created dynamically that contains a field for
each registered SdkMemberType, corresponding to that type's
SdkPropertyName().
* A list of sdkMemberListProperty instances is also created, one for
each registered SdkMemberType.
The dynamicSdkMemberTypes instance is cached using a key that uniquely
identifies the set of registered types just in case new types are
registered after one has been created, e.g. by tests.
Bug: 142918168
Test: m checkbuild
Change-Id: I4bf2bf56a2a49025aa41454048bc1e8ccc6baca2
Header include directories are copied into the snapshot separately for
each cc library that exports them. However, the same include
directories can be exported by multiple libraries which caused ninja
error because two separate rules (albeit identical) were defined to
create the same files.
This avoids the duplicate ninja rules by detecting and discarding
duplicate copies, i.e. where the source and destination are the same.
It will also report an error if two or more different source files are
copied to the same destination.
Bug: 142918168
Test: m nothing
added test and verified it produced two identical copy rules
fixed code and verified duplicate copy rules had been eliminated
Change-Id: I39e37405035bee5093f96e03248e9e29ed30962c
Prevents the META-INF/MANIFEST.MF file from the source jar from being
copied to the snapshot zip file.
Bug: 143678475
Test: m conscrypt-module-sdk
manually check contents of generated zip
Change-Id: I6eca1435dfc25b562e49de46b049fa81cf8daf90
Exports visibility and package mutator registration functions so they
can be used in sdk testing. Updates sdk test to support visibility and
package modules.
Adds EffectiveVisibility(...)[]string function to make the effective
visibility rules available to sdk snapshot creation.
Extracts compositeRule.Strings() []string from compositeRule.String()
method so that it can be used by above func.
Adds visibility property to sdk snapshot and prebuilt modules along
with a test to ensure it works properly.
Adds dir parameter to CheckSnapshot so that it can check the snapshot
generated for a non-root package. That is required in order to ensure
that visibility of :__subpackages__ on a source module in package
<pkg> is resolved to an effective visibility of
//<pkg>:__subpackages__ on its corresponding prebuilt.
Test: m conscrypt-module-sdk
Bug: 143678475
Change-Id: Icaacac5b9c04726d28e6fec93e49715ac45df7f4
This is needed for a follow up change that makes sure that the
prebuilt modules have the same visibility as the source modules.
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I9461c8c094ab19ee9ececb5e5fd50565789f2fa2
Adds TestHelper to provide general test helper functionality for use by
any test.
Adds testSdkResult, composed with TestHelper that encapsulates the
result of processing and sdk {..} definition and provides specialized
support for testing the build rules.
Dedups the analysis of the sdk build rules, and improves it to extract
more information, and in different forms. That is represented by the
snapshotBuildInfo struct.
Adds a CheckSnapshot() method which checks the snapshot for a specified
sdk version. It takes a list of functions that can each perform a check
on a specific facet of the supplied snapshotBuildInfo.
Methods are provided for tests to use to check the following facets:
* Generated Android.bp contents.
* Copy rules
* Merge zip inputs
This approach makes it possible for each test to customize what is
being checked without either duplicating functionality, causing a
proliferation of specialized forms of the CheckSnapshot method for
different types of tests or adding arguments for every possible check
that any test would need which would lead to lots of churn to existing
tests when new arguments are added.
The main testing improvement is for CheckSnapshot() to actually try and
load the Android.bp that is generated. In order to do that it was
necessary to create a mock filesystem populated with information from
the build rules, i.e. the destination files from every Cp command as
well as the destination directory from every repackage zip command.
That helps detect a number of sources of errors:
* Failing to copy a file/directory that is mentioned in the generated
Android.bp file.
* Invalid properties.
* Invalid format of the .bp file.
* Integrity issues within the .bp file.
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I4d3fe18f86698186d18e7e8b32d2e319183f7f0c
Extracts the type specific functionality into the SdkMemberType
interface which has to be implemented by each module type that can
be added as a member of the sdk. It provides functionality to add
the required dependencies for the module type, check to see if a
resolved module is the correct instance and build the snapshot.
The latter was previously part of SdkAware but was moved because
it has to be able to process multiple SdkAware variants so delegating
it to a single instance did not make sense.
The custom code for handling each member type specific property,
e.g. java_libs, has been replaced with common code that processes
a list of sdkMemberListProperty struct which associates the
property (name and getter) with the SdkMemberType and a special
DependencyTag which is passed to the SdkMemberType when it has to add
dependencies.
The DependencyTag contains a reference to the appropriate
sdkMemberListProperty which allows the resolved dependencies to be
grouped by type.
Previously, the dependency collection methods would ignore a module if
it was an unsupported type because they did not have a way of
determining which property it was initially listed in. That meant it
was possible to add say a droidstubs module to the java_libs property
(and because they had the same variants) it would work as if it was
added to the stubs_sources property. Or alternatively, a module of an
unsupported type could be added to any property and it would just be
ignored.
However, the DependencyTag provides information about which property
a resolved module was referenced in and so it can detect when the
resolved module is of the wrong type and report an error. That check
identified a bug in one of the tests where the sdk referenced a
java_import module (which is not allowed in an sdk) instead of a
java_library module (which is allowed). That test was fixed as part
of this.
A list of sdkMemberListProperty structs defines the member properties
supported by the sdk and are processed in order to ensure consistent
behaviour.
The resolved dependencies are grouped by type and each group is then
processed in defined order. Within each type dependencies are grouped
by name and encapsulated behind an SdkMember interface which includes
the name and the list of variants.
The Droidstubs and java.Library types can only support one variant and
will fail if given more.
The processing for the native_shared_libs property has been moved into
the cc/library.go file so the sdk package code should now have no type
specific information in it apart from what is if the list of
sdkMemberListProperty structs.
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I10203594d33dbf53441f655aff124f9ab3538d87
It is easier to extract information out of static build rules than it
is out of custom build rules built using the builder as the former
provides access to the in/out and args separate from the rule whereas
the latter only provides access to in/out.
Also, cleans up some warnings that appear in Intellij.
There is a lot of duplication in the testing code. That will be
resolved in a follow up change.
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I973bc0c90b0affd84487f1b222dd3e6c22c07ec0
Adds HostSupported() method to ModuleBase for use by sdk. Adds
host_supported/device_supported to all prebuilt modules and the
sdk snapshot (where necessary).
Adds TestHostSnapshot to verify the behavior is correct.
Bug: 143678475
Test: m nothing
Change-Id: I8b4d097e46d5804f62cb8f651a83069299a3e639
Having each module type generate the contents of the snapshot's .bp
file results in lots of duplicated code. This adds an intermediate
model for use by the module types and then generates the .bp file
contents from that.
This not only removes the duplicated formatting code but it also
allows consistent handling of shared properties such as name further
reducing duplication. It also makes it possible to duplicate the
versioned and unversioned prebuilt modules from the same model.
Extracts generatedContents from generatedFile to allow the contents
to be populated without creating an output file, for testing.
Cleans up unused code.
Bug: 143678475
Test: m nothing
Change-Id: If21b84db0ef3fdfb5dc11ea0973ce6cb73603ea3
When generating the list of members the names of the resolved
modules are used in the case of java_libs and stubs_source whereas
the name stored in the nativeLibInfo is used for native_shared_libs.
Those names are identical to the names provided in the corresponding
sdk property so this switches to use those instead as it is more
consistent and simplifies future refactorings.
Bug: 143678475
Test: m nothing
Change-Id: I9c08a8175fa9a37bb0cd89a361ef4c0f613b0dab
Without this the generated snapshot build file will not load.
Bug: 143678475
Test: m nothing and check output for dummy sdk
Change-Id: I252ed01280aeceb4ff40b2d9de2bc2bf88858ea9
Adds stubs_sources property to sdk and unzips the droidstubs srcjar
into the snapshot directory.
Adds an UnzipToSnapshot method to the SnapshotBuilder which creates
a rule that uses zip2zip to repackage the supplied zip content into a
temporary zip file that matches what the required snapshot structure.
e.g. if the supplied zip contains foo/Foo.java and that needs to be in
the snapshot directory java/foo/stubs then it will create a zip that
contains java/foo/stubs/foo/Foo.java.
The temporary zip that is the output of that rule is added to the
zipsToMerge field for merging later.
If the zipsToMerge is empty then the snapshot zip is created as
before. Otherwise, a temporary zip file is created. That is then
merged with the other zip files in zipsToMerge to create the final
snapshot zip.
Adds prebuilt_stubs_sources for use by the generated .bp module.
Bug: 143678475
Test: added conscrypt sdk module and attempted to build it
Change-Id: Ie274263af3a08e36a73c61c0dbf0c341fd6967e2
Creates a SnapshotBuilder and GeneratedSnapshotFile interfaces to allow
the java library snapshot work to be moved into the java package.
Test: m -j60 checkbuild
Change-Id: I857167616026149d5e85885621b53876b419ba9b
`m module_sdk dist` produces snapshots of all SDKs in the source tree.
A snapshot is a zip file consists of Android.bp, exported headers,
exported AIDL files, stubs for native libs and jars. The zip file is
expected to be downloaded from the build server and extracted to a
directory (which probably will be
/prebuilts/module_sdks/<module_name>/current).
Bug: 138182343
Test: m (sdk_test.go updated)
Change-Id: Idbe4bc24795fe08f26fc1cf7497028f9d162053a
The snapshot script can now handle native shared libs in an SDK.
Bug: 138182343
Test: create following sdk module:
sdk {
name: "mysdk",
native_shared_libs: ["libc", "libdl"],
}
, then execute `m mysdk` and execute the update_prebuilt-1.sh as
prompted. Following directories are generated under the directory where
mysdk is defined at:
1
├── aidl
├── Android.bp
├── arm64
│ ├── include
│ ├── include_gen
│ └── lib
│ ├── libc.so
│ └── libdl.so
├── include
│ └── bionic
│ └── libc
│ └── include
│ ├── alloca.h
│ ├── android
│ │ ├── api-level.h
<omitted>
Change-Id: Ia1dcc5564c1cd17c6ccf441d06d5995af55db9ee
`m <sdk_name>` generates two scripts each of which is use to update the
current snapshot of the SDK and to freeze ToT as a new version,
respectively. Executing the scripts will copy necessary files (stub
libraries, AIDL files, etc.) along with Android.bp into the ./<apiver>
directory under the directory where the sdk is defined.
This change also introduces a new module type 'sdk_snapshot' that
represents a snapshot of an SDK. It will be auto-generated by the above
scripts, so developers are not expected to write this manually.
The module type 'sdk' is now used to simply specify the list of modules
that an SDK has.
Finally, this change changes the version separator from '#' to '@'
because '#' confuses Make.
Bug: 138182343
Test: m
Change-Id: Ifcbc3a39a2f6ad5b4f4b200ba55a1ce3281498cf
