Commit graph

57 commits

Author SHA1 Message Date
Paul Duffin
36474d322b Stop sdk package depending on testing.T being embedded in TestResult
This change is in preparation for removing testing.T from TestResult.

Bug: 181070625
Test: m nothing
Change-Id: I67535aff0d894e6e3d8456b75540f340af853355
2021-03-13 15:59:33 +00:00
Paul Duffin
c10ee77ea6 Merge changes Ifc96992e,Ic76523ba
* changes:
  Support test fixtures in sdk package
  Add apexFixtureFactory to apex package
2021-03-12 08:01:31 +00:00
Paul Duffin
fe9a9e3f7d Add missing // to clarify comment
Bug: 181070625
Test: m nothing
Change-Id: Ia22a1be7a05b3a9efd34f2bf70216eb046396be0
2021-03-11 17:41:01 +00:00
Paul Duffin
4a2a29ce6a Support test fixtures in sdk package
Bug: 181070625
Test: m nothing
Change-Id: Ifc96992e54c1b1d89a82b88ab27e555ae267a51e
2021-03-11 17:25:29 +00:00
Paul Duffin
981b94b460 Switch CheckSnapshot from a testSdkResult method to a function
This will allow the testSdkResult to be replaced with the TestResult
when switching sdk package to use test fixtures.

Bug: 181070625
Test: m nothing
Change-Id: Ieca63f4c189f5e804102aeb4073289ea03143b6e
2021-03-11 13:39:24 +00:00
Paul Duffin
8306f25679 Make testSdkResult compatible with android.TestResult
This change makes it easier to switch the sdk package over to using the
new fixture mechanism by removing inconsistencies between the
testSdkResult and TestResult structures.

Bug: 181070625
Test: m nothing
Change-Id: Ic4c06e08ea5060fd09123f2ca65580e18b4d2ef6
2021-03-11 13:38:56 +00:00
Paul Duffin
3b2636fdf3 Merge "Remove duplicate sdk.TestHelper" 2021-03-10 19:59:56 +00:00
Paul Duffin
a3cb2b396f Remove duplicate sdk.TestHelper
As part of the work on the new fixture mechanism some of the TestHelper
functionality was moved into the android/fixture.go package. This moves
the rest and removes the now duplicated TestHelper from the sdk
package.

Also removed some unnecessary & operators.

Bug: 181070625
Test: m nothing
Change-Id: Ia09a5d05e4fab3a4e28cf44b2d947a33541e3925
2021-03-10 14:07:20 +00:00
Paul Duffin
6d9108f047 Extract sdk registration code into function for reuse
Test: m nothing
Change-Id: I0d0e36324808831deb9a32f07ca3696125703873
2021-03-09 23:03:45 +00:00
Paul Duffin
d6ceb8600c Clean up cc.RegisterRequiredBuildComponentsForTest()
Changes this function so it only registers components from the cc
package by pushing the call to genrule.RegisterGenruleBuildComponents()
down into those packages whose tests need it.

This will make it easier to migrate cc package tests to test fixtures
as the RegisterRequiredBuildComponentsForTest() no longer overlaps with
preparers from the genrule packages.

Bug: 181070625
Test: m nothing
Change-Id: Ic00c7e480dc738d7a88d038aca6ab95a1502a24a
2021-03-05 18:20:33 +00:00
Paul Duffin
42dd4e6cd6 Fix the snapshot handling of generated headers
Previously, the snapshot handling code did not preserve the directory
structure of generated include directories and instead just copied the
headers into the same module specific directory and added that single
directory to the export_include_dirs (or similar) property.

That had a couple of issues:
* The include directory was repeated in the ..._include_dirs property.
* It did not work when the include directories overlapped.

In the latter case it had a couple of issues:
* Code which compiled fine against the source would not compile against
  the prebuilt.
* Header files were duplicated in the output.

e.g. assume the following generated header file structure:
  foo/
      foo.h
  bar/
      bar.h
      baz/
          baz.h

When the sdk snapshot was passed include directories of "foo", "bar" and
headers of "foo/foo.h", "bar/bar.h", "bar/baz/baz.h" it would generate a
snapshot with the structure:
  include_gen/
      foo.h
      bar.h
      baz/
         baz.h

And:
  export_include_dirs: ["include_gen", "include_gen"]

However, when the include directories overlapped and include directories
of "foo", "bar" and "bar/baz" were passed in the directory structure
would be the same and the export_include_dirs would contain 3 usages of
"include_gen".

That meant that source code which used the following would build
against the source (because it would find "baz.h" in the "bar/baz"
include directory) but would fail when built against the prebuilts
because the "include_gen" directory did not contain "baz.h":
    #include "baz.h"

This change preserves the input directory structure for generated files
in a similar way to how it does it for source files. So, the snapshot
structure looks something like this:

  include_gen/
      foo/
          foo.h
      bar/
          bar.h
          baz/
              baz.h

And:
  export_include_dirs: [
    "include_gen/foo",
    "include_gen/bar",
    "include_gen/bar/baz",
  ],

Bug: 180427921
Test: m nothing
Change-Id: Id69eef8cf5eecd033841d3b7cd0c044a697ce404
2021-02-22 18:23:22 +00:00
Paul Duffin
d075907d29 Support testing versioned/unversioned sdk Android.bp files separately
Previously, the only way to test the Android.bp file generated by the
sdk module was to test both the unversioned and versioned parts
together. This change allows them to be tested separately.

Bug: 180479010
Test: m nothing
Change-Id: I3d695bcfbff030a6da393283ab30ec0979fb2826
2021-02-22 18:23:22 +00:00
Paul Duffin
c059c8c9a0 Add java.RegisterRequiredBuildComponentsForTest function
Insulate tests that exercise code in the java package from having to
register the build components provided by the java package by providing
a single function that registers them all. This follows the pattern
currently used in the cc and rust packages.

This change is in preparation for switching the dex_bootjars singleton
from a singleton, which does not require a module definition in order
to be instantiated, to a singleton module which does. That will require
adding a module definition into java.GatherRequiredDepsForTest() and
this change ensures that the required components will have been
registered in every test.

Bug: 177892522
Test: m nothing
Change-Id: I6475db8240894947dd07c89a940a3e4f201aa598
2021-01-21 11:47:32 +00:00
Anton Hansson
dff2c78a20 Add attribute to disable last-api compat tracking
Setting this to true by default is dangerous as it can mask bugs. Create
a dedicated attribute for java_sdk_library to enable this behavior
instead. The default will be flipped in a future CL when all the current
offenders have been fixed.

Fix all the tests to have the right API files.

Bug: 176092454
Test: m nothing
Change-Id: Ieab94bcb74abf8d018365a56fb447fe3dbd46957
2020-12-21 17:25:30 +00:00
Jingwen Chen
cda22c9bb9 Refactor 'in_make' to mean Kati is not skipped.
In Nougat and before, Make wrote a marker file to indicate that
soong_build was invoked from Make to change certain behaviors of Soong
at build time.
https://cs.android.com/android/platform/superproject/+/android-7.1.2_r36:build/core/soong.mk;l=70-73;drc=ae18638b0406ad107b0882a02a13cdd8b92f2a4e

Things have changed, and now soong_build is invoked from soong_ui, which
supports a --skip-make configuration flag:
https://cs.android.com/android/platform/superproject/+/master:build/soong/ui/build/build.go;l=31-33;drc=680387bf1d3ce7cbc77f535be7c42cec411b1687

Thus, the various remnants of 'EmbeddedInMake' and 'inMake'
configuration are misleading, since soong_build is no longer invoked
from Make. This CL refactors all instances to actually mean that
Kati is enabled (not skipped with --skip-make), and will run after
soong_build finishes, so Kati-specific behavior like the AndroidMk
singleton should run.

Test: TH presubmit

Change-Id: I576ab8e54f99f5c8ddf9feaf9a828019b279e266
2020-11-23 00:29:18 -05:00
Colin Cross
ae8600b507 Pass Config to NewTestContext instead of ctx.Register
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.

Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
2020-11-12 10:07:49 -08:00
Paul Duffin
a2ae7e0358 Allow java_sdk_library to specify doctags_files
When generating Javadoc the processor needs to be given information
about the doctags that are present in the source. This change allows
that information to be managed with the java_sdk_library that generates
the stubs source from which the Javadoc is generated.

Bug: 168301990
Test: Built offline-sdk-docs with and without the change and
      diffed them. The only difference was the timestamp.js
      file.
Change-Id: I4adbeb0781bc2191461fec856ffa90ea185e7434
2020-09-28 18:35:57 +01:00
Martin Stjernholm
79bee057a3 Merge "Handle property structs and BpPropertySets as values to AddProperty." 2020-09-28 10:06:04 +00:00
Martin Stjernholm
191c25f589 Handle property structs and BpPropertySets as values to AddProperty.
Both will create a nested property set, that may be merged with an
existing one.

Test: m nothing
Bug: 151303681
Change-Id: I30696ba3eb8960ca6fa54c9ee2cf6229ab9f5da9
2020-09-25 00:33:04 +01:00
Jiyong Park
1613e5541f HostCross is an attribute of a Target, not OsType
A host target is considered as being cross-compiled when the target
can't run natively on the build machine. For example, linux_glibc/x86_64
is a non-cross target on a standard x86/Linux machine, but is a cross
host on Mac. Previously, whether cross or not was a static attribute of
an OsType. For example, Windows was always considered as cross host,
while linux_bionic was not. This becomes a problem when we support more
host targets like linux_bionic/arm64 which should be cross-host on
standard x86/Linux machines.

This change removes HostCross from the OsClass type and instead adds a
property HostCross to the Target type. When a target is being added, it
is initialized to true when the target can't run natively on the current
build machine.

Bug: 168086242
Test: m
Change-Id: Ic37c8db918873ddf324c86b12b5412952b0f2be2
2020-09-24 14:14:31 +09:00
Paul Duffin
021f4e525f Fix prebuilt mutator ordering in tests
Previously, the prebuilt mutators were added by the
cc.RegisterRequiredBuildComponentsForTest() function as a convenience
but unfortunately it lead to some of the mutators being in a different
order in the tests than in the normal build.

This change:
* Extracts the RegisterPrebuiltMutators() call from
  cc.RegisterRequiredBuildComponentsForTest()
* Makes sure that the prebuilt mutators are registered before the
  visibility gatherer and enforcer mutators.

Bug: 162505935
Test: m nothing
Change-Id: I7d959b558200b502f0a5e4653c41ea01414e142a
2020-07-31 16:12:01 +01:00
Martin Stjernholm
7feceb2354 Add support for SDK tests with LinuxBionic OS type.
Break apart test helpers a bit to make it possible to enable
LinuxBionic in a single test, and add LinuxBionic support to
cc.GatherRequiredDepsForTest.

Test: m nothing
Bug: 160349757
Change-Id: Iace1024c964cee2308c130c945daef9e46c18c66
2020-07-16 21:55:15 +01:00
Martin Stjernholm
3ff2e661bd Disable all SDK snapshot generation tests on mac.
We frequently miss putting in the per-test skip condition, and since
mac isn't tested in presubmit this leads to build breakages and time
consuming reverts. To avoid that this blanket disables all the SDK
tests on mac. It's not used there and we got test coverage in many
linux-based builds.

Bug: 145598135
Bug: 161315642
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I2aea92fef2c0f8c2742396fe36610501dc5a6f0f
2020-07-15 16:58:05 +01:00
Treehugger Robot
2c79c871d9 Merge changes Iefcf4fbe,I450d4769,I1ffc7e7b
* changes:
  Prepend default to allow overriding in the bp file.
  Do not follow SDK member dependencies for APEX payloads.
  Do not follow prebuiltDependencyTags in APEX payload walks.
2020-07-09 12:32:31 +00:00
Paul Duffin
44f1d8404b Stop java_sdk_library_import from depending on source modules
Previously, java_sdk_library_import added the dependencies on its child
components in the deps mutator after prebuilts without a matching
source module are renamed to the source module name. That meant that
the java_sdk_library_import has to use the source module name and ended
up depending on the source module unless it was preferred.

This change adds a new component deps mutator that runs before the
PrebuiltRenameMutator so that the java_sdk_library_import can add
dependencies onto the prebuilt modules. It also updates an affected
test.

Bug: 159902351
Test: m nothing
Change-Id: I3576c4873302743e51aff547ea1497bef6d748ac
2020-07-08 16:46:16 +01:00
Martin Stjernholm
cc77601d10 Do not follow SDK member dependencies for APEX payloads.
Test: m nothing
Test: `m` with prebuilts/runtime in the manifest (along with other
  fixes)
Bug: 151303681
Change-Id: I450d476975c7ab4434228b8c4baf3af192142211
2020-07-07 18:14:26 +01:00
Paul Duffin
1110827b3f Defer \n quoting generatedContents until creating the ninja rule
Previously, when writing to generatedContents \n characters were
quoted (replaced with \\n) so as to allow them to be preserved through
ninja/rsp/bash and were unquoted (replaced \\n with \n) just before
redirecting to the output file. That meant that any code which wanted
to access the contents for other purposes, e.g. testing had to unquote
\\n.

This change moves the quoting to be part of the code that generates the
ninja rule which simplifies any other code that has to access the
contents.

Without quoting the generated Android.bp files are not formatted
properly, are all on one line and completely unreadable.

Bug: 156286550
Test: m art-module-sdk and check generated Android.bp file to make sure
      it is properly formatted.
Change-Id: I768c3b96ed08a3daf251730e2a10d9d72338c49a
2020-05-11 22:59:25 +01:00
Paul Duffin
4b8b79394f Allow extractCommonProperties to return an error
Refactoring in preparation for follow up changes.

Also:
* Adds a new AssertErrorMessageEquals() helper method.
* Improved error reporting in the accessor and added name to
  extractorProperty to ensure meaningful errors are reported.
* Added String() string method to propertiesContainer.
* Reports errors using the field name as the errors are not really
  fixable by developers and it is more meaningful to the build team.

Bug: 155628860
Test: m nothing
Change-Id: I5c5b8436bcbc39e4e7cd35df2577b2dac53e702a
2020-05-06 18:40:41 +01:00
Paul Duffin
3d1248ceb6 Export stub sources as part of the java_sdk_library
Minor refactoring of checkMergeZip(string) -> checkMergeZips(...string)
to allow testing of multiple merge zips.

Bug: 153443117
Test: m nothing
Change-Id: I8db00f611ced15f8476ba16f2834a72e8c913596
2020-04-09 01:32:38 +01:00
Paul Duffin
dd46f71493 Support java_sdk_library as member of sdk
Bug: 153443117
Test: m nothing
Change-Id: I9d8089b2555038e3f10ad5939a6a7b01839c67ea
2020-04-09 01:32:38 +01:00
Paul Duffin
b07fa51196 Support extracting common values from embedded structures
This change also added support for excluding properties from common
value extraction by using a struct tag of `sdk:"keep"` That was needed
to prevent the fields in SdkMemberPropertiesBase from having their
values cleared.

The purpose of this change is to make it easier to share functionality
across sdk member types.

Bug: 142935992
Test: m nothing
Change-Id: Ie5160a8f854056920e411801ca20721eab7c8578
2020-03-13 10:47:14 +00:00
Paul Duffin
a04c107bfa Add support for multiple os types
Updates the member snapshot creation code to support multiple os types.
It basically sorts the variants by os type, then applies the code to
optimize the arch properties and then it optimizes the properties that
are common across architectures and extracts any properties that are
common across os types.

The java and cc member types needed to be modified to make the location
of the generated files within the snapshot os type dependent when there
is more than one os type. That was done by adding an OsPrefix() method
to the SdkMemberPropertiesBase which returns the os prefix to use when
there is > 1 os type and otherwise returns an empty string.

Added three tests, one for cc shared libraries, one for cc binary and
one for java header libraries.

Bug: 150451422
Test: m nothing
Change-Id: I08f5fbdd7852b06c9a9a2f1cfdc364338a3d5bac
2020-03-09 15:54:31 +00:00
Paul Duffin
8c3fec4c37 Enable androidmk processing in sdk testing
Previously, while sdk tests would pass the sdk code would often fail
in androidmk processing. This change makes the tests more realistic
and will catch the errors earlier.

Bug: 142935992
Test: m nothing
Change-Id: Ifd0b2d7cf24e941c919f6b6e0beb2403a67d4308
2020-03-06 09:45:54 +00:00
Nicolas Geoffray
1228e9c0ba Add a nice install paths for module SDKs and exports.
Following how NDK also creates its own install path.

Bug: 142935992
Test: cc_sdk_test.go, java_sdk_test.go
Change-Id: I98a3656903f37f6d7c90e6cf609431b2461a6161
2020-03-04 14:20:46 +00:00
Paul Duffin
e1ddcc9d5a Only check copy rules into the snapshot directory
Test: m nothing
Change-Id: I6d9ab5a9f003193c5791a4ee6d3ee35730ddf96c
2020-03-03 16:02:26 +00:00
Paul Duffin
1356d8c0f3 Add CommonOS variant for sdk
Adds a CommonOS variant for sdk that depends on the os specific
variants and is used to generate a single sdk for multiple OsTypes,
e.g. host linux and android.

At the minute the member types only support a single OsType but the
basic mechanism for managing the CommonOS variant and collating the
variants across all of them is there.

The only visible effect of this change is that the location of the
generated snapshot is changed, it is no longer os specific and instead
is in the same location irrespective of which os it is built for.

A lot of tests needed to be changed to specify "common_os" as the
variant type instead of the specific os type. As that is the same across
all tests it is hard coded in CheckSnapshot method.

Test: m nothing
Bug: 150451422
Change-Id: If36be39b06d6910453649f7c288c2d34f688b2f4
2020-03-02 19:31:26 +00:00
Paul Duffin
08798aae0b Add windows to the list of available OS's in sdk tests
Makes the tests more realistic as they cover what happens when an sdk
OS specific variant is disabled (Windows is disabled by default). This
will allow issues around disabled variants to be detected early during
Soong bootstrapping.

Test: m nothing
Change-Id: I7ec47dbca52e38750166d755daf706aadea12d15
2020-02-28 11:21:06 +00:00
Colin Cross
f28329de73 Move NDK test modules into cc/testing.go
Move the NDK modules defined in app_test.go into cc/testing.go in
preparation for using sdk_version: "current" in more tests.

Test: all soong tests
Change-Id: I80bc31225fb4562fce42781219cefbbd6affae06
2020-02-19 18:59:15 -08:00
Paul Duffin
7b81f5e9d7 Add java_system_modules to sdk/module_exports
Adds an SdkMemberType implementation for java_system_modules. It
specifies that java_system_modules can be used with sdk as well as
module_exports, and also that the libs property should be included
as transitive members in the sdk.

It also adds support for treating appropriate tagged properties in
the snapshot prebuilts module as references to sdk members so that
they are correctly transformed when creating the versioned modules.

Bug: 142940300
Test: m nothing
Change-Id: Ic10b5a6d5b92b6018334fe876f06feaf79cc55e9
2020-02-07 14:03:03 +00:00
Paul Duffin
c132742c96 Dedup package build components registration
Bug: 146540677
Test: m nothing
Change-Id: Iff2d7063b7f06313e9068c61a5627229463c98dd
2020-01-15 11:17:57 +00:00
Paul Duffin
8150da6e9d Added module_exports/_snapshot as alias for sdk/_snapshot
Bug: 146341462
Test: m nothing
Change-Id: I27e1ef494a2b0874074aa43614612189b17e7860
2019-12-30 18:03:30 +00:00
Paul Duffin
e2bc36f12c Merge "Generate .srcjar for prebuilt_stubs_sources" 2019-12-23 13:32:08 +00:00
Paul Duffin
6c26dc7392 Dedup cc library module type registration
Test: m checkbuild
Bug: 146540677
Change-Id: I6685338550f587212e70f3eba8d4342b66f566e2
2019-12-20 15:22:52 +00:00
Paul Duffin
9b478b0831 Generate .srcjar for prebuilt_stubs_sources
Changes prebuilt_stubs_sources to generate a .srcjar from its input
instead of just exposing the srcs it is given. This ensures that it can
be used as a drop in replacement for a droidstubs module.

Updates the test for prebuilt_stubs_sources to be more representative
of the actual use made of it by sdk snapshot which outputs a directory
not a glob pattern. Added some documentation of the
prebuilts_stubs_sources srcs property to make it clear that it is
supposed to be a set of directories.

Extracts common code from sdk/testing.go for normalizing path/paths
for testing.

Bug: 143678475
Test: m conscrypt-module-sdk conscrypt-module-host-sdk conscrypt-module-test-sdk
      unzip those in place of external/conscrypt
	  build core-current-stubs-source which expects it to provide a .srcjar.
Change-Id: I8204a022557a9b0b45e19eac79ecba98ff95213d
2019-12-20 08:13:45 +00:00
Paul Duffin
d686791c16 Improve cc tests by adding sanitizer mutators
This fixes an issue that arises when deduping cc build component
registration code.

The sanitize_runtime_deps and sanitize_runtime post deps mutators were
not previously added when running cc tests. That meant the tests were
not actually testing the same behavior as at runtime.

Adding the mutators breaks the TestFuzzTarget test as the mutator adds
libclang_rt.ubsan_standalone-aarch64-android as a dependency of libc++
and the former is not available.

This fixes the test by adding the missing dependency as a cc prebuilt
shared library.

Test: m checkbuild
Bug: 146540677
Change-Id: Ie13c7e6fcefef7d9cb1cc5364be3dc563ce40de5
2019-12-19 19:19:40 +00:00
Paul Duffin
0c4979bbdf Dedup prebuilt mutator registration
Test: m checkbuild
Bug: 146540677
Change-Id: I9f8a11adf61fe6f352ec1eabc7c5bac332348ba2
2019-12-19 19:19:40 +00:00
Paul Duffin
77980a8bb9 Dedup registration for cc default test config
The cc.GatherRequiredDepsForTest() method returns some default module
definitions that are required when using cc module types like
cc_library. Previously, the registration of the module types and
mutators needed to process those default definitions was duplicated
in the test config initialization.

This change removes that duplicated code and replaces it with calls
to cc.RegisterRequiredBuildComponentsForTest(ctx) which registers all
the required build components.

Test: m checkbuild
Bug: 146540677
Change-Id: I80b6913c5691ff164ce9d308b9e1da24940f2d42
2019-12-19 19:19:40 +00:00
Paul Duffin
59986b23cc Dedup cc prebuilts module type registration
Test: m checkbuild
Bug: 146540677
Change-Id: I7e9440a075ef9c683729ed83e0033ab529fe4ac0
2019-12-19 19:19:40 +00:00
Paul Duffin
884363e782 Dedup droiddoc module type registration
Some tests were changed as they were using the wrong name for the
droiddoc_exported_dir module type.

Test: m checkbuild
Bug: 146540677
Change-Id: If57404760d2df3757c93e237696359355cf0c96e
2019-12-19 19:19:40 +00:00
Paul Duffin
f9b1da0fcb Dedup registration code for module types and singletons
The registration of module types and singletons is duplicated between
init() functions that register them for use in the build runtime and
test context creation code that registers them for testing.

This is a proof of concept for a mechanism that will allow the code
to be shared. It defines a RegistrationContext interface that is
implemented by both the TestContext and the new initRegistrationContext
type. An instance of the the latter is available through the
InitRegistrationContext variable.

The intent is that the registration of the module types and singleton
types will be extracted from the init() function into a separate
function that takes a RegistrationContext parameter. That method is
called from init() passing in the InitRegistrationContext and from a
test passing in the TestContext. Something like this:

  func init() {
    RegisterBuildComponents(android.InitRegistrationContext)
  }

  func RegisterBuildComponents(ctx android.RegistrationContext) {
    ctx.RegisterModuleType(....)
    ....
  }

A test would do something like this:

  ctx := android.NewTestContext()
  RegisterBuildComponents(ctx)

Test: m nothing
Change-Id: I97173cabb6d6cf7ce98fdb5f73418438b1997b35
2019-12-19 10:09:53 +00:00