Commit graph

16873 commits

Author SHA1 Message Date
Alexander Smundak
863d5ded9d Merge "Expand namespaces explanation." 2019-12-10 00:06:20 +00:00
Andrew Chant
ed75f18e25 Merge "Revert "soong: upgrade Android platform to clang-r370808"" 2019-12-09 23:37:23 +00:00
Treehugger Robot
455e3a089d Merge "Move/dedupe some host path functions in package_ctx.go." 2019-12-09 23:20:16 +00:00
Treehugger Robot
8fc9eeacc8 Merge "Fix generated headers of snapshot" 2019-12-09 23:15:55 +00:00
Martin Stjernholm
7260d066ea Move/dedupe some host path functions in package_ctx.go.
These functions don't need a PackageContext object.

Test: m
Bug: 145934348
Change-Id: Ia1da2c76cbad292f9ca79617199b2b5b6b265568
2019-12-09 22:04:21 +00:00
Sasha Smundak
d42609e15c Expand namespaces explanation.
Test: N/A
Change-Id: Iec7d40e6886285046fbe00aea8e19073c6c1ba73
2019-12-09 13:26:09 -08:00
Nick Desaulniers
862eb4648a Revert "soong: upgrade Android platform to clang-r370808"
This reverts commit 4f49e35902.

Causes test failures in ndk_translate.
Bug: 145807809
Bug: 145827049
Test: N/A
Change-Id: I8f42a15fec44475b74b1926c70bc98e03fe66f52
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
2019-12-09 19:16:08 +00:00
Paul Duffin
593b3c9fb0 Ensure prebuilt modules have same visibility as source modules
Exports visibility and package mutator registration functions so they
can be used in sdk testing. Updates sdk test to support visibility and
package modules.

Adds EffectiveVisibility(...)[]string function to make the effective
visibility rules available to sdk snapshot creation.

Extracts compositeRule.Strings() []string from compositeRule.String()
method so that it can be used by above func.

Adds visibility property to sdk snapshot and prebuilt modules along
with a test to ensure it works properly.

Adds dir parameter to CheckSnapshot so that it can check the snapshot
generated for a non-root package. That is required in order to ensure
that visibility of :__subpackages__ on a source module in package
<pkg> is resolved to an effective visibility of
//<pkg>:__subpackages__ on its corresponding prebuilt.

Test: m conscrypt-module-sdk
Bug: 143678475
Change-Id: Icaacac5b9c04726d28e6fec93e49715ac45df7f4
2019-12-09 13:32:28 +00:00
Paul Duffin
9d8d609fcd Pass SdkMember to AddPrebuiltModule instead of the name
This is needed for a follow up change that makes sure that the
prebuilt modules have the same visibility as the source modules.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I9461c8c094ab19ee9ececb5e5fd50565789f2fa2
2019-12-09 13:28:36 +00:00
Inseob Kim
d110f878a7 Fix generated headers of snapshot
For VNDK snapshot and SDK snapshot, deps files have been used to capture
generated headers. But exported deps might contain intermediate phony
files instead of actual header files, which are for optimization of
ninja. To correctly capture all headers, exported generated header files
are gathered separately.

Bug: 65377115
Test: m nothing
Change-Id: Ia03fa69186490a818578190e3c0bfb0261d1fd6e
2019-12-09 12:38:34 +09:00
Treehugger Robot
667039f404 Merge "Cleanup java.DefaultsFactory()" 2019-12-07 08:52:21 +00:00
Treehugger Robot
8cda6d93de Merge "Support header and implementation jars in sdk" 2019-12-07 08:52:09 +00:00
Treehugger Robot
e2f3b5dfe5 Merge "Move ImageMutator after archMutator" 2019-12-07 00:37:30 +00:00
Jeff Sharkey
e2e4a5dd56 Merge "Overridden APEX doesn't contribute to the file contexts" 2019-12-06 23:02:01 +00:00
Elliott Hughes
b5d11197d3 Merge "Enable -Wpragma-pack warnings as errors." 2019-12-06 22:07:20 +00:00
Colin Cross
ae6c5207cc Move ImageMutator after archMutator
Move the ImageMutator to be registered just after the archMutator
in preparation for moving it between osMutator and archMutator.
Requries updating variants in a few tests that now run the
ImageMutator.

Bug: 142286466
Test: no change to build.ninja
Test: all soong tests
Change-Id: Ia9d2a7bc0e225bedec3c9a83ea04f471a931bf47
2019-12-06 12:37:14 -08:00
Treehugger Robot
f3c24c12ef Merge "Link type checking for java_library" 2019-12-06 18:59:48 +00:00
Dan Willemsen
ad1997eebc Merge "Add sharding support to multiproduct_kati" 2019-12-06 17:40:23 +00:00
Ulyana Trafimovich
a4f02412a1 Merge "Pass correct --image argument to oatdump." 2019-12-06 16:55:39 +00:00
Paul Duffin
4735766ef8 Cleanup java.DefaultsFactory()
This function took a props varargs parameter but it was only ever
called with no parameters. The props parameter meant it could not be
used directly with RegisterModuleType() so the defaultsFactory() func
was added which simply called through to DefaultsFactory().

Removing the props varargs parameter allowed it to be used directly
with RegisterModuleType() and so the defaultsFactory() could be
removed.

This was needed because a follow up change adds a test in another
package that uses java_defaults and so it needs the factory method
(defaultsFactory()) to be exported.

Test: m nothing
Change-Id: I33d5c4d4ce8f349b1e6fc706d5e9656faf303b4f
2019-12-06 16:13:58 +00:00
Paul Duffin
a0dbf43a82 Support header and implementation jars in sdk
Add a new java_header_libs property that exports header jars (as
java_libs currently does) and switch java_libs to export implementation
jars instead.

Refactors implementation of the existing library sdk member type so
both properties can be supported from common code as they differ only
on the jar being exported.

Bug: 143678475
Test: m nothing
Change-Id: I04642122f72d083bbdfd3290624f957b71ee8875
2019-12-06 16:13:58 +00:00
Paul Duffin
d835daaade Separate out module type specific test config
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I71433a7c89dfae4c27537f1f66bd677cf9fba0b3
2019-12-06 16:13:58 +00:00
Treehugger Robot
3a7caaa80c Merge changes Ia7dbcd41,I93140450,I4d3fe18f,Ib00870dd,I10203594
* changes:
  Organize sdk member properties
  Extract the cc and java sdk related tests out into their own file
  Improve testing of sdk snapshot generation
  Separate sdk testing infrastructure from sdk tests
  Parameterize the sdk member processing
2019-12-06 15:55:24 +00:00
Paul Duffin
7467a714b0 Merge "Use static build rules in snapshot generation" 2019-12-06 15:52:00 +00:00
Jeongik Cha
e403e9e8dc Link type checking for java_library
Link type checking for java_library has not been working unintentionally.
So turn on link type checking for these types.

And also add tests for link type checking.

Bug: 145799020
Test: cherry-pick aosp/1182522 and check if build fails
Test: m nothing and there is no error(soong unittest)

Change-Id: Ifc347f657885de1028ac0076ddd103c0387b597a
2019-12-07 00:45:02 +09:00
Anton Hansson
d57bd3c84f Revert "Move odexes of non-system apps into system_other"
This reverts commit 12b8d427c7.

Bug: 142537657
Test: No
Change-Id: Iefd882ec11c060877671f5bf0f2429ba4d811f8b
Merged-In: Iefd882ec11c060877671f5bf0f2429ba4d811f8b
2019-12-06 13:42:25 +00:00
Ulya Trafimovich
163664a548 Pass correct --image argument to oatdump.
This was broken after CL:
  Ida40dfae8c83bf7c2e737d5c7ea418e1197ad826
..which split boot image into primary and extension.
The argument to oatdump was not updated properly.

The old form before the split was:

  --image=out/soong/generic_x86/dex_bootjars/system/framework/boot.art

The new form after the split is:

  --image=out/soong/generic_x86/dex_artjars/apex/com.android.art/javalib/boot.art:out/soong/generic_x86/dex_bootjars/system/framework/boot-framework.art

Test: lunch aosp_x86-userdebug && m dump-oat

Test: Manually inspect build command to make sure correct image
  location is used:

  $ fgrep -e 'oatdump' $ANDROID_BUILD_TOP/out/soong/build.ninja | egrep -e '--image[^ ]*' -o
  --image=out/soong/generic_x86/dex_artjars/apex/com.android.art/javalib/boot.art:out/soong/generic_x86/dex_bootjars/system/framework/boot-framework.art

Change-Id: I54a5146c404aefb6eaeadbe3e9363db935ecad86
2019-12-06 13:42:21 +00:00
Paul Duffin
a6e737b078 Organize sdk member properties
Grouping in alphabetical order by package and then by name within the
package should minimize conflicts when making changes.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: Ia7dbcd41ce8b8dd8675a90b1b6868fcaeaf72ee4
2019-12-06 12:16:59 +00:00
Paul Duffin
a80fdec00d Extract the cc and java sdk related tests out into their own file
Makes it easier for multiple people to work on different areas without
conflicts.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I93140450c0b049946a9d0432225bccff82f5a873
2019-12-06 12:16:59 +00:00
Paul Duffin
c3c5d5e351 Improve testing of sdk snapshot generation
Adds TestHelper to provide general test helper functionality for use by
any test.

Adds testSdkResult, composed with TestHelper that encapsulates the
result of processing and sdk {..} definition and provides specialized
support for testing the build rules.

Dedups the analysis of the sdk build rules, and improves it to extract
more information, and in different forms. That is represented by the
snapshotBuildInfo struct.

Adds a CheckSnapshot() method which checks the snapshot for a specified
sdk version. It takes a list of functions that can each perform a check
on a specific facet of the supplied snapshotBuildInfo.

Methods are provided for tests to use to check the following facets:
* Generated Android.bp contents.
* Copy rules
* Merge zip inputs

This approach makes it possible for each test to customize what is
being checked without either duplicating functionality, causing a
proliferation of specialized forms of the CheckSnapshot method for
different types of tests or adding arguments for every possible check
that any test would need which would lead to lots of churn to existing
tests when new arguments are added.

The main testing improvement is for CheckSnapshot() to actually try and
load the Android.bp that is generated. In order to do that it was
necessary to create a mock filesystem populated with information from
the build rules, i.e. the destination files from every Cp command as
well as the destination directory from every repackage zip command.

That helps detect a number of sources of errors:
* Failing to copy a file/directory that is mentioned in the generated
  Android.bp file.
* Invalid properties.
* Invalid format of the .bp file.
* Integrity issues within the .bp file.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I4d3fe18f86698186d18e7e8b32d2e319183f7f0c
2019-12-06 12:16:59 +00:00
Paul Duffin
82d90438be Separate sdk testing infrastructure from sdk tests
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: Ib00870ddefc4c1dd9e42ca594e6ebe8e24c42e05
2019-12-06 12:16:59 +00:00
Paul Duffin
1387957727 Parameterize the sdk member processing
Extracts the type specific functionality into the SdkMemberType
interface which has to be implemented by each module type that can
be added as a member of the sdk. It provides functionality to add
the required dependencies for the module type, check to see if a
resolved module is the correct instance and build the snapshot.

The latter was previously part of SdkAware but was moved because
it has to be able to process multiple SdkAware variants so delegating
it to a single instance did not make sense.

The custom code for handling each member type specific property,
e.g. java_libs, has been replaced with common code that processes
a list of sdkMemberListProperty struct which associates the
property (name and getter) with the SdkMemberType and a special
DependencyTag which is passed to the SdkMemberType when it has to add
dependencies.

The DependencyTag contains a reference to the appropriate
sdkMemberListProperty which allows the resolved dependencies to be
grouped by type.

Previously, the dependency collection methods would ignore a module if
it was an unsupported type because they did not have a way of
determining which property it was initially listed in. That meant it
was possible to add say a droidstubs module to the java_libs property
(and because they had the same variants) it would work as if it was
added to the stubs_sources property. Or alternatively, a module of an
unsupported type could be added to any property and it would just be
ignored.

However, the DependencyTag provides information about which property
a resolved module was referenced in and so it can detect when the
resolved module is of the wrong type and report an error. That check
identified a bug in one of the tests where the sdk referenced a
java_import module (which is not allowed in an sdk) instead of a
java_library module (which is allowed). That test was fixed as part
of this.

A list of sdkMemberListProperty structs defines the member properties
supported by the sdk and are processed in order to ensure consistent
behaviour.

The resolved dependencies are grouped by type and each group is then
processed in defined order. Within each type dependencies are grouped
by name and encapsulated behind an SdkMember interface which includes
the name and the list of variants.

The Droidstubs and java.Library types can only support one variant and
will fail if given more.

The processing for the native_shared_libs property has been moved into
the cc/library.go file so the sdk package code should now have no type
specific information in it apart from what is if the list of
sdkMemberListProperty structs.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I10203594d33dbf53441f655aff124f9ab3538d87
2019-12-06 12:16:59 +00:00
Paul Duffin
375058f67d Use static build rules in snapshot generation
It is easier to extract information out of static build rules than it
is out of custom build rules built using the builder as the former
provides access to the in/out and args separate from the rule whereas
the latter only provides access to in/out.

Also, cleans up some warnings that appear in Intellij.

There is a lot of duplication in the testing code. That will be
resolved in a follow up change.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I973bc0c90b0affd84487f1b222dd3e6c22c07ec0
2019-12-06 12:16:53 +00:00
Treehugger Robot
cea93afe76 Merge "apex_available is defaultable" 2019-12-06 04:23:37 +00:00
Dan Willemsen
9609ad9c21 Add sharding support to multiproduct_kati
This is so that we can split the build_test build on large branches into
multiple builds, each testing a subset of the products.

Test: m blueprint_tools  (runs the go unit tests)
Test: build/soong/build_test.bash --only-config
      vs
Test: build/soong/build_test.bash --only-config --shard-count=2
Test: build/soong/build_test.bash --only-config --shard-count=2 --shard=2

Change-Id: I40ccc1aa477bc0ffa74ff564d155068509be18f0
2019-12-05 19:54:40 -08:00
Ivan Lozano
1fdb0b55a5 Merge "Exclude dirs from binder interface whitelisting" 2019-12-06 02:25:10 +00:00
Treehugger Robot
337cd8e493 Merge "Fix: vendor and odm are restricted to use public platform properties" 2019-12-06 02:00:11 +00:00
Treehugger Robot
6cfd3f0319 Merge "Add CUJ tests" 2019-12-06 01:17:04 +00:00
Calin Juravle
af7c2fcb71 Merge "Export art dexpreopt image zip file" 2019-12-05 22:11:28 +00:00
Treehugger Robot
5eb62bc2fe Merge "Fix cc_fuzz ignoring soong namespaces." 2019-12-05 21:41:41 +00:00
Nick Desaulniers
ac0336d81f Merge "soong: upgrade Android platform to clang-r370808" 2019-12-05 20:12:08 +00:00
Colin Cross
d0be2109cf Add CUJ tests
Add a cuj_tests binary that runs through a sequence of critical
user journey builds and collects metrics.

Bug: 145688720
Test: cuj_tests
Change-Id: I6bb1a3fc4a458fd8cea7b7c924f565d0cf3e6df3
2019-12-05 11:11:37 -08:00
Mitch Phillips
6a9bf214f9 Fix cc_fuzz ignoring soong namespaces.
cc_fuzz ignored soong namespaces, and built all cc_fuzz modules into the
architecture package. This caused naming conflicts where two modules had
the same name but were in different namespaces.

Now, cc_fuzz will only build modules that are in the same namespace as
the product definition - resolving the duplicate definitions error.

Bug: 145240569
Test: m haiku
Change-Id: I0a0acd139426301586ac5e3aba8ce5527cb7a1b5
2019-12-05 09:38:56 -08:00
Treehugger Robot
bed70f837b Merge "Test par file argument handling" 2019-12-05 16:23:56 +00:00
Treehugger Robot
dc0e8fd131 Merge changes Id261b5a3,Ice6f34d0
* changes:
  Fix errorProtoLog error messages
  Write BUILD_DATETIME_FILE after SetupOutDir
2019-12-05 09:10:13 +00:00
Treehugger Robot
5f9e5244c5 Merge "Remove passing -forceprocessing flag to R8" 2019-12-05 08:56:38 +00:00
Jiyong Park
b326ac2b65 Fix: vendor and odm are restricted to use public platform properties
There was a mistake in the condition.

Bug: N/A
Test: N/A

Change-Id: I07f79fdcc3c946a4b9e957e5a42b6694b5eeb251
2019-12-05 13:35:35 +09:00
Jiyong Park
317645e84c Overridden APEX doesn't contribute to the file contexts
This change fixes the problem that when an apex module is overridden by
another override_apex, the <apex_name>-file_contexts are duplicated when
creating the system-level file-contexts.

Fixing this by not emitting the file_context info for the overridden
apex.

In doing so, OverridableModule interface was extended to have
GetOverriddenBy() method which can be used to test whether a module is
an overridden one or not.

Bug: 144338929
Test: m (apex_test amended)
Test: add "override_apex {name:"com.googlge.android.tzdata",
Change-Id: I5e9401c32899bb9987c90cba4185f571dc1a87f0
base:"com.android.tzdata"}" and the build is successful
2019-12-05 13:25:39 +09:00
Colin Cross
31bf00dac3 Export art dexpreopt image zip file
Test: m  out/target/product/generic_arm64/boot.zip
Bug: 144491400
Change-Id: I84342703ac614e072b0cde8c6f5597dc8cffbc65
2019-12-04 16:50:52 -08:00
Treehugger Robot
7b34ebf447 Merge "Remove implicit outputs for the flattened APEX Make module" 2019-12-05 00:21:29 +00:00