Commit graph

215 commits

Author SHA1 Message Date
Paul Duffin
a2ae7e0358 Allow java_sdk_library to specify doctags_files
When generating Javadoc the processor needs to be given information
about the doctags that are present in the source. This change allows
that information to be managed with the java_sdk_library that generates
the stubs source from which the Javadoc is generated.

Bug: 168301990
Test: Built offline-sdk-docs with and without the change and
      diffed them. The only difference was the timestamp.js
      file.
Change-Id: I4adbeb0781bc2191461fec856ffa90ea185e7434
2020-09-28 18:35:57 +01:00
Martin Stjernholm
79bee057a3 Merge "Handle property structs and BpPropertySets as values to AddProperty." 2020-09-28 10:06:04 +00:00
Martin Stjernholm
191c25f589 Handle property structs and BpPropertySets as values to AddProperty.
Both will create a nested property set, that may be merged with an
existing one.

Test: m nothing
Bug: 151303681
Change-Id: I30696ba3eb8960ca6fa54c9ee2cf6229ab9f5da9
2020-09-25 00:33:04 +01:00
Jiyong Park
1613e5541f HostCross is an attribute of a Target, not OsType
A host target is considered as being cross-compiled when the target
can't run natively on the build machine. For example, linux_glibc/x86_64
is a non-cross target on a standard x86/Linux machine, but is a cross
host on Mac. Previously, whether cross or not was a static attribute of
an OsType. For example, Windows was always considered as cross host,
while linux_bionic was not. This becomes a problem when we support more
host targets like linux_bionic/arm64 which should be cross-host on
standard x86/Linux machines.

This change removes HostCross from the OsClass type and instead adds a
property HostCross to the Target type. When a target is being added, it
is initialized to true when the target can't run natively on the current
build machine.

Bug: 168086242
Test: m
Change-Id: Ic37c8db918873ddf324c86b12b5412952b0f2be2
2020-09-24 14:14:31 +09:00
Paul Duffin
d6abaa71f8 Propagate recovery_available property through to sdk snapshot
Needed for jni_headers which is marked as recovery_available because libchrome
is also marked and that library is used by others that are the recovery
partition. There may be other dependency paths that require jni_headers to be
recovery_available that was just the first one found.

Bug: 168301990
Test: m nothing
Change-Id: I916ecfe7700b9c153b29115e849da148fc7e7a76
2020-09-11 16:09:28 +01:00
Paul Duffin
ee9ad5d526 Remove framework-modules naming scheme
The naming_scheme property was not removed as it may be useful for
future when migrating to java_sdk_library.

Bug: 168301990
Test: m nothing
Change-Id: Ie97dd60355a207f1312a2dd910f1fb25b46fd737
2020-09-11 16:09:15 +01:00
Paul Duffin
d1edbd4eed Propagate vendor_available property through to sdk snapshot
Test: m nothing
Change-Id: I68dcba111d290ae8a84c2bc023e1a4e0eaa43408
2020-09-03 18:49:35 +01:00
Treehugger Robot
4c2e4f381a Merge "Support ninja rsp files in soong_zip" 2020-08-24 18:40:56 +00:00
Martin Stjernholm
fbb486fc7a Propagate sanitize.never flag in SDK snapshots.
This is necessary to not get dependencies on libclang_rt sanitizer libs
for the CRT objects which lead to cyclic dependencies in sanitizer
builds.

Test: `lunch aosp_blueline_hwasan-userdebug && m nothing`
  in a tree that has the prebuilts created and unzipped from
  `build/soong/scripts/build-aml-prebuilts.sh runtime-module-{sdk,host-exports}`
Bug: 151303681
Change-Id: I3f848a084280bdc3ade4b74df03e981d8cc61222
2020-08-21 18:43:51 +01:00
Colin Cross
053fca10c9 Support ninja rsp files in soong_zip
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.

Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
2020-08-19 21:18:56 +00:00
Colin Cross
aede88c1c7 Reland: Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-19 10:21:17 -07:00
Pete Bentley
0c7b26e0e2 Revert "Deduplicate APEX variants that would build identically"
This reverts commit d6b2525b00.

Reason for revert: <Breaks tests - confirmed by Forrest: b/165188843>

Change-Id: I7cb68cb87522415004390c0672dc774e0067b122
2020-08-18 13:44:59 +00:00
Colin Cross
d6b2525b00 Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I0529837476a253c32b3dfb98dcccf107427c742c
2020-08-17 15:18:31 -07:00
Martin Stjernholm
caa47d7eed Disable all host OS variants except those explicitly included.
Necessary to avoid problems with implicitly enabled host OS'es, e.g.
linux_glibc getting enabled when we only supply a linux_bionic linker
in runtime-module-host-exports. That will then cause a non-functional
prebuilt to take precedence over source if the prebuilt is preferred.

We don't do this for device since we so far only support a single
device OS (android).

This introduces the notion that SDK member types can be host OS
dependent or not. That way java members with host prebuilts don't get
restricted to a specific host OS.

Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh runtime-module-host-exports
  Check that the generated Android.bp correctly disables the bionic
  linker prebuilt for linux_glibc.
Test: art/build/apex/runtests.sh
  on master-art with an updated runtime SDK snapshot
Test: art/tools/buildbot-build.sh {--host,--target}
  on master-art with an updated runtime SDK snapshot
Bug: 160349757
Change-Id: Idad7ef138cdbcbd209d390bf6c10ca8365d4619f
2020-08-03 21:06:50 +01:00
Martin Stjernholm
b4bfd148eb Merge "Set compile_multilib for SDK snapshots for device." 2020-08-03 17:55:25 +00:00
Paul Duffin
021f4e525f Fix prebuilt mutator ordering in tests
Previously, the prebuilt mutators were added by the
cc.RegisterRequiredBuildComponentsForTest() function as a convenience
but unfortunately it lead to some of the mutators being in a different
order in the tests than in the normal build.

This change:
* Extracts the RegisterPrebuiltMutators() call from
  cc.RegisterRequiredBuildComponentsForTest()
* Makes sure that the prebuilt mutators are registered before the
  visibility gatherer and enforcer mutators.

Bug: 162505935
Test: m nothing
Change-Id: I7d959b558200b502f0a5e4653c41ea01414e142a
2020-07-31 16:12:01 +01:00
Paul Duffin
a37eca27c8 Avoid creating APEX variant for sdk member
Previously, an APEX variant was created for a module that was a member
of an SDK just in case it had to be replaced with an APEX requested
snapshotted version of that member. That was necessary because that was
the only way to have APEX specific replacements.

Since then a new method called ReplaceDependenciesIf() has been added
which provides fine grained control over which dependencies are
replaced. This change uses that new method to handle the replacements
which allows the APEX variants to be removed.

Bug: 161928524
Test: m nothing
Change-Id: If3869dd6753dc182b099af566b20fbc9c9c6eff7
2020-07-24 11:10:10 +01:00
Pete Gillin
84c3807b99 Enforce the legacy core/platform API restriction.
This change silently decides whether modules which depend on either
sdkPrivate or sdkCorePlatform get the legacy or the stable version of
the core/platform API, based on whether the module's name is on a
hard-coded list or not.

Test: m java
Test: make a target from the list when its entry is commented out, which correctly fails
Bug: 157640067
Change-Id: I15e5a6c2f07e73718803501d705de0d7ab9bec90
Merged-In: I15e5a6c2f07e73718803501d705de0d7ab9bec90
Merged-In: Iaa97ddaa015e8079fcb3426585c5101c7ec9e22a
(cherry picked from commit c0f4373106)
2020-07-23 09:06:52 +01:00
Treehugger Robot
0bcda81dd5 Merge "Generate module lib API and scope together" 2020-07-21 04:21:07 +00:00
Paul Duffin
15f34ef811 Generate module lib API and scope together
Previously, due to limitations in metalava the module lib API and stubs
source were generated separately. That limitation has since been
removed so this change optimizes the generating code.

Changes:
* Add an annotation field to apiScope to contain the annotation that
  identifies the scope.
* Rename droidstubArgs to extraArgs to better reflect that they are not
  passed to droidstubs.
* Rename droidstubsArgsForGeneratingStubsSource to droidstubsArgs.
* Remove droidstubsArgsForGeneratingApi and
  createStubsSourceAndApiTogether fields along with code that supports
  generating stubs source and the API separately.
* Fix the module lib test.

Test: m checkapi
Bug: 161364853
Change-Id: I821886bb68645addf5ae9c96ed2f4f3649151d1a
2020-07-20 20:51:25 +01:00
Martin Stjernholm
4cfa2c692b Set compile_multilib for SDK snapshots for device.
Previously compile_multilib was ignored for device and defaulted to
"both", which could lead to an internal inconsistency in the snapshot
if a member was 32- or 64-only.

This also implements common property optimisation for compile_multilib,
by adding a struct that can be extended to other properties in the
future.

Test: m nothing
Bug: 151303681
Change-Id: I3e96222858e5e755173aca03508a23c431977e2a
2020-07-17 14:32:15 +01:00
Martin Stjernholm
89238f4703 Add compile_multilib properties to all native prebuilts (reland).
There was a special case in cc_prebuilt_binary only, which resulted in
prebuilt libraries getting both 32 and 64 bit variants even when their
sources only had one of them, and the other variant would be defunct
since it wouldn't get any prebuilt artefact.

This moves the handling of compile_multilib completely to the common
update code, so that SDK members don't need to deal with it. It doesn't
take SDK member defaults into account, which means a bit more
boilerplate in the snapshots, but it's simpler and less error prone
(different SDK member types have different defaults).

This relands https://r.android.com/1359962 after disabling the SDK
tests on mac once and for all.

Bug: 151303681
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I05f6603b2ac9b8676b25c3e297165ca23284e9cc
2020-07-17 14:32:15 +01:00
Martin Stjernholm
7feceb2354 Add support for SDK tests with LinuxBionic OS type.
Break apart test helpers a bit to make it possible to enable
LinuxBionic in a single test, and add LinuxBionic support to
cc.GatherRequiredDepsForTest.

Test: m nothing
Bug: 160349757
Change-Id: Iace1024c964cee2308c130c945daef9e46c18c66
2020-07-16 21:55:15 +01:00
Martin Stjernholm
3ff2e661bd Disable all SDK snapshot generation tests on mac.
We frequently miss putting in the per-test skip condition, and since
mac isn't tested in presubmit this leads to build breakages and time
consuming reverts. To avoid that this blanket disables all the SDK
tests on mac. It's not used there and we got test coverage in many
linux-based builds.

Bug: 145598135
Bug: 161315642
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I2aea92fef2c0f8c2742396fe36610501dc5a6f0f
2020-07-15 16:58:05 +01:00
Jakub Pawlowski
ca7b2c3eb7 Merge "Revert "Add compile_multilib properties to all native prebuilts."" 2020-07-15 10:33:41 +00:00
Charles Chen
29936ae5a7 Revert "Add compile_multilib properties to all native prebuilts."
This reverts commit 1e9c2677fd.

Reason for revert: Break build

Bug: 161315642
Change-Id: Ibf3d5e88412327e8f08d08272b75cd3df34a71e5
2020-07-15 04:01:45 +00:00
Treehugger Robot
6da69732e4 Merge "Add compile_multilib properties to all native prebuilts." 2020-07-15 02:51:21 +00:00
Treehugger Robot
9d98ac15bb Merge "Fix string concatenation error" 2020-07-12 23:08:40 +00:00
Martin Stjernholm
1e9c2677fd Add compile_multilib properties to all native prebuilts.
There was a special case in cc_prebuilt_binary only, which resulted in
prebuilt libraries getting both 32 and 64 bit variants even when their
sources only had one of them, and the other variant would be defunct
since it wouldn't get any prebuilt artefact.

This moves the handling of compile_multilib completely to the common
update code, so that SDK members don't need to deal with it. It doesn't
take SDK member defaults into account, which means a bit more
boilerplate in the snapshots, but it's simpler and less error prone
(different SDK member types have different defaults).

Bug: 151303681
Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh art-module-host-exports
  Check that the generated Android.bp passes Soong.
Change-Id: Ib73444c6788ee1c78480bdb103aa2b8ae8f2c63c
2020-07-11 00:38:24 +01:00
Colin Cross
afa6a77c64 Fix string concatenation error
Fix an error caught by golint that prevents using go test.  string(int)
produces a rune, not a string representation of the int.  Use fmt.Errorf
instead to format the string.

Test: go test
Change-Id: I4c2e1e30a0f1b3dc24afd123c38c4e41c2abb47a
2020-07-10 16:32:49 -07:00
Paul Duffin
ed62b9cc2a Differentiate sdk snapshot include dir by os type
Test: m nothing
Change-Id: Ide032aa64bb1fee69969c5e007bee6643d475309
2020-07-10 12:18:03 +01:00
Paul Duffin
cee7e66b07 Ensure that sdk/module_exports depends on source members
Previously, preferring a prebuilt of an sdk/module_exports's member
would cause the sdk/module_exports to depend on the prebuilt instead
of the source and cause problems with the build.

This chance prevents the dependency from an sdk/module_exports to its
members from being replaced with prebuilts.

Bug: 160785918
Test: m nothing
Change-Id: Iee4bcd438c11929e30fb5766701b05a0e89956d9
2020-07-09 17:51:06 +01:00
Treehugger Robot
2c79c871d9 Merge changes Iefcf4fbe,I450d4769,I1ffc7e7b
* changes:
  Prepend default to allow overriding in the bp file.
  Do not follow SDK member dependencies for APEX payloads.
  Do not follow prebuiltDependencyTags in APEX payload walks.
2020-07-09 12:32:31 +00:00
Paul Duffin
44f1d8404b Stop java_sdk_library_import from depending on source modules
Previously, java_sdk_library_import added the dependencies on its child
components in the deps mutator after prebuilts without a matching
source module are renamed to the source module name. That meant that
the java_sdk_library_import has to use the source module name and ended
up depending on the source module unless it was preferred.

This change adds a new component deps mutator that runs before the
PrebuiltRenameMutator so that the java_sdk_library_import can add
dependencies onto the prebuilt modules. It also updates an affected
test.

Bug: 159902351
Test: m nothing
Change-Id: I3576c4873302743e51aff547ea1497bef6d748ac
2020-07-08 16:46:16 +01:00
Martin Stjernholm
26ab8e80e4 Prepend default to allow overriding in the bp file.
Bug: 143948100
Bug: 151303681
Test: m nothing
Change-Id: Iefcf4fbe9d2971ce267242185165f0c38f68db30
2020-07-07 18:14:26 +01:00
Martin Stjernholm
cc77601d10 Do not follow SDK member dependencies for APEX payloads.
Test: m nothing
Test: `m` with prebuilts/runtime in the manifest (along with other
  fixes)
Bug: 151303681
Change-Id: I450d476975c7ab4434228b8c4baf3af192142211
2020-07-07 18:14:26 +01:00
Jingwen Chen
40fd90ae52 Support multiple dists per Android.bp module, and dist output selection.
This CL adds "dists" to the base property struct to support multiple
dist file configurations, and generic tag support to dist tagged outputs
of modules.

Fixes: b/152834186
Test: soong tests and `m sdk dist`

Change-Id: I80c86bc9b7b09e671f640a4480c45d438bdd9a2a
Signed-off-by: Jingwen Chen <jingwen@google.com>
2020-06-25 12:42:07 +00:00
Paul Duffin
4e77284ab4 Include resources in sdk snapshot of java library impl
Bug: 159607838
Test: m nothing
Change-Id: Ie7ddbffc3957f832abfadb27292c436e8e5dcbf3
2020-06-24 13:04:15 +01:00
Martin Stjernholm
47ed352227 Propagate unique_host_soname to cc_library prebuilts.
Test: m nothing
Test: Check on go/Android.bp that cc_prebuilt_library supports
  unique_host_soname
Bug: 158743135
Change-Id: Ie02a5cae057fb9092c226d8c5b7f63e1f66ad066
2020-06-17 23:06:43 +01:00
Paul Duffin
0c5bae5e83 java_sdk_library: Add system-server scope
Bug: 155164730
Test: m nothing
Merged-In: I49a2dab5c064b05f16691a3fae65f2b4ffc53bfd
Change-Id: I49a2dab5c064b05f16691a3fae65f2b4ffc53bfd
(cherry picked from commit 5a757b1ebb)
2020-06-15 15:02:19 +01:00
Colin Cross
440e0d0542 Use inclusive language in build/soong
Test: m checkbuild
Change-Id: Id07890b7cbc2397291a658ca00e86b43c743aafc
2020-06-11 15:33:16 -07:00
Pete Gillin
1f41dbff64 Split the core/platform API into stable and legacy versions.
For now, everything outside libcore still uses the legacy version.

Test: treehugger
Bug: 157640067
Change-Id: If5234e9ee533ff537926801a0af045d36b1caf01
2020-06-08 10:53:51 +01:00
Jaewoong Jung
264700345d Merge "Soong package structure refactoring" 2020-06-02 16:30:50 +00:00
Paul Duffin
3dbf9fd892 Copy removed.txt file to the snapshot correctly
Previously, the code copied the current.txt file to both the
current_api and removed_api properties. This change copies the
removed.txt file to the removed_api property instead.

Bug: 157980685
Test: m nothing
Change-Id: Iad34e91051da43222d22c240c16f50887c43d73e
2020-06-02 14:40:28 +01:00
Jaewoong Jung
4b79e98a6e Soong package structure refactoring
Give prebuilt_etc and sh_binary their own packages and split the
gigantic main Android.bp up to small, per-package ones.

Test: m nothing, TreeHugger
Bug: 156980228
Change-Id: I7b00cd344b9f16861f1ff39edf0029f016b853d0
2020-06-01 13:44:48 -07:00
Martin Stjernholm
a6a224586b Merge "Propagate flags necessary for a prebuilt of the Bionic linker." 2020-06-01 09:55:10 +00:00
Martin Stjernholm
7130fabd20 Propagate flags necessary for a prebuilt of the Bionic linker.
The flags are necessary to avoid implicit dependencies on crt libs,
libc, and the linker (recursively). The reason cc_prebuilt_binary react
to these flags is that they can affect the install rules.

Test: m nothing
Test: art/test/testrunner/run_build_test_target.py art-linux-bionic-x64
  on a master-art branch using snapshot built from runtime-module-sdk
  and runtime-module-host-exports
Bug: 152255951
Change-Id: I6b2fe92d105d1f446fffd00bd3267a416f75efb7
2020-05-28 22:58:01 +01:00
Paul Duffin
d7eb1c2a76 java_sdk_library: Propagate shared_library to snapshot
Passes the shared_library property through to the snapshot. It does not
optimize away the default value in order to make it easier to invert
the default value in future. The current default value was only chosen
for convenience because most existing usages were already treated as
shared libraries. It would be safer if modules had to opt in to be used
as shared libraries.

Bug: 155164730
Test: m nothing
Change-Id: I33c7323f2389b44ed49cebe517ae2cce349117f1
2020-05-28 17:39:25 +01:00
Paul Duffin
6b3ca917b0 Merge "Syntax check generated Android.bp snapshot" 2020-05-19 07:53:27 +00:00
Treehugger Robot
e3fa145692 Merge "Fix special case when appending a visibility rule to //visibility:private." 2020-05-15 14:08:24 +00:00