Commit graph

24870 commits

Author SHA1 Message Date
Jiyong Park
89f7bf1635 bootstrap is added for cc_binary only when the module is in an APEX
Like shared libraries, the bootstrap subdirectory is appended to the
install path only when the module is in an APEX. Currently, this change
doesn't make any difference because only the Bionic binary (linker) is
satisfying the existing condition InstallToBootstrap() and it already
satisfies DirectlyInAnyApex.

However, this change makes a diference when we build the binary for the
host targets. Then without this change, it is installed to
out/soong/host/<OS>-<arch>/bin/bootstrap. This doesn't make sense
because we don't use APEXes for the host targets. With this change,
the binary is correctly installed to out/soong/host/<OS>-<arch>/bin
because DirectlyInAnyApex returns false for hosts.

Bug: 159685774
Test: m
Change-Id: I0d63bd8c11e3a96ee6a27b2295bc14d6dc15ff9e
2020-08-25 18:57:39 +09:00
Jingwen Chen
ce3d46fe05 Bazel overlay: rename 'deps' to 'module_deps'.
'deps' is a module prop used by some module types, like
prebuilt_build_tool:

https://cs.android.com/android/platform/superproject/+/master:build/soong/android/prebuilt_build_tool.go;l=25-26;drc=751ae87ed29c67629f068a8590af80023a457294

Use `module_deps` to prevent conflicts in the generated overlay
attributes.

Bug: 162720644
Test: m bazel_overlay && cd out/soong/bazel_overlay && bazel cquery
//...
Test: soong_build tests

Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: Ia075a860983960b2d9121d30ead8f808be7e0c5b
2020-08-25 06:11:53 +00:00
Jingwen Chen
dc4df2dd4a Merge "Surface module properties as Bazel BUILD target attributes in the Bazel overlay" am: 81a0b66d93
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1393254

Change-Id: Iaaf6ce5bd5e9aacc2ec577bbb60fe2544ad945ba
2020-08-24 23:40:11 +00:00
Jingwen Chen
81a0b66d93 Merge "Surface module properties as Bazel BUILD target attributes in the Bazel overlay" 2020-08-24 22:59:11 +00:00
Treehugger Robot
1e4fbc23bf Merge "Support ninja rsp files in soong_zip" am: 4c2e4f381a
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1405368

Change-Id: I99276f84d12144624385012f8f5593c02d990d60
2020-08-24 19:33:37 +00:00
Treehugger Robot
4c2e4f381a Merge "Support ninja rsp files in soong_zip" 2020-08-24 18:40:56 +00:00
Treehugger Robot
a5c0b11970 Merge "Propagate sanitize.never flag in SDK snapshots." am: e819bec240
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1406917

Change-Id: Ia8a02f8570ca0619c1b4696aa72c2dd1a2a393ee
2020-08-24 17:31:15 +00:00
Liz Kammer
380c67920e Merge "Add imports_sdk_version to prebuilt_apis" am: 9c07d24477
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1397557

Change-Id: Ic536dfe4c2f3a3a2e610520266cdca76f0d9ecfd
2020-08-24 17:30:51 +00:00
Treehugger Robot
e819bec240 Merge "Propagate sanitize.never flag in SDK snapshots." 2020-08-24 17:20:58 +00:00
Liz Kammer
9c07d24477 Merge "Add imports_sdk_version to prebuilt_apis" 2020-08-24 16:50:51 +00:00
Jooyung Han
c2a1d70eaf apex: pass "apex name" as literal to apex variants
When a cc module sets UseApexNameMacro(mutated property), it is built
with __ANDROID_APEX_NAME__ for its apex variants.

For now the new prop is used by aidl_interface-generated modules only.

Note that we already have __ANDROID_APEX_<NAME>__ macro. The new macro
can be used when we need to pass the name as data while the old one is
useful when we want conditional compilation.

Bug: 165017590
Test: m com.android.aidltest
     check build.ninja if -D__ANDROID_APEX_NAME__ is defined for apex
     varaints
Change-Id: Ia81ba8f833d23254e58c9777daf184d7861f07a7
2020-08-24 10:08:27 +09:00
Colin Cross
6419a88214 Update go.mod to go 1.15 am: 42f135b7b0
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1405909

Change-Id: Ic3bfd0cf94cd477d8a0f8830cb9b9bb3764968f0
2020-08-22 00:51:33 +00:00
Colin Cross
9e93b1bb12 Remove TestApexWithJniLibs_Errors
AddFarVariationDependencies was broken, which allowed dependencies
on non-libraries to be added even though a link variation was
requested.  When AddFarVariationDependencies is fixed, attempting
to add a non-library to jni_libs results in the usual missing
variant error.

Test: none
Change-Id: I6aa281491b8af5f141a2d132e248c75f95d68ab5
2020-08-21 16:18:11 -07:00
Colin Cross
cc83efbb18 Create image variations for sh_binary
AddFarVariationDependencies was broken, which allowed apex modules
to depend on sh_binary even though they were requesting an image
variation and sh_binary did not have one.  Create image variations
for sh_binary.

Test: TestApexWithShBinary
Change-Id: I1ff35ea9967edafbb7812a5e56667e4385af7e2f
2020-08-21 14:27:34 -07:00
Colin Cross
42f135b7b0 Update go.mod to go 1.15
The version in go.mod is used by `go build` to select the version
of the Go language to target.  If it is older than the version of Go
in the tree then command line tools like `go test` will give different
results from inside the build.  Update the version to match the version
of Go in the tree.

Test: go build ./java
Change-Id: I418f401580be7696e2829c970938c7c82715b932
2020-08-21 11:13:21 -07:00
Martin Stjernholm
fbb486fc7a Propagate sanitize.never flag in SDK snapshots.
This is necessary to not get dependencies on libclang_rt sanitizer libs
for the CRT objects which lead to cyclic dependencies in sanitizer
builds.

Test: `lunch aosp_blueline_hwasan-userdebug && m nothing`
  in a tree that has the prebuilts created and unzipped from
  `build/soong/scripts/build-aml-prebuilts.sh runtime-module-{sdk,host-exports}`
Bug: 151303681
Change-Id: I3f848a084280bdc3ade4b74df03e981d8cc61222
2020-08-21 18:43:51 +01:00
Treehugger Robot
82448f2d2d Merge "Add extra_test_configs option" am: ec7a91f9fa
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1402168

Change-Id: I6de14462095a64c3ccb3c501de7e4feaa04394c9
2020-08-21 16:50:54 +00:00
Treehugger Robot
ec7a91f9fa Merge "Add extra_test_configs option" 2020-08-21 16:22:43 +00:00
Jingwen Chen
69d4cbeb04 Surface module properties as Bazel BUILD target attributes in the Bazel overlay
This patchset changes bazel_overlay to generate soong_module as a macro,
instead of a rule, and generate module properties in the BUILD files as
kwargs to the soong_module macro.

Here's a sample of the new BUILD files with module properties:

bionic/libdl/BUILD.bazel:
https://paste.googleplex.com/6484466996346880?raw

art/build/apex/BUILD.bazel:
https://paste.googleplex.com/5461276001042432?raw

bionic/apex/BUILD.bazel:
https://paste.googleplex.com/4932795173437440?raw

soong_module is now a macro that conditionally expands to underlying
soong_<module type> rules with statically defined attributes. In this
CL, we are starting with a hardcoded filegroup rule definition to
demonstrate the conditional rule loading within the soong_module macro.
If the module_type matches an existing Bazel rule, soong_module forwards
the entire **kwargs into the rule, which Bazel typechecks.

Non-filegroup module types will be expanded into generic_soong_module,
but with the kwargs dropped.

This approach allows us to:

1) Programmtically generate soong_<module type> rules for all module
types available in Soong, together with the statically defined attribute
types in `attrs`.
2) Incrementally migrate and test individual module types from
generic_soong_module to their module rule shims.
3) Swap out the module rule shims to the actual Bazel rules (e.g
cc_library, java_library) and perform attribute manipulation in Starlark
itself.

Example of querying against the 'srcs' attribute in soong_filegroup:

```
$ bazel cquery 'kind(soong_filegroup, //...)' | wc -l
590

$ bazel cquery --output=build 'attr(srcs, "linker.cpp",
kind(soong_filegroup, //bionic/...))'
INFO: Analyzed 3907 targets (0 packages loaded, 0 targets configured).
INFO: Found 3907 targets...
/usr/local/google/home/jingwen/aosp/out/soong/bazel_overlay/bionic/linker/BUILD.bazel:4144:13
soong_filegroup(
  name = "linker_sources",
  generator_name = "linker_sources",
  generator_function = "soong_module",
  generator_location = "bionic/linker/BUILD.bazel:4144:13",
  srcs = ["dlfcn.cpp", "linker.cpp", "linker_block_allocator.cpp",
"linker_dlwarning.cpp", "linker_cfi.cpp", "linker_config.cpp",
"linker_debug.cpp", "linker_gdb_support.cpp", "linker_globals.cpp",
"linker_libc_support.c", "linker_libcxx_support.cpp",
"linker_namespaces.cpp", "linker_logger.cpp",
"linker_mapped_file_fragment.cpp", "linker_phdr.cpp",
"linker_relocate.cpp", "linker_sdk_versions.cpp", "linker_soinfo.cpp",
"linker_tls.cpp", "linker_utils.cpp", "rt.cpp"],
  deps = [],
)
/usr/local/google/home/jingwen/aosp/out/soong/bazel_overlay/soong_module.bzl:32:23
in <toplevel>
```

This CL is known to be lacking the following features, and will be looked at in follow up CLs:

1) Pretty printing reflect.Interface properties, like arch, multilib and
dists.
2) Generating module Bazel rule shims for all module types, instead of
hardcoding them like `soong_filegroup`.

Bug: 162720644
Test: bazel_overlay_test.go (soong build test)
Test: m bazel_overlay && cd out/soong/bazel_overlay && bazel cquery //...
Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: Ic1e448887eb540ed15a55bc4090cf75a4d832d41
2020-08-21 11:45:46 +00:00
Treehugger Robot
bc9a8dfc01 Merge "Remove duplicated use of UsesLibraryDependency" am: f5a2b8a641
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1405677

Change-Id: I6e095d9426d3c83f361293ecda441d6983fa5275
2020-08-21 10:06:53 +00:00
Martin Stjernholm
b5326f2d1c Merge "Resolve prebuilt module explicitly for dex2oat tool dependencies." am: 4ef4d55944
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1404636

Change-Id: I712c2e6b3a19ed65e6624d1e712b2b11790aa411
2020-08-21 10:06:46 +00:00
Ulyana Trafimovich
d972866eb5 Merge "Use common helper functions for getting sorted map keys." am: 49b1e9b6ce
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1404519

Change-Id: I2ad29eeb5a9b7a2b69854f945f8ed8f912655eff
2020-08-21 10:06:32 +00:00
Ulyana Trafimovich
df14a1b9c2 Merge "Propagate transitive <uses-library> dependencies through static libraries." am: bde92fc6be
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1404787

Change-Id: I03a83e7a2cab6cdb5d8e74df979440bb243d6ec3
2020-08-21 10:04:32 +00:00
Treehugger Robot
f5a2b8a641 Merge "Remove duplicated use of UsesLibraryDependency" 2020-08-21 10:01:58 +00:00
Martin Stjernholm
4ef4d55944 Merge "Resolve prebuilt module explicitly for dex2oat tool dependencies." 2020-08-21 09:57:40 +00:00
Ulyana Trafimovich
49b1e9b6ce Merge "Use common helper functions for getting sorted map keys." 2020-08-21 09:36:30 +00:00
Ulyana Trafimovich
bde92fc6be Merge "Propagate transitive <uses-library> dependencies through static libraries." 2020-08-21 09:36:10 +00:00
Treehugger Robot
a3d1c127a2 Merge "Reland: Deduplicate APEX variants that would build identically" am: 68bfe074c6
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1405087

Change-Id: I3839e80972dbc025b504f4593538f589be079389
2020-08-21 07:21:53 +00:00
Treehugger Robot
68bfe074c6 Merge "Reland: Deduplicate APEX variants that would build identically" 2020-08-21 06:46:51 +00:00
Treehugger Robot
0d1812df60 Merge "Add Goma deprecation PSA to soong." am: c70043aec2
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1403640

Change-Id: I828f1770c3ad4f3d6a5c561dadd93735e413581f
2020-08-21 04:39:42 +00:00
Treehugger Robot
c70043aec2 Merge "Add Goma deprecation PSA to soong." 2020-08-21 04:26:11 +00:00
Treehugger Robot
305935590d Merge "Capture list of unused methods when shrinking in R8" am: c23d6031b0
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1401848

Change-Id: Iff923f6fe8ba1d4cd0d93cc182ad6aaaa7d7e6bd
2020-08-21 00:36:49 +00:00
Treehugger Robot
c23d6031b0 Merge "Capture list of unused methods when shrinking in R8" 2020-08-21 00:28:24 +00:00
Liz Kammer
9b7ecd76f5 Remove duplicated use of UsesLibraryDependency
This was resulting in a soong java test failure:
./java.go:994:80: ambiguous selector dep.DexJarBuildPath
./java.go:994:103: ambiguous selector dep.DexJarInstallPath
./java.go:2595:58: ambiguous selector dep.DexJarBuildPath
./java.go:2595:81: ambiguous selector dep.DexJarInstallPath
./sdk_library.go:879:2: duplicate method DexJarBuildPath
./sdk_library.go:879:2: duplicate method DexJarInstallPath

Test: go test java tests
Test: m
Change-Id: I34465907f917d94be643be7322215fc35fc5fbe6
2020-08-20 13:50:42 -07:00
Treehugger Robot
b8638926f4 Merge "Allow globally disabling some clang-tidy checks." am: 66790d8943
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1403127

Change-Id: Ia2351d00d6952fea7f82268e4d345306e6118bff
2020-08-20 20:13:44 +00:00
Treehugger Robot
66790d8943 Merge "Allow globally disabling some clang-tidy checks." 2020-08-20 19:51:31 +00:00
Jeffrey Vander Stoep
507f3f0c0f Merge "Rust: use 1.45.2 prebuilts" am: 2e8a617bca
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1398476

Change-Id: I2d5e9e8d887f1bf3ed1c0de2dd34370b28bd372f
2020-08-20 17:23:46 +00:00
Jeffrey Vander Stoep
2e8a617bca Merge "Rust: use 1.45.2 prebuilts" 2020-08-20 17:14:19 +00:00
Liz Kammer
2d2fd859bb Add imports_sdk_version to prebuilt_apis
This allows setting the prebuilt sdk's generated java_imports
sdk_version to "none".

Test: go tests
Test: m
Bug: 160455085
Change-Id: Id24aca811f0f09692971e63418da6685d8351737
2020-08-20 09:48:50 -07:00
Martin Stjernholm
c004862309 Resolve prebuilt module explicitly for dex2oat tool dependencies.
Test: m nothing
Bug: 145934348
Change-Id: If27984be26976a3abe8cb5e21cf174d5e71beb8a
2020-08-20 16:05:45 +01:00
Ulya Trafimovich
b8063c6a86 Use common helper functions for getting sorted map keys.
Add a new helper SortedIntKeys similar to SortedStringKeys.

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I08a43ec2cae7d1a82531295aca1a0658e3a0dd6f
2020-08-20 12:37:03 +01:00
Ulya Trafimovich
4b6d4c12cc Propagate transitive <uses-library> dependencies through static libraries.
Bug: 163037089
Test: lunch aosp_cf_x86-userdebug && m nothing
Test: added testcase in Soong
Change-Id: I943d497d779ca218a16e7208029189a1a69086c1
2020-08-20 12:34:42 +01:00
Ulyana Trafimovich
9b67bbd11c Merge "Collect paths to transitive SDK Java library dependencies." am: 9ce2221791
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1402668

Change-Id: I2f12e92ce08bd75d19073a0ba19aca2b07142422
2020-08-20 11:02:10 +00:00
Ulyana Trafimovich
9ce2221791 Merge "Collect paths to transitive SDK Java library dependencies." 2020-08-20 10:37:58 +00:00
Dan Shi
95d19422a6 Add extra_test_configs option
Bug: 163344047
Test: m -j HelloWorldHostTest hello_world_test HelloWorldTests
Change-Id: I237e3aa3f40df1f6387ba8ff6a16e56535cdeae9
2020-08-19 15:27:03 -07:00
Colin Cross
053fca10c9 Support ninja rsp files in soong_zip
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.

Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
2020-08-19 21:18:56 +00:00
Colin Cross
cb6143a142 Capture list of unused methods when shrinking in R8
Use the -printusage flag in R8 to output a list of the unused
methods.  Some of the files can be large (2MB for DocumentsUI,
87MB for all of AOSP), so immediately zip them and remove the
originals.  The zipped files will be merged and disted.

Bug: 151857441
Test: m TARGET_BUILD_APPS=DocumentsUI dist
Change-Id: I780e84e80eba7fe4d4fa15fec0f461890afd900b
2020-08-19 11:19:14 -07:00
Colin Cross
aede88c1c7 Reland: Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-19 10:21:17 -07:00
Ulya Trafimovich
31e444e101 Collect paths to transitive SDK Java library dependencies.
Previously only the names were collected, and later used in the
manifest_fixer to add missing <uses-library> entries to the manifest.
Now we also need to collect build-time and on-device paths, to be used
in class loader context for dexpreopt. This commit only collects paths,
but does not pass them to dexpreopt yet.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I34b229ee68f16ba215ba03770feadb4d890ec2bf
2020-08-19 13:51:21 +01:00
Treehugger Robot
2a3d0ade4c Merge "Add AIDL compiler to allowed Rust paths" am: 2d815963ba
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1394307

Change-Id: Ib892f2d01237bf56592356fdefa32859d13a6378
2020-08-18 22:12:44 +00:00