Commit graph

21266 commits

Author SHA1 Message Date
Dan Willemsen
1b1a1e92c1 Merge "Reland "Add prebuilt_build_tool to allow genrules to use prebuilt tools"" am: a047f0120c
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1392887

Change-Id: I4e8f1980d0bf131984f10aedd02e5b10c11fbaff
2020-08-07 20:45:15 +00:00
Dan Willemsen
a047f0120c Merge "Reland "Add prebuilt_build_tool to allow genrules to use prebuilt tools"" 2020-08-07 20:27:40 +00:00
Treehugger Robot
a7586115c7 Merge "Add BoardKernelModuleInterfaceVersions." am: f3ab72936b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1392499

Change-Id: Id7677cf74428661cf2ed6fec8c587b32aeff7bfa
2020-08-07 18:41:52 +00:00
Treehugger Robot
f3ab72936b Merge "Add BoardKernelModuleInterfaceVersions." 2020-08-07 18:37:36 +00:00
Treehugger Robot
84beaa0135 Merge "Don't export link dirs from proc-macros." am: 2a73c2d060
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1393144

Change-Id: Ibbf8e3d230bac6d363093c211501230da382fbfc
2020-08-07 18:34:30 +00:00
Kelvin Zhang
74fa47f962 Merge "Fix zip64 reader when file size is < 4GB and 32 bit fields are -1" am: 49023a992f
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1381161

Change-Id: Ib0d36e96c81c2604f7be03aadee9b6a5c53305b6
2020-08-07 18:34:20 +00:00
Treehugger Robot
2a73c2d060 Merge "Don't export link dirs from proc-macros." 2020-08-07 18:20:21 +00:00
Kelvin Zhang
49023a992f Merge "Fix zip64 reader when file size is < 4GB and 32 bit fields are -1" 2020-08-07 18:20:15 +00:00
Patrice Arruda
add7ea9cc8 Set the OS name in the metrics protobuf file
There are cases where the developer is not building a product so
the metadata information is not being set. For now, set the OS
name so we can track which OS the user is using it.

Fixes: b/163118212
Test: m nothing and checked the blueprint
Change-Id: I9b521eee2d266b50625aa8e916ddaacb4e2feaea
2020-08-07 17:55:23 +00:00
Ivan Lozano
2bbcacf0c9 Don't export link dirs from proc-macros.
We shouldn't use exported link dirs from proc-macros as these may be
libraries for the wrong architecture or OS.

This doesn't cause a problem unless both the proc-macro and the
dependent library share a dependency. If that's the case, in some
circumstances it will cause the depdent library to fail compilation
when using a library with incorrect architecture.

Bug: 163088025
Test: Test case no longer colliding.
Test: cd external/rust/; mma
Change-Id: I46cbd850c3a89d10aa844b5c1b383e1070452d00
2020-08-07 09:00:50 -04:00
Thiébaud Weksteen
4318e1c650 rust: fully disable all lints when no_lint is set
If the no_lint attribute is set, we now explicitly disable any warning
or error.

Bug: 162897623
Test: cd external/selinux; mma; check the build log for warnings
Change-Id: I5338bda77ab25ded7cc43cc3afd29419fcbd9808
2020-08-07 13:56:35 +02:00
Martin Stjernholm
0be556121d Merge "Only output make targets for uninstallable static libs in the APEX unavailable-to-platform case." am: 9f381d5ce3
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1393187

Change-Id: Id8618275a3468759c25c4eba5c02356072ab5fc5
2020-08-07 11:41:41 +00:00
Martin Stjernholm
9f381d5ce3 Merge "Only output make targets for uninstallable static libs in the APEX unavailable-to-platform case." 2020-08-07 11:30:29 +00:00
Thiébaud Weksteen
d0a77bc0ea Merge changes I329d0fe3,I8af9115b am: 9bf3042493
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1393179

Change-Id: I4f0e488def53b6d808a7ea2e7133ffab0ba7e798
2020-08-07 06:54:18 +00:00
Thiébaud Weksteen
9bf3042493 Merge changes I329d0fe3,I8af9115b
* changes:
  rust: validate existence of library source
  rust: handle modules with same crate_name
2020-08-07 06:43:44 +00:00
Jeongik Cha
816a23a50d Introduce AlwaysUsePrebuiltSdks
Instead of UnbundledBuild, use AlwaysUsePrebuiltSdks
to determine if java modules needs to be built against prebuilt sdks.
And rename UnbundledBuildUsePrebuiltSdks to AlwaysUsePrebuiltSdks to
express its behavior more correctly.(It can be orthgonal to "Unbundled")

Bug: 160390776
Test: TARGET_BUILD_UNBUNDLED_IMAGE=true m vendorimage

Change-Id: I0be7265c1959d8774c295372cd7a9250169f6df9
2020-08-07 12:15:52 +09:00
Inseob Kim
7fe143638d Merge changes from topic "cfi_vendor_snapshot" am: eaebec1312
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1376079

Change-Id: Iedcaec8072ea3509c8aa15c0c32bcb9e33b0fcd7
2020-08-07 02:43:49 +00:00
Inseob Kim
9e6f024ad5 Refactor vendor snapshot modules am: 2d34ad94f9
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1382316

Change-Id: Ia367878b6de80b123d680a74073c1cd63a841c55
2020-08-07 02:43:46 +00:00
Inseob Kim
eaebec1312 Merge changes from topic "cfi_vendor_snapshot"
* changes:
  Add cfi static libraries to vendor snapshot
  Refactor vendor snapshot modules
2020-08-07 01:48:10 +00:00
Colin Cross
81ca6cd407 Add alias variations to linkageMutator
Alias the shared variation if it exists, otherwise the static variation.
This allows modules that are not aware of shared library variations
(like cc_genrule) to depend on shared libraries by depending on
just the normal image, os and arch variations.

Bug: 162437057
Test: TestLibraryGenruleCmd
Change-Id: Icec57d43538e01ab05cc50d4e3f9a11cc55f0162
2020-08-06 18:46:06 -07:00
Colin Cross
094faa5b26 Use local variations for versionsMutator
Use local variations and an alias for the "" variation for
versionMutator.  Local variations are used here because the variation
of one module doesn't affect the variation it should depend on - a
module with variation version:29 will manually link against dependencies
with the correct version variation, and don't need it to be automatically
resolved.  The alias allows other modules to depend on a module
mutated by the version mutator without specifying the version: ""
variation.  This allows removing the variation on genrule modules.

The motiviation here is to make sure versionMutator calls AliasVariation,
which will allow the previous linkageMutator to also use AliasVariation.

Bug: 162437057
Test: no change to build.ninja, make_vars-aosp_crosshatch.mk, Android-aosp_crosshatch.mk or late-aosp_crosshatch.mk
Change-Id: Ibb030cc4334e47511b8ab5bc3eb8a5ae39ad0023
2020-08-06 18:46:06 -07:00
Treehugger Robot
2e93f60a21 Merge "Reland "Support per-module MakeVars"" am: 8b073fc7f1
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1392886

Change-Id: I3debab6511963ad15d2710f469e5c094be71ef78
2020-08-07 01:14:34 +00:00
Treehugger Robot
8b073fc7f1 Merge "Reland "Support per-module MakeVars"" 2020-08-07 00:50:24 +00:00
Jingwen Chen
a9af2cefaf Merge "Create a AOSP Bazel overlay workspace with Soong" am: 2106bf8458
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1359245

Change-Id: I167ee06ac23afebcf38676b490d6ce75773b2639
2020-08-07 00:10:07 +00:00
Jingwen Chen
2106bf8458 Merge "Create a AOSP Bazel overlay workspace with Soong" 2020-08-06 23:56:54 +00:00
Dan Willemsen
751ae87ed2 Reland "Add prebuilt_build_tool to allow genrules to use prebuilt tools"
Includes fixes for out dirs at absolute paths, and symlinks to absolute
paths. Also squashes support for exporting make vars, which previously
happened in a later patch (which got moved first, since it's mostly
infrastructure changes).

Original message:

cc_prebuilt_binary doesn't work well for host tools because they'll
often be using their own versions of shared libraries that may not be
compatible with what we build.

So add a module type that allows genrules to use one of these prebuilts
as a tool. Like other prebuilts, we'll use the source module if we have
it, or the prebuilt otherwise.

It supports adding extra dependencies for shared libraries or other data
files that are necessary to run the tool. Any genrules using the tool
will be rerun if any of the dependencies change.

Bug: 128690776
Test: treehugger
Test: build one-true-awk using genrule with bison
Test: with OUT_DIR=$PWD/out
Test: with OUT_DIR pointing to an absolute symlink
Change-Id: I662296cb55503b6a10ec5e5465b138e9edeb71f3
2020-08-06 15:56:38 -07:00
Dan Willemsen
6a6478d49e Reland "Support per-module MakeVars"
This allows setting per-module make variables earlier in the build,
particularly for prebuilt_build_tool users like LEX/M4/BISON (which will
come in a later CL). I moved filegroup over because it's a simpler
common interface, but it doesn't strictly need it.

Test: Inspect out/soong/make_vars-*.mk, out/soong/late-*.mk
Test: treehugger
Change-Id: I38631129d82b0aad2438e83610bb6a97568bf932
2020-08-06 15:56:09 -07:00
Andrei Homescu
c7767922e0 Export Rust SourceProvider types and methods
The AIDL compiler now uses SourceProvider to compiler the generated Rust
code from system/tools/aidl/build/aidl_interface.go using its own
SourceProvider object, which needs access to baseSourceProvider and all
methods of SourceProvider.

Test: mmma system/tools/aidl with 1357705 applied
Change-Id: I226609a7fccca2e7e1bfbad5d69d1821d37e43a1
2020-08-06 15:50:27 -07:00
Yifan Hong
42bef8d0cc Add BoardKernelModuleInterfaceVersions.
This is a list of KMI versions for this board. Only gki_apex modules
with matching kmi_version field is enabled.

Test: build GKI apexes
Bug: 162888350
Change-Id: Ieb2116b79e3985ba15155a3cb8a2c340b46473c7
2020-08-06 15:47:58 -07:00
Martin Stjernholm
9e9bb7f223 Only output make targets for uninstallable static libs in the APEX
unavailable-to-platform case.

This removes the special case added in https://r.android.com/1274763
from SkipInstall(), so that it doesn't cause conflicting AndroidMk
entries when a cc_prebuilt_library_static module has prefer:true and
the corresponding source module exists.

Test: `m` in a tree with a snapshot created from art-module-sdk where
  the libartimagevalues module has prefer:true
Bug: 151303681
Change-Id: I651ae325753b707296892adb4cae80daaddb6af2
2020-08-06 22:52:58 +01:00
Matthew Maurer
86fdf15721 Merge "[rust] Escape flags for bindgen" am: 217a1e406e
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1388348

Change-Id: I70a49bf440a2286c7c562538069eda56252993dd
2020-08-06 21:44:50 +00:00
Mitch Phillips
0ae16213d9 Merge "Added write only sanitizer for ASAN and HWASAN" am: 8dc2af873b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1372178

Change-Id: I23e8b419658f01af16337a4ba2674a9422dacad4
2020-08-06 21:44:29 +00:00
Matthew Maurer
217a1e406e Merge "[rust] Escape flags for bindgen" 2020-08-06 21:33:49 +00:00
Mitch Phillips
8dc2af873b Merge "Added write only sanitizer for ASAN and HWASAN" 2020-08-06 21:17:12 +00:00
Kelvin Zhang
45e2f1405d Fix zip64 reader when file size is < 4GB and 32 bit fields are -1
When CopyFrom writes a zipentry, it strips extra fields and generates
data descriptors. When writing data descriptors, it only writes 64 bit
values if the relevant sizes are >4GB. In some cases, the sizes are <4GB
but 32 bit sizes are set to -1. In this situation, CopyFrom will write
incorrect local file header, resulting in a zip file that can't be
parsed by standard zip tools.

Test: Unit Tests
Bug: 161922066
Change-Id: I64319a80647013eaf7693cf8bf5c6120016913a3
2020-08-06 15:48:04 -04:00
Inseob Kim
c42f2f2e9d Add cfi static libraries to vendor snapshot
CFI modules can't link against non-CFI static libraries, and vice versa.
So without capturing both CFI and non-CFI static libraries, vendor
modules won't be able to use CFI, which will be a critical security
hole.

This captures both CFI and non-CFI variants of all static libraries for
vendor snapshot, except for those whose cfi are explicitly disabled.

For example, suppose that "libfoo" is defined as follows.

cc_library_static {
    name: "libfoo",
    vendor_available: true,
}

As it doesn't have cfi disabled, two libraries "libfoo.a" and
"libfoo.cfi.a" will be captured. When installed, vendor snapshot module
for "libfoo" will look like:

vendor_snapshot_static {
    name: "libfoo",
    src: "libfoo.a",
    cfi: {
        src: "libfoo.cfi.a",
    },
}

The build system will recognize the "cfi" property, and will create both
CFI and non-CFI variant, allowing any modules to link against "libfoo"
safely, no matter whether CFI is enabled or not.

Two clarification:

1) The reason why we don't create separate modules is that DepsMutator
runs before sanitize mutators. CFI and non-CFI variant of a library
should exist in a single module.

2) We can't capture CFI variant if the source module explicitly disables
cfi variant by specifying the following.

sanitize: {
    cfi: false,
}

In this case, only non-CFI variant will be created for the vendor
snapshot module.

Bug: 65377115
Test: m dist vendor-snapshot && install && build against snapshot
Change-Id: Idbf3e3205d581800d6093c8d6cf6152374129ba4
2020-08-06 19:47:57 +00:00
Inseob Kim
2d34ad94f9 Refactor vendor snapshot modules
They have too many duplicated fields and functions.

Bug: 65377115
Test: manual
Change-Id: If3e2d67f02f3db7fa836c9550c978073ab7190e0
2020-08-06 19:47:43 +00:00
Steven Moreland
34fabfeffa Merge "allow empty phony modules" am: bd60c83271
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1390950

Change-Id: Id575e4e89f2dee5d727dc99d7beb9b4ecc5b2c67
2020-08-06 17:40:47 +00:00
Steven Moreland
bd60c83271 Merge "allow empty phony modules" 2020-08-06 17:22:11 +00:00
Ulyana Trafimovich
434e708101 Merge "Add structured representation for colon-separated jar lists." am: 910eb70891
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1354005

Change-Id: I46ca0273cda33ca194400744fc19583d21118ffc
2020-08-06 15:24:29 +00:00
Ulyana Trafimovich
910eb70891 Merge "Add structured representation for colon-separated jar lists." 2020-08-06 15:07:27 +00:00
Nicolas Geoffray
484464e4ba Merge "Add platform-mainline-test-exports in list of modules to build." am: 1f109cd376
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1391356

Change-Id: Ie30b12b1d3ea5e9e67f716c75e67c65d402cd525
2020-08-06 14:02:15 +00:00
Nicolas Geoffray
1f109cd376 Merge "Add platform-mainline-test-exports in list of modules to build." 2020-08-06 13:49:25 +00:00
Thiébaud Weksteen
83ee52f4cd rust: validate existence of library source
While rust_bindgen modules may be interpreted as libraries, they do not
have a reference to the generated source until bindgen has been
executed. For now, ignore these modules.

Update the unit tests to cover the rust_bindgen case.

Test: SOONG_GEN_RUST_PROJECT=1 m nothing
Bug: 162881856
Change-Id: I329d0fe3d94b77d395c3684f55ab01544ff7c18f
2020-08-06 15:17:46 +02:00
Thiébaud Weksteen
2f628baeae rust: handle modules with same crate_name
crate_name do not need to be unique. A library may depend on two
different versions of the same crate. Use the module name instead of the
crate name when indexing the modules for rust-project.json.

Test: SOONG_GEN_RUST_PROJECT=1 m nothing
Bug: 162896400
Change-Id: I8af9115b106fda2bed5b9fe96059cc29a33ac247
2020-08-06 15:17:42 +02:00
Thiébaud Weksteen
2cb2bf7134 Revert "rust: validate existence of library source" am: c810510898
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1393037

Change-Id: I50fc8f00dcbd528e80dc9551364250b1620d371a
2020-08-06 12:53:33 +00:00
Thiébaud Weksteen
4e62bf5357 Revert "rust: handle modules with same crate_name" am: 246e69c586
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1393036

Change-Id: Ieb7b7048d7f619e35d9d8a152754a7374c8fa288
2020-08-06 12:53:30 +00:00
Thiébaud Weksteen
c810510898 Revert "rust: validate existence of library source"
Revert submission 1391076

Reason for revert: Broken downstream Darwin build (b/162975597)
Reverted Changes:
I275f04639:rust: handle modules with same crate_name
Ie736d7ebb:rust: validate existence of library source

Change-Id: I1732d50a08546864f17cd826c8af574c1064c670
2020-08-06 12:28:08 +00:00
Thiébaud Weksteen
246e69c586 Revert "rust: handle modules with same crate_name"
Revert submission 1391076

Reason for revert: Broken downstream Darwin build (b/162975597)
Reverted Changes:
I275f04639:rust: handle modules with same crate_name
Ie736d7ebb:rust: validate existence of library source

Change-Id: I995923153c11db26b4af985f2eabe94912fb04d3
2020-08-06 12:27:56 +00:00
Jingwen Chen
5ba7e479d1 Create a AOSP Bazel overlay workspace with Soong
The Bazel overlay is a directory at out/soong/bazel_overlay that
replicates the layout of the AOSP Soong module tree, but as a Bazel
workspace. Each Soong module variant is represented as a BUILD target
created with the `soong_module` rule.

To create this overlay, run `m bazel_overlay`.

A `soong_module` target can depend on other `soong_module` targets.
These dependencies replicate each module's `directDeps` in the Blueprint
graph, just before `PrepareBuildActions`.

This enables users to use bazel query as a way to introspect the Soong
module graph. For example,

- Direct reverse dependencies of //bionic/libc:generated_android_ids in
//bionic/libc/...:

$ bazel query 'rdeps(//bionic/libc/...,
//bionic/libc:generated_android_ids, 1)'
//bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static
//bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static
//bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime
//bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static
//bionic/libc:generated_android_ids

- Why does com.android.runtime depend on lzma?

$ bazel query
'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image,
//external/lzma/...)'
//bionic/apex:com.android.runtime--android_common_com.android.runtime_image
//bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime
//system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime
//external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime

- What does the dep graph of //bionic/libc:crtbegin_so look like?

$ bazel query
'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)'
--output=graph > graph.in && dot -Tpng < graph.in > graph.png

https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8

Test:  croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel
query //... && bazel query 'rdeps(//bionic/libc/...,
//bionic/libc:generated_android_ids, 1)'

Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
2020-08-06 10:24:21 +00:00