Commit graph

2569 commits

Author SHA1 Message Date
Colin Cross
cec8171420 Add integration testing infrastructure
Fix mutator registration for tests to allow different tests
in the same package to register different mutators.

Allow tests to track the resulting ModuleBuildParams objects
to use in assertions, and provide helpers for getting them.
For example:
    config := android.TestConfig(buildDir)
    ctx := android.NewTestContext()
    ctx.RegisterModuleType(...)
    ctx.MockFileSystem(...)
    ctx.ParseBlueprintsFile("Android.bp")
    ctx.PrepareBuildActions(config)
    ctx.Register()
    // Get the Inputs value passed to the javac rule for the foo module
    inputs := ctx.ModuleForTests("foo".Rule("javac").Inputs

Test: java_test.go
Change-Id: I10c82967f5f3586d2c176f169906b571ed82fc73
2017-07-14 14:19:51 -07:00
Colin Cross
eb54da6ebe Fix incorrect package for variable
java doesn't use a config package for its variables

Test: java_test.go
Change-Id: I84c396b794d6eef971b1ef174a6f6e529071c452
2017-07-14 08:24:36 -07:00
Colin Cross
d48633a564 Use ExistentPathForSource
Converting java to use Path objects incorrectly used PathForModuleSrc
for a path that might not exist, use ExistentPathForSource instead.

Test: java_test.go
Change-Id: I50e4e315a10ab08c4e72065094ae29844874f5e4
2017-07-14 08:24:36 -07:00
Dan Willemsen
38cef8ac39 microfactory: Sort dependencies
This was causing android/soong/ui/build to rebuild sometimes when it
didn't need to.

Test: `m -j nothing; grep "B compile" out/.soong_ui.trace` repeatedly
Change-Id: I5d33271a7b037f53674a0d312d8da1746eed8eaa
2017-07-13 15:09:44 -07:00
Dan Willemsen
cae59bc29d Add microfactory tracing
Test: m clean; load out/build.trace.gz in chrome://tracing
Test: m -j nothing; load again
Change-Id: I67d4e006a4bdde593b54c20d6d93a48278fe696d
2017-07-13 15:09:44 -07:00
Dan Willemsen
1b82286736 Support running soong_ui.bash from anywhere in the tree
It will cd to $TOP before running soong_ui itself, so that soong_ui
still runs from the top of the tree. ORIGINAL_PWD is saved so that we
can reference that later (for example, to move 'mma' implementation into
Go).

Test: cd system; ../build/soong/soong_ui.bash
Test: Set absolute / relative OUT_DIR or OUT_DIR_COMMON_BASE and repeat
Change-Id: Icb67a3ee6a3358cca50300755c8953419fc19437
2017-07-12 16:41:09 -07:00
Dan Willemsen
c40e10bd6b Drop make flags in soong_ui
MAKELEVEL and MAKEFLAGS were are both unset by makeparallel when invoked
from Make. In preparation for removing the Make wrapper (and
makeparallel), strip them in soong_ui as well.

Add MFLAGS for completeness.

Test: m -j
Change-Id: I029f9464d11b432e979b32ede360450f4facb038
2017-07-11 15:23:55 -07:00
Dan Willemsen
091525e15c Parse environment options from the command line
Make supports specifying all types of variables on the command line
(using =, :=, +=, and other variable references. When running soong_ui
through make/makeparallel these all effectively become environment
variables.

So in preparation to remove the Make wrapper, support a simplified form
of this syntax, roughly equivalent to what the shell supports if
specified before the command (<NAME>=<VALUE>).

Test: m -j blueprint_tools
Change-Id: I08fa2b86710f282e619b0cc324a3e5bbaf62d26e
2017-07-11 15:23:51 -07:00
Dan Willemsen
9b58749f30 Support more ways to pass -j/-k
In preparation to remove Make/makeparallel from soong_ui startup, we
need to preserve compatibility with the different ways that make
supports the -j option.

Nothing changes unless Make/makeparallel is removed from the startup.
Once that is removed, not specifying a -j value will be equivalent to
'-j' instead of '-j1', like Ninja. A value will also be supported when
specifying -k, like Ninja (though specifying it alone will be equivalent
to '-k 0').

Test: m -j blueprint_tools
Change-Id: I9d5d59bedd4f6e5cca76bdb4cd47e0b5b7d523f0
2017-07-11 14:51:26 -07:00
Dan Willemsen
fd7310d95a Fix kernel_headers module registration
This was breaking SANITIZE_TARGET (and likely others)

Test: SANITIZE_TARGET=address m -j nothing
Test: out/soong/build.ninja is identical before/after
Test: out/soong/Android-aosp_arm64.mk is identical before/after
Change-Id: Ic0d6774025761ec9ae9240c109895ab8032e4784
2017-07-10 12:30:35 -07:00
Jiyong Park
d773eb3e86 add device_kernel_headers module for vendor-specific kernel headers
device_kernel_headers is a built-in heder-only lib that provides device-specific
kernel headers. The header path is configured via a new product variable
SystemIncludeDirs, which is currently the mirror of
TARGET_PROJECT_SYSTEM_INCLUDES in the make world.

Note: generic kernel headers (bionic/libc/kernel) have been added to the
include path by default. "device_kernel_headers" module is for device-specific
kernel headers such as /device/*/*/kernel-headers.

Note 2: this is opt-in for Android.bp modules (i.e. header_libs :
["device_kernel_headers"] required.) while it is always provided to
Android.mk modules.

Bug: 62939405
Test: choosecombo to aosp_sailfish (or any other Pixel/Nexus targets)
BOARD_VNDK_VERSION=current m -j gralloc.msm8996 (or any other vendor
libs using vendor-specific kernel headers)

Change-Id: I81c60abc13942c89fff723d1544b27a81b300db0
2017-07-08 09:29:04 +09:00
Ivan Lozano
0c3a1efae4 Add integer_overflow sanitization build option.
Adds the SANITIZE_TARGET=integer_overflow build option to apply signed and
unsigned integer overflow sanitization globally. This implements the
Soong side of the build option.

An additional build option is provided to control whether or not to run
in diagnostics mode, controlled by SANITIZE_TARGET_DIAG. This works the
same way that SANITIZE_TARGET does and currently only supports
'integer_overflow' as an option.

A default sanitizer blacklist is added to avoid applying sanitization
to functions that are likely to exhibit benign overflows.

Bug: 30969751
Test: Building with and without the new flags, device boot-up, tested
various permutations of controlling the new flags from build files.

Change-Id: Ibc8a8615d3132f1a23faaf1cb4861f24c5ef734a
2017-07-07 09:52:23 -07:00
Treehugger Robot
e9425b0277 Merge "Fix install location for vendor tests" 2017-07-07 03:30:13 +00:00
Dan Willemsen
00269f23ee Fix install location for vendor tests
These should be install in /data/nativetest* with the rest of the tests,
but had been moved to /vendor/nativetest* accidentally. Add some tests
so that this doesn't happen again.

Bug: 63393698
Test: m -j blueprint_tools
Test: compare out/soong/Android-aosp_arm64.mk
Test: compare out/soong/build.ninja
Change-Id: Id3b08a7e3908955df18a043a02ea576dc88086c3
2017-07-06 18:09:46 -07:00
Treehugger Robot
195fb4183c Merge "Fixed the unexpected scenario for "device_supported"" 2017-07-06 18:30:04 +00:00
Jiyong Park
4c35af09ea fix: a vendor variant of a lib doesn't re-export llndk headers
If a vendor (or vendor_available) lib has a dependency to a llndk
library, the name of the llndk library is modified to <name>.llndk.

There was a bug that this name modification is done only for shared_libs
property, but not for export_shared_lib_headers property. As a result,
the headers of the llndk was not re-exported. Fix the bug by modifying
export_shared_lib_headers property as well.

Bug: 63178444
Test: build
Test: look into out/soong/Android-<target>.mk. libutils.vendor has
system/core/liblog/include_vndk in its LOCAL_EXPORT_CFLAGS.

Change-Id: I6fcb1036cdb81ba167ead815b436a5cada283ede
2017-07-05 22:44:09 +00:00
Nan Zhang
1a0f09bc56 Fixed the unexpected scenario for "device_supported"
cc_defaults {
    name: boo,
    device_supported: false,
}

cc_library_static {
    name: foo,
    defaults: [boo],
}

Soong still tried to build foo_android_arm/arm64 device target which is not what we
expected.

Test: m -j checkbuild

Change-Id: I26a67c9ea024f5458f0818def0fa10cecc5fb7cf
2017-07-05 10:37:57 -07:00
Colin Cross
36242850fd Refactor factories
Change module factories from returning a blueprint.Module and a list
of property structs to returning an android.Module, which holds the
list of property structs.

Test: build.ninja identical except for Factory: comment lines
Change-Id: Ica1d823f009db812c518f271a386fbff39c9766f
2017-06-30 21:08:36 +00:00
Treehugger Robot
4817389148 Merge "Add header-abi-linker dependency on so file." 2017-06-30 17:38:53 +00:00
Dan Austin
8241abba87 Update coverage sanitizer flags in soong
Update the coverage sanitizer flags in soong to use the new
flavor of coverage sanitization.

Bug: 63108942

Test: Fuzzer runs with coverage guards.
Change-Id: I2e817a5282cf7dab49aee6985743defd4fb74628
2017-06-29 08:55:13 -07:00
Lazar Trsic
397866653e Revert "[mips] Disable compact branch generation"
With the latest Clang we do not have issue with beqc instruction,
so allow generation of compact branches for mips.

This reverts commit 3d6a38d601.

Change-Id: I6dbb5d1fce08bd714cc9a0318d50de5765cff038
2017-06-28 11:22:58 +02:00
Dimitry Ivanov
ba6b55240b Add libdl.so to the list of system shared libs
Note that libdl is already (always) implicitly loaded
because libc.so has a dependency on it.

This change also makes sure that libc.so always goes before
libdl.so in DT_NEEDED list.

Bug: http://b/62815515
Test: make
Change-Id: Idb231f8a8443ee859828bac6462236a1b7199d05
2017-06-27 16:08:03 -07:00
Jayant Chowdhary
6ab3d846b2 Add header-abi-linker dependency on so file.
Whenever a symbol file is not available, header-abi-linker can look at
the symbols with STV_DEFAULT or STV_PROTECTED visibility for further abi
filtering.

Bug: 62463617

Bug: 62307940

Test: make -j64

Change-Id: I5dc618be104a49d47375d37cf78c4490b1529987
2017-06-27 11:19:10 -07:00
Colin Cross
06382991e0 Make '**' implicit
Default to copying all entries in the input zip to the output zip.

Test: zip2zip_test
Change-Id: I0d70620c621bfeee2b7fe2a64b350b4866ad2877
2017-06-23 14:08:42 -07:00
Colin Cross
8936b02b58 Add jar sorting to zip2zip
Jars have a strange sorting order; the META-INF/ directory should
come first, then META-INF/MANIFEST.MF, then any other files in
META-INF/, and then any files outside META-INF.  Add a -j argument
to zip2zip that sorts using jar ordering.

Test: zip2zip_test
Change-Id: I80e2bc7e284ef74f6561c26cb6541298834db1bc
2017-06-23 14:08:32 -07:00
Colin Cross
3e3e72da90 Add java config and share it with make
Add a java/config package to hold config information, and share
it with make through makevars.

Test: builds
Change-Id: I46c088bda0fe97a1823bfdd80fa692d0bf61da1b
2017-06-23 11:24:54 -07:00
Colin Cross
540eff8e5f Split java device properties into separate struct
Prevent host modules from having fields that only make sense for
device modules.

Test: builds
Change-Id: I20278e029a38fb9a6b75ef3c2cf3c1a97cef2b87
2017-06-23 10:57:36 -07:00
Colin Cross
46c9b8bfd5 Remove non-idiomatic inheritance
Remove inheritance implemented with the bad "superclass calls
subclass through interface" pattern, and replace it with composition.

Test: builds
Change-Id: If323f89360455b3f98b40777edaaaa265bb3b5fc
2017-06-23 10:57:36 -07:00
Colin Cross
ba211130bd Update obsolete references
Test: builds
Change-Id: Ic37628b78704a03200b6326485fb4667e1c5a665
2017-06-23 10:57:36 -07:00
Colin Cross
20e1365e61 Fix panic in builds with no device
Running prebuilts/build-tools/build-prebuilts.sh in a repo that
contains art/build/art.go panics because config.Targets[Device] is
empty.  Check the length before accessing the slice.

Test: prebuilts/build-tools/build-prebuilts.sh
Change-Id: Ifb9fe0fad07b22d6b574f505c08c5c761278aad0
2017-06-23 10:57:36 -07:00
Treehugger Robot
405ce4d352 Merge "zip2zip: Support sorting globbed arguments, '**'" 2017-06-23 01:00:29 +00:00
Dan Willemsen
82218f285c zip2zip: Support sorting globbed arguments, '**'
When '-s' is passed, any globbed arguments will have their results
sorted. When there are multiple arguments, the files will still be
inserted in argument order.

A bare '**' is now special cased to mean every file in the input zip.

This allows zip2zip to sort entire zip files efficiently by using
`zip2zip -s -i <> -o <> '**'`. This can be useful if your original zip
program used filesystem ordering which was not reproducible.

Test: m -j blueprint_tools (new tests pass)
Change-Id: Ic3512c5fe14c94c6f3e134296905121d2ff8b58a
2017-06-22 15:55:27 -07:00
Treehugger Robot
12e23182a6 Merge "Add OWNERS in build/soong" 2017-06-22 01:04:55 +00:00
Treehugger Robot
c527b76775 Merge "Turn find emulator warnings into errors" 2017-06-21 20:52:45 +00:00
Dan Willemsen
418420e49d Turn find emulator warnings into errors
Test: build/soong/build_test.bash (multiproduct_kati)
Change-Id: Ieb65a457c8c1ed4bff44a01aa2cc11203172ba9b
Merged-In: Ieb65a457c8c1ed4bff44a01aa2cc11203172ba9b
Merged-In: I70b696525a90c64cc05ad4caa1c3647d57ec9e9e
(cherry picked from commit dc5b70f97c)
(cherry picked from commit e8a25d5577)
(cherry picked from commit 14b3f7f12b)
2017-06-21 12:44:18 -07:00
Treehugger Robot
d5696dd836 Merge "add exclude_shared_libs for vendor_available:true libs" 2017-06-21 19:15:50 +00:00
Dan Willemsen
e7efcbe34e Capture backtraces during build_test
We're seeing random ckati segfaults, and ASAN only hides the problem, so
attempt to capture more information.

Bug: 36182021
Test: build/soong/build_test.bash, kill -SIGSEGV <ckati>
Change-Id: I854ae846df0c47f69ec8d39b9a3d9e6f482e0854
2017-06-20 23:27:45 -07:00
Jiyong Park
44cf1a7779 add exclude_shared_libs for vendor_available:true libs
Adding a mechanism to conditionally exclude some shared library
dependencies when a lib is built for vendors.

Without this, some libraries cannot be earily marked as vendor_available
if they are depending on shared libs can shouldn't be marked as
vendor_available.

By using exclude_shared_libs with exclude_srcs (or __ANDROID_VNDK__
macro), we can eliminate the unnecessary dependency for vendors.

Bug: 62471389
Test: build

Change-Id: If94277b45c3769223cea371d0028e75277640356
2017-06-21 10:13:27 +09:00
Jeff Gaston
83cf1cee58 Merge changes I0b67e89f,I60f02a8d
* changes:
  Revert "Revert "Run bpfix at the end of androidmk""
  Revert "Revert "Initial implementation of bpfix""
2017-06-20 19:29:41 +00:00
Jayant Chowdhary
9677e8c6ad Black-list for clang LibTooling Cflags.
Add a list of flags which are not understood by clang LibTooling tools
and filter them out of the Cflags the tools are invoked with.

Test: In frameworks/av, make libmedia vendor_available (this invokes
header-abi-dumper on this module), mm -j64.

Bug: 62447349

Change-Id: I46f017212b89f4331145c999103d0ed44da0abaf
2017-06-19 19:43:03 -07:00
Jeff Gaston
c381fd0282 Revert "Revert "Run bpfix at the end of androidmk""
Bug: 38351765
Test: androidmk Android.mk > Android.bp

This reverts commit eb3186abbe.

Change-Id: I0b67e89f5096aae60c97eb6af4fd205509aac5c9
2017-06-19 15:52:25 -07:00
Jeff Gaston
aff66e55a9 Revert "Revert "Initial implementation of bpfix""
Bug: 38351765
Test: bpfix Android.bp

This reverts commit a8cc9c53fa.

Change-Id: I60f02a8dd920346aa17b9044f834ffe94fa693c6
2017-06-19 15:52:15 -07:00
Ivan Lozano
f47b048782 Merge "Add support for misc_undefined diagnostics" 2017-06-19 20:16:48 +00:00
Ivan Lozano
651275ba90 Add support for misc_undefined diagnostics
Setting Sanitize.Diag.Undefined to true does not enable diagnostics for
all possible UBSan sanitizers that might be declared in Sanitize.Misc_undefined
(specifically unsigned-integer-overflow and nullability-*). This adds an
equivalent Sanitize.Diag.Misc_undefined string array to allow diagnostics to
be enabled in these sanitizers.

Bug: 62546512
Test: make works and device boots.
Test: Tested an Android.bp generated binary with the new field enabling
unsigned-integer-overflow diagnostics.

Change-Id: I9aa7cb2555ab4f94ae1aa290715a617718d936ff
2017-06-19 10:11:56 -07:00
Chih-Hung Hsieh
81a6cb3295 Add OWNERS in build/soong
* Owners are selected from top CL approvals or owners.
  They will be suggested to review/approve future CLs.
* OWNERS files are recognized by the new find-owners plugin, see .md files in
  https://gerrit.googlesource.com/plugins/find-owners/+/master/src/main/resources/Documentation/

Test: build/make/tools/checkowners.py -c -v OWNERS
Change-Id: I5184ff6ddec602de238eb6bde033503fad1f8887
2017-06-15 13:49:57 -07:00
Yoshisato Yanagisawa
572324ac33 Add JAVAC_WRAPPER support
To run javac with AndroidGomaStaticRule, let me make JAVAC_WRAPPER
also show up to javac/builder.go.

Test: USE_GOMA=true JAVAC_WRAPPER=gomacc m -j32
Bug: b/62334576

Change-Id: I19e1e4315f71d8706800da810b5212f5964171a4
2017-06-14 01:04:50 +00:00
Treehugger Robot
2a60959b65 Merge "Require that NDK symbol files end with .map.txt." 2017-06-14 00:21:45 +00:00
Dan Albert
15be0c6a36 Require that NDK symbol files end with .map.txt.
We're making an NDK API Council +1 a requirment for changes to these
files, so we need to know that all symbol files will match a given
file pattern.

Test: Changed a symbol file to foo.txt, `make ndk` failed
Test: make ndk
Bug: http://b/35870541
Change-Id: I29690fb97746ad682e57845fea3f1114cfd77598
2017-06-13 15:14:56 -07:00
Jeff Gaston
a6697e8b7f Only check for ninja stuckness after it's been running for a bit
Bug: 62580037
Test: rm -rf out && m -j showcommands | grep "ninja may be stuck" || echo ok

Change-Id: I8ff1bd216b5f8349ce9e06e5465a9f8d0663f8c0
2017-06-13 14:15:02 -07:00
Jeff Gaston
193f2fb092 Make absolute OUT_DIR work with sbox
Test: make OUT_DIR=/tmp/abspath
Test: make OUT_DIR=relative-out
Bug: 35562758

Change-Id: I688d5f6117b194440c5d01c1040033b5671187c2
2017-06-12 16:36:17 -07:00