Commit graph

10713 commits

Author SHA1 Message Date
Colin Cross
ae58e2bfa5 Don't prune $(OUT_DIR) from findleaves.py
Pruning $(OUT_DIR) can be overaggressive if $(OUT_DIR) matches the name
of any directory in the source tree that contains Android.mk files.
Pruning $(OUT_DIR) is no longer necessary, as one of the first things
the build system does is put an empty Android.mk file in there to avoid
recursing into it.

Bug: 31941856
Test: m -j OUT_DIR=angler
Change-Id: I7c5e95816f35bd5906845716d56cca76de7fa1ad
2016-10-04 16:35:21 -07:00
David Zeuthen
70209ad9f0 Merge "Make room for AVB hashtree and metadata." 2016-10-04 22:32:27 +00:00
Chih-hung Hsieh
4b4b2da13d Merge "Accept environment variables DEFAULT_*_TIDY_CHECKS" 2016-10-04 22:25:26 +00:00
David Zeuthen
4014a9daa7 Make room for AVB hashtree and metadata.
While the system.img images currently built with AVB support verify
correctly, mounting the filesystem content fails. This is because
'avbtool add_hashtree_footer' used to claim some of the unused /
DONT_CARE space for stashing the verity tables and this resulting in the
mapped device ending up being smaller causing the mount failure.

Fix this by leaving enough room for AVB hashtree and metadata before
building the image. This is achieved by moving the AVB hashtree support
into build_image.py and using a just added '--calc_max_image_size'
option to 'avbtool add_hashtree_footer' to figure out how much space to
leave out.

This depends on https://android-review.googlesource.com/#/c/281821/

Bug: 31264226
Test: Mounting dm-verity set up from system.img now works.

Merged-In: I4c5de1004c1059f8c582e76b3b8517d427aa1a87

Change-Id: I945a5f1f6782791736cd319f216cfa6b448fb04d
2016-10-04 17:32:16 -04:00
Chih-Hung Hsieh
8fdda6e495 Accept environment variables DEFAULT_*_TIDY_CHECKS
* Define DEFAULT_GLOBAL_TIDY_CHECKS/DEFAULT_EXTERNAL_VENDOR_TIDY_CHECKS
  only if it is not already defined.

Bug: http://b/27779618
Test: build with WITH_TIDY=1 and define those variables.
Change-Id: I201c9da3769d6ef658a75d4a1bf42ec6816d4069
2016-10-04 10:18:02 -07:00
Colin Cross
d2ff157159 Enable toc optimization for host builds
The toc optimization had been disabled for host builds to ensure that
the timestamp of the final binary changed whenever its implementation
changed, in order to support rerunning host tools that were modified
during incremental builds.  However, only the final install rule must be
re-run to update the timestamp, and not the link rule.

Update the shared library install dependencies to use normal
dependencies instead of order-only dependencies for host modules, and
then enable the the toc optimization for host modules.  If the
implementation of a library changes it will be reinstalled, and
libraries or binaries that depend on it will also be reinstalled.

Bug: 26015464
Test: m -j; touch art/disassembler/disassembler_x86.cc; m -j, verify
      out/host/linux-x86/bin/oatdump is updated
Change-Id: I0a14decc1994eb55ad269d841943aef66e320c63
2016-10-03 17:53:44 -07:00
Colin Cross
a5471a6723 exec makeparallel
bash doesn't kill children when it receives SIGTERM.  Since makeparallel
is the last command run by the shell, run it with exec to replace the
shell with makeparallel so make can send the signal directly to
makeparallel.

Bug: 31907490
Test: m -j & killall make; pgrep -a makeparallel
Change-Id: If8aeb51ec224234da5f05eae3382ce77f48a57e2
2016-10-03 15:33:06 -07:00
Colin Cross
647d383788 Revert "Remove -d keepdepfile from ninja command line"
This reverts commit 4aca571552.

Test: builds
Change-Id: Iddb3517a4596e84404db1738fa95eb15f435a42d
2016-10-03 15:28:58 -07:00
Dan Willemsen
f009531772 Merge "Optimize rebuilds by reducing $(shell) usage" 2016-10-03 21:20:36 +00:00
Dan Willemsen
b011810c25 Optimize rebuilds by reducing $(shell) usage
$(shell) isn't particularly fast in Kati, and they have to be executed
both when reading the makefiles and determining whether the ninja file
needs to be regenerated.

Right now, the regen time is mostly hidden because we run them in
parallel. We've also configured it to ignore any commands that contain
"echo", "date", or the output directory. That happens to remove most
commands that contain side effects, so running them in parallel is fine.

But the side effects contain some important things, like the clean up
necessary when switching products. So I'm removing those filters, and
then we'll need to run the shell commands in sequence, since there will
be side-effects. That makes regen take longer though, so use pure-Make
implementations instead of $(shell) where possible.

This set of changes reduces aosp/master aosp_arm64-eng build $(shell)
usage and time by 2/3:

*kati*: func shell time: 3.135095 / 709
*kati*: func shell time: 1.067331 / 236

Bug: 30947985
Test: Manual test lines for math functions
Test: Compare build-aosp_arm64.ninja before/after
Change-Id: I4fc9d6318957992921972994f277c17918e7e1eb
2016-09-30 21:14:13 -07:00
Dan Willemsen
847d965b9e Merge "Remove LOCAL_CTS_GTEST_LIST_EXECUTABLE" 2016-10-01 01:43:37 +00:00
Treehugger Robot
3d38042a00 Merge "Use DEFAULT_EXTERNAL_VENDOR_TIDY_CHECKS for vendor/google_devices" 2016-09-30 23:25:17 +00:00
Chih-Hung Hsieh
10efc73c06 Use DEFAULT_EXTERNAL_VENDOR_TIDY_CHECKS for vendor/google_devices
Many source files under vendor/google_devices
are old third party code with too many clang-tidy
misc-macro-parentheses warnings.

Test: build with WITH_TIDY=1
Change-Id: I79248af3ece9c7a4436b8cf5e6c25f224fd8201b
2016-09-30 14:50:43 -07:00
Dan Willemsen
3aa39a4903 Remove LOCAL_CTS_GTEST_LIST_EXECUTABLE
The one user has now been removed, so this is no longer necessary.

Test: None
Change-Id: I8fb9e7b90f664af616368685d81f8b10d0687306
2016-09-29 22:37:04 -07:00
Treehugger Robot
1c0b5eaf83 Merge "Add LOCAL_EXPORT_C_INCLUDE_DEPS" 2016-09-30 01:38:04 +00:00
Dan Willemsen
ba6a164da7 Add LOCAL_EXPORT_C_INCLUDE_DEPS
This shouldn't be necessary for most makefiles, since we add
$(LOCAL_GENERATED_SOURCES) to this already included, but for prebuilts
that aren't actually prebuilts (Soong modules), we need to pass
dependencies so that the generated headers are built before being used.

Bug: 31742855
Test: Use, inspect build-*.ninja
Change-Id: I8d9f675af639d3f40780d48a016fc079a82531c2
2016-09-29 12:08:29 -07:00
David Zeuthen
8ee6d119d1 Merge "Update for new Android Verified Boot (AVB)." 2016-09-29 17:48:40 +00:00
David Zeuthen
2ce63edab7 Update for new Android Verified Boot (AVB).
This updates the build system for the new Android Verified Boot
codebase. As this is based on Brillo Verified Boot, this change replaces
the existing BVB support.

Android Verified Boot is enabled by the BOARD_AVB_ENABLE variable

 BOARD_AVB_ENABLE := true

This will make the build system create vbmeta.img which will contain a
hash descriptor for boot.img, a hashtree descriptor for system.img, a
kernel-cmdline descriptor for setting up dm-verity for system.img and
append a hash-tree to system.img.

Additionally, the descriptors are left in boot.img and system.img so a
third party can create their own vbmeta.img file linking - using the
option --chain_partition - to these images. If this is not needed
footers can be erased using the 'avbtool erase_footer' command. It's
also harmless to just leave them in the images.

By default, the algorithm SHA256_RSA4096 is used with a test key from
the AVB source directory. This can be overriden by the
BOARD_AVB_ALGORITHM and BOARD_AVB_KEY_PATH variables to use e.g. a
4096-bit RSA key and SHA-512:

 BOARD_AVB_ALGORITHM := SHA512_RSA4096
 BOARD_AVB_KEY_PATH := /path/to/rsa_key_4096bits.pem

To prevent rollback attacks, the rollback index should be increased on a
regular basis. The rollback index can be set with the
BOARD_AVB_ROLLBACK_INDEX variable:

 BOARD_AVB_ROLLBACK_INDEX := 5

If this is not set, the rollback index defaults to 0.

The variable BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS can be used to specify
additional options passed to 'avbtool make_vbmeta_image'. Typical
options to be used here include '--prop', '--prop_from_file', and
'--chain_partition'.

The variable BOARD_AVBTOOL_BOOT_ADD_HASH_FOOTER_ARGS can be used to
specify additional options passed to 'avbtool add_hash_footer' for
boot.img. Typical options to be used here include '--hash_algorithm' and
'--salt'.

The variable BOARD_AVBTOOL_SYSTEM_ADD_HASHTREE_FOOTER_ARGS can be used
to specify additional options passed to 'avbtool add_hashtree_footer'
for systems.img. Typical options to be used here include
'--hash_algorithm', '--salt', and '--block_size'.

BUG=31264226
TEST=Manually tested on edison-eng by inspecting {boot, system,
  vbmeta}.img in out/ directory as well as their counterparts in
  the IMAGES/ directory of edision-target_files-eng.zeuthen.zip

Merged-In: Ic9a61cfc65c148b12996e57f04da5432eef6b982

Change-Id: I97042655bca15e7eac899f12c5bada2f6184d307
2016-09-29 11:19:36 -04:00
Stephen Hines
183b142ac9 Switch to using clang-3289846.
Bug: http://b/30252254
Test: Built/boot platform using latest prebuilts.
Change-Id: I7c9357a99afec323fd766e93b5d5e941e9e06244
2016-09-28 08:51:55 -07:00
Treehugger Robot
c114bd9dbf Merge "Remove WITH_SYNTAX_CHECK" 2016-09-28 01:44:24 +00:00
Evgenii Stepanov
2358c55d70 Merge "Enable LOCAL_SANITIZE:=cfi and add LOCAL_SANITIZE_DIAG." 2016-09-27 20:51:41 +00:00
Dan Willemsen
e68100e8bb Remove WITH_SYNTAX_CHECK
It's no longer used. The majority of the platform is on clang anyways.

Test: m -j
Change-Id: I4fb4375eaad2469e169d34c47ae3fe2a0540aeec
2016-09-26 17:37:19 -07:00
Dan Willemsen
69811f89d6 Remove spaces from DEFAULT_TIDY_HEADER_DIRS
I'm preparing to move this to Soong, and it's simpler to compare if the
spaces are removed here (to match the other lists in this file).

Test: WITH_TIDY=true; compare build.ninja before and after change
Change-Id: Ib27a19a36ebb1132300a8c1eeace1202685dfc4c
2016-09-26 16:02:15 -07:00
Treehugger Robot
8870658397 Merge "link_type check: Support modules installed into /data" 2016-09-23 04:30:02 +00:00
Treehugger Robot
19f24492db Merge "Pass userdebug/eng variant to Soong" 2016-09-23 00:16:52 +00:00
Dan Willemsen
f2649570c5 Pass userdebug/eng variant to Soong
Test: Run build with eng/userdebug/user and check the soong.variables
Change-Id: I7cd71a94e92526b6cc61303b13b7bc593cbbfc3a
2016-09-22 15:22:48 -07:00
Dan Willemsen
3594cee467 link_type check: Support modules installed into /data
Test: compare build.ninja before/after, look at warnings.html
Change-Id: I1b78748dd80b73d2601b2adf0aaac8a6b2b350d5
2016-09-22 13:05:02 -07:00
Benoit Lamarche
52eafee87a Merge changes from topic 'jack-d-b3-as-stable'
* changes:
  Fix bad JACK_PLUGIN variable
  Add support for Jack plugin
2016-09-22 09:34:27 +00:00
Dan Willemsen
f7d1a6d75f Merge "Package up proguard jack_dictionary" 2016-09-21 19:39:47 +00:00
Yohann Roussel
16243bdbec Fix bad JACK_PLUGIN variable
Changed to LOCAL_JACK_PLUGIN.

(cherry picked from commit 17924b136cb5bfc60d138e8c7e5c18a90b5965c3)

Test: Manually tested by some manual activations of the coverage plugin.
Change-Id: I1be4aaa502103fc308d35a5672dc4fe5900c4ebe
2016-09-21 10:54:42 +02:00
Yohann Roussel
53b7fbbe78 Add support for Jack plugin
And ensure compatibility with coverage plugin.

(cherry picked from commit 862bb84d37e3e6ca61080de58f206e512e09d4e0)

Bug: 28876950
Test: Manually tested by some manual activations of the coverage plugin.
Change-Id: I804558a501825357bf0812de626d2957eedbdc13
2016-09-21 10:53:45 +02:00
Colin Cross
3388670e78 Replace usages of my_register_name with my_all_targets
A few module types add extra dependencies on $(my_register_name), move
them to $(my_all_targets) so that they are built for mm and mma too.

Bug: 31526036
Test: mmm framework
Change-Id: I2e594ce771451a99691739b963f4ce517e9dd595
2016-09-20 16:03:50 -07:00
Dan Willemsen
b6c061e9ff Package up proguard jack_dictionary
Bug: 31182185
Test: m dist in an unbundled branch
Change-Id: I7ee1ba076cfa03f294f8231f6bdd8fa8df5557af
2016-09-20 14:21:53 -07:00
Paul Duffin
cb6f7c2d86 Merge "Add targets for building individual modules using javac" 2016-09-20 11:51:40 +00:00
Dan Willemsen
665cda08f0 Merge "Fix link_type checking" 2016-09-19 21:54:09 +00:00
Paul Duffin
32bd0adf9c Add targets for building individual modules using javac
This is needed in order to be able to run error prone (which is
a plugin replacement for javac) on a subset of the java code.

Bug: 31507496
Test: Build javac-check-core-tests and check the resulting JAR
Change-Id: I5a556920572fcdad7b39d67e3f00fe0e0f1f8f46
2016-09-19 13:38:51 +01:00
Dan Willemsen
2cfbdf460d Only update previous product config when changed
I'm working on removing the --ignore_dirty=$(OUT_DIR)/% argument from
Kati. If we're always writing to a file that we're also reading, then
with that flag removed, we'll always reparse all the makefiles.

So instead, use the method from the generated java file cleanup where we
read from a "previous" file, write to a "current" file, then use a
cmp || mv to overwrite the "previous" file if necessary. When the flag
is removed, this will cause Kati to need to run twice any time this
changes, but it's better than forever.

Bug: 30947985
Test: Switch between aosp_arm-eng and full-eng
Change-Id: Id2d1445809b60ce26700bb2aca765df3b5c2b360
2016-09-16 13:17:08 -07:00
Dan Willemsen
da7d25f2d1 Remove objclean
We don't need to manually remove objects when switching SANITIZE_TARGET
-- that will be handled by Ninja noticing that the command lines are
different.

Bug: 30947985
Test: None
Change-Id: I83c15981cf6ea258bb7fa85490b0b1ede1058aa6
2016-09-16 12:53:20 -07:00
Treehugger Robot
7b351cadfc Merge changes from topics 'ndk-compiler-rt_extras', 'ndk-libc++-libdl'
* changes:
  Use libcompiler-rt_extras with NDK builds.
  Always link libdl with libc++ for the NDK.
2016-09-16 06:40:32 +00:00
Ryan Campbell
737bc02ee8 Fix typo in coverage target out path.
Change-Id: I4a448708ab74e08b2c515039ee5d71c32861214e
2016-09-15 17:32:13 -07:00
Dan Willemsen
121e284b46 Fix link_type checking
This was printing "error:", but not actually triggering an error.
Instead of trying to write a single line bash script to handle this,
move the actual check into python. This allows us to print all of the
errors for a single module before triggering the failure.

Also updates the warning format and the warn.py script to properly parse
these warning. Many of the java:sdk -> java:platform warnings are false
positives due to the lack of LOCAL_SDK_VERSION markings on prebuilts.

Individual tags can be marked as warnings now, which lets us check for
system libraries linking against vendor libraries (which won't work on
AOSP). I'm not sure this is a completely valid check, which one reason
that it's just a warning.

Test: m all_link_types (with some missing libs commented out)
Change-Id: I333e418c9a4511b7c7e826891ae481da08fbf6f9
2016-09-15 14:40:39 -07:00
Treehugger Robot
4a6862e59b Merge "Remove SRC_SYSTEM_HEADERS" 2016-09-15 18:27:45 +00:00
Colin Cross
4ed13c38a5 Remove SRC_SYSTEM_HEADERS
It is inherited from Soong now.

Bug: 31492149
Change-Id: Ia9bfd9662dba16032dac68b55b4f0743c0688139
2016-09-15 09:29:15 -07:00
Treehugger Robot
766c348748 Merge "goma.mk: do not start goma compiler_proxy if NOSTART_GOMA is set." 2016-09-15 03:41:24 +00:00
Dan Albert
c28e4bcb1c Use libcompiler-rt_extras with NDK builds.
We want to build things in CTS (things built against the NDK) with
integer overflow checks. Some projects in the tree also make explicit
calls to the overflow checked builtins, and those projects need to be
built with the NDK (external/dng_sdk is the specific example that
brought this up).

Test: make checkbuild tests
Bug: None
Change-Id: Iab27dd0d931f723140de242049fa72f3b1ce0fb8
2016-09-14 16:08:48 -07:00
Dan Albert
2936a1e84b Always link libdl with libc++ for the NDK.
libc++ requires libdl whether it's statically linked or not.

Test: make checkbuild tests
Bug: None
Change-Id: I8abdcfaef779b358196716ab1193cebe278317ac
2016-09-14 16:08:04 -07:00
Dan Willemsen
5bdd08b2d9 Fix shared library install dependencies outside of /system
For properly picking the shared library install dependencies, HOST_OUT
and HOST_CROSS_OUT are necessary so that we don't add a dependency
between Linux and Windows. But for target modules, we must use
TARGET_OUT_ROOT instead of TARGET_OUT, or modules in /data and /vendor
won't have proper install dependencies.

Bug: 31462199
Test: m bionic-unit-tests; manual diff of generated ninja file
Change-Id: I828fa2664613296cc6c431813f5657abffdd2b8f
2016-09-13 13:47:06 -07:00
Ryan Campbell
1e1ecd5d07 Merge "Implement path-based enabling of code coverage." 2016-09-13 19:48:34 +00:00
Dan Willemsen
c9aa6fca41 Fix building static linux binaries
BUILD_HOST_static has been broken because I was stripping
-Wl,--start-group/-Wl,--end-group into a separate set of ldflags instead
of keeping them wrapped around some of the ldlibs. So don't pass some of
the build system flags through the ldlibs checker.

Bug: 30898128
Change-Id: I332b42679695cdc5da3cb0036290b6a3544699c0
2016-09-13 17:48:07 +00:00
Ryan Campbell
81c9d29dad Implement path-based enabling of code coverage.
Native coverage is enabled by setting NATIVE_COVERAGE to true
and specifying a list of paths in the COVERAGE_PATHS
environment variable. Files are exported to a zip file in the
target out directory.

Change-Id: I66a2ddd88e849bec1cc0cdae1b51fe18a007e2c3
2016-09-13 10:27:25 -07:00