Commit graph

41279 commits

Author SHA1 Message Date
Colin Cross
ce8c05196d Partially revert BUILD_DATETIME change
Change Ibe4ceb1b224c5e8f3e1a8a59cfee89217765a901 caused kati to
regenerate on every build:
$(shell cat out/build_date.txt) was changed, regenerating...
Switch back to $(shell date +%s) which is special cased in kati
to not cause a regeneration.  Everything besides DATE will still
use BUILD_DATETIME_FROM_FILE from soong_ui.

Once BUILD_NUMBER is moved to soong_ui $(DATE) is no longer
necessary and the whole thing can be removed.

Bug: 70351683
Bug: 73352417
Test: m && m doesn't regenerate
Test: m BUILD_DATETIME=1234 shows up in ro.build.version.incremental
Change-Id: I8444beb2b47a49095ef760083fc85f220821bffe
2018-02-16 09:41:24 -08:00
Treehugger Robot
dc6e822374 Merge "NO_STANDARD_LIBRARIES := true blocks ALL resource libraries" 2018-02-16 03:35:28 +00:00
Treehugger Robot
87774191b3 Merge "Record the compilation reson in oat files for prebuilts" 2018-02-16 02:28:31 +00:00
Treehugger Robot
304be2ec74 Merge "Build: Move LOCAL_ERROR_PRONE_FLAGS" 2018-02-16 01:04:45 +00:00
Adam Lesinski
6b6283a042 NO_STANDARD_LIBRARIES := true blocks ALL resource libraries
LOCAL_RES_LIBRARIES and LOCAL_APK_LIBRARIES should still be honored.

Test: make CtsDevicePrereleaseSdkApp
Change-Id: I3360d0fe2f7e2f3038e48f41905923a72609ceec
2018-02-15 16:53:18 -08:00
Calin Juravle
55f149263c Record the compilation reson in oat files for prebuilts
Test: build
Bug: 73102540
Change-Id: Ia2ba0670f36b3eeb481bc7b2ae44a1c89777bfc1
2018-02-15 16:00:15 -08:00
Treehugger Robot
b9a66a6250 Merge "Remove core/clang/versions.mk and scan-build wrapper" 2018-02-15 23:56:36 +00:00
Andreas Gampe
21a1c6d03f Build: Move LOCAL_ERROR_PRONE_FLAGS
Move application of module flags to host_java_library_common.mk
so that dalvik host builds can also have flags.

Bug: 72451126
Test: m javac-check RUN_ERROR_PRONE=true
Change-Id: Idbf43afd886536cc7ea674aa4487da3c22987850
2018-02-15 15:02:15 -08:00
Treehugger Robot
4babe39d0f Merge "Remove BUILD_DATETIME from Make." 2018-02-15 22:30:49 +00:00
Chih-Hung Hsieh
7ed0db8d1d Remove core/clang/versions.mk and scan-build wrapper
* Use the definitions of ClangDefaultBase and ClangDefaultVersion
  in build/soong/cc/config/global.go.
* scan-build is too old;
  replaced by better clang-tidy's -clang-analyzer-* checks.
* WITH_STATIC_ANALYZER is still effective
  to invoke ccc-analyzer or c++-analyzer.

Test: make checkbuild
Change-Id: Ifd159dd28e529d5d3c9807f5e391911365da8cdb
2018-02-15 13:38:50 -08:00
Meng Wang
bd48f84f17 Merge "AOSP telephony uses device/sample APN XML" 2018-02-15 17:37:57 +00:00
Treehugger Robot
b2cc4ddafd Merge "emulator: grant fingerprint hal data permission" 2018-02-15 16:34:36 +00:00
Meng Wang
5151c6c2b8 AOSP telephony uses device/sample APN XML
Bug: 73091197
Test: make - /system/etc/apns-conf.xml same as
      device/sample/etc/apns-full-conf.xml

Change-Id: I5e9651066e7a5d4de31d846daf3fe3120476b89f
2018-02-15 05:54:52 +00:00
bohu
72fa12db3c emulator: grant fingerprint hal data permission
Test: build sdk_gphone_x86-userdebug
launch emulator, enroll fingerprint, close emulator
relaunch emulator, check previously enrolled fingerprint still work

Change-Id: I27552509368f08cc6a7aa52b0e9efc5ae952469d
2018-02-14 15:37:31 -08:00
Nan Zhang
975f41ed34 Remove BUILD_DATETIME from Make.
Soong_UI will handle build timestamp and export date file to Make.

Test: m -j32
Bug: b/70351683
Change-Id: Ibe4ceb1b224c5e8f3e1a8a59cfee89217765a901
2018-02-14 13:30:53 -08:00
Treehugger Robot
2a519d5a9b Merge "Disable duplicates warnings when merging jars" 2018-02-14 20:52:30 +00:00
Colin Cross
eefc171d1f Merge "Revert "Remove art targets from PARSE_TIME_MAKE_GOALS"" 2018-02-14 19:14:17 +00:00
Colin Cross
6bd8e28571 Revert "Remove art targets from PARSE_TIME_MAKE_GOALS"
This reverts commit 86a596c711.

Reason for revert: Broke aosp_x86_64-eng checkbuild

Change-Id: Ib13b967ee56efdf3053ef1a292a07c1b82972588
2018-02-14 19:13:04 +00:00
Colin Cross
b060ba4b92 Merge "Remove art targets from PARSE_TIME_MAKE_GOALS" 2018-02-14 18:36:15 +00:00
Colin Cross
5b73ab34ed Disable duplicates warnings when merging jars
There are too many duplicates warnings, disable them until someone
is actually looking at them.

Bug: 69316739
Test: m checkbuild
Change-Id: Ifb67dbf7734b5f53941d110f578c4fe642e36005
2018-02-14 18:35:37 +00:00
Treehugger Robot
9c339a76c3 Merge "BUILD_DROIDDOC supports LOCAL_NO_STANDARD_LIBRARIES" 2018-02-14 18:27:08 +00:00
Jiyong Park
889773f7ab BUILD_DROIDDOC supports LOCAL_NO_STANDARD_LIBRARIES
When LOCAL_NO_STANDARD_LIBRARIES is set to true, droiddoc only omits
framework libraries from the required libs. This enables us to use
droiddoc even when frameworks/base is absent (i.e. master-art branch).

Test: m -j core.current.stubs
Test: build in master-art branch
Change-Id: Ia9f0b8bda84b006881b3e62e6573a7784ca56a8c
2018-02-14 21:16:15 +09:00
Treehugger Robot
895a8172ee Merge "Create symbolic boot vdex files for different ISAs" 2018-02-14 00:59:56 +00:00
Christopher Ferris
184b4d8560 Merge "Add libc_malloc_hook to list of libraries." 2018-02-14 00:49:48 +00:00
Treehugger Robot
42c7cf0631 Merge "Enable BOARD_PROPERTY_OVERRIDES_SPLIT_ENABLED for full-treble products" 2018-02-14 00:30:23 +00:00
Tao Bao
9452d84b97 Merge "releasetools: Allow generating BBOTA for images with shared blocks." 2018-02-13 21:53:25 +00:00
Tao Bao
6392e05813 Merge "releasetools: Remove the unconditional fallback to bsdiff." 2018-02-13 21:53:17 +00:00
Christopher Ferris
649b8c1127 Add libc_malloc_hook to list of libraries.
Bug: 30561479

Test: Build from scratch and verify the library is present.
Change-Id: I53f30d694cff202ffc93491485ed3c845de07862
(cherry picked from commit 8c70d4de97)
2018-02-13 13:38:26 -08:00
Tao Bao
4f0b725439 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload." 2018-02-13 20:20:30 +00:00
Treehugger Robot
73e886ec62 Merge "Pass PGO_ADDITIONAL_PROFILE_DIRS to soong" 2018-02-13 19:48:21 +00:00
Tao Bao
e709b094e4 releasetools: Allow generating BBOTA for images with shared blocks.
When target defines 'BOARD_EXT4_SHARE_DUP_BLOCKS := true', the generated
system/vendor images may contain shared blocks (i.e. some blocks will
show up in multiple files' block list), which violates the current
assumptions in BBOTA script.

This CL allows generating BBOTAs by considering the first occurrence as
the "owner" of the shared blocks. All the later users of the shared
blocks will have an incomplete block list, whose RangeSet's will be
tagged with 'uses_shared_blocks'.

Files with 'uses_shared_blocks' tag will not be diff'd with imgdiff,
potentially with patch size penalty. Such files will be accounted for in
imgdiff stats report, where we can revisit for a better solution.

Bug: 64109868
Test: Generate BBOTA full and incremental package with targets defining
      'BOARD_EXT4_SHARE_DUP_BLOCKS := true'.
Change-Id: I87fbc22eef7fafe2a470a03fdcfa1babf088ea8d
2018-02-13 11:14:23 -08:00
Tao Bao
4ccea8549e releasetools: Remove the unconditional fallback to bsdiff.
This CL uses the 'incomplete' tag to skip applying imgdiff to files with
incomplete block list. It's not the ideal fix to address the holes in
ext4 images, but would unhide other imgdiff issues covered by the
unconditional fallback.

Bug: 68016761
Test: Generate an incremental OTA package from images with incomplete
      block list. Check the imgdiff stats report.
Test: `python -m unittest test_blockimgdiff`
Change-Id: Ice77686414e70f5e42de35c1757fb31cf02e4fd4
2018-02-13 11:06:20 -08:00
Pirama Arumuga Nainar
676fbe7e5e Pass PGO_ADDITIONAL_PROFILE_DIRS to soong
This variable can be set in BoardConfig.mk to specify a list of
additional paths that contain PGO profiles.  These directories are
searched before the predefined paths in soong/cc/pgo.go while finding
PGO profiles.

Test: Set this variable in a BoardConfig and verify that such profiles
are found and that these paths are searched before the predefined paths
in soong/cc/pgo.go.

Change-Id: Ied3ebf0ce6e6e2ead57f4828998e9617424e4573
2018-02-13 09:32:44 -08:00
Tao Bao
4a066a2cd2 Merge changes Ia7ecd0e1,I33982473
* changes:
  releasetools: Make blockimgdiff.py pylint-clean.
  releasetools: Remove the global diff_done in blockimgdiff.py.
2018-02-13 17:13:17 +00:00
Treehugger Robot
5502f9958a Merge "Add bpfloader binary to system image" 2018-02-13 05:49:46 +00:00
bowen_lai
4570fdbb4c Create symbolic boot vdex files for different ISAs
Create only one vdex file and share between different ISAs.
By sharing vdex file, zygote32 and zygote64 can share the same file and
the system partition size is reduced by about 32MB.

Test: make framework & check boot.vdex and boot-*.vdex files
Test: test system boots with no selinux violations

Bug: 64211239

Change-Id: I31ede4b6aa342f2047d9309b8c631217e6699202
2018-02-12 19:45:21 -08:00
Tao Bao
508b087943 releasetools: Make blockimgdiff.py pylint-clean.
************* Module blockimgdiff
C:433, 0: Unnecessary parens after 'if' keyword (superfluous-parens)
C:687, 0: Wrong hanging indentation (add 4 spaces).
            max_stashed_blocks, self._max_stashed_size, max_allowed,
            ^   | (bad-continuation)
C:688, 0: Wrong hanging indentation (add 4 spaces).
            self._max_stashed_size * 100.0 / max_allowed))
            ^   | (bad-continuation)
C:691, 0: Wrong hanging indentation (remove 2 spaces).
            max_stashed_blocks, self._max_stashed_size))
          | ^ (bad-continuation)
C:898, 0: Wrong hanging indentation (add 4 spaces).
                  "imgdiff" if imgdiff else "bsdiff",
                  ^   | (bad-continuation)
C:899, 0: Wrong hanging indentation (add 4 spaces).
                  xf.tgt_name if xf.tgt_name == xf.src_name else
                  ^   | (bad-continuation)
C:901, 0: Wrong hanging indentation (add 4 spaces).
                  xf.tgt_ranges, xf.src_ranges, e.message))
                  ^   | (bad-continuation)
C:909, 0: Wrong hanging indentation (add 4 spaces).
                      xf.tgt_name,))
                      ^   | (bad-continuation)
C:917, 0: Wrong hanging indentation (add 4 spaces).
                      xf.tgt_name, e.message))
                      ^   | (bad-continuation)
C:961, 0: Wrong hanging indentation (remove 2 spaces).
                xf.patch_len, tgt_size, xf.patch_len * 100.0 / tgt_size,
              | ^ (bad-continuation)
C:962, 0: Wrong hanging indentation (remove 2 spaces).
                xf.style,
              | ^ (bad-continuation)
C:963, 0: Wrong hanging indentation (remove 2 spaces).
                xf.tgt_name if xf.tgt_name == xf.src_name else (
              | ^ (bad-continuation)
C:965, 0: Wrong hanging indentation (remove 2 spaces).
                xf.tgt_ranges, xf.src_ranges))
              | ^ (bad-continuation)
C:1422, 0: Wrong continued indentation (add 28 spaces).
                tgt_skipped.size() * 100.0 / tgt_size, tgt_name))
                ^                           | (bad-continuation)
C:1560, 0: Wrong continued indentation (add 8 spaces).
            split_src_ranges) in enumerate(split_info_list):
            ^       | (bad-continuation)
R:566, 6: Redefinition of src_str type from list to str (redefined-variable-type)
C:1198,25: More than one statement on a single line (multiple-statements)
C:1211,25: More than one statement on a single line (multiple-statements)
C:1220,16: More than one statement on a single line (multiple-statements)
C:1277,38: More than one statement on a single line (multiple-statements)
C:1284,19: More than one statement on a single line (multiple-statements)
C: 19, 0: standard import "import copy" comes before "import common" (wrong-import-order)
C: 20, 0: standard import "import functools" comes before "import common" (wrong-import-order)
C: 21, 0: standard import "import heapq" comes before "import common" (wrong-import-order)
C: 22, 0: standard import "import itertools" comes before "import common" (wrong-import-order)
C: 23, 0: standard import "import multiprocessing" comes before "import common" (wrong-import-order)
C: 24, 0: standard import "import os" comes before "import common" (wrong-import-order)
C: 25, 0: standard import "import os.path" comes before "import common" (wrong-import-order)
C: 26, 0: standard import "import re" comes before "import common" (wrong-import-order)
C: 27, 0: standard import "import subprocess" comes before "import common" (wrong-import-order)
C: 28, 0: standard import "import sys" comes before "import common" (wrong-import-order)
C: 29, 0: standard import "import threading" comes before "import common" (wrong-import-order)
C: 31, 0: standard import "from collections import deque, OrderedDict" comes before "import common" (wrong-import-order)
C: 32, 0: standard import "from hashlib import sha1" comes before "import common" (wrong-import-order)

Test: `pylint --rcfile=pylintrc blockimgdiff.py`
Test: Generate an incremental BBOTA package.
Change-Id: Ia7ecd0e1fa48daf4e43251bdcdfcd08fb316015d
2018-02-12 13:01:34 -08:00
Tao Bao
9739514769 releasetools: Remove the global diff_done in blockimgdiff.py.
pylint complains about undefined `diff_done`:

W:754, 8: Global variable 'diff_done' undefined at the module level (global-variable-undefined)
W:820,14: Global variable 'diff_done' undefined at the module level (global-variable-undefined)

It would still warn about using global statement after adding the
definition.

W:859, 8: Using the global statement (global-statement)
W:925,14: Using the global statement (global-statement)

This CL computes 'diff_done' via 'len(diff_queue)' instead. It also
moves the progress reporting _before_ the diff work. This way it avoids
showing 100% progress with still changing filenames (because multiple
workers could see an empty queue simultaneously upon finishing their own
works).

There're possible alternatives, such as using the 'nonlocal' keyword in
Python 3 (which we're not there yet), or by using mutable object instead
(e.g. 'diff_done = [0]'). This CL looks cleaner, since it just kills the
var.

Test: Generate a BBOTA incremental. Check the on-screen progress
      report.
Test: `pylint --rcfile=pylintrc blockimgdiff.py` no longer complains
      about the global diff_done.
Change-Id: I339824735527e1f794b5b1dc99ff3fdb2da85744
2018-02-12 12:55:25 -08:00
Tao Bao
7eb2afb226 Merge "releasetools: Add an ImgdiffStats class that reports imgdiff stats." 2018-02-12 19:01:24 +00:00
Tom Cherry
3d9fa3615d Merge "Place TARGET_FS_CONFIG_GEN passwd/group files in /vendor/etc" 2018-02-12 16:27:20 +00:00
Tao Bao
667ff57272 releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload.
The secondary payload should always be applied with
SWITCH_SLOT_ON_REBOOT=0. This CL moves the 'secondary' parameter from
Payload.WriteToZip() to Payload.__init__(). So it can append the flag to
secondary/payload_properties.txt.

Bug: 35724498
Test: Generate an A/B OTA with --include_secondary. Check
      secondary/payload_properties.txt entry in the generated ZIP.
Test: `python -m unittest test_ota_from_target_files`
Change-Id: I816c07ab57a1c8a52eff785801634b8b1cb134d4
2018-02-10 00:13:07 -08:00
Tao Bao
294651d7b4 releasetools: Add an ImgdiffStats class that reports imgdiff stats.
We have a couple of active imgdiff workarounds (and likely with one more
inbounding that allows having shared blocks in ext4 image). Most of
these workarounds need extending imgdiff's capability. While us not
getting there anytime soon, collect the stats to better understand the
impact of each kind so we can prioritize accordingly.

A sample report is as follows.

  Imgdiff Stats Report
========================

 APK files diff'd with imgdiff (count: 88)
-------------------------------------------

  /system/priv-app/Shell/Shell.apk
  ...

 Large APK files split and diff'd with imgdiff (count: 4)
----------------------------------------------------------

  /system/priv-app/Settings/Settings.apk
  ...

Bug: 68016761
Test: Generate an incremental BBOTA package. Check the stats report.
Test: python -m unittest test_blockimgdiff
Change-Id: I27ad862cde472ab2806db877632ce5a0607420f2
2018-02-09 23:02:32 -08:00
Tao Bao
bec8be51a5 Merge "releasetools: Refactor the condition checking for using imgdiff." 2018-02-10 00:47:42 +00:00
Chenbo Feng
2b948a2259 Add bpfloader binary to system image
The bpfloader is a executable binary used by netd to load and attach
networking related bpf program at run time. It need to be flashed to
device when updating the system image. Currently this binary is only
used by devices with kernel version 4.9 or above.

Test: hikey boot and bpf program show up under sys/fs/bpf
Bug: 30950746
Change-Id: I3d14d270a04ea57319ad4292ff51b1cf933fc975
2018-02-09 15:17:22 -08:00
Tao Bao
cb73aed1f0 releasetools: Refactor the condition checking for using imgdiff.
In Transfer class, unbundle 'intact' with the monotonicity of the input
ranges. Negate the logic of 'intact', and thus rename it to 'trimmed'.
Move this property from an attribute of Transfer class as the one in
RangeSet.extra. 'trimmed' indicates whether the source / target ranges
have been modified after creating the Transfer() instance.

The logic that determines whether we can apply imgdiff has been
refactored and consolidated into BlockImageDiff.CanUseImgdiff(). Now
both of the two paths call this single copy, i.e. the one that detects
large APKs (before creating Transfer()'s), and the other that's about to
generate the patch for a given Transfer instance.

Bug: 68016761
Test: python -m unittest test_blockimgdiff
Test: Generate an incremental BBOTA package.
Change-Id: Id07195f63f1fa6c3af6e9091940d251cf09fa104
2018-02-09 12:46:01 -08:00
Dan Willemsen
2744cbf190 Merge "Revert "Propagate proguard rules from LOCAL_STATIC_ANDROID_LIBRARIES"" 2018-02-09 19:22:13 +00:00
Dan Willemsen
30d6869faf Revert "Propagate proguard rules from LOCAL_STATIC_ANDROID_LIBRARIES"
This reverts commit df5d7da7d8.

Reason for revert: breaking builds

Bug: 72969557
Change-Id: I27aad65587933c3f760a89c541d27469c916abe8
2018-02-09 19:21:11 +00:00
Tom Cherry
9f25d873bf Merge "Add compile time check that friendly AID names are < 32 characters" 2018-02-09 18:46:27 +00:00
Tom Cherry
ee0610e86c Add compile time check that friendly AID names are < 32 characters
There is an internal buffer in bionic for user/group names that is 32
characters long including the trailing null, so we must restrict the
length of these names during compile time.

Bug: 27999086
Test: Successfully compile a valid config.fs
Test: Fail to compile a config.fs with AID name > 31 characters
Change-Id: I7fe887c630dd4d1033b86a5d8332480eb3b0fa07
2018-02-09 18:46:13 +00:00
Tom Cherry
55d0104f1b Place TARGET_FS_CONFIG_GEN passwd/group files in /vendor/etc
These entries are vendor provided and belong on the /vendor partition.

Bug: 27999086
Test: end to end user/group check via config.fs and getpwnam, etc.
Change-Id: I9a5d56da594bf0d04de2b9ce7fd7d9a8151d4682
2018-02-09 10:44:52 -08:00