Commit graph

47299 commits

Author SHA1 Message Date
Anton Hansson
700db42d54 Quote message in echo-error/echo-warning macros
Makes pretty-error and pretty-warning behave more like $(error) and
$(warning), where you don't need to do things like quote parentheses.

Bug: 118833208
Test: trigger the private_apis && sdk set error in sdk_check.mk,
      which no longer produces bash syntax errors

Change-Id: I766ff98ad4e652f59dbef9dd5654f1cd10a2d038
2018-11-09 08:59:25 +00:00
Anton Hansson
41ec0a65a4 Merge "Add vendor restrict to mainline targets" 2018-10-30 07:43:55 +00:00
Treehugger Robot
6b38c9bb4c Merge changes Ia8fb5696,Id5ae6645
* changes:
  build_image setup ext4 dedupe on more images.
  build_image tune right-size image
2018-10-29 23:33:24 +00:00
Mark Salyzyn
f0cef8d8bf build_image setup ext4 dedupe on more images.
Add ext4 dedupe to system_other, product, product_services, odm and
oem images.  Experimental savings without any other configuration
changes was 0%, .75%, 3.4% respectively for the raw image file size
for the first three.

Test: manual
Bug: 111302946
Change-Id: Ia8fb5696151acad59bb144ea93f2c2ddac962bbd
2018-10-29 13:37:10 -07:00
Mark Salyzyn
c3d408e3c1 build_image tune right-size image
To more quickly settle on a no free space result, recognize that
there is roughly 1/58.5 overhead managing the free space pool as
determined experimentally.  This algorithm carries with it some
risk of being sensitive to any alterations in ext4.

This also addresses the issue of a much larger apparent free space
available calculated on the sample pass when deploying ext4 dedupe,
and resolves the poorer estimation that happens without accounting
for the overhead.

The alternative of adding a second pass works to a similar, and
likely more comforting algorithm and result.  But doing so adds
a minute to the typical incremental build time, or +50% to the
time it takes to determine ext4 right sized images.

Test: manual
Bug: 111302946
Change-Id: Id5ae6645b9aa2d036e6fefe2fb17672e8f8ef6f0
2018-10-29 13:37:04 -07:00
Anton Hansson
89fe71180b Merge "Add a mainline arm64 device" 2018-10-29 19:03:41 +00:00
Anton Hansson
123f8e73ed Add vendor restrict to mainline targets
Prevents additions of non-aosp artifacts to these products.

Bug: 80410283
Test: lunch mainline_arm64; m nothing
Test: lunch mainline_system_arm64; m nothing
Change-Id: Id12a77bd7f4cc9917aecbca76e8a2842d7c77613
2018-10-29 16:25:49 +00:00
Anton Hansson
284b81c1ae Add a mainline arm64 device
The existing 'generic' targets are emulator devices, and targeting
it has the side-effect of including emulator-specific artifacts.

Create a mainline device instead, which is really generic. This also
allows specifying tweaks unsuitable for the emulator.

Include BoardConfigGsiCommon for now, even though it includes a few
things we likely won't want in the future.

Bug: 80410283
Bug: 111538404
Test: lunch mainline_arm64; m
Change-Id: I15e7cb41fcbd3c0592a091ba7a8575ddf10bda71
2018-10-29 16:15:04 +00:00
Treehugger Robot
df45f28a82 Merge "Really mark sdk_addon as PHONY to fix m snod" 2018-10-27 00:28:23 +00:00
Yifan Hong
779dc30020 Merge changes from topic "dpgroup"
* changes:
  Add dynamic partition info to target files
  BOARD_{GROUP}_SIZE must not be empty.
  Add _a and _b to group names to super.
2018-10-26 18:23:53 +00:00
Dan Willemsen
ea898f1c53 Really mark sdk_addon as PHONY to fix m snod
When we're using a dont_bother_goals (including snod), we don't load
build/make/core/tasks/*.mk, so sdk_addon isn't defined as a PHONY rule.

This causes a build break because build/make/core/Makefile uses
dist-for-goals with sdk_addon. dist-for-goals requires its arguments to
be PHONY targets.

Test: m snod
Change-Id: I69937fe9360be57766d17ed4c28b5d2a1e348308
2018-10-26 11:17:25 -07:00
Paul Hobbs
27ec8b868e Merge "Revert "Revert^2: Change command-line flags of hiddenapi"" 2018-10-26 17:22:35 +00:00
Luman Qu
d8c427fbba Revert "Revert^2: Change command-line flags of hiddenapi"
This reverts commit e9540386e2.

Reason for revert: Caught by droidcop https://b.corp.google.com/issues/118481351

Change-Id: I66e45ef5a84d63682e5c33c9aa1e708160606611
2018-10-26 16:53:09 +00:00
Treehugger Robot
43930504e1 Merge "Mark BUILD_NUMBER as obsolete, more vars as readonly" 2018-10-26 15:42:36 +00:00
David Brazdil
6d0b0f6103 Merge "Revert^2: Change command-line flags of hiddenapi" 2018-10-26 09:19:23 +00:00
Colin Cross
e0006c91ca Merge "Pass privileged dexpreopt module configuration to Soong" 2018-10-26 00:00:12 +00:00
Yifan Hong
07d3d79e6e Add dynamic partition info to target files
Test: builds and manual inspect
Bug: 117182932
Change-Id: Ic84860d976d18ef7aa8d050fd1118d1e75a0456d
2018-10-25 16:23:42 -07:00
Yifan Hong
a50b5e34a4 BOARD_{GROUP}_SIZE must not be empty.
Otherwise, OTA generation will fail.
Test: OTA generation
Test: do not define it, build error

Change-Id: I63da80bd35d1982db0510428560727c759212771
2018-10-25 16:23:42 -07:00
Yifan Hong
16a07e1806 Add _a and _b to group names to super.
For A/B devices, append slot suffix to dynamic
partition group names. This ensures that during an
update, partitions from target slot doesn't take
more than half of the super partition.

Test: flash and lpdump
Bug: 111610495
Change-Id: I07a082b67d0f9faada150c7ac3527eb737a81655
2018-10-25 16:23:42 -07:00
Treehugger Robot
fea4263ec1 Merge "releasetools: Fix the error message." 2018-10-25 22:35:37 +00:00
Tao Bao
f3fc62c8a0 releasetools: Fix the error message.
This avoids showing a message as below:

raise BuildImageError("Failed to get tune2fs usage:\n{}".format(output))
UnboundLocalError: local variable 'output' referenced before assignment

The `output` won't be set under such a case, which unfortunately hides
the real cause. This CL addresses the issue by letting it throw
implicitly, which contains the command and outputs.

Test: `m dist`
Test: `python -m unittest test_build_image`
Change-Id: Icf015bab8869d150516246ed73552f6502127cdf
2018-10-25 12:29:44 -07:00
David Brazdil
e9540386e2 Revert^2: Change command-line flags of hiddenapi
Change command-line flags of hiddenapi

The `hiddenapi` build tool now reads a dex file from one filename
and writes a modified dex file to another. Change the build rule
to reflect it.

This reverts commit 5381eb684e.

Test: m
Change-Id: I8d37935c5ea381348278ec668d474f6ffb8e8411
2018-10-25 17:25:23 +00:00
Treehugger Robot
1bc81731c8 Merge "build_image: improvements to right size for dynamic partitions" 2018-10-25 15:25:49 +00:00
Treehugger Robot
73b998ad2e Merge "Revert "ATest: GTest auto gen config support run-test-as."" 2018-10-25 08:53:24 +00:00
Bill Yang
a8fcf6c893 Revert "ATest: GTest auto gen config support run-test-as."
This reverts commit fab820c5f1.

Reason for revert: sed -i requires an argument on mac, break build on mac.

Change-Id: I0b72e28968072afbbb862bd827ee59764dd8a0be
2018-10-25 07:11:26 +00:00
Bill Yi
470dacd220 Merge pie-platform-release to aosp-master - DO NOT MERGE
Change-Id: I37435c7f41f3c040f4aaecfb7efc259b353a6388
2018-10-24 14:54:21 -07:00
Treehugger Robot
79c6c5b8be Merge "Allow target to specify LOCAL_PREFER_INTEGRITY" 2018-10-24 21:46:23 +00:00
Victor Hsieh
e53dd7b20b Allow target to specify LOCAL_PREFER_INTEGRITY
The option will:
 - Produce APK with uncompressed dex
 - Declare to prefer integrity in manifest

Test: unzip -vl, dex compression looks correct with the option provided
      or not.
Test: similarly, aapt dump correct attribute
Bug: None

Change-Id: I16b9a37255150c2ad84af84087dfabb536a3b07a
2018-10-24 11:40:07 -07:00
Colin Cross
aefeae367e Merge "Reduce the propagation of LOCAL_DEX_PREOPT := nostripping" 2018-10-24 16:47:12 +00:00
Treehugger Robot
2fc5669b62 Merge "ATest: GTest auto gen config support run-test-as." 2018-10-24 13:53:50 +00:00
David Brazdil
70b26e5c98 Merge "Revert "Change command-line flags of hiddenapi"" 2018-10-24 12:38:57 +00:00
David Brazdil
5381eb684e Revert "Change command-line flags of hiddenapi"
This reverts commit 6b00bfd2fb.

Reason for revert: marlin running out of space

Exempted-From-Owner-Approval: Needed to unblock build, no
  buildcops on call in EMEA today.
Change-Id: I9e86545c8ada2523bc21116c6e958f69fb94dc26
2018-10-24 12:37:41 +00:00
David Brazdil
7658657ee4 Merge "Change command-line flags of hiddenapi" 2018-10-24 09:19:00 +00:00
Colin Cross
3aa2f8f69b Pass privileged dexpreopt module configuration to Soong
Pass DONT_UNCOMPRESS_PRIV_APPS_DEXS, PRODUCT_LOADED_BY_PRIVILEGED_MODULES,
DEX_PREOPT_DEFAULT, and WITH_DEXPREOPT to Soong so it can determine when
to store uncompressed dex files in APKs and when to strip them.

Also convert DEXPREOPT.$(TARGET_PRODUCT).$(LOCAL_MODULE).CONFIG=disabled
to a DEXPREOPT_DISABLED_MODULES list, and export it to Soong.

Test: m checkbuild
Change-Id: I8652032d4b9f5d2e119e73acdf91b8148563eedd
2018-10-23 22:33:07 -07:00
Colin Cross
fc782ad949 Reduce the propagation of LOCAL_DEX_PREOPT := nostripping
nostripping causes confusing double negatives, allow nostripping
in LOCAL_DEX_PREOPT and DEFAULT_DEX_PREOPT, but convert to
LOCAL_STRIP_DEX outside of dex_preopt_odex_install.mk.

Test: m checkbuild
Change-Id: I996e9258ce20c394900d9fe937d638bc2ab8589d
2018-10-23 22:33:07 -07:00
Treehugger Robot
f26f313d17 Merge "libvixl-arm and -arm64 are no longer VNDK libraries." 2018-10-24 01:41:57 +00:00
Julien Desprez
b8ca22ca30 Merge "Remove host proto lib from cts jars" 2018-10-23 23:05:09 +00:00
Mark Salyzyn
780f595fac build_image: improvements to right size for dynamic partitions
If partition_reserved_size is 0 or undefined, and
use_dynamic_partition_size is true, we should approach no space
and no free inodes automatically.

Estimate the space and number of inodes required, then do a first
pass build to see how much space actually used, and use those values
to refine the estimate.

Depends on tune2fs to report the characteristics of the filesystem,
so only support for ext filesystems.  In the future if there has to
be a more generic ability, either a tool per a filesystem has to be
found, or we will need root capabilities to mount the filesystem to
acquire the characteristics live from the host system.

Test: manual + python -m unittest test_build_image
Bug: 111302946
Change-Id: I933a388be43516b6de7b5007b296765bd5556fde
2018-10-23 15:14:00 -07:00
Tao Bao
7ceac730cf Merge "releasetools: Include descriptor of vbmeta_system.img in vbmeta.img." 2018-10-23 20:03:47 +00:00
Julien Desprez
e483544cf5 Remove host proto lib from cts jars
The lib is already statically included in tradefed.jar.

Test: run cts-unit-tests
Bug: 118297021
Change-Id: I073b68216870b1e744ee35db82a7e03f11db6782
2018-10-23 09:38:09 -07:00
Tianjie Xu
1e0742e2c3 Merge "Install the ota keys under recovery as a zipfile" 2018-10-23 02:40:55 +00:00
Treehugger Robot
b64a0f3b03 Merge "Only set set_hashtree_disabled_flag in top-level vbmeta.img." 2018-10-22 22:44:54 +00:00
Tao Bao
7106420add releasetools: Include descriptor of vbmeta_system.img in vbmeta.img.
Chained vbmeta images should be included into the top-level vbmeta.img
as chained partitions. It's done in the path of `m`, but not `m dist`.

Bug: 118115607
Test: `m dist` a target that uses chained vbmeta.
    a) Check that the `vbmeta.img` in target-img.zip contains the image
       descriptor of `vbmeta_system.img` (`avbtool info_image`).
    b) `avbtool verify_image --image vbmeta.img \
            --expected_chain_partition \
                vbmeta_system:1:/path/to/vbmeta_system/pubkey`
    c) `avbtool verify_image --image vbmeta_system.img`
Change-Id: I064e583b247c44b9b2f19355838550bb5dbb8f26
2018-10-22 15:27:54 -07:00
Dan Willemsen
6dbb33dfad Mark BUILD_NUMBER as obsolete, more vars as readonly
In preparation for moving BUILD_NUMBER and FILE_NAME_TAG handling to
soong_ui, mark BUILD_NUMBER as obsolete and FILE_NAME_TAG as readonly.

Later changes will remove BUILD_NUMBER entirely, and move FILE_NAME_TAG
handling into the kati packaging step via dist-for-goals renames.

Test: build_test on master
Change-Id: I0e340d32fe0cb6296d3b5a7fec349b50632d044e
2018-10-22 15:20:26 -07:00
Tianjie Xu
ffbe6b9785 Install the ota keys under recovery as a zipfile
Recovery can now parse the pem encoded x509 keys from a zipfile. So
instead of dumping the keys into a text file with some intermediate format,
we can simply create a zipfile with the keys.

Bug: 116655889
Test: make bootimage and check the generated zipfile, run sign_target_files_apks
Change-Id: Ib76feecfb26d6be713a07644e80ec96133759004
2018-10-22 14:47:15 -07:00
Treehugger Robot
4c95ffa8ef Merge "Remove USE_CLANG_LLD[=false]" 2018-10-22 20:26:27 +00:00
Tao Bao
240dcb8c45 Only set set_hashtree_disabled_flag in top-level vbmeta.img.
It's sufficient to have the flag in top-level vbmeta.img, and libavb
expects no disable flag in chained vbmeta.img.

Bug: 118115607
Test: `m vbmetaimage` with an -eng target using vbmeta_system.img. Check
      the flag in the generated image
      (`avbtool info_image --image vbmeta_system.img`).
Change-Id: Ia429dd8a5cf0243e16ae305da210d3ea27c68abf
2018-10-22 13:06:49 -07:00
Colin Cross
1e0b2296ad Merge "Always define sdk_addon as phony" 2018-10-22 18:22:48 +00:00
Colin Cross
d0dbeb3758 Always define sdk_addon as phony
dist-write-files will add dist targets to the sdk_addon target,
but the sdk_addon target is not declared as phony when building
with mm (ONE_SHOT_MAKEFILE set), causing:
build/make/core/main.mk:1495: error: writing to readonly directory: "sdk_addon"

Always define sdk_addon as phony even if ONE_SHOT_MAKEFILE is set.

Bug: 118144231
Test: mm
Change-Id: I99d14a98a7597ebd694c765f94b0d6f4486860dc
2018-10-22 10:42:44 -07:00
Treehugger Robot
5546a543ce Merge "Put maxTargetSdk=P APIs onto light greylist." 2018-10-22 15:56:49 +00:00