Commit graph

5739 commits

Author SHA1 Message Date
Neil Fuller
4f2051b1bb Merge "Make host targets use non-repackaged targets"
am: 09c81c303a

Change-Id: I5606474b66c13dba18987163e7ff0dc184f3f497
2018-08-05 08:12:39 -07:00
Treehugger Robot
09c81c303a Merge "Make host targets use non-repackaged targets" 2018-08-05 15:05:48 +00:00
Isaac Chen
13d3426f3c Merge "Work around e2fsck issue for building sparse GSIs"
am: a0d3277b8d

Change-Id: I53b45da39145cb10f37bd7bcede55f13cfed801b
2018-08-03 17:07:18 -07:00
Treehugger Robot
a0d3277b8d Merge "Work around e2fsck issue for building sparse GSIs" 2018-08-03 23:29:33 +00:00
Isaac Chen
ec7fa1cd09 Work around e2fsck issue for building sparse GSIs
When trying to support sparse format for GSIs, aosp/720949, a bug in
e2fsck was triggered, and hence failed the build for sparse GSI.

This workaround disables the detection of ASAN ODR so we can build
sparse GSIs successfully like we previously did for the vendors,
some of which can't handle unsparse images.

Bug: 112062612
Test: $ lunch aosp_x86_64-eng
      $ m SANITIZE_HOST=address ASAN_OPTIONS=detect_odr_violation=0 -j
      $ emulator

Change-Id: Ifafd27aa1e57d785b914f78a4d9bfef6a0c33fc9
2018-08-03 11:51:08 +08:00
Yifan Hong
635887d5ac Merge "target_files: copy to PRODUCT-SERVICES instead"
am: 567df6214f

Change-Id: I95bf48eba07daa2b0f6d5b84c45de9b22f7aac0b
2018-08-02 15:31:56 -07:00
Yifan Hong
567df6214f Merge "target_files: copy to PRODUCT-SERVICES instead" 2018-08-02 21:53:57 +00:00
Bryan Henry
a2bf1e5c5c Merge "Fix 2009-01-01 timestamps in releasetools to always be UTC"
am: e118d5bc4a

Change-Id: I12c8cc0664306ba116076b3f6d43e1c01fb9120f
2018-08-01 17:24:41 -07:00
Bryan Henry
e118d5bc4a Merge "Fix 2009-01-01 timestamps in releasetools to always be UTC" 2018-08-02 00:16:06 +00:00
Yifan Hong
4d6b38c47a target_files: copy to PRODUCT-SERVICES instead
The add_img_to_target_files.py script uses upper() to determine
the name of sub-directory under target_files_intermediates. The
name of the image is product-services.img but the sub-directory
name was PRODUCT_SERVICES. Change it to PRODUCT-SERVICES so that
'm dist' works.

Test: m dist
Change-Id: I8cad841c674b5d62c1d803d0383fc3aee14f272a
2018-08-01 16:46:16 -07:00
Yifan Hong
9a7d525849 Merge "Split PRODUCT_USE_LOGICAL_PARTITIONS into sub-flags."
am: d1c308bf40

Change-Id: I834e75dcfc04af0dd8655027bfc251c3d0162815
2018-08-01 11:39:59 -07:00
Bryan Henry
e6d547d53a Fix 2009-01-01 timestamps in releasetools to always be UTC
The usage of datetime.fromtimestamp previously resulted in the build or
signing machine's local timezone affecting the Unix timestamp ultimately
applied to images generated by add_img_to_target_files. The go/ab build
outputs would use 2009-01-01 00:00 UTC, for example, but local builds
and the release signed images (generated through go/ab-sign) would use
2009-01-01 00:00 PST. This change makes the timestamps always use UTC.

Bug: 80600931
Bug: 80093599
Test: 'm -j droid dist' and verified timestamps in resulting
target_files zip.

Change-Id: Ic2a19591519850c249f78254e1464aa6839bfc6c
2018-07-31 18:32:00 -07:00
Yifan Hong
2dae5721d4 Split PRODUCT_USE_LOGICAL_PARTITIONS into sub-flags.
Split the flag into PRODUCT_USE_DYNAMIC_PARTITION_SIZE
and PRODUCT_BUILD_SUPER_PARTITION. More sub-flags can be
added with the same way.

Also change some checks with USE_LOGICAL_PARTITIONS with
the appropriate sub-flags.

This allows easier device bring-up to fulfill the requirements
separately.

This also enables dynamic partition size without logical patition.
Originally, to enable BOARD_*_PARTITION_RESERVED_SIZE must also
enable PRODUCT_USE_LOGICAL_PARTITIONS. The patch fix the rule to
let dynamic partition size is able to be used independently.

Fixes: 111966003

Test: make superimage with PRODUCT_USE_LOGICAL_PARTITIONS
Change-Id: I45b29c87bf94356d8416bbd58a14c8982039c222
2018-08-01 00:39:13 +00:00
Hung-ying Tyan
3868a7fd9a Merge "Add more ro.vendor.build.*"
am: 2e0fb33af8

Change-Id: I9357d2e338198eb59ad7a9472bd82ca75177412a
2018-07-31 01:13:20 -07:00
Treehugger Robot
2e0fb33af8 Merge "Add more ro.vendor.build.*" 2018-07-31 08:07:57 +00:00
Yifan Hong
8883a739b3 Merge changes I1459d62f,I4c2b975e
am: bb937a6b69

Change-Id: Ifdedee1b0251bc9c2a8baf30d28b5d315b5c9e20
2018-07-30 11:35:33 -07:00
Yifan Hong
37c0c7c82d Build image for super partition
... for bootstrapping / initializing the device.
Image is built to $(PRODUCT_OUT)/super.img when running
`m dist`. For A/B devices, the image contains other
partitions in the _a slot.

Change-Id: I1459d62f02b95f142dfb3b7608f88ec6801dbf37
Fixes: 111758129
Test: m superimage -j
2018-07-27 13:45:43 -07:00
Yifan Hong
ebc041af56 productservices => product_services
Make names consistent.
Some things that does not change:
- productservicesimage target and intermediates

Test: build product services image
Bug: 111609632
Change-Id: I4c2b975e194577aad3d51b908b103880625deb5e
2018-07-26 17:19:02 -07:00
Tao Bao
0adc598d82 Merge "releasetools: Fix an issue in image size computation."
am: d578232c95

Change-Id: Idd139fe594e95cd92926da42beb670be239ee113
2018-07-26 12:53:51 -07:00
Tao Bao
4f35d47160 Merge "releasetools: Return the actual image size when building logical partitions."
am: f42fc1a7c3

Change-Id: I714fbd9458c2d6708ade24b36cbfd965a135b840
2018-07-26 10:09:53 -07:00
Tao Bao
c2606eb548 releasetools: Fix an issue in image size computation.
When building a system image with system_root_image enabled, the size
computation should include files under both of in_dir (i.e. /system
files) and root (pointed by 'root_dir'). Because files from both
locations will end up into the built image. The files under root are
usually only a few MiBs, but should be accounted for especially in the
context of logical partitions (where the partition size will be
allocated based on the actual need). Note that we will still need some
"reserved space" (defined via BOARD_*_PARTITION_RESERVED_SIZE) to cover
the cost for filesystem and/or verity metadata.

This CL moves the combination of the two dirs up, before parsing and
computing other properties. This doesn't affect anything for a
successful image building path. It may however increase the time to
error out in certain error path, since it copies the files earlier now.

Test: python -m unittest test_build_image
Test: `make dist`
Test: Setup a target with PRODUCT_USE_LOGICAL_PARTITIONS == true and
      BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE == 20MiB. Build system
      image successfully.
Test: Setup a target with PRODUCT_USE_LOGICAL_PARTITIONS == true and
      BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE == 20MiB. Write a large
      file to root dir (PRODUCT_OUT_ROOT). The image building fails, but
      reporting a size that accounts for both of /system and root.
Change-Id: Idfb26b8e259626ba57ec3bd4f85d357c30e56163
2018-07-26 16:59:57 +00:00
Tao Bao
f42fc1a7c3 Merge "releasetools: Return the actual image size when building logical partitions." 2018-07-26 16:59:36 +00:00
Hung-ying Tyan
c565f5c2ed Add more ro.vendor.build.*
to facilitate independent updates of vendor/odm images.

Bug: 78600951 (#56,#57)
Bug: 111491184
Test: build aosp_marlin-userdebug and check vendor/build.prop

Change-Id: I798c13822695259375fbc8d5c7fe08d43a540e46
2018-07-26 17:39:47 +08:00
Tao Bao
4251fe97aa releasetools: Return the actual image size when building logical partitions.
When verity is enabled, `partition_size` has the "adjusted" partition
size for holding the filesystem files (i.e. excluding verity hashtree,
and FEC metadata if applicable), whereas `original_partition_size` has
the one for the actual image returned by build_image.py.
`partition_size` must be smaller than `original_partition_size`. As a
result, the later assertion in Makefile (e.g. assert-max-image-size) may
fail to hold when using `partition_size`.

This CL addresses the issue by returning the value in
`original_partition_size` instead, if available. It also changes to
include both values when dumping the size info on image creation
failure.

Fixes: 79106666
Test: `m dist` with aosp_walleye-userdebug
Test: Set up walleye to use logical partitions, with a reserved size of
      20MiB. `m systemimage` no longer fails.
Test: Set up walleye that doesn't use verified boot. Use `m systemimage`
      to build an oversized image. Check the error message.
Change-Id: I41d77bffa85e8eadfbf4a2a7dcd2e798fadfdab7
2018-07-24 11:57:10 -07:00
Tao Bao
fdcb3b5829 Merge "releasetools: Support verifying AVB signed images with chained partitions."
am: 78078646e9

Change-Id: If7656208f92fde69dad358d298cd1c9eb1002ecd
2018-07-24 11:53:57 -07:00
Tao Bao
02a0859b3c releasetools: Support verifying AVB signed images with chained partitions.
For example, verify a target_files.zip that has system AVB-signed as a
chained partition.

  $ build/make/tools/releasetools/validate_target_files.py \
      signed-target_files-4904652.zip \
      --verity_key verifiedboot_pub.pem \
      --avb_system_key_path system_pub.pem

Note that verifiedboot_pub.pem should be the key (either public or
private) to verify vbmeta.img, and 'system_pub.pem' should be the key
(either public or private) for the chained partition of system.

testdata/testkey.key is the private key converted from
testdata/testkey.pk8 for testing purpose (`openssl pkcs8 -in
testdata/testkey.pk8 -inform DER -out testdata/testkey.key -nocrypt`).

Bug: 63706333
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Test: `m dist` on aosp_walleye-userdebug; Run validate_target_files.py
      on the generated target_files.zip.
Test: Set up walleye with chained system partition; `m dist`; Run
      validate_target_files.py on the generated target_files.zip.
Change-Id: I38517ab39baf8a5bc1a6062fab2fe229b68e897d
2018-07-23 15:20:14 -07:00
Tao Bao
f013dc3fed Merge "releasetools: Clean up AppendVBMetaArgsForPartition()."
am: 540a71d2b3

Change-Id: If13ee90c84f75c2452c1e1e8504928414a971ed4
2018-07-23 14:56:22 -07:00
Tao Bao
540a71d2b3 Merge "releasetools: Clean up AppendVBMetaArgsForPartition()." 2018-07-23 21:40:37 +00:00
Yifan Hong
fe421ed263 Merge changes from topic "board_partition_reserved_size"
am: 2d90c246a9

Change-Id: I460ea917b47a61acb0cbb0f17a867d15f61c7d6d
2018-07-23 11:58:52 -07:00
Yifan Hong
2d90c246a9 Merge changes from topic "board_partition_reserved_size"
* changes:
  build_image.py recognize BOARD_PRODUCT_SERVICESIMAGE_PARTITION_RESERVED_SIZE
  build_image.py recognize BOARD_PRODUCTIMAGE_PARTITION_RESERVED_SIZE
2018-07-23 18:38:24 +00:00
Tao Bao
f87ea90d76 Merge "releasetools: Rename the ramdisk_dir in prop dict to root_dir."
am: d094713952

Change-Id: I013c82846a143c5eac050325e66092afc66c958f
2018-07-23 09:41:25 -07:00
Tao Bao
d094713952 Merge "releasetools: Rename the ramdisk_dir in prop dict to root_dir." 2018-07-23 16:33:46 +00:00
Tao Bao
1a3d83d667 Merge "releasetools: Make scripts pylint clean."
am: 23a4070483

Change-Id: I4f4b35565f7d154dd47e4b5b8444e76c6c7227c6
2018-07-23 07:22:59 -07:00
Tao Bao
23a4070483 Merge "releasetools: Make scripts pylint clean." 2018-07-23 14:11:37 +00:00
Neil Fuller
35f744966b Make host targets use non-repackaged targets
A host build target was depending on a target that is intended for
on-device, repackaged (com.android.) use. This switches to using the
unbundled target instead.

Test: Build
Bug: 111055375
Bug: 111734251
Change-Id: Ie81dd7257a14756fc21fa6f956175e5bd2ff80c1
2018-07-23 13:00:32 +01:00
Tao Bao
3e53ef7b1c releasetools: Clean up AppendVBMetaArgsForPartition().
Drop the parameter of public_key_dir. It's not necessary to stage all
the extracted public keys into the same temporary dir.

Redirect the stderr output to STDOUT while calling avbtool, so that it
can dump the error message accordingly.

Also remove the check for having empty img_path arg. It doesn't look
like a clean logic to allow the function to be called with invalid /
empty arg. The only caller (i.e. AddVBMeta) already asserts img_path
being valid as well.

Test: python -m unittest test_add_img_to_target_files
Test: `m dist` with aosp_walleye-userdebug
Change-Id: Id58c5ae780ac8a22661ffea629144d4836839175
2018-07-22 22:36:27 -07:00
Chih-Hung Hsieh
d83446d789 Merge "Escape special HTML characters in warning messages."
am: a14c131154

Change-Id: Ia8a2afa3d765d7a07a5c7c732af9f66912c1483c
2018-07-20 17:24:14 -07:00
Treehugger Robot
a14c131154 Merge "Escape special HTML characters in warning messages." 2018-07-21 00:20:02 +00:00
Tao Bao
708ca5f60b Merge "releasetools: Track the change to applypatch arguments."
am: 80810b9d36

Change-Id: I9df643323e5624596b5981e9a4d48fdafb1db2f3
2018-07-20 17:12:39 -07:00
Yifan Hong
9c35a022e1 build_image.py recognize BOARD_PRODUCT_SERVICESIMAGE_PARTITION_RESERVED_SIZE
Test: builds
Bug: 79106666
Change-Id: I00c1a96c09cf7bd9274b65ac0afc47462686171d
2018-07-20 16:51:26 -07:00
Yifan Hong
56a6c3bb34 build_image.py recognize BOARD_PRODUCTIMAGE_PARTITION_RESERVED_SIZE
Test: builds
Bug: 79106666
Change-Id: I26aba694fd3d6a2feed7601524dde52569ba41e2
2018-07-20 16:51:26 -07:00
Tao Bao
8bfd3c7483 releasetools: Rename the ramdisk_dir in prop dict to root_dir.
This property indicates the path to the root directory (i.e.
TARGET_ROOT_OUT at build time, or ROOT/ in a target_files.zip).
build_image.py will only use this property when building a target using
system_root_image. Under such a configuration, files under this dir
never land into a ramdisk, but as part of system.img. This CL renames
the property name to avoid confusion.

Note that with this change releasetools keeps working with older
target_files zips that contain 'ramdisk_dir'. Because common.py doesn't
read the value of that property, but sets it to ROOT/ directly.

Test: `m dist` on aosp_marlin-userdebug.
Test: `m dist` on aosp_angler-userdebug.
Test: Run add_img_to_target_files.py with a new target_files.zip and an
      old target_files.zip respectively.
Change-Id: Ie9e19dd17d8c2abb4c4a97449e436774d5dd2a36
2018-07-20 16:05:36 -07:00
Tao Bao
80810b9d36 Merge "releasetools: Track the change to applypatch arguments." 2018-07-20 23:04:33 +00:00
Chih-Hung Hsieh
b2afb63ac9 Escape special HTML characters in warning messages.
Bug: 111699078
Test: run warn.py with a build.log file
Change-Id: I553dad57a6d81ea6cc89289741d9963f49cc3b47
2018-07-20 15:36:26 -07:00
Dario Freni
5f9ddc95a2 Merge "Add support for /product-services partition"
am: 3cf03b775a

Change-Id: I1b0b3628d1b96967e569093b93b12fbc5acd7ad5
2018-07-20 02:09:37 -07:00
Dario Freni
3cf03b775a Merge "Add support for /product-services partition" 2018-07-20 09:00:25 +00:00
Tao Bao
b1750e2825 Merge "releasetools: Remove the assertion of not allowing thumbprint in Treble world."
am: 805ab25e05

Change-Id: Iee41170414e8b17f05d2defa405b18834bc7e8f3
2018-07-19 15:05:39 -07:00
Tao Bao
76def243ec releasetools: Make scripts pylint clean.
Clean up the following scripts.
  blockimgdiff.py
  common.py
  edify_generator.py
  img_from_target_files.py
  ota_from_target_files.py

Seems we don't have a way to force pylint-ing the scripts using
per-directory pylintrc files (and we don't have pylint tool in AOSP
repo), per
https://android.googlesource.com/platform/tools/repohooks/#todo_limitations.

Test: `m dist`
Test: pylint --rcfile=pylintrc <script.py>
Change-Id: Ia6fd1ddc86f4d84c68e500f225d4a89d0fea8ec7
2018-07-19 10:39:05 -07:00
Tao Bao
4223012099 releasetools: Remove the assertion of not allowing thumbprint in Treble world.
The claim in the original CL doesn't seem to hold. We may still have
devices using both of thumbprint and Treble compatibility check.

Bug: 111640956
Test; python -m unittest test_ota_from_target_files
Test: Build an incremental package for a target using thumbprints.
Change-Id: Ia491d5279901c2385d1c30802abcb99d65d919cc
2018-07-19 10:16:45 -07:00
Dario Freni
5f681e1dd7 Add support for /product-services partition
This CL is largely an adaptation of Change-Id
I774e6a38003734421591e51bed103802ff84f432

It adds the following variables:

- BOARD_AVB_PRODUCT_SERVICES_KEY_PATH
- BOARD_AVB_PRODUCT_SERVICES_ALGORITHM
- BOARD_AVB_PRODUCT_SERVICES_ROLLBACK_INDEX_LOCATION
- BOARD_PRODUCT_SERVICESIMAGE_FILE_SYSTEM_TYPE
- BOARD_PRODUCT_SERVICESIMAGE_EXTFS_INODE_COUNT
- BOARD_PRODUCT_SERVICESIMAGE_EXTFS_RSV_PCT
- BOARD_PRODUCT_SERVICESIMAGE_PARTITION_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_JOURNAL_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_BLOCK_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_COMPRESSOR
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_COMPRESSOR_OPT
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_DISABLE_4K_ALIGN
- BOARD_PREBUILT_PRODUCT_SERVICESIMAGE
- BOARD_USES_PRODUCT_SERVICESIMAGE
- LOCAL_PRODUCT_SERVICES_MODULE
- PRODUCT_PRODUCT_SERVICES_BASE_FS_PATH
- PRODUCT_PRODUCT_SERVICES_VERITY_PARTITION
- PRODUCT_PRODUCT_SERVICES_PROPERTIES
- TARGET_COPY_OUT_PRODUCT_SERVICES
- TARGET_OUT_PRODUCT_SERVICES
- TARGET_OUT_PRODUCT_SERVICES_*

Bug: 80741439
Test: Successfully built product-services.img with one module in it, and flashed
on device. Also successfully built image with /system/product-services directory
and no /product-services partition.
Change-Id: I5d229f6ac729ea6df9ff1f14cee2e28972cd9b4d
2018-07-19 11:15:11 +01:00