Commit graph

2067 commits

Author SHA1 Message Date
Daniel Norman
d5fe862628 Uses a per-partition fingerprint for building images and avb_salt.
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.

Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
      `add_img_to_target_files -a out/dist/target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
      the partial target-files.zip IMAGES are identical to the merged
      target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
2020-03-02 19:05:20 +00:00
Tianjie Xu
f02ececc7f Preserve the monotonic property when subtracting 0 block
The monotonic property of the range is lost after the recent change to
subtract 0th block from the file range. As a result, we read the block
in the wrong order when pull files from the raw image. This yields a
incorrect hash during validation.

Bug: 150334561
Test: run validate_target_file. unittests
Change-Id: Id4fedaaf273c42c8d21a4549fda4c9a45e22066a
2020-02-26 21:50:47 -08:00
Bowgo Tsai
2fe786a528 Support removing AVB public keys
AVB public keys might be installed into the first-stage ramdisk.
This CL supports removing some AVB keys during the signing process,
which can be useful when a key needs to be revoked.

Also correcting the path of force_debuggable detection.

Bug: 150109393
Test: make dist
Test: ./build/tools/releasetools/sign_target_files_apks \
      --remove_avb_public_keys key1.avbpubkey,key2.avbpubkey \
      out/dist/*-target_files-*.zip signed-target_files.zip
Change-Id: I1af95adbbec655f40121f3392965dd1950aca4e5
2020-02-24 17:53:10 +08:00
David Anderson
ab9c7e6d04 Exclude holes from the block map.
e2fsdroid can export holes as a "0" block, in order to make sure logical
and physical block assignments are maintained properly. Exclude these
hole blocks from range sets in LoadFileBlockMap.

Bug: 139201772
Test: m dist on non-A/B
Change-Id: Ic04e869572229678fdeaaa6fbf6f464d6c672051
2020-02-21 17:04:00 +00:00
Tamas Petz
c0a8c63935 build_image: check value of "ext4_share_dup_blocks"
Regardless of the value of "ext4_share_dup_blocks" "-c" flag is passed
unconditionally. The correct behavior is to check if
"ext4_share_dup_blocks" is set to "true".

Test: 1. build bonito with "BOARD_EXT4_SHARE_DUP_BLOCKS := false"
      2. check for "shared_blocks" feature of system.img
         ./out/host/linux-x86/bin/tune2fs -l \
         ./out/target/product/bonito/system.img | \
         grep -o shared_blocks

Change-Id: I8c0e5606658dd0f47b71cfc6c453a77a57da4f42
2020-02-10 11:45:06 +01:00
Tianjie Xu
9bd832a012 Remove the key parameter when verifying avb images in validate_target_files
If a key is specified, the avbtool always use the input key to verify
all the chained images. And this will cause failures when the vbmeta &
system use different keys (e.g. RSA 4096 vs RSA2048). Because the public
key to vbmeta will always fail to verify the system image. Remove the
'--key' parameter in the verification command, so the avbtool will use
the embedded public in the image.

Test: validate target-file from sdk_gphone_x86_64
Bug: 148916990
Change-Id: I9d31be0f8c32af605af94fa73d07818f40f51ec4
2020-02-06 13:31:26 -08:00
Tianjie Xu
0e38d64e49 Add quote to the signing arguments when passing to apexer
We should add quote when passing the signing arguments to apexer. So
the final argument whould look like
apexer ... --signing_args "--signing_helper_with_files=%path"

And the argument parser in avbtool will eventually parse the
"--signing_helper_with_files" correctly.

Bug: 148627666
Test: unittest pass, run a smoke sign_target_files_apk with signing arguments
Change-Id: Ie203d26a508cb98ca94e6c364a2640b681ad79f2
2020-01-31 20:19:18 +00:00
Tianjie Xu
cea6ad1fb0 Pass signing_args to apexer to repack apex
In the build signing environment, we don't necessary have access to the
physical private key. And we work around it by passing an extra
signing_helper to the avbtool instead of a private key. Since we need
to perform signing to add the hashtree footer when rebuilding the apex
image. We should propogate the signing_helper to apexer and eventually
to avbtool.

Bug: 148627666
Test: unit tests pass. Smoke test by pass a public key to apexer and
hardcode the private key in signing helper script.
Change-Id: I56d57485493a5ba2cbe4c49e493ee9604f68088b
2020-01-31 01:37:36 +00:00
Tianjie Xu
ba5de57120 Merge "Assets should be optional in apex repacking" 2020-01-29 22:59:30 +00:00
Tianjie Xu
83bd55c1db Assets should be optional in apex repacking
We should only provide the --assets_dir option when the assets directory
is available in the original apex.

Bug: 148452817
Test: unit tests pass
Change-Id: I44308b80a43ff3f441223f0ecc248991d1d83f31
2020-01-29 11:40:42 -08:00
Tianjie Xu
8fff584906 Merge "Make the error more instructive on missing android_jar_path" 2020-01-29 03:19:23 +00:00
Daniel Rosenberg
dac25a96df Merge "Add build properties to enable casefold and prj_quota" 2020-01-28 23:53:25 +00:00
Tianjie Xu
61a792f323 Make the error more instructive on missing android_jar_path
So when $ANDROID_BUILD_TOP is missing, the error message print out the
path of the missing file; instead of a type error.

Bug: 148452817
Test: check result with missing ANDROID_BUILD_TOP
Change-Id: I5f841f6626eb3843364f20e78b278d3244505396
2020-01-28 10:58:14 -08:00
Daniel Rosenberg
6cc2c81f4d Add build properties to enable casefold and prj_quota
This adds the ability to format f2fs and ext4 in a way that supports
casefolding and project quotas. Project quotas require wider inodes, and
thus cannot be enable apart from at build time. Casefolding can be
configured with f2fs.fsck/tune2fs later.

Both features are enabled via inheriting emulated_storage.mk in the
device.mk

Bug: 138322712
Bug: 138321217
Test: Enable on device with related patches, view fs configuration on /data
Change-Id: Ib1ab0589006ff1b3fb7478b7e9b0a8aad71e4b09
2020-01-27 12:45:47 -08:00
Tianjie Xu
88a759d651 Resign apks contained in apex
Some apex payload images contain apk files. And these apks need to be
signed during the signing processed when sign_target_files_apks is
called. To support the signing, we can extract the payload and repack
the apex file with the (de)apexer tool. Add the signing support in the
apex_util.

Bug: 146508800
Test: unit tests pass, run sign_apex, sign_target_files_apks
Change-Id: If6d58975248709a144b07dbabf47c27916e5695e
2020-01-27 19:48:39 +00:00
Treehugger Robot
b4484dc33b Merge "Rebuild the vendor install-recovery if necessary." 2020-01-10 13:49:55 +00:00
Treehugger Robot
88ab531831 Merge "Go one directory up looking for releasetools path" 2020-01-10 12:31:32 +00:00
Robin Lee
da427de124 Rebuild the vendor install-recovery if necessary.
Test: Manual on a device with VENDOR/bin/install-recovery.sh
Bug: 146504238
Bug: 68319577
Change-Id: I33c253716bda5e7e655dac5c0636e9995e7b64c1
2020-01-09 00:29:38 +01:00
Treehugger Robot
620a2f2366 Merge "Add system_ext to VINTF search paths." 2020-01-07 23:04:54 +00:00
Robin Lee
34ea73940e Go one directory up looking for releasetools path
Change I366099c3dfd5fa4282745ef258a8cf35338e1e42 introduced a more
restricted search where we only look in out/host/x/bin instead of
out/host/x when looking for files.

Because signapk is in out/host/x/framework, and lib64 is in out/host/x,
we need to go one level up for this to work properly.

Fix: 147096688
Test: atest --host releasetools_test releasetools_py3_test
Change-Id: I597e1d95c09eb0e27b2faebb107218761901d9ac
2020-01-02 19:25:20 +00:00
P Adarsh Reddy
7e9b5c4204 Handle virtual A/B defines during dynamic_partitions_info merge.
While merging the Dynamic Partition info Dicts in the case of
split builds, pick the virtual-a/b flags like virtual_ab,
virtual_ab_retrofit from the vendor half.

Bug: 146811936
Test: Merged build contains the virtual a/b defines.
Change-Id: I0171b074094e47cfd12dc6af2565acdcd1b65163
2019-12-24 06:32:03 +00:00
Yifan Hong
dafae5228c Merge "Add /system/vendor/odm for VINTF search path" 2019-12-23 22:26:55 +00:00
Yifan Hong
9cbb624abd Add system_ext to VINTF search paths.
Bug: 140280874
Test: pass

Change-Id: Ifd5b784fc3c4553284f74a1f06cbac0a9bef4c75
2019-12-19 14:00:10 -08:00
Yifan Hong
2870d1e3c1 Add /system/vendor/odm for VINTF search path
Compared with board_config.mk and found this one is missing.

Also add docs to explain where are these values from.

Test: pass

Change-Id: Ic22d75a538657bd286edcae6d95f9ee462911975
2019-12-19 14:00:03 -08:00
David Anderson
117b52f74e releasetools: Pass --virtual-ab to lpmake.
Note that we are currently excluding this flag for retrofit devices,
since it necessitates support for a newer super.img format, which
retrofit devices may not fully support.

Bug: 134949511
Test: m -j, lpdump super_empty.img
Change-Id: Id69a8d755b9f4c4bb5fbd456a416ced1a5f1d135
2019-12-16 22:17:28 +00:00
Treehugger Robot
d6aa90ce9a Merge "sign_target_apks: Add networkstack to default key_map" 2019-12-13 22:22:08 +00:00
Dan Austin
5290364e9b Add AFTL relevant options to OPTIONS in common.py.
Adds AFTL relevant options in preparation for avbtool changes. The
options are:
aftl_server: A string representing the URL of the transparency log.
aftl_key_path: The path to the public key associated with the
transparency log (in PEM format).
aftl_manufacturer_key_path: The path to the key associated with the
manufacturer for AFTL message signing.
aftl_signing_helper: The signing helper for use with AFTL message
signing.

Change-Id: I73d466f24b171c16dbdaa40c589779bed52be551
Test: Ran add_img_to_target_files with all AFTL options.
Bug: 138779249
2019-12-12 15:48:52 -08:00
David Anderson
0ec64ac0aa releasetools: Generalize base_fs_file handling.
Share the code to handle base_fs files rather than duplicate it for
system/vendor. At the same time, add support for system_ext and product
partitions.

Bug: 139201772
Test: manual test
Change-Id: Ib1b7c018c828901e24eefc01e3faead03e77bc2d
2019-12-11 22:32:00 +00:00
Oleh Cherpak
982e608085 sign_target_apks: Add networkstack to default key_map
Test: run cts -m CtsSecurityTestCases -t \
	android.security.cts.PackageSignatureTest#testPackageSignatures
Bug: 145955635
Change-Id: I1a1498562e2b5983010cb98e3edcd03ceb2cce19
Signed-off-by: Oleh Cherpak <oleh.cherpak@globallogic.com>
2019-12-10 15:03:57 +00:00
David Anderson
c55b2f516c Merge "build_image: Do not use base_fs file for system_other." 2019-12-04 21:28:53 +00:00
David Anderson
3269f25476 build_image: Do not use base_fs file for system_other.
system_other is not OTA'd (outside of factory OTA), and its block map differs greatly from
system.img, so do not attempt to re-use it.

Bug: 139201772
Test: make with base_fs file for system
Change-Id: I1ed2091da381ebdb93522a081e5d2f7c18c9d069
2019-12-04 21:28:42 +00:00
Baligh Uddin
ac936fd573 Fix syntax error for SignApexFile
BUG: 145662097
TEST: Local smoke test which invokes sign_apex for local apex files.
2019-12-04 08:30:32 -08:00
Alistair Delva
1cfe41269f Merge "Allow f2fs sparse mode to be disabled" 2019-12-02 21:54:20 +00:00
Treehugger Robot
e7c1f6314c Merge "Removes custom prefix/suffix from MergeDynamicPartitionInfoDicts." 2019-11-27 20:30:34 +00:00
Daniel Norman
55417148f9 Removes custom prefix/suffix from MergeDynamicPartitionInfoDicts.
All callers of this function now always pass the same values, so this
change hardcodes those values within the function body.

Fix: 145008064
Test: python -m unittest test_common
Test: build & boot a merged target that uses DAP

Change-Id: I0051c5ba507983231825edfcaf349e574efa451a
2019-11-26 11:09:17 -08:00
Yifan Hong
874a5aa95e Merge "Move check partition size logs to a file" 2019-11-26 00:32:52 +00:00
Steven Laver
615385ee8b releasetools: merge_target_files.py should handle new contents of dynamic_partitions_info.txt
With a recent change (cb8e2755ff) to the
generation of dynamic_partitions_info.txt,
process_dynamic_partitions_info_txt() needs a corresponding update.

Change-Id: I30ec92176fd9cd737160b579214eb5b3d6a44c10
2019-11-22 09:34:27 -08:00
Donghoon Yu
92420db7dd releasetools: fix avbtool arguments.
The following error occurred when building dist.
Separate these arguments

error log:
avbtool: error: unrecognized arguments: --partition_name vendor_boot

Bug: 144820089

Change-Id: I3891ef3bd9a2f7d2cd1377228b9894b599dc1cc3
Signed-off-by: Donghoon Yu <hoony.yu@samsung.com>
2019-11-21 06:15:02 +00:00
Alistair Delva
91238cce14 Allow f2fs sparse mode to be disabled
Bug: 142424832
Bug: 142803849
Change-Id: Ic112128f15e3f29208332afceea4fbc3fca40b36
2019-11-15 19:05:41 +00:00
Treehugger Robot
e24ef86473 Merge "releasetools: build_super_image.py should unpack */build.prop" 2019-11-15 04:34:02 +00:00
Steven Laver
f43fa9dd99 releasetools: build_super_image.py should unpack */build.prop
build_super_image.py attempts to retrieve the "ro.product.device"
property, which requires parsing the */build.prop files from the target
files package. This fixes the following error:
"ExternalError: couldn't resolve ro.product.device"

Test: ./otatools/releasetools/build_super_image.py -p ./otatools
aosp_arm64-target_files.zip super.img

Change-Id: I592524a27b1a4e7544644b1a344071cff9718942
2019-11-14 23:47:31 +00:00
Yifan Hong
309109351e Move check partition size logs to a file
Don't contaminate the build log.
Test: m check-all-partition-sizes -j (verbose logs stored to file)
Test: m check-all-partition-sizes-nodeps -j (see verbose logs)
Test: manually run the script with[out] -v and with[out] --logfile

Change-Id: I345a340deab3e29bb9cb05d4970a55d8758607a7
2019-11-05 14:06:27 -08:00
Yifan Hong
96527f375f Fix Virtual A/B size checks
Test: build
Test: test_check_partition_sizes
Bug: 143111912
Change-Id: I4e056c25948e4169a0b5b098168141e27c31a0d4
2019-11-05 14:06:23 -08:00
Treehugger Robot
f5ad6e5443 Merge "Support Factory OTA for Virtual A/B devices" 2019-11-05 18:47:01 +00:00
Yifan Hong
fe07343fed Support Factory OTA for Virtual A/B devices
In Factory OTA, secondary payload should be generated without
virtual_ab flag set, so that update package generator generates
a regular A/B OTA (with snapshot_enabled set to false), and
OTA client writes system_b without snapshots.

Test: apply secondary OTA
Fixes: 138733621
Change-Id: I05d534a12d4a7891ce5597f3c29e55f5185e5a8d
2019-11-04 15:14:26 -08:00
Yifan Hong
3a7c2ef7cc Reland "Move partition size checks to python script"
This reverts commit 0141e45b96.

Reason for revert: Fixed in follow-up CL
Test: forrest
Bug: 143734706

Change-Id: I007acf228d4fb4d6a16ae9089e3f04cf33a567bb
2019-11-01 11:41:26 -07:00
Andrei-Valentin Onea
15e75e8807 Merge "Revert "Move partition size checks to python script"" 2019-11-01 15:36:10 +00:00
Raff Tsai
0141e45b96 Revert "Move partition size checks to python script"
This reverts commit a4906cd306.

Reason for revert: b/143734706

Change-Id: I905bd9c0bfcbd123c8f9e78986bfcb5600a9eca9
2019-11-01 05:36:20 +00:00
Yifan Hong
8244f27722 Merge "Move partition size checks to python script" 2019-11-01 01:52:11 +00:00
Yifan Hong
a4906cd306 Move partition size checks to python script
Easier to read and modify its logic.

Test: builds
Test: test_check_partition_sizes
Test: atest releasetools_test
Test: atest releasetools_py3_test
Change-Id: I73204664b64a0bcee7375fa6fca46758d5a2bb72
2019-10-31 11:22:20 -07:00