Commit graph

5025 commits

Author SHA1 Message Date
Tao Bao
bcd1d161e8 Add compatibility metadata for Treble-enabled non-A/B OTA packages.
Commit 21803d3574 only added compatibility
metadata for A/B OTA packages, because by that time we didn't use
'ro.treble.enabled' property to determine if a target was Treble-enabled.

This CL uses 'ro.treble.enabled' to guard the packing of compatibility
metadata for both of A/B and non-A/B OTA packages. It also switches to
checking the system/vendor fingerprints to determine if there's an
update to the partition (previously it was computing the SHA-1 of the
images, which may have unintentionally changed due to issues that give
non-repetitive builds).

Bug: 64339310
Test: Generate OTA packages (full and incremental) on Treble-enabled
      targets (sailfish, as well as a non-A/B angler target with
      Treble-enabled locally); check that the compatibility.zip entry
      exists.
Test: Generate OTA packages on Treble-unenabled non-A/B target (angler);
      check that the compatibility.zip entry doesn't exist.
Test: Generate OTA packages on Treble-unenabled A/B target; check that
      the compatibility.zip entry doesn't exist.

Change-Id: I2a1fcf612439d849ba8ccea217a0faf5d5ba8e14
2017-08-28 15:19:44 -07:00
Tao Bao
ac7b7db12c Remove file_contexts.bin from non-A/B OTA package.
This entry only exists in non-A/B full OTA path. However, this file has
never been loaded by recovery - recovery was only reading the one from
the recovery image (which also has been removed by a recent change in
[1]).

[1]: commit e35926e1aff2e6b9b54656bd59c8178e295a1b7e in
platform/bootable/recovery.

Bug: 63538434
Test: `m dist` on angler. file_contexts.bin is gone from full OTA
      package. Sideloading the generated full OTA on angler.
Change-Id: Iec2e2d36599fa1ed499516a3e74f06cd36ade494
2017-08-25 14:14:53 -07:00
Jeff Gaston
03e7e81837 Merge "Remove the need to find Android.mk files via directory symlinks"
am: 6ab53d7211

Change-Id: I0495d45001821d095531d5be7c049fc2836888f7
2017-08-08 00:14:51 +00:00
Jeff Gaston
5c1f3fd068 Remove the need to find Android.mk files via directory symlinks
Bug: 64397960
Test: m -j
Change-Id: I5b77dd654dfced63900913c447b5448b632158d7
2017-08-07 14:21:20 -07:00
Tao Bao
d91be38c5e Merge "Look for non-existent files listed in avb_vbmeta_args."
am: 5180c3b97b

Change-Id: I6a035f46f09f70f11c271693158392f98927aa0f
2017-07-21 20:52:24 +00:00
Tao Bao
9a5f419ecf Look for non-existent files listed in avb_vbmeta_args.
In BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS, if we have defined
"--include_descriptors_from_image" with an image file whose path points
to source tree, add_img_to_target_files.py or sign_target_files_apks.py
may fail to find the file. Because these scripts may run without a
source tree, by taking target_files.zip as the only input.

This CL scans additional locations in the input target_files.zip to find
those missing files in avb_vbmeta_args. As long as the files are included
in the target_files.zip, they get a second chance to be found.

Bug: 63910867
Test: As follows:
 1. Setup BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS with a local file path;
 2. Remove the local file;
 3. sign_target_files_apks.py fails without this CL;
 4. sign_target_files_apks.py works.
Change-Id: I3c58f80a5535db02b74cfe40d0c0beff72587cf8
(cherry picked from commit 1dc5d47653)
2017-07-21 11:57:00 -07:00
Tianjie Xu
a07a04c737 Merge "Generate brotli compressed {}.new.dat for full OTA"
am: 1b28eca1a1

Change-Id: I3474e8efb634513fbc4876e5c272cf56b3443e12
2017-07-14 23:27:21 +00:00
Tianjie Xu
1b28eca1a1 Merge "Generate brotli compressed {}.new.dat for full OTA" 2017-07-14 23:21:54 +00:00
Tianjie Xu
b0a29ad8c2 Generate brotli compressed {}.new.dat for full OTA
Brotli has a better compression ratio than deflation. So for non-AB
full OTA, we can compress the new.dat with brotli first and store it
in the zip package.
This increase the OTA generation time by ~5 minutes for a full OTA
(measured based on 2.0G system.new.dat)

Bug: 34220646
Test: Generate a full OTA for bullhead
Change-Id: I9c0550af8eafcfa68711f74290c8e2d05a96648f
2017-07-14 10:53:21 -07:00
Tao Bao
fcc2cf6c1e Merge "Track the AVB property name change."
am: c45161536f

Change-Id: If20f1f0880680b8da185d846a9e0f17139389b50
2017-07-07 23:33:41 +00:00
Tao Bao
3f72176da0 Track the AVB property name change.
system_avb_hashtree_enable has been renamed to
avb_system_hashtree_enable in commit
3e599ead66. 'care_map.txt' is missing due
to the change.

Bug: 63142730
Test: `m dist` and check IMAGES/care_map.txt exists in target_files zip.
Change-Id: I60c269b41df844b50353d357bf67c20f15548642
2017-06-29 15:19:09 -07:00
Tao Bao
ce10b6e2f3 Merge "Allow packing custom radio images into target_files zip."
am: 9ea6a8b817

Change-Id: Ifdcdfa2214e4fa75bb0c7a5df1cb1796814fa7bc
2017-06-29 03:16:17 +00:00
Tao Bao
9ea6a8b817 Merge "Allow packing custom radio images into target_files zip." 2017-06-28 22:03:19 +00:00
Tao Bao
b5363b2dbc Merge "Support replacing AVB signing keys."
am: 682849976d

Change-Id: Iedbd4ce27ffdfeb3d88406f75636221739427add
2017-06-28 19:41:25 +00:00
Tao Bao
95a95c3a5e Allow packing custom radio images into target_files zip.
Radio images are added to INSTALLED_RADIOIMAGE_TARGET, which by default
will be packed under RADIO/ in a target_files zip. This CL introduces
BOARD_PACK_RADIOIMAGES that allows additionally copying them into
IMAGES/, which will then be included into <product>-img.zip.

Bug: 62195105
Test: Define BOARD_PACK_RADIOIMAGES and `m dist`. Check the generated
      target_files zip and <product>-img.zip.
Change-Id: I3deafd2dfecd1d4dbfdfc2d002fc40ef22fb60ea
2017-06-27 13:50:12 -07:00
Tao Bao
639118ff4d Support replacing AVB signing keys.
sign_target_files_apks.py now supports signing targets using verified
boot 2.0 (aka AVB). It allows replacing AVB signing keys for both of
chained and non-chained partitions.

An example command line looks as follows.
 $ ./build/tools/releasetools/sign_target_files_apks.py \
     --avb_vbmeta_key external/avb/test/data/testkey_rsa2048.pem \
     --avb_vbmeta_algorithm SHA256_RSA2048 \
     --avb_vbmeta_extra_args \
         "--signing_helper_with_files ./signing-helper.sh" \
     --avb_system_key external/avb/test/data/testkey_rsa4096.pem \
     --avb_system_algorithm SHA256_RSA4096 \
     --avb_system_extra_args \
         "--signing_helper_with_files ./signing-helper.sh" \
     product-target_files.eng.zip signed-product-target_files.zip

To verify the signed images:
 $ unzip signed-product-target_files.zip IMAGES/\*
 $ avbtool verify_image --image IMAGES/vbmeta.img \
     --expected_chain_partition system:1:testkey_rsa4096_pub.pem \
     --key external/avb/test/data/testkey_rsa2048.pem

Bug: 38315721
Test: sign_target_files_apks.py on AVB-enabled target w/ and w/o chained
      partitions respectively. Check the signing command lines; validate
      the signed images with 'avbtool verify_image'.
Change-Id: Ia009555b16ddb6d8ba6a0858d5ca7d983bbab887
2017-06-27 10:50:46 -07:00
Tianjie Xu
35f6f99056 Merge "Remove the obsolete location check for install-recovery.sh" am: 8fb65a5d63
am: ea5dad9010

Change-Id: I26e314431a87732a486566d2702106d28d323d92
2017-06-22 18:27:49 +00:00
Tianjie Xu
e0bda97ea1 Merge "Verify the contents in install-recovery.sh" am: 924c1c0598
am: 01c463e406

Change-Id: Ie8177614c855e3f554184d570b2f98b29aefc8c6
2017-06-22 18:27:39 +00:00
Tianjie Xu
8fb65a5d63 Merge "Remove the obsolete location check for install-recovery.sh" 2017-06-22 18:12:44 +00:00
Tianjie Xu
924c1c0598 Merge "Verify the contents in install-recovery.sh" 2017-06-22 18:12:44 +00:00
Chih-hung Hsieh
877a7a43d2 Merge "Add OWNERS in build/make" am: d79ce64555
am: 9af11fce7a

Change-Id: I44e3689a9280de474055ec4f109278ef2ff7d8cf
2017-06-22 04:34:38 +00:00
Chih-hung Hsieh
d79ce64555 Merge "Add OWNERS in build/make" 2017-06-22 04:27:13 +00:00
Chih-Hung Hsieh
044fc0c4a4 Add OWNERS in build/make
* Owners are selected from top CL approvals or owners.
  They will be suggested to review/approve future CLs.
* OWNERS files are recognized by the new find-owners plugin, see .md files in
  https://gerrit.googlesource.com/plugins/find-owners/+/master/src/main/resources/Documentation/

Test: build/make/tools/checkowners.py -c -v OWNERS
Change-Id: Ibe6765e5448c471ca413413e3d052507ccee4ea6
2017-06-21 16:09:56 -07:00
Tianjie Xu
9c384d2ca5 Verify the contents in install-recovery.sh
Check the SHA1 of recovery.img and boot.img embedded in the
install-recovery.sh.

Bug: 35411009
Test: validation script detects mismatch for both full recovery and
recovery-from-boot.
Change-Id: I5f07a869d9fa17fad26a22ef9ca3ecb06b1b28e3
2017-06-20 17:28:35 -07:00
Tianjie Xu
78de9f173d Remove the obsolete location check for install-recovery.sh
The script has been moved to /system/bin since L.

Bug: 35411009
Test: Generate a exact same OTA for bullhead.
Change-Id: Iaecb86e3dabc5b659605837bb06f16d20fa80334
2017-06-20 16:52:54 -07:00
Bowgo Tsai
abbba2c832 Merge "AVB: support chain partition signing" am: 0effed4b94
am: dcad0945ab

Change-Id: I5bc9f2dcc94b2db846d9956c0def6b3c9910b2c5
2017-06-20 07:07:33 +00:00
Treehugger Robot
0effed4b94 Merge "AVB: support chain partition signing" 2017-06-20 06:57:34 +00:00
Tao Bao
5edb6ee6fd Merge "Allow sign_target_files_apks.py to create zip64 signed TF.zip." am: 9baf675285
am: 7d5f8ec704

Change-Id: I23d08c4a7b3311afafa857e4efb5a982f8cf0f25
2017-06-14 05:49:58 +00:00
Treehugger Robot
9baf675285 Merge "Allow sign_target_files_apks.py to create zip64 signed TF.zip." 2017-06-14 05:36:51 +00:00
Tianjie Xu
f5fd565275 Merge "Fix duplicate 'META/care_map.txt' when calling add_img_to_target_files" am: 463c25f890
am: fa0542dced

Change-Id: Ia1d5b0b478f700b4df8ebbf9db10d4035d394792
2017-06-14 00:34:06 +00:00
Tao Bao
2b8f489e30 Allow sign_target_files_apks.py to create zip64 signed TF.zip.
We should only disallow zip64 for the image and OTA zips (because we
don't have zip64 support in libziparchive yet). But target_files zips
are fine to use zip64 with host tools (and we already do that in
add_img_to_target_files.py).

This CL also sets the default compression method to DEFLATED when
creating the signed TF.zip.

Test: sign_target_files.apks.py signing a large TF.zip passes.
Change-Id: I8043739860604134fa1166e920c95c28797bbcc1
2017-06-13 13:01:23 -07:00
Tianjie Xu
9ac4cb0984 Fix duplicate 'META/care_map.txt' when calling add_img_to_target_files
In case the system/vendor image is updated for an A/B target file, we'll
delay the write of META/care_map.txt. Specifically, we'll run "zip -d"
to remove the old entry and then zip in the new one from tmp dir.

Bug: 62345693
Test: Run add_img_to_target_files.py on sailfish target files, and
care_map.txt is updated.

Change-Id: Id2f9997aac50c0740e6944d0d0d43d98f5fc6c29
2017-06-13 11:20:29 -07:00
Bowgo Tsai
3e599ead66 AVB: support chain partition signing
Current build system will include AVB metadata from each partition and
store them into /vbmeta partiton when BOARD_AVB_ENABLE is set, which makes
each partition tightly-coupled.

Add the support for 'chain partition':
  - The vbmeta of each partition is stored on the same partition itself.
  - The public key used to verify each partition is stored in /vbmeta.

For example, the following build variables are required to enable chain
partition for system partition:
  - BOARD_AVB_SYSTEM_KEY_PATH := path/to/system_private_key
  - BOARD_AVB_SYSTEM_ALGORITHM := SHA512_RSA8192
  - BOARD_AVB_SYSTEM_ROLLBACK_INDEX := 1
  - BOARD_AVB_SYSTEM_ROLLBACK_INDEX_LOCATION := 2

The corresponding settings will be added into META/misc_info.txt for
build_image.py and/or add_img_to_target_files.py:
  - avb_system_key_path=path/to/system_private_key
  - avb_system_algorithm=SHA512_RSA8192
  - avb_system_add_hashtree_footer_args=--rollback_index 1
  - avb_system_rollback_index_location=2

To enable chain partition for other partitions, just replace SYSTEM with
BOOT, VENDOR and/or DTBO in the build variables.

Also switch from  `avbtool make_vbmeta_image --setup_rootfs_from_kernel system.img ...`
to `avbtool add_hashtree_footer --image system.img --setup_as_rootfs_from_kernel...`
when BOARD_BUILD_SYSTEM_ROOT_IMAGE is true. This works for both chained
and no-chained:
  - chained: `avbtool add_hashtree_footer --setup_as_rootfs_from_kernel` will
    add dm-verity kernel cmdline descriptor to system.img
  - no-chained: `avbtool make_vbmeta_image --include_descriptors_from_image
    system.img` will include the kernel cmdline descriptor from system.img into
    vbmeta.img

Bug: 38399657
Test: `make` pass, flash images from $OUT and boot device without chain partitions
Test: `make` pass, flash images from $OUT and boot device with chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device without chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device with chain partitions
Test: follow the same steps in
      https://android-review.googlesource.com/#/c/407572/

Change-Id: I344f79290743d7d47b5e7441b3a21df812a69099
2017-06-13 17:45:10 +08:00
Tao Bao
ddb9391d6b Merge "Delay the write-back of misc_info in sign_target_files_apks.py." am: d6e8002734
am: 6da0f9c8fd

Change-Id: Ifbaea16ab19a30c74132319b199cc2f41c3a3bff
2017-06-08 22:34:22 +00:00
Treehugger Robot
d6e8002734 Merge "Delay the write-back of misc_info in sign_target_files_apks.py." 2017-06-08 22:24:53 +00:00
Chih-Hung Hsieh
30e550cfb7 Merge "Find target keywords in the first 100 lines." am: 61ddffc6af
am: 4bf15225d4

Change-Id: I8375de1b5359e4bcbb028f0e158bc20545fbf8fe
2017-06-08 01:50:50 +00:00
Chih-Hung Hsieh
655c542dc8 Find target keywords in the first 100 lines.
Bug: 62416199
Test: warn.py build.log
Change-Id: I8c5ef310758fdd44519aeb1efe2f0bd92fe1ac87
2017-06-07 23:17:13 +00:00
Tao Bao
46a5999a02 Delay the write-back of misc_info in sign_target_files_apks.py.
Currently we're writing META/misc_info.txt to the new TF.zip during
ReplaceVerityPrivateKey(). We should delay that until we have replaced
everything in need. Otherwise we won't be able to replace/overwrite
that zip entry (unless `zip -d` first).

This CL also cleans up the return value of ReplaceVerityPublicKey() and
ReplaceVerityKeyId(), since the caller no longer needs the values.

Test: sign_target_files_apks.py and check the generated signed TF.zip.
Change-Id: I9fbd7182247728281519e5e3971557f6b018ad65
2017-06-07 15:13:50 -07:00
Tao Bao
1629a7e663 Merge "Fix the broken script in AssertFingerprintOrThumbprint." am: 9718140570 am: d791faf34e
am: ee8a023c4e

Change-Id: I66067ffc253be23c11ff30a2805677f23105b40e
2017-06-05 22:52:15 +00:00
Treehugger Robot
9718140570 Merge "Fix the broken script in AssertFingerprintOrThumbprint." 2017-06-05 22:33:05 +00:00
Tao Bao
f6737d7506 Merge "Remove three board_avb_* args from META/misc_info.txt." am: 401b1d2509 am: 6466063839
am: 26912a3a57

Change-Id: Ief85e4efe05855b4486ed7e2f8bac22302ddd6a9
2017-06-05 20:24:06 +00:00
Tao Bao
d2d01e519e Fix the broken script in AssertFingerprintOrThumbprint.
Bug: 62252466
Test: Re-generate an incremental OTA that goes from the fingerprint to
      thumbprint, and check the updater-script.
Change-Id: I6e2cbf68cbd22fbcf0d200fc3fdc8a33da510a53
2017-06-05 12:27:52 -07:00
Tao Bao
3ebfddeabe Remove three board_avb_* args from META/misc_info.txt.
board_avb_algorithm and board_avb_key_path are overlapping with
avb_signing_args. In core/Makefile, only avb_signing_args (i.e.
INTERNAL_AVB_SIGNING_ARGS) will be used in the AVB-signing command. It
covers the contents in board_avb_{algorithm,key_path}. We should do the
same thing in tools/releasetools to avoid potential inconsistency.

This CL cleans up the logic in tools/releasetools, by always using
avb_signing_args. This also allows easier signing key replacement (so we
can replace the key/algorithm/signer in 'avb_signing_args').

board_avb_system_add_hashtree_footer_args is unused in releasetools
script, and the same information has been covered by
system_avb_add_hashtree_footer_args. This CL removes this arg as well.

Test: `m dist`. Then a) check the removed three args no longer exist in
      META/misc_info.txt; b) check that rebuilding images with
      add_img_to_target_files.py uses the same parameters.
Change-Id: I7db890b5c942de5b6868d8d1ebf937586d4729c0
2017-06-05 10:33:52 -07:00
Tao Bao
891c29766e Merge "Support re-generating DTBO image from add_img_to_target_files.py." am: 4536e45f66 am: b88e3f20a6
am: cdabed4ee8

Change-Id: Ic890baa277142baf355c1196afd7705c117761a3
2017-06-02 22:39:44 +00:00
Tao Bao
4536e45f66 Merge "Support re-generating DTBO image from add_img_to_target_files.py." 2017-06-02 22:13:36 +00:00
Tianjie Xu
9989175c0e Merge "Generate care_map.txt when AVB is enabled" am: 65b98cd3b1 am: 1e0b14eb33
am: 72274464c4

Change-Id: I7549c4f1b66ee1bf8dfaab26e90a2022f9f19dea
2017-06-02 18:24:35 +00:00
Tianjie Xu
65b98cd3b1 Merge "Generate care_map.txt when AVB is enabled" 2017-06-02 18:09:58 +00:00
Tianjie Xu
6b2e155ee6 Generate care_map.txt when AVB is enabled
When AVB is enabled, generate care_map.txt and add it to the target
files. Also copy it into the OTA package where it will later be used
by the update_verifier.

Bug: 62208947
Test: \
1. Run add_img_to_target_files on the TF of a new pixel device,
and care_map.txt generates successfully.

2. Make dist in oc-dr1-release and find care_map.txt in the OTA package.

3. update_verifier succeeds in reading all the blocks on the care_map,
and fails to read out-of-bound blocks.

Change-Id: I2881711e6f87789cb7de150dbeca18b756fed68a
2017-06-01 21:36:43 -07:00
Tao Bao
c633ed0230 Support re-generating DTBO image from add_img_to_target_files.py.
This is a step to enable signing a given target_files zip with release
keys.

When calling sign_target_files_apks.py, we will delete all the entries
under IMAGES/ in order to re-generate them (with the proper release
keys). In order to support that, we need to pack everything in need into
TF.zip.

Steps to test the CL.
a) Choose a target that has both AVB and DTBO enabled.
 $ m dist

b) Check IMAGES/dtbo.img and PREBUILT_IMAGES/dtbo.img both exist in the
   generated out/dist/TF.zip.

c) Remove the entries under IMAGES/ from the generated TF.zip.
 $ zip -d TF.zip IMAGES/\*

d) Re-generate the images with TF.zip.
 $ build/make/tools/releasetools/add_img_to_target_files.py TF.zip

e) Check that IMAGES/dtbo.img is re-generated, and it's identical to the
   image in b). Note that by default the re-generated image will carry a
   different footer, because of the random salt. This CL is verified by
   specifying the same salt.

Bug: 38315721
Test: see above.
Change-Id: I0bdc4e1cd4800962dc3902ca550dad6a8ca56c78
2017-06-01 20:07:33 -07:00
Tao Bao
ffa864c5ed Merge "Pack avbtool into otatools.zip." am: db5fa05429 am: 308b234593
am: 0c2bc7a7b1

Change-Id: I1f15f62536e3030bee7fec0fdacb1ef8ee57a69e
2017-06-01 07:30:30 +00:00