Commit graph

1992 commits

Author SHA1 Message Date
Tao Bao
08f9da1898 Merge "Don't generate hashtree when signing bundled APEXes." 2019-09-23 14:22:59 +00:00
Tao Bao
448004af9d Don't generate hashtree when signing bundled APEXes.
Bug: 139957269
Test: Sign a target_files.zip. Extract a re-signed APEX and check the
      hashtree size (being zero).
Test: Use sign_apex to sign an APEX file. Check the hashtree size (not
      being zero).
Test: python -m unittest test_apex_utils
Test: python -m unittest test_sign_apex
Change-Id: I927b7681d66920d7732b700ec3a8f7a65b4cb351
2019-09-19 14:41:34 -07:00
Treehugger Robot
c4f6f36dc4 Merge "Creates img.zip from target files zip, not dir." 2019-09-19 01:03:33 +00:00
Daniel Norman
74eb74b146 Creates img.zip from target files zip, not dir.
Directory support for img_from_target_files was dropped in aosp/1095755.

Test: Create & boot a merged build.
Change-Id: Ib976ea60a24c2f172c597130d6bcce25199a93b3
2019-09-18 14:03:34 -07:00
Tao Bao
3e759462d9 releasetools: Remove more images out of secondary payload.
This CL additionally removes boot/dtbo/modem/vbmeta_* images out of
secondary payload. We essentially only keep system_other.img and
bootloader images there.

For Pixel devices, this additionally saves ~80MiB (mostly because of
the removal of boot and radio images).

Bug: 140771390
Test: ota_from_target_files \
          --include_secondary --skip_postinstall \
          -i input-target_files-1.zip \
          input-target_files-2.zip \
          output-ota.zip
Test: python -m unittest test_ota_from_target_files
Change-Id: If47b27c52b3547a4cc86223a988c53960bc6af40
2019-09-17 22:56:19 -07:00
Tao Bao
d5b6094ed3 Merge "releasetools: Sanity check the build fingerprint." 2019-09-17 18:30:45 +00:00
Tao Bao
c99819311c Also install verity_key to ramdisk for non-system-as-root target.
The commit in d14b895665
(https://android-review.googlesource.com/c/platform/build/+/728287)
changed partition layout, to always build the root dir into system.img,
even for devices not using system-as-root (i.e. the ones with separate
boot ramdisk).

With the new layout, there will be two root dirs for non-system-as-root
targets during the boot. If such a device uses Verified Boot 1.0,
/verity_key needs to be available in both roots, to establish the chain
of trust.
 - bootloader uses the baked-in key to verify boot.img; it then loads
   the ramdisk from the verified boot.img
 - First stage init uses /verity_key (in ramdisk) to verify and mount
   system.img at /system, then chroot's to it
 - Second stage init uses /verity_key (in system.img) to verify and
   mount other partitions

This CL adds rules to additionally install verity_key into ramdisk for
such targets.

Bug: 139770257
Test: Set up a target to use non-system-as-root
      (BOARD_BUILD_SYSTEM_ROOT_IMAGE != true). `m dist`.
Test: Check that both ROOT/verity_key and BOOT/RAMDISK/verity_key exist
      in the built target_files.zip.
Test: Run validate_target_files to validate the above target_files.zip.
      $ validate_target_files \
          --verity_key_mincrypt /path/to/verity_key \
          target_files.zip
Test: Run sign_target_files_apks to sign the above target. Re-run
      validate_target_files on the signed target_files.zip.
Test: python -m unittest test_validate_target_files
Change-Id: Ibe7e771c8c376429add85851ac86055564765d3c
2019-09-17 08:53:16 -07:00
Tao Bao
c4011cd719 releasetools: Sanity check the build fingerprint.
Bug: 140298338
Test: python -m unittest test_ota_from_target_files
Change-Id: I4add5f82d5b684c441b0484732da6991f8e7301a
2019-09-17 08:43:54 -07:00
Tianjie Xu
1c8080098a Remove vendor & product in secondary payload
We used to carry the full OTA for vendor and product partitions when
generating the secondary payload. Since they are not actually used, we
can remove them to reduce the size of factory OTA packages. For example,
the size for a sargo package reduces from 454M to 283M after the change.

Bug: 140771390
Test: generate and apply a second payload on Pixel
Change-Id: I091dc07db8c345b182d9f5dd8d48f5c209245641
2019-09-11 15:23:41 -07:00
Yifan Hong
708cbdbd41 Merge "releasetools: add dep to checkvintf"
am: fd4a3de77d

Change-Id: Ie5ffcd9b96de64129555980d77d851f7672f2083
2019-09-04 17:53:30 -07:00
Yifan Hong
09298552e2 releasetools: add dep to checkvintf
'required' only works on python_binary_host.
Test: modify checkvintf source code, build ota_from_target_files

Change-Id: Ieac68b39b1ccabae679a1679f184cb3aeaaeefb6
2019-09-04 12:37:46 -07:00
Tao Bao
bce0264348 Merge "releasetools: img_from_target_files uses zip2zip."
am: f833a75283

Change-Id: I1731934e5585185167db9638c7b6652871fbf913
2019-08-30 12:29:42 -07:00
Tao Bao
f833a75283 Merge "releasetools: img_from_target_files uses zip2zip." 2019-08-30 19:13:59 +00:00
Tianjie Xu
8b106fd2bc Merge "Add a new flag "--disable_fec_computation" for OTA generation"
am: 9c9993c706

Change-Id: Ic077215542e20c2bc9fbaaaa2fa602f15d3f4a87
2019-08-29 13:27:20 -07:00
Tianjie Xu
9c9993c706 Merge "Add a new flag "--disable_fec_computation" for OTA generation" 2019-08-29 20:05:41 +00:00
Tianjie Xu
1b07983c6d Add a new flag "--disable_fec_computation" for OTA generation
We have added the flag to disable on device fec computation in
brillo_update_payload. Exposed it in ota_from_target_files as well.

Bug: 139723500
Test: generate an incremental package with the new flag
Change-Id: I683e3ec7791963a3c7e141ecb4e5a3fb53fe2108
2019-08-28 21:38:05 -07:00
Yifan Hong
ee276acce8 Merge changes from topic "vintf_ota_delete"
am: bcba1cfdaf

Change-Id: If11df7c409b9d1087defc119bf3a132f5b7016c5
2019-08-28 17:21:36 -07:00
Yifan Hong
a592de9c61 Check VINTF compatibility at OTA generation time.
am: 9276cf0226

Change-Id: Iee0efa785ccbf2aaadb1be7b3dc106f805242adb
2019-08-28 17:21:27 -07:00
Yifan Hong
96961bf94f releasetools: add tests for check_target_files_vintf
am: ccb86fe7d4

Change-Id: I3ecddd78ccce7f97bd02abea133ad84187b312ac
2019-08-28 17:21:07 -07:00
Yifan Hong
d14a6110c5 Add a script to check VINTF compat of target files package.
am: e3ba82cff2

Change-Id: Iccf4b7afa93749f441f48a3270f3d4beac8a0f06
2019-08-28 17:20:52 -07:00
Tao Bao
57f8ed6b5e releasetools: img_from_target_files uses zip2zip.
Test: Build the following targets and check the built image archive.
      bonito-userdebug (contains flashable images);
      cf_x86_phone-userdebug (contains super.img);
      aosp_arm64-eng (contains VerifiedBootParams.textproto)
Test: m -j otatools-package; Check that zip2zip is included; Use
      bin/img_from_target_files to generate image archive.
Change-Id: I9f28765bd702849f0df309faebd6aa3072920536
2019-08-28 16:36:25 -07:00
Yifan Hong
bcba1cfdaf Merge changes from topic "vintf_ota_delete"
* changes:
  merge_target_files: Checks VINTF metadata of resulting target files
  Check VINTF compatibility at OTA generation time.
  releasetools: add tests for check_target_files_vintf
  Add a script to check VINTF compat of target files package.
2019-08-28 23:19:21 +00:00
Yifan Hong
9276cf0226 Check VINTF compatibility at OTA generation time.
Instead of relying on META/{system,vendor}_{manifest,matrix}.xml
and blindly copy compatibility.zip from target files package to
OTA package, do a static check on the input target files package
before generating the OTA package.

META/{system,vendor}_{manifest,matrix} does not contain ODM manifest
fragments, which is incorrect.

Also, the on-device check of compatibility.zip uses a old libvintf
binary on the device, which may not understand the incoming VINTF
metadata. This change removes the on-device check. Hence, it removes
the requirement of forwards compatibility of libvintf.

This behavior can be skipped with --skip-compatibility-check.

Test: build OTA package
Bug: 139300422
Bug: 131425279
Change-Id: I7fb93be9eb73f578fc05a182c6c9d1f073db2800
2019-08-28 11:02:26 -07:00
Yifan Hong
ccb86fe7d4 releasetools: add tests for check_target_files_vintf
Test: run it
Bug: 139300422
Change-Id: I265d1e6313177dc661bb5cdf1415b11684dec0a0
2019-08-28 11:02:26 -07:00
Yifan Hong
e3ba82cff2 Add a script to check VINTF compat of target files package.
Instead of checking META/{system,vendor}_{manifest,matrix}.xml (which
is error-prone because ODM SKU-specific manifests are not checked),
this script read the target files package, remaps its directory
structure so that checkvintf understands it, and check VINTF
compatibility.

Also, put it in otatools.zip.

Test: run it on an extracted target files package
Bug: 131425279
Change-Id: I06036f9a8d7242d4bc11524028be40e780c508e8
2019-08-28 11:02:26 -07:00
Yifan Hong
ade0d3f4a2 merge_target_files: Checks VINTF metadata of resulting target files
Bug: 139300422
Bug: 131425279
Test: test_merge_target_files
Change-Id: I972db2fc40555ab0fc38316f0a6e6f5e8f163371
2019-08-28 11:02:26 -07:00
Tao Bao
3a85b1a466 Merge "releasetools: Clean up img_from_target_files.py."
am: 1f8e8738c1

Change-Id: Ic5718fd194395f86f99da41791ed9c909f7638cd
2019-08-27 16:45:09 -07:00
Tao Bao
ac63a9d728 releasetools: Clean up img_from_target_files.py.
Bug: N/A
Test: Run img_from_target_files to generate an image archive.
Change-Id: I802640b88fe0c7a6d1aab63411b8194d61bceaba
2019-08-26 20:33:11 -07:00
Daniel Mentz
1e8961033e Merge "Consider search_path to locate avb_*_key_path files"
am: 11dc247ba9

Change-Id: Ie0479fe88e37b330bc80fb6cb68d906546c70b0a
2019-08-26 12:21:24 -07:00
Daniel Mentz
25478181b9 Consider search_path to locate avb_*_key_path files
avb_*_key_path are usually relative file paths. If these files cannot be
located from the current directory, try prepending OPTIONS.search_path.

Bug: 139836256
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Change-Id: I020b257f458a1f5762691d0499feaf4027e37126
2019-08-26 18:17:14 +00:00
Baligh Uddin
6fac8bab91 Merge "Add a guard against deleting AVB_PUBKey."
am: 667c1aa83d

Change-Id: I255c746b3f6c8b98787f157800d143523c82161f
2019-08-25 22:16:58 -07:00
Baligh Uddin
15881282d7 Add a guard against deleting AVB_PUBKey.
The avb_pubkey may not be present, if the apex was initially
unsigned or generated from a bundle.  In this case, running
sign_apex to generate a signed apex binary would result in an error.
This fix checks for presense of avbpubkey before attempting the
deletion

BUG: 139994107
Change-Id: I3cb2e88a11ad8797e38ba5fb98c96a4ec4135fc8
2019-08-25 22:01:45 -07:00
Tianjie Xu
d4400ca15f Merge "Generate block based OTA for product partitions"
am: 2211352eed

Change-Id: Ib36f9dde2982d77fdd2a8a74d2e80d7a8b21326e
2019-08-20 13:39:52 -07:00
Tianjie Xu
f67dd8059d Generate block based OTA for product partitions
Currently only the system and vendor partitions are updated in the
generic block based OTA generation script. Since the product partition
is quite large and consist most of APK files, it would be beneficial to
update it similar to system. Handle the odm and system_ext partitions in
the same way as well.

Bug: 132683080
Test: Run unit tests, generate full and incremental OTA with product partitions.
Change-Id: I13478cf9bd32137c6729b8c9cb102080147093f2
2019-08-20 00:18:00 -07:00
Paul Trautrim
6c3aa71d1b Merge "Use --add_missing in merge_target_files.py"
am: cc875c1482

Change-Id: I0abe618a8e1900c57b67060361b46807531cddd7
2019-08-19 04:05:40 -07:00
Paul Trautrim
cc875c1482 Merge "Use --add_missing in merge_target_files.py" 2019-08-19 10:50:35 +00:00
Tao Bao
f4157c5859 Merge "releasetools: Add more module building rules."
am: 39369d08fb

Change-Id: Ic422023eceae40922a6251f32982cc1141f86aac
2019-08-14 09:08:50 -07:00
Tao Bao
39369d08fb Merge "releasetools: Add more module building rules." 2019-08-14 15:33:33 +00:00
Paul Trautrim
bc3600c030 Use --add_missing in merge_target_files.py
This is needed when including prebuilt images that do not need to be
regenerated. Otherwise the script will exit upon detecting an existing
IMAGES/ directory.

Test: test_merge_target_files
Bug: 139111475
Change-Id: I55d82bd7dc4fa5f27bc01e481362e79fecc200ff
2019-08-13 19:24:12 +09:00
Tao Bao
cd79edb331 Merge "releasetools: Fix the target files diffing."
am: 6a52b34473

Change-Id: I852a255f7ad99718a96bae0df92ce54ceb543c0a
2019-08-08 16:35:04 -07:00
Tao Bao
3c7b2a658c releasetools: Add more module building rules.
This CL adds the building rules for the following tools:
- add_img_to_target_files
- check_ota_package_signature
- check_target_files_signatures
- img_from_target_files
- merge_target_files
- ota_package_parser
- sign_apex
- sign_target_files_apks
- validate_target_files

Bug: 63866463
Test: mmma -j build/make/tools/releasetools
Test: Run each tool above to ensure there's no missing dependency (in
      the exercised code path).
Change-Id: Ide5b9775939ed02c822109cbcf4224000d5472df
2019-08-07 22:35:42 -07:00
Tao Bao
ced94b337c Merge "releasetools: ota_from_target_files uses target_files_diff."
am: 69340f6885

Change-Id: I822a3bfa774c79d2c4789a84fa084361753cb24b
2019-08-07 21:41:21 -07:00
Tao Bao
f0c4aa204c releasetools: Fix the target files diffing.
This CL refactors the non-A/B OTA generation code into a function,
so it allows adding common post-generation code for both flows.

It moves the code for target files diffing to common path, and fixes the
diffing functionality (broken due to the use of unzip pattern).

Test: Run ota_from_target_files with `--log_diff` flag to generate
      non-A/B and A/B OTAs respectively.
Change-Id: I92e5549a58344fc56a151a709c06534913d265d5
2019-08-07 14:53:08 -07:00
Tao Bao
f882684d85 Merge "Move make_recovery_path and sparse_img to modules."
am: 4c9527b314

Change-Id: I13a67cfe58450696c37a0e7e4ecc3261d8bd7d5a
2019-08-07 13:15:15 -07:00
Tao Bao
496748378d releasetools: ota_from_target_files uses target_files_diff.
The dependency was overlooked as the module was conditionally imported.

Bug: 63866463
Test: `m -j ota_from_target_files`; Run ota_from_target_files with
      `--log_diff` flag (on a non-A/B target).
Change-Id: I71eff2e55cc28fd649f0ca1b38da53907dac6597
2019-08-07 13:01:27 -07:00
Tao Bao
4c9527b314 Merge "Move make_recovery_path and sparse_img to modules." 2019-08-07 18:49:20 +00:00
Tao Bao
e11a46000c Move make_recovery_path and sparse_img to modules.
These two tools are only used within build system, so not adding them
to otatools.zip.

Bug: 63866463
Test: TreeHugger
Test: Build a non-A/B target that calls make_recovery_patch.
Change-Id: Iaeddc98a2c4a27c6b49d91047500029e460db9e3
2019-08-06 23:27:48 -07:00
Tao Bao
7552bbdfb6 Merge "releasetools: merge_target_files sets verbose in main."
am: 3738f37e22

Change-Id: I0e193d444af6e8a05289e22bbc02fc084972750a
2019-08-06 19:47:51 -07:00
Tao Bao
abb806b3f0 releasetools: merge_target_files sets verbose in main.
The flag would otherwise be turned on unconditionally even when
importing merge_target_files as a module. As a result, `atest
releasetools_test` was broken / flaky prior to this change, as
PythonUnitTestResultParser doesn't like unexpected outputs in stderr.

Test: `atest releasetools_test releasetools_py3_test`
Change-Id: Ifd614c797d2cee3b33e4c2a7ad8e598850e9d60c
2019-08-06 14:13:19 -07:00
Tao Bao
e5f0839c4b Merge "releasetools: Build build_super_image as a lib."
am: 7edc2b1994

Change-Id: Ib9fb0144b79d183aacc08ed0f661a0c7356b5372
2019-08-06 12:18:31 -07:00