Commit graph

1674 commits

Author SHA1 Message Date
Treehugger Robot
d72873dc41 Merge "Reland "Only assert-max-image-size for static partitions."" 2019-04-10 20:09:00 +00:00
Treehugger Robot
e3ef041244 Merge "Stop using build/target -> build/make/target symlink" 2019-04-10 19:17:25 +00:00
Yifan Hong
3f8c09f8a5 Merge "Fix DataImage.ReadRangeSet()" 2019-04-10 17:20:06 +00:00
Dan Willemsen
0ab1be6fe2 Stop using build/target -> build/make/target symlink
Instead, fully specify build/make/target/... everywhere

Test: treehugger
Change-Id: Idf89b2e6a0b777adbfb6370ea34f35faee6d4965
2019-04-09 21:35:37 -07:00
xunchang
376cc7c452 Pass the correct signature size to payload generator
The signature size will be 512 bytes when signing the payload
with 4096 bits key. This cl determines the key size with
"openssl rsa -modulus"

The new key in testdata is generated by
"openssl genrsa -out testkey 4096"

Bug: 129163830
Test: generate and verify an OTA package
Change-Id: I6662b0a0c553dc0fd84711312a1256b887e332fd
2019-04-09 17:41:00 -07:00
Yifan Hong
6f3eaeb84d Fix DataImage.ReadRangeSet()
It returns a list of one generator object, not a list
of strings.

Test: test_blockimgdiff
Change-Id: I8962c539c2ce3fae90d428b38c4b0e52c5a2cdad
2019-04-09 17:01:53 -07:00
Yifan Hong
8c3dce0c56 Reland "Only assert-max-image-size for static partitions."
This reverts commit 6e099095d1.

Reason for revert: reland the CL

Bug: 122377935
Test: build blueline_mainline

Change-Id: Iee594b64e687decff186c0fa60f82b88608febe9
2019-04-09 10:56:34 -07:00
Florian Mayer
1d26ca95c3 Merge "Revert "Only assert-max-image-size for static partitions."" 2019-04-09 13:21:23 +00:00
Florian Mayer
6e099095d1 Revert "Only assert-max-image-size for static partitions."
This reverts commit 67e4336912.

Reason for revert: Broke master

Change-Id: I8a5054a42f145881437e2cb5b372f276bfdaa3d2
2019-04-09 09:54:59 +00:00
Treehugger Robot
3c4889fc79 Merge changes from topic "dap_size_check"
* changes:
  Only assert-max-image-size for static partitions.
  sparse_img.py --get_partition_size return size of partition
  Revert "Fix dynamic partition size check for devices with recovery"
2019-04-09 02:17:36 +00:00
Yifan Hong
67e4336912 Only assert-max-image-size for static partitions.
assert-max-image-size doesn't make sense for
dynamic partitions, as build_image.py always find the
right size for the output image. Hence:

- build_image.py no longer need to write generated_*_info.txt
  (which contains the size of the image).
- assert-max-image-size on the static BOARD_*IMAGE_PARTITION_SIZE. If
  a partition is dynamic, that variable isn't set, and
  assert-max-image-size becomes a no-op. If the partition is static,
  assert-max-image-size checks the static partition size as it used
  to be.
- Fix read-size-of-partitions to use the size of the partition by
  reading the image directly (instead of using generated_*_info.txt).

For devices without AVB, with DAP enabled, and does not have
RESERVED_SIZE for partitions, because of right sizing, the original
code always warns about approaching size limits. Since such checks
doesn't make sense for dynamic partitions, remove them.

Test: builds on device with dynamic partitions
Test: builds on cuttlefish with DAP enabled (without AVB), no
      more size limit warnings:
WARNING: out/target/product/vsoc_x86/vendor.img approaching size limit (X now; limit X)

Fixes: 122377935
Change-Id: I75e1b8322197cb18cf397d02aefd49d777bb6405
2019-04-04 15:09:51 -07:00
Yifan Hong
7ad83b634e sparse_img.py --get_partition_size return size of partition
Also, move code from build_super_image.py to sparse_img.py.

Test: sparse_img.py on sparse and non-sparse images
Bug: 122377935
Change-Id: Ie91fdfdbb54298ea27eb20d1b5363aeb1470356e
2019-04-04 12:59:00 -07:00
Yifan Hong
cb35bd9e51 Build non-sparse super image if necessary.
If TARGET_USERIMAGES_SPARSE_EXT_DISABLED is set, don't provide
--sparse to lpmake, so that a non-sparse super image is built.

Test: build with the flag set.
Fixes: 120041578
Change-Id: I5a26e4c793b0e2ddc89e9c38c8828ac21044e78a
2019-04-03 10:58:25 -07:00
Tao Bao
289fa3cea3 Merge "releasetools: Update the embedded AVB pubkey in an APEX." 2019-04-02 16:56:45 +00:00
Tao Bao
9c0f443264 releasetools: Update the embedded AVB pubkey in an APEX.
APEX keys are now embedded in the APEX [1].

[1] https://android-review.googlesource.com/c/platform/system/apex/+/936942

Fixes: 128297564
Test: Run sign_target_files_apks.py on a target_files.zip built with the
      CL above.
Change-Id: I219cc3e6ce6f652537fcc37322c8902f295c1447
2019-04-01 21:30:07 -07:00
Bill Peckham
364c1ccaa5 Add support merging system and other for the non-A/B configuration.
For non-A/B, you must supply the --system-item-list, --other-item-list, and
--system-misc-info-keys parameters approrpriate for merging two partial target
files. Additionally, you must supply the --rebuild-recovery option to correctly
generate the filesystem config and file contexts. With all of these parameters,
the script will generate a merged target files containing the correct recovery
files.

Also fix the --keep-tmp option to be consistent with the other options.

Bug: 122813742
Test: verify that merged target image boots and can perform OTA.
Change-Id: I5a942ac0cd9924fec419a686794a2340304594c8
2019-04-01 14:55:33 -07:00
Daniel Norman
6d82fa3961 Adds some tests for merge_target_files.
Test: python -m unittest test_merge_target_files
Bug: 129262038
Change-Id: Ibcb3b384b81ceaa1924a6cd402a8880bd3cfbba6
2019-03-28 14:43:30 -07:00
Tao Bao
c292edc22c Merge "releasetools: Remove the sanity check on APEX payload key names." 2019-03-27 15:44:29 +00:00
Tao Bao
5ed75d74e3 Merge "releasetools: Re-enable verifying AVB-signed images." 2019-03-27 03:20:13 +00:00
Tao Bao
6d9e3da5f9 releasetools: Remove the sanity check on APEX payload key names.
Unlike container keys/certs, we'll always carry full names for APEX
payload signing keys.

Test: Run sign_target_files_apks.py on a target_files.zip with
      mismatching payload signing keys (e.g. shim apexes).
Test: `python -m unittest test_sign_target_files_apks`
Change-Id: Ifa329478f43b4f5ae665821564dbc14af7841330
2019-03-26 13:05:46 -07:00
Tao Bao
a81d429928 releasetools: Re-enable verifying AVB-signed images.
This reverts commit 9788b4ed31. All the
blocking issues have been addressed.

Fixes: 120517892
Test: Run validate_target_files.py on crosshatch signed
      target_files.zip.
Change-Id: I95de241e159998e002dedddafea65953b1a1b263
2019-03-26 12:13:37 -07:00
Daniel Norman
c21a8448d1 Merge "Merges dynamic partition misc_info.txt keys from both system and other." 2019-03-25 23:15:09 +00:00
Tao Bao
ffc9a30946 releasetools: Fix alignment issue when signing APEXes.
Previously it was following a wrong order by doing `zipalign` after
calling SignApk, which effectively compromised the signature. This CL
corrects the logic, and follows the same flow as in build system:
 - Pack APEX file;
 - `zipalign -f 4096`;
 - Call SignApk to sign the container with `-a 4096` flag.

Bug: 129148142
Test: Run sign_target_files_apks.py on taimen target_files.zip. Boot the
      image after signing.
Change-Id: I91bd3dce4f45c1891c5e122212a699f4808618fa
(cherry picked from commit 0e06cb0a8b)
2019-03-23 10:27:43 -07:00
Daniel Norman
19b9fe9fdf Merges dynamic partition misc_info.txt keys from both system and other.
To build a complete list of the dynamic partitions and partitions
groups, we need to merge the contribution from the system and other
target files.

Bug: 127687287
Test: Running merge_target_files.py and observing partition lists are
merged as expected.
Change-Id: I5bb9bd0e3179d48c9bfacdb3aca8253158f61cf6
2019-03-22 14:46:01 -07:00
Tao Bao
1e4cbfaf5c releasetools: Fix an issue in handling PRESIGNED APEX keys.
For PRESIGNED APEXes, we should keep carrying the matching public keys
at /system/etc/security/apex.

Bug: 129148142
Test: Run sign_target_files_apks.py on a target_files.zip with presigned
      APEXes. Check the output zip.
Change-Id: I2e941fd9b10e99d2db9df1e5308cbbe8c760177b
(cherry picked from commit bf3fb024cd)
2019-03-22 14:11:24 -07:00
Tao Bao
c62b9a2347 Merge changes I662aab3d,I5e741c27
* changes:
  Revert^2 "releasetools: check_target_files_signatures.py checks APEXes."
  releasetools: common.UnzipTemp() filters out non-matching patterns.
2019-03-21 16:38:04 +00:00
Tao Bao
359862db12 Revert^2 "releasetools: check_target_files_signatures.py checks APEXes."
This reverts commit 5516d37f41.

The previous issue in unzipping non-matching files has been addressed
with commit a49054ca2f2959f50f3188914ec0faebc90ebcbe. This CL rolls
forward to allow dumping container certifcates for APEXes.

Bug: 128848294
Test: Run check_target_files_signatures.py on target_files.zips w/ and
      w/o APEX files.
Change-Id: I662aab3d96fc40ac8e5e206e32b73ac763220b70
2019-03-20 12:40:34 -07:00
Tao Bao
0ff15de32a releasetools: common.UnzipTemp() filters out non-matching patterns.
common.UnzipTemp() calls `unzip` to do the unzipping, which will
complain if there's non-existent names in the given list. Prior to this
CL, callers had to do the work to remove non-existent entries. This CL
filters out the given patterns in common.UnzipTemp()/common.UnzipToDir()
to make callers' works easier.

Bug: 128848294
Test: `m dist` with aosp_taimen-userdebug (which calls
      ota_from_target_files.py on a target_files.zip that doesn't
      contain RADIO/*).
Test: `python -m unittest test_common.CommonZipTest`
Change-Id: I5e741c27ea8d0b8126c398a7e1b56a8deb4a3d7f
2019-03-20 12:38:09 -07:00
Bowgo Tsai
e4544b1c02 Moving system_other key into product.img
Currently system_other AVB public key is placed in system.img.
However, this makes it's harder to have a *generic* system.img
across different product configs. Moving the key to /product
partition to allow more product-specific AVB keys.

Device board config can add /product/etc/fstab.postinstall,
to mount system_other with this key in /product. It can specify
different mount options, file systems, verity settings, etc., in
this product-specific fstab as well.

Bug: 123611926
Test: `make productimage` checks the following is generated.
      $OUT/product/etc/security/avb/system_other.avbpubkey
      Also checks it's included in $OUT/installed-files-product.{json, txt}

Test: run the following command and checks that
      PRODUCT/etc/security/avb/system_other.avbpubkey is updated:
      ./build/tools/releasetools/sign_target_files_apks \
        --avb_system_other_algorithm SHA256_RSA2048 \
        --avb_system_other_key external/avb/test/data/testkey_rsa2048.pem \
        out/dist/*-target_files-*.zip signed-target_files.zip

Change-Id: I6804f29941bec54375d80bd68a5aedb5c23b842e
2019-03-20 16:18:11 +08:00
Daniel Norman
a4ab1d8490 Merge "Validates the config lists used by the script." 2019-03-19 23:56:25 +00:00
Tao Bao
2812fd32ac Merge "releasetools: Allow skipping PRESIGNED APEXes." 2019-03-19 23:04:07 +00:00
Daniel Norman
fab3f29e5a Merge "Adds optional --rebuild_recovery flag to merge_target_files." 2019-03-19 21:32:28 +00:00
Tao Bao
e134399cab releasetools: Allow skipping PRESIGNED APEXes.
This CL adds support that allows treating an APEX as pre-signed. We can
skip signing an APEX with `-e <apex-name>=` and
`--extra_apex_payload_key <apex-name>=`. Note that the payload_key and
container_key must be in consistent state - either they're both
PRESIGNED or none of them is. CheckApkAndApexKeysAvailable() has been
updated to perform the sanity check.

Bug: 123716522
Test: Run sign_target_files_apks.py with the above flags.
Test: python -m unittest test_sign_target_files_apks
Change-Id: Id1e2f3f2facd4a97a385983cc9b78c028f7e7e73
2019-03-19 12:25:02 -07:00
Daniel Norman
e596452f03 Validates the config lists used by the script.
This validation is to help ensure that any usage of custom merge config
files does not accidentally exclude any item that has been added to the
default config lists.

Bug: 124197349
Test: Run merge_target_files with custom merge config files.
Change-Id: I34c51cb75212368146a2944d37621f311060d24d
2019-03-19 17:47:49 +00:00
Tao Bao
5516d37f41 Revert "releasetools: check_target_files_signatures.py checks APEXes."
This reverts commit d8469727bc. The script
is broken on target_files.zip that don't contain any APEX.

Bug: 128848294
Test: Run check_target_files_signatures.py on target_files.zip w/o APEX.
2019-03-19 09:30:08 -07:00
Tao Bao
e3f9dc6113 Merge "releasetools: Kill the use of os.getenv('AVBTOOL')." 2019-03-19 15:53:26 +00:00
Tao Bao
f88e0ce01d releasetools: Kill the use of os.getenv('AVBTOOL').
OTA tools should pick up the avbtool, as listed in dict['avb_avbtool'],
from the current PATH (plus bin/ under the dir specified via `--path`),
the same way as handling all other host tools.

Test: `m dist`
Change-Id: I3eb4d2c61979b03d9c23b2403d9a38cf052d87ea
2019-03-18 21:01:43 -07:00
Yifan Hong
691defc4bd Merge "Add supernod build target." 2019-03-18 23:46:05 +00:00
Daniel Norman
a4911da57d Adds optional --rebuild_recovery flag to merge_target_files.
Also makes AddSystem check that an output_zip exists before attempting
to add the recovery patch to the output zip.

Bug: 128838154
Test: Running merge_target_files with --rebuild_recovery and verifying
it passes --rebuild_recovery to add_img_to_target_files.
Change-Id: I19347b2c0dabf29b7196045b18551b5d0687df2c
2019-03-18 12:29:31 -07:00
Tao Bao
fa9de0aae8 releasetools: Fix an issue when using APEX container key mapping.
The keys_info in the touched code is a tuple, which is immutable.

Bug: 123716522
Test: Run sign_target_files_apks.py with '-e foo.apex=bar' that replaces
      the APEX container key.

Change-Id: I4e57e46c93a56b7f6646764d021ebb42c19bf7f5
2019-03-18 10:26:09 -07:00
Tao Bao
aa7e993a22 releasetools: Support signing APEXes.
Bug: 123716522
Test: Run sign_target_files_apks.py to sign a target_files with APEXes.
Test: Run check_target_files_signatures.py on signed artifact.
Test: python -m unittest test_sign_target_files_apks
Change-Id: I3fa13e3d9461cf5e0838e0572d436e218164fe41
2019-03-15 22:15:51 -07:00
Tao Bao
1cd59f2a26 releasetools: Add apex_utils.py.
The added `testdata/testkey_with_passwd.key` is generated with:

$ openssl pkcs8 -inform DER -in testdata/testkey_with_passwd.pk8 \
    -out testdata/testkey_with_passwd.key -passin pass:foo

Bug: 123716522
Test: python -m unittest test_apex_utils
Change-Id: I7b7f00178ecab35c3e41ecf4d10295fe9bf480ab
2019-03-15 22:08:42 -07:00
Tao Bao
d8469727bc releasetools: check_target_files_signatures.py checks APEXes.
Only the container certs will be checked and reported. For the payload
within an APEX, we can't easily extract the cert info.

It needs to go along a longer path, if ever needed, by:
 - extracting public keys from all the available certs;
 - using each of them to verify against an APEX payload to find a match
   (`avbtool verify_image --image payload --key public_key`).

Bug: 123716522
Test: Run check_target_files_signatures.py on target_files with APEXes.
Change-Id: I2ef318e05433d2d65ab84e2dff9e01fb6ee3373d
2019-03-15 20:56:13 -07:00
Tao Bao
2cc0ca1770 releasetools: Add common.ExtractAvbPublicKey().
Bug: 123716522
Test: python -m unittest test_common
Change-Id: I1f645008a14cc882ef280f169c36e4b14a53ef88
2019-03-15 10:47:46 -07:00
Tao Bao
badceb2d60 releasetools: Initalize logger when signing.
Other modules have switched to logging module. sign_target_files_apks.py
needs to init the logger to get the logs.

Test: Run `sign_target_files_apks.py -v`. Check outputs.
Test: Run `check_target_files_signatures.py -v`.
Change-Id: Ic68c019f6fb14840561885f1194ad6efdfdb7d82
2019-03-15 09:53:00 -07:00
Yifan Hong
69e0d61cbe Add supernod build target.
superimage-nodeps and supernod depends
on images from $(ANDROID_PRODUCT_OUT) (not from
target files package). It doesn't rebuild source
images if they are present.

A typical workflow is:

    m -j
    # change code in system
    m snod -j
    m supernod -j

Test: For non retrofit, run:
      `m snod -j; m supernod -j`
Fixes: 128321505
Change-Id: Ib8c011cadb9c0cd334234aef39f19be6a48fee62
2019-03-13 11:15:21 -07:00
Daniel Norman
2c99c5bbd2 Adds support for optional config files to configure the merge.
Bug: 124467065
Test: Running `python merge_target_files.py` using the three new flags
and observing that their contents are passed to the merge_target_files() function.
Change-Id: I4de46f041f5ae8bc8be2730313ce873a952bf78e
2019-03-12 16:51:18 +00:00
Treehugger Robot
b6f2f8b2d9 Merge "releasetools: Add compatibility for custom out directory." 2019-03-11 19:51:14 +00:00
Pavel Salomatov
3267655004 releasetools: Add compatibility for custom out directory.
sign_target_files_apks script looks for the signapk.jar inside the out dir.
If the our dir is set to a different directory via OUT_DIR_COMMON_BASE the script does not work properly.

From now script checks if the OUT_DIR_COMMON_BASE is set, then searches the jar in the proper path.
If OUT_DIR_COMMON_BASE is unset, searches in "out" like it did before.

Test: Build with OUT_DIR_COMMON_BASE set and unset and verify that sign_target_files_apks works in both cases

Change-Id: I9218b98ff79526184f8353705640193405afac9e
2019-03-11 19:22:29 +03:00
Tao Bao
28a11ef040 Merge "releasetools: Include retrofit info into package metadata." 2019-03-07 18:11:30 +00:00