Commit graph

5968 commits

Author SHA1 Message Date
Tao Bao
ffc9a30946 releasetools: Fix alignment issue when signing APEXes.
Previously it was following a wrong order by doing `zipalign` after
calling SignApk, which effectively compromised the signature. This CL
corrects the logic, and follows the same flow as in build system:
 - Pack APEX file;
 - `zipalign -f 4096`;
 - Call SignApk to sign the container with `-a 4096` flag.

Bug: 129148142
Test: Run sign_target_files_apks.py on taimen target_files.zip. Boot the
      image after signing.
Change-Id: I91bd3dce4f45c1891c5e122212a699f4808618fa
(cherry picked from commit 0e06cb0a8b)
2019-03-23 10:27:43 -07:00
Tao Bao
1e4cbfaf5c releasetools: Fix an issue in handling PRESIGNED APEX keys.
For PRESIGNED APEXes, we should keep carrying the matching public keys
at /system/etc/security/apex.

Bug: 129148142
Test: Run sign_target_files_apks.py on a target_files.zip with presigned
      APEXes. Check the output zip.
Change-Id: I2e941fd9b10e99d2db9df1e5308cbbe8c760177b
(cherry picked from commit bf3fb024cd)
2019-03-22 14:11:24 -07:00
Tao Bao
c62b9a2347 Merge changes I662aab3d,I5e741c27
* changes:
  Revert^2 "releasetools: check_target_files_signatures.py checks APEXes."
  releasetools: common.UnzipTemp() filters out non-matching patterns.
2019-03-21 16:38:04 +00:00
Logan Chien
81589605fd Merge "Update check_elf_file.py for clang-r353983" 2019-03-21 08:05:16 +00:00
Vic Yang
30dda1da5e Merge changes from topic "vndk-no-vendor-variant"
* changes:
  Add support for no-vendor-variant VNDK
  Add module-target-built-files function
2019-03-21 04:30:47 +00:00
Treehugger Robot
7e87c72fb0 Merge "Moving system_other key into product.img" 2019-03-21 04:26:21 +00:00
Tao Bao
359862db12 Revert^2 "releasetools: check_target_files_signatures.py checks APEXes."
This reverts commit 5516d37f41.

The previous issue in unzipping non-matching files has been addressed
with commit a49054ca2f2959f50f3188914ec0faebc90ebcbe. This CL rolls
forward to allow dumping container certifcates for APEXes.

Bug: 128848294
Test: Run check_target_files_signatures.py on target_files.zips w/ and
      w/o APEX files.
Change-Id: I662aab3d96fc40ac8e5e206e32b73ac763220b70
2019-03-20 12:40:34 -07:00
Tao Bao
0ff15de32a releasetools: common.UnzipTemp() filters out non-matching patterns.
common.UnzipTemp() calls `unzip` to do the unzipping, which will
complain if there's non-existent names in the given list. Prior to this
CL, callers had to do the work to remove non-existent entries. This CL
filters out the given patterns in common.UnzipTemp()/common.UnzipToDir()
to make callers' works easier.

Bug: 128848294
Test: `m dist` with aosp_taimen-userdebug (which calls
      ota_from_target_files.py on a target_files.zip that doesn't
      contain RADIO/*).
Test: `python -m unittest test_common.CommonZipTest`
Change-Id: I5e741c27ea8d0b8126c398a7e1b56a8deb4a3d7f
2019-03-20 12:38:09 -07:00
Vic Yang
51512c558c Add support for no-vendor-variant VNDK
When TARGET_VNDK_USE_CORE_VARIANT is set to true, the vendor variant of
VNDK libraries are by default not installed.  Instead, the core variant
will be used by vendor binaries at runtime.

To ensure the core variant of VNDK libraries are installed, we also add
a flag LOCAL_VNDK_DEPEND_ON_CORE_VARIANT to indicate that the vendor
variant module depends on the core variant module.  This flag should be
set by Soong for all VNDK libraries without the vendor variant
installed.  When the flag is set, the vendor variant binary is also
compared against the core variant binary to ensure they are
functionally identical.

As we are merging the two variants for some libraries, we need a new
link type to denote a module is usable as both native:vndk and
native:platform.  We add native:platform_vndk for this.

Bug: 119423884
Test: With the corresponding Soong change, build with
      TARGET_VNDK_USE_CORE_VARIANT set to true.
Test: Add a dummy VNDK library and a dummy vendor binary that depends
      on it.  Build with no-vendor-variant VNDK and check the core
      variant is installed.
Test: Add conditional compilation based on __ANDROID_VNDK__ in the
      dummy VNDK library and check build fails.

Change-Id: I40000f2728e8193212113c1ee950e9d697f2d40d
2019-03-20 10:23:04 -07:00
Anton Hansson
bec4aa868a Merge "Refactor generate_enforce_rro" 2019-03-20 15:18:19 +00:00
Anton Hansson
c371f19e89 Merge "Rename DEVICE arg in common buildprops script" 2019-03-20 10:25:01 +00:00
Bowgo Tsai
e4544b1c02 Moving system_other key into product.img
Currently system_other AVB public key is placed in system.img.
However, this makes it's harder to have a *generic* system.img
across different product configs. Moving the key to /product
partition to allow more product-specific AVB keys.

Device board config can add /product/etc/fstab.postinstall,
to mount system_other with this key in /product. It can specify
different mount options, file systems, verity settings, etc., in
this product-specific fstab as well.

Bug: 123611926
Test: `make productimage` checks the following is generated.
      $OUT/product/etc/security/avb/system_other.avbpubkey
      Also checks it's included in $OUT/installed-files-product.{json, txt}

Test: run the following command and checks that
      PRODUCT/etc/security/avb/system_other.avbpubkey is updated:
      ./build/tools/releasetools/sign_target_files_apks \
        --avb_system_other_algorithm SHA256_RSA2048 \
        --avb_system_other_key external/avb/test/data/testkey_rsa2048.pem \
        out/dist/*-target_files-*.zip signed-target_files.zip

Change-Id: I6804f29941bec54375d80bd68a5aedb5c23b842e
2019-03-20 16:18:11 +08:00
Logan Chien
99cdf5385d Update check_elf_file.py for clang-r353983
This commit updates how `check_elf_file.py` parses the symbol name
because the `llvm-readobj` (from clang-r353983) does not print "@" if
the symbol is not versioned.

See also. https://reviews.llvm.org/D56319

Bug: 128959554
Test: CHECK_ELF_FILES=true make check-elf-files
Change-Id: I0dee5e505225e57750a2c86cf0d25a151c218eb1
2019-03-20 15:24:00 +08:00
Daniel Norman
a4ab1d8490 Merge "Validates the config lists used by the script." 2019-03-19 23:56:25 +00:00
Tao Bao
2812fd32ac Merge "releasetools: Allow skipping PRESIGNED APEXes." 2019-03-19 23:04:07 +00:00
Daniel Norman
fab3f29e5a Merge "Adds optional --rebuild_recovery flag to merge_target_files." 2019-03-19 21:32:28 +00:00
Tao Bao
e134399cab releasetools: Allow skipping PRESIGNED APEXes.
This CL adds support that allows treating an APEX as pre-signed. We can
skip signing an APEX with `-e <apex-name>=` and
`--extra_apex_payload_key <apex-name>=`. Note that the payload_key and
container_key must be in consistent state - either they're both
PRESIGNED or none of them is. CheckApkAndApexKeysAvailable() has been
updated to perform the sanity check.

Bug: 123716522
Test: Run sign_target_files_apks.py with the above flags.
Test: python -m unittest test_sign_target_files_apks
Change-Id: Id1e2f3f2facd4a97a385983cc9b78c028f7e7e73
2019-03-19 12:25:02 -07:00
Daniel Norman
e596452f03 Validates the config lists used by the script.
This validation is to help ensure that any usage of custom merge config
files does not accidentally exclude any item that has been added to the
default config lists.

Bug: 124197349
Test: Run merge_target_files with custom merge config files.
Change-Id: I34c51cb75212368146a2944d37621f311060d24d
2019-03-19 17:47:49 +00:00
Tao Bao
5516d37f41 Revert "releasetools: check_target_files_signatures.py checks APEXes."
This reverts commit d8469727bc. The script
is broken on target_files.zip that don't contain any APEX.

Bug: 128848294
Test: Run check_target_files_signatures.py on target_files.zip w/o APEX.
2019-03-19 09:30:08 -07:00
Tao Bao
e3f9dc6113 Merge "releasetools: Kill the use of os.getenv('AVBTOOL')." 2019-03-19 15:53:26 +00:00
Tao Bao
f88e0ce01d releasetools: Kill the use of os.getenv('AVBTOOL').
OTA tools should pick up the avbtool, as listed in dict['avb_avbtool'],
from the current PATH (plus bin/ under the dir specified via `--path`),
the same way as handling all other host tools.

Test: `m dist`
Change-Id: I3eb4d2c61979b03d9c23b2403d9a38cf052d87ea
2019-03-18 21:01:43 -07:00
Yifan Hong
691defc4bd Merge "Add supernod build target." 2019-03-18 23:46:05 +00:00
Daniel Norman
a4911da57d Adds optional --rebuild_recovery flag to merge_target_files.
Also makes AddSystem check that an output_zip exists before attempting
to add the recovery patch to the output zip.

Bug: 128838154
Test: Running merge_target_files with --rebuild_recovery and verifying
it passes --rebuild_recovery to add_img_to_target_files.
Change-Id: I19347b2c0dabf29b7196045b18551b5d0687df2c
2019-03-18 12:29:31 -07:00
Tao Bao
fa9de0aae8 releasetools: Fix an issue when using APEX container key mapping.
The keys_info in the touched code is a tuple, which is immutable.

Bug: 123716522
Test: Run sign_target_files_apks.py with '-e foo.apex=bar' that replaces
      the APEX container key.

Change-Id: I4e57e46c93a56b7f6646764d021ebb42c19bf7f5
2019-03-18 10:26:09 -07:00
Anton Hansson
08231e1786 Refactor generate_enforce_rro
A few tweaks to make it easier to extend to generating RROs in
multiple partitions:
- deduce the module name inside generate_enforce_rro
- dedup rule definition
- tweak framework-res check to use source module name instead

Bug: 127758779
Test: verify noop on presubmit targets
Change-Id: I2f0d6270b21f5427c372c04a5c6e7fb712e72a9a
2019-03-18 14:21:09 +00:00
Tao Bao
aa7e993a22 releasetools: Support signing APEXes.
Bug: 123716522
Test: Run sign_target_files_apks.py to sign a target_files with APEXes.
Test: Run check_target_files_signatures.py on signed artifact.
Test: python -m unittest test_sign_target_files_apks
Change-Id: I3fa13e3d9461cf5e0838e0572d436e218164fe41
2019-03-15 22:15:51 -07:00
Tao Bao
1cd59f2a26 releasetools: Add apex_utils.py.
The added `testdata/testkey_with_passwd.key` is generated with:

$ openssl pkcs8 -inform DER -in testdata/testkey_with_passwd.pk8 \
    -out testdata/testkey_with_passwd.key -passin pass:foo

Bug: 123716522
Test: python -m unittest test_apex_utils
Change-Id: I7b7f00178ecab35c3e41ecf4d10295fe9bf480ab
2019-03-15 22:08:42 -07:00
Tao Bao
d8469727bc releasetools: check_target_files_signatures.py checks APEXes.
Only the container certs will be checked and reported. For the payload
within an APEX, we can't easily extract the cert info.

It needs to go along a longer path, if ever needed, by:
 - extracting public keys from all the available certs;
 - using each of them to verify against an APEX payload to find a match
   (`avbtool verify_image --image payload --key public_key`).

Bug: 123716522
Test: Run check_target_files_signatures.py on target_files with APEXes.
Change-Id: I2ef318e05433d2d65ab84e2dff9e01fb6ee3373d
2019-03-15 20:56:13 -07:00
Tao Bao
2cc0ca1770 releasetools: Add common.ExtractAvbPublicKey().
Bug: 123716522
Test: python -m unittest test_common
Change-Id: I1f645008a14cc882ef280f169c36e4b14a53ef88
2019-03-15 10:47:46 -07:00
Tao Bao
badceb2d60 releasetools: Initalize logger when signing.
Other modules have switched to logging module. sign_target_files_apks.py
needs to init the logger to get the logs.

Test: Run `sign_target_files_apks.py -v`. Check outputs.
Test: Run `check_target_files_signatures.py -v`.
Change-Id: Ic68c019f6fb14840561885f1194ad6efdfdb7d82
2019-03-15 09:53:00 -07:00
Yifan Hong
69e0d61cbe Add supernod build target.
superimage-nodeps and supernod depends
on images from $(ANDROID_PRODUCT_OUT) (not from
target files package). It doesn't rebuild source
images if they are present.

A typical workflow is:

    m -j
    # change code in system
    m snod -j
    m supernod -j

Test: For non retrofit, run:
      `m snod -j; m supernod -j`
Fixes: 128321505
Change-Id: Ib8c011cadb9c0cd334234aef39f19be6a48fee62
2019-03-13 11:15:21 -07:00
Chih-Hung Hsieh
cc47729719 Accept the "file:(project:)?filePath" directive.
Test: checkowners.py some_owner_files
Bug: 126777637
Bug: 126761563
Change-Id: Iee03521681bc38fa28970a015058e1eba3c8b87a
2019-03-12 11:56:33 -07:00
Anton Hansson
ef0b949be3 Rename DEVICE arg in common buildprops script
A followup change will make it not always equal TARGET_DEVICE,
so that name is unsuitable. Make its name follow the other product
variables.

Bug: 110206836
Test: presubmit
Change-Id: Icb1422ec5e7af658c5cc3070993c472e99805c6b
2019-03-12 18:03:23 +00:00
Daniel Norman
2c99c5bbd2 Adds support for optional config files to configure the merge.
Bug: 124467065
Test: Running `python merge_target_files.py` using the three new flags
and observing that their contents are passed to the merge_target_files() function.
Change-Id: I4de46f041f5ae8bc8be2730313ce873a952bf78e
2019-03-12 16:51:18 +00:00
Treehugger Robot
b6f2f8b2d9 Merge "releasetools: Add compatibility for custom out directory." 2019-03-11 19:51:14 +00:00
Pavel Salomatov
3267655004 releasetools: Add compatibility for custom out directory.
sign_target_files_apks script looks for the signapk.jar inside the out dir.
If the our dir is set to a different directory via OUT_DIR_COMMON_BASE the script does not work properly.

From now script checks if the OUT_DIR_COMMON_BASE is set, then searches the jar in the proper path.
If OUT_DIR_COMMON_BASE is unset, searches in "out" like it did before.

Test: Build with OUT_DIR_COMMON_BASE set and unset and verify that sign_target_files_apks works in both cases

Change-Id: I9218b98ff79526184f8353705640193405afac9e
2019-03-11 19:22:29 +03:00
Tao Bao
28a11ef040 Merge "releasetools: Include retrofit info into package metadata." 2019-03-07 18:11:30 +00:00
Treehugger Robot
fa2e796ce2 Merge "Remove logical partitions from ab_partitions.txt in retrofit package." 2019-03-07 01:06:42 +00:00
Tao Bao
393eeb4918 releasetools: Include retrofit info into package metadata.
Fixes: 126754990
Test: Generate retrofit and non-retrofit OTAs. Check the package
      metadata entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I076a013a6224a2ffea8e0b8581890b66f847f165
2019-03-06 16:19:11 -08:00
Yifan Hong
b433eba6a6 Remove logical partitions from ab_partitions.txt in retrofit package.
For retrofit updates:
ab_partitions -= dynamic_partition_list
ab_partitions += super_block_devices

For example,
if super_block_devices == ["system", "vendor", "odm"] and
   dynamic_partition_list == ["system", "vendor", "product"],
product partition needs to be removed from ab_partitions.txt
otherwise brillo_update_payload will generate a payload with
product partition in it.

Test: retrofit update that adds a partition
Fixes: 127425410

Change-Id: Id79a410cee3c611ac50d27f14282916aea34f938
2019-03-06 13:35:20 -08:00
Bill Peckham
8ff3fbdd08 Extract common.UnzipToDir, invoke that from merge_target_files.py
This change adds another utility function to common.py: UnzipToDir, which is
generally useful. Refactor merge_target_files.py to use it, and also refactor
other uses in common.py to use it.

Test: ota_from_target_files.py, validate_target_files.py, test_common.py
Bug: 124464492
Change-Id: Ia571070bceb7d3c8002304836bdf688485bf0dd9
2019-02-22 10:57:43 -08:00
Bill Peckham
889b0c6b09 Convert common.RunAndWait method to raise an exception on subprocess failure.
Then refactor the code in merge_target_files.py to adapt to this semantic
change. This makes the code more consistent with existing releasetools code,
and it's easier to follow.

Test: Failure cases (verify exception), success cases (merged target generated)
Bug: 124521133
Change-Id: I56f04e360d8ff8ffcd6245359cdeb79f4565a9c4
2019-02-21 19:02:07 -08:00
Bill Peckham
30ee35f908 Merge "Include merge_target_files.py deps in otatools.zip, use common argv processing." 2019-02-21 04:13:23 +00:00
Tianjie Xu
ef7737f921 Merge "Check the completeness of original blocks during target files validation" 2019-02-21 03:41:35 +00:00
Bill Peckham
f753e15791 Include merge_target_files.py deps in otatools.zip, use common argv processing.
The merge_target_files.py script needs fc_sort and sefcontext_compile, so
include these tools into otatools.zip via core/Makefile.

Modify tools/releasetools/merge_target_files.py to use the otatools common argv
processing to take advantage of the '--path' option so that we add point the
'--path' option to an extracted otatools.zip package to gain access to fc_sort,
sefcontext_compile, and soong_zip (previously included in otatools.zip).

Bug: 123600124
Test: extract otatools.zip, use --path option to point to it, verify result
Change-Id: I7d84525981b8741c6bdbcac9984256920fc7f417
2019-02-21 00:50:54 +00:00
xunchang
c0f77ee489 Check the completeness of original blocks during target files validation
The validate_target_files.py checks the 'incomplete' field of the range
in file_map. And range has already considered the shared blocks and
could be smaller in size than the original file range. Therefore, the
'incomplete' flag was set on the original range in common.py; and we
should switch to use the original range also during validation.

I also checked another flag usage in CanUseImgdiff(), and it has
explicitly rejected cases of shared blocks.

Bug: 124868891
Test: unit tests pass
Change-Id: I03959625d7b81fd83420db98f01d23f54064bcd2
2019-02-20 15:47:30 -08:00
xunchang
1cfe25155e OTA: Add an option to dump the metadata in ota package
Add a flag to write a copy of the metadata to a separate file. Therefore,
users can read the post build fingerprint without extracting the OTA package.

Bug: 124783265
Test: Check the dumped metadata file after generate A/B and non-A/B OTA.
Change-Id: I8918aec87bb81906ef0a7eee774178e9f689d91d
2019-02-20 15:32:09 -08:00
xunchang
abfa265f81 Add a flag to skip compatibility check
This serves as a workaround to skip the compatibility check for devices
with incompatible kernels.

Bug: 114240221
Test: generate and check the OTA package for wear device
Change-Id: I65b523a66648af7a77fc3ea79176764fe8ae8d02
2019-02-19 16:27:10 -08:00
Treehugger Robot
01a05483ed Merge "Support verifying system_other" 2019-02-18 05:21:27 +00:00
Bill Peckham
194b1c8508 Merge "Adding initial merge_target_files.py script to merge two target files packages." 2019-02-16 01:40:36 +00:00