Commit graph

1739 commits

Author SHA1 Message Date
Tao Bao
338c1b7e90 releasetools: Remove some legacy paths for prop rewrites.
sign_target_files_apks.py only needs to take care of the current
release. The legacy paths of ODM/build.prop, VENDOR/odm/build.prop, and
BOOT/RAMDISK/default.prop no longer exist in the target_files.zip from
master.

The other two cases of ROOT/default.prop and
RECOVERY/RAMDISK/default.prop are still kept in the code, as they will
still exist (as symlink or conditionally).

Test: Run sign_target_files_apks.py against
      aosp_taimen-target_files.zip. Check the rewritten prop files.
Test: `python -m unittest test_sign_target_files_apks`
Change-Id: I5e70bc2ccc0f3dcf0eace0718c59a3b0f89a9ff4
2019-06-21 10:26:15 -07:00
Tao Bao
b813120af1 releasetools: Make blockimgdiff Python 3 compatible.
Bug: 131631303
Test: `python -m unittest test_blockimgdiff`
Test: `python3 -m unittest test_blockimgdiff`
Change-Id: I8cf072ca4af6a525bc675cba0be998ff7be7f4f1
2019-06-20 14:30:30 -07:00
Tao Bao
1f1aafa30d Merge "releasetools: Make rangelib Python 3 compatibile." 2019-06-20 14:51:21 +00:00
Tao Bao
d660c8d8f3 releasetools: Make rangelib Python 3 compatibile.
Bug: 131631303
Test: `python -m unittest test_rangelib`
Test: `python3 -m unittest test_rangelib`
Change-Id: Ia8e26623d5967f2eea003252ee56b861350d626b
2019-06-19 10:30:34 -07:00
Tao Bao
c1a1ec30df releasetools: Make common.ZipWriteStr Python 3 compatible.
Python 2 and 3 behave differently when calling ZipFile.writestr() with
zinfo.external_attr being 0. Python 3 uses `0o600 << 16` as the value
for such a case (since
18ee29d0b8),
which seems to make more sense. Otherwise the entry will end up with
0o000 as the permission bits. This CL updates common.ZipWriteStr to
follow the logic in Python 3, in order to get consistent behavior
between using the two versions.

Bug: 131631303
Test: `python -m unittest test_common.CommonZipTest`
Test: `python3 -m unittest test_common.CommonZipTest`
Change-Id: If8429855d922ef1ad76591f703215a0ce5089f0f
2019-06-18 23:53:53 -07:00
Tao Bao
f1113e97ae releasetools: Update partitions in deterministic order.
Previously it was using regular dict.

Test: python -m unittest test_common.DynamicPartitionsDifferenceTest
Change-Id: If108a4512aeaf9d3c8775c030cad6e44342b9d3d
2019-06-18 12:14:32 -07:00
Treehugger Robot
87ebb3013f Merge "Allows skipping building the super images for partial builds." 2019-06-14 21:42:33 +00:00
Daniel Norman
0bf940cace Allows skipping building the super images for partial builds.
Previously, setting PRODUCT_BUILD_SUPER_PARTITION to false for a partial
build (with PRODUCT_USE_DYNAMIC_PARTITIONS == true) would fail to
include necessary keys in misc_info.txt that are required when merging
two partial builds to create a dynamic-partition-enabled mixed build.

This change ensures these necessary keys are included even when
PRODUCT_BUILD_SUPER_PARTITION is false. Setting
PRODUCT_BUILD_SUPER_PARTITION to false causes partial builds to skip
building super.img and super_empty.img, instead relying on these images
to come from the final merged build.

Bug: 134764140
Test: Building & booting a dynamic-partition-enabled mixed build, and
inspecting partial builds' logs / out folder to ensure that
super.img/super_empty.img were not created.

Change-Id: I99431a9a342e9b0617510e250597f3024ef39322
2019-06-14 11:42:45 -07:00
Tao Bao
cd951b2bdb releasetools: No-op change to use common.AVB_VBMETA_PARTITIONS.
common.AVB_VBMETA_PARTITIONS was recently added (commit
08c190fc89) for the same purpose.

Test: TreeHugger
Change-Id: I65572d54c22a753fdef80677377fcc9b684ee16f
2019-06-10 15:29:19 -07:00
Chris Gross
eab4f0e2e9 Add a logging statement after soong ZIP runs in the
merge_target_files.py step.

Bug: 134681035
Test: Built a merged build and ensured that the new timestamp was
visible in the log.

Change-Id: Ia6bbda48c7f57afdb6482253eaf0b3b0ea067468
2019-06-07 13:15:52 -07:00
Daniel Norman
6728fe850e Merge "Renames system to framework and other to vendor." 2019-06-06 21:14:14 +00:00
Daniel Norman
d5d70ea555 Renames system to framework and other to vendor.
Also fixes small nit from previous change to write_sorted_data().

Bug: 132788610
Test: python -m unittest test_merge_target_files
Test: Creating and booting a merged build.
Change-Id: I3dc43a4fe55b86b436dec08feb5d70096d38de36
2019-06-05 16:47:15 -07:00
Tao Bao
4978fa99d1 Rebuild recovery-from-boot patch when calling add_img_to_target_files.
When using Verified Boot 2.0, releasetools specifies a salt value based
on build fingerprint, so that to give idempotent images.

However, the change that removed static `ro.build.fingerprint` [1] broke
the behavior, as common.LoadInfoDict still relies on fingerprints.
Without a fixed salt, the first call to make_recovery_patch.py and the
second one (which writes IMAGES/{boot,recovery}.img) will see different
images, which leads to install-recovery.sh failure.

Note that currently there's a dependency that requires getting bootable
images through two separate calls. make_recovery_patch.py has to happen
first to get (placeholder) files in the system image. We then generate
canned fs_config files, and finally use add_img_to_target_files.py to
write the images.

This CL adds a quick workaround to force rebuilding the
recovery-from-boot patch while calling add_img_to_target_files.py.

[1] https://android-review.googlesource.com/c/platform/build/+/892933

Bug: 134123803
Bug: 134525174
Test: TreeHugger
Test: Build a non-A/B target that uses AVB. Run validate_target_files.py
      on the generated target_files.zip.
Change-Id: I5859e30be63bfd54398cf41fd2d907f15285f560
2019-06-04 22:44:45 -07:00
Tao Bao
08c190fc89 releasetools: Allow verifying chained vbmeta partitions.
Chained vbmeta partitions (vbmeta_system, vbmeta_vendor) were added to
support dynamic partitions. validate_target_files.py misses the logic in
handling such partitions.

Bug: 132882632
Test: Run validate_target_files.py on a target_files.zip that uses
      chained vbmeta_system partition.
Change-Id: Id06c575d13d5e9cc1b621f485ceb75d3e354c39f
Merged-In: Id06c575d13d5e9cc1b621f485ceb75d3e354c39f
(cherry picked from commit 814b14b3f7)
2019-06-04 12:49:41 -07:00
Daniel Norman
edf124780f Adds validation check that certain partitions come from a single build.
This is to prevent a user from accidentally including files from the
wrong build. For example, adding any SYSTEM/ line to other_item_list
while keeping SYSTEM/* in system_item_list would cause the other build
to introduce an extra or changed file in the system image.

Bug: 132730710
Test: python -m unittest test_merge_target_files
Change-Id: Ic1178cdc9b991114f293ff3f2b4e6054e06647c6
2019-06-01 04:39:46 +00:00
Daniel Norman
c9fc1120c1 Merge "Adds image-specific selinux_fc props to allow different file_contexts." 2019-05-23 23:42:13 +00:00
Daniel Norman
72c626f017 Adds image-specific selinux_fc props to allow different file_contexts.
This enables mixed builds to use the file_contexts.bin from the system
build when regenerating images that come from system target files, and
similarly for file_contexts.bin from the other build when regenerating
images from other target files.

In monolithic (non-mixed) builds all image-specific selinux_fc props
point to the same file_contexts.

Bug: 132108151
Test: Built and booted mixed build devices.
Change-Id: Id51ed6d96ea6337879f1ab21d47c93c67bc25312
2019-05-23 11:21:02 -07:00
Bowgo Tsai
71a4d5cdd5 Moving /odm/build.prop to /odm/etc/buid.prop
In device root directory, we have the following symlinks:
  - /odm/app -> /vendor/odm/app
  - /odm/bin -> /vendor/odm/bin
  - /odm/etc -> /vendor/odm/etc
  ...

This allows the Generic System Image (GSI) to be used on both devices:
  1) Has a physical odm partition, where those symlink will be hidden
     when /odm is used as the mount point
  2) Has no physical odm partition and fallback to /vendor/odm/.

We can't just have the symlink /odm -> /vendor/odm, because the former
devices won't have /vendor/odm directory, which leads to mount failure
when the mount point /odm is resolved to /vendor/odm.

The existing /vendor/odm/build.prop won't be loaded in the latter
devices, because there is no symlink:
    - /odm/build.prop -> /vendor/odm/build.prop.

Note that init blocks reading through direct symlinks (O_NOFOLLOW) so
the above symlink won't work either. This CL moves the odm build.prop
to /odm/etc/build.prop for init to load it (symlinks in earlier
components of the path will still be followed by O_NOFOLLOW).

Bug: 132128501
Test: boot a device and checks /odm/etc/build.prop is loaded
Test: make dist with an odm.img, checks $OUT/odm/etc/build.prop is loaded
Change-Id: I6f88763db755c9ec6068bfdd9cee81c19d72e9d7
Merged-In: I6f88763db755c9ec6068bfdd9cee81c19d72e9d7
(cherry picked from commit 6c62884000)
2019-05-22 14:04:39 +08:00
Chris Gross
fabf50a577 Add merging of apexkeys and apkcerts metadata
Bug: 131437873
Test: Built system-only and vendor builds for merge. Compared
resulting apkcertx and apexkeys text files to that of a monolithic
build,
Test: Created colliding entries in both apexkeys and apkcerts text
files and ensure the script exited with an appropriate error message.
Test: Created unit tests to cover both non-colliding and colliding
entries
Change-Id: I6e42ce682ffa9059344e8cd63ba3a720c1f93452
2019-05-20 18:35:22 +00:00
Tao Bao
bc7e3f98f4 Merge "Add sign_apex.py that signs a given APEX file." 2019-05-13 22:47:24 +00:00
Tao Bao
e7354ba447 Add sign_apex.py that signs a given APEX file.
This CL moves SignApex() from sign_target_files_apks into apex_utils,
and adds sign_apex that allows signing a standalone APEX file directly.

Test: Run the following command and check the output file.
$ build/make/tools/releasetools/sign_apex.py \
     -v \
     --container_key \
         build/make/target/product/security/testkey.x509.pem \
     --payload_key external/avb/test/data/testkey_rsa4096.pem \
     --payload_extra_args \
         "--signing_helper_with_files ./signing-helper.sh" \
     foo.apex \
     signed-foo.apex
Test: Run sign_target_files_apks.py on crosshatch target_files.zip.
Change-Id: I4b2422fd5cb1c60a3aa94511475e2a0e5b1666ca
2019-05-13 12:41:26 -07:00
Tao Bao
1dac1d4164 releasetools: Unzip all files when generating image archive.
Commit 7df64c3e starts to call common.LoadInfoDict() when generating
image archive, which reads additional files under BOOT/, RECOVERY/ and
ROOT/. Unzip everything from the target_files.zip.

Bug: 132456827
Test: Run img_from_target_files.py on previously failing
      target_files.zip.
Change-Id: I22ee57c4f765bee9494478bf115b1581877401f4
2019-05-10 15:57:20 -07:00
Daniel Norman
714bd12861 Fixes typo when calling process_dynamic_partitions_info_txt.
Test: Running merge_target_files.py
Change-Id: Ieddfbfe524824844fc12cc10347f667332149f7c
2019-05-08 16:20:02 -07:00
Tao Bao
13ebc021ec releasetools: Update the comments regarding selinux_fc.
The former comment no longer applies, as we have been always packing
META/file_contexts.bin in a target_files.zip (commit aa7318c3, since
Nougat), and we no longer look for the one under BOOT/RAMDISK/ (commit
d14b8956, since Q).

Test: N/A
Change-Id: I03f361234bf440e942f21e5a624862590248544b
2019-05-08 12:40:27 -07:00
Chris Gross
84c96eba1e Merge "Add BUILD_RECOVERY_IMAGE and BUILD_BOOT_IMAGE flags" 2019-05-08 18:21:42 +00:00
Daniel Norman
a61cde0747 Begins merging META/dynamic_partitions_info.txt.
This file is used by OTA generation so it needs to appear in mixed
builds with the combined content from the system and other versions of
the file.

Test: python -m unittest test_merge_target_files
Test: Running merge_target_files on a dynamic-partition-enabled build
and observing the resulting target files.
Bug: 131889742

Change-Id: I4ddbebc087e430f6307d0bd5461121a374e58ea4
2019-05-07 11:53:58 -07:00
Chris Gross
a784ef197c Add BUILD_RECOVERY_IMAGE and BUILD_BOOT_IMAGE flags
Bug: 123428770
Test: Built system-only image and checked that no boot.img or
recovery.img files where created. Booted the resulting merged build on
device.

Change-Id: I760476502775e68125907c39e66b8665e789a798
2019-05-07 11:00:37 -07:00
Bill Peckham
7dbbc3075b Merge "Use file_contexts.bin from other build to regenerate images." 2019-05-07 17:26:25 +00:00
Bill Peckham
736b223d5f Use file_contexts.bin from other build to regenerate images.
The old process_file_contexts_bin function did not properly generate a usable
file_contexts.bin to regenerate images, so instead use the file_contexts.bin
from the other partial target files package. When combining any one of several
other partial target files packages with a single system partial target files
package, this file will properly apply contexts as long as the same source is
used for the system partial target files.

Test: Verify that file contexts are properlty applied to vendor image.
Bug: 131584454
Change-Id: I16f8cc3b7f2eb7f09746f0ddcb2c1daf3fd19da6
2019-05-06 23:23:23 +00:00
Tao Bao
d6085d6834 releasetools: Support replacing the signing keys for chained vbmeta.
Bug: 131710801
Test: Run sign_target_files_apks.py on a target that uses vbmeta_system.
Change-Id: I3bc526af3ec9f2680ca17ee5535607cff3ae9523
2019-05-06 12:56:52 -07:00
Magnus Strandh
234f4b418f Ensure that 'release-keys' are set on properties
Some properties had 'test-keys' still set
after signing the target files zip for release.

These properties are now added to the RewriteProps
method.

Bug: 131810966
Test: manual
Test: `atest releasetools_test`
Change-Id: Ifb352ed28f5100f1e9f686d77e935723f7f6d3ae
2019-05-03 10:42:35 -07:00
Tao Bao
31c2bf905e Merge "Update the references to verity tools." 2019-05-03 13:41:45 +00:00
Tao Bao
d5ab10eed4 Update the references to verity tools.
Test: `m -j dist` with aosp_marlin-userdebug
Test: `atest releasetools_test`
Test: Run signing with otatools.
Test: TreeHugger
Change-Id: I60d5bad4435d588c5f97b2245ca67ed3d55299b7
2019-05-02 18:32:11 -07:00
Tao Bao
77a4aa0c06 releasetools: Skip on empty care_map.
common.GetCareMap() may return an empty list on unavailable care_map
since the change in commit 8bdfb990ea.
Caller needs to handle such a case accordingly. This CL fixes the caller
in add_img_to_target_files.py, and changes the return value to None to
break legacy callers loudly.

Fixes: 131794385
Test: `atest releasetools_test`
Change-Id: I7c94f456064199237e84ef75732bdd10ebe31736
2019-05-01 22:21:25 -07:00
Yifan Hong
a2488b7d85 Merge "Add BOARD_SUPER_IMAGE_IN_UPDATE_PACKAGE flag." 2019-04-30 19:49:12 +00:00
Yifan Hong
0e97dbb8ca Add BOARD_SUPER_IMAGE_IN_UPDATE_PACKAGE flag.
When set, product-img-tag.zip contains super.img instead of individual
user images from target files. For virtual devices, super.img is needed
to boot the device, but individual user images aren't needed.

Test: on A/B DAP, with flag set:
     - m updatepackage and look at img.zip
     - img_from_target_files
     both have super.img and not system / vendor / system_other
Test: on non-A/B DAP, with the flag set:
     - m updatepackage and look at img.zip
     - img_from_target_files
     both have super.img and not system / vendor
Test: on A/B retrofit, with the flag set:
     - m updatepackage and look at img.zip
     - img_from_target_files
     both have super_*.img and system_other.img, but not system / vendor
Bug: 113175337

Change-Id: I94e33091d0c837cae40776176b4dcfdd338aba90
2019-04-29 10:50:42 -07:00
Tao Bao
0376d7dcbb Merge "releasetools: GetCareMap supports non-sparse image." 2019-04-26 22:59:30 +00:00
Bill Peckham
3129d2c971 Merge "Fix missing import." 2019-04-26 02:38:08 +00:00
Yifan Hong
001ebfcb30 Merge changes from topic "productmanifest"
* changes:
  ota_from_target_files: include metadata when odm / product is changed.
  Add product manifest.
2019-04-25 22:51:27 +00:00
Bill Peckham
540d91ae3f Fix missing import.
Test: Verify that script runs properly now that import is present.
Change-Id: I255bc6ca2ba1f2cc387001c6f4bf9690a90eb47c
2019-04-25 14:18:16 -07:00
Yifan Hong
51d375610c ota_from_target_files: include metadata when odm / product is changed.
When odm is changed, device manifest/matrices should be included.
When product is changed, framework manifest/matrices should be included.

Bug: 130714844
Bug: 126770403
Test: build with odm and product VINTF metadata

Change-Id: I49c8083e0e7185ae7b96047d68f1f624b1113dfc
2019-04-25 10:22:39 -07:00
Bill Peckham
d540f5a9a5 Merge "Sort content of merged target files package." 2019-04-25 16:19:47 +00:00
Tao Bao
8bdfb990ea releasetools: GetCareMap supports non-sparse image.
Test: `atest --host releasetools_test`
Test: `m dist` with a target that uses non-sparse images.
Test: Run UpdateVerifierTest on blueline.
Change-Id: I8fdebee42fcaac78c2d1be2a84ddb69f46ec701d
2019-04-25 09:15:07 -07:00
Tao Bao
f454c3a0b4 releasetools: Accept PRESIGNED keys in apexkeys.txt.
For an PRESIGNED APEX, it has the following format, which should be
considered as a valid input.

name="foo.apex" public_key="PRESIGNED" private_key="PRESIGNED" container_certificate="PRESIGNED" container_private_key="PRESIGNED"

Bug: 131153746
Test: Run sign_target_files_apks.py on a target_files.zip with PRESIGNED
      APEXes.
Test: python -m unittest sign_target_files_apks
Change-Id: I51076b0c6eddfb75637d37659a08009f0a88e931
2019-04-25 00:10:07 -07:00
Tao Bao
a212ab59df Merge "releasetools: PRESIGNED APEX container entails PRESIGNED payload." 2019-04-25 02:03:50 +00:00
Bill Peckham
9662cfb234 Sort content of merged target files package.
By sorting the content of the final output merged target files package, the
merged target files package is more like the target files packages generated by
a build.

Test: Generate merged target files package, verify that content is sorted.
Change-Id: Ic0c198630ebd7692a3f3f9663d85e4b45229175c
2019-04-24 17:59:01 -07:00
Tao Bao
f98fa1094e releasetools: PRESIGNED APEX container entails PRESIGNED payload.
We used to require explicitly setting both (e.g. `-e foo.apex=` and
`--extra_apex_payload_key foo.apex=` to skip signing `foo.apex`).

This CL allows specifying `-e` alone to achieve the same result.
However, if a conflicting `--extra_apex_payload_key` is also specified,
that would be considered as a config error.

Bug: 131153746
Test: Run sign_target_files_apks.py with `-e foo.apex=` alone to skip
      signing foo.apex.
Test: Run sign_target_files_apks.py with `-e foo.apex=` and
      `--extra_apex_payload_key foo.apex=key` and expect assertion error.
Change-Id: Ia747f59ee726b60bdb1445024e749320171064c2
2019-04-24 15:17:23 -07:00
Daniel Norman
b8a2f9df51 Adds support to create img from target files directory instead of zip.
This is used by merge_target_files to prevent an unnecessary unzip and
copy.

Test: Ran merge_target_files.py and booted using the img.zip.
Change-Id: I6fe0dd025b30b3f4965c9b22fb6943019bf5899b
2019-04-24 20:53:22 +00:00
Bowgo Tsai
78369ebbc1 Avoid signing debuggable boot-debug.img
The boot-debug.img should NOT be release signed and can only be used
if the device is unlocked. Adding a check to prevent the tool from
signing this debuggable boot.img.

See the following for more details about boot-debug.img:

  https://android-review.googlesource.com/c/platform/build/+/947857

Bug: 126493225
Test: put a file /force_debuggable into boot.img, checks the following
      command fails:
        ./build/tools/releasetools/sign_target_files_apks \
        out/dist/*-target_files-*.zip signed-target_files.zip
Change-Id: Ia5232949cb9582d2b4eaa171d9e9f3fe7317d418
2019-04-23 12:55:10 +08:00
Tao Bao
7b3e97b1ea Merge "releasetools: Update the comment on common.AVB_PARTITIONS." 2019-04-23 03:09:26 +00:00