Although the SHA-1 of the source boot image has been encoded in the
filename, the updater code currently doesn't extract that and would fail
to find a matching backup from /cache. This could lead to corrupt
install, if it's interrupted while patching the boot image.
Bug: 80630342
Test: Generate an non-A/B incremental package. Verify the script.
Change-Id: Ie2b1a90836c3ccb4758e1a2d9b69a88de94ae0c7
Bug: 79951743
Test: Run validate_target_files.py on target_files.zip that has
non-monotonic ranges listed in IMAGES/system.map.
Change-Id: I28d3ca6972d361362589cd51e60731af9994a551
Targets can define 'TARGET_USERIMAGES_SPARSE_EXT_DISABLED := true' to
generate non-sparse system images, but validate_target_files.py doesn't
work with such images. This CL adds a workaround to temporarily skip the
file consistency check for such images.
Bug: 79616357
Test: Run validate_target_files.py on a target_files.zip that's not
using sparse image.
Test: Run validate_target_files.py on marlin target_files.zip (which
uses sparse image).
Change-Id: I1f4066c5b3fec595b10cab10283d62c1c5a6c624
We used to write a fake post-timestamp into package metadata for
"--override_timestamp", which allows a package to be pushed based on
the fake timestamp. This CL stops doing that by using the designated
"ota-downgrade=yes" instead, but keeping the "--override_timestamp" flag
for backward compatibility.
Now both "--override_timestamp" and "--downgrade" allow pushing packages
with downgrade timestamps. The only differenece is that "--downgrade"
enforces a data wiping, because it's intended a real downgrade (based on
Android version or security patch level). "--override_timestamp" serves
the path of a legit "upgrade", but unfortunately with build timestamps
in reverse order (e.g. the two builds are cut from different branches).
With this CL, we write "post-timestamp" to all packages.
Bug: 77811049
Test: Generate an incremental pakcage with "--override_timestamp".
Test: Generate an incremental pakcage with "--downgrade".
Test: python -m unittest test_ota_from_target_files
Change-Id: I4ae4ce77cb600a06ffd9b15b6d4c730314aa0b79
ota_from_target_files.py:
Rename PropertyFiles._GetPropertyFilesString to
PropertyFiles.GetPropertyFilesString.
Currently only PropertyFiles.Compute and
PropertyFiles.Finalize gives access to _GetPropertyFilesString.
But Compute force sets reserve_space=True,
and Finalize requires reserved_length.
_GetPropertyFilesString is useful method that can
be used outside releasetools.
One of the usage is in bootable/recovery/updater_sample/.
Test: python -m unittest test_ota_from_target_files
Change-Id: I2cc44ec46a0e68ba071531b003af8cdbfe90e588
Signed-off-by: Zhomart Mukhamejanov <zhomart@google.com>
This reverts commit 2a40cc6996.
Reason for revert: Broke a bunch of builds, platform/system/tools/bpt must be missing from more manifests.
Change-Id: I4c367a1fec5c62c247544227e9d3b596227e31da
...instead of copying from IMAGES/partition-table.bpt during
sign_target_files_apks. This addresses a comment on go/oag/665565, but
is done as a separate change so that it can be cherry-picked later (or
not at all) to downstream branches to avoid breaking signing of existing
target-files zips.
Bug: 72837107
Test: Local sign_target_files_apks run of locally built target-files
Change-Id: Id14c859eefe075fc56b15869f2f21c570eb07e65
For Android Things targets (or any other target which has
BOARD_BPT_INPUT_FILES defined), add_img_to_target_files will generate a
partition-table.img using bpttool. It also adds the final combined .bpt
definition file into target-files in IMAGES/partition-table.bpt.
When we're signing using sign_target_files_apks, add_img_to_target_files
needs to regenerate the partition table, but META/misc_info.txt still
contains the original list of bpt input files from the build that aren't
available. This change extracts the final bpt from the input
target-files, adds it to META/ in the output target-files, and then
updates the board_bpt_input_files property to point to it.
Bug: 72837107
Test: Local sign_target_files_apks run of locally built target-files
Change-Id: Id79125208f31c78b1ac2079172f9c91a9203849b
Output is useful for debugging.
Bug: 72837107
Test: Local sign_target_files_apks run of locally built target-files
Change-Id: I5c27fcc86fa3a51080e0502eb8f5f01a40b033c3
This CL separates the options into three groups (excluding the global
options provided via common.py).
- Non-A/B OTA specific options;
- A/B OTA specific options;
- Common options that apply to both.
It mostly reshuffles the lines, with minor change to "--verify" that
removes the obselete remounting behavior (which doesn't apply to
block-based OTA).
Hopefully this makes the expected behavior of some options less
confusing.
Test: `build/make/tools/releasetools/ota_from_target_files.py`
Change-Id: I194ea52c4f7d6a3c7f34531abbcf3fdc7b7f4fa8
This CL defaults the journal size to 0 for system images (i.e. system,
vendor, system_other, oem, product). We used to do this by explicitly
defining BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_JOURNAL_SIZE to 0 in
device-specific BoardConfig. Targets can still specify a non-zero
journal size as needed.
With this CL, marlin/sailfish gets 32MB / 16MB free space back for
system and vendor respectively.
Bug: 75975085
Test: `m dist`. Check the journal size in the generated images.
Change-Id: Ib3185d07c49be2b4f0fac5fe17ec1a82093ba0c0
(cherry picked from commit 965542fe69)
Defaults the reserved blocks for root to 0% on read only partitions
(system, system_other, vendor, oem). It also adds support for
explicitly specifying the extfs reserved percentage via
BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_EXTFS_RSV_PCT.
This eventually translates down to the -m option for mkfs.
Removing the reserved space can save at least 5% from the default.
dumpe2fs system:
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
Bug: 75975085
Test: Build, verify reserved space is changed accordingly
Change-Id: I212d82741908b636db0d658a1c4847bbaadfd5ba
(cherry picked from commit 5ff758799c)
We've added support in brillo_update_payload that allows additionally
looking for images under RADIO/ in the given target_files zips [1]. This
avoids having duplicate radio images in target_files zips.
Also adjust the unittest in test_ota_from_target_files.py to cover this
path.
As a result of this CL, the radio images will no longer appear in the
image archive (i.e. <target>-img.zip) as well - they are less useful
anyway because we have packed only the _updatable_ pieces that are part
of full bootloader/radio images.
Bug: 77218220
Test: `python -m unittest test_ota_from_target_files`
Test: `python -m unittest test_add_img_to_target_files`
Test: `m dist` produces the same full OTA package
Test: Build marlin-userdebug in internal branch. Check the image zip.
Change-Id: I05579480f0bb9ab90aaeecf75969ee29b6904ad6
It was initially introduced in commit
2e735ca34e, where it packs additional
vendor images into target_files zip in order to generate OTAs. We can
acheive the same goal with INSTALLED_RADIOIMAGE_TARGET, which is the way
being actively used across all targets, including IoT (the former
Brillo) targets.
Bug: 78201540
Test: `m dist` with aosp_marlin-userdebug
Test: Code search shows no active user of BRILLO_VENDOR_PARTITIONS.
Test: `python -m unittest test_add_img_to_target_files`
Change-Id: I8803d5377b5a39304a701cceafb243f9a228347d
Non-A/B devices need to include the DTBO image
within the recovery partition to be self-sufficient
and prevent OTA failures.
Test: Ran 'm dist' and verified that the DTBO image
was included in recovery.img using unpack_bootimg.
Also ran 'make' and verified that the DTBO image was
included in recovery.img using unpack_bootimg.
Also tested that aosp_angler-userdebug could
boot into recovery(including a fake dtbo.img
in recovery image).
Bug: 74763691
Change-Id: I38c9c395c95d21f4da42cfa646063bd4416f6bd8
The following is a buggy pattern that won't capture anything into err.
The issue is benign, since a failed run would be eventually captured by
a subsequent check.
p = Run(["aapt", ...], stdout=subprocess.PIPE)
output, err = p.communicate()
if err:
raise ...
This CL changes the error detection to be based on the return code from
aapt. It also adds some sanity test to ensure the call to aapt works.
The test app is built from AOSP com.android.cts.ctsshim (chosen mostly
because of its small size).
Test: python -m unittest test_common
Change-Id: I337f141bd0fc5f0801dfc628c601b88b7640789c
For the functions in common.py, capture the stdout/stderr outputs when
shelling out to external tools. Dump the outputs on errors.
Bug: 76123422
Test: Inject errors to signapk.jar / brotli / unzip. Check the outputs.
Change-Id: Ib2d4272528b83a50bb727788cf8a5211e2ddade8
build_image.py used to invoke build_verity_metadata.py with a hard-coded
path of 'system/extras/verity/build_verity_metadata.py', which makes it
hard to run unittests from non-$(ANDROID_BUILD_TOP) directory.
This CL adds the dependency on the tool, so that it gets installed to
$(HOST_OUT_EXECUTABLES), then removes the hard-coded path.
Bug: 74544459
Bug: 76015688
Test: `m dist`
Test: python -m unittest test_validate_target_files
Change-Id: I0dcf4eb067a0db6f099cb589eb99a151a05c7f2b
This CL handles a path that uses system-root-image on non-A/B device.
For this path, we can't generate recovery-from-boot patch with imgdiff,
because boot/recovery images contain different number of entries (only
recovery iamge has ramdisk image now).
Using BOARD_USES_FULL_RECOVERY_IMAGE can work around the issue, at the
cost of extra size. Compared to carrying full recovery image, this CL
saves the cost of the kernel size, by putting a patch that's roughly the
size of the recovery ramdisk.
The applypatch executable already detects and handles a bsdiff patch
automatically. No change required to that end.
Note that it won't further reduce the patch size by handling that
ramdisk entry specially, because (a) that's the only difference between
the two images; and (b) there's no corresponding data in boot image to
be diff'd against.
Bug: 72731506
Test: `m dist` with aosp_angler-userdebug. Check the device can install
recovery image successfully (after intentionally corrupting the
recovery image).
Test: Build aosp_angler-userdebug with BOARD_BUILD_SYSTEM_ROOT_IMAGE set.
Verify the generated patch files.
Test: Run validate_target_files.py with the target_files.zips.
Change-Id: I69c06f51ba8c39ae059c5e9a6872a9f10600cf17
In FinalizeMetadata and PropertyFiles, we need to reserve space between
the calls to Compute() and Finalize(). We used to put a 10-byte
placeholder, in the hope of covering the 'offset:length' space for the
metadata entry, as well as the possible value changes in other entries.
However, this could fail in two possible cases: (a) metadata entry
itself has a large offset (e.g. staying near the end of a 1-GiB package,
where the offset itself has 10-digit); or (b) the offsets for other
entries change substantially due to entry reordering. Note that for case
(b), it's space inefficient to always reserve 15-byte for _each_ token
in the property-files.
This CL handles both of these two cases. For (a), we bump up the 10-byte
to 15-byte, which is large enough to cover a package size up to 10-digit
number (i.e. ~9GiB) with a metadata entry size of 4-digit. All these
15-byte will be used for the metadata token alone.
For (b), we add a fallback flow that would retry one more time, but
based on the already signed package that has entries in desired order.
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate aosp-bullhead full OTA with '--no_signing' flag.
Change-Id: If20487602d2ad09b3797465c01972f2fa792a1f1
This CL exposes ota-property-files flag for non-A/B OTA packages.
Currently the line only contains the info for the METADATA entry, for
example "ota-property-files=metadata:69:286". This allows system updater
to just download the METADATA entry, as opposed to downloading the
entire package, to learn about the info regarding the OTA package (e.g.
post-OTA build fingerprint). Note that this requires the OTA server-side
support to pass down the flag along with the update URL.
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate a non-A/B package and check the property-files string.
Change-Id: I1482c587e18ea7101c8328777ea988c2d8ca06ac
AbOtaPropertyFiles writes 'ota-property-files' into package metadata.
Comparing to StreamingPropertyFiles, AbOtaPropertyFiles additionally
writes a token of 'payload-metadata.bin' for a virtual entry that's head
of the 'payload.bin'. This entry can be used by the system updater to
verify if a payload is applicable to the underlying device.
AbOtaPropertyFiles intends to replace StreamingPropertyFiles, as it
covers all the info of the latter. We keep them both in P, and will
remove the latter in later release.
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate full and incremental A/B OTAs. Check the property-files
in the METADATA entry.
Change-Id: If5872c8ee0a97a8a95b07d4c69b463aa3fe1b9b9
And move StreamingPropertyFiles as its subclass. We will need similar
PropertyFiles instance for non-A/B OTA as well (to expose the
offset/size for the METADATA entry).
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate an A/B OTA. Check the generated property-files string.
Test: pylint --rcfile=pylintrc \
ota_from_target_files.py \
test_ota_from_target_files.py
Change-Id: If90d97f0b330749fd8a6cde2ed9d0d6cd6ea60a8
For a given (signed) target-files.zip, this CLs allows verifying the
Verified Boot related images. It works with both of VB 1.0 and VB 2.0
images.
As part of the CL, it also moves validate_target_files.py to argparse,
which is more flexible than the traditional getopt module.
Also add unittests for the VB 1.0 path. VB 2.0 tests will be added in
follow-up CL.
Example usage:
- Run the script on aosp_bullhead target-files.zip.
$ ./build/make/tools/releasetools/validate_target_files.py \
--verity_key build/target/product/security/verity.x509.pem \
--verity_key_mincrypt build/target/product/security/verity_key \
aosp_bullhead-target_files-4522605.zip
- Run the script on aosp_walleye target-files.zip.
$ ./build/make/tools/releasetools/validate_target_files.py \
--verity_key external/avb/test/data/testkey_rsa4096.pem \
aosp_walleye-target_files-4627254.zip
Bug: 63706333
Bug: 65486807
Test: Run validate_target_files.py on target_files.zip files.
Test: PYTHONPATH=build/make/tools/releasetools python -m unittest \
test_validate_target_files
Change-Id: I170f14d5828d15f3687d8af0a89a816968069057
This CL breaks down ComputeStreamingMetadata() into mutiple member
functions of StreamingPropertyFiles class, which correspond to the
two-pass logic when generating streaming property files (aka streaming
metadata).
StreamingPropertyFiles.Compute() does the work for the first pass, by
putting placeholders before doing initial signing. Finalize()
corresponds to the second pass, where the placeholders get replaced with
actual data. Verify() can be optionally called to assert the correctness
of the work.
The separation between Compute() and Finalize() is to allow having
multiple StreamingPropertyFiles instances (in coming up CLs). This way
we can call Compute() multiple times for each instance, followed by only
one call to SignOutput(). And similarly for Finalize().
Bug: 74210298
Test: Generate an A/B OTA package. Check the METADATA entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I45be0372a4863c4405e6d8e20bcb9ccdc29e7e11
The file was broken due to earlier touches:
- Missing 'import zipfile';
- Mismatching arguments when calling GetSparseImage().
Bug: 73996151
Test: Run validate_target_files.py with a walleye-target_files.zip.
Test: pylint --rcfile=pylintrc validate_target_files.py
Change-Id: I3692bd51fb27a3da698e06b75155e84502549f66
And add unittests for ComputeStreamingMetadata().
This prepares for the changes that add additional property-files (for
both of A/B and non-A/B).
Bug: 74210298
Bug: 72751683
Test: python -m unittest test_ota_from_target_files
Test: Generate A/B OTA package. Check the ota-streaming-property-files
in the METADATA entry.
Change-Id: Ib4b069f61c2c06c035c0cff73a55112f3936b969
There's a mismatch in WriteABOTAPackageWithBrilloScript().
temp_zip_file = tempfile.NamedTemporaryFile()
...
common.ZipClose(temp_zip_file)
It's benign since common.ZipClose() happens to be calling
"temp_zip_file.close()". This CL moves the use of tempfile to
common.MakeTempFile(), so that the tempfile will be cleaned up
automatically as part of the call to common.Cleanup(). (Not fixing the
close() directly, since the nearby lines will be refactored into another
function shortly.)
Also remove one assert in the same function, which trivially holds in
the current code.
Test: Generate an A/B OTA.
Change-Id: I53b375d1150820de718dec0ead55abf5f4951071
Also minor cleanups to make it pylint clean.
Test: Run check_target_files_signatures.py with a target-files.zip.
Test: pylint --rcfile=pylintrc check_target_files_signatures.py
Change-Id: Ife3b54c7805c2f2562e87e91ab4b4de355782012
In addition to the unzipping work, common.UnzipTemp() kindly bundles an
open ZipFile object as part of the return value. It doesn't look very
helpful to the callers though. It also looks less obvious that the
caller needs to properly close the handle (missing the close here is
benign though). This CL just removes the ZipFile object out of the
return value, and leaves the work to callers.
Test: `m dist` on both of A/B and non-A/B target.
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_common
Test: python -m unittest test_ota_from_target_files
Test: Check the callers to common.UnzipTemp() in code search.
Change-Id: Id47da3fd42a0e76d6ae8851f05780db319ee48cf
This should be the last case to be moved over.
Test: Generate an incremental BBOTA (which exercises the changed code).
Test: `rgrep mkdtemp` gives no more instance.
Change-Id: I76db069476201cdfaf3a2de9d9635dfe54507f7a
... from the following functions in add_img_to_target_files.py.
AddSystem
AddSystemOther
AddVendor
AddProduct
AddDtbo
AddUserdata
AddVBMeta
AddPartitionTable
AddCache
The last user of the parameter in img_from_target_files.py has been
removed in commit 2bb109709a (in O).
Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Test: Check all the callers to the above functions.
Test: m dist
Change-Id: I551d1683def8f8535062fc90f68dafa0f4252822
The file has been removed from target-files.zip since commit
c19a8d5590 (Gingerbread), whose info has
been consolidated into META/misc_info.txt.
Test: `m dist`
Change-Id: Ic144457954f5742ea082dcd9ffbea71df4afe46e
With test_utils.construct_sparse_image(), creating valid sparse image is
no longer a blocker for testing these functions.
Test: python -m unittest test_add_img_to_target_files
Change-Id: Iff1f15837cc744bad52e2120f9a9ad94d7db50d5
This CL adds a new flag '--skip_postinstall' that allows skipping all
the postinstall hooks when generating an A/B OTA package (default:
False). Note that this discards ALL the hooks, including non-optional
ones. Should only be used if caller knows it's safe to do so (e.g. all
the postinstall work is to dexopt apps and a data wipe will happen
immediately after).
Bug: 73547992
Test: python -m unittest test_ota_from_target_files
Test: Generate a full OTA package for walleye. Examine the generated
payload.
Change-Id: Ifc069e897b4019605051eabfd221230a6a37867c
Since we have been carrying test certificates in testdata/ for other
tests, do the same for test_sign_target_files_apks.py. Copy
verity.x509.pem from build/target/product/security/ to testdata/ for
that purpose.
Also capture the stderr output in ReplaceVerityKeyId().
Test: python -m unittest test_sign_target_files_apks
Change-Id: Ie11e042086952e8a4a5a63950cb0b16cc436b7e6
Also minor clean up to the function, to suppress the following lint
warning.
R:1813, 2: Redefinition of cert type from list to unicode (redefined-variable-type)
Test: python -m unittest -v test_common.CommonApkUtilsTest
Change-Id: Ib4b1dfa8b19e505fc70684b648efc36171c73bbf
Add construct_sparse_image() to test_utils.py, which is a util function
to create sparse images. The new tests also partially cover the recent
changes that add 'incomplete' and 'uses_shard_blocks' tags.
Test: python -m unittest test_common
Change-Id: Ia15f5c4ad12423691216ebbad2c28f95c8427d7e
