Previously it was following a wrong order by doing `zipalign` after
calling SignApk, which effectively compromised the signature. This CL
corrects the logic, and follows the same flow as in build system:
- Pack APEX file;
- `zipalign -f 4096`;
- Call SignApk to sign the container with `-a 4096` flag.
Bug: 129148142
Test: Run sign_target_files_apks.py on taimen target_files.zip. Boot the
image after signing.
Change-Id: I91bd3dce4f45c1891c5e122212a699f4808618fa
(cherry picked from commit 0e06cb0a8b)
common.UnzipTemp() calls `unzip` to do the unzipping, which will
complain if there's non-existent names in the given list. Prior to this
CL, callers had to do the work to remove non-existent entries. This CL
filters out the given patterns in common.UnzipTemp()/common.UnzipToDir()
to make callers' works easier.
Bug: 128848294
Test: `m dist` with aosp_taimen-userdebug (which calls
ota_from_target_files.py on a target_files.zip that doesn't
contain RADIO/*).
Test: `python -m unittest test_common.CommonZipTest`
Change-Id: I5e741c27ea8d0b8126c398a7e1b56a8deb4a3d7f
OTA tools should pick up the avbtool, as listed in dict['avb_avbtool'],
from the current PATH (plus bin/ under the dir specified via `--path`),
the same way as handling all other host tools.
Test: `m dist`
Change-Id: I3eb4d2c61979b03d9c23b2403d9a38cf052d87ea
sign_target_files_apks script looks for the signapk.jar inside the out dir.
If the our dir is set to a different directory via OUT_DIR_COMMON_BASE the script does not work properly.
From now script checks if the OUT_DIR_COMMON_BASE is set, then searches the jar in the proper path.
If OUT_DIR_COMMON_BASE is unset, searches in "out" like it did before.
Test: Build with OUT_DIR_COMMON_BASE set and unset and verify that sign_target_files_apks works in both cases
Change-Id: I9218b98ff79526184f8353705640193405afac9e
This change adds another utility function to common.py: UnzipToDir, which is
generally useful. Refactor merge_target_files.py to use it, and also refactor
other uses in common.py to use it.
Test: ota_from_target_files.py, validate_target_files.py, test_common.py
Bug: 124464492
Change-Id: Ia571070bceb7d3c8002304836bdf688485bf0dd9
Then refactor the code in merge_target_files.py to adapt to this semantic
change. This makes the code more consistent with existing releasetools code,
and it's easier to follow.
Test: Failure cases (verify exception), success cases (merged target generated)
Bug: 124521133
Change-Id: I56f04e360d8ff8ffcd6245359cdeb79f4565a9c4
This script takes as input two partial target files (one contains system bits,
and the other contains non-system, or other, bits). The script merges the
contents of the two partial target files packages to produce a complete target
files package.
Bug: 123430711
Test: Build two partial target files, merge, compare with full target files.
Test: Validate merged target files via validate_target_files.py.
Change-Id: Ic24acf43b86fc703fb4c970688b006291a1861f8
- Add hashtree_info to EmptyImage so that BlockDifference.Compute()
can accept EmptyImage() as target image, which is the case when
a partition is removed.
- BlockDifference also checks source_info_dict to determine
whether a partition is dynamic. When a partition is removed,
its name does not appear in target_info_dict.
- Add tests to ensure DynamicPartitionDifference() still works.
Test: DynamicPartitionDifferenceTest
Test: test_blockimgdiff
Change-Id: Iadb1db075f5dc344db6d5ade358c83b01231e443
Bug: 111136242
Test: When BOARD_PREBUILT_DTB_DIR is set correctly,
generated $OUT/boot.img contains the DTB image.
Change-Id: I282e31b04cc60383377b9e9b54f8fe64a8140242
- When removing a partition, BlockDifference() object
will have tgt=EmptyImage(). Fix the asserts accordingly.
Also, BlockDifference object now allow tgt=None case.
- When adding a partition, BlockDifference() object
will have src=None. Fix the asserts accordingly.
Also, add unit tests to DynamicPartitionsDifference.
Test: create incremental OTA
Test: test_common.DynamicPartitionsDifferenceTest
Bug: 111801737
Change-Id: I3a35378ecf93111b8f44545cff6ae9696b6b4851
Test: sideload full OTA on cuttlefish
Test: sideload incremental OTA on cuttlefish (that grows
system, shrinks vendor, and move vendor to group foo)
Bug: 111801737
Change-Id: Ie8a267a90b4df9e9e0a2fbcc1b582ab2e353df52
Setting 'uses_shared_blocks' indicates that the block list for a given
file is incomplete, as some of the blocks are "owned" by others. As a
result, such a file will be skipped from imgdiff'ing.
This CL makes a copy of the original block list before removing the
shared blocks. It uses the original RangeSet as the value of
extra['uses_shared_blocks']. validate_target_files.py will try to read
the file as in the original RangeSet, unless the original list is also
incomplete or has non-monotonic ranges.
Test: Run validate_target_files on a target that uses
`BOARD_EXT4_SHARE_DUP_BLOCKS := true`.
Change-Id: I259e871ecc249ba0c14b5796bef413185a1b8242
Converts the following files to Python logging.
add_img_to_target_files.py
blockimgdiff.py
build_image.py
check_ota_package_signature.py
common.py
img_from_target_files.py
make_recovery_patch.py
ota_from_target_files.py
sparse_img.py
verity_utils.py
This separates logging outputs from normal outputs, and allows easier
switching between different verbosity levels. It also supports adjusting
logging config via environment variable (LOGGING_CONFIG).
Test: `m dist`
Test: `python -m unittest discover build/make/tools/releasetools`
Change-Id: Idfc3f7aff83e03f41f3c9b5067d64fd595d5192d
This prepares for upcoming changes that refactor verity-related
functions into classes.
This CL makes minimal changes to the moved functions, by creating a new
BuildVerityImageError class in verity_utils.py replacing the former
BuildImageError that's specific to build_image.py. As part of the
change, it also moves the tests for AVBCalcMinPartitionSize into
test_verity_utils.py.
Test: python -m unittest test_verity_utils
Test: `m dist` with aosp_marlin-userdebug (Verified Boot 1.0)
Test: `m dist` with aosp_taimen-userdebug (Verified Boot 2.0)
Change-Id: I19b52714d8980705ea1f9484ac03eb0af9483240
build_image.RunCommand and common.Run are essentially doing the same
work, but with different interfaces and return values. This CL
consolidates them by moving build_image.RunCommand into common.py as
common.RunAndCheckOutput, with the former calling common.Run internally.
common.RunAndCheckOutput checks the exit code: it returns the output
string on success, or raises common.ExternalError otherwise.
Test: `m dist` with aosp_marlin-userdebug
Test: python -m unittest test_build_image
Test: python -m unittest test_common
Change-Id: I3cd0e9301c576572194618eba8dc00db06c3c35f
stdout and stderr will default to subprocess.PIPE and subprocess.STDOUT
respectively (which is the expected behavior from most of the existing
callers), unless caller specifies any of them.
Test: `m dist`
Test: python -m unittest \
test_common \
test_add_img_to_target_files \
test_ota_from_target_files \
test_validate_target_files
Change-Id: I43b3f08edfa8a9bcfe54baf9848dc705c048e327
Generate the transfer command "compute_hash_tree" for incremental
updates of the non-A/B devices that enable verified boot 1.0
Other changes include:
i. factor out verity_utils to use both in build_image and blockimgdiff
ii. add a new flag 'hashtree_info_generator' in sparse_image to generate
the hashtree information.
Bug: 25170618
Test: generate a package for aosp_angler; and run simulator
Change-Id: I4d4d7a3e41dc3da810d2cbf8988e85d187c9ab0e
Reads the fingerprint of a partition from the build props and embeds it
into the care_map. Later, the update_verifier will compare the
fingerprint before it performs the verification.
Bug: 114778109
Test: generate care_map for sailfish and new pixel
Change-Id: I06afa01bf6dd3de4456a08ee7960954facc775fb
Bug: 112007947
Test: `m dist` with aosp_taimen-userdebug.
Test: Set up a target that uses chained VBMeta images of
`vbmeta_mainline` and `vbmeta_vendor`. `m dist` and check the
build log, as well as outputs from
`avbtool info_image --image vbmeta.img`,
`avbtool info_image --image vbmeta_mainline.img`,
`avbtool info_image --image vbmeta_vendor.img`.
Change-Id: Ib1d4e97f583b65245703eae15d211adcd9e83741
It used to accept a non-None input_dir to indicate the need to
re-generate images (called from add_img_to_target_files.py). During that
flow, both of the two parameters redundantly refer to the same input
dir. This CL replaces the second parameter with a bool value instead.
Test: `python -m unittest test_common`
Test: `m dist` with aosp_taimen-userdebug
Test: `zip -d aosp_taimen-target_files-eng.zip IMAGES/\* &&
add_img_to_target_files.py -a aosp_taimen-target_files-eng.zip`
Change-Id: I0a5a164366acb116407f94bb350872a3a0b207d1
Support a first stage ramdisk which will include the targets placed
into TARGET_RAMDISK_OUT. This replaces the existing ramdisk on
existing devices.
All system images are now built to be mounted as the root dir.
Devices with a first stage ramdisk will switch root to the system
partition.
BOARD_BUILD_SYSTEM_ROOT_IMAGE remains and is used to specify if the
system partition is going to be directly used as rootfs without the
ramdisk.
Bug: 79173823
Bug: 79758715
Test: hikey boots, sailfish boots
Test: OTA walleye from P to master
Change-Id: Idbb2dccc6340b0235a4bef03e11e420a9ed154b6
The change in [1] moved the recovery etc files from /etc to /system/etc.
However, we may use the latest OTA tools to build incremental OTAs for
old target_files zips. This CL adds a workaround to look at both of the
old and new locations.
[1] commit 696bb33676
Bug: 113073663
Test: Build a previously failing incremental OTA for taimen.
Change-Id: Ie07aa1713e616d523838b1260a992b20f5a11612
They used to be installed under recovery/root/etc. This CL moves the
files to the new location and creates a symlink from /etc to /system/etc
(done by the rule in system/core/rootdir). This gives similar layout
between normal boot and recovery, and allows installing prebuilt_etc
files with Soong (`recovery_available: true`).
As part of the change, we no longer need the whitelisting rule for
mke2fs.conf.
Bug: 112780007
Test: Build with other changes in the topic (aosp_taimen-userdebug).
Check the generated files under recovery (/etc being a symlink to
/system/etc).
Test: Boot into recovery. Verify basic functionalities (`adb shell` and
`adb sideload`, factory reset).
Test: `build/soong/build_test.bash --dist`
Change-Id: Ibb6dea6f179a339f0c2d0fd8ba05ec0085b79a12
Attempting to reduce the number of different spellings we have for
"product services" partition in the codebase.
Bug: 112431447
Test: m
Change-Id: I6debd3f5f5abc4bd1e22f2ef11c5fe131915976c
Merged-In: I6debd3f5f5abc4bd1e22f2ef11c5fe131915976c
The usage of datetime.fromtimestamp previously resulted in the build or
signing machine's local timezone affecting the Unix timestamp ultimately
applied to images generated by add_img_to_target_files. The go/ab build
outputs would use 2009-01-01 00:00 UTC, for example, but local builds
and the release signed images (generated through go/ab-sign) would use
2009-01-01 00:00 PST. This change makes the timestamps always use UTC.
Bug: 80600931
Bug: 80093599
Test: 'm -j droid dist' and verified timestamps in resulting
target_files zip.
Change-Id: Ic2a19591519850c249f78254e1464aa6839bfc6c
For example, verify a target_files.zip that has system AVB-signed as a
chained partition.
$ build/make/tools/releasetools/validate_target_files.py \
signed-target_files-4904652.zip \
--verity_key verifiedboot_pub.pem \
--avb_system_key_path system_pub.pem
Note that verifiedboot_pub.pem should be the key (either public or
private) to verify vbmeta.img, and 'system_pub.pem' should be the key
(either public or private) for the chained partition of system.
testdata/testkey.key is the private key converted from
testdata/testkey.pk8 for testing purpose (`openssl pkcs8 -in
testdata/testkey.pk8 -inform DER -out testdata/testkey.key -nocrypt`).
Bug: 63706333
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Test: `m dist` on aosp_walleye-userdebug; Run validate_target_files.py
on the generated target_files.zip.
Test: Set up walleye with chained system partition; `m dist`; Run
validate_target_files.py on the generated target_files.zip.
Change-Id: I38517ab39baf8a5bc1a6062fab2fe229b68e897d
This property indicates the path to the root directory (i.e.
TARGET_ROOT_OUT at build time, or ROOT/ in a target_files.zip).
build_image.py will only use this property when building a target using
system_root_image. Under such a configuration, files under this dir
never land into a ramdisk, but as part of system.img. This CL renames
the property name to avoid confusion.
Note that with this change releasetools keeps working with older
target_files zips that contain 'ramdisk_dir'. Because common.py doesn't
read the value of that property, but sets it to ROOT/ directly.
Test: `m dist` on aosp_marlin-userdebug.
Test: `m dist` on aosp_angler-userdebug.
Test: Run add_img_to_target_files.py with a new target_files.zip and an
old target_files.zip respectively.
Change-Id: Ie9e19dd17d8c2abb4c4a97449e436774d5dd2a36
Clean up the following scripts.
blockimgdiff.py
common.py
edify_generator.py
img_from_target_files.py
ota_from_target_files.py
Seems we don't have a way to force pylint-ing the scripts using
per-directory pylintrc files (and we don't have pylint tool in AOSP
repo), per
https://android.googlesource.com/platform/tools/repohooks/#todo_limitations.
Test: `m dist`
Test: pylint --rcfile=pylintrc <script.py>
Change-Id: Ia6fd1ddc86f4d84c68e500f225d4a89d0fea8ec7
This CL is largely an adaptation of Change-Id
I774e6a38003734421591e51bed103802ff84f432
It adds the following variables:
- BOARD_AVB_PRODUCT_SERVICES_KEY_PATH
- BOARD_AVB_PRODUCT_SERVICES_ALGORITHM
- BOARD_AVB_PRODUCT_SERVICES_ROLLBACK_INDEX_LOCATION
- BOARD_PRODUCT_SERVICESIMAGE_FILE_SYSTEM_TYPE
- BOARD_PRODUCT_SERVICESIMAGE_EXTFS_INODE_COUNT
- BOARD_PRODUCT_SERVICESIMAGE_EXTFS_RSV_PCT
- BOARD_PRODUCT_SERVICESIMAGE_PARTITION_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_JOURNAL_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_BLOCK_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_COMPRESSOR
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_COMPRESSOR_OPT
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_DISABLE_4K_ALIGN
- BOARD_PREBUILT_PRODUCT_SERVICESIMAGE
- BOARD_USES_PRODUCT_SERVICESIMAGE
- LOCAL_PRODUCT_SERVICES_MODULE
- PRODUCT_PRODUCT_SERVICES_BASE_FS_PATH
- PRODUCT_PRODUCT_SERVICES_VERITY_PARTITION
- PRODUCT_PRODUCT_SERVICES_PROPERTIES
- TARGET_COPY_OUT_PRODUCT_SERVICES
- TARGET_OUT_PRODUCT_SERVICES
- TARGET_OUT_PRODUCT_SERVICES_*
Bug: 80741439
Test: Successfully built product-services.img with one module in it, and flashed
on device. Also successfully built image with /system/product-services directory
and no /product-services partition.
Change-Id: I5d229f6ac729ea6df9ff1f14cee2e28972cd9b4d
This writes /system/bin/install-recovery.sh with the updated argument
formats. Requires matching changes to bootable/recovery/applypatch.
Bug: 110106408
Test: python -m unittest test_common.InstallRecoveryScriptFormatTest
Test: See additional tests in the other CL.
Change-Id: I2dec6f064819c01b5bb34c3d6fd0e247a5f48f08
With system_root_image, e2fsdroid writes file map with extra leading
slashes in filenames (e.g. "//system/framework/am.jar"). This breaks the
detection of files with incomplete ranges, and thus fails the patch
generation. This CL addresses the issue by stripping out leading
slashes.
Additionally, non-/system files (e.g "//sbin/charger") are not packed
under SYSTEM/ in a target_files.zip, despite being part of system.img.
We need to look for these files under ROOT/ instead.
This CL also asserts the availability of all files listed on a file map,
to avoid silently missing other edge cases.
Bug: 80380658
Test: python -m unittest test_common
Test: Successfully generated an incremental for a target using
system_root_image that was previously failing.
Change-Id: I62a2460e882f3930e99add4d2b44291edf7a51a0
This reverts commit 2a40cc6996.
Reason for revert: Broke a bunch of builds, platform/system/tools/bpt must be missing from more manifests.
Change-Id: I4c367a1fec5c62c247544227e9d3b596227e31da
For Android Things targets (or any other target which has
BOARD_BPT_INPUT_FILES defined), add_img_to_target_files will generate a
partition-table.img using bpttool. It also adds the final combined .bpt
definition file into target-files in IMAGES/partition-table.bpt.
When we're signing using sign_target_files_apks, add_img_to_target_files
needs to regenerate the partition table, but META/misc_info.txt still
contains the original list of bpt input files from the build that aren't
available. This change extracts the final bpt from the input
target-files, adds it to META/ in the output target-files, and then
updates the board_bpt_input_files property to point to it.
Bug: 72837107
Test: Local sign_target_files_apks run of locally built target-files
Change-Id: Id79125208f31c78b1ac2079172f9c91a9203849b
This CL separates the options into three groups (excluding the global
options provided via common.py).
- Non-A/B OTA specific options;
- A/B OTA specific options;
- Common options that apply to both.
It mostly reshuffles the lines, with minor change to "--verify" that
removes the obselete remounting behavior (which doesn't apply to
block-based OTA).
Hopefully this makes the expected behavior of some options less
confusing.
Test: `build/make/tools/releasetools/ota_from_target_files.py`
Change-Id: I194ea52c4f7d6a3c7f34531abbcf3fdc7b7f4fa8
Non-A/B devices need to include the DTBO image
within the recovery partition to be self-sufficient
and prevent OTA failures.
Test: Ran 'm dist' and verified that the DTBO image
was included in recovery.img using unpack_bootimg.
Also ran 'make' and verified that the DTBO image was
included in recovery.img using unpack_bootimg.
Also tested that aosp_angler-userdebug could
boot into recovery(including a fake dtbo.img
in recovery image).
Bug: 74763691
Change-Id: I38c9c395c95d21f4da42cfa646063bd4416f6bd8
The following is a buggy pattern that won't capture anything into err.
The issue is benign, since a failed run would be eventually captured by
a subsequent check.
p = Run(["aapt", ...], stdout=subprocess.PIPE)
output, err = p.communicate()
if err:
raise ...
This CL changes the error detection to be based on the return code from
aapt. It also adds some sanity test to ensure the call to aapt works.
The test app is built from AOSP com.android.cts.ctsshim (chosen mostly
because of its small size).
Test: python -m unittest test_common
Change-Id: I337f141bd0fc5f0801dfc628c601b88b7640789c
For the functions in common.py, capture the stdout/stderr outputs when
shelling out to external tools. Dump the outputs on errors.
Bug: 76123422
Test: Inject errors to signapk.jar / brotli / unzip. Check the outputs.
Change-Id: Ib2d4272528b83a50bb727788cf8a5211e2ddade8
