We want the generated images being identical for the same source files.
Currently the generated ext4 image (either from make_ext4fs or mke2fs)
is reproducible, but the AVB footer added by avbtool contain changes
because of the random salt being used.
This CL changes the avbtool invocation to specify "--salt <hexstring>"
(already supported by avbtool) to use reproducible salt that's computed
based on fingerprints (or thumbprints if applicable).
Bug: 67023482
Test: Regenerate images from the same source as follows:
Use a target_files.zip from an AVB-enabled target.
$ zip -d target_files.zip IMAGES/\*
$ ./build/make/tools/releasetools/add_img_to_target_files.py \
-v target_files.zip
Repeat the above commands and compare the generated images.
Change-Id: Id9db17ae0132ca3a820b4be5a5ef06ca3fef71ed
We have added support in mkuserimg_mke2fs.sh that allows specifying
filesystem UUID and hash seed UUID. This CL generates and passes the
UUIDs based on the image name and build fingerprint. This way we can
rebuild and get identical images.
Note that this only applies to images generated with `m dist` and with
signing tools (sign_target_files_apks.py and
add_img_to_target_files.py). Images under $OUT (built with `make`) are
not affected.
Bug: 64994964
Test: lunch marlin-userdebug; run `m dist` twice (after deleting the
intermediate files under $OUT/obj/PACKAGING), and compare the
generated images.
Change-Id: I41cf4e5869582bb930af2f35a8e9c79bff43b2a2
vendor.img could be built from the source - in which case we have
unpacked files under VENDOR/ - or dropped in as a prebuilt binary blob.
We should consider either of them as target having a vendor partition.
Because we need to add its AVB hashtree info into vbmeta.img if target
is using AVB. Otherwise libfs_mgr would refuse to mount this
"AVB-enabled" vendor.img.
For targets not using AVB, this change is no-op.
Bug: 65462819
Test: Having vendor.img as prebuilt, `make dist`. Check that the
generated vbmeta.img contains the info from vendor.img.
Test: Build, flash and boot the above image.zip.
Change-Id: Iaeb30e2059cb33fb39f23e5ffd28f338d00ccbfc
Make the build log less verbose; also print the result for
AdjustPartitionSizeForVerity().
Bug: 65292710
Test: Run add_img_to_target_files with and without "-v" and check
the logs.
Change-Id: Icdc86197ce69db036658ea9e77cc4ebe1750463b
We pack _all_ the APK certs info into META/apkcerts.txt at build time,
including the ones that are not installed for the target. When
detecting for compressed APKs, we should only count the ones that are
actually installed.
Bug: 65498015
Test: `check_target_files_signatures.py bullhead-target_files.zip`
Change-Id: I2f32d4667be0efbfd5f9365dde819f009572533d
When creating ext4 images with mke2fs, it may skip allocating some
blocks if they contain all zeros. As a result, there could be less
blocks listed in the block map than the actual file length.
For example, for a file with a length of 112200-byte (27+ blocks),
the listed blocks in block.map could be '43665-43688' (24 blocks).
Because some all-zero blocks are not taking actual space.
The generated ext4 images are perfectly valid - kernel will figure out
that data block is not allocated and writes all zeros into user buffer.
However, we can't fully reconstruct a file from its block list in our
Python script. Ideally this can be avoided by mounting or parsing an
ext4 image directly, which is yet to be supported in our script.
This CL skips checking for such files to avoid failing
validate_target_files.py.
Bug: 65213616
Test: validate_target_files.py passes on targets with mke2fs generated
images (e.g. marlin).
Change-Id: Id9cc59e345b9283844044ef94ceb5702f0ca0526
(cherry picked from commit b418c30e3a)
When creating ext4 images with mke2fs, it may skip allocating some
blocks if they contain all zeros. As a result, there could be less
blocks listed in the block map than the actual file length.
For example, for a file with a length of 112200-byte (27+ blocks),
the listed blocks in block.map could be '43665-43688' (24 blocks).
Because some all-zero blocks are not taking actual space.
The generated ext4 images are perfectly valid - kernel will figure out
that data block is not allocated and writes all zeros into user buffer.
However, we can't fully reconstruct a file from its block list in our
Python script. Ideally this can be avoided by mounting or parsing an
ext4 image directly, which is yet to be supported in our script.
This CL skips checking for such files to avoid failing
validate_target_files.py.
Bug: 65213616
Test: validate_target_files.py passes on targets with mke2fs generated
images (e.g. marlin).
Change-Id: Id9cc59e345b9283844044ef94ceb5702f0ca0526
$ PYTHONPATH=$PYTHONPATH:system/update_engine/scripts \
./build/make/tools/releasetools/check_ota_package_signature.py \
build/target/product/security/testkey.x509.pem \
out/dist/aosp_marlin-ota-eng.zip
Package: out/dist/aosp_marlin-ota-eng.zip
Certificate: build/target/product/security/testkey.x509.pem
...
Whole package signature VERIFIED
Verifying A/B OTA payload signatures...
...
Payload signatures VERIFIED
Bug: 65261072
Test: Signed a package and its payload with the right keys; ran the
command above.
Test: Signed the payload with a different key; ran the command above and
observed the reported verification failure.
Change-Id: If626ecb327a9826cd0956eef94914c939068a7d1
Commit 21803d3574 only added compatibility
metadata for A/B OTA packages, because by that time we didn't use
'ro.treble.enabled' property to determine if a target was Treble-enabled.
This CL uses 'ro.treble.enabled' to guard the packing of compatibility
metadata for both of A/B and non-A/B OTA packages. It also switches to
checking the system/vendor fingerprints to determine if there's an
update to the partition (previously it was computing the SHA-1 of the
images, which may have unintentionally changed due to issues that give
non-repetitive builds).
Bug: 64339310
Test: Generate OTA packages (full and incremental) on Treble-enabled
targets (sailfish, as well as a non-A/B angler target with
Treble-enabled locally); check that the compatibility.zip entry
exists.
Test: Generate OTA packages on Treble-unenabled non-A/B target (angler);
check that the compatibility.zip entry doesn't exist.
Test: Generate OTA packages on Treble-unenabled A/B target; check that
the compatibility.zip entry doesn't exist.
Change-Id: I2a1fcf612439d849ba8ccea217a0faf5d5ba8e14
This entry only exists in non-A/B full OTA path. However, this file has
never been loaded by recovery - recovery was only reading the one from
the recovery image (which also has been removed by a recent change in
[1]).
[1]: commit e35926e1aff2e6b9b54656bd59c8178e295a1b7e in
platform/bootable/recovery.
Bug: 63538434
Test: `m dist` on angler. file_contexts.bin is gone from full OTA
package. Sideloading the generated full OTA on angler.
Change-Id: Iec2e2d36599fa1ed499516a3e74f06cd36ade494
Compressed APKs can be identified by a "compressed=<ext>" entry in
the apkcerts.txt file. When we encounter such an entry, we need to
decompress the file to a temporary location before we process its
certs. When we're signing, we should also recompress the package
after it's signed.
Bug: 64531948
Test: ./build/tools/releasetools/check_target_files_signatures.py
Test: ./build/tools/releasetools/sign_target_files_apks.py
Test: compared signed output before / after this change, verify that
it's bitwise identical when no compressed APKs are present.
Change-Id: Id32e52f9c11023955330c113117daaf6b73bd8c2
In BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS, if we have defined
"--include_descriptors_from_image" with an image file whose path points
to source tree, add_img_to_target_files.py or sign_target_files_apks.py
may fail to find the file. Because these scripts may run without a
source tree, by taking target_files.zip as the only input.
This CL scans additional locations in the input target_files.zip to find
those missing files in avb_vbmeta_args. As long as the files are included
in the target_files.zip, they get a second chance to be found.
Bug: 63910867
Test: As follows:
1. Setup BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS with a local file path;
2. Remove the local file;
3. sign_target_files_apks.py fails without this CL;
4. sign_target_files_apks.py works.
Change-Id: I3c58f80a5535db02b74cfe40d0c0beff72587cf8
(cherry picked from commit 1dc5d47653)
In BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS, if we have defined
"--include_descriptors_from_image" with an image file whose path points
to source tree, add_img_to_target_files.py or sign_target_files_apks.py
may fail to find the file. Because these scripts may run without a
source tree, by taking target_files.zip as the only input.
This CL scans additional locations in the input target_files.zip to find
those missing files in avb_vbmeta_args. As long as the files are included
in the target_files.zip, they get a second chance to be found.
Bug: 63910867
Test: As follows:
1. Setup BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS with a local file path;
2. Remove the local file;
3. sign_target_files_apks.py fails without this CL;
4. sign_target_files_apks.py works.
Change-Id: I3c58f80a5535db02b74cfe40d0c0beff72587cf8
Brotli has a better compression ratio than deflation. So for non-AB
full OTA, we can compress the new.dat with brotli first and store it
in the zip package.
This increase the OTA generation time by ~5 minutes for a full OTA
(measured based on 2.0G system.new.dat)
Bug: 34220646
Test: Generate a full OTA for bullhead
Change-Id: I9c0550af8eafcfa68711f74290c8e2d05a96648f
Accidentally broken by the cherry-pick in commit
f829b40c48 - the original CL in oc-dev
doesn't require the 'import stat' line.
Bug: 63629728
Test: `pylint --rcfile=pylintrc sign_target_files_apks.py`
Test: Successfully sign a build with sign_target_files_apks.py.
Change-Id: I94be613fb2219597148c4339ac33fc93d0cdae47
system_avb_hashtree_enable has been renamed to
avb_system_hashtree_enable in commit
3e599ead66. 'care_map.txt' is missing due
to the change.
Bug: 63142730
Test: `m dist` and check IMAGES/care_map.txt exists in target_files zip.
Change-Id: I60c269b41df844b50353d357bf67c20f15548642
(cherry picked from commit 3f72176da0)
system_avb_hashtree_enable has been renamed to
avb_system_hashtree_enable in commit
3e599ead66. 'care_map.txt' is missing due
to the change.
Bug: 63142730
Test: `m dist` and check IMAGES/care_map.txt exists in target_files zip.
Change-Id: I60c269b41df844b50353d357bf67c20f15548642
Currently we're writing META/misc_info.txt to the new TF.zip during
ReplaceVerityPrivateKey(). We should delay that until we have replaced
everything in need. Otherwise we won't be able to replace/overwrite
that zip entry (unless `zip -d` first).
This CL also cleans up the return value of ReplaceVerityPublicKey() and
ReplaceVerityKeyId(), since the caller no longer needs the values.
Test: sign_target_files_apks.py and check the generated signed TF.zip.
Change-Id: I9fbd7182247728281519e5e3971557f6b018ad65
(cherry picked from commit 46a5999a02)
Radio images are added to INSTALLED_RADIOIMAGE_TARGET, which by default
will be packed under RADIO/ in a target_files zip. This CL introduces
BOARD_PACK_RADIOIMAGES that allows additionally copying them into
IMAGES/, which will then be included into <product>-img.zip.
Bug: 62195105
Test: Define BOARD_PACK_RADIOIMAGES and `m dist`. Check the generated
target_files zip and <product>-img.zip.
Change-Id: I3deafd2dfecd1d4dbfdfc2d002fc40ef22fb60ea
This patch tries to fix the problem where the default properties need
to go with the system image especially on non-AB devices where
/default.prop is on the ramdisk image. A symlink is created at
/default.prop for backward compatibility.
Bug: 37815285
Test: Tested with ag/2416542. Booted pixel phones, checked the location
of prop.default, verified the symlink, checked a few properties
(via adb shell getprop) and manually tested a few apps (Camera,
Maps etc).
sign_target_files_apks.py was tested with:
sign_target_files_apks -o -e DynamiteLoader.apk= -e DynamiteModulesA.apk= \
-e DynamiteModulesB.apk= -e DynamiteModulesC.apk= -e DynamiteModulesD.apk= \
-e GoogleCertificates.apk= out/dist/*-target_files-*.zip signed-target_files.zip
Booted to recovery and ran 'adb sideload' successfully.
Change-Id: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
Merged-In: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
(cherry-picked from 4fbbe4578bb10d54292d9b243edf4999fddf1c93)
This patch tries to fix the problem where the default properties need
to go with the system image especially on non-AB devices where
/default.prop is on the ramdisk image. A symlink is created at
/default.prop for backward compatibility.
Bug: 37815285
Test: Tested with ag/2416542. Booted pixel phones, checked the location
of prop.default, verified the symlink, checked a few properties
(via adb shell getprop) and manually tested a few apps (Camera,
Maps etc).
sign_target_files_apks.py was tested with:
sign_target_files_apks -o -e DynamiteLoader.apk= -e DynamiteModulesA.apk= \
-e DynamiteModulesB.apk= -e DynamiteModulesC.apk= -e DynamiteModulesD.apk= \
-e GoogleCertificates.apk= out/dist/*-target_files-*.zip signed-target_files.zip
Booted to recovery and ran 'adb sideload' successfully.
Change-Id: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
