This verifies the init rc files in the merged result.
Bug: 163089173
Test: test_common.py
Test: Run merge_target_files.py to merge two target-files packages where
one has init_rc errors. Observe script failure.
Test: Run merge_target_files.py on two good target-files packages,
observe no failure.
Change-Id: I86c8e5a2bc07c2c1896ac40afd32bc1d055447ee
This check is used when merging target files to ensure that a merged
build does not contain any APKs that share UID across builds.
Bug: 171431774
Test: test_common
Test: Use merge_target_files.py to merge two partial builds,
observe no failures for inputs without colliding APKs.
Test: Use merge_target_files.py to merge two partial builds,
observe failure for inputs that have an APK that shares a
UID across input partition groups.
Change-Id: I9dc57216882741ae46a99cfd7847f34702c75582
These changes were necessary to begin merging a new cross-release target
whose vendor half is frozen.
- MergeDynamicPartitionInfoDicts
- Filters combined fields to remove duplicates
- Merges `super_block_devices` as well as other keys that were not
previously used by other targets consuming this tool.
- Introduces --allow-duplicate-apkapex-keys. This gives a warning rather
than fatal error on duplicate apk/apex keys. This flag is needed for
targets that cannot update a frozen half.
- (Formats merge_target_files.py)
Bug: 170683837
Test: Use merge_target_files to merge an S+R build, and boot.
Change-Id: Id5f787e730de8f8ef697e1f2f29ac6514221e58d
When opening an zip file through zipfile.ZipFile(), python2 by default
disables zip64 support. To support update files >4GB, we manually add
allowZip64 to override the setting.
Test: generate && serve an OTA
Change-Id: I9645e963ced830cc2d3a4b72bc63b9369a1cefe8
This reverts commit b21e48b499.
In practice, some partners use the 'import' statement to override
the device fingerprint at runtime. The runtime fingerprint will
later add to the metadata of OTA package, so that the OTA server
can deliver the package to corresponding devices correctly.
This CL supports parsing a subset of import statement that the init
process recognizes. And we loose the restriction based on how the
dynamic fingerprint is used in practice. Right now, we only searches
for the override of brand, name and device. And the placeholder
format should be ${placeholder}, with its value supplied by the
script caller.
As part of the implementation, we generate all the possible
combinations of the input boot variables. And recalculate the
fingerprint for each of the combination. Though we load the
build.prop multiple times, the logic is easier to follow. Also,
it's more convenient to enhance the logic if we only want to
allow some of the boot variables combination later.
Bug: 152167826
Change-Id: I4a9fa35c7ac037ff1cf4f9a4bdff602beac3894b
Test: unittests pass
This reverts commit 6022545272.
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.
Bug: 152167826
Change-Id: I5c952cd2a976ba1a09ddc66d56c2b8b55a61986b
Test: unittests pass
In practice, some partners use the 'import' statement to override
the device fingerprint at runtime. The runtime fingerprint will
later add to the metadata of OTA package, so that the OTA server
can deliver the package to corresponding devices correctly.
This CL supports parsing a subset of import statement that the init
process recognizes. And we loose the restriction based on how the
dynamic fingerprint is used in practice. Right now, we only searches
for the override of brand, name and device. And the placeholder
format should be ${placeholder}, with its value supplied by the
script caller.
As part of the implementation, we generate all the possible
combinations of the input boot variables. And recalculate the
fingerprint for each of the combination. Though we load the
build.prop multiple times, the logic is easier to follow. Also,
it's more convenient to enhance the logic if we only want to
allow some of the boot variables combination later.
Bug: 152167826
Test: unittests pass
Change-Id: Iac093a40dc6f873c5e1858efa44cb2bd6082508a
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.
Bug: 152167826
Test: unittests pass
Change-Id: I2fe7e93a2f4de8e55f5f8051b000b96b5efdc85a
The oem_dicts in BuildInfo is only available to ota_from_target_files
when it's called with "--oem_settings" input. However, aosp/1135332
starts to use BuildInfo in add_img_to_target_files w/o oem_dicts. An
assert is triggered due to it when oem_fingerprint_properties is in the
info_dict.
This change skip _oem_props reading if oem_dicts is None to allow to
this case.
BUG: 155360923
BUG: 154171021
Test: Sign images with oem_fingerprint_properties.
Change-Id: I6a73feecd9567fd4d85f4eab3d4e11c5df28fe39
When loading build info from a previous version of Android, the set of
allowed property sources should match those available in that version.
In this particular case, the product_services partition was a valid
property source in Android 10.
Bug: 155053195
Test: ran unit tests from test_common.py
Test: generated an incremental OTA which previously failed
Change-Id: Ic0b0a112656533eca78dee31517deff7e3c8d7cc
The arguments transparency_log_servers and transparency_log_pub_keys
have been merged. Update the caller in the OTA script accordingly.
Also disable the test to contact aftl server until we have
a public server.
Bug: 153940575
Test: check the argument
Change-Id: If6a7e7d644884d395c75c2fcdfd6aa7c2380d851
Since we might use ToT release tools to sign a package
generated by an older build, we make the new
`partition=` tag optional. This also means we need to be
careful to use non-greedy regex matching.
Bug: 153133823
Test: python3 -m unittest
Test: input with and without the new `partition=` tag
Test: new test_ReadApkCerts_WithWithoutOptionalFields
Change-Id: Ic57efd34e745ad302ae17150c6f2318f0b4524cb
Right now we assert that the aftltool should exist since the one
in aosp won't work due to grpc dependencies.
Bug: 147870995
Test: build
Change-Id: Iabb2c375167572a965493a7648fdc1abe287af67
The otatools should talk to the aftl server and append the inclusion
proofs when building the vbmeta image. We should only do this during the
signing process when the network is always available.
Also the inclusion proof doesn't impact the final vbmeta image size on
coral, where the final size is 8192 after padding. This is below the
physical image of 65536.
Bug: 147870995
Test: unit tests pass, run sign_target_file_apks
Change-Id: If84c6bf5198c9b05f5e0c16ae6335971915f47e3
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.
Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
`zip -d out/dist/target_files.zip IMAGES/\*`
`add_img_to_target_files -a out/dist/target_files.zip`.
Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
the partial target-files.zip IMAGES are identical to the merged
target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
All callers of this function now always pass the same values, so this
change hardcodes those values within the function body.
Fix: 145008064
Test: python -m unittest test_common
Test: build & boot a merged target that uses DAP
Change-Id: I0051c5ba507983231825edfcaf349e574efa451a
It's a bug introduced in commit 1aeef725a7, which affects unittest only.
Bug: 130351427
Test: Use a lunch'd target. Run
`atest --host releasetools_test releasetools_py3_test`.
Change-Id: I7ff01a6af47d002e1203bd376d477b60d769cbd1
There is no change to module functionalities. Testcases are moved around
accordingly.
Bug: 134525174
Test: TreeHugger
Test: lunch a target; atest --host releasetools_test releasetools_py3_test
Change-Id: I7bc8f49cc239e7c6655fe5e375508f01c1743b94
The previous change in commit 22632cc82c
changed the behavior in loading file map. It now always puts a copy of
the input text in `extra` field. Update the tests accordingly.
Bug: 79951650
Test: Use a lunch'd target; `atest --host releasetools_test`
Change-Id: Iccf06c817c1305bf9946d7759c6f6f6af21fe85e
Bug: 137853921
Bug: 138671115
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Test: Ran 'merge_builds --build_vbmeta' for two devices, one with the
vbmeta struct on system.img and another with vbmeta_system.img. Flashed
the regenerated vbmeta.img files on devices, devices boot.
Change-Id: I8d7585c7af468be3d242d8aceeed6d27e6fc6d96
- Removes functions that can be replaced with one-line external methods
- Moves read_config_list to common alongside similar methods LoadDictionaryFrom*
- Runs pyformat on merge_target_files.py
Bug: 137853921
Test: python -m unittest test_merge_target_files
Test: Using merge_target_files.py to create a merged build, & booting.
Change-Id: I833c1086db41e1374057cc7447fc50d1915734a7
This helps to break the circular dependency between common and
blockimgdiff.
Bug: 32379627
Test: unit tests pass
Change-Id: I90b5ff34782acbfac86f36265bd96c207d898bf6
This allows a consistent logic in using the avbtool which could be
board-specific.
Test: `atest releasetools_test`
Test: Run sign_target_files_apks.py on a target_files.zip.
Change-Id: I8cd93b8e71146985734f85c31f4662f5e2e9534c
Python 2 and 3 behave differently when calling ZipFile.writestr() with
zinfo.external_attr being 0. Python 3 uses `0o600 << 16` as the value
for such a case (since
18ee29d0b8),
which seems to make more sense. Otherwise the entry will end up with
0o000 as the permission bits. This CL updates common.ZipWriteStr to
follow the logic in Python 3, in order to get consistent behavior
between using the two versions.
Bug: 131631303
Test: `python -m unittest test_common.CommonZipTest`
Test: `python3 -m unittest test_common.CommonZipTest`
Change-Id: If8429855d922ef1ad76591f703215a0ce5089f0f
Previously it was using regular dict.
Test: python -m unittest test_common.DynamicPartitionsDifferenceTest
Change-Id: If108a4512aeaf9d3c8775c030cad6e44342b9d3d
About half of the testcases rely on external tools (i.e. the ones in
`otatools.zip`, which are external to releasetools module, but still
built by Android). It's WAI as releasetools scripts are mostly for
gluing purpose.
However, the current support in Soong doesn't allow packing the helper
modules as part of the built releasetools_test. This CL adds a decorator
that allows declaring external dependencies in testcases, which will be
skipped while running in presubmit. It doesn't affect local invocation
of `atest releasetools_test`.
Fixes: 112080715
Test: `atest releasetools_test`
Test: TreeHugger; check that releasetools_test is invoked (and test
passes).
Change-Id: I8fdeb6549023cf5ddeb79d610c7c37cf9f13d3cc
The function used to be serving system and vendor partitions only (as
they were the only partitions using sparse image at the point). The code
itself doesn't rely on anything specific to system/vendor.
Test: python -m unittest test_common
Change-Id: Ia4ecdeedb262f3d9db082128eaf9bab299983333
common.UnzipTemp() calls `unzip` to do the unzipping, which will
complain if there's non-existent names in the given list. Prior to this
CL, callers had to do the work to remove non-existent entries. This CL
filters out the given patterns in common.UnzipTemp()/common.UnzipToDir()
to make callers' works easier.
Bug: 128848294
Test: `m dist` with aosp_taimen-userdebug (which calls
ota_from_target_files.py on a target_files.zip that doesn't
contain RADIO/*).
Test: `python -m unittest test_common.CommonZipTest`
Change-Id: I5e741c27ea8d0b8126c398a7e1b56a8deb4a3d7f
- Add hashtree_info to EmptyImage so that BlockDifference.Compute()
can accept EmptyImage() as target image, which is the case when
a partition is removed.
- BlockDifference also checks source_info_dict to determine
whether a partition is dynamic. When a partition is removed,
its name does not appear in target_info_dict.
- Add tests to ensure DynamicPartitionDifference() still works.
Test: DynamicPartitionDifferenceTest
Test: test_blockimgdiff
Change-Id: Iadb1db075f5dc344db6d5ade358c83b01231e443
- When removing a partition, BlockDifference() object
will have tgt=EmptyImage(). Fix the asserts accordingly.
Also, BlockDifference object now allow tgt=None case.
- When adding a partition, BlockDifference() object
will have src=None. Fix the asserts accordingly.
Also, add unit tests to DynamicPartitionsDifference.
Test: create incremental OTA
Test: test_common.DynamicPartitionsDifferenceTest
Bug: 111801737
Change-Id: I3a35378ecf93111b8f44545cff6ae9696b6b4851
All the releasetools unittests extend the common base class of
test_utils.ReleaseToolsTestCase. Define tearDown() in the base class to
do the clean-up works.
Test: `pylint --rcfile=pylintrc test_*.py`
Test: `python -m unittest discover .`
Change-Id: I51775d964ef032dcdf3bb89c55e1a31371cde708
build_image.RunCommand and common.Run are essentially doing the same
work, but with different interfaces and return values. This CL
consolidates them by moving build_image.RunCommand into common.py as
common.RunAndCheckOutput, with the former calling common.Run internally.
common.RunAndCheckOutput checks the exit code: it returns the output
string on success, or raises common.ExternalError otherwise.
Test: `m dist` with aosp_marlin-userdebug
Test: python -m unittest test_build_image
Test: python -m unittest test_common
Change-Id: I3cd0e9301c576572194618eba8dc00db06c3c35f
It used to accept a non-None input_dir to indicate the need to
re-generate images (called from add_img_to_target_files.py). During that
flow, both of the two parameters redundantly refer to the same input
dir. This CL replaces the second parameter with a bool value instead.
Test: `python -m unittest test_common`
Test: `m dist` with aosp_taimen-userdebug
Test: `zip -d aosp_taimen-target_files-eng.zip IMAGES/\* &&
add_img_to_target_files.py -a aosp_taimen-target_files-eng.zip`
Change-Id: I0a5a164366acb116407f94bb350872a3a0b207d1
For example, verify a target_files.zip that has system AVB-signed as a
chained partition.
$ build/make/tools/releasetools/validate_target_files.py \
signed-target_files-4904652.zip \
--verity_key verifiedboot_pub.pem \
--avb_system_key_path system_pub.pem
Note that verifiedboot_pub.pem should be the key (either public or
private) to verify vbmeta.img, and 'system_pub.pem' should be the key
(either public or private) for the chained partition of system.
testdata/testkey.key is the private key converted from
testdata/testkey.pk8 for testing purpose (`openssl pkcs8 -in
testdata/testkey.pk8 -inform DER -out testdata/testkey.key -nocrypt`).
Bug: 63706333
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Test: `m dist` on aosp_walleye-userdebug; Run validate_target_files.py
on the generated target_files.zip.
Test: Set up walleye with chained system partition; `m dist`; Run
validate_target_files.py on the generated target_files.zip.
Change-Id: I38517ab39baf8a5bc1a6062fab2fe229b68e897d
With system_root_image, e2fsdroid writes file map with extra leading
slashes in filenames (e.g. "//system/framework/am.jar"). This breaks the
detection of files with incomplete ranges, and thus fails the patch
generation. This CL addresses the issue by stripping out leading
slashes.
Additionally, non-/system files (e.g "//sbin/charger") are not packed
under SYSTEM/ in a target_files.zip, despite being part of system.img.
We need to look for these files under ROOT/ instead.
This CL also asserts the availability of all files listed on a file map,
to avoid silently missing other edge cases.
Bug: 80380658
Test: python -m unittest test_common
Test: Successfully generated an incremental for a target using
system_root_image that was previously failing.
Change-Id: I62a2460e882f3930e99add4d2b44291edf7a51a0
The following is a buggy pattern that won't capture anything into err.
The issue is benign, since a failed run would be eventually captured by
a subsequent check.
p = Run(["aapt", ...], stdout=subprocess.PIPE)
output, err = p.communicate()
if err:
raise ...
This CL changes the error detection to be based on the return code from
aapt. It also adds some sanity test to ensure the call to aapt works.
The test app is built from AOSP com.android.cts.ctsshim (chosen mostly
because of its small size).
Test: python -m unittest test_common
Change-Id: I337f141bd0fc5f0801dfc628c601b88b7640789c
In addition to the unzipping work, common.UnzipTemp() kindly bundles an
open ZipFile object as part of the return value. It doesn't look very
helpful to the callers though. It also looks less obvious that the
caller needs to properly close the handle (missing the close here is
benign though). This CL just removes the ZipFile object out of the
return value, and leaves the work to callers.
Test: `m dist` on both of A/B and non-A/B target.
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_common
Test: python -m unittest test_ota_from_target_files
Test: Check the callers to common.UnzipTemp() in code search.
Change-Id: Id47da3fd42a0e76d6ae8851f05780db319ee48cf
Also minor clean up to the function, to suppress the following lint
warning.
R:1813, 2: Redefinition of cert type from list to unicode (redefined-variable-type)
Test: python -m unittest -v test_common.CommonApkUtilsTest
Change-Id: Ib4b1dfa8b19e505fc70684b648efc36171c73bbf
