The man page for the password file (man 5 passwd) does not mention any
provisions for comment line other than comment field
Bug: 109748241
Test: Build and check generated file
Change-Id: I2912d0844cf3f077184129d1a0121b50b09ef1e3
We used to write a fake post-timestamp into package metadata for
"--override_timestamp", which allows a package to be pushed based on
the fake timestamp. This CL stops doing that by using the designated
"ota-downgrade=yes" instead, but keeping the "--override_timestamp" flag
for backward compatibility.
Now both "--override_timestamp" and "--downgrade" allow pushing packages
with downgrade timestamps. The only differenece is that "--downgrade"
enforces a data wiping, because it's intended a real downgrade (based on
Android version or security patch level). "--override_timestamp" serves
the path of a legit "upgrade", but unfortunately with build timestamps
in reverse order (e.g. the two builds are cut from different branches).
With this CL, we write "post-timestamp" to all packages.
Bug: 77811049
Test: Generate an incremental pakcage with "--override_timestamp".
Test: Generate an incremental pakcage with "--downgrade".
Test: python -m unittest test_ota_from_target_files
Change-Id: I4ae4ce77cb600a06ffd9b15b6d4c730314aa0b79
Merged-In: I4ae4ce77cb600a06ffd9b15b6d4c730314aa0b79
(cherry picked from commit faa8e0b7e9)
Bug: 79951743
Test: Run validate_target_files.py on target_files.zip that has
non-monotonic ranges listed in IMAGES/system.map.
Change-Id: I28d3ca6972d361362589cd51e60731af9994a551
Merged-In: I28d3ca6972d361362589cd51e60731af9994a551
(cherry picked from commit d32936d657)
Targets can define 'TARGET_USERIMAGES_SPARSE_EXT_DISABLED := true' to
generate non-sparse system images, but validate_target_files.py doesn't
work with such images. This CL adds a workaround to temporarily skip the
file consistency check for such images.
Bug: 79616357
Test: Run validate_target_files.py on a target_files.zip that's not
using sparse image.
Test: Run validate_target_files.py on marlin target_files.zip (which
uses sparse image).
Change-Id: I1f4066c5b3fec595b10cab10283d62c1c5a6c624
Merged-In: I1f4066c5b3fec595b10cab10283d62c1c5a6c624
(cherry picked from commit 63e2f49fdd)
Bug: 77298768
Test: test that config.fs AIDs are visible through this library
Change-Id: Ifbf3276212ea6904533ac23021bfce29d9a3c5d9
(cherry picked from commit 7a95c15c15)
This CL defaults the journal size to 0 for system images (i.e. system,
vendor, system_other, oem, product). We used to do this by explicitly
defining BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_JOURNAL_SIZE to 0 in
device-specific BoardConfig. Targets can still specify a non-zero
journal size as needed.
With this CL, marlin/sailfish gets 32MB / 16MB free space back for
system and vendor respectively.
Bug: 75975085
Test: `m dist`. Check the journal size in the generated images.
Change-Id: Ib3185d07c49be2b4f0fac5fe17ec1a82093ba0c0
Defaults the reserved blocks for root to 0% on read only partitions
(system, system_other, vendor, oem). It also adds support for
explicitly specifying the extfs reserved percentage via
BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_EXTFS_RSV_PCT.
This eventually translates down to the -m option for mkfs.
Removing the reserved space can save at least 5% from the default.
dumpe2fs system:
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
Bug: 75975085
Test: Build, verify reserved space is changed accordingly
Change-Id: I212d82741908b636db0d658a1c4847bbaadfd5ba
Non-A/B devices need to include the DTBO image
within the recovery partition to be self-sufficient
and prevent OTA failures.
Test: Ran 'm dist' and verified that the DTBO image
was included in recovery.img using unpack_bootimg.
Also ran 'make' and verified that the DTBO image was
included in recovery.img using unpack_bootimg.
Bug: 74763691
Change-Id: I38c9c395c95d21f4da42cfa646063bd4416f6bd8
Merged-In: I38c9c395c95d21f4da42cfa646063bd4416f6bd8
(cherry picked from commit e74a38bc6d)
In FinalizeMetadata and PropertyFiles, we need to reserve space between
the calls to Compute() and Finalize(). We used to put a 10-byte
placeholder, in the hope of covering the 'offset:length' space for the
metadata entry, as well as the possible value changes in other entries.
However, this could fail in two possible cases: (a) metadata entry
itself has a large offset (e.g. staying near the end of a 1-GiB package,
where the offset itself has 10-digit); or (b) the offsets for other
entries change substantially due to entry reordering. Note that for case
(b), it's space inefficient to always reserve 15-byte for _each_ token
in the property-files.
This CL handles both of these two cases. For (a), we bump up the 10-byte
to 15-byte, which is large enough to cover a package size up to 10-digit
number (i.e. ~9GiB) with a metadata entry size of 4-digit. All these
15-byte will be used for the metadata token alone.
For (b), we add a fallback flow that would retry one more time, but
based on the already signed package that has entries in desired order.
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate aosp-bullhead full OTA with '--no_signing' flag.
Change-Id: If20487602d2ad09b3797465c01972f2fa792a1f1
(cherry picked from commit 3bf8c65029)
This CL handles a path that uses system-root-image on non-A/B device.
For this path, we can't generate recovery-from-boot patch with imgdiff,
because boot/recovery images contain different number of entries (only
recovery iamge has ramdisk image now).
Using BOARD_USES_FULL_RECOVERY_IMAGE can work around the issue, at the
cost of extra size. Compared to carrying full recovery image, this CL
saves the cost of the kernel size, by putting a patch that's roughly the
size of the recovery ramdisk.
The applypatch executable already detects and handles a bsdiff patch
automatically. No change required to that end.
Note that it won't further reduce the patch size by handling that
ramdisk entry specially, because (a) that's the only difference between
the two images; and (b) there's no corresponding data in boot image to
be diff'd against.
Bug: 72731506
Test: `m dist` with aosp_angler-userdebug. Check the device can install
recovery image successfully (after intentionally corrupting the
recovery image).
Test: Build aosp_angler-userdebug with BOARD_BUILD_SYSTEM_ROOT_IMAGE set.
Verify the generated patch files.
Test: Run validate_target_files.py with the target_files.zips.
Change-Id: I69c06f51ba8c39ae059c5e9a6872a9f10600cf17
Merged-In: I69c06f51ba8c39ae059c5e9a6872a9f10600cf17
(cherry picked from commit 6d5d623987)
This CL exposes ota-property-files flag for non-A/B OTA packages.
Currently the line only contains the info for the METADATA entry, for
example "ota-property-files=metadata:69:286". This allows system updater
to just download the METADATA entry, as opposed to downloading the
entire package, to learn about the info regarding the OTA package (e.g.
post-OTA build fingerprint). Note that this requires the OTA server-side
support to pass down the flag along with the update URL.
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate a non-A/B package and check the property-files string.
Change-Id: I1482c587e18ea7101c8328777ea988c2d8ca06ac
Merged-In: I1482c587e18ea7101c8328777ea988c2d8ca06ac
(cherry picked from commit c0746f4e94)
AbOtaPropertyFiles writes 'ota-property-files' into package metadata.
Comparing to StreamingPropertyFiles, AbOtaPropertyFiles additionally
writes a token of 'payload-metadata.bin' for a virtual entry that's head
of the 'payload.bin'. This entry can be used by the system updater to
verify if a payload is applicable to the underlying device.
AbOtaPropertyFiles intends to replace StreamingPropertyFiles, as it
covers all the info of the latter. We keep them both in P, and will
remove the latter in later release.
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate full and incremental A/B OTAs. Check the property-files
in the METADATA entry.
Change-Id: If5872c8ee0a97a8a95b07d4c69b463aa3fe1b9b9
(cherry picked from commit b63046750c)
And move StreamingPropertyFiles as its subclass. We will need similar
PropertyFiles instance for non-A/B OTA as well (to expose the
offset/size for the METADATA entry).
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate an A/B OTA. Check the generated property-files string.
Test: pylint --rcfile=pylintrc \
ota_from_target_files.py \
test_ota_from_target_files.py
Change-Id: If90d97f0b330749fd8a6cde2ed9d0d6cd6ea60a8
(cherry picked from commit 432f374a1701909ca324f8b047666614684568c0)
This CL breaks down ComputeStreamingMetadata() into mutiple member
functions of StreamingPropertyFiles class, which correspond to the
two-pass logic when generating streaming property files (aka streaming
metadata).
StreamingPropertyFiles.Compute() does the work for the first pass, by
putting placeholders before doing initial signing. Finalize()
corresponds to the second pass, where the placeholders get replaced with
actual data. Verify() can be optionally called to assert the correctness
of the work.
The separation between Compute() and Finalize() is to allow having
multiple StreamingPropertyFiles instances (in coming up CLs). This way
we can call Compute() multiple times for each instance, followed by only
one call to SignOutput(). And similarly for Finalize().
Bug: 74210298
Test: Generate an A/B OTA package. Check the METADATA entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I45be0372a4863c4405e6d8e20bcb9ccdc29e7e11
(cherry picked from commit ae5e4c30fe)
The file was broken due to earlier touches:
- Missing 'import zipfile';
- Mismatching arguments when calling GetSparseImage().
Bug: 73996151
Test: Run validate_target_files.py with a walleye-target_files.zip.
Test: pylint --rcfile=pylintrc validate_target_files.py
Change-Id: I3692bd51fb27a3da698e06b75155e84502549f66
(cherry picked from commit c63626b4a3)
And add unittests for ComputeStreamingMetadata().
This prepares for the changes that add additional property-files (for
both of A/B and non-A/B).
Bug: 74210298
Bug: 72751683
Test: python -m unittest test_ota_from_target_files
Test: Generate A/B OTA package. Check the ota-streaming-property-files
in the METADATA entry.
Change-Id: Ib4b069f61c2c06c035c0cff73a55112f3936b969
(cherry picked from commit f5110498c0)
There's a mismatch in WriteABOTAPackageWithBrilloScript().
temp_zip_file = tempfile.NamedTemporaryFile()
...
common.ZipClose(temp_zip_file)
It's benign since common.ZipClose() happens to be calling
"temp_zip_file.close()". This CL moves the use of tempfile to
common.MakeTempFile(), so that the tempfile will be cleaned up
automatically as part of the call to common.Cleanup(). (Not fixing the
close() directly, since the nearby lines will be refactored into another
function shortly.)
Also remove one assert in the same function, which trivially holds in
the current code.
Test: Generate an A/B OTA.
Change-Id: I53b375d1150820de718dec0ead55abf5f4951071
Also minor cleanups to make it pylint clean.
Test: Run check_target_files_signatures.py with a target-files.zip.
Test: pylint --rcfile=pylintrc check_target_files_signatures.py
Change-Id: Ife3b54c7805c2f2562e87e91ab4b4de355782012
In addition to the unzipping work, common.UnzipTemp() kindly bundles an
open ZipFile object as part of the return value. It doesn't look very
helpful to the callers though. It also looks less obvious that the
caller needs to properly close the handle (missing the close here is
benign though). This CL just removes the ZipFile object out of the
return value, and leaves the work to callers.
Test: `m dist` on both of A/B and non-A/B target.
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_common
Test: python -m unittest test_ota_from_target_files
Test: Check the callers to common.UnzipTemp() in code search.
Change-Id: Id47da3fd42a0e76d6ae8851f05780db319ee48cf
This should be the last case to be moved over.
Test: Generate an incremental BBOTA (which exercises the changed code).
Test: `rgrep mkdtemp` gives no more instance.
Change-Id: I76db069476201cdfaf3a2de9d9635dfe54507f7a
... from the following functions in add_img_to_target_files.py.
AddSystem
AddSystemOther
AddVendor
AddProduct
AddDtbo
AddUserdata
AddVBMeta
AddPartitionTable
AddCache
The last user of the parameter in img_from_target_files.py has been
removed in commit 2bb109709a (in O).
Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Test: Check all the callers to the above functions.
Test: m dist
Change-Id: I551d1683def8f8535062fc90f68dafa0f4252822
