Converts the following files to Python logging.
add_img_to_target_files.py
blockimgdiff.py
build_image.py
check_ota_package_signature.py
common.py
img_from_target_files.py
make_recovery_patch.py
ota_from_target_files.py
sparse_img.py
verity_utils.py
This separates logging outputs from normal outputs, and allows easier
switching between different verbosity levels. It also supports adjusting
logging config via environment variable (LOGGING_CONFIG).
Test: `m dist`
Test: `python -m unittest discover build/make/tools/releasetools`
Change-Id: Idfc3f7aff83e03f41f3c9b5067d64fd595d5192d
Clean up the following scripts.
blockimgdiff.py
common.py
edify_generator.py
img_from_target_files.py
ota_from_target_files.py
Seems we don't have a way to force pylint-ing the scripts using
per-directory pylintrc files (and we don't have pylint tool in AOSP
repo), per
https://android.googlesource.com/platform/tools/repohooks/#todo_limitations.
Test: `m dist`
Test: pylint --rcfile=pylintrc <script.py>
Change-Id: Ia6fd1ddc86f4d84c68e500f225d4a89d0fea8ec7
In addition to the unzipping work, common.UnzipTemp() kindly bundles an
open ZipFile object as part of the return value. It doesn't look very
helpful to the callers though. It also looks less obvious that the
caller needs to properly close the handle (missing the close here is
benign though). This CL just removes the ZipFile object out of the
return value, and leaves the work to callers.
Test: `m dist` on both of A/B and non-A/B target.
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_common
Test: python -m unittest test_ota_from_target_files
Test: Check the callers to common.UnzipTemp() in code search.
Change-Id: Id47da3fd42a0e76d6ae8851f05780db319ee48cf
img_from_target_files.py used to handle the case that a given TF.zip not
containing the image entries under IMAGES/. That is only the case for
pre-Lollipop releases.
Also unzip the needed files only since we know that for sure now.
Test: img_from_target_files.py with an existing bullhead-TF.zip gives
the same bullhead-img.zip.
Change-Id: I892379ba388df80ae63be9d3ce647fbb77fd4753
In two-step OTAs, we write recovery image to /boot as the first step so
that we can reboot from there and install a new recovery image to
/recovery. However, bootloader will show "Your device is corrupt"
message when booting /boot with the recovery image. Because the recovery
image encodes the path of "/recovery" as part of the signature metadata,
which fails the verified boot.
This CL generates a special "recovery-two-step.img" in addition to the
regular recovery.img. This image encodes "/boot" when being signed,
which will be flashed to /boot at stage 1/3 in a two-step OTA.
Here are the desired changes:
- 'IMAGES/recovery-two-step.img' exists in target_files.zip for non-A/B
targets (e.g. bullhead). The image should not exist for targets that
don't have a recovery partition (e.g. A/B devices like sailfish).
- <device>-img.zip should not contain 'recovery-two-step.img'.
- Nothing should change when building non-two-step OTAs. For two-step
OTAs, 'recovery-two-step.img' should be included in the OTA package;
'updater-script' should flash this image to /boot at stage 1/3.
- When building a two-step OTA with an input TF.zip that doesn't have
IMAGES/recovery-two-step.img, it should use the existing
IMAGES/recovery.img instead.
Bug: 32986477
Test: Tested the steps above on bullhead and sailfish.
Change-Id: I34e6c599bcf2011d4cd5c926999418b3975d6d0f
Don't generate recovery.img when calling 'make dist' if
TARGET_NO_RECOVERY is set. The build system passes the flag to the
packaging script which then generates recovery.img conditionally.
Bug: 25329471
Change-Id: Ifbc999300d5c31e897878f81e231ae7dd2aca660
For system images that contain the root directory, we need to find the
root directory at ROOT/ instead of BOOT/RAMDISK/.
Change-Id: Ica345c8b1b03475f8ac6c44fd576045fcf17c882
file_contexts (specified by SELINUX_FC) is needed both when building
and (re)packaging. We used to use the copy in out/ when building, and
looked for the copy in BOOT/RAMDISK/ when packaging from target_files
zip. With system_root_image enabled, the file_contexts needed for
building and packaging might be different from the one on device. So
we explicitly pack the file as META/file_contexts in target_files zip.
Also refactor out the overriding of selinux_fc property into
common.LoadInfoDict().
Change-Id: I94f9ea6671b3792c12c1c21573840743d63da39a
(cherry picked from commit aa7318c384)
In order to work around the zip 2GiB limit, we need to wrap the related
functions in zipfile. Calls to those functions should always be replaced
with calls to the wrappers instead.
Bug: 18015246
Change-Id: Ice494371ca6654e88ded2ae0eb680f51082effcb
This caught a few bugs/syntax errors (a few character classes were not
escaped properly in regex patterns, some indentation was illegal,
etc).
Change-Id: I50637607524e68c4fb9cad7167f58a46b8d26b2c
We need to patch zipfile during close() too, because it refers to the
ZIP64 file size threshold when writing out the central directory
Bug: 18015246
Bug: 19888174
Change-Id: I1b49d653d0831fcc2106808f86c929d7a2b22ff3
Signed-off-by: Greg Hackmann <ghackmann@google.com>
Python 2.7's zipfile implementation wrongly thinks that zip64 is
required for files larger than 2GiB. We can work around this by
adjusting their limit. Note that `zipfile.writestr()` will not work
for strings larger than 2GiB. The Python interpreter sometimes rejects
strings that large (though it isn't clear to me exactly what
circumstances cause this). `zipfile.write()` must be used directly to
work around this.
This mess can be avoided if we port to python3.
The bug (b/19364241) in original commit has been fixed.
Bug: 18015246
Bug: 19364241
Bug: 19839468
(cherry picked from commit cd082d4bfe)
Change-Id: I7b5cc310e0a9ba894533b53cb998afd5ce96d8c6
make_ext4fs can now output a file listing the blocks used for each
file in the image. Request this file and save it in the target_files;
it will be used for future improvements to block OTAs.
Bug: 16984795
Change-Id: Id1e60465e3b5a9d126a7934b4d089cf34d8fec44
Store sparse images in the target-files, and use those (when they're
available) for building block OTAs.
- New script add_img_to_target_files is added to make the images and
add them to the IMAGES/ subdir in the target-files. It gets run
from the Makefile when building a target-files.
- img_from_target_files becomes mostly vestigial: it creates the
img.zip by just copying the images out of the target-files. (It
still knows how to build images for use on older target-files.)
- ota_from_target_files uses images from the target-files in
preference to rebuilding images from the source files.
- sign_apk_target_files builds images and includes them in its output
target files (even if the input target-files didn't have them).
Bug: 16488065
Change-Id: I444e0d722d636978209467ffc01750a585c6db75
In the past, there was an exception for ext-base fs types to
deal with the lack of image size.
Back then it was only yaffs and ext*.
So now we explicitely only allow yaffs to have no userdata image size.
Change-Id: Ie354ee6222a58228dbcce2c6934971a0737422af
Signed-off-by: JP Abgrall <jpa@google.com>
Make vendor partition a first-class member of the OTA system (for
target_files that contain a VENDOR/ subdirectory).
Build vendor images in a way that is compatible with block-based OTA.
Support updating the vendor partition in both full and incremental,
block and file OTAs. In most cases this is handled by refactoring the
existing code to handle the system partition to handle either, and
then calling it twice.
Currently we don't support incremental OTAs from a target-files
without a VENDOR subdirectory to one with one, or vice versa. To add
or remove a vendor partition a full OTA will need to be done.
Bug: 15544685
Change-Id: I9cb9a1267060bd9683a9bea19b43a26b5a43800d
When building images, we want to use the file_contexts and fs_config
data contained in the target_files zip, rather than whatever happens
to be in the current client.
Change-Id: I13df2405898039f5a9b4bb4837147e76b31b068a
img_from_target_files.py just skipps the boot.img and recovery.img since
there is no kernel or recovery.fstab for emulator.
Bug: 15383279
Change-Id: I4035193e6ab933194ff1417dfae4eab963fe5301
NamedTemporaryFile's aggressive caching behavior can cause an issue
where changes made by another process aren't visible even after the
fseek() below or a flush(). To avoid this, simply open the file
again and read from the fresh version.
This fixes an issue where verity metadata written by append2simg
doesn't become visible to img_from_target_files.
Change-Id: I291fb3a95d5b532218ac6205ecc9e9b4f3a36bd4
The system partitions has regions that we shouldn't write and can't
depend on the contents of. Adds a new script to generate a map of
these regions (using the sparse image as input), and include the map
in the package zip so it can be used when writing or patching the
system partition.
Also fixes a bug where the wrong SELinux file contexts are used when
generating incrementals.
Change-Id: Iaca5b967a3b7d1df843c7c21becc19b3f1633dad
These scripts already use some post-2.4 features, so let's make it
official: Python 2.7 is needed to run them.
Change-Id: I256e9ed99b0b62abe4e22a7b1f811acb7419e88e
Instead of writing individual files and fixing up their metadata, make
full OTAs contain a system image and simply write it to the block
device.
This is only done for target-files that already contain the recovery
flashing information, older target-files still get a file-based full
OTA.
Bug: 12893978
Change-Id: If7586083c8f275e24fec49d260af5b5aff4a0a88
