applypatch() was initially designed for file-based OTA, operating on
individual files. It was later extended to allow patching eMMC targets
as a whole, in favor of block-based updates.
As we have deprecated file-based OTA since Oreo, part of the code in
applypatch() has become obsolete. This CL refactors the related
functions, by removing the obsolete logic and focusing on eMMC targets.
Since this CL substantially changes applypatch APIs, it adds new
functions to avoid unintentionally mixing them together. In particular,
it removes `applypatch()`, `applypatch_check()`, `applypatch_flash()`,
and adds `PatchPartition()`, `PatchPartitionCheck()`, `FlashPartition()`
and `CheckPartition()`. It also replaces the old Edify functions
`apply_patch()` and `apply_patch_check()` with `patch_partition()` and
`patch_partition_check()` respectively.
This CL requires matching changes to OTA generation script (in the same
topic).
Bug: 110106408
Test: Run recovery_unit_test and recovery_component_test on marlin.
Test: `m dist` with non-A/B target. Verify
/system/bin/install-recovery.sh on device.
Test: `m dist` with non-A/B target using BOARD_USES_FULL_RECOVERY_IMAGE.
Verify /system/bin/install-recovery.sh on device.
Test: Install an incremental OTA with the new updater and scripts.
Change-Id: Ia34a90114bb227f4216eb478c22dc98c8194cb7f
This applies to the standalone applypatch executable
(/system/bin/applypatch on device). This executable is only used when
installing (via patching or flashing) a recovery image on non-A/B
device.
This CL removes the support for patching non-eMMC targets from
applypatch that has been deprecated as part of file-based OTA. For
patching eMMC targets, it also drops the support for accepting multiple
patches (not useful, since the source file must be fixed).
This CL needs the matching change in the same topic, which writes the
script of "/system/bin/install-recovery.sh". Note that this CL doesn't
chanage the applypatch API signatures, in order to minimize the CL size.
*BEFORE*
usage: /system/bin/applypatch [-b <bonus-file>] <src-file> <tgt-file> <tgt-sha1> <tgt-size> [<src-sha1>:<patch> ...]
or /system/bin/applypatch -c <file> [<sha1> ...]
or /system/bin/applypatch -l
Filenames may be of the form
EMMC:<partition>:<len_1>:<sha1_1>:<len_2>:<sha1_2>:...
to specify reading from or writing to an EMMC partition.
*AFTER*
Usage:
check mode
applypatch --check EMMC:<target-file>:<target-size>:<target-sha1>
flash mode
applypatch --flash <source-file>
--target EMMC:<target-file>:<target-size>:<target-sha1>
patch mode
applypatch [--bonus <bonus-file>]
--patch <patch-file>
--target EMMC:<target-file>:<target-size>:<target-sha1>
--source EMMC:<source-file>:<source-size>:<source-sha1>
show license
applypatch --license
Bug: 110106408
Test: Run recovery_component_test and recovery_unit_test on marlin.
Test: Build a non-A/B target that has /system/bin/install-recovery.sh.
Verify that it installs recovery image successfully.
Test: Build a non-A/B target that has /system/bin/install-recovery.sh in
flashing mode. Verify that it installs recovery image successfully.
Change-Id: I71f9a71fb457e6f663e0b5511946949e65b4b78c
They are doing exactly the same thing, except for the slightly different
error return value (1 vs -1).
int CacheSizeCheck(size_t bytes);
int MakeFreeSpaceOnCache(size_t bytes_needed);
This CL consolidates the two functions and uses bool as its return type.
// Checks whether /cache partition has at least 'bytes'-byte free space. Returns true immediately
// if so. Otherwise, it will try to free some space by removing older logs, checks again and
// returns the checking result.
bool CheckAndFreeSpaceOnCache(size_t bytes);
Test: Run recovery_unit_test and recovery_component_test on marlin.
Change-Id: I94a96934d2b18713f8f39ad5aa96a02c98d87963
Prior to this CL, FreeSpaceForFile() was returning `size_t`, which may
overflow on ILP32 when called on a partition with 4GiB+ free space.
Additionally, it was returning static_cast<size_t>(-1) on error, but the
caller in freecache.cpp didn't check for that.
This CL changes its return type to `int64_t`, and moves the function
into freecache.cpp since there's no external caller.
Test: Run recovery_unit_test on marlin.
Test: Code search shows no external user of FreeSpaceForFile().
Change-Id: I00f501a057726e1f1ab69f367c46c77b30f2d774
Also fix an error-pone behavior in previous code when verifying an eMMC
target. As long as it loads the partition content successfully according
to the SHAs embedded in the filename, it shouldn't further check against
the SHAs given in the second argument. Because the loaded contents
relate to a specific partition size.
For example:
apply_patch_check(
"EMMC:/boot.img:src_size:src_hash:tgt_size:tgt_hash",
"src_hash");
Assume "/boot.img" already has the desired hash of "tgt_hash", the
previous code would give wrong verification result. The issue can be
addressed by additionally listing "tgt_hash" as one of the desired SHAs
(or by applying this CL).
Bug: 110106408
Test: Run recovery_unit_test and recovery_component_test on marlin.
Change-Id: I8daafdbecd083f687e24d563ab089caa25667633
Also two minor changes (other than renaming some parameters):
- Added constness to the first parameter of FindMatchingPatch();
- Declared WriteToPartition() as static.
Bug: 110106408
Test: mmma -j bootable/recovery
Change-Id: I388958c944a23ce4a38a757ce2249f6a89dd4f03
The imgpatch will fail on empty deflates because the bspatch won't call
the sink function if the target length is zero.
Instead of compressing an empty string, it's cleaner to not generate such
empty deflate chunks in the patch. Therefore, we can just convert the
chunk type to raw if the target length is smaller than the patch data.
Also adjust some unit tests and add the testdata gzipped_source &
gzipped_target. These two files are ~1K each and are generated by
gzipping two slightly different regular files.
Bug: 79265132
Test: unit tests pass, imgpatch applys successfully on the given src/tgt
Change-Id: I6bfff3251918137f6762a6f9e9551642371a1124
As they're accepting the SinkFn callback, it makes more sense to leave
the work to their callers.
Test: mmma -j bootable/recovery
Test: Run recovery_component_test on marlin.
Test: No other active user of the two functions.
Change-Id: I8d67b38ce037925442296f136b483e0c71983777
We set the limit of the max stash size to 80% of cache size. But the
cache space can still be insufficient for the update if the log files
occupy a large chunk of /cache. So remove the old logs for now to make
room for the update.
Bug: 77528881
Test: unit tests pass
Change-Id: Ia8bcb0ace11f8164ad9290bfb360e08e31d282cb
This class allows us to set the following locations dynamically:
cache_temp_source, last_command_file, stash_directory_base.
In the updater's main function, we reset the values of these variables
to their default locations in /cache; while we can set them to temp
files in unit tests or host simulation.
Test: unit tests pass
Change-Id: I528652650caa41373617ab055d41b1f1a4ec0f87
In the split mode of imgdiff, we used to assume that the size of a split
target chunk is always greater than the blocksize i.e. 4096. This may
lead to the following assertion failure:
I0221 04:57:33.451323 818464 common.py:205 imgdiff F 02-21 04:57:33 821203 821203 imgdiff.cpp:999]
Check failed: tgt_size >= BLOCK_SIZE (tgt_size=476, BLOCK_SIZE=4096)
This CL removes the assumption and handles the edge cases.
Test: generate and verify the incremental update for TFs in the bug; unit test passes
Bug: 73757557
Bug: 73711365
Change-Id: Iadbb4ee658995f5856cd488f3793980881a59620
It used to keep track of the stat(2) info (e.g. st_mode/st_gid/st_uid)
while patching a file in file-based OTA.
Test: Build and use the new updater to apply an update on bullhead.
Change-Id: Ibf8f0f4b14298a9489bf24a2678bb279c5d9c8f3
Also add a verbose option. And we won't print messages of 'info'
severity unless '-v' is present.
Test: run imgdiff and check the logs.
Change-Id: I1b90874baea8e72e2a2323a0b63bc5d35e653e6b
It used to be "const Value*", but nullptr won't be a valid input.
Test: recovery_host_test; recovery_component_test
Change-Id: I904b5689ac3e64504088bf0544c9fb5d45a52243
We encountered segfaults in Imgdiff host tests due to the failure to
reset states of getopt. The problem can be solved by switching to use
bionic's gtest where a new process is forked for each test.
Also modify the recovery_component_test to make sure it runs in parallel.
Changes include:
1. Merge the writes to misc partition into one single test.
2. Change the hard coded location "/cache/saved.file" into a configurable
variable.
Bug: 67849209
Test: recovery tests pass
Change-Id: I165d313f32b83393fb7922c5078636ac40b50bc2
bsdiff interface is changing such that it hides the suffix array
pointer from the public interface. This allows to use a different
suffix array data size depending on the input size, running much faster
in the normal case.
Bug: 34220646
Test: `make checkbuild`; Ran an incremental update generation on a non-A/B device.
Change-Id: I78e766da56cf28bc7774b8c8e58527bc11d919fb
Also drop the "bootable/recovery" path in LOCAL_C_INCLUDES from
applypatch modules.
Test: lunch aosp_{angler,bullhead,fugu,dragon,sailfish}-userdebug;
mmma bootable/recovery
Change-Id: Idd602a796894f971ee4f8fa3eafe36c42d9de986
And move '#include "edify/expr.h"' into .cpp files. This breaks the
transitive dependency on libedify. Modules that include
"applypatch/applypatch.h" don't need to add libedify into their
dependency list, unless they really need anything from libedify.
Build libedify static library for host, which is needed by
libimgpatch.
Test: mmma bootable/recovery
Change-Id: Ibb53d322579fcbf593438d058d9bcee240625941
Add a mandatory option in imgdiff to write the split info (i.e.
patch_size, tgt_size, src_ranges) to file when handling large apks.
Therefore, the caller of imgdiff can create split transfers based on
the info.
Bug: 63542719
Test: unit tests pass
Change-Id: I853d55d1f999fd576474faa81077f7307f4d856d
Due to the cache size limit for OTA generation, we used to split large
zip files linearly into pieces and do bsdiff on them. As a result, i) we
lose the advantage of imgdiff; ii) if there's an accidental order change
of some huge files inside the zip, we'll create an insanely large patch.
This patch splits the src&tgt more smartly based on the zip entry_name.
If the entry_name is empty or no matching source is found for a target
chunk, we'll skip adding its source and later do a bsdiff against the
whole split source image (this rarely happens in our use cases except
for the metadata inside a ziparchive).
After the split, the target pieces are continuous and block aligned,
while the sources pieces are mutually exclusive. (Some of the source
blocks may not be used if there's no matching entry_name in the target.)
Then we will generate patches accordingly between each split image
pairs.
Afterwards, if we apply imgpatch to each pair of split source/target
images and add up the patched result, we can get back the original
target image.
For example:
Input: [src_image, tgt_image]
Split: [src-0,tgt-0; src-1,tgt-1, src-2,tgt-2]
Diff: [ patch-0; patch-1; patch-2]
Patch: [(src-0,patch-0)=tgt-0; (src-1,patch-1)=tgt-1;
(src-2,patch-2)=tgt-2;]
Append: [tgt-0 + tgt-1 + tgt-2 = tgt_image]
Peformance:
For the small package in b/34220646, we decrease the patch size of
chrome.apk dramatically from 30M to 400K due to the order change of
two big .so files.
On two versions of angler, I also observe decent patch size decrease.
For chrome.apk, we reduced the size from 5.9M to 3.2M; and for
vevlet.apk from 8.0M to 6.5M.
Bug: 34220646
Test: recovery component test && apply imgdiff & imgpatch on two
chrome.apk
Change-Id: I145d802984fa805efbbac9d01a2e64d82ef9728b
1. Move the declaration of the Image classes to the header file to make
testing easier.
2. Also move rangeset.h to bootable/recovery to allow access in imgdiff.
Test: recovery component test
Change-Id: I68a863e60a3f2e7ae46ee48f48eb15391f5f4330
This will help us to identify the patch corruption.
Meanwhile fix a wrong size parameter passed to bspatch.
(patch->data.size() into patch->data.size() - patch_offset).
Also remove the only usage of "ApplyBSDiffPatchMem()" and inline its
Sink function for simplicity.
Bug: 37855643
Test: Prints SHA1 for corrupted patch in imgdiff_test.
Change-Id: Ibf2db8c08b0ded1409bb7c91a3547a6bf99c601d
Mostly for applypatch family APIs like ApplyBSDiffPatch() and
ApplyImagePatch(). Changing to size_t doesn't indicate they would
necessarily work with very large size_t (e.g. > ssize_t), just
similar to write(2). But otherwise accepting negative length doesn't
make much sense.
Also change the return type of SinkFn from ssize_t to size_t. Callers
tell a successful sink by comparing the number of written bytes against
the desired value. Negative return values like -1 are not needed. This
also makes it consistent with bsdiff::bspatch interface.
Test: recovery_component_test
Test: Apply an incremental with the new updater.
Change-Id: I7ff1615203a5c9854134f75d019e266f4ea6e714
Factor out libimgdiff static library for testing purpose.
This CL adds the imgdiff tests on host and on target both (similar to
libimgpatch). In practice, we only need imgdiff binary on host, and
libimgpatch on target. But they should build and pass tests on both
platforms.
Test: recovery_host_test passes; recovery_component_test passes.
Change-Id: I0eafb7faf727cdf70066310e845af6ee245d4f60
Changing the field of 'Value' in edify to std::string from char*.
Meanwhile cleaning up the users of 'Value' and switching them to
cpp style.
Test: compontent tests passed.
Bug: 31713288
Change-Id: Iec5a7d601b1e4ca40935bf1c70d325dafecec235
If two libraries both use LOCAL_WHOLE_STATIC_LIBRARIES and include a same
library, there would be linking errors when generating a shared library
(or executable) that depends on the two libraries both.
Also clean up Android.mk files.
Remove the "LOCAL_MODULE_TAGS := eng" line for the updater module. The
module will then default to "optional" which won't be built until needed.
Change-Id: I3ec227109b8aa744b7568e7f82f575aae3fe0e6f
update_engine need it for the new IMGDIFF operation.
Also removed __unused in ApplyImagePatch() as I got error building it
for the host, and I think it's dangerous not checking the size of the
input.
Test: mma
Bug: 26628339
Change-Id: I22d4cd55c2c3f87697d6afdf10e8106fef7d1a9c
