This is a stress test that instantiates multiple testcases that
interrupt a BBOTA update at every transfer command. Each testcase
asserts the last_command_file after the interruption, verifies the
update resumability, then resumes the update and asserts the updated
image.
The transfer list in the testcase covers most of the transfer commands
(stash/free/move/bsdiff/zero/new), as well as some special pattern like
having duplicate stash ids.
This CL also addresses one issue in the updater code, by resetting the
stash_map before each run. The stash map should be valid only per
block_image_verify/block_image_update run. Having leftover may cause
issue in subsequent runs, in particular when calling block_image_verify
after a previous run of block_image_{update,verify}.
Test: Run recovery_component_test on marlin.
Change-Id: I6f9a0368d194a754ce41a9c9819c6d5be2657248
Otherwise tests may interfere with each other by using the same / default
location.
Test: Run recovery_component_test on marlin.
Change-Id: I6b0455489f2fdce819009964dd92bfd9bfeb06ef
The imgpatch will fail on empty deflates because the bspatch won't call
the sink function if the target length is zero.
Instead of compressing an empty string, it's cleaner to not generate such
empty deflate chunks in the patch. Therefore, we can just convert the
chunk type to raw if the target length is smaller than the patch data.
Also adjust some unit tests and add the testdata gzipped_source &
gzipped_target. These two files are ~1K each and are generated by
gzipping two slightly different regular files.
Bug: 79265132
Test: unit tests pass, imgpatch applys successfully on the given src/tgt
Change-Id: I6bfff3251918137f6762a6f9e9551642371a1124
Move the common codes into RunBlockImageUpdate(). Also clean up the
partition updated marker after running each test.
Test: Run recovery_component_test on marlin.
Change-Id: Id4302e4da4c664231b737a1e83d2e164ef58ed97
We have a general need for overriding more paths (e.g. "/tmp"), mostly
for testing purpose. Rename CacheLocation to Paths, and use that to
manage TEMPORARY_{INSTALL,LOG}_FILE.
Test: mmma -j bootable/recovery
Test: recovery_component_test
Change-Id: Ia8ce8e5695df37ca434f13ac4d3206de1e8e9396
Dump the SHA1 of the uncompressed data in applypatch to confirm if we
are at least doing the bspatch part correctly. (I expect so since the actual
length of the uncompressed data matches the expected length).
Also try to decompress the deflate chunk inside the recovery image for
these two flacky tests. In theory, there shouldn't be randomness in
zlib; so we would know if we process the data wrongly if the deflate fails
to decompress.
Bug: 67849209
Test: recovery_component_test
Change-Id: Id947522153b1eeb0d10d161298a96fb045f92018
The apply patch test should have a deterministic way to append patch
data. Add debug logs to dump the length and SHA1 of each step to further
track down the flakiness.
Also redirect the debug logging to stdout in case the logcat becomes too
chatty.
Bug: 67849209
Test: Run recovery_component_test
Change-Id: I42bafef2d9dee599719ae57840b3d8c00d243ebd
This allows the update_verifier in a general system image to work across
devices that have different verified boot versions (i.e. not supported /
verified boot 1.0 / verified boot 2.0 / disabled).
Bug: 78283982
Test: Run recovery_component_test on both of marlin and walleye.
Test: Generate an OTA that has this CL. Install this OTA and check the
update_verifier log during the post-reboot verification, on both
of marlin (VB 1.0) and walleye (VB 2.0).
Test: Build and flash walleye image with verified boot disabled. Check
that update_verifier marks the slot as successfully booted.
Change-Id: I828d87d59f911786531f774ffcf9b2ad7c2ca007
After splitting the previously flaky
ApplyPatchModesTest#PatchModeEmmcTarget tests,
PatchModeEmmcTargetWithMultiplePatches now becomes the sole victim. This
CL dumps additional info to narrow down the cause.
Bug: 67849209
Test: `recovery_component_test` on marlin.
Test: It dumps additional info after using corrupt bonus.file.
Change-Id: Ic5436de457cc882a51d03f49d5cee70077f7d3df
Although the tests were initially written for checking the validity of
the text images, it doesn't hurt to run them continuously as part of the
component test (recovery_manual_test requires reboots during the run,
due to the nature of the tests of recovery-{refresh,persist}). This also
allows detecting breaking changes to libminui or libpng.
There's a catch that the ResourcesTest won't be triggered via `atest`,
as the res-* testdata won't be picked up via AndroidTest.xml. Explored
a few options but not addressing that in this CL:
- `atest` is not fully working in AOSP yet (missing support in
tools/tradefederation/core/atest/atest.py).
- `atest` doesn't allow specifying the testdata with path in the 'push'
option.
- It won't fail the test run though, as ResourcesTest will skip the
tests automatically when it finds no text image file.
- APCT and manual `adb sync data` are not affected, and I don't see an
active user of `atest` other than a tool for manual test invocation.
- Unrelated to this CL, `atest` doesn't seem to work well with
recovery_component_test or recovery_unit_test while we have both of
them in one AndroidTest.xml. It randomly triggers only one of them,
despite of the given test name. When splitting AndroidTest.xml into
two, it tends to pick up the wrong testdata subdir and gives wrong
results.
Test: Run recovery_manual_test and recovery_component_test on marlin.
Change-Id: I3a237499a7770356e14085674bc8b9cb4551db85
We set the limit of the max stash size to 80% of cache size. But the
cache space can still be insufficient for the update if the log files
occupy a large chunk of /cache. So remove the old logs for now to make
room for the update.
Bug: 77528881
Test: unit tests pass
Change-Id: Ia8bcb0ace11f8164ad9290bfb360e08e31d282cb
We have added the support for building /product partition in build
system (the CL in [1]), where /product is an optional partition that
contains system files. This CL adds the matching support if /product
needs to be verified during A/B OTA (i.e. listed in care_map file).
[1]: commit b7735d81054002961b681f4bdf296d4de2701135,
https://android-review.googlesource.com/c/platform/build/+/598454
Bug: 63974895
Test: Run update_verifier test on walleye.
Change-Id: Ia1c35e9583b8e66c98a4495b1f81a5ea7e65036f
We have been seeing flakiness in continuous test, but unable to
reproduce locally. Break it down into smaller units to narrow down the
cause.
Bug: 67849209
Test: Run recovery_component_test on marlin.
Change-Id: Ia24b0c5d137bad27d502575fcd18d3ca9c9828b6
/system/bin/applypatch on device is expected to work with bsdiff based
recovery-from-boot patch automatically. Adding a test to ensure that's
always the case.
Bug: 72731506
Test: Run recovery_component_test on marlin.
Change-Id: I56283cd3ce7cf0215cc3bb3619b206fa01d552c4
This class allows us to set the following locations dynamically:
cache_temp_source, last_command_file, stash_directory_base.
In the updater's main function, we reset the values of these variables
to their default locations in /cache; while we can set them to temp
files in unit tests or host simulation.
Test: unit tests pass
Change-Id: I528652650caa41373617ab055d41b1f1a4ec0f87
In the split mode of imgdiff, we used to assume that the size of a split
target chunk is always greater than the blocksize i.e. 4096. This may
lead to the following assertion failure:
I0221 04:57:33.451323 818464 common.py:205 imgdiff F 02-21 04:57:33 821203 821203 imgdiff.cpp:999]
Check failed: tgt_size >= BLOCK_SIZE (tgt_size=476, BLOCK_SIZE=4096)
This CL removes the assumption and handles the edge cases.
Test: generate and verify the incremental update for TFs in the bug; unit test passes
Bug: 73757557
Bug: 73711365
Change-Id: Iadbb4ee658995f5856cd488f3793980881a59620
When performing an update, save the index and cmdline of the current
command into the last command file if this command writes to the stash
either explicitly of implicitly. This mitigates the overhead to update
the last command file for every command. I ran a simple test on angler
and the time to update 1000 times is ~2.3 seconds.
Upon resuming an update, read the saved index first; then
1. In verification mode, check if all commands before the saved index
have already produced the expected target blocks. If not, delete the
last command file so that we will later resume the update from the
start of the transfer list.
2. In update mode, skip all commands before the saved index. Therefore,
we can avoid deleting stashes with duplicate id unintentionally;
and also speed up the update.
If an update succeeds or is unresumable, delete the last command file.
Bug: 69858743
Test: Unittest passed, apply a failed update with invalid cmd on angler
and check the last_command content, apply a failed update with invalid
source hash and last_command is deleted.
Change-Id: Ib60ba1e3c6d111d9f33097759b17dbcef97a37bf
This CL mainly changes:
a) moving the interface in struct provider_vtab to std::function;
b) code cleanup, such as moving the declaration closer to the uses,
using explicit type conversion.
Test: recovery_component_test
Test: minadbd_test
Test: Sideload a package on marlin.
Change-Id: Id0e3c70f1ada54a4cd985b54c84438c23ed4687e
We encountered segfaults in Imgdiff host tests due to the failure to
reset states of getopt. The problem can be solved by switching to use
bionic's gtest where a new process is forked for each test.
Also modify the recovery_component_test to make sure it runs in parallel.
Changes include:
1. Merge the writes to misc partition into one single test.
2. Change the hard coded location "/cache/saved.file" into a configurable
variable.
Bug: 67849209
Test: recovery tests pass
Change-Id: I165d313f32b83393fb7922c5078636ac40b50bc2
~TemporaryDir() calls rmdir(2) directly, which works with empty
directories only.
Test: Run recovery_host_test; No leftover on host.
Test; Run recovery_component_test on marlin; No leftover on device.
Change-Id: Ib510efb16eeda61b34161e2b386499e6cb79a4ca
As we construct the deflate entries of the target zip file with
random data, the total size of the zip file may vary from case
to case. This leads to occasional failures in the split test for
deflate large apk files. This CL fixes the issue by adding two static
zip files in the testdata instead of generating them dynamically.
Bug: 67849209
Test: run the deflate_large_test repeatedly
Change-Id: Iaeffad9205adefa10c9f62f9f088c33c4360a650
bsdiff interface is changing such that it hides the suffix array
pointer from the public interface. This allows to use a different
suffix array data size depending on the input size, running much faster
in the normal case.
Bug: 34220646
Test: `make checkbuild`; Ran an incremental update generation on a non-A/B device.
Change-Id: I78e766da56cf28bc7774b8c8e58527bc11d919fb
The thread to receive new data may still be alive after we exit
PerformBlockImageUpdate() upon failures. This caused memory corruption
when we run the unittest repeatedly. Set the receiver_available flag
to false and make sure the receiver exits when the update fails.
Bug: 65430057
Test: unittests passed with tsan
Change-Id: Icb232d13fb96c78262249ffbd29cdbe5b77f1fce
Since it has grown much larger, users of the header shouldn't compile
and carry their full copies.
Also add missing header includes in imgdiff.cpp and imgdiff_test.cpp.
Test: mmma bootable/recovery
Test: recovery_unit_test; recovery_component_test; recovery_host_test
Change-Id: I88ca54171765e5606ab0d61580fbc1ada578fd7d
Also drop the "bootable/recovery" path in LOCAL_C_INCLUDES from
applypatch modules.
Test: lunch aosp_{angler,bullhead,fugu,dragon,sailfish}-userdebug;
mmma bootable/recovery
Change-Id: Idd602a796894f971ee4f8fa3eafe36c42d9de986
This reverts commit 26436d6d60 to re-land
"Move error_code.h into otautil.".
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".
This CL needs to land with device-specific module changes (e.g. adding
the dependency on libotautil).
Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
mmma bootable/recovery
Change-Id: If193241801af2dae73eccd31ce57cd2b81c9fd96
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".
Test: mmma bootable/recovery
Change-Id: Ia4649789cef2aaeb2785483660e9ea5a8b389c62
Add a mandatory option in imgdiff to write the split info (i.e.
patch_size, tgt_size, src_ranges) to file when handling large apks.
Therefore, the caller of imgdiff can create split transfers based on
the info.
Bug: 63542719
Test: unit tests pass
Change-Id: I853d55d1f999fd576474faa81077f7307f4d856d
Otherwise the test may fail after a large number of iterations due to
file open failure.
Bug: 65430057
Test: run recovery_component_test on sailfish for 2000 iterations.
Change-Id: I0d456284d6064467038911d63eade95740cbec2c
Due to the cache size limit for OTA generation, we used to split large
zip files linearly into pieces and do bsdiff on them. As a result, i) we
lose the advantage of imgdiff; ii) if there's an accidental order change
of some huge files inside the zip, we'll create an insanely large patch.
This patch splits the src&tgt more smartly based on the zip entry_name.
If the entry_name is empty or no matching source is found for a target
chunk, we'll skip adding its source and later do a bsdiff against the
whole split source image (this rarely happens in our use cases except
for the metadata inside a ziparchive).
After the split, the target pieces are continuous and block aligned,
while the sources pieces are mutually exclusive. (Some of the source
blocks may not be used if there's no matching entry_name in the target.)
Then we will generate patches accordingly between each split image
pairs.
Afterwards, if we apply imgpatch to each pair of split source/target
images and add up the patched result, we can get back the original
target image.
For example:
Input: [src_image, tgt_image]
Split: [src-0,tgt-0; src-1,tgt-1, src-2,tgt-2]
Diff: [ patch-0; patch-1; patch-2]
Patch: [(src-0,patch-0)=tgt-0; (src-1,patch-1)=tgt-1;
(src-2,patch-2)=tgt-2;]
Append: [tgt-0 + tgt-1 + tgt-2 = tgt_image]
Peformance:
For the small package in b/34220646, we decrease the patch size of
chrome.apk dramatically from 30M to 400K due to the order change of
two big .so files.
On two versions of angler, I also observe decent patch size decrease.
For chrome.apk, we reduced the size from 5.9M to 3.2M; and for
vevlet.apk from 8.0M to 6.5M.
Bug: 34220646
Test: recovery component test && apply imgdiff & imgpatch on two
chrome.apk
Change-Id: I145d802984fa805efbbac9d01a2e64d82ef9728b
