Switch to 64 bit integers since the size of the entire src/tgt images may
not fit in size_t of ILP32.
There are other theoretical overflow cases in memory allocation and I/O
functions. However, they reside within a single transfer command and are
less likely to happen. I will evaluate and address them in separate
cls.
Test: unit tests pass
Bug: 122461124
Change-Id: Ib719ee695920877458fcfaa25c6ac058a5bbabf2
Test: sideload full OTA on cuttlefish
Test: sideload incremental OTA on cuttlefish (that grows
system, shrinks vendor, and move vendor to group foo)
Test: verify that /cache/recovery/cc46ebfd04058569d0c6c1431c6af6c1328458e4
exists (sha1sum of "system")
Bug: 111801737
Change-Id: Ibdf6565bc1b60f3665c01739b4c95a85f0261ae5
When applying an OTA package onto the device in OTA mode,
if the recovery logs haven't been viewed, there is a chance
that /cache/recovery does not exist. Then, stash creation will
fail. Create stash directories recursively to avoid this error.
Test: without /cache/recovery, sideload the OTA on cuttlefish
Change-Id: I5cc01a067d866476a3594e795dcb5b15649e817b
Now it's less beneficial to inject I/O faults since we don't see many of
them. Remove the library that mocks I/O failures. And switch to android::base
I/O when possible.
Bug: 113032079
Test: unit tests pass
Change-Id: I9f2a92b7ba80f4da6ff9e2abc27f2680138f942c
The new command is part of the transfer.list and allows us to compute the hash
tree on non-ab devices.
The required arguments for the hash_tree computation are:
hash_tree_ranges
source_ranges
hash_algorithm
salt_hex
root_hash
Bug: 25170618
Test: unit tests pass; run simulator with compute_hash_tree
Change-Id: I8ff0d582cc8adabb8a060db7845f38b35b28e62c
They are doing exactly the same thing, except for the slightly different
error return value (1 vs -1).
int CacheSizeCheck(size_t bytes);
int MakeFreeSpaceOnCache(size_t bytes_needed);
This CL consolidates the two functions and uses bool as its return type.
// Checks whether /cache partition has at least 'bytes'-byte free space. Returns true immediately
// if so. Otherwise, it will try to free some space by removing older logs, checks again and
// returns the checking result.
bool CheckAndFreeSpaceOnCache(size_t bytes);
Test: Run recovery_unit_test and recovery_component_test on marlin.
Change-Id: I94a96934d2b18713f8f39ad5aa96a02c98d87963
Test: mmma -j bootable/recovery
Test: Run recovery_component_test and recovery_unit_test on marlin.
Change-Id: I4b240e3e771c387b9694be9c0f2f74e0265ab4cb
This will be used for testing purpose only, replacing the previously
used "fail", to intentionally abort an update.
As we're separating the logic between commands parsing and execution,
"abort" needs to be considered as a valid command during the parsing.
Test: recovery_unit_test and recovery_component_test on marlin.
Change-Id: I47c41c423e62c41cc8515fd92f3c5959be08da02
During block verification, load stash reads from the source blocks on
the device instead of the stashed files. And for interrupted update,
it's pretty common that the source blocks has already been overwritten
by subsequent commands.
In that case the hash printing is mostly useless. Moreover we should have
already printed the hash when the first update failed.
Bug: 80241799
Test: Unit tests pass. No longer prints mismatching stashed source blocks on retry.
Change-Id: I4effe684280b0325199f6cc4b2cc26e91295c2d7
This avoids leaving the created new data writer thread unjoined, in the
presence of transfer list parsing errors, or the early exit case on
`total_blocks == 0`.
Also fix a minor issue when dumping the errno on pthread_create error
(pthread_create returns the error number, as opposed to setting errno).
Test: Run recovery_component_test on marlin.
Change-Id: Icfac27fef0c64736eb8c76264da73c223b4960cb
As suggested by the style guide
(https://google.github.io/styleguide/cppguide.html#Reference_Arguments),
all parameters passed by reference must be labeled const. This CL moves
most of the non-const references in blockimg.cpp to pointers, except for
the CommandParameters& parameter in PerformCommand* functions, which
will be handled in separate CLs.
Test: mmma -j bootable/recovery
Test: Run recovery_component_test on marlin.
Change-Id: I84299208e9a1699f5381fb2228d4120f0c8dacb3
This is a stress test that instantiates multiple testcases that
interrupt a BBOTA update at every transfer command. Each testcase
asserts the last_command_file after the interruption, verifies the
update resumability, then resumes the update and asserts the updated
image.
The transfer list in the testcase covers most of the transfer commands
(stash/free/move/bsdiff/zero/new), as well as some special pattern like
having duplicate stash ids.
This CL also addresses one issue in the updater code, by resetting the
stash_map before each run. The stash map should be valid only per
block_image_verify/block_image_update run. Having leftover may cause
issue in subsequent runs, in particular when calling block_image_verify
after a previous run of block_image_{update,verify}.
Test: Run recovery_component_test on marlin.
Change-Id: I6f9a0368d194a754ce41a9c9819c6d5be2657248
None of the callers actually uses the value.
(Even in the earlier versions, e.g. the one in M, the value wasn't used
either.)
Test: Run recovery_component_test on marlin.
Change-Id: I53e61a1afa211f71a200889ed3aa4046763b46ea
Move the commands map parsing out of PerformBlockImageUpdate(), as this
can be done more easily by the caller.
The goal (not done in this CL) is to decouple command parsing logic from
the performers. This allows (a) focusing on the command logic in the
performer; and (b) extending BBOTA commands syntax separately.
Test: Run recovery_unit_test and recovery_component_test.
Change-Id: Ife202398a7660b152d84a3ba17b90f93d19c55f2
This avoids the signedness issue, and makes the code more readable.
Test: mmma bootable/recovery
Test: Run recovery_component_test.
Change-Id: I01c3a0357887cfd7c9d4aba4239ef650cfa18388
It no longer needs to be conditional, as we've removed the support for
BBOTA v1/v2.
Test: mmma bootable/recovery
Change-Id: I881de8afa38cc5b41b48e6d48d85170699ea5eb4
The last command file has the last executed stash command by convention.
So we should update the file after we actually finishes executing the command;
instead of after loading src/tgt.
Bug: 79756267
Test: unit tests pass
Change-Id: I4535b5836e7eb13b3abe3d02f9f362ec5e9ec969
We cannot skip "new" commands while resuming an update with
last_command_file, because they read in the data sequentially from the
package.
Bug: 69858743
Test: Interrupt an update that has new commands. Check the update log.
Change-Id: I05fb67246f5ea3ba2a0f78f10255c0d9b0bc2f5a
Prior to the change, the BBOTA updater would try to re-run all the
commands for a given partition on retry, including creating stashes
according to the list of commands. This could fail a retry when the
previous update had moved on to next stage, with leftovers in /cache.
This CL creates a marker on /cache upon successfully updating a
partition. The update commands will be skipped when trying to apply
updates on an updated partition. Note that the marker is expected to be
removed while doing a normal boot (in particular, handled by
RecoverySystem#handleAftermath). If that didn't happen, the updater
would also remove the marker before starting next fresh update.
Alternatively, we can achieve the same goal by changing the OTA script,
which needs to additionally compare the checksum against the target
build. For example,
range_sha1("/system", "ranges") == SHA1_of_updated_system ||
block_image_update("/system");
The downside is that we need to pay that cost on each install, as the
edify script doesn't support caching the result in a variable.
Bug: 79165963
Test: Simulate the process on device (by triggering a reboot while
updating /vendor). Check the update log and result.
Change-Id: I731031fa336133e1221b33edfc469969706e8091
We have a general need for overriding more paths (e.g. "/tmp"), mostly
for testing purpose. Rename CacheLocation to Paths, and use that to
manage TEMPORARY_{INSTALL,LOG}_FILE.
Test: mmma -j bootable/recovery
Test: recovery_component_test
Change-Id: Ia8ce8e5695df37ca434f13ac4d3206de1e8e9396
As they're accepting the SinkFn callback, it makes more sense to leave
the work to their callers.
Test: mmma -j bootable/recovery
Test: Run recovery_component_test on marlin.
Test: No other active user of the two functions.
Change-Id: I8d67b38ce037925442296f136b483e0c71983777
This class allows us to set the following locations dynamically:
cache_temp_source, last_command_file, stash_directory_base.
In the updater's main function, we reset the values of these variables
to their default locations in /cache; while we can set them to temp
files in unit tests or host simulation.
Test: unit tests pass
Change-Id: I528652650caa41373617ab055d41b1f1a4ec0f87
In BlockImageVerify some commands are undefined, e.g. "erase", "new",
"zero". And we should not error out if the corresponding function
pointer of these commands is null; otherwise we will fail the
verification.
The old code is:
if (cmd->f != nullptr && cmd->f(params) == -1)
return false;
In the last_command_file change the logic was wrongly modified to
if (cmd->f == nullptr)
return false;
...
if (cmd->f(params) == -1)
return false;
Test: sideload an incremental OTA twice on bullhead
Change-Id: I2561c365badb850da0e416629ccd61f0df7da5d7
When performing an update, save the index and cmdline of the current
command into the last command file if this command writes to the stash
either explicitly of implicitly. This mitigates the overhead to update
the last command file for every command. I ran a simple test on angler
and the time to update 1000 times is ~2.3 seconds.
Upon resuming an update, read the saved index first; then
1. In verification mode, check if all commands before the saved index
have already produced the expected target blocks. If not, delete the
last command file so that we will later resume the update from the
start of the transfer list.
2. In update mode, skip all commands before the saved index. Therefore,
we can avoid deleting stashes with duplicate id unintentionally;
and also speed up the update.
If an update succeeds or is unresumable, delete the last command file.
Bug: 69858743
Test: Unittest passed, apply a failed update with invalid cmd on angler
and check the last_command content, apply a failed update with invalid
source hash and last_command is deleted.
Change-Id: Ib60ba1e3c6d111d9f33097759b17dbcef97a37bf
The AbortFn() used to overwrite the error message, hiding the real
failure reported in ErrorAbort(). And we will miss the failure in
the script patterns like 'blockimageupdate() || abort()'
We will ensure there's one line break at the end of ErrorAbort's
error message; and append to the existing error message when calling
abort().
Test: Message from ErrorAbort shows up in the log
Change-Id: I3aebd06629c5129330250c7fe5e8cdead2ae85bc
It used to be "const Value*", but nullptr won't be a valid input.
Test: recovery_host_test; recovery_component_test
Change-Id: I904b5689ac3e64504088bf0544c9fb5d45a52243
We used to CHECK and abort on parsing errors. While it works fine for
the updater use case (because recovery starts updater in a forked
process and collects the process exit code), it's difficult for other
clients to use RangeSet as a library (e.g. update_verifier).
This CL switches the aborts to returning empty RangeSet instead. Callers
need to check the parsing results explicitly.
The CL also separates RangeSet::PushBack() into a function, and moves
SortedRangeSet::Clear() into RangeSet.
Test: recovery_unit_test
Test: Sideload an OTA package with the new updater on angler.
Test: Sideload an OTA package with injected range string errors. The
updater aborts from the explicit checks.
Change-Id: If2b7f6f41dc93af917a21c7877a83e98dc3fd016
The thread to receive new data may still be alive after we exit
PerformBlockImageUpdate() upon failures. This caused memory corruption
when we run the unittest repeatedly. Set the receiver_available flag
to false and make sure the receiver exits when the update fails.
Bug: 65430057
Test: unittests passed with tsan
Change-Id: Icb232d13fb96c78262249ffbd29cdbe5b77f1fce
Also drop the "bootable/recovery" path in LOCAL_C_INCLUDES from
applypatch modules.
Test: lunch aosp_{angler,bullhead,fugu,dragon,sailfish}-userdebug;
mmma bootable/recovery
Change-Id: Idd602a796894f971ee4f8fa3eafe36c42d9de986
This reverts commit 26436d6d60 to re-land
"Move error_code.h into otautil.".
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".
This CL needs to land with device-specific module changes (e.g. adding
the dependency on libotautil).
Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
mmma bootable/recovery
Change-Id: If193241801af2dae73eccd31ce57cd2b81c9fd96
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".
Test: mmma bootable/recovery
Change-Id: Ia4649789cef2aaeb2785483660e9ea5a8b389c62
1. Move the declaration of the Image classes to the header file to make
testing easier.
2. Also move rangeset.h to bootable/recovery to allow access in imgdiff.
Test: recovery component test
Change-Id: I68a863e60a3f2e7ae46ee48f48eb15391f5f4330
receive_new_data may exit too early if the zip processor has sent all
the raw data. As a result, the last few 'new' commands will fail even
though the brotli decoder has more output in its buffer.
Restruct the code so that 'NewThreadInfo' owns the decoder state solely;
and receive_brotli_new_data is responsible for the decompression.
Also reduce the test data size to 100 blocks to avoid the test timeout.
Bug: 63802629
Test: recovery_component_test. on bullhead, apply full updates with and
w/o brotli compressed entries, apply an incremental update.
Change-Id: I9442f2536b74e48dbf7eeb062a8539c82c6dab47
Add a new writer that can decode the brotli-compressed system/vendor
new data stored in the OTA zip.
Brotli generally gives better compression rate at the cost of slightly
increased time consumption. The patch.dat is already compressed
by BZ; so there's no point to further compress it.
For the given 1.9G bullhead system image:
Size: 875M -> 787M; ~10% reduction of package size.
Time: 147s -> 153s; ~4% increase of the block_image_update execution time.
(I guess I/O takes much longer time than decompression.)
Also it takes 4 minutes to compress the system image on my local
machine, 3 more minutes than zip.
Test: recovery tests pass && apply a full OTA with brotli compressed
system/vendor.new.dat on bullhead
Change-Id: I232335ebf662a9c55579ca073ad45265700a621e
We have seen one case when bspatch failed likely due to patch
corruption. Since the package has passed verification before, we want
to reboot and retry the patch command again since there's no
alternative for users.
We won't delete the stash before reboot, and the src has passed SHA1
check. If there's an error on the patch, it will fail the package
verification during retry.
Bug: 37855643
Test: angler reboots and retries the update when bspatch fails.
Change-Id: I2ebac9621bd1f0649bb301b9a28a0dd079ed4e1d
Right now the update stuck in a deadlock if there's less new data than
expection. Add some checkers and abort the update if such case happens.
Also add a corresponding test.
Bug: 36787146
Test: update aborts correctly on bullhead && recovery_component_test passes
Change-Id: I914e4a2a4cf157b99ef2fc65bd21c6981e38ca47
This CL makes the following changes to RangeSet:
- Uses std::pair<size_t, size_t> to represent a Range;
- Uses std::vector<Range> to represent a RangeSet;
- Provides const iterators (forward and reverse);
- Provides const accessor;
- 'blocks()' returns the number of blocks (formerly 'size');
- 'size()' returns the number of Range's (formerly 'count').
Test: recovery_unit_test
Test: Apply an incremental update with the new updater.
Change-Id: Ia1fbb343370a152e1f7aa050cf914c2da09b1396
We don't need to take raw pointers out of the parsed arguments.
std::unique_ptr handles the dereferencing automatically.
Test: mmma bootable/recovery
Change-Id: I1beabf6e04dc350bdad7b36cee5fb345c82b28f2
Also move RangeSet into a header file to make it testable, and add unit
tests.
In RangeSet::Parse() (the former parse_range()), use libbase logging to
do assertions. This has the same effect as the previous
exit(EXIT_FAILURE) to terminate the updater process and abort an update.
The difference lies in the exit status code (i.e. WEXITSTATUS(status) in
install.cpp), which changes from 1 (i.e. EXIT_FAILURE) to 0.
Test: recovery_unit_test
Test: Apply an incremental update with the new updater.
Change-Id: Ie8393c78b0d8ae0fd5f0ca0646d871308d71fff0
