Commit graph

79 commits

Author SHA1 Message Date
Tianjie Xu
57dd961995 Move Image/ImageChunk/PatchChunk declaration into header files
1. Move the declaration of the Image classes to the header file to make
testing easier.
2. Also move rangeset.h to bootable/recovery to allow access in imgdiff.

Test: recovery component test
Change-Id: I68a863e60a3f2e7ae46ee48f48eb15391f5f4330
2017-08-18 17:56:22 -07:00
Tianjie Xu
6ed175d541 Fix a case when brotli writer fails to write last few blocks of data
receive_new_data may exit too early if the zip processor has sent all
the raw data. As a result, the last few 'new' commands will fail even
though the brotli decoder has more output in its buffer.

Restruct the code so that 'NewThreadInfo' owns the decoder state solely;
and receive_brotli_new_data is responsible for the decompression.

Also reduce the test data size to 100 blocks to avoid the test timeout.

Bug: 63802629
Test: recovery_component_test. on bullhead, apply full updates with and
w/o brotli compressed entries, apply an incremental update.

Change-Id: I9442f2536b74e48dbf7eeb062a8539c82c6dab47
2017-07-21 11:44:00 -07:00
Tianjie Xu
107a34f9fc Add support to decompress brotli compressed new data
Add a new writer that can decode the brotli-compressed system/vendor
new data stored in the OTA zip.

Brotli generally gives better compression rate at the cost of slightly
increased time consumption. The patch.dat is already compressed
by BZ; so there's no point to further compress it.

For the given 1.9G bullhead system image:
Size: 875M -> 787M; ~10% reduction of package size.
Time: 147s -> 153s; ~4% increase of the block_image_update execution time.
(I guess I/O takes much longer time than decompression.)

Also it takes 4 minutes to compress the system image on my local
machine, 3 more minutes than zip.

Test: recovery tests pass && apply a full OTA with brotli compressed
system/vendor.new.dat on bullhead

Change-Id: I232335ebf662a9c55579ca073ad45265700a621e
2017-07-07 16:08:18 -07:00
Tianjie Xu
6957555e29 Retry the update if ApplyBSDiffPatch | ApplyImagePatch fails
We have seen one case when bspatch failed likely due to patch
corruption. Since the package has passed verification before, we want
to reboot and retry the patch command again since there's no
alternative for users.

We won't delete the stash before reboot, and the src has passed SHA1
check. If there's an error on the patch, it will fail the package
verification during retry.

Bug: 37855643
Test: angler reboots and retries the update when bspatch fails.
Change-Id: I2ebac9621bd1f0649bb301b9a28a0dd079ed4e1d
2017-05-23 17:36:56 -07:00
Tianjie Xu
3a8d98dd90 Abort the update if there's not enough new data
Right now the update stuck in a deadlock if there's less new data than
expection. Add some checkers and abort the update if such case happens.
Also add a corresponding test.

Bug: 36787146
Test: update aborts correctly on bullhead && recovery_component_test passes
Change-Id: I914e4a2a4cf157b99ef2fc65bd21c6981e38ca47
2017-04-07 17:19:46 -07:00
Tao Bao
bf5b77dbf7 Change the internal representation in RangeSet.
This CL makes the following changes to RangeSet:
 - Uses std::pair<size_t, size_t> to represent a Range;
 - Uses std::vector<Range> to represent a RangeSet;
 - Provides const iterators (forward and reverse);
 - Provides const accessor;
 - 'blocks()' returns the number of blocks (formerly 'size');
 - 'size()' returns the number of Range's (formerly 'count').

Test: recovery_unit_test
Test: Apply an incremental update with the new updater.
Change-Id: Ia1fbb343370a152e1f7aa050cf914c2da09b1396
2017-04-03 09:22:48 -07:00
Tao Bao
c97edcb4f4 updater: Keep the parsed parameters in std::unique_ptr.
We don't need to take raw pointers out of the parsed arguments.
std::unique_ptr handles the dereferencing automatically.

Test: mmma bootable/recovery
Change-Id: I1beabf6e04dc350bdad7b36cee5fb345c82b28f2
2017-03-31 12:24:22 -07:00
Tao Bao
8f23757ad4 Move parse_range() and range_overlaps() into RangeSet.
Also move RangeSet into a header file to make it testable, and add unit
tests.

In RangeSet::Parse() (the former parse_range()), use libbase logging to
do assertions. This has the same effect as the previous
exit(EXIT_FAILURE) to terminate the updater process and abort an update.
The difference lies in the exit status code (i.e. WEXITSTATUS(status) in
install.cpp), which changes from 1 (i.e. EXIT_FAILURE) to 0.

Test: recovery_unit_test
Test: Apply an incremental update with the new updater.
Change-Id: Ie8393c78b0d8ae0fd5f0ca0646d871308d71fff0
2017-03-31 00:04:56 -07:00
Tao Bao
65c065b4bb Merge "updater: Move RangeSinkWrite into RangeSinkState." 2017-03-30 23:56:14 +00:00
Tao Bao
0bbc764bbe updater: Don't append newline when calling uiPrint().
LOG(INFO) already appends a newline. Don't print redundant newline.

Test: No extra blank lines when calling ui_print(). And on-screen UI
      shows the same.
Change-Id: I74e9a8504a7146a6cb3dae02fe2406d0dd54069b
2017-03-30 10:28:37 -07:00
Tao Bao
60a70afc0a updater: Move RangeSinkWrite into RangeSinkState.
Then rename RangeSinkState to RangeSinkWriter. RangeSinkWriter reads
data from the given FD, and writes them to the desination RangeSet.

Test: Apply an incremental with the new updater.
Change-Id: I5e3ab6fc082efa1726562c55b56e2d418fe4acaf
2017-03-29 11:31:41 -07:00
Tao Bao
c0e1c46a70 applypatch: Let Apply{BSDiff,Image}Patch accept std::function.
Test: mmma bootable/recovery system/update_engine
Test: recovery_component_test
Change-Id: I93c2caa87bf94a53509bb37f98f2c02bcadb6f5c
2017-03-28 10:14:53 -07:00
Tao Bao
f7eb760fe7 applypatch: Change the ssize_t length parameters to size_t.
Mostly for applypatch family APIs like ApplyBSDiffPatch() and
ApplyImagePatch(). Changing to size_t doesn't indicate they would
necessarily work with very large size_t (e.g. > ssize_t), just
similar to write(2). But otherwise accepting negative length doesn't
make much sense.

Also change the return type of SinkFn from ssize_t to size_t. Callers
tell a successful sink by comparing the number of written bytes against
the desired value. Negative return values like -1 are not needed. This
also makes it consistent with bsdiff::bspatch interface.

Test: recovery_component_test
Test: Apply an incremental with the new updater.
Change-Id: I7ff1615203a5c9854134f75d019e266f4ea6e714
2017-03-28 10:13:38 -07:00
Tao Bao
d2aecd465b updater: Clean up LoadSrcTgtVersion2().
Rename to LoadSourceBlocks() by moving the target blocks parsing part
into the caller. This allows detecting whether the target blocks have
already had the expected data before loading the source blocks. It
doesn't affect anything when applying an update package for the first
time, but it skips loading the unneeded source blocks when resuming an
update. It additionally avoids unnecessarily dumping the "corrupt"
source/stash blocks when resuming an update.

Bug: 33694730
Test: Apply an incremental update with the new updater.
Test: Resume an incremental update with the new updater.
Change-Id: I794fd0d1045be7b3b7f8619285dc0dade01398d0
2017-03-27 16:21:55 -07:00
Tao Bao
5212a00588 Merge "updater: Remove some redundant arguments." 2017-03-25 23:11:37 +00:00
Tao Bao
5479506c9c Merge "updater: Drop the support for BBOTA v1 and v2." 2017-03-24 23:06:56 +00:00
Tao Bao
110155a604 Merge "Removed C-style casts" 2017-03-24 20:28:00 +00:00
Tao Bao
bcf464988e updater: Remove some redundant arguments.
Clean up a few functions that take CommandParameters& as the first
parameter. We don't need to take duplicate arguments if they always come
from CommandParameters. This redundancy came from the point we replaced
strtok()s (commit baad2d454d).

Test: Apply an incremental update with the new updater.
Change-Id: I2912b8ce6bc7580bf7f566e125f12270e679e155
2017-03-24 00:34:36 -07:00
Tao Bao
3356777cae updater: Drop the support for BBOTA v1 and v2.
The script support for BBOTA v1 and v2 has been dropped in commit
8fad03e7712082eb880ffaaffb69eb13252ce220 (platform/build).

Bug: 33694730
Test: Apply an incremental with the new updater.
Test: recovery_component_test
Change-Id: I038b1bf8d10f030cab8ec0aa6ee565c5a9545dfd
2017-03-23 23:58:02 -07:00
Mikhail Lappo
20791bdcd7 Removed C-style casts
In c++ code would be cleaner to use
c++ retinterpret cast instead of old
c-style notation

Change-Id: Ibeef5e0c374addf108c0a8876a6be45063d8e396
2017-03-23 21:52:04 +01:00
Mikhail Lappo
bb8bce932e Redundant checking of STL container element
As of C++ specification size_type erase( const key_type& key );
removes the element (if one exists). There is no need to perform
the check twice.

Change-Id: I4b057c08526abc7c2a483a60f9e166e4d8f56a74
2017-03-23 17:20:47 +01:00
Tianjie Xu
c444732540 Remove malloc in edify functions
And switch them to std::vector & std::unique_ptr

Bug: 32117870
Test: recovery tests passed on sailfish
Change-Id: I5a45951c4bdf895be311d6d760e52e7a1b0798c3
2017-03-22 14:20:57 -07:00
Tianjie Xu
a946b9e241 Fix the permission of stashed blocks created by updater
Our updater created the stashes with root permission. This causes an
access denial when the RecoverySystem service tries to clean up these
blocks after a failing update. As a result, the subsequent OTA updates
may fail due to insufficient cache space.

Bug: 36457133
Test: stashed blocks cleaned successfully after reboot
Change-Id: If0ca99638cdfa1033646f29d9cc92b5ff1bacac1
2017-03-21 17:40:01 -07:00
Tianjie Xu
2cd36ba522 Revert "Revert "Print SHA-1 in hex for corrupted blocks""
This reverts commit 90eff6a340.
Also fix the bug where stashed blocks are not freed.

Bug: 21124445
Test: Previous failed update succeeded on bullhead
Change-Id: I23d232331a2beb51b6dcc82c957c87bc247d0268
2017-03-16 23:33:01 -07:00
Tao Bao
ec8272f6e3 updater: Minor clean up to EnumerateStash().
Test: Apply an incremental BBOTA package with the new updater.
Test: Resume an interrupted BBOTA (so it cleans up the partial stash).
Change-Id: I620cc57ee6366845bcffbc19210f7a01e2196052
2017-03-16 00:00:46 -07:00
Tao Bao
90eff6a340 Revert "Print SHA-1 in hex for corrupted blocks"
This reverts commit bb0cd75a0e.

Broke the 'free' command that deletes a stash.

Bug: 36242722
Test: The previously failed incremental applies successfully.
Change-Id: I1237cb0a33adfbeea57e0465b629704862ba13aa
2017-03-15 09:57:06 -07:00
Tianjie Xu
bb0cd75a0e Print SHA-1 in hex for corrupted blocks
It will be helpful for debug if we know which blocks are corrupted after
a verification failure. This CL prints the SHA-1 for each source block
in a transfer command if these blocks don't have an expected hash. And
along with the correct SHA-1, we will catch the corrupted blocks.

Bug: 21124445
Test: Printed the mismatched SHA-1 for bullhead during an update.
Change-Id: I683d4bdaf9a335035045b3f532b3a265b2fcbbfc
2017-03-13 11:32:56 -07:00
Tao Bao
3da880156b Replace _exit(-1) with _exit(EXIT_FAILURE).
-1 is not a valid exit status.

Also replace a few exit(1) with exit(EXIT_FAILURE).

Test: mmma bootable/recovery
Change-Id: I4596c8328b770bf95acccc06a4401bd5cabd4bfd
2017-02-03 14:11:11 -08:00
Tao Bao
91f74c82cd Merge "updater: Clean up CreateStash()." 2017-01-05 21:14:43 +00:00
Tao Bao
c844c06faf updater: Refactor parse_range().
Returning the parsed RangeSet directly (as opposed to using some pointer
parameter) to make the code cleaner.

Test: Apply an incremental with the new updater.
Change-Id: I8c99e701f189eb6a3eacc0d647e5a3a85fbeb3eb
2017-01-03 17:51:30 -08:00
Tao Bao
5141221211 updater: Clean up CreateStash().
Change the stash size computation from int to size_t.

Test: Apply an incremental BBOTA with the new updater.
Change-Id: Ib45b71b826fec6aa0ffafc67c17735825634eae0
2016-12-28 16:22:16 -08:00
Tao Bao
5354f60f2b updater: Fix the operator order.
Shift operator ("<<") has a higher precedence level than ternary
operator ("?").

Test: BBOTA update log says "performing update" as opposed to
      "performing 0".

Change-Id: I0cf60cbfc11415e94f1f9f6effe75f14d13a1874
2016-12-14 11:34:49 -08:00
Tao Bao
039f2da3e4 updater: Switch to libbase logging.
Test: Build an updater into a package and apply it on device.
Change-Id: I289b5768e9b1e44ef78e0479c64dbaa36fb1a685
2016-11-30 23:44:41 -08:00
Tianjie Xu
5fe280ac96 Cleanup ReadArgs & ReadValueArgs usage
ReadArgs will switch to using std::string and std::unique_ptr. Also
cleanup the callers.

Test: mma & component test passed.
Change-Id: I4724406ae6c0c134a27bbd1cdd24ad5d343b2a3b
2016-11-01 15:03:06 -07:00
Tianjie Xu
8cf5c8f60f Replace minzip with libziparchive
Clean up the duplicated codes that handle the zip files in
bootable/recovery; and rename the library of the remaining
utility functions to libotautil.

Test: Update package installed successfully on angler.
Bug: 19472796

Change-Id: Iea8962fcf3004473cb0322b6bb3a9ea3ca7f679e
2016-10-17 17:41:51 -07:00
Tianjie Xu
aced5d9e4e Change StringValue to use std::string
Changing the field of 'Value' in edify to std::string from char*.
Meanwhile cleaning up the users of 'Value' and switching them to
cpp style.

Test: compontent tests passed.
Bug: 31713288

Change-Id: Iec5a7d601b1e4ca40935bf1c70d325dafecec235
2016-10-15 01:18:23 +00:00
Tao Bao
0c7839ac14 Refactor libupdater into a seperate module.
So that we can write native tests for updater functions. This CL adds a
testcase for getprop() function.

Test: mmma bootable/recovery; Run recovery_component_test on device.

Change-Id: Iff4c1ff63c5c71aded2f9686fed6b71cc298c228
2016-10-10 16:49:50 -07:00
Tianjie Xu
71e182bc38 Check an edge case when read(2) returns 0
We might end up in an infinite loop if read(2) reached EOF unexpectedly.
The problematic code in uncrypt mentioned in the bug has been fixed
by switching to libbase ReadFully(). So I grepped through the recovery
code and fixed some other occurences of the issue.

Bug: 31073201
Change-Id: Ib867029158ba23363b8f85d61c25058a635c5a6b
2016-08-31 19:11:41 -07:00
Tianjie Xu
93110c767f Check the results from applypatch
am: 31f8cc84cf

Change-Id: I2ea241fb3e688f5f8b0558d638dd3ed502223f97
2016-06-16 18:08:59 +00:00
Tianjie Xu
31f8cc84cf Check the results from applypatch
Check the results from applypatch in PerformCommandDiff; and abort the
update on failure.

Bug:29339536
Change-Id: I5087d79ba532b54250f4c17560524255c8a4fabc
2016-06-15 23:26:59 +00:00
Tianjie Xu
64f46fb16c resolve merge conflicts of 7ce287d to nyc-dev-plus-aosp
Change-Id: I2194d1170281f58eb508f2ef63b39c8729125f76
2016-06-03 15:44:52 -07:00
Tianjie Xu
7ce287d432 Call ioctl before each write on retry
If the update is a retry, ioctl(BLKDISCARD) the destination blocks before
writing to these blocks.

Bug: 28990135
Change-Id: I1e703808e68ebb1292cd66afd76be8fd6946ee59
2016-06-03 12:12:50 -07:00
Tianjie Xu
84478e8823 resolve merge conflicts of 50f6417 to nyc-dev-plus-aosp
Change-Id: I42c127f7946e678acf6596f6352f090abc0ca019
2016-05-23 12:24:28 -07:00
Tianjie Xu
162558382b Allow recovery to return error codes
Write error code, cause code, and retry count into last_install. So we
can have more information about the reason of a failed OTA.

Example of new last_install:
@/cache/recovery/block.map     package name
0                              install result
retry: 1                       retry count (new)
error: 30                      error code (new)
cause: 12                      error cause (new)

Details in:
go/android-ota-errorcode

Bug: 28471955
Change-Id: I00e7153c821e7355c1be81a86c7f228108f3dc37
2016-05-20 13:56:53 -07:00
Tianjie Xu
142b864ba1 Add time and I/O info to last_install
am: dd874b1c87

* commit 'dd874b1c87eb04f28db0db2629df0adde568a74c':
  Add time and I/O info to last_install

Change-Id: I02aa858d5ce488d3acbf5400811e2565cf7d9c75
2016-05-18 18:44:31 +00:00
Tianjie Xu
dd874b1c87 Add time and I/O info to last_install
One example of last_install is:

/sideload/package.zip
1
time_total: 101
bytes_written_system: 14574000
bytes_stashed_system: 100
bytes_written_vendor: 5107400
bytes_stashed_vendor: 0

Bug: 28658632
Change-Id: I4bf79ea71a609068d38fbce6b41bcb892524aa7a
2016-05-16 14:54:37 -07:00
Chih-hung Hsieh
fafa6a6531 Merge "Fix google-explicit-constructor warnings." 2016-04-29 22:26:41 +00:00
Chih-Hung Hsieh
49c5c79df1 Fix google-explicit-constructor warnings.
Bug: 28341362
Change-Id: I5b35ae16c069e7e9229e66963386f322bd808af1
2016-04-29 14:16:35 -07:00
Tao Bao
730646199b updater: Don't zero out CommandParameters with memset(3).
[1] switched a few things to android::base::unique_fd including
CommandParameters.fd. However, we were using memset(3) to zero out the
struct, which effectively assigned unique_fd(0) to fd. When it called
fd.reset(), file descriptor 0 was unintentionally closed. When FD 0 was
later reassigned via open(2), it led to lseek(2) errors: "Bad file
descriptor".

This CL switches to using braced-init (i.e. '= {}') instead, so that the
default constructor unique_fd(-1) would be called.

[1]: commit bcabd09293

Bug: 28391985
Change-Id: If1f99932b15552714c399e65c8b80550344b758a
2016-04-26 17:36:13 -07:00
Elliott Hughes
bcabd09293 Switch to <android-base/unique_fd.h>.
Change-Id: I13ba3f40bd52b5f3e3fe9002a45a9a8630040129
2016-03-29 08:18:34 -07:00