Commit graph

309 commits

Author SHA1 Message Date
Jaegeuk Kim
1a8bb0f542 recovery: format f2fs with encrypt/quota
Change-Id: Ia393b7b78b45f09964449ec0e255aa26bb3b8ddf
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-11-13 14:05:00 -08:00
Tao Bao
1e0941f4f6 applypatch: Change the patch parameter to const Value& in Apply{BSDiff,Image}Patch.
It used to be "const Value*", but nullptr won't be a valid input.

Test: recovery_host_test; recovery_component_test
Change-Id: I904b5689ac3e64504088bf0544c9fb5d45a52243
2017-11-10 12:18:34 -08:00
Tao Bao
6798315327 otautil: Remove the aborts in RangeSet::Parse().
We used to CHECK and abort on parsing errors. While it works fine for
the updater use case (because recovery starts updater in a forked
process and collects the process exit code), it's difficult for other
clients to use RangeSet as a library (e.g. update_verifier).

This CL switches the aborts to returning empty RangeSet instead. Callers
need to check the parsing results explicitly.

The CL also separates RangeSet::PushBack() into a function, and moves
SortedRangeSet::Clear() into RangeSet.

Test: recovery_unit_test
Test: Sideload an OTA package with the new updater on angler.
Test: Sideload an OTA package with injected range string errors. The
      updater aborts from the explicit checks.
Change-Id: If2b7f6f41dc93af917a21c7877a83e98dc3fd016
2017-11-07 12:50:02 -08:00
Jin Qian
502fd1c5e8 recovery: remove make_ext4fs from updater
Bug: 64395169
Change-Id: I6f6a4f82b225435c6ad5c828e110fa135e6f7579
2017-11-03 13:54:59 -07:00
Tianjie Xu
5450c84ba4 Finish the new data receiver when update fails
The thread to receive new data may still be alive after we exit
PerformBlockImageUpdate() upon failures. This caused memory corruption
when we run the unittest repeatedly. Set the receiver_available flag
to false and make sure the receiver exits when the update fails.

Bug: 65430057
Test: unittests passed with tsan
Change-Id: Icb232d13fb96c78262249ffbd29cdbe5b77f1fce
2017-10-20 11:14:56 -07:00
Tao Bao
99f0d9e52b Drop -Wno-unused-parameter.
The only one left is libedify. Will handle that in a separate CL.

Test: mmma bootable/recovery
Change-Id: I732a5f85229da90fd767bee2e46c5c95f529c396
2017-10-11 16:56:12 -07:00
Tao Bao
09e468f84c Move rangeset.h and print_sha1.h into otautil.
Also drop the "bootable/recovery" path in LOCAL_C_INCLUDES from
applypatch modules.

Test: lunch aosp_{angler,bullhead,fugu,dragon,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: Idd602a796894f971ee4f8fa3eafe36c42d9de986
2017-10-10 15:52:11 -07:00
Tao Bao
1fc5bf353a Revert "Revert "Move error_code.h into otautil.""
This reverts commit 26436d6d60 to re-land
"Move error_code.h into otautil.".

This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".

This CL needs to land with device-specific module changes (e.g. adding
the dependency on libotautil).

Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: If193241801af2dae73eccd31ce57cd2b81c9fd96
2017-10-09 14:07:54 -07:00
Tao Bao
0bf20d5133 Don't include "error_code.h" in edify/expr.h.
Use forward declartion to avoid pull in the module that contains
error_code.h (trying to move it into libotautil). Otherwise all the
modules that include "edify/expr.h" need to depend on the module that
exports error_code.h.

.cpp sources should include "error_code.h" explicitly to use the enums.

Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: Ic82db2746c7deb866e8cdfb3c57e0b1ecc71c4dc
2017-10-05 12:46:18 -07:00
Tao Bao
26436d6d60 Revert "Move error_code.h into otautil."
This reverts commit 623fe7e701.

Reason for revert: Need to address device-specific modules.

Change-Id: Ib7a4191e7f193dfff49b02d3de76dda856800251
2017-10-05 17:16:31 +00:00
Tao Bao
623fe7e701 Move error_code.h into otautil.
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".

Test: mmma bootable/recovery
Change-Id: Ia4649789cef2aaeb2785483660e9ea5a8b389c62
2017-10-04 08:55:24 -07:00
Tao Bao
d33b2f86b7 otafault: Move headers under otafault/.
Test: mmma bootable/recovery
Change-Id: I3ceb72f703c7c2857d656c137d71baa1fccd8238
2017-09-29 10:29:53 -07:00
Tianjie Xu
c89d1e7e2a Turn on -Wall for recovery modules
Turn on -Wall for all modules. Also remove the obsolete file_cmp() in
apply_patch test and now() in wear_ui.

The only exception is lib_edify due to the unused functions in the
intermediate cpp files generated from the lex files. It will be handled
in a seperate CL.

Bug: 64939312
Test: mma, unit tests pass
Change-Id: Ic53f76b60b6401ab20db3d98130d674c08e3702f
2017-08-28 21:56:33 -07:00
Tianjie Xu
b127fddf09 Merge "Move Image/ImageChunk/PatchChunk declaration into header files"
am: b4bc57ed39

Change-Id: If254ed9e24bc0cafa19db9766ed36643ca0fed49
2017-08-19 04:27:34 +00:00
Tianjie Xu
57dd961995 Move Image/ImageChunk/PatchChunk declaration into header files
1. Move the declaration of the Image classes to the header file to make
testing easier.
2. Also move rangeset.h to bootable/recovery to allow access in imgdiff.

Test: recovery component test
Change-Id: I68a863e60a3f2e7ae46ee48f48eb15391f5f4330
2017-08-18 17:56:22 -07:00
Tianjie Xu
fbd4b10310 Merge "Add implemention of SortedRangeSet"
am: 64cba55fbc

Change-Id: I2174e4f55c85fe57014b31625dbc2d06e41350be
2017-08-01 00:31:20 +00:00
Tianjie Xu
b9e7fc7fa9 Add implemention of SortedRangeSet
This is useful in imgdiff to maintain the block ranges of
splitted source image.

Bug: 34220646
Test: mma && unit tests pass
Change-Id: I6427f2ea50f0e3b0aa3dd01880ec0206679b7429
2017-07-31 15:13:55 -07:00
Tao Bao
ac634e3c9d Merge "updater: Remove dead make_parents()."
am: f5396eb8b2

Change-Id: Ief0557813324d49c5da64265dc49ab757ed3ae3f
2017-07-25 19:09:40 +00:00
Tao Bao
5902691764 updater: Remove dead make_parents().
Its former callers in RenameFn() and SymlinkFn() have been removed in
commit 63d786cf22.

Test: mmma -j bootable/recovery
Change-Id: I26ed126202554fc5840811ec7ae162da70593213
2017-07-25 08:42:52 -07:00
Tao Bao
248b4f9dca Merge "Remove the obsolete reference to /file_contexts."
am: 6b09b895be

Change-Id: I2b26ced9ffeb278ce01ff2ca7be4057b0ab6c61c
2017-07-23 04:03:34 +00:00
Tao Bao
338be53ed3 Remove the obsolete reference to /file_contexts.
This file no longer exists:
- /file_contexts has been split into plat_file_contexts and
  nonplat_file_contexts since commit
  b236eb6ca204cefcb926e19bd5682f9dcad4021d (system/sepolicy).
- It was named /file_contexts.bin prior to the split.

'-S file_contexts' is also no longer required by e2fsdroid, since commit
2fff6fb036cbbb6dedd7da3d208b312a9038a5ce (external/e2fsprogs). It will
load the file contexts via libselinux.

Test: Trigger the path by performing a data wipe for converting to FBE.
Change-Id: I179939da409e5c0415ae0ea0bf5ddb23f9e6331e
(cherry picked from commit 7af933b6a6)
2017-07-22 16:16:21 -07:00
Tianjie Xu
80acaab52f Merge "Fix a case when brotli writer fails to write last few blocks of data"
am: e45c8f0057

Change-Id: I337e8ec26f59a5245ab299080d7251331823e2da
2017-07-21 21:04:17 +00:00
Tianjie Xu
e45c8f0057 Merge "Fix a case when brotli writer fails to write last few blocks of data" 2017-07-21 20:59:24 +00:00
Jin Qian
9203641742 Merge "recovery: replace make_ext4 with e2fsprogs"
am: f7c00ddaf6

Change-Id: I17167b59242c709b18d09fccb52058b0bc25bf09
2017-07-21 19:06:06 +00:00
Tianjie Xu
6ed175d541 Fix a case when brotli writer fails to write last few blocks of data
receive_new_data may exit too early if the zip processor has sent all
the raw data. As a result, the last few 'new' commands will fail even
though the brotli decoder has more output in its buffer.

Restruct the code so that 'NewThreadInfo' owns the decoder state solely;
and receive_brotli_new_data is responsible for the decompression.

Also reduce the test data size to 100 blocks to avoid the test timeout.

Bug: 63802629
Test: recovery_component_test. on bullhead, apply full updates with and
w/o brotli compressed entries, apply an incremental update.

Change-Id: I9442f2536b74e48dbf7eeb062a8539c82c6dab47
2017-07-21 11:44:00 -07:00
Jin Qian
ded2dac082 recovery: replace make_ext4 with e2fsprogs
Execute mke2fs to create empty ext4 filesystem.
Execute e2fsdroid to add files to filesystem.

Test: enter recovery mode and wipe data
Bug: 35219933
Change-Id: I10a9f4c1f4754ad864b2df45b1f879180ab33876
(cherry picked from commit ac31808cd3)
2017-07-20 11:42:17 -07:00
Tianjie Xu
43bdf6cad6 Merge "Add support to decompress brotli compressed new data"
am: 918e6ea1b2

Change-Id: I4fd9cea71716ad1574ecb4bb7f612bc8734711c5
2017-07-10 22:33:37 +00:00
Tianjie Xu
107a34f9fc Add support to decompress brotli compressed new data
Add a new writer that can decode the brotli-compressed system/vendor
new data stored in the OTA zip.

Brotli generally gives better compression rate at the cost of slightly
increased time consumption. The patch.dat is already compressed
by BZ; so there's no point to further compress it.

For the given 1.9G bullhead system image:
Size: 875M -> 787M; ~10% reduction of package size.
Time: 147s -> 153s; ~4% increase of the block_image_update execution time.
(I guess I/O takes much longer time than decompression.)

Also it takes 4 minutes to compress the system image on my local
machine, 3 more minutes than zip.

Test: recovery tests pass && apply a full OTA with brotli compressed
system/vendor.new.dat on bullhead

Change-Id: I232335ebf662a9c55579ca073ad45265700a621e
2017-07-07 16:08:18 -07:00
Jeff Vander Stoep
eba11fa574 Fix "No file_contexts" warning
am: e35926e1af

Change-Id: Ia050561286c30d8198f3185da9e3cd31372b1d79
2017-06-16 02:57:57 +00:00
Jeff Vander Stoep
e35926e1af Fix "No file_contexts" warning
Fixed by Loading the file_contexts specified in libselinux, whereas
previously recovery loaded /file_contexts which no longer exists.

Bug: 62587423
Test: build and flash recovery on Angler. Warning is gone.
Test: Wipe data and cache.
Test: sideload OTA
Change-Id: I11581c878b860ac5f412e6e8e7acde811f37870f
(cherry picked from commit 2330dd8733)
2017-06-15 21:24:29 +00:00
Tianjie Xu
99e7216907 Merge "kill package_extract_dir" 2017-05-31 21:49:03 +00:00
Tianjie Xu
6957555e29 Retry the update if ApplyBSDiffPatch | ApplyImagePatch fails
We have seen one case when bspatch failed likely due to patch
corruption. Since the package has passed verification before, we want
to reboot and retry the patch command again since there's no
alternative for users.

We won't delete the stash before reboot, and the src has passed SHA1
check. If there's an error on the patch, it will fail the package
verification during retry.

Bug: 37855643
Test: angler reboots and retries the update when bspatch fails.
Change-Id: I2ebac9621bd1f0649bb301b9a28a0dd079ed4e1d
2017-05-23 17:36:56 -07:00
Tianjie Xu
53c38b1538 kill package_extract_dir
It's only used by file-based OTA which has been deprecated for O.

Test: mma
Change-Id: I439c93155ca94554d827142c99aa6c0845cc7561
2017-05-23 17:09:45 -07:00
Tao Bao
397a8137a0 updater: Update the mkfs.f2fs argument to match f2fs-tools 1.8.0.
Commit adeb41a8c0 has switched the
argument for recovery. This CL handles the case for updater.

Note that there's a chance the updater may run against the old
recovery (and f2fs 1.4.1 binary). Not sending a 0-sector argument to
f2fs 1.4.1 also works.

Bug: 37758867
Test: Make an OTA package that calls format f2fs, with mkfs.f2fs 1.8.0
      and 1.4.1 binaries respectively.
Change-Id: I4d4bbe8c57544d1c514b7aa37fbf22a0aab14e2c
2017-05-12 12:09:16 -07:00
Tianjie Xu
89394632b1 Merge "Add a default error code when updater script aborts" 2017-05-03 17:34:45 +00:00
Tianjie Xu
e0c88793d1 Add a default error code when updater script aborts
We didn't report error/cause codes unless there's an explict "Abort()"
call inside the updater script. As a result, some cause codes set by
ErrorAbort() didn't show up in last_install.

To fix the issue, add a default error code when the script terminates
abnormally (i.e. with non zero status).

Bug: 37912405
Test: error/cause code shows up in last_install when argument parsing fails
Change-Id: Ic6d3bd1855b853aeaa0760071e593a00cf6f0209
2017-05-03 05:52:03 +00:00
Tao Bao
b656a154ea Move sysMapFile and sysReleaseMap into MemMapping class.
Test: recovery_component_test
Test: recovery_unit_test
Test: Apply an OTA on angler.
Change-Id: I7170f03e4ce1fe06184ca1d7bcce0a695f33ac4d
2017-05-01 21:51:54 -07:00
Dmitri Plotnikov
ed9db0fd73 Adding support for quiescent reboot to recovery
Bug: 37401320
Test: build and push OTA and hit adb reboot recovery,quiescent. The screen should remain off throughout the upgrade process.

(cherry picked from commit 8706a98aa6)

Change-Id: I79789a151f6faafda8ecc6198c2182cc2a91da70
2017-04-27 16:31:11 -07:00
Dmitri Plotnikov
8706a98aa6 Adding support for quiescent reboot to recovery
Bug: 37401320
Test: build and push OTA and hit adb reboot recovery,quiescent. The screen should remain off throughout the upgrade process.
Change-Id: Ibed3795c09e26c4fa73684d40b94e40c78394d3f
2017-04-19 14:43:08 -07:00
Tianjie Xu
3a8d98dd90 Abort the update if there's not enough new data
Right now the update stuck in a deadlock if there's less new data than
expection. Add some checkers and abort the update if such case happens.
Also add a corresponding test.

Bug: 36787146
Test: update aborts correctly on bullhead && recovery_component_test passes
Change-Id: I914e4a2a4cf157b99ef2fc65bd21c6981e38ca47
2017-04-07 17:19:46 -07:00
Tao Bao
bf5b77dbf7 Change the internal representation in RangeSet.
This CL makes the following changes to RangeSet:
 - Uses std::pair<size_t, size_t> to represent a Range;
 - Uses std::vector<Range> to represent a RangeSet;
 - Provides const iterators (forward and reverse);
 - Provides const accessor;
 - 'blocks()' returns the number of blocks (formerly 'size');
 - 'size()' returns the number of Range's (formerly 'count').

Test: recovery_unit_test
Test: Apply an incremental update with the new updater.
Change-Id: Ia1fbb343370a152e1f7aa050cf914c2da09b1396
2017-04-03 09:22:48 -07:00
Tao Bao
c97edcb4f4 updater: Keep the parsed parameters in std::unique_ptr.
We don't need to take raw pointers out of the parsed arguments.
std::unique_ptr handles the dereferencing automatically.

Test: mmma bootable/recovery
Change-Id: I1beabf6e04dc350bdad7b36cee5fb345c82b28f2
2017-03-31 12:24:22 -07:00
Tao Bao
8f23757ad4 Move parse_range() and range_overlaps() into RangeSet.
Also move RangeSet into a header file to make it testable, and add unit
tests.

In RangeSet::Parse() (the former parse_range()), use libbase logging to
do assertions. This has the same effect as the previous
exit(EXIT_FAILURE) to terminate the updater process and abort an update.
The difference lies in the exit status code (i.e. WEXITSTATUS(status) in
install.cpp), which changes from 1 (i.e. EXIT_FAILURE) to 0.

Test: recovery_unit_test
Test: Apply an incremental update with the new updater.
Change-Id: Ie8393c78b0d8ae0fd5f0ca0646d871308d71fff0
2017-03-31 00:04:56 -07:00
Tao Bao
65c065b4bb Merge "updater: Move RangeSinkWrite into RangeSinkState." 2017-03-30 23:56:14 +00:00
Tao Bao
0bbc764bbe updater: Don't append newline when calling uiPrint().
LOG(INFO) already appends a newline. Don't print redundant newline.

Test: No extra blank lines when calling ui_print(). And on-screen UI
      shows the same.
Change-Id: I74e9a8504a7146a6cb3dae02fe2406d0dd54069b
2017-03-30 10:28:37 -07:00
Tao Bao
60a70afc0a updater: Move RangeSinkWrite into RangeSinkState.
Then rename RangeSinkState to RangeSinkWriter. RangeSinkWriter reads
data from the given FD, and writes them to the desination RangeSet.

Test: Apply an incremental with the new updater.
Change-Id: I5e3ab6fc082efa1726562c55b56e2d418fe4acaf
2017-03-29 11:31:41 -07:00
Tao Bao
c0e1c46a70 applypatch: Let Apply{BSDiff,Image}Patch accept std::function.
Test: mmma bootable/recovery system/update_engine
Test: recovery_component_test
Change-Id: I93c2caa87bf94a53509bb37f98f2c02bcadb6f5c
2017-03-28 10:14:53 -07:00
Tao Bao
f7eb760fe7 applypatch: Change the ssize_t length parameters to size_t.
Mostly for applypatch family APIs like ApplyBSDiffPatch() and
ApplyImagePatch(). Changing to size_t doesn't indicate they would
necessarily work with very large size_t (e.g. > ssize_t), just
similar to write(2). But otherwise accepting negative length doesn't
make much sense.

Also change the return type of SinkFn from ssize_t to size_t. Callers
tell a successful sink by comparing the number of written bytes against
the desired value. Negative return values like -1 are not needed. This
also makes it consistent with bsdiff::bspatch interface.

Test: recovery_component_test
Test: Apply an incremental with the new updater.
Change-Id: I7ff1615203a5c9854134f75d019e266f4ea6e714
2017-03-28 10:13:38 -07:00
Tao Bao
d2aecd465b updater: Clean up LoadSrcTgtVersion2().
Rename to LoadSourceBlocks() by moving the target blocks parsing part
into the caller. This allows detecting whether the target blocks have
already had the expected data before loading the source blocks. It
doesn't affect anything when applying an update package for the first
time, but it skips loading the unneeded source blocks when resuming an
update. It additionally avoids unnecessarily dumping the "corrupt"
source/stash blocks when resuming an update.

Bug: 33694730
Test: Apply an incremental update with the new updater.
Test: Resume an incremental update with the new updater.
Change-Id: I794fd0d1045be7b3b7f8619285dc0dade01398d0
2017-03-27 16:21:55 -07:00
Tao Bao
5212a00588 Merge "updater: Remove some redundant arguments." 2017-03-25 23:11:37 +00:00