Prior to this CL, the block verification works were assigned based on
the pattern of the ranges, which could lead to unbalanced workloads. This
CL adds RangeSet::Split() and moves update_verifier over.
a) For the following care_map.txt on walleye:
system
20,0,347,348,540,556,32770,33084,98306,98620,163842,164156,229378,229692,294914,295228,524289,524291,524292,524348,529059
vendor
8,0,120,135,32770,32831,94564,98304,98306
Measured the time costs prior to and with this CL with the following
script.
$ cat test_update_verifier.sh
#!/bin/sh
adb shell stop
adb shell "cp /data/local/tmp/care_map.txt /data/ota_package/"
for i in $(seq 1 50)
do
echo "Iteration: $i"
adb shell "bootctl set-active-boot-slot 0"
adb shell "echo 3 > /proc/sys/vm/drop_caches"
adb shell "time /data/local/tmp/update_verifier"
sleep 3
done
Without this CL, the average time cost is 5.66s, while with the CL it's
reduced to 3.2s.
b) For the following care_map.txt, measured the performance on marlin:
system
18,0,271,286,457,8350,32770,33022,98306,98558,163842,164094,196609,204800,229378,229630,294914,295166,501547
vendor
10,0,42,44,85,2408,32770,32806,32807,36902,74242
It takes 12.9s and 5.6s without and with the CL respectively.
Fixes: 68553827
Test: recovery_unit_test
Test: Flash new build and trigger update_verifier. Check the balanced
block verification.
Change-Id: I5fa4bf09a84e6b9b0975ee5f522724464181333f
We used to CHECK and abort on parsing errors. While it works fine for
the updater use case (because recovery starts updater in a forked
process and collects the process exit code), it's difficult for other
clients to use RangeSet as a library (e.g. update_verifier).
This CL switches the aborts to returning empty RangeSet instead. Callers
need to check the parsing results explicitly.
The CL also separates RangeSet::PushBack() into a function, and moves
SortedRangeSet::Clear() into RangeSet.
Test: recovery_unit_test
Test: Sideload an OTA package with the new updater on angler.
Test: Sideload an OTA package with injected range string errors. The
updater aborts from the explicit checks.
Change-Id: If2b7f6f41dc93af917a21c7877a83e98dc3fd016
This CL mainly changes:
a) moving the interface in struct provider_vtab to std::function;
b) code cleanup, such as moving the declaration closer to the uses,
using explicit type conversion.
Test: recovery_component_test
Test: minadbd_test
Test: Sideload a package on marlin.
Change-Id: Id0e3c70f1ada54a4cd985b54c84438c23ed4687e
We encountered segfaults in Imgdiff host tests due to the failure to
reset states of getopt. The problem can be solved by switching to use
bionic's gtest where a new process is forked for each test.
Also modify the recovery_component_test to make sure it runs in parallel.
Changes include:
1. Merge the writes to misc partition into one single test.
2. Change the hard coded location "/cache/saved.file" into a configurable
variable.
Bug: 67849209
Test: recovery tests pass
Change-Id: I165d313f32b83393fb7922c5078636ac40b50bc2
~TemporaryDir() calls rmdir(2) directly, which works with empty
directories only.
Test: Run recovery_host_test; No leftover on host.
Test; Run recovery_component_test on marlin; No leftover on device.
Change-Id: Ib510efb16eeda61b34161e2b386499e6cb79a4ca
As we construct the deflate entries of the target zip file with
random data, the total size of the zip file may vary from case
to case. This leads to occasional failures in the split test for
deflate large apk files. This CL fixes the issue by adding two static
zip files in the testdata instead of generating them dynamically.
Bug: 67849209
Test: run the deflate_large_test repeatedly
Change-Id: Iaeffad9205adefa10c9f62f9f088c33c4360a650
bsdiff interface is changing such that it hides the suffix array
pointer from the public interface. This allows to use a different
suffix array data size depending on the input size, running much faster
in the normal case.
Bug: 34220646
Test: `make checkbuild`; Ran an incremental update generation on a non-A/B device.
Change-Id: I78e766da56cf28bc7774b8c8e58527bc11d919fb
The thread to receive new data may still be alive after we exit
PerformBlockImageUpdate() upon failures. This caused memory corruption
when we run the unittest repeatedly. Set the receiver_available flag
to false and make sure the receiver exits when the update fails.
Bug: 65430057
Test: unittests passed with tsan
Change-Id: Icb232d13fb96c78262249ffbd29cdbe5b77f1fce
Since it has grown much larger, users of the header shouldn't compile
and carry their full copies.
Also add missing header includes in imgdiff.cpp and imgdiff_test.cpp.
Test: mmma bootable/recovery
Test: recovery_unit_test; recovery_component_test; recovery_host_test
Change-Id: I88ca54171765e5606ab0d61580fbc1ada578fd7d
Also drop the "bootable/recovery" path in LOCAL_C_INCLUDES from
applypatch modules.
Test: lunch aosp_{angler,bullhead,fugu,dragon,sailfish}-userdebug;
mmma bootable/recovery
Change-Id: Idd602a796894f971ee4f8fa3eafe36c42d9de986
This reverts commit 26436d6d60 to re-land
"Move error_code.h into otautil.".
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".
This CL needs to land with device-specific module changes (e.g. adding
the dependency on libotautil).
Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
mmma bootable/recovery
Change-Id: If193241801af2dae73eccd31ce57cd2b81c9fd96
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".
Test: mmma bootable/recovery
Change-Id: Ia4649789cef2aaeb2785483660e9ea5a8b389c62
Add a mandatory option in imgdiff to write the split info (i.e.
patch_size, tgt_size, src_ranges) to file when handling large apks.
Therefore, the caller of imgdiff can create split transfers based on
the info.
Bug: 63542719
Test: unit tests pass
Change-Id: I853d55d1f999fd576474faa81077f7307f4d856d
Otherwise the test may fail after a large number of iterations due to
file open failure.
Bug: 65430057
Test: run recovery_component_test on sailfish for 2000 iterations.
Change-Id: I0d456284d6064467038911d63eade95740cbec2c
Due to the cache size limit for OTA generation, we used to split large
zip files linearly into pieces and do bsdiff on them. As a result, i) we
lose the advantage of imgdiff; ii) if there's an accidental order change
of some huge files inside the zip, we'll create an insanely large patch.
This patch splits the src&tgt more smartly based on the zip entry_name.
If the entry_name is empty or no matching source is found for a target
chunk, we'll skip adding its source and later do a bsdiff against the
whole split source image (this rarely happens in our use cases except
for the metadata inside a ziparchive).
After the split, the target pieces are continuous and block aligned,
while the sources pieces are mutually exclusive. (Some of the source
blocks may not be used if there's no matching entry_name in the target.)
Then we will generate patches accordingly between each split image
pairs.
Afterwards, if we apply imgpatch to each pair of split source/target
images and add up the patched result, we can get back the original
target image.
For example:
Input: [src_image, tgt_image]
Split: [src-0,tgt-0; src-1,tgt-1, src-2,tgt-2]
Diff: [ patch-0; patch-1; patch-2]
Patch: [(src-0,patch-0)=tgt-0; (src-1,patch-1)=tgt-1;
(src-2,patch-2)=tgt-2;]
Append: [tgt-0 + tgt-1 + tgt-2 = tgt_image]
Peformance:
For the small package in b/34220646, we decrease the patch size of
chrome.apk dramatically from 30M to 400K due to the order change of
two big .so files.
On two versions of angler, I also observe decent patch size decrease.
For chrome.apk, we reduced the size from 5.9M to 3.2M; and for
vevlet.apk from 8.0M to 6.5M.
Bug: 34220646
Test: recovery component test && apply imgdiff & imgpatch on two
chrome.apk
Change-Id: I145d802984fa805efbbac9d01a2e64d82ef9728b
Turn on -Wall for all modules. Also remove the obsolete file_cmp() in
apply_patch test and now() in wear_ui.
The only exception is lib_edify due to the unused functions in the
intermediate cpp files generated from the lex files. It will be handled
in a seperate CL.
Bug: 64939312
Test: mma, unit tests pass
Change-Id: Ic53f76b60b6401ab20db3d98130d674c08e3702f
The metadata file now can have multiple serial numbers in the format:
serialno=serialno1|serialno2|serialno3 ...
Verifier will pass the check if the device serial number matches any of
these numbers.
Bug: 64802465
Test: Create a metadata file with 1000 numbers and sideload in sailfish.
The checker detects both match and mismatch cases.
Change-Id: I3f12b75e15f4179df260778e37f4563d65db0fa8
1. Move the declaration of the Image classes to the header file to make
testing easier.
2. Also move rangeset.h to bootable/recovery to allow access in imgdiff.
Test: recovery component test
Change-Id: I68a863e60a3f2e7ae46ee48f48eb15391f5f4330
It fails to build recovery_component_test with the following errors:
out/soong/.intermediates/hardware/interfaces/boot/1.0/android.hardware.boot@1.0_genc++_headers/gen/android/hardware/boot/1.0/types.h:14:
error: undefined reference to 'android::hardware::hidl_string::hidl_string(android::hardware::hidl_string const&)'
out/soong/.intermediates/hardware/interfaces/boot/1.0/android.hardware.boot@1.0_genc++_headers/gen/android/hardware/boot/1.0/types.h:14:
error: undefined reference to 'android::hardware::hidl_string::operator=(android::hardware::hidl_string
const&)'
out/soong/.intermediates/hardware/interfaces/boot/1.0/android.hardware.boot@1.0_genc++_headers/gen/android/hardware/boot/1.0/types.h:14:
error: undefined reference to 'android::hardware::hidl_string::~hidl_string()'
libupdate_verifier includes <android/hardware/boot/1.0/IBootControl.h>,
which includes the 'types.h' above. In 'types.h', it defines struct
CommandResult that's using android::hardware::hidl_string.
Since libhidlbase doesn't have a static library target, remove
'LOCAL_FORCE_STATIC_EXECUTABLE := true', which isn't required for
running tests.
Test: mmma -j bootable/recovery
Bug: 64538692
Change-Id: Iaa7c08adc241128d787274fcaea9b363e7ff93f4
(cherry picked from commit 102016ce1f)
