It used to be "const Value*", but nullptr won't be a valid input.
Test: recovery_host_test; recovery_component_test
Change-Id: I904b5689ac3e64504088bf0544c9fb5d45a52243
The function ApplyBSDiffPatch() defined in bspatch.cpp is declared in
applypatch.h, but it includes "bspatch.h" from the bsdiff/ project,
which is at least confusing. There is no "bspatch.h" in this repo, so
the include actually reffers to the one in bsdiff. This patch uses the
"bsdiff/bspatch.h" form instead to avoid confusion.
Bug: None
Test: It builds.
Change-Id: I6b6ffd6725b2b34ff644aed93683f69779103661
We encountered segfaults in Imgdiff host tests due to the failure to
reset states of getopt. The problem can be solved by switching to use
bionic's gtest where a new process is forked for each test.
Also modify the recovery_component_test to make sure it runs in parallel.
Changes include:
1. Merge the writes to misc partition into one single test.
2. Change the hard coded location "/cache/saved.file" into a configurable
variable.
Bug: 67849209
Test: recovery tests pass
Change-Id: I165d313f32b83393fb7922c5078636ac40b50bc2
$ valgrind --leak-check=full out/host/linux-x86/nativetest64/recovery_host_test/recovery_host_test
==36755== 112 bytes in 1 blocks are definitely lost in loss record 4 of 16
==36755== at 0x40307C4: malloc (valgrind/coregrind/m_replacemalloc/vg_replace_malloc.c:270)
==36755== by 0x40C1669: operator new(unsigned long) (external/libcxxabi/src/cxa_new_delete.cpp:46)
==36755== by 0x18D6A8: ApplyImagePatch(unsigned char const*, unsigned long, Value const*, std::__1::function<unsigned long (unsigned char const*, unsigned long)>, sha_state_st*, Value const*) (bootable/recovery/applypatch/imgpatch.cpp:62)
==36755== by 0x18D02B: ApplyImagePatch(unsigned char const*, unsigned long, unsigned char const*, unsigned long, std::__1::function<unsigned long (unsigned char const*, unsigned long)>) (bootable/recovery/applypatch/imgpatch.cpp:134)
==36755== by 0x160D15: GenerateTarget(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) (bootable/recovery/tests/component/imgdiff_test.cpp:85)
==36755== by 0x11FA7D: verify_patched_image(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (bootable/recovery/tests/component/imgdiff_test.cpp:96)
==36755== by 0x12966C: ImgdiffTest_zip_mode_smoke_trailer_zeros_Test::TestBody() (bootable/recovery/tests/component/imgdiff_test.cpp:295)
==36755== by 0x235EF9: testing::Test::Run() (external/googletest/googletest/src/gtest.cc:2455)
==36755== by 0x236CBF: testing::TestInfo::Run() (external/googletest/googletest/src/gtest.cc:2653)
==36755== by 0x2372D6: testing::TestCase::Run() (external/googletest/googletest/src/gtest.cc:2771)
==36755== by 0x23EEE6: testing::internal::UnitTestImpl::RunAllTests() (external/googletest/googletest/src/gtest.cc:4648)
==36755== by 0x23EB45: testing::UnitTest::Run() (external/googletest/googletest/src/gtest.cc:2455)
std::unique_ptr<z_stream, decltype(&deflateEnd)> strm(new z_stream(), deflateEnd);
Only the internally allocated buffers inside 'strm' would be free'd by
deflateEnd(), but not 'strm' itself.
This CL fixes the issue by moving 'strm' to stack variable. Note that we
only need to call deflateEnd() on successful return of deflateInit2().
Test: recovery_host_test && recovery_component_test
Change-Id: I39b9bdf62376b8029f95cab82c8542bfcb874009
bsdiff interface is changing such that it hides the suffix array
pointer from the public interface. This allows to use a different
suffix array data size depending on the input size, running much faster
in the normal case.
Bug: 34220646
Test: `make checkbuild`; Ran an incremental update generation on a non-A/B device.
Change-Id: I78e766da56cf28bc7774b8c8e58527bc11d919fb
Since it has grown much larger, users of the header shouldn't compile
and carry their full copies.
Also add missing header includes in imgdiff.cpp and imgdiff_test.cpp.
Test: mmma bootable/recovery
Test: recovery_unit_test; recovery_component_test; recovery_host_test
Change-Id: I88ca54171765e5606ab0d61580fbc1ada578fd7d
We have shared lib target for libbz now (and libbz.so is already on
device because of /system/bin/bzip2).
Test: m applypatch
Change-Id: I5fe2468a8d535840245f081a92d436240dddbf6b
Also drop the "bootable/recovery" path in LOCAL_C_INCLUDES from
applypatch modules.
Test: lunch aosp_{angler,bullhead,fugu,dragon,sailfish}-userdebug;
mmma bootable/recovery
Change-Id: Idd602a796894f971ee4f8fa3eafe36c42d9de986
This reduces the size of /system/bin/applypatch by ~69KB
(aosp_bullhead-userdebug).
Also remove the unneeded libcutils dependency.
Test: mmma bootable/recovery
Test: Check that /system/bin/install-recovery.sh successfully installs
the recovery image.
Change-Id: I5063be9a9b7b8029d45ab5c2a7c45ef2cda81d26
Also make matching changes to applypatch modules which include
edify/expr.h.
Test: mmma bootable/recovery
Change-Id: Ia72be3caa010d7f56a70add2da345e631b306378
This reverts commit 26436d6d60 to re-land
"Move error_code.h into otautil.".
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".
This CL needs to land with device-specific module changes (e.g. adding
the dependency on libotautil).
Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
mmma bootable/recovery
Change-Id: If193241801af2dae73eccd31ce57cd2b81c9fd96
And move '#include "edify/expr.h"' into .cpp files. This breaks the
transitive dependency on libedify. Modules that include
"applypatch/applypatch.h" don't need to add libedify into their
dependency list, unless they really need anything from libedify.
Build libedify static library for host, which is needed by
libimgpatch.
Test: mmma bootable/recovery
Change-Id: Ibb53d322579fcbf593438d058d9bcee240625941
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".
Test: mmma bootable/recovery
Change-Id: Ia4649789cef2aaeb2785483660e9ea5a8b389c62
Add a mandatory option in imgdiff to write the split info (i.e.
patch_size, tgt_size, src_ranges) to file when handling large apks.
Therefore, the caller of imgdiff can create split transfers based on
the info.
Bug: 63542719
Test: unit tests pass
Change-Id: I853d55d1f999fd576474faa81077f7307f4d856d
Due to the cache size limit for OTA generation, we used to split large
zip files linearly into pieces and do bsdiff on them. As a result, i) we
lose the advantage of imgdiff; ii) if there's an accidental order change
of some huge files inside the zip, we'll create an insanely large patch.
This patch splits the src&tgt more smartly based on the zip entry_name.
If the entry_name is empty or no matching source is found for a target
chunk, we'll skip adding its source and later do a bsdiff against the
whole split source image (this rarely happens in our use cases except
for the metadata inside a ziparchive).
After the split, the target pieces are continuous and block aligned,
while the sources pieces are mutually exclusive. (Some of the source
blocks may not be used if there's no matching entry_name in the target.)
Then we will generate patches accordingly between each split image
pairs.
Afterwards, if we apply imgpatch to each pair of split source/target
images and add up the patched result, we can get back the original
target image.
For example:
Input: [src_image, tgt_image]
Split: [src-0,tgt-0; src-1,tgt-1, src-2,tgt-2]
Diff: [ patch-0; patch-1; patch-2]
Patch: [(src-0,patch-0)=tgt-0; (src-1,patch-1)=tgt-1;
(src-2,patch-2)=tgt-2;]
Append: [tgt-0 + tgt-1 + tgt-2 = tgt_image]
Peformance:
For the small package in b/34220646, we decrease the patch size of
chrome.apk dramatically from 30M to 400K due to the order change of
two big .so files.
On two versions of angler, I also observe decent patch size decrease.
For chrome.apk, we reduced the size from 5.9M to 3.2M; and for
vevlet.apk from 8.0M to 6.5M.
Bug: 34220646
Test: recovery component test && apply imgdiff & imgpatch on two
chrome.apk
Change-Id: I145d802984fa805efbbac9d01a2e64d82ef9728b
Turn on -Wall for all modules. Also remove the obsolete file_cmp() in
apply_patch test and now() in wear_ui.
The only exception is lib_edify due to the unused functions in the
intermediate cpp files generated from the lex files. It will be handled
in a seperate CL.
Bug: 64939312
Test: mma, unit tests pass
Change-Id: Ic53f76b60b6401ab20db3d98130d674c08e3702f
1. Move the declaration of the Image classes to the header file to make
testing easier.
2. Also move rangeset.h to bootable/recovery to allow access in imgdiff.
Test: recovery component test
Change-Id: I68a863e60a3f2e7ae46ee48f48eb15391f5f4330
This way we can keep the input images const when calling
genetatepatches().
Test: recovery component test; diff and patch on chrome.apk; generate
recovery-from-boot.p for angler.
Change-Id: I65b5689b88f6719c6ede46bb82def0c4caeb8a61
This helps us to add a new mode to handle large APKs in the follow
up CL.
Changes include:
1. Create a new interface class 'Image'
1. Create subclasses 'ZipModeImage' and 'ImageModeImage' and move the
related functions there.
Bug: 63542719
Test: recovery_component_test passes
Change-Id: I7729b0ba39b19a9c84811636a60dd0a0b1acc2f0
In a rare case, a random chunk will pass both the gzip header check
and the inflation process; but fail the uncompressed length check in the
footer. This leads to a imgdiff failure. So, we should treat this chunk
as 'normal' instead of 'inflated' while generating the patch.
Bug: 63334984
Test: imgdiff generates patch successfully on previous failing images.
Change-Id: Ice84f22d3653bce9756bda91e70528c0d2f264a0
This new sink function works as a wrapper of the old sink. It deflates
the available patch data on the fly. Therefore, we don't need to store
the full uncompressed patch data in memory.
Test: recovery_component_test && apply an incremental update on angler
Change-Id: I2274ec50a1607089abcc9d0954a2a748f28c3122
This will help us to identify the patch corruption.
Meanwhile fix a wrong size parameter passed to bspatch.
(patch->data.size() into patch->data.size() - patch_offset).
Also remove the only usage of "ApplyBSDiffPatchMem()" and inline its
Sink function for simplicity.
Bug: 37855643
Test: Prints SHA1 for corrupted patch in imgdiff_test.
Change-Id: Ibf2db8c08b0ded1409bb7c91a3547a6bf99c601d
The SpaceMode (applypatch -s) was used in amend script (cupcake) only,
which has been removed since commit
9ce2ebf5d300eba5f6086583b0941ef68a3e4b42 (platform/build). The later
(and current) edify script uses apply_patch_space().
Note that other modes (PatchMode, CheckMode) of applypatch executable
are mostly used by install-recovery.sh script.
Test: No active user of "applypatch -s".
Test: recovery_component_test
Change-Id: I1d689b7fedd3884077e88ed1d6c22f7a2198859d
capture the error flow.
Construct ota package which is bsdiff exception scene ,then do
simulation test, native code can not capture exception scenes.
Test: recovery_component_test
Test: Apply an bsdiff exception scene ota package.
Change-Id: Icd9f6eac78739bd35c74b9fcaaf8154335d680a5
Mostly for applypatch family APIs like ApplyBSDiffPatch() and
ApplyImagePatch(). Changing to size_t doesn't indicate they would
necessarily work with very large size_t (e.g. > ssize_t), just
similar to write(2). But otherwise accepting negative length doesn't
make much sense.
Also change the return type of SinkFn from ssize_t to size_t. Callers
tell a successful sink by comparing the number of written bytes against
the desired value. Negative return values like -1 are not needed. This
also makes it consistent with bsdiff::bspatch interface.
Test: recovery_component_test
Test: Apply an incremental with the new updater.
Change-Id: I7ff1615203a5c9854134f75d019e266f4ea6e714
Also remove the utils in applypatch and replace them with the
corresponding libbase functions.
Test: recovery tests pass.
Change-Id: I77254c141bd3e7d3d6894c23b60e866009516f81
Patching regular files is used in file-based OTA only, which has become
obsolete.
Bug: 35853185
Test: Apply an incremental that patches the boot.img.
Test: /system/bin/install-recovery.sh works.
Test: recovery_component_test passes.
Change-Id: Id44e42c4bc63f2162ecc8a6df1cb528b7ae6b0a9
Put ImageChunk and some helper functions into a class. Also switch to
using std::vector instead of malloc.
Bug: 18606652
Test: imgdiff_test passed on host. Also generate a complete incremental OTA package.
The file content is the same and time consumption is similar.
Change-Id: Id603ada4e130ef521218400761a119001a86ca79
Now ApplyBSDiffPatch() will stream the output to sink as we go instead
of sinking everything at the end.
Test: recovery_host_test
Bug: 26982501
Change-Id: I05b6ed40d45e4b1b19ae72784cf705b731b976e3
In zip mode, if a chunk is not deflate or its filename can't be found
in source chunks, the entire source file is used as old data for bsdiff,
To avoid repeatedly construct the suffix array used by bsdiff, we cache
the suffix array of the entire source file.
Bug: 34281147
Test: =time -v imgdiff -z Chrome-ORF74B.apk Chrome-ORF76B.apk Chrome.imgdiff
Change-Id: Ifd957ccecf7226fcb44dbf28c58969a06ef74f4b
When the input image ends with the magic value sequence of 0x1f, 0x8b,
0x0b (optionally with 0x00), the image parsing code will be stuck in an
infinite loop.
Test: recovery_component_test passes.
Change-Id: Ie3629dfdc41360387b19cc3e0359c95ae4fb998e
Factor out libimgdiff static library for testing purpose.
This CL adds the imgdiff tests on host and on target both (similar to
libimgpatch). In practice, we only need imgdiff binary on host, and
libimgpatch on target. But they should build and pass tests on both
platforms.
Test: recovery_host_test passes; recovery_component_test passes.
Change-Id: I0eafb7faf727cdf70066310e845af6ee245d4f60
https://code.google.com/p/android/issues/detail?id=230602
On the second attempt， open the file with O_RDONLY，
which causing a write failure。
Change-Id: If89165b8c7619fe25722073a46b3cc7c61530a71
Signed-off-by: katao <ustckato@gmail.com>
bootable/recovery/applypatch/imgdiff.cpp:195:5: warning: Potential leak
of memory pointed to by 'img' [clang-analyzer-unix.Malloc]
Bug: 26936282
Test: WITH_TIDY=1 WITH_STATIC_ANALYZER=1 mm
Change-Id: Ie79c780233ddfebf85686a24df3bf2561f831580
bootable/recovery/applypatch/imgdiff.cpp:322:11: warning: Value stored to 'ret' during its initialization is never read [clang-analyzer-deadcode.DeadStores]
bootable/recovery/applypatch/imgdiff.cpp:447:11: warning: Value stored to 'ret' during its initialization is never read [clang-analyzer-deadcode.DeadStores]
bootable/recovery/applypatch/imgdiff.cpp:553:3: warning: Value stored to 'ret' is never read [clang-analyzer-deadcode.DeadStores]
Bug: 26936282
Test: WITH_TIDY=1 WITH_STATIC_ANALYZER=1 mm
Change-Id: I3f865e3e9b9d19e5ea5e8dfd2fe2c644254ffbb5
We should always use unique_fd or unique_file to hold the FD or FILE*
pointer when opening via ota_(f)open functions.
This CL avoids accidentally closing raw FDs or FILE* pointers that are
managed by unique_fd/unique_file.
Test: recovery_component_test passes.
Change-Id: If58eb8b5c5da507563f85efd5d56276472a1c957
We were using the below sequence prior to the CL in [1].
unique_fd fd(ota_open(...));
ota_close(fd);
fd.reset(ota_open(...));
fd.reset() may unintentionally close the newly opened FD if it
has the same value as the early ota_open. The CL in [1] changed to
"ota_close(fd.release())" to avoid the issue. This CL adds a new
overloaded function ota_close(unique_fd&) to handle the release
automatically.
Similarly add ota_fclose(std::unique_ptr<FILE>&).
[1] commit 48cf770471.
Bug: 33034669
Test: recovery_component_test passes.
Change-Id: Ief91edc590e95a7426e33364b28754173efb1056
We use android::base::unique_fd() to avoid leaking FD. We also want to
call close (or ota_close) to explicitly check the close result. When
combining the two together, we need to release the unique_fd to avoid
closing the same FD twice.
Bug: 33034669
Test: Trigger applypatch with install-recovery.sh.
Change-Id: I1a4f5d5fba7a23ef98d8bd7b7b07e87ae6f705c5
Add unique_fd that calls ota_close() instead of the default closer.
Test: recovery_component_test passes.
Test: Apply a package that calls apply_patch().
Change-Id: I0c19921731757934f76cf7d5215916673a8f2777
We don't need three vectors to sort the (size, SHA-1) pairs.
Test: recovery_component_test passes.
Test: Apply a package that calls apply_patch_check() to patch EMMC
partitions.
Change-Id: I4a6620630a6711f490822cf30f1e7fe5cea6ce49
static_cast is preferable to reinterpret_cast when casting from void*
pointers returned by malloc/calloc/realloc/mmap calls.
Discovered while looking at compiler warnings (b/26936282).
Test: WITH_TIDY=1 WITH_STATIC_ANALYZER=1 mma
Change-Id: Iaffd537784aa857108f6981fdfd82d0496eb5592
Merged-In: I151642d5a60c94f312d0611576ad0143c249ba3d
bootable/recovery/applypatch/imgdiff.cpp:1065:3: warning: Potential leak of memory pointed to by 'patch_data'
bootable/recovery/applypatch/imgdiff.cpp:1065:3: warning: Potential leak of memory pointed to by 'patch_size'
bootable/recovery/applypatch/imgdiff.cpp:226:7: warning: Potential leak of memory pointed to by 'temp_entries'
Bug: 26936282
Test: WITH_TIDY=1 WITH_STATIC_ANALYZER=1 mm
Change-Id: I3cac945d7677d367934d5619ef7419daf6f48d6f
Refactor applypatch/main.cpp into libapplypatch_modes so that we can add
testcases.
Some changes to applypatch/main.cpp:
- Replace char** argv with const char**;
- Use android::base::Split() to split ":";
- Use android::base::ParseUInt().
Bug: 32383590
Test: Unit tests pass, install-recovery.sh works.
Change-Id: I44e7bfa5ab717d439ea1d0ee9ddb7b2c40bb95a4
When constructing std::string from C-string, the string may be truncated
at null char. Use range constructor instead.
Bug: 32380016
Test: Use applypatch to install a previously failed recovery image.
Change-Id: Id3e2afb4a810594243cd21db526933d1fea5044b
Applypatch_check should be skipped if no sha is specified. As the
comments said: "It's okay to specify no sha1s; the check will pass if
the LoadFileContents is successful. Useful for reading partitions,
where the filename encodes the sha1s."
Test: The update package applied on angler successfully.
Bug: 32243751
Change-Id: Ib8f3dadf19f745c2dbd350d60da46ab12d75bc87
Clean up the duplicated codes that handle the zip files in
bootable/recovery; and rename the library of the remaining
utility functions to libotautil.
Test: Update package installed successfully on angler.
Bug: 19472796
Change-Id: Iea8962fcf3004473cb0322b6bb3a9ea3ca7f679e
Changing the field of 'Value' in edify to std::string from char*.
Meanwhile cleaning up the users of 'Value' and switching them to
cpp style.
Test: compontent tests passed.
Bug: 31713288
Change-Id: Iec5a7d601b1e4ca40935bf1c70d325dafecec235
Also remove the 0xff comparison when validating the bootloader
message fields. As the fields won't be erased to 0xff after we
remove the MTD support.
Bug: 28202046
Test: The recovery folder compiles for aosp_x86-eng
Change-Id: Ibb30ea1b2b28676fb08c7e92a1e5f7b6ef3247ab
Clean up the recovery image and switch to libbase logging.
Bug: 28191554
Change-Id: Icd999c3cc832f0639f204b5c36cea8afe303ad35
Merged-In: Icd999c3cc832f0639f204b5c36cea8afe303ad35
We might end up in an infinite loop if read(2) reached EOF unexpectedly.
The problematic code in uncrypt mentioned in the bug has been fixed
by switching to libbase ReadFully(). So I grepped through the recovery
code and fixed some other occurences of the issue.
Bug: 31073201
Change-Id: Ib867029158ba23363b8f85d61c25058a635c5a6b
We may have expanded_len == 0 when calling inflate(). After switching to
using std::vector, it passes a nullptr buffer to inflate() and leads to
Z_STREAM_ERROR.
Bug: 29312140
Change-Id: Iab7c6c07a9e8488e844e7cdda76d02bd60d2ea98
external/bsdiff uses divsufsort which is much faster, and also include
some bug fixes.
Bug: 26982501
Test: ./imgdiff_test.sh
Change-Id: I089a301c291ee55d79938c7a3ca6d509895440d1
The applypatch heaaders were recently moved to the include/ directory.
This patch reflects that change in the Makefile.
Bug: None
TEST=sudo emerge imgdiff
Change-Id: I0bf3f991b5e0c98054033c5939ed69b3e3c827a9
Cherry pick this patch because it fixes the problem that
a newed Value is released by free().
Bug: 26906416
Change-Id: Ib53b445cd415a1ed5e95733fbc4073f9ef4dbc43
(cherry picked from commit d6c93afcc2)
To make the static library position independent.
Bug: 26866274
Test: emerge-peppy imgdiff; sudo emerge imgdiff; emerge nyan imgdiff
Change-Id: I319e721ccfb6a51f63d31afa49f54aa7f607f4bf
If two libraries both use LOCAL_WHOLE_STATIC_LIBRARIES and include a same
library, there would be linking errors when generating a shared library
(or executable) that depends on the two libraries both.
Also clean up Android.mk files.
Remove the "LOCAL_MODULE_TAGS := eng" line for the updater module. The
module will then default to "optional" which won't be built until needed.
Change-Id: I3ec227109b8aa744b7568e7f82f575aae3fe0e6f
Treat it as a normal chunk if inflate() fails.
Test: run imgdiff on corrupted gzip and apply the patch
Bug: 27153028
Change-Id: Idcbb3c1360ec0774e6c7861b21d99af8ee10604a
