Commit graph

124 commits

Author SHA1 Message Date
Elliott Hughes
55fd293dfe Add libziparchive-based unzip.
Bug: N/A
Test: new toybox unzip.test
Change-Id: I00be388578be9a0a0390a9e2ecfac664c6ab39ca
2017-06-04 11:53:48 -07:00
Treehugger Robot
f8e96ea2d1 Merge "add a performance benchmark for libziparchive" 2017-05-30 19:44:52 +00:00
Sebastian Pop
0a94c56463 add a performance benchmark for libziparchive
This patch adds two benchmarks that measure the performance of some
operations of libziparchive.

Both benchmarks are creating a temporary zip file containing file
names of uniformly distributed lengths.  The creation of the zip
file is not timed in the benchmarks.

- FindEntry_no_match tries to find an inexisting entry in the files
  of the zip archive, in order to force the code to examine all the
  files in the archive.

- Iterate_all_files uses the iterate function to list all the files
  in the archive.

Bug: N/A
Test: adb shell /data/ziparchive-benchmarks
Change-Id: Ibdb524ba1c5ae55caddf0416ebbc09f8b6df0021
2017-05-30 10:50:14 -07:00
Jayant Chowdhary
2c87bccf46 Mark libziparchive vendor_available.
libziparchive is a library which belongs to vndk-cap. Mark it
vendor_available to enable header-abi-dumper's invocation to identify it
as a part of the vndk.

Details: https://android-review.googlesource.com/368372

Test: mm -j64

Bug: 38244611

Change-Id: Ibe490cc6c2cfca0f8d58df45317bb3a491d530f0
2017-05-24 12:33:12 -07:00
Adam Lesinski
d987c9dd9e ZipWriter: Keep LFH and CFH in sync
We change the GPB in the LocalFileHeader if the entry can not
have a trailing DataDescriptor. Make sure to patch the
CentralFileHeader to have the same bits set.

Modify ZipArchive to check that the data descriptor bit is consistent
between Central and Local file headers.

(cherry-pick of commit e0eca55fe6)

Test: make ziparchive-tests
Bug: 36686974
Change-Id: Ied167570abcf6426b1c678cd40123e5ad65909db
2017-04-11 01:45:25 +00:00
Adam Lesinski
47f5581927 Merge "ZipWriter: Do not write DataDescriptor for STORED files" 2017-04-07 00:33:20 +00:00
Adam Lesinski
e2fa70bcb0 ZipWriter: Do not write DataDescriptor for STORED files
Older implementations of ZIP (Java's ZipInputStream) don't
like a Data Descriptor trailing after an uncompressed entry.

If the FILE is seekable, and the entry is uncompressed, seek
back to the header and write out the crc and sizes.

(cherry-pick of commit 639814d946)

Bug: 36686974
Test: make ziparchive_tests
Change-Id: I61664515e5afa3e2ba814874eeac847a2aaac319
2017-04-06 17:03:32 -07:00
Adam Lesinski
b02d690336 libziparchive: fix mac os breakage
Add ftruncate64 to utils/Compat.h definitions for mac.

Change-Id: I82cb46927be911e867b606f4f4429a5e1b1987f7
Test: builds on mac
2017-03-23 12:02:09 -07:00
Adam Lesinski
537713bace libziparchive: Add ability to backup in ZipWriter
Based on the compressed size of a file entry,
the decision needs to be made to instead store the file
uncompressed. This adds support to ZipWriter to backup
its last file entry.

The file is now always truncated when the EOCD is written out,
to account for the case where a file entry is backed-up and the
resulting file written is much smaller, leaving garbage data at
the end of the file.

This change also includes a rename of FileInfo -> FileEntry.
This struct was private (now public), so it shouldn't affect any
clients.

Bug: 35461578
Test: make ziparchive-tests
Change-Id: I23dc584406274ab7b8ce62b3fbc3562ca4c2603e
2017-03-22 16:46:42 -07:00
Mark Salyzyn
30f991f251 liblog: use log/log.h when utilizing ALOG macros
Test: compile
Bug: 30465923
Change-Id: Id6d76510819ebd88c3f5003d00d73a0dbe85e943
2017-01-11 09:31:15 -08:00
Treehugger Robot
4bf9c82857 Merge "Use shared lib of libutils, libz and libbase." 2017-01-05 23:24:21 +00:00
Jaekyun Seok
86e80b9be0 Use shared lib of libutils, libz and libbase.
libutils, libz and libbase are being used as shared lib by many other
modules.
So using their shared lib will reduce total image size.

Size diffs on angler build image are as follows.

libziparchive.so  : 103844 -> 41680 (-62164)
libnativeloader.so:  50824 -> 25104 (-25720)
total             : (-87884)

Test: building succeeded, and the image was tested on angler.
Bug: 33056637
Change-Id: I015afe5b8f4d87d495b706e2e78d60f44a910e87
2016-12-22 07:03:26 +09:00
Christian Poetzsch
3081c5ebfb libziparchive: use _FILE_OFFSET_BITS=64
On 32 bit system those calls may fail if one tries to unpack files which
are bigger than 2GB.

Use large file system support to fix this problem.

Test: unpack a file bigger than 2GB on 32 bit Android systems

Change-Id: Ibd9bd5fc4a2f8dc7df98bd595f4fd1638a4f0d4a
Signed-off-by: Christian Poetzsch <christian.potzsch@imgtec.com>
2016-12-21 12:05:10 +00:00
Dan Willemsen
69b0123549 Enable libbase/libutils/libziparchive on host bionic
am: ab34b47b35

Change-Id: If8e762eace58a494efede9ef5cb09be99b74e5b8
2016-11-30 06:05:59 +00:00
Dan Willemsen
ab34b47b35 Enable libbase/libutils/libziparchive on host bionic
Bug: 31559095
Test: Test linux_bionic compile
Change-Id: Ib42c063532c1cbeb3e1418ab16a1fb54d5902aab
2016-11-29 13:38:53 -08:00
Tao Bao
af2a8858c4 Merge "ziparchive: Allow ExtractEntryToFile() to work with block device."
am: 787482ecd9

Change-Id: Icdce4207cb64e084e200b6f7f8ae194831d5c4f0
2016-11-15 22:11:21 +00:00
Tao Bao
a456c21348 ziparchive: Allow ExtractEntryToFile() to work with block device.
FileWriter::Create() calls ftruncate(2) to truncate the destination
file, which doesn't work with FD that references a block device. It
leads to kIoError when calling ExtractEntryToFile() to extract an entry
to block device FD.

As a result, it fails the package_extract_file() command in OTA updates
(e.g. 'package_extract_file("boot.img",
"/dev/block/platform/soc.0/f9824900.sdhci/by-name/boot")').

This CL skips the call to ftruncate(2) if FD references a block device.

Bug: 32903624
Test: ziparchive-tests works.
Test: Build an OTA updater (m updater) and call package_extract_file().

Change-Id: Ia81116f1a8d7cab802396bdc32c6096b4cb56a3c
2016-11-15 10:19:04 -08:00
Mark Salyzyn
8a7297a09f Merge "system/core: preparation to pull back interfaces from android/log.h"
am: 27d2d49f48

Change-Id: I604bb1d4cf62636663fa92e3d14a55887dbcae23
2016-10-20 18:03:57 +00:00
Mark Salyzyn
cfd5b080af system/core: preparation to pull back interfaces from android/log.h
Point to log/log.h where necessary, define LOG_TAG where necessary.
Accept that private/android_logger.h is suitable replacement for
log/logger.h and android/log.h.

Correct liblog/README

Effectively a cleanup and controlled select revert of
'system/core: drop or replace log/logger.h' and
'system/core: Replace log/log.h with android/log.h'.

Test: compile
Bug: 30465923
Change-Id: Ic2ad157bad6f5efe2c6af293a73bb753300b17a2
2016-10-20 08:11:39 -07:00
Tianjie Xu
b2ca85184b Merge "Add functions in recovery/minzip to libziparchive"
am: 4bc51d1ea4

Change-Id: Ica94e8050286bed1c5bc23a33305050ab8da8ec5
2016-10-18 21:57:30 +00:00
Tianjie Xu
18c25920c2 Add functions in recovery/minzip to libziparchive
Add two functions libziparchive that libminzip has. And create
corresponding unit tests.
1. Open a zip archive from a memory mapped region.
2. A new writer that takes a call back function pointer.
(Used by the OTA updater to stream the data.)

Test: Unit tests passed
Bug: 19472796
Change-Id: I2b2daec71174afe221030357e39bff5faea51e72
2016-10-15 23:16:34 -07:00
Tianjie Xu
206150bf78 Fix out of bound access in libziparchive am: 1ee4892e66 am: c24dd97654 am: 8788591224 am: 167562aa76 am: 398adf9b55 am: 680c3f1dc4 am: f0ce69815f
am: 6fb68bb196

Change-Id: Id8f75debfae94d35dbbee9799632cba3f72648d0
2016-10-07 00:30:02 +00:00
Tianjie Xu
6fb68bb196 Fix out of bound access in libziparchive am: 1ee4892e66 am: c24dd97654 am: 8788591224 am: 167562aa76 am: 398adf9b55 am: 680c3f1dc4
am: f0ce69815f

Change-Id: I8e69df083b6d5cd3a7e6bf7113ae836932c2e9f2
2016-10-07 00:26:31 +00:00
Tianjie Xu
167562aa76 Fix out of bound access in libziparchive am: 1ee4892e66 am: c24dd97654
am: 8788591224

Change-Id: I2f0ca98c74f23b1125f29b1eaf810143a83754b4
2016-10-07 00:11:29 +00:00
Mark Salyzyn
ff2dcd9af9 system/core Replace log/log.h with android/log.h
Should use android/log.h instead of log/log.h as a good example
to all others.  Adjust header order to comply with Android Coding
standards.

Test: Compile
Bug: 26552300
Bug: 31289077
Change-Id: I33a8fb4e754d2dc4754d335660c450e0a67190fc
2016-09-30 12:47:05 -07:00
Tianjie Xu
1ee4892e66 Fix out of bound access in libziparchive
The boundary check of an invalid EOCD record may succeed due to the
overflow of uint32_t. Fix the check.

Test: Open the crash.apk and libziparchive reports the offset error as expected.

Bug: 31251826
Change-Id: I1d8092a19b73886a671bc9d291cfc27d65e3d236
(cherry picked from commit ae8180c06d)
2016-09-22 18:24:53 -07:00
Colin Cross
7c6c7f0b9f Fix warnings in libziparchive
system/core/include is included in the global include path using
-isystem, which hides all warnings.  zlib.h is included through
system/core/include/ziparchive/zip_archive.h, which was hiding warnings
in it.  Use a #pragma around the call to deflateInit2, it is a macro
that expands to an old-style cast, in preparation for moving from
-isystem to -I.  Also move the ZipString constructor to zip_archive.cc
so it can assert on the length of the string parameter and fix an
implicit conversion from size_t to uint16_t.

Test: m -j native
Bug: 31492149
Change-Id: I74cdad7fe9c723859b5cfbea73c8f27d9d9ca265
2016-09-20 09:23:47 -07:00
Narayan Kamath
388a65d146 Merge "zip_archive: Improve error message on fallocate failure."
am: 704d818ec1

Change-Id: Ic1ab5e824dcc2b6a78e8ceb29732185ad5da1987
2016-08-16 09:03:04 +00:00
Narayan Kamath
d5d7abe3d1 zip_archive: Improve error message on fallocate failure.
Noticed on b/30765660.

Change-Id: I1a98f83fef02a7368bdc2e1f8f24a0771eebe468
2016-08-10 12:24:05 +01:00
Colin Cross
2fedbf79bb resolve merge conflicts of 41e82a3 to stage-aosp-master
Change-Id: I5a7faca3249f0c038914540b1dd29edf0084c0e1
2016-07-12 23:53:04 -07:00
Colin Cross
45566ba879 Convert libziparchive from Android.mk to Android.bp
Change-Id: I6f0df9046b848ee8b841ae0a608a6adc981788e7
2016-07-12 21:01:08 -07:00
Chih-hung Hsieh
63eca85308 Merge "Fix google-explicit-constructor warnings." am: e1d5c25
am: e641a2f

* commit 'e641a2fa619c29dc25c329520ea300c21f908dcc':
  Fix google-explicit-constructor warnings.

Change-Id: I461fe586fd8d23fa5abbe94859e609c29fbe889a
2016-04-26 00:49:15 +00:00
Chih-Hung Hsieh
3a114e0a81 Fix google-explicit-constructor warnings.
Bug: 28341362
Change-Id: I5d1988efc1563efbdb25c867713c54f41439a82a
2016-04-25 14:57:11 -07:00
Adam Lesinski
b1911401e0 libziparchive: ensure ReadAtOffset is atomic
ag/880725 modified ReadAtOffset to seek then read from the open
file descriptor. Previously pread64 was used to provide atomic
behaviour.

This causes races when multiple threads are trying to access data from
the file. This is supported, so this change reverts the relevant parts
of the above CL to restore the old behaviour.
Bug:27563413

Change-Id: I7bffd78da8c558745dfc3c072ba9691b1b15bb5b
2016-03-09 17:17:19 -08:00
Yabin Cui
b2a770042e libziparchive: port unit tests to darwin and windows.
Also use ReadFully to replace read, because read can
return reading bytes less than requested. And use
WriteFully to replace write.

Bug: 26962895
Change-Id: Iff0b2bc6d925619a537f7fef682c2a7ad89a2dc2
(cherry picked from commit 8e6f722764)
2016-03-07 11:21:24 -08:00
Yabin Cui
8e6f722764 libziparchive: port unit tests to darwin and windows.
Also use ReadFully to replace read, because read can
return reading bytes less than requested. And use
WriteFully to replace write.

Bug: 26962895
Change-Id: Iff0b2bc6d925619a537f7fef682c2a7ad89a2dc2
2016-02-08 16:52:55 -08:00
Christopher Ferris
5e9f3d44a3 Allow setting an arbitrary alignment for an entry.
The current code only allows the creation of an entry that is 32 bit
aligned.

Bug: 25446938

Change-Id: I6c924df12ef2bc067b3de7789257af7e3db7e904
2016-01-19 13:55:53 -08:00
Christopher Ferris
e6884ce56f Add a ZipArchiveStreamEntry class.
This allows someone to stream the data out of a zip archive
instead of extracting to a file or to memory.

Included in this change is a small cleanup of the makefile.

Change-Id: I8b679a679c3502ff4ea0bc4f9e918303657fa424
2016-01-14 12:30:20 -08:00
Elliott Hughes
4f71319df0 Track rename of base/ to android-base/.
Change-Id: Idf9444fece4aa89c93e15640de59a91f6e758ccf
2015-12-04 22:00:26 -08:00
Christopher Ferris
a2a32b0ec6 Fix bug in writing zips.
The code does not handle an edge case where writing a compressed
image can overflow the avail_out data when doing a flush. Add a
loop to keep writing the data while deflate indicates that it doesn't
have enough space to write out the compressed data during the flush.

Change-Id: I1f1d1646457ed9b67ed24f6582688c300186c23c
2015-11-05 10:48:05 -08:00
Adam Lesinski
25322eeddd Merge "ZipWriter: archive disk numbers are 0 index based" 2015-10-30 16:56:11 +00:00
Adam Lesinski
ac60abdd70 Merge "Add deflate compression to ZipWriter" 2015-10-30 16:55:55 +00:00
beonit
0e99a2f79a libziparchive: fix calculation of mod_time.
Take mod_date into account as well, and fixes the issue where
all entries had creation dates in 1979.

Signed-off-by: beonit <beonit@gmail.com>
Change-Id: Id101794fa08218d15f2d1ba4e4a313c1807ea7aa
2015-10-30 10:01:38 +00:00
Adam Lesinski
044c790b85 ZipWriter: archive disk numbers are 0 index based
Misunderstood that disk numbers were 1 index based.

Change-Id: I0c1f43436f1c5e9394818ad12164f6d7b982cbd8
2015-10-20 12:41:49 -07:00
Adam Lesinski
591fd3953d Add deflate compression to ZipWriter
Use zlib to deflate compress data into the zip file.

Change-Id: If094b926a67e199be976c3190781cd0101dd66e1
2015-10-20 12:40:33 -07:00
Adam Lesinski
ad4ad8cfc8 Implement ZipWriter for quickly writing ZipFiles.
The ZipWriter implementation exposes a stateful interface that allows
bytes of data to be streamed in as they arrive. ZipEntries can be
compressed and/or aligned on a 32-bit boundary for mmapping at runtime.

Change-Id: I43ac9e661aa5022f00d9e12b247c4314d61c441c
2015-10-15 16:27:44 -07:00
Dan Willemsen
87a419c8b1 Remove USE_MINGW/CYGWIN; Whitelist windows modules
CYGWIN is not supported, USE_MINGW and HOST_OS==windows are being
replaced with LOCAL_..._windows variables.

Bug: 23566667
Change-Id: I3e4a1e4097dc994cf5abdce6939e83a91758fd75
2015-09-02 17:10:35 -07:00
Evgenii Stepanov
953314fc21 Fix ODR issues in ART.
Linking against both libziparchive-host and libz-host results in
libz loaded twice (because it is also linked statically in
libziparchive).

Change-Id: Ib34eef60200a702c2d7fbca3589b324992e78eab
2015-06-29 16:27:44 -07:00
Yusuke Sato
07447544ad Rename ZipEntryName to ZipString
since the struct is now used for other purposes. Also add some
comparator functions to the struct to simplify zip_archive.cc.

This is a follow-up CL for f1d3d3b247.

Bug: 21957428
Change-Id: I60d4171eeacc561d59226d946e9eb5f9c96d80cf
2015-06-26 10:34:01 -07:00
Yusuke Sato
f1d3d3b247 Add |optional_suffix| to StartIteration()
so that PackageManagerService can iterate over files with a
specific file extension like ".so".

(cherry picked from commit a4a80693d9)

Bug: 21957428
Change-Id: I36ba3c33a8b366a65f67cb6d156067c5caca1151
2015-06-25 14:10:05 -07:00