Remove dependencies on zip_archive specific data structures. In follow
up changes, this method will be promoted to a public API and used from
androidfw/ZipUtils as well as tools/zipalign, thereby allowing us to
remove a near-exact copy of this code and to simplify depdencies.
Test: zip_archive_test
Bug: 35246701
Change-Id: If24a9965fbd6fff308c8758859026684fd2af3b9
As a VNDK module, Android.bp must have 'vndk' tag as well as
'vendor_available: true'.
The 'vndk' tag for VNDK module is formated as below:
vndk: {
enabled: true,
},
VNDK modules will be installed both in system/lib(64) as normal and
in system/lib(64)/vndk as a vendor variant.
Bug: 63866913
Test: build and boot with BOARD_VNDK_VERSION=current
Merged-In: Iec5d3496e91a99f3e6b0c816c67ad279672ff36a
Change-Id: Iec5d3496e91a99f3e6b0c816c67ad279672ff36a
(cherry picked from commit 4e7e5b3ba053d013f2c4ae79d02722b874c629fb)
As a VNDK module, Android.bp must have 'vndk' tag as well as
'vendor_available: true'.
The 'vndk' tag for VNDK module is formated as below:
vndk: {
enabled: true,
},
VNDK modules will be installed both in system/lib(64) as normal and
in system/lib(64)/vndk as a vendor variant.
Bug: 63866913
Test: build and boot with BOARD_VNDK_VERSION=current
Change-Id: Iec5d3496e91a99f3e6b0c816c67ad279672ff36a
The use of ReadAtOffset is meant to allow concurrent access
to the zip archive once it has been loaded. There were places
where this was the case, and some places that did a seek + read
combination, which could lead to data races.
NOTE: On Windows, we are not using pread as the implementation of
ReadAtOffset, therefore the guarantees on Windows are weaker.
On Linux, pread allows the file descriptor to be read at a specific
offset without changing the read pointer. This allows inherited fd's
and duped fds to be read concurrently.
On Windows, we use the ReadFile API, which allows for an atomic seek +
read operation, but modifies the read pointer. This means that any mix
use of ReadAtOffset and Read will have races. Just using ReadAtOffset is
safe.
For the Windows case, this is fine as the libziparchive code now only
uses ReadAtOffset.
Bug: 62184114
Bug: 62101783
Test: make ziparchive-tests (existing tests pass)
Change-Id: Ia7f9a30af2216682cdd9d578d26e84bc46773bb9
libziparchive-host needs to include the headers correctly, too.
Bug: 37342627
Test: mmma system/core/libziparchive
Change-Id: I88a6d38ff9e494273040f9b913c71bccdda117ad
By forward-declaring zstream, we don't need to re-export zlib headers.
Bug: 37342627
Test: build
Test: BOARD_VNDK_VERSION=current m -j dumpstate (or any other module
using libziparchive)
Change-Id: Ia69e5d459c79878d491cfe4ca21a7cc8fed4a5d8
libziparchive headers are moved from the global include directory
(/system/core/include) to the local directory inside libziparchive.
Note: /system/core/include/ziparchive still exists as a symlink to
libarchive/include/ziparchive. This will be removed when there is no
header-only dependency to libziparchive.
Bug: 37342627
Test: build
Change-Id: I3631ffc2df7be8a064d64a625d10436090c3bb0f
We should check the boundary of central directory before checking its
signature. Swap the order of these two checks.
Bug: 36392138
Test: libziparchive doesn't read the signature after boundary check fails.
Merged-In: Ie89f709bb2d1ccb647116fb7ccb1e23c943e5ab8
Change-Id: Ie89f709bb2d1ccb647116fb7ccb1e23c943e5ab8
(cherry picked from commit 74464a1361)
Add a check to ensure the filename boundary doesn't exceed the mapped
memory region. Also add the corresponding unit test.
Bug: 28802225
Test: New unit test passes.
Merged-In: Ibf543a7da3d7898952e9eb332c84cdfc67cf5aa4
Change-Id: Ibf543a7da3d7898952e9eb332c84cdfc67cf5aa4
(cherry picked from commit bcc4431f24)
The boundary check of an invalid EOCD record may succeed due to the
overflow of uint32_t. Fix the check and add a unit test.
Test: Open the crash.apk and libziparchive reports the offset error as expected.
Bug: 31251826
Merged-In: I1d8092a19b73886a671bc9d291cfc27d65e3d236
Change-Id: I1d8092a19b73886a671bc9d291cfc27d65e3d236
(cherry picked from commit ae8180c06d)
Make them available to tests so that they can compare values
directly instead of comparing error code strings.
Test: zip_archive_test
Change-Id: I1c78c24651ff7549b5125a49a4e67bbd8295c617
- Ensure that the compressed and uncompressed lengths in the data
descriptor match those in the central directory.
- Calculate CRCs for compressed entries and ensure that those CRCs
match the information in the central directory / data descriptor.
This is currently guarded by a flag and will be turned on at a
later stage.
- Finally, always trust the LFH when it comes to indicating the
existence of a data descriptor. We no longer require this info to
match the central directory to bring this in line with other zip
processing tools. (see NOTE in the included change).
Bug: 62047801
Test: zip_archive_test
Change-Id: Icceea66a8b1937c08a4af9aee97bc77ad7218c0f
The Central File Header and Local File Header should
have the same general purpose bit flags set for
consistency.
Bug: 36686974
Test: existing tests pass (ziparchive-tests)
(cherry picked from commit 73b3aa541f)
Change-Id: Ia672a28732154a564ea2f2e3520238bb77924a56
This patch adds two benchmarks that measure the performance of some
operations of libziparchive.
Both benchmarks are creating a temporary zip file containing file
names of uniformly distributed lengths. The creation of the zip
file is not timed in the benchmarks.
- FindEntry_no_match tries to find an inexisting entry in the files
of the zip archive, in order to force the code to examine all the
files in the archive.
- Iterate_all_files uses the iterate function to list all the files
in the archive.
Bug: N/A
Test: adb shell /data/ziparchive-benchmarks
Change-Id: Ibdb524ba1c5ae55caddf0416ebbc09f8b6df0021
libziparchive is a library which belongs to vndk-cap. Mark it
vendor_available to enable header-abi-dumper's invocation to identify it
as a part of the vndk.
Details: https://android-review.googlesource.com/368372
Test: mm -j64
Bug: 38244611
Change-Id: Ibe490cc6c2cfca0f8d58df45317bb3a491d530f0
We change the GPB in the LocalFileHeader if the entry can not
have a trailing DataDescriptor. Make sure to patch the
CentralFileHeader to have the same bits set.
Modify ZipArchive to check that the data descriptor bit is consistent
between Central and Local file headers.
(cherry-pick of commit e0eca55fe6)
Test: make ziparchive-tests
Bug: 36686974
Change-Id: Ied167570abcf6426b1c678cd40123e5ad65909db
Older implementations of ZIP (Java's ZipInputStream) don't
like a Data Descriptor trailing after an uncompressed entry.
If the FILE is seekable, and the entry is uncompressed, seek
back to the header and write out the crc and sizes.
(cherry-pick of commit 639814d946)
Bug: 36686974
Test: make ziparchive_tests
Change-Id: I61664515e5afa3e2ba814874eeac847a2aaac319
Based on the compressed size of a file entry,
the decision needs to be made to instead store the file
uncompressed. This adds support to ZipWriter to backup
its last file entry.
The file is now always truncated when the EOCD is written out,
to account for the case where a file entry is backed-up and the
resulting file written is much smaller, leaving garbage data at
the end of the file.
This change also includes a rename of FileInfo -> FileEntry.
This struct was private (now public), so it shouldn't affect any
clients.
Bug: 35461578
Test: make ziparchive-tests
Change-Id: I23dc584406274ab7b8ce62b3fbc3562ca4c2603e
libutils, libz and libbase are being used as shared lib by many other
modules.
So using their shared lib will reduce total image size.
Size diffs on angler build image are as follows.
libziparchive.so : 103844 -> 41680 (-62164)
libnativeloader.so: 50824 -> 25104 (-25720)
total : (-87884)
Test: building succeeded, and the image was tested on angler.
Bug: 33056637
Change-Id: I015afe5b8f4d87d495b706e2e78d60f44a910e87
On 32 bit system those calls may fail if one tries to unpack files which
are bigger than 2GB.
Use large file system support to fix this problem.
Test: unpack a file bigger than 2GB on 32 bit Android systems
Change-Id: Ibd9bd5fc4a2f8dc7df98bd595f4fd1638a4f0d4a
Signed-off-by: Christian Poetzsch <christian.potzsch@imgtec.com>
FileWriter::Create() calls ftruncate(2) to truncate the destination
file, which doesn't work with FD that references a block device. It
leads to kIoError when calling ExtractEntryToFile() to extract an entry
to block device FD.
As a result, it fails the package_extract_file() command in OTA updates
(e.g. 'package_extract_file("boot.img",
"/dev/block/platform/soc.0/f9824900.sdhci/by-name/boot")').
This CL skips the call to ftruncate(2) if FD references a block device.
Bug: 32903624
Test: ziparchive-tests works.
Test: Build an OTA updater (m updater) and call package_extract_file().
Change-Id: Ia81116f1a8d7cab802396bdc32c6096b4cb56a3c
Point to log/log.h where necessary, define LOG_TAG where necessary.
Accept that private/android_logger.h is suitable replacement for
log/logger.h and android/log.h.
Correct liblog/README
Effectively a cleanup and controlled select revert of
'system/core: drop or replace log/logger.h' and
'system/core: Replace log/log.h with android/log.h'.
Test: compile
Bug: 30465923
Change-Id: Ic2ad157bad6f5efe2c6af293a73bb753300b17a2
Add two functions libziparchive that libminzip has. And create
corresponding unit tests.
1. Open a zip archive from a memory mapped region.
2. A new writer that takes a call back function pointer.
(Used by the OTA updater to stream the data.)
Test: Unit tests passed
Bug: 19472796
Change-Id: I2b2daec71174afe221030357e39bff5faea51e72
Should use android/log.h instead of log/log.h as a good example
to all others. Adjust header order to comply with Android Coding
standards.
Test: Compile
Bug: 26552300
Bug: 31289077
Change-Id: I33a8fb4e754d2dc4754d335660c450e0a67190fc
The boundary check of an invalid EOCD record may succeed due to the
overflow of uint32_t. Fix the check.
Test: Open the crash.apk and libziparchive reports the offset error as expected.
Bug: 31251826
Change-Id: I1d8092a19b73886a671bc9d291cfc27d65e3d236
(cherry picked from commit ae8180c06d)
system/core/include is included in the global include path using
-isystem, which hides all warnings. zlib.h is included through
system/core/include/ziparchive/zip_archive.h, which was hiding warnings
in it. Use a #pragma around the call to deflateInit2, it is a macro
that expands to an old-style cast, in preparation for moving from
-isystem to -I. Also move the ZipString constructor to zip_archive.cc
so it can assert on the length of the string parameter and fix an
implicit conversion from size_t to uint16_t.
Test: m -j native
Bug: 31492149
Change-Id: I74cdad7fe9c723859b5cfbea73c8f27d9d9ca265
ag/880725 modified ReadAtOffset to seek then read from the open
file descriptor. Previously pread64 was used to provide atomic
behaviour.
This causes races when multiple threads are trying to access data from
the file. This is supported, so this change reverts the relevant parts
of the above CL to restore the old behaviour.
Bug:27563413
Change-Id: I7bffd78da8c558745dfc3c072ba9691b1b15bb5b
Also use ReadFully to replace read, because read can
return reading bytes less than requested. And use
WriteFully to replace write.
Bug: 26962895
Change-Id: Iff0b2bc6d925619a537f7fef682c2a7ad89a2dc2
(cherry picked from commit 8e6f722764)
Also use ReadFully to replace read, because read can
return reading bytes less than requested. And use
WriteFully to replace write.
Bug: 26962895
Change-Id: Iff0b2bc6d925619a537f7fef682c2a7ad89a2dc2
This allows someone to stream the data out of a zip archive
instead of extracting to a file or to memory.
Included in this change is a small cleanup of the makefile.
Change-Id: I8b679a679c3502ff4ea0bc4f9e918303657fa424
The code does not handle an edge case where writing a compressed
image can overflow the avail_out data when doing a flush. Add a
loop to keep writing the data while deflate indicates that it doesn't
have enough space to write out the compressed data during the flush.
Change-Id: I1f1d1646457ed9b67ed24f6582688c300186c23c
Take mod_date into account as well, and fixes the issue where
all entries had creation dates in 1979.
Signed-off-by: beonit <beonit@gmail.com>
Change-Id: Id101794fa08218d15f2d1ba4e4a313c1807ea7aa
The ZipWriter implementation exposes a stateful interface that allows
bytes of data to be streamed in as they arrive. ZipEntries can be
compressed and/or aligned on a 32-bit boundary for mmapping at runtime.
Change-Id: I43ac9e661aa5022f00d9e12b247c4314d61c441c
CYGWIN is not supported, USE_MINGW and HOST_OS==windows are being
replaced with LOCAL_..._windows variables.
Bug: 23566667
Change-Id: I3e4a1e4097dc994cf5abdce6939e83a91758fd75
Linking against both libziparchive-host and libz-host results in
libz loaded twice (because it is also linked statically in
libziparchive).
Change-Id: Ib34eef60200a702c2d7fbca3589b324992e78eab
since the struct is now used for other purposes. Also add some
comparator functions to the struct to simplify zip_archive.cc.
This is a follow-up CL for f1d3d3b247.
Bug: 21957428
Change-Id: I60d4171eeacc561d59226d946e9eb5f9c96d80cf
so that PackageManagerService can iterate over files with a
specific file extension like ".so".
(cherry picked from commit a4a80693d9)
Bug: 21957428
Change-Id: I36ba3c33a8b366a65f67cb6d156067c5caca1151
so that PackageManagerService can iterate over files with a
specific file extension like ".so".
Bug: 21957428
Change-Id: I76ed9560d4d1e00d297a97d518ec357166be1981
The objective of fallocate call seems to be to
make sure that we have enough enough space left
in the disk to house the uncompressed file.
But, fallocate is only supported in the following
file systems:
btrfs, ext4, ocfs2, and xfs
Return error only when fallocate fails due to
lack of space. The immediate ftruncate call is
going to take of the majority of other errors.
Bug: http://b/21558406
Bug: 21561449
Change-Id: I7083f3c7e5d745bd6e8a190ac9020297d638d9d4
(cherry picked from commit a68d0d1fe4)
We're already linking against libbase but we'll have to add
a libbase dependency to every target that includes libziparchive
as a STATIC_LIBRARY dependency, given that there's no way to
express that what we want (except by adding a LOCAL_WHOLE_STATIC_LIBRARY
dependency on libbase to libziparchive but that seems bad too)
Bug: http://b/21558406
Change-Id: I294ad389a9c61a1134a7bc323da25b0004a8f1e0
(cherry picked from commit e97e66ea7c)
It's unnecessary, and causes issues when the uncompressed
output is large.
Bug: http://b/21558406
Change-Id: I99cfb3933b094c2826c7e6c6de9aab03478fcc53
(cherry picked from commit f899bd534b)
The objective of fallocate call seems to be to
make sure that we have enough enough space left
in the disk to house the uncompressed file.
But, fallocate is only supported in the following
file systems:
btrfs, ext4, ocfs2, and xfs
Return error only when fallocate fails due to
lack of space. The immediate ftruncate call is
going to take of the majority of other errors.
Bug: 21561449
Change-Id: I7083f3c7e5d745bd6e8a190ac9020297d638d9d4
Linking against both libziparchive-host and libz-host results in
libz loaded twice (because it is also linked statically in
libziparchive).
bug: 21365909
Change-Id: Ie924e0b38e4c0811daff06e7c0bfed048e7ca82f
We're already linking against libbase but we'll have to add
a libbase dependency to every target that includes libziparchive
as a STATIC_LIBRARY dependency, given that there's no way to
express that what we want (except by adding a LOCAL_WHOLE_STATIC_LIBRARY
dependency on libbase to libziparchive but that seems bad too)
Change-Id: I294ad389a9c61a1134a7bc323da25b0004a8f1e0
There were crashes when android failed to extract
zip archives due to out of free space on disk, with stacks like:
#00 pc 00000000000156b0 /system/lib64/libc.so (memcpy+176)
#01 pc 000000000002e5b8 /system/lib64/libandroidfw.so
#02 pc 000000000002f488 /system/lib64/libandroidfw.so (ExtractToMemory+328)
#03 pc 000000000002f5c8 /system/lib64/libandroidfw.so (ExtractEntryToFile+268)
#04 pc 00000000000287d8 /system/lib64/libandroidfw.so (android::ZipFileRO::\
uncompressEntry(void*, int) const+12)
Space for the file is now allocated using fallocate rather than
ftruncate, since ftruncate dont actually reserve space on disk. When writes
to the mmaped pointer fails to be realized due to out of space SIGBUS
is the result. With this change we make sure that there is space available
before mmaping the pointer.
Change-Id: Ic5e8c33dbe729edb57919dacff73811b34cc2dc2
libziparchive fails to iterate some bootanimation archives
reporting invalid offset error. This is caused by failure
to process a directory with one file
(when name_offset + file_name_length == cd_offset).
Change-Id: I2733e7f782c14a6fadd5491bb94318ac968df206
Nobody ever called acquire() so release() was always
equivalent to delete. Just use delete instead so that
people can use unique_ptr directly (or shared_ptr if
they really want refcounts).
Change-Id: I9e3ad5e0f6a4fcc4e02e5a2ff7ef9514fe234415
Removed LOCAL_C_INCLUDES which are no longer necessary as zlib
now uses the LOCAL_EXPORT_C_INCLUDE_DIRS mechanism.
Removed a couple of redundant LOCAL_MODULE entries
Tested compiling some of the projects that make use of libziparchive
and libziparchive-host using a clean out dir with aosp_flounder-userdebug
aosp_arm-eng targets
make -j3 libziparchive libandroidfw libartd libart libjavacore libjavacoretests aapt
Change-Id: I2988f1b3a62d1df2b5eada4ad39a9246ac1184fb
With mingw 4.8, the new default packed behavior is broken and makes
zip archive fails due to unexpected misalignment in
CentralDirectoryRecord.
This patch uses -mno-ms-bitfields to revert to the old packed
behavior.
Change-Id: Ic977c841e330e19451db1d31ddb22e570a525062
There should never be a need of an entry name with \0 character.
Bug: 16162465
(cherry picked from commit 78271ba97b)
Change-Id: I68c72fb45e8ec70eb125cfc887488bc18ba5447d
It's important because entry names can be encoded in UTF-8 and can have \0
character in the middle.
Use vector instead of char* for prefix in IterationHandle.
Bug: 16162465
Change-Id: Ie34c8d7c6231cc258530c22bdde5542895213649
Add new public method to allow checkisc if an archive has entry names encoded in
UTF-8. If not then they will be encoded in IBM PC character encoding.
Bug: 16162465
Change-Id: I4468d76accca8a9b0b31cae8d43399ffc22cad42
in StartIteration. This method should always be called when the
iteration is over to make sure that we don't leak memory.
Change-Id: I5205c754dfafbab9bb5f06003c3663d2ec4e8a35
Given that all current & future android ABIs are
little endian, we can get rid of the explicit conversions
from memory regions to little endian data members.
Also cleans up a few C style casts that snuck in during
several -Werror efforts and fixes temporary file generation
on target.
bug: 15448202
Change-Id: I4fcbb3c1124cb82c82139d328344e54fc7895353
Currently CloseArchive doesn't call free and call sites don't appear
to either. I could not find any call sites which manually freed the
archive by deleting the handle. This fixes several memory leaks.
Change-Id: I21f187dde60fd87e6e54bde06de9e76fd0791104
Two minor issues were fixed:
- The offset to entry data can be the same as the
central directory offset when the last entry in the
file has length 0 and is stored (not deflated). Fix
a check that disallowed this. We already have a strict
check that entry data must end before the central directory,
so we're covered.
- We would attempt to map a segment of length 0 when writing
an entry whose length is 0. We should just return early in
this case.
bug: 12623277
Change-Id: I2a4ca0c4d170cc3cbf326e5ca13894acd9c434c9
The issue here is that the makefile sets LOCAL_LDFLAGS
and asks for -llog without declaring a dependency on it.
This will cause build breakages if liblog hasn't already
been built yet.
Change-Id: Ia38761cda44a38b6c2a88902d6108cbe64a328ce
Unlike ALOGV, messages from ALOGD are logged on
all configurations. Not finding an entry in a zip
file is a "normal" occurrence so using an ALOGD
message for it amounts to spam.
Change-Id: I2c60d11e8a750be5106afd65c3c5e335f53f01b6
We would always write uncompressed data at offset 0 instead
of the current filedescriptor offset.
Also adds a unit-test & a clarifying comment on the API.
Change-Id: If44757e96dde504ce63d81b4dec7115fc6f6d5fb
We don't need a warning if an entry isn't found in a zip
file. It can happen as part of normal operation.
Change-Id: I86c132a040371f36f0dd981b49c02b3173821439
- Add a build rule for host tests
- Add basic tests for Find / Iterate and Extract
for both deflated & stored entries.
- Fix an off by one error that the test uncovered.
Change-Id: If72009b1ea9791d5a265829f05c32ffe1c2752c4
Extract zip file processing logic from libdvm into a
standalone library.
This library is a stricter than the libdvm library in
several ways:
- Duplicate zip entry names are now disallowed. Files with
such entries will fail to parse.
- We now verify CD file size information with the individual
file header information. (This was pointed out as a deficiency
of this implementation in past discussions.)
- We also add support for crc checking, which means we might
need to parse the optional data descriptor footer (if one
exists).
We also provide an API for iterating over the entries of
a zip file. This library is optimized for two use cases :
- Lookup for a single entry in the file, with the intention
of processing or extracting the data associated with that
entry
- Iterating over all entries in a file *and* processing
/ extracting their data.
Change-Id: Ia87de6184ef753cc470b0af755c47a4f92ac8198
