system/core/libziparchive/zip_archive.cc:847:36: error: use of undeclared identifier 'lseek64'; did you mean 'lseek'?
const off64_t current_offset = lseek64(fd, 0, SEEK_CUR);
Bug: N/A
Test: builds
Change-Id: If762011722d53376bb6dab35c6ee8031762e5a95
The libziparchive public headers that refer to `off64_t` also need the
Mac workaround.
In fastboot, there's a stray `lseek64` but since it's only for offset 0,
any kind of seek is fine.
Bug: N/A
Test: builds
Change-Id: I68b4f95202623ebf07ffe6c3e0e21437e7922c5b
This allows us to remove libziparchive's dependency on libutils.
Bug: http://b/79112958
Test: ran libbase and libziparchive tests, ran fastboot manually
Change-Id: I95c651976dad222863e5b8c37d4514b778f5dce7
A typedef to void* allows an implicit conversion from ZipArchiveHandle*
(or any other pointer type) to ZipArchiveHandle.
See I95d79809b6e118fb3c39c7b98b8055c8e324db1a in platform/bionic.
Bug: none
Test: m checkbuild
Change-Id: I3dd426cb64c46ef81e1dd81b4a2e4f40ac2701df
Bug: http://b/91353691
Enable -Wold-style-cast only for non-Windows targets. _islower_l,
_isupper_l etc. in MinGW locale_win32.h (included from
libcxx/include/__locale) have an old-style-cast.
Test: Build and test Windows modules under Wine.
Change-Id: Ib7594559a43096885b0cc1c656cf59db8b52d38b
Debugging memory allocations on the nexus launcher unveiled significant memory allocations for the hashatable used in libziparchive, ~1MB.
This is partly because of the ZipString struct storing an entry in the table. The struct stored a pointer to a string (on 64 bit, 8 bytes) and the length to read from that pointer, 2 bytes. Because of alignment, the structure consumed 16 bytes, wasting 6 bytes.
Now, we store entries in the hashtable as a ZipStringOffset. This new structure stores a 4 byte offset from a fixed location in the memory mapped file instead of the entire address, consuming 8 bytes with alignment.
Bug: 79416399
Test: Builds successfully and manual testing by opening launcher on Pixel 2 shows precisely 50% decrease in memory allocated for the hashtable. From 909312 bytes to 454656.
Change-Id: I28b43699233fbee7f63fccae2d4fe96fcc07e5c4
Allows for opening zip files usng paths longer than 260 characters and
with unicode characters on Windows.
Bug: 113110184
Test: manual and libziparchive_tests
Change-Id: I9ce96ac2f1b1e448ae2a2f69c1d4cb3395ea79ee
adbd has been built as a static executable since the same binary was
copied to the recovery partition where shared library is not supported.
However, since we now support shared library in the recovery partition,
adbd is built as a dynamic executable.
In addition, the dependency from adbd to libdebuggerd_handler is removed
as debuggerd is handled by the dynamic linker.
A few more modules in /system/core are marked as recovery_available:
true as they are transitive dependencies of the dynamic linker.
This change also includes ld.config.recovery.txt which is the linker
config file for the recovery mode. It is installed to /etc/ld.config.txt
and contains linker namespace config for the dynamic binaries under
/sbin.
Bug: 63673171
Test: `adb reboot recovery; adb devices` shows the device ID
Test: Select 'mount /system' in the recovery mode, then `adb shell`.
$ lsof -p `pidof adbd` shows that libm.so, libc.so, etc. are loaded from
the /lib directory.
Change-Id: I363d5a787863f1677ee40afb5d5841321ddaae77
We need to (a) tell soong to copy our data and (b) automatically find
our data relative to our executable.
The real point of this is to be able to run these tests in APCT and
presubmit.
Bug: N/A
Test: ran tests on host and device, from a variety of directories
Change-Id: I4c0be1ac60f03953fdd5ba6e3d15b1aaa37ed019
libziparchive is explicitly marked as double_loadable since it is one of the
(indirect) dependencies of the LLNDK library libvulkan
and at the same time the lib itself is marked as VNDK. Such lib can be
double loaded inside a vendor process.
Note: even without this change, the library is already capable of being
double loaded due to the dependency graph around it. This change is to
make it explicit so that double loading of a library is carefully
tracked and signed-off by the owner of the lib.
Bug: 77155589
Test: m -j
Merged-In: Id0a731d553bbb68b84bca421500c94b7b35eca14
Change-Id: Id0a731d553bbb68b84bca421500c94b7b35eca14
(cherry picked from commit 730728cbb4)
Fix failures in DataDescriptor related tests due to to a bad call to
SetZipString (undefined behaviour). Also fix a typo in the test for
invalid descriptors, we were asserting things on the wrong array.
Test: zip_archive_test
Change-Id: I8c9a632443fdf1d5c115670d6e9317e1f4bf6ef4
Moving to std::hash changed iteration order but these tests should
not have relied on hash_map iteration order anyway.
Test: zip_archive_test
Change-Id: I712bf2307c8770f03ea6f074bfc506a40cdcb066
Both Extract...() functions don't need dynamic allocation
for the writers, as those are strictly scoped. This CL
changes heap allocation to stack allocation.
Test: zip_archive_test
Change-Id: Id727e4b9848235cd063cc67ecbe052d21ca21326
Only compute the crc32 if required. In addition :
- Add unit tests for Inflate that cover this addition.
- Fix an inconsistency in return codes that was revealed
by this new test.
Bug: 35246701
Test: zip_archive_tests
Test: make; zipalign.
Merged-In: I31d7554378f94fc8995f707471d57cb98311e2c2
Change-Id: I05111bfa665c610f93d1c1dee987a509bf87aa65
Previously fastboot would carry on regardless if decompression failed:
fastboot: archive does not contain 'vbmeta.img'
fastboot: extracting vendor.img (260 MB)...
fastboot: W/ziparchive(56777): Zip: unable to allocate 272781472 bytes at offset 0 : No space left on device
fastboot: failed to extract 'vendor.img': I/O error
fastboot: archive does not contain 'vendor_other.img'
fastboot: wiping userdata...
This is because all but "boot" and "system" are considered "optional",
and the implementation of "optional" was "ignore any failures". What it
_should_ have meant was "it's okay if these don't exist, but if they do,
failures matter".
Fix this logic, use die() more aggressively, and remove spurious "\n"s
from die() format strings.
Also fix spurious whitespace in the libziparchive format string. Before:
Zip: unable to allocate 272781472 bytes at offset 0 : No space left on device
After:
Zip: unable to allocate 272781472 bytes at offset 0: No space left on device
Bug: http://b/68383022
Test: `fastboot update` on marlin
Change-Id: I3cbf55f1a33ca125f293f873eafbcfb86c880ba8
Remove dependencies on zip_archive specific data structures. In follow
up changes, this method will be promoted to a public API and used from
androidfw/ZipUtils as well as tools/zipalign, thereby allowing us to
remove a near-exact copy of this code and to simplify depdencies.
Test: zip_archive_test
Bug: 35246701
Change-Id: If24a9965fbd6fff308c8758859026684fd2af3b9
As a VNDK module, Android.bp must have 'vndk' tag as well as
'vendor_available: true'.
The 'vndk' tag for VNDK module is formated as below:
vndk: {
enabled: true,
},
VNDK modules will be installed both in system/lib(64) as normal and
in system/lib(64)/vndk as a vendor variant.
Bug: 63866913
Test: build and boot with BOARD_VNDK_VERSION=current
Merged-In: Iec5d3496e91a99f3e6b0c816c67ad279672ff36a
Change-Id: Iec5d3496e91a99f3e6b0c816c67ad279672ff36a
(cherry picked from commit 4e7e5b3ba053d013f2c4ae79d02722b874c629fb)
As a VNDK module, Android.bp must have 'vndk' tag as well as
'vendor_available: true'.
The 'vndk' tag for VNDK module is formated as below:
vndk: {
enabled: true,
},
VNDK modules will be installed both in system/lib(64) as normal and
in system/lib(64)/vndk as a vendor variant.
Bug: 63866913
Test: build and boot with BOARD_VNDK_VERSION=current
Change-Id: Iec5d3496e91a99f3e6b0c816c67ad279672ff36a
The use of ReadAtOffset is meant to allow concurrent access
to the zip archive once it has been loaded. There were places
where this was the case, and some places that did a seek + read
combination, which could lead to data races.
NOTE: On Windows, we are not using pread as the implementation of
ReadAtOffset, therefore the guarantees on Windows are weaker.
On Linux, pread allows the file descriptor to be read at a specific
offset without changing the read pointer. This allows inherited fd's
and duped fds to be read concurrently.
On Windows, we use the ReadFile API, which allows for an atomic seek +
read operation, but modifies the read pointer. This means that any mix
use of ReadAtOffset and Read will have races. Just using ReadAtOffset is
safe.
For the Windows case, this is fine as the libziparchive code now only
uses ReadAtOffset.
Bug: 62184114
Bug: 62101783
Test: make ziparchive-tests (existing tests pass)
Change-Id: Ia7f9a30af2216682cdd9d578d26e84bc46773bb9
libziparchive-host needs to include the headers correctly, too.
Bug: 37342627
Test: mmma system/core/libziparchive
Change-Id: I88a6d38ff9e494273040f9b913c71bccdda117ad
By forward-declaring zstream, we don't need to re-export zlib headers.
Bug: 37342627
Test: build
Test: BOARD_VNDK_VERSION=current m -j dumpstate (or any other module
using libziparchive)
Change-Id: Ia69e5d459c79878d491cfe4ca21a7cc8fed4a5d8
libziparchive headers are moved from the global include directory
(/system/core/include) to the local directory inside libziparchive.
Note: /system/core/include/ziparchive still exists as a symlink to
libarchive/include/ziparchive. This will be removed when there is no
header-only dependency to libziparchive.
Bug: 37342627
Test: build
Change-Id: I3631ffc2df7be8a064d64a625d10436090c3bb0f
We should check the boundary of central directory before checking its
signature. Swap the order of these two checks.
Bug: 36392138
Test: libziparchive doesn't read the signature after boundary check fails.
Merged-In: Ie89f709bb2d1ccb647116fb7ccb1e23c943e5ab8
Change-Id: Ie89f709bb2d1ccb647116fb7ccb1e23c943e5ab8
(cherry picked from commit 74464a1361)
Add a check to ensure the filename boundary doesn't exceed the mapped
memory region. Also add the corresponding unit test.
Bug: 28802225
Test: New unit test passes.
Merged-In: Ibf543a7da3d7898952e9eb332c84cdfc67cf5aa4
Change-Id: Ibf543a7da3d7898952e9eb332c84cdfc67cf5aa4
(cherry picked from commit bcc4431f24)
The boundary check of an invalid EOCD record may succeed due to the
overflow of uint32_t. Fix the check and add a unit test.
Test: Open the crash.apk and libziparchive reports the offset error as expected.
Bug: 31251826
Merged-In: I1d8092a19b73886a671bc9d291cfc27d65e3d236
Change-Id: I1d8092a19b73886a671bc9d291cfc27d65e3d236
(cherry picked from commit ae8180c06d)
Make them available to tests so that they can compare values
directly instead of comparing error code strings.
Test: zip_archive_test
Change-Id: I1c78c24651ff7549b5125a49a4e67bbd8295c617
- Ensure that the compressed and uncompressed lengths in the data
descriptor match those in the central directory.
- Calculate CRCs for compressed entries and ensure that those CRCs
match the information in the central directory / data descriptor.
This is currently guarded by a flag and will be turned on at a
later stage.
- Finally, always trust the LFH when it comes to indicating the
existence of a data descriptor. We no longer require this info to
match the central directory to bring this in line with other zip
processing tools. (see NOTE in the included change).
Bug: 62047801
Test: zip_archive_test
Change-Id: Icceea66a8b1937c08a4af9aee97bc77ad7218c0f
The Central File Header and Local File Header should
have the same general purpose bit flags set for
consistency.
Bug: 36686974
Test: existing tests pass (ziparchive-tests)
(cherry picked from commit 73b3aa541f)
Change-Id: Ia672a28732154a564ea2f2e3520238bb77924a56
This patch adds two benchmarks that measure the performance of some
operations of libziparchive.
Both benchmarks are creating a temporary zip file containing file
names of uniformly distributed lengths. The creation of the zip
file is not timed in the benchmarks.
- FindEntry_no_match tries to find an inexisting entry in the files
of the zip archive, in order to force the code to examine all the
files in the archive.
- Iterate_all_files uses the iterate function to list all the files
in the archive.
Bug: N/A
Test: adb shell /data/ziparchive-benchmarks
Change-Id: Ibdb524ba1c5ae55caddf0416ebbc09f8b6df0021
libziparchive is a library which belongs to vndk-cap. Mark it
vendor_available to enable header-abi-dumper's invocation to identify it
as a part of the vndk.
Details: https://android-review.googlesource.com/368372
Test: mm -j64
Bug: 38244611
Change-Id: Ibe490cc6c2cfca0f8d58df45317bb3a491d530f0
We change the GPB in the LocalFileHeader if the entry can not
have a trailing DataDescriptor. Make sure to patch the
CentralFileHeader to have the same bits set.
Modify ZipArchive to check that the data descriptor bit is consistent
between Central and Local file headers.
(cherry-pick of commit e0eca55fe6)
Test: make ziparchive-tests
Bug: 36686974
Change-Id: Ied167570abcf6426b1c678cd40123e5ad65909db
Older implementations of ZIP (Java's ZipInputStream) don't
like a Data Descriptor trailing after an uncompressed entry.
If the FILE is seekable, and the entry is uncompressed, seek
back to the header and write out the crc and sizes.
(cherry-pick of commit 639814d946)
Bug: 36686974
Test: make ziparchive_tests
Change-Id: I61664515e5afa3e2ba814874eeac847a2aaac319
Based on the compressed size of a file entry,
the decision needs to be made to instead store the file
uncompressed. This adds support to ZipWriter to backup
its last file entry.
The file is now always truncated when the EOCD is written out,
to account for the case where a file entry is backed-up and the
resulting file written is much smaller, leaving garbage data at
the end of the file.
This change also includes a rename of FileInfo -> FileEntry.
This struct was private (now public), so it shouldn't affect any
clients.
Bug: 35461578
Test: make ziparchive-tests
Change-Id: I23dc584406274ab7b8ce62b3fbc3562ca4c2603e
libutils, libz and libbase are being used as shared lib by many other
modules.
So using their shared lib will reduce total image size.
Size diffs on angler build image are as follows.
libziparchive.so : 103844 -> 41680 (-62164)
libnativeloader.so: 50824 -> 25104 (-25720)
total : (-87884)
Test: building succeeded, and the image was tested on angler.
Bug: 33056637
Change-Id: I015afe5b8f4d87d495b706e2e78d60f44a910e87
On 32 bit system those calls may fail if one tries to unpack files which
are bigger than 2GB.
Use large file system support to fix this problem.
Test: unpack a file bigger than 2GB on 32 bit Android systems
Change-Id: Ibd9bd5fc4a2f8dc7df98bd595f4fd1638a4f0d4a
Signed-off-by: Christian Poetzsch <christian.potzsch@imgtec.com>
FileWriter::Create() calls ftruncate(2) to truncate the destination
file, which doesn't work with FD that references a block device. It
leads to kIoError when calling ExtractEntryToFile() to extract an entry
to block device FD.
As a result, it fails the package_extract_file() command in OTA updates
(e.g. 'package_extract_file("boot.img",
"/dev/block/platform/soc.0/f9824900.sdhci/by-name/boot")').
This CL skips the call to ftruncate(2) if FD references a block device.
Bug: 32903624
Test: ziparchive-tests works.
Test: Build an OTA updater (m updater) and call package_extract_file().
Change-Id: Ia81116f1a8d7cab802396bdc32c6096b4cb56a3c
Point to log/log.h where necessary, define LOG_TAG where necessary.
Accept that private/android_logger.h is suitable replacement for
log/logger.h and android/log.h.
Correct liblog/README
Effectively a cleanup and controlled select revert of
'system/core: drop or replace log/logger.h' and
'system/core: Replace log/log.h with android/log.h'.
Test: compile
Bug: 30465923
Change-Id: Ic2ad157bad6f5efe2c6af293a73bb753300b17a2
Add two functions libziparchive that libminzip has. And create
corresponding unit tests.
1. Open a zip archive from a memory mapped region.
2. A new writer that takes a call back function pointer.
(Used by the OTA updater to stream the data.)
Test: Unit tests passed
Bug: 19472796
Change-Id: I2b2daec71174afe221030357e39bff5faea51e72
Should use android/log.h instead of log/log.h as a good example
to all others. Adjust header order to comply with Android Coding
standards.
Test: Compile
Bug: 26552300
Bug: 31289077
Change-Id: I33a8fb4e754d2dc4754d335660c450e0a67190fc
The boundary check of an invalid EOCD record may succeed due to the
overflow of uint32_t. Fix the check.
Test: Open the crash.apk and libziparchive reports the offset error as expected.
Bug: 31251826
Change-Id: I1d8092a19b73886a671bc9d291cfc27d65e3d236
(cherry picked from commit ae8180c06d)
system/core/include is included in the global include path using
-isystem, which hides all warnings. zlib.h is included through
system/core/include/ziparchive/zip_archive.h, which was hiding warnings
in it. Use a #pragma around the call to deflateInit2, it is a macro
that expands to an old-style cast, in preparation for moving from
-isystem to -I. Also move the ZipString constructor to zip_archive.cc
so it can assert on the length of the string parameter and fix an
implicit conversion from size_t to uint16_t.
Test: m -j native
Bug: 31492149
Change-Id: I74cdad7fe9c723859b5cfbea73c8f27d9d9ca265
