Commit graph

13 commits

Author SHA1 Message Date
Hyeongseok Kim
e8d02c50d7 libsparse: Fix overflow of merged sparse chunk length
Merging sparse chunk can make sparse map block bigger than 4GiB,
that can't be covered by unsigned integer type. Fix this by
changing unsigned int to uint64_t type.

Test: sparse build
Bug: 162808120
Change-Id: Id4d3f88f9d531c25c3937c99b2c81efb915605ee
Signed-off-by: Hyeongseok Kim <hyeongseok@gmail.com>
Cc: hyeongseok.kim <hyeongseok.kim@lge.com>
2020-08-11 08:34:28 +09:00
Tobias Thierer
fca4a9c279 Revert "libsparse: Add sparse typed callback"
This reverts commit db69f0d47f.

Reason for revert: Broke the build:

In file included from system/core/libsparse/sparse.cpp:26:
system/core/libsparse/output_file.h:34:72: error: unknown type name 'off64_t'; did you mean 'off_t'?
int (*skip_write)(void*, off64_t), void* priv,
^~~~~~~
off_t
/Applications/Xcode6.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/sys/_types/_off_t.h:30:25: note: 'off_t' declared here
typedef __darwin_off_t off_t;
^
1 error generated.

Bug: 78793464
Change-Id: I0f8bc4e9aa2f74612bfd8721d00d961e3f7e695f
2018-07-26 05:23:45 +00:00
Jerry Zhang
db69f0d47f libsparse: Add sparse typed callback
Currently, sparse_file_callback uses libsparse's
own logic for reading a file into a buffer. However,
a caller may want to use their own logic for doing
this in order to customize the buffer size and parallelize
the reads/writes. Also, a caller may want to implement
fill blocks with their own logic as well. To do this
add sparse_file_typed_callback which calls a different
callback function depending on the type of the sparse
chunk being written.

Test: Use typed callback for fd writes
Bug: 78793464
Change-Id: I75955a464fc05991f806339830fdfa05fda354b9
2018-07-25 11:04:03 -07:00
Jerry Zhang
7b444f08c1 libsparse: Add .clang-format and reformat to google3 style
Bug: 78793464
Test: compiles
Change-Id: I8e44ba77195a12fc2bac7d4276bbc4aa95149b31
2018-06-13 20:32:24 +00:00
Tao Bao
e18c03165b libsparse: Use 'size_t' for the 'len' parameter in callbacks.
This CL updates the callback function signature in
sparse_file_callback() and sparse_file_foreach_chunk().

Before:
int sparse_file_callback(
    struct sparse_file *s, bool sparse, bool crc,
    int (*write)(void *priv, const void *data, int len), void *priv);

int sparse_file_foreach_chunk(
    struct sparse_file *s, bool sparse, bool crc,
    int (*write)(
        void *priv, const void *data, int len, unsigned int block,
        unsigned int nr_blocks),
    void *priv);

After:
int sparse_file_callback(
    struct sparse_file *s, bool sparse, bool crc,
    int (*write)(void *priv, const void *data, size_t len), void *priv);

int sparse_file_foreach_chunk(
    struct sparse_file *s, bool sparse, bool crc,
    int (*write)(
        void *priv, const void *data, size_t len, unsigned int block,
        unsigned int nr_blocks),
    void *priv);

The length (i.e. 'len') comes from the size of a chunk, which could be
legitimately larger than INT_MAX. Prior to this CL, callers (e.g.
write_sparse_data_chunk()) were already passing unsigned int to the
callbacks. When a chunk size exceeds INT_MAX, the callback would see a
negative value, which could lead to undesired behavior. For example,
out_counter_write(), as one of the internal callbacks in libsparse,
gives a wrong sum of chunk sizes, which in turn fails the fastboot
flashing when given a huge sparse image.

It also defines SPARSE_CALLBACK_USES_SIZE_T that allows clients to keep
their codes compatibile with both versions.

Bug: 78432315
Test: `m dist` (with matching changes to all the clients)
Test: Build fastboot and successfully flash a previously failing (huge)
      sparse image.
Change-Id: Iac4bcf7b57039d08af3c57f4be00d75f6b693d93
2018-04-25 10:29:22 -07:00
Chris Fries
a7eeb22efb libsparse: Fix verbose_error string usage
Switch vnsprintf to StringPrintf to avoid off-by-one, and switch
sparse_read.c to cpp.

Test: fastboot flash normal image with smaller sparse limit

Change-Id: Ia399b167625deb271bfd0ee3273071306d71c4d4
2017-04-18 10:53:49 -05:00
Colin Cross
b43828b247 libsparse: rename symbols that conflict with libext4_utils
Until ext4_utils switches to using libsparse, libext4_utils defines some
of the same symbols as libsparse.  Fastboot links statically against
both of them, and there is no easy way to make the symbols hidden, so
just rename them in libsparse.

Change-Id: Idc2cfe20efe3c3a7fb8233f453a89bbbeb0dcc8b
2012-07-09 22:09:37 -07:00
Colin Cross
1e17b313a6 libsparse: add callback output file type
Add a new output file subclass that will call a callback for
each block as it is written.  Will be used to measure the space
used by each sparse block to allow resparsing files.

Also add sparse_file_callback, which will write out a sparse
file by calling the provided write function.

Change-Id: I18707bd9c357b68da319cc07982e93d1c2b2bee2
2012-07-09 22:09:37 -07:00
Colin Cross
b4cd267db3 libsparse: pseudo-subclass output_file for normal and gz files
Create two subclasses of output_file that can handle normal
and gzipped files, and refactor open_output_fd.  Will allow
adding support for an output_file type that is not file
backed.

Change-Id: I26744c74d13f205cf17df1ea9caac1eea9c57357
2012-07-09 22:09:37 -07:00
Colin Cross
13a5606593 libsparse: fix windows image writing
Fix write_fd_chunk on windows.  Uses malloc and read instead of mmap.

Change-Id: I75f10db2e04f19e7f3a6ff46b6978d143cb5254e
2012-07-09 22:09:37 -07:00
Colin Cross
9e1f17e926 libsparse: add support for including fds
Add sparse_file_add_fd to include all or part of the contents
of an fd in the output file.  Will be useful for re-sparsing files
where fd will point to the input sparse file.

Change-Id: I5d4ab07fb37231e8e9c1912f62a2968c8b0a00ef
2012-07-09 22:09:37 -07:00
Colin Cross
b55dceea98 libsparse: cleanups
Move block loops into sparse.c with iterator helpers in backed_block.c.
Simplify chunk writing by moving skip chunk calls from output_file.c to
sparse.c.
Rename variables to be consistent with new naming.
Remove use of u8, u32, u64.

Change-Id: Ic138ad58bef9f96239266ccee12ee83ea285e7eb
2012-07-09 22:09:37 -07:00
Colin Cross
28fa5bc347 system/core: move libsparse into system/core
This moves an exact copy of libsparse from
system/extras/ext4_utils/libsparse to system/core/libsparse in
preparation for linking tools in system/core against it.

Change-Id: If664e4fcfd6612844ac745589beb1517e7f9fe58
2012-07-09 22:09:36 -07:00