platform_system_core/fs_mgr/libsnapshot
Akilesh Kailash 59fa486703 libsnapshot_cow: Support multi-block compression
This patch supports compression for bigger block size.

3 bits [57-59] in the COW Operation "source_info_" field is used to store
the compression factor. Supported compression factors are power of 2
viz: 4k, 8k, 16k, 32k, 64k, 128k, 256k.

Only REPLACE operations will have the bigger block size support for now.
This can be extended to other operations later.

The write path in EmitBlocks() has the core logic wherein consecutive
sequence of REPLACE ops are compressed based on the compression factor
settings. Thus, for a 64k compression factor, there will be just one
COW operation which encodes all the 16 operation and the entire 64k
block is compressed in one shot.

NOTE: There is no read I/O path support in this patch. Subsequent patch
will have the read support.

Performance data (with read I/O path support in subsequent patch):

go/variable-block-vabc-perf covers detail performance runs
on Pixel 6 for full and incremental OTA.

TL;DR:

Performance of a full OTA (All numbers are compared against 4k block
size)
=======================================
Snapshot-size:

~10-11% decrease in snapshot-size (disk-space) for zstd with 256k block
size.

~8% decrease in snapshot-size (disk-space) for lz4

Install time:

~13% decrease in OTA install time for zstd with 256k block size.

Snapshot-merge:

~50% decrease in snapshot-merge time with 256k block size for zstd

Post OTA boot-time:

~10.5 decrease in boot time for 64k block size for zstd

In-memory footprint for COW operations:

~80% decrease in memory footprint for 256k block size. (58MB -> 9.2MB)

============================================

For more improvements, further tuning of zstd/lz4 is
required primariy the compression levels, zstd compression window,
performance of gz with compression levels.

Bug: 319309466

Test: cow_api test covering all the supported block sizes for v3 writer.

On Pixel 6:

=======================================
COW Writer V3:

for OTA in full, incremental OTA
   for block_size in 4k, 16k, 32k, 64k, 128k, 256k
       for compression_algo in lz4, zstd, gz, none
          install OTA, reboot, verify merge
=======================================
COW Writer V2:

for OTA in full, incremental OTA
   for block_size in 4k
      for compression_algo in lz4, zstd, gz, none
          install OTA, reboot, verity merge

=====================================
Change-Id: I96201f1609582aa9d44d8085852e284b0c4a426d
Signed-off-by: Akilesh Kailash <akailash@google.com>
2024-01-31 12:52:31 -08:00
..
android/snapshot libsnapshot: use compression_factor in ota 2024-01-30 09:25:02 -08:00
include/libsnapshot libsnapshot_cow: Support multi-block compression 2024-01-31 12:52:31 -08:00
include_test/libsnapshot libsnapshot: Remove ISnapshotWriter. 2023-06-14 12:03:58 -07:00
libsnapshot_cow libsnapshot_cow: Support multi-block compression 2024-01-31 12:52:31 -08:00
scripts Add LOCAL_LICENSE_KINDS to system/core/fs_mgr 2021-09-15 10:57:11 -07:00
snapuserd Set taskprofile to snapshot merge thread 2024-01-09 03:24:10 +00:00
tools libsnapshot: Add a test case for recent decompress regression. 2023-11-11 00:29:41 +00:00
Android.bp libsnapshot_cow: Support multi-block compression 2024-01-31 12:52:31 -08:00
device_info.cpp Convert fastboot/libsnapshot to new BootControl client 2022-07-08 10:57:53 -07:00
device_info.h Convert fastboot/libsnapshot to new BootControl client 2022-07-08 10:57:53 -07:00
dm_snapshot_internals.h libsnapshot: Use std::unordered_set in DmSnapCowSizeCalculator. 2021-07-30 11:02:28 -07:00
OWNERS Update bug component from Android Systems to OTA client 2023-08-22 10:18:47 +01:00
partition_cow_creator.cpp libsnapshot: get options from protobuf fields 2024-01-03 11:09:37 -08:00
partition_cow_creator.h libsnapshot: use compression_factor in ota 2024-01-30 09:25:02 -08:00
partition_cow_creator_test.cpp libsnapshot: Clarify the meaning of "compression.enabled". 2022-07-27 18:32:15 -07:00
return.cpp libsnapshot: delete WaitForMerge. 2020-03-03 13:19:49 -08:00
snapshot.cpp libsnapshot: update type 2024-01-30 09:25:23 -08:00
snapshot_metadata_updater.cpp Merge "libsnapshot: Properly set partial_update_" 2020-07-31 23:02:05 +00:00
snapshot_metadata_updater.h Update libsnapshot to handle partial update 2020-06-09 12:53:45 -07:00
snapshot_metadata_updater_test.cpp libsnapshot_test: skip global setup on non-VAB devices. 2020-08-06 17:21:50 -07:00
snapshot_stats.cpp libsnapshot: Add more feature flags to SnapshotMergeReport. 2022-05-25 13:40:31 -07:00
snapshot_stub.cpp libsnapshot: Remove ISnapshotWriter. 2023-06-14 12:03:58 -07:00
snapshot_test.cpp libsnapshot: Detach the daemon explicitly before stopping the service 2023-12-18 17:22:06 -08:00
snapshotctl.cpp snapshotctl: fsync after writing every 1MB buffer 2023-11-29 11:26:46 -08:00
test_helpers.cpp libsnapshot: Remove ISnapshotWriter. 2023-06-14 12:03:58 -07:00
utility.cpp libsnapshot: Fix test failures on certain configurations. 2023-03-06 23:21:25 +00:00
utility.h libsnapshot: Fix test failures on certain configurations. 2023-03-06 23:21:25 +00:00
vts_ota_config_test.cpp Some device that platform base on android12 doesn't support A/B update, it looks virtual A/B is also not mandatory. so this test case should check ro.vendor.api_level first. 2022-09-17 06:47:02 +00:00