This patch supports compression for bigger block size.
3 bits [57-59] in the COW Operation "source_info_" field is used to store
the compression factor. Supported compression factors are power of 2
viz: 4k, 8k, 16k, 32k, 64k, 128k, 256k.
Only REPLACE operations will have the bigger block size support for now.
This can be extended to other operations later.
The write path in EmitBlocks() has the core logic wherein consecutive
sequence of REPLACE ops are compressed based on the compression factor
settings. Thus, for a 64k compression factor, there will be just one
COW operation which encodes all the 16 operation and the entire 64k
block is compressed in one shot.
NOTE: There is no read I/O path support in this patch. Subsequent patch
will have the read support.
Performance data (with read I/O path support in subsequent patch):
go/variable-block-vabc-perf covers detail performance runs
on Pixel 6 for full and incremental OTA.
TL;DR:
Performance of a full OTA (All numbers are compared against 4k block
size)
=======================================
Snapshot-size:
~10-11% decrease in snapshot-size (disk-space) for zstd with 256k block
size.
~8% decrease in snapshot-size (disk-space) for lz4
Install time:
~13% decrease in OTA install time for zstd with 256k block size.
Snapshot-merge:
~50% decrease in snapshot-merge time with 256k block size for zstd
Post OTA boot-time:
~10.5 decrease in boot time for 64k block size for zstd
In-memory footprint for COW operations:
~80% decrease in memory footprint for 256k block size. (58MB -> 9.2MB)
============================================
For more improvements, further tuning of zstd/lz4 is
required primariy the compression levels, zstd compression window,
performance of gz with compression levels.
Bug: 319309466
Test: cow_api test covering all the supported block sizes for v3 writer.
On Pixel 6:
=======================================
COW Writer V3:
for OTA in full, incremental OTA
for block_size in 4k, 16k, 32k, 64k, 128k, 256k
for compression_algo in lz4, zstd, gz, none
install OTA, reboot, verify merge
=======================================
COW Writer V2:
for OTA in full, incremental OTA
for block_size in 4k
for compression_algo in lz4, zstd, gz, none
install OTA, reboot, verity merge
=====================================
Change-Id: I96201f1609582aa9d44d8085852e284b0c4a426d
Signed-off-by: Akilesh Kailash <akailash@google.com>
Intermediate CL needed before variable block size can land. Since v3 is
enabled on cuttlefish, the base build needs to write the
compression_factor in order for reader to properly parse. Otherwise
we'll fail OTA test
Test: th
Change-Id: Ia353aae8e668858851073f09308909ae70d7854e
In the case that op_count_max is read in as zero, we should use the
upper bound of max blocks as the estimation. One case in which this
error can happen is if a v2 cow estimator is used, we should still be
able to run an OTA if we upper bound our ops buffer size estimation.
Test: th
Change-Id: I97ca66368d6631bf43c8911ed66f99c9e8096e2d
Parse manifest compression_factor and set CowOptions appropriately. This
allows v3 writer to use compression factor in OTA. Updating some
comments about supported compression algorithms
Test: th
Change-Id: I88f254087e536d9e5925064f85317f0acce280ee
Make the makefile safer by requiring a specific value for the
environment variable that turns on Secretkeeper
Bug: 306364873
Test: TreeHugger
Change-Id: Ic5bb5e7411a19941f58ec8c973104c1e53f3834f
The test is not eligible for CTS. Reasons:
1. The init behavior does not directly affect app compat. App interact
with init only for the property service and that part is covered by
the Bionic test already.
2. This test doesn't run against the init binary installed on the
device. libinit where most of the init functionalities are
implemented is statically linked to this test binary. In other words,
this test is closer to a unit test for init.
3. This test is not compatible with Trunk stable where test and DUT are
built in different branches. The test depends on several (private)
libraries like libbase and libutils. Since the interfaces of the
libraries may have changed in the main branch, the test binary built
from the old test-dev branch may break.
This change does not remove the test. The test will still run as a unit
test during pre/post submit.
I didn't drop the `Cts` prefix from the name, because that requires
broader changes.
Bug: 320800872
Test: N/A
Change-Id: I1402c08b79b57ad6daa7948fe37f14fbbe36f1d6
With variable block size compression being added, the number of ops
written cannot be calculated directly as easily since one op can cover
the data for multiple ops previously. We can get rid of this check for
XOR and Raw blocks as
within WriteOperation() we already make a check to see if we are
exceeding op_count_max limit.
We still need to keep this check for EmitZeroBlocks and EmitCopyBlocks
since the number of operations is determined ahead of time in those
function calls. Without this check in place, the ops will be added to
cached ops and return true when ops cannot be written.
with this change, v3 cow ota now works on cuttlefish with support for
variable block size compression.
Test: th
Change-Id: Ia55f152f5deb67a9022d0feff112345e72741dd3
Changes to structure of v3 header + operation needed for variable block
size. Seperating this CL from the variable block size one so we can get
v3 enabled on cuttlefish
the op count type changes are so that op count matches the type of
max_blocks. Max_blocks is used when op buffer size is not set -> we
default to upper bound of one operation per block in the partition.
Test: th
Bug: 307452468
Change-Id: I1a2581763a4fd6be5d5795f7e4781023e9984256
* changes:
libprocessgroup: Use pid_t consistently for TIDs
libprocessgroup: Use pid_t consistently for PIDs
libprocessgroup: Use uid_t consistently for UIDs
When doing an auto remount & reboot command and is running as the SHELL
uid, just try to gain root on behalf of the user and retry the remount
command.
If "gain root" failed, then print the message to tell the user to run
"adb root" and retry.
Bug: 322285923
Test: adb unroot && adb remount -vR
Test: adb unroot && adb shell remount -vR
Change-Id: If8e04dc602573c73178c108ef4944f0a985b590e
am skip reason: Merged-In Ia77ace7513c48b1a14290c6ecc0222b46d6bf927 with SHA-1 7e5925ef02 is already in history
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2913222
Change-Id: If019daaffca8d306980639276242eb441396a2d9
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
am skip reason: Merged-In Ia77ace7513c48b1a14290c6ecc0222b46d6bf927 with SHA-1 7e5925ef02 is already in history
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2913222
Change-Id: I1ded9c4c6d8b16020431c799352f55ec0f560f9b
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
am skip reason: Merged-In Ia77ace7513c48b1a14290c6ecc0222b46d6bf927 with SHA-1 7e5925ef02 is already in history
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2913222
Change-Id: I63c2e10b890a989627f744614d76d970a6bdb968
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
am skip reason: Merged-In Ia77ace7513c48b1a14290c6ecc0222b46d6bf927 with SHA-1 7e5925ef02 is already in history
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2913222
Change-Id: If890bd3499d5049024b7c2e4ab2125ec71d43822
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
am skip reason: Merged-In Ia77ace7513c48b1a14290c6ecc0222b46d6bf927 with SHA-1 d5ffbdfc56 is already in history
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2911926
Change-Id: I4d5617aae7ced0ca27e4a2fcf3e2edfbbdab39b1
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
am skip reason: Merged-In Ia77ace7513c48b1a14290c6ecc0222b46d6bf927 with SHA-1 d5ffbdfc56 is already in history
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2911926
Change-Id: I8b3c4bea0f60b7bbd44942179cf071a72cb41309
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
* changes:
Merge changes from topic "am-42c0dbea3ac34fc69f4fc2fd082669b3" into android13-tests-dev am: bc49a17b24 -s ours
[automerger skipped] Merge "Merge "sched_policy_test: set_sched_policy also set the cpuset policy as fallback" into android12-tests-dev am: ce34e32556" into android12L-tests-dev am: 421a3c69ad -s ours am: db8555c78e -s ours
am skip reason: Merged-In Ia77ace7513c48b1a14290c6ecc0222b46d6bf927 with SHA-1 7e5925ef02 is already in history
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2913222
Change-Id: I9534b7bb318e94e937d81fffa9cb5f6b5e3e6e16
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
