Adding small binary to write a simple COW for version 2. We will then
use updated reader to make sure we can read this version 2 cow. Think it
would be a good idea to keep the binary here to see how exactly we
generated the cow (useful for debugging and if we ever need to recreate
this cow)
Test: m basic_v2_cow_writer
Change-Id: I28435025b7a8280fc5c4574876cc9110b391cb0e
Target block hash was in-correctly getting added to map thereby
overriding the source-hash.
Bug: 299011882
Test: Flash Pixel 6 Pro from A->B and from B->A
Change-Id: Ib3887c29cd6b8f2abd50e932273a5cfc2a096bd5
Signed-off-by: Akilesh Kailash <akailash@google.com>
std::async returns a std::future whose destructor blocks until the
async function has completed, which defeats the purpose of using
std::async. The future needs to be kept alive to allow the function to
run concurrently.
Starting in C++20, std::async marked [[nodiscard]] to help catch this
misuse. Upgrading libc++ adds the [[nodiscard]] attribute to
std::async, so fixing this bug is necessary to keep the code compiling
after libc++ is updated.
Bug: 175635923
Test: treehugger
Test: m && m snapuserd_test
Change-Id: Id2e820248c2b6111aa843fb709e08a2c19677066
Updating comments to match current format. Scratch space exists in
between header and operation
Test: th
Change-Id: I2f86e9dc4078f03370cdc38918136c894c6ca484
this is going to replace the current version of CowOperation and will
work with writer v2 and parser v2. This will be the on disk format of
the cow, while CowOperation will be updated to be the in memory format
of 15 bytes (implicitly will be the v3 version).
Test: m libsnapshot
Test: m libsnapshot
Change-Id: Ibd00ef014a9fc11cdf2bad97c52462db8eef9502
This is to support when partition sizes are greater than 2GB (2^31)
on 32-bit userspace.
Bug: 300178204
Test: OTA on device with 32-bit userspace + product partition > 2GB
Change-Id: I7074682352d8388ed410c684cb7cb0fa346ba24c
Signed-off-by: Akilesh Kailash <akailash@google.com>
Adding block sz to compressor classes to prepare for variable block size
compression
Test: m libsnapshot
Change-Id: I84db20c80c0f95188f79ccc73b5c30678bd75e78
The fstab provided by the user/caller might not be the default fstab,
which might not include the /cache mount entry. We should just use the
procfs mount info to determine if /cache is currently mounted.
Bug: 300036012
Test: adb_remount test
Change-Id: I4643d0a21ae21f3513f715de424f0be1fe64ff9e
Two new API's have been added:
1: BootFromSnapshotsWithoutSlotSwitch: This will create a new marker
which indicates first-stage init to mount the partitions off snapshots.
We need this marker as during boot, there are couple of places during
mounting snapshots wherein the marker is used. However, there is no
change in the existing I/O path related to OTA.
2: PrepareDeviceToBootWithoutSnapshot: This will delete the marker so
that subsequent reboot will not have the partitions mounted without the
snapshots.
VTS tests covers both these API's. Additionally, when these
markers are present, new OTA's cannot be installed. All these
are covered in VTS tests.
===========================================================
snapshotctl: General flow to apply and revert pre-created snapshots
1: To install the pre-created snapshots:
$snapshotctl map-snapshots <directory path containing snapshots patches>
Now the device is ready to boot from snapshots.
2: After device reboots, partitions are mounted off the snapshots. There
is no snapshot-merge.
3: In order to go back to previous build:
$snapshotctl revert-snapshots
Now the device is ready to boot from base build.
4: After device reboots back to previous build, all the snapshot states
and COW images are removed.
============================================
Additional commands:
To delete the pre-created snapshots:
$snapshotctl delete-snapshots
======================================
Tested it on Pixel 6 Pro between two builds which are ~24 hours apart.
1: Creating snapshots on a linux-host - ~4-6 seconds
2: Applying pre-created snapshots - ~10-15 seconds (includes intermediate
transfer of patches to the device). This depends on the size of snapshot patches.
3: Device reboot - ~12-14 seconds.
Bug: 299011882
Test: 1: Apply pre-created snapshots
2: Reboot device: Verify new build
3: Apply OTA when partitions are mounted of snapshots and verify OTA
fails.
3: Revert-snapshot and reboot.
4: Verify device goes back to base build.
Full OTA on Pixel. vts_libsnapshot_test
Change-Id: I36a72d973d8f70ae49773ebd45dd996fac22a4e3
Signed-off-by: Akilesh Kailash <akailash@google.com>
Similar to fastboot binaries. This will ease the
automation to use this tool
Test: presubmit
Bug: 290951369
Change-Id: I5c879acc7cdecbafebfa074ef76034403cb0cd72
Extend snapshotctl binary on the device
to create and apply pre-created snapshots.
snapshotctl map-snapshots <directory-where snapshot patches are present>
1: snapshotctl creates the block device based on the size of the
snapshot patches.
2: snapshotctl will copy the data to the block device.
unmap and delete snapshots will cleanup.
No change in libsnapshot library except with minor refactoring.
This patch doesn't yet prepare the device to reboot from these
snapshots.
On Pixel 6 Pro, applying pre-created snapshots for all partitions
takes ~3 seconds wherein the delta between two builds are 24 hours apart.
Bug: 299011882
Test: snapshotctl map-snapshots /data/test-snapshots/
snapshotctl unmap-snapshots
snapshotctl delete-snapshots
Change-Id: I98a0fbd9cf7234c2188bad85cdd092ded8997710
Signed-off-by: Akilesh Kailash <akailash@google.com>
This is a host based tool wherein it compares two Android images and generates snapshot patches which are similar to Android OTA format.
There are few advantages:
1: All the computation of snapshot logic is pushed onto the host.
2: Each partition can have different compression algorithm.
3: All the libsnapshot_cow changes can be tested very quickly.
Here is the test run. This compares two builds which are 24 hours apart.
```
create_snapshot --source=$ANDROID_PRODUCT_OUT/system.img --target=./images/system.img --compression="zstd" &
create_snapshot --source=$ANDROID_PRODUCT_OUT/product.img --target=./images/product.img --compression="lz4" &
create_snapshot --source=$ANDROID_PRODUCT_OUT/vendor.img --target=./images/vendor.img &
create_snapshot --source=$ANDROID_PRODUCT_OUT/system_ext.img --target=./images/system_ext.img --compression="gz" &
create_snapshot --source=$ANDROID_PRODUCT_OUT/vendor_dlkm.img --target=./images/vendor_dlkm.img &
echo "Waiting for snapshot patch creation"
wait $(jobs -p)
echo "Snapshot patch creation completed"
```
========================================
Waiting for snapshot patch creation
Snapshot patch: vendor_dlkm.patch created successfully
Snapshot patch: vendor.patch created successfully
Snapshot patch: system_ext.patch created successfully
Snapshot patch: product.patch created successfully
Snapshot patch: system.patch created successfully
Snapshot patch creation completed
real 0m3.848s
user 0m14.239s
sys 0m8.045s
========================================
It takes ~4 seconds to generate the snapshot patches on the host. Snapshot patches are named as <partition-name>.patch.
Bug: 299011882
Test: create_snapshot between two builds as mentioned above.
Change-Id: Ic87dd3349a866b5626fa03f1f879f417a8116cc2
Signed-off-by: Akilesh Kailash <akailash@google.com>
Adding test to measure performance differences between cow compression
algorithms + levels. This gives us an easy way to test performance
between the tunables without having to run an OTA every time. Ultimately we want this to be separate from
cow_api_test so it would be nice to have this be it's own binary. Can
add some tests for decompression + compressing from a part of an actual
img file too.
Test: m cow_benchmark
Change-Id: Iba92ae3c0b2ad4ff6f842556b701b223d7d37823
Ensuring that the expression is evaluated before being outputed. In
response to comment from aosp/2708333
Test: th
Change-Id: I073c143f1c4e32af11e235ce782947b250117dc5
It's been a while since the Android Systems bug component does not
access the creation of new bugs for triage, instead, new bugs must
be created in the subcomponents of the team.
This change updates the bug component originally associated to this
subsystem from the Android Systems' 30545 to OTA client's 1014951.
Test: none
Bug: 270571229
Change-Id: I5455086bf777dd20abb2e2f0360f76a003120126
Signed-off-by: Alessio Balsini <balsini@google.com>
Fix zstd to read from ignore_buf rather than buf since that is where we
are first copying the date
Test: zstd ota
Change-Id: I5032300e4628ecd7e49f1fa9f76dc9a828fb58e6
remove some unused headers and fix warning: Moving a temporary object prevents copy elision
Test: m libsnapshot
Change-Id: Idec3e051837dab5f1b95e677d1cdb09e9a57e73e
ZSTD_c_windowLog should be set as log base 2 of max block size. Changing
hardcoded value to be determined by this variable.
Test: m libsnapshot
Change-Id: I09be447b7f1e95cb72a6a42eddb4035f61a43aad
The pre-conditions and post-conditions to cv.wait() in this function are
different, which leads to a race. If the merge is never initiated, but
I/O is terminated, the wait() will hang.
This is easy to hit in tests where I/O terminates quickly, but should
not affect actual OTA flow in practice.
Bug: 269361087
Test: snapuserd_test
Change-Id: I85ed9c357031ec6fd74d6a3f49d85e6cbb037eee
reuse the same context for zstd compression. One context should be used
per compression thread for optimal performance. Discussion and
reccomendations can be found at go/android-hw-compression-vendor
Results (level 3 compression)
full ota on raven with optimizations: 1191.304 seconds
without optimizations: 1461.854 seconds
compression ratio remains unchanged and merge time difference are
negligible ~1% delta (probably just noise)
Test: ota_from_target_files, update_device.py
Change-Id: I3feede9f1f119874e369c54b29c594475fbf7376
Adding in compressor class to clean up code for cow_compress.cpp.
Since we are making some api changes (to zstd) that are unique to each
compression methods, these should be implementation details should be
hidden to the parent class
Test: m libsnapshot
Change-Id: I9194e2c615aefed078458f525382253228bc1b69
We may have snapshot files in /metadata/ota/snapshot/ which ends with
.tmp such as system_a.tmp - This happens if the device
reboots just before `rename` in `WriteStringToFileAtomic`. This
can lead to spurious merge failures.
Log the error and skip these snapshot files. It is ok to skip
as we will still have original snapshot status files since
we are already in the merge path. Additionally, try to remove
these files when snapshot is deleted.
Bug: 292198189
Test: OTA
Change-Id: I5db3dbd5a919b263ae577185de3e7f79a5e9b89a
Signed-off-by: Akilesh Kailash <akailash@google.com>
