This enables read-ahead functionality by having
scratch space in the COW
Bug: 183863613
Test: OTA tests with new COW format
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I7988687c81d0ea239e71695818199db4653ddb80
kCowVersionManifest will be 2. This should now
be in sync with kCowVersionMajor.
Bug: 183863613
Test: OTA with new COW format (by enabling scratch space option)
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ia6c31e399de723ee83459b59d6b076b48f5c88d5
* changes:
libsnapshot:snapuserd:Add unit test for read-ahead code path.
libsnapshot: Flush data to scratch space only for overlapping regions
libsnapshot:snapuserd: read-ahead COW copy ops
libsnapshot: Retrieve COW version from update engine manifest
libsnapshot:snapuserd: Add 2MB scratch space in COW file
libsnapshot:snapuserd: mmap + msync header after merge
Now that mkbootfs is in prebuilt build tools, make it have no dynamic
dependency so that the binary is portable.
Bug: 184490452
Test: Presubmit
Change-Id: Ida4ee9af3c51ba9d163cf9c1e7b7098fd24e0de1
When read-ahead thread caches the data from base device, flush the data
only if there are overlapping regions. If there is crash, subsequent
reboot will not recover the data from scratch space. Rather, data
will be re-constructed from base device.
Additionally, allow batch merge of blocks by the kernel even for
overlapping region given that we have the read-ahead thread
taking care of the overlapping blocks.
Bug: 183863613
Test: 1: Incremental OTA from build 7284758 to 7288239. Merge time
reduces from ~6 minutes to ~2.5 minutes
2: Reboot and crash kernel multiple times when merge was in
progress
3: Verify read-ahead thread re-constructs the data for overlapping
region.
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I50e0d828f4fb36a23f0ca13b07a73229ba68874d
Introduce read-ahead mechanism for COW copy ops.
1: Read-ahead thread will read from base device
and store the data in scratch space along with the metadata.
2: Worker threads during merge will retrieve the data
from read-ahead cache
3: Fixed set of blocks are read during each cycle by the read-ahead
thread.
4: When the last block in the region is merged, read-ahead thread
makes forward progress.
Scratch space is set to 2MB and is only used from COW copy operations.
We can extend this to Replace Ops based on performance evaluation.
Performance:
As mentioned in bug 181883791, Incremental OTA of size 55M with
235K copy operations where every block is moved by 4k:
Without read-ahead: 40 Minutes for merge completion
With read-ahead: 21 Minutes for merge completion
Bug: 183863613
Test: 1: Full OTA - no regression observed.
2: Incremental OTA - with older COW format. Daemon will just skip
the read-ahead feature for older COW format.
3: Incremental OTA - with new COW format.
4: Reboot and crash kernel when multiple times when incremental OTA is in-flight.
Verify post reboot, read-ahead thread re-constructs the data from scratch
space.
5: No regression observed in RSS-Anon memory usage when merge in-flight.
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ic565bfbee3e9fcfc94af694596dbf44c0877639f
update_metadata.proto will have the COW version. Retrieve
that from the manifest and compare it with the COW library.
If the versioning doesn't match, disable VABC.
The primary use case of this is during downgrade tests
in pre-submit. Whenever we have a COW format changes,
we may have to disable VABC for that specific transition
build. At a high level, the flow of version check will be:
1: Create a initial COW version of 1 in manifest (update_metadata.proto)
2: The latest COW version of libsnapshot is 2
3: libsnapshot will return VABC disabled
4: Check-in the CL and changes to manifest
5: Once the CL is baked in and the build is green, bump up the COW version to 2 in the manifest
6: Next set of tests, since both versions match, libsnapshot will enable VABC
7: Downgrade should be done to the build which was checked in at (5)
Bug: 183863613
Test: Apply OTA and verify if VABC is disabled if the versions don't
match
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Id55f33a90bb31b417e72f4fbe370daf05a68f05a
Add 2MB scratch space in the COW file. This is a preparation
patch for read-ahead patch. This just add the buffer
space right after the header. Bump up the version number
in the header in order to distiguish between older and newer
COW formats.
No operation is done on this buffer with this patch-set.
Scratch space option is disabled by default.
Bug: 183863613
Test: 1: Create Full OTA with the new COW format.
2: Incremental OTA with older COW format.
3: vts_libsnapshot_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I42a535a48ec22adb893dfe6f86a4f51650e1f88a
mmap the CowHeader and use msync to flush only the
first 4k page after merge is complete.
This cuts down ~30 seconds of merge completion time
on a 55M incremental OTA with 235k copy operations.
Although, this isn't a significant gain but this patch
creates a scaffolding for the next set of read-ahead patches.
Bug: 183863613
Test: Incremental and Full OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I15bfec91ea1d5bdf4390670bcf406e1015b79299
Adds the -b option to show the bad data block that failed to decompress.
If the block is large enough, display the front as though it were a
CowOperation, as this is the most likely culprit.
Change-Id: I287f13e0794a1ca9d647d4b1099ab238a6202b23
Bug: 183985866
Test: inspect_cow -db <COW_FILE>
On first boot, FDE devices hang on the command
'wait_for_prop apexd.status activated'. This is because apexd was
already started with the tmpfs /data, then was stopped by
vold.decrypt=trigger_shutdown_framework. Then when apexd is started
again with the real /data, it sees that apexd.status="ready" already, so
it doesn't consider itself to be starting from scratch again. So it
doesn't move apexd.status back to "activated" as expected.
Fix the above by resetting apexd.status to its initial value of the
empty string before trying to start apexd in the post-fs-data trigger.
Note that this also takes care of the userspace reboot case which was
previously handled in the userspace-reboot-requested trigger.
Also, FDE devices hang at the same place on non-first boots with default
encryption (i.e., when no PIN is set) because apexd is still running
after having been started with the tmpfs /data. This is because
vold.decrypt=trigger_shutdown_framework isn't run in that case, but
rather vold manually kills processes that have open files on /data --
which doesn't include apexd. But, apexd should be restarted too.
Fix that by using 'restart apexd' rather than 'start apexd'.
Note that these changes are needed even though FDE devices don't support
updatable APEXes, as apexd is needed regardless.
This is one of a set of changes that is needed to get FDE working again
so that devices that launched with FDE can be upgraded to Android 12.
Bug: 186165644
Test: Tested FDE on Cuttlefish. Also tested userspace reboot (with FBE)
Change-Id: I4fa57cf15d77b64d1167eaf966347d2a9d6a9b72
If precompiled vendor policy has system_ext hash, system_ext also has to
have its hash, to use precompiled sepolicy.
Bug: 186727553
Test: remove system_ext's hash and see sepolicy compiled in runtime
Change-Id: I4af3418d614156b5e9cd0b0116c2814ba994ee81
The existing code has a lot of references to the
`ro.boot.qemu` and `ro.boot.qemu.something` properties
which is not supported by the bootconfig if we place
everything under `androidboot.qemu`.
Bug: 182291166
Test: getprop | grep qemu
Signed-off-by: Roman Kiryanov <rkir@google.com>
Change-Id: Icb9d29c8dc39e1fa52a6f2ce43b4f42182b7995d
This test previously expected failure - a machine which does not have
2GiB of memory. However, while today is becoming the past,
2GiB allocations working is no longer a dream of the future!
Fixes: 186569165
Test: libutils_test
Change-Id: I6a9ed608c0989d415b4e7735b8a451b8928b4083
And use the new fs_mgr C++ API.
Bug: 186504948
Test: atest CtsFsMgrTestCases
Test: atest CtsFsMgrTestCases in DSU
Change-Id: I80d68d72fb3aa9f5a0dc7d9400b4d0f80cc7deb3
Currently the gki_4_19_pixel5 presubmit test uses an old
vendor_boot-debug.img from a release branch. Adding fallback
paths to load debug resources from /first_stage_ramdisk dir to
pass the presubmit.
This CL should be reverted later once the vendor_boot-debug.img
gets updated to store the debug resources on the root dir.
Bug: 186082603
Test: boot a device with boot-debug.img
Test: boot a device with vendor_boot-debug.img
Change-Id: I9fcd77fc5a60a15cff254e432e05f1c9122ad80d
Currently the debug resources might under /first_stage_ramdisk/*
of the ramdisk, if there is androidboot.force_normal_boot=1 in the
kernel cmdline to request init chroot into /first_stage_ramdisk dir.
To make a generic boot-debug.img works on devices with and without
this chroot, moving the debug resources to the root of the ramdisk.
And copy them for later use before the chroot.
Bug: 186082603
Test: boot a device with boot-debug.img
Test: boot a device with vendor_boot-debug.img
Change-Id: I052a92b2d26c7fdf749991fc55015ff68743efc2
If one end of the communication socket is closed for some reason, there
is no need to terminate the daemon or the client. Mask
the SIGPIPE using MSG_NOSIGNAL flag - we will
still get EPIPE error but process will not be terminated.
Bug: 186213024
Test: Full OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Iaa53545c0c4059618f6b49afb9ec24ea5372c7e0
Issue a warning about missing cpu/schedtune controller only if both of
them are missing.
Bug: 185437398
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Change-Id: I3a9d3c9a8c91c8d2c5346bcb431bb0407c64a811
Soong generates classpaths.proto config and puts it into
/system/etc/classpaths/ for derive_classpath to read at runtime. There
is no need to plumb these values via make anymore.
Bug: 180105615
Test: m && launch_cvd; presubmit / DeviceBootTest
Change-Id: I514c5036871233ae865b972effea8321dbe4aea9
Remove the vestigial llndk_library modules and replace them with
properties in the llndk clause of the implementation cc_library.
Bug: 170784825
Test: m checkbuild
Test: compare out/soong/build.ninja
Change-Id: Ie3a1bffcf29bb1b6747f7f708826c61bd43ba5a1
