The user interface of Android devices becomes very slow under memory
pressure. This is because Android uses the zram driver on top of the loop
driver for swapping, because under memory pressure the swap code alternates
reads and writes quickly, because mq-deadline is the default scheduler for
loop devices and because mq-deadline delays writes by five seconds for such
a workload with default settings. Fix this by selecting I/O scheduler 'none'
for loop devices.
Bug: 194450129
Test: Built Android images, installed these and verified that the I/O scheduler of all loop devices is 'none' instead of 'mq-deadline'.
Change-Id: Ia5f606504b663948ab56955cad5a71885a356430
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Improve code readability by renaming 'device' into 'loop_device' and
'device_fd' into 'loop_fd'.
Bug: 194894000
Test: Built Android images and installed these on an Android device.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Change-Id: Ia9c2d7a525e727f8706e66631b97fc4678c6a4d9
When the device is rebooted after OTA is done,
daemon will read through all the dynamic partitions
to bring the blocks to memory. When update-verifier
runs, all the blocks would be in page-cache thereby
cutting down the boot time.
Boot time improvements on Pixel:
Full OTA:
===========
VABC (Without this patch): 37.308 seconds
VABC (With this patch): 28.604 seconds
Incremental OTA:
=================
VABC (Without this patch): 39.072 seconds
VABC (With this patch): 27.523 seconds
We read the blocks only during second stage transition. Thus,
it shouldn't impact when snapuserd is spin up during
post-install phase or during first-stage init.
Bug: 193863442
Test: Full and Incremental OTA on pixel
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Id654449238e22125f7d6288c7100fde512cc2ced
* changes:
libsnapshot: Don't PrepMergeOps on resume
snapuserd: Add support for Xor ops in snapuserd
snapuserd: Rename Read Ahead Iterator
snapuserd: Add XorSink
libsnapshot: Clone worker readers from snapuserd
libsnapshot: Add support for Xor ops in Cow Format
If this flag is given, then fs_mgr_mount_overlayfs_fstab_entry() shall
filter out missing directories in the lowerdir= list.
For example,
test /mnt/vendor/overlay_test overlay \
ro,lowerdir=/dir1:/dir2:/missing_dir3 \
first_stage_mount,overlayfs_remove_missing_lowerdir
should mount the overlayfs device with "lowerdir=/dir1:/dir2".
Bug: 186342252
Test: Manual boot test with modified fstab on CF
Change-Id: Id06b37d0c236528cef981e495280b4f4e9c2b4bb
Look for the fstab file in /system/etc as an alternative to /, in order
to allow fstab files to be installed using the "prebuilt_etc" Soong
module.
This new path is meant to be used by the vendor ramdisk only. As before,
fstabs should *not* be placed in /system/etc on the system partition.
In more detail: sometimes, multiple nearly-identical fstabs need to be
installed to a device, with the correct one being selected at boot time
(b/191417025 as well as other cases that partners have run into). To
avoid error-prone duplication of configuration files, these fstabs
should be generated from a template by the build system instead of being
duplicated in the source tree. But if this is done, the usual way of
installing fstabs (PRODUCT_COPY_FILES) can't be used; they need to be
made into real build system modules instead.
Currently, the "prebuilt_etc" Soong module can't correctly install the
vendor_ramdisk copy of the fstab(s), since it will install it into the
/system/etc directory whereas Android currently requires that the
vendor_ramdisk copy of the fstab(s) be placed in the root directory.
Earlier I proposed adding a "prebuilt_fstab" module to handle this quirk
(https://r.android.com/1744033). However, it was requested to instead
always look for the fstabs in /etc too, in order to allow "prebuilt_etc"
to be used and because /etc is the appropriate place for this file.
This change implements that suggestion (but actually using /system/etc,
since that is where "prebuilt_etc" actually installs it).
Bug: 191417025
Test: Tested that a device boots both with this, both before and after
http://ag/15075136 which uses the new location.
Change-Id: Id083070e51ae85959167e4615cd96b31a0b1bd6a
If we're reading up to a label, we're resuming setting up the file, and
there is no reason to expect the ops we require for sequence ops to be
present. In that case, skip prepping for merge, and return an empty
merge iterator if it is mistakenly requested.
Test: cow_api_test CowTest.ResumeSeqOp
Change-Id: Idd93bd4c4209197b9728fcb21a7191aae971b62d
This adds the ability to read the blocks corresponding to xor operations
from snapuserd. Xor Operations should be treated the same as copy
operations during the merge, but before their data is written to disk,
it must be xor'ed against the operation's data. For the purposes of
readahead, this acts as a copy op. Post readahead, it acts more like a
replace op.
Change-Id: I7d74bbdd43bfc5277ef4c8aade57dd375be9180a
Bug: 177104308
Test: vts_libsnapshot_test and cow_snapuserd_test
We can compute the xor while decompessing by computing it within a
buffer. This would allow us to use a smaller buffer than the full block
size if we wish to cut down on the extra memory required.
Bug: 177104308
Test: cow_snapuserd_test (Snapuserd_Test.xor_buffer)
Change-Id: Id18505841c77340760cea2d20e83454f1142b9b5
We'll need to have the Cow Files parsed to deal with xor ops, since
their data location is implicit from the format. Since the relevant data
is all stored in shared pointers, we can pass that data into the workers
without needing to reparse or copy.
Bug: 177104308
Test: builds
Change-Id: I96ac3da1ae620be48e5340c9f146c523b3ce74b6
This adds support for Xor Ops in the Cow Format. These represent store
possibly compressed data which must be xor'ed against the given section
of source data to be interpreted as a block in the new image. The cow
reader and writer do not have access to this data, so they assume the
provider and user of the data will handle the xor-ing.
Bug: 177104308
Test: cow_api_test (ReadWriteXor)
Change-Id: I7a313d2643773d6b81a878a3e5bc87e3bdfc387b
There is a check here if vector resize fails. In practice, this would
throw bad_alloc or length_error and cause a runtime abort, so the check
is dead code.
To protect against bad chunk_ids we can switch to unordered_set instead.
The original memory concerns for std::set are less applicable since
unordered_set is bucketed.
Bug: 194431534
Test: apply OTA; run vts_libsnapshot_tests
Change-Id: I09c108b700d2f83acf80a9eaa5099b46aedcab89
This uses a property to communicate when snapuserd is ready to accept
connections, to eliminate some non-determinism from the connect loop.
Bug: 193833730
Test: update_engine connects to snapuserd
Change-Id: I2fc0063486c3df542db1e60d12564a5bc1581a37
This is a corner case wherein a crash during OTA
merge can lead to missing of some COW operations to be
merged thereby some blocks may end up with stale data.
Fix here is to avoid any re-ordering of COW operations.
Merge the COW operations as present in the COW file.
New tests have been added to cow_snapuserd.
Bug: 194955361
Test: cow_snapuserd_test, Incremental OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Id895fe7a3d6b4510676490a86d0caf62dec9b079
Enhanced the checkpoint=disable GC threshold fallback mechanism by first
dropping to checkpoint=disable before dropping the option completely for
kernel compatibility.
Test: atest vts_kernel_checkpoint_test
Bug: 193216993
Change-Id: I50ecdf3a82eb7bc677965e8fd3821227fd41bfc2
Signed-off-by: Jerry Wong <jerry.wong@broadcom.com>
Currently there is no socket for daemon instances launched during the
selinux phase of init. We don't create any sockets due to the complexity
of the required sepolicy.
This workaround will allow us to create the socket with very minimal
sepolicy changes. init will launch a one-off instance of snapuserd in
"proxy" mode, and then the following steps will occur:
1. The proxy daemon will be given two sockets, the "normal" socket that
snapuserd clients would connect to, and a "proxy" socket.
2. The proxy daemon will listen on the proxy socket.
3. The first-stage daemon will wake up and connect to the proxy daemon
as a client.
4. The proxy will send the normal socket via SCM_RIGHTS, then exit.
5. The first-stage daemon can now listen and accept on the normal
socket.
Ordering of these events is achieved through a snapuserd.proxy_ready
property.
Some special-casing was needed in init to make this work. The snapuserd
socket owned by snapuserd_proxy is placed into a "persist" mode so it
doesn't get deleted when snapuserd_proxy exits. There's also a special
case method to create a Service object around a previously existing pid.
Finally, first-stage init is technically on a different updateable
partition than snapuserd. Thus, we add a way to query snapuserd to see
if it supports socket handoff. If it does, we communicate this
information through an environment variable to second-stage init.
Bug: 193833730
Test: manual test
Change-Id: I1950b31028980f0138bc03578cd455eb60ea4a58
If the sequence op contains a Cow Operation that doesn't exist, we
should log the error and abort. Otherwise we'll try to merge an op that
doesn't exist.
Test: cow_api_test CowTest.MissingSeqOp
Bug: 177104308
Change-Id: I1a96f09798bc16126ae1338c6259506b18c88f51
Move all the code relevant to snapuserd to a seperate
directory. Add OWNERS file.
No other code changes apart from moving files around
and fixing couple location of header paths
at few places.
Bug: 194642092
Test: Compile, Full OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ib1d852bfeda4eca5c996d6cd7b057f141cb5ddad
If a split fiemap header gets corrupted, we could get ENAMETOOLONG and
removal fails. Handle this gracefully.
Bug: N/A
Test: fiemap_writer_test
Change-Id: I4af9ca7ad4b4fa87a91ff05b8fadac4258006ab6
If a merge fails we write a new snapshot status indicating that the merge
failed. If this happens to occur during the second merge phase, we fail
to propagate the phase counter to the new status. This means the merge
is unlikely to make progress and succeed later.
Bug: 193549218
Test: manual test
Change-Id: I31fdae6bde48e3a71b6f3fcc663541257f7ebd8f
If device abnormal restart during upgrade, update_engine just
created split cow image file and wrote the name to xxx-cow-img.img.
After restart, update_engine will report following err:
06-30 08:48:06.466 root 8918 8920 E gsid : Error removing image
product_b-cow-img: File name too long
That is because SplitFiemap::Create->DetermineMaximumFileSize
open xxx-cow-img.img to get some info, then fsync and unlink this
file.
On device use f2fs, after restart, f2fs_recover_fsync_data will
recover this file with inode block sync to ufs in following path:
DetermineMaximumFileSize
->FiemapWriter::Open
->AllocateFile
This make xxx-cow-img.img file content corruption.
Update_engine call TryCancelUpdate to clean up the residual files
of last upgrade, cannot read valid split file name from
xxx-cow-img.img, then report err log mentioned above.
Add fsync, f2fs can use correct inode to recover related files.
This can reduce the probability of this issue.
Test: abnormal restart when upgrade
Change-Id: Iceb3e5604d90495350c5fb1fb3433732372f3fec
Signed-off-by: lijiazi <lijiazi@xiaomi.com>
Judging from local experiments, it looks like device-mapper doesn't
always generate a uevent after DM_DEV_CREATE ioctl.
Test: presubmit
Bug: 193462349
Change-Id: I8a74375631b20c14a32a41dbaf38380ebc0078e6
A recent change to libsnapshot caused us to filter out duplicate COW
ops. The merge consistency check relied on the old method of manually
counting ops, causing it to come up with a different number. Fix this by
using the already computed "official" count.
Bug: 193532829
Test: new test case in vts_libsnapshot_test
manual test with incremental OTA
Change-Id: I68d1e41f5c140af20a04ba80e3db0780a916ecf8
This switches merge code over from using the old RevOpItr to the new
MergeRevOpItr. Since there are no other users, RevOpItr is no longer
needed.
Changed names of copy_ops_ and total_data_ops_ to more accurately
reflect their meaning.
Bug: 177104308
Test: cow_snapuserd_test
Change-Id: Ic053be4877cfdc86656551f5a3d5d95f3825f937
This adds the -m option to inspect_cow, allowing it to display the ops
according to CowRevMergeOpItr
Bug: 177104308
Test: inspect_cow -m on file with and without seq ops
Change-Id: Ic9a062b8175c77084e76c437414c23fd09e34cc6
This presents cow ops in the reverse of their merge order, as requested
when we perform a merge. It starts with sorted non-ordered ops sorted by
blocks, and then the ordered ops. Already merged ops are removed.
Bug: 177104308
Test: cow_api_test
Change-Id: I864bcab5cda447cf4a197abc02d954f9db08b744
Ordered Ops are Cow Ops with source block dependencies.
Currently this is only Copy Ops, but will extend to XOR
blocks when they're added.
Bug: 177104308
Test: Builds, does not impace cow_api_test/cow_snapuserd_test
Change-Id: I0a50dfe3316ec3a3b1d035335f094c62296649a5
The libsnapshot iterators can all use the same interface instead of
duplicating the interface. We don't have any need for the internal class
variable iterators, so remove them.
Test: Builds, does not impact cow_api_test/cow_snapuserd_test
Change-Id: I5f008401e067a55a57812b7bf101a472ad97df18
