Commit graph

1480 commits

Author SHA1 Message Date
Yifan Hong
c42f5d64e4 Merge "libsnapshot: collapse fix sanity checks" 2019-09-10 23:47:15 +00:00
Yifan Hong
24815a49d4 Merge "libsnapshot: MapSnapshot: use device string for inner device" 2019-09-10 23:46:22 +00:00
David Anderson
878930be92 Merge "liblp: Add a helper to differentiate empty from non-empty images." 2019-09-10 21:36:39 +00:00
Yifan Hong
af32f9377e libsnapshot: collapse fix sanity checks
Fix sanity checks of the snapshot device sizes.
Test: libsnapshot_test

Change-Id: If2945eaa84be59e14ea1fbc9028022db3a145698
2019-09-09 19:45:14 -07:00
Yifan Hong
f4cd54e512 libsnapshot: MapSnapshot: use device string for inner device
Previously, the device path for the -inner device is used, which
may not be present. Use the major:minor instead.

Test: libsnapshot (not flaky)
Change-Id: I8b8cbe8af27a49a50876385bb0c0353ecf7ed1de
2019-09-09 19:45:14 -07:00
Yifan Hong
26a2e6eb7b libsnapshot: cancel ok if unverified and old slot
Allow canceling an update if the update has been applied
but not rebooted into it.

Test: libsnapshot_test
Change-Id: I694d74e200908ec622855074ab811e3029328f43
2019-09-09 14:41:57 -07:00
David Anderson
922cfefbb9 liblp: Add a helper to differentiate empty from non-empty images.
Also, finally explain the difference between these somewhere.

Bug: 140204341
Test: lpdump
Change-Id: I128780b01a28e893afac65f3aa4f4a8d36032d63
2019-09-09 13:16:02 -07:00
Yifan Hong
233c32a454 Merge changes from topic "libsnapshot_prep"
* changes:
  libsnapshot: MapPartitionWithSnapshot cleanup itself if failed
  libsnapshot: MapPartitionWithSnapshot: fix timeout
  libsnapshot: require ex lock for {Create}{CowImage,Snapshot} / DeleteSnapshot
  libsnapshot: Refactor: add {Create,Map,Unmap}CowImage
2019-09-06 19:28:09 +00:00
Treehugger Robot
0b18c6735f Merge "fs_mgr: overlayfs: README merge in public document" 2019-09-06 03:28:42 +00:00
Treehugger Robot
95773aa4b6 Merge "Moving skip_mount.cfg to system_ext partition" 2019-09-06 03:17:22 +00:00
Yifan Hong
f35687df6e libsnapshot: MapPartitionWithSnapshot cleanup itself if failed
When we run MapPartitionWithSnapshot, intermediate devices aren't
cleaned up if the call fails. Hence, record these intermediate devices
we have created along the way using the new AutoDevices class. Upon
failure, the AutoDevices object will be destroyed, and all the
intermediate devices will be deleted from device mapper or image
manager. Upon success, AutoDevices::Release() makes sure the
intermediate devices aren't deleted.

Test: libsnapshot_test

Change-Id: Iff4c1297528288a27765c0224b67254b68c89776
2019-09-05 15:14:34 -07:00
Yifan Hong
5576f7cc13 libsnapshot: MapPartitionWithSnapshot: fix timeout
The MapPartitionWithSnapshot call doesn't respect params.timeout.
Fix it.

Test: libsnapshot_test
Change-Id: I2c5a2889e4687449319bb2018e39405682b458a6
2019-09-05 15:14:34 -07:00
Yifan Hong
caaf9a617a libsnapshot: require ex lock for {Create}{CowImage,Snapshot} / DeleteSnapshot
Create / Deleting the COW image / snapshot changes states, so it makes
sense to require an exclusive lock before doing so. If caller doesn't hold
an exclusive lock, parallel calls to MapCowImage / MapSnapshot / UnmapCowImage /
UnmapSnapshot may have weird results.

Test: libsnapshot_test

Change-Id: I4be660df1059ec24144f8baf43a1c8c05d9e372b
2019-09-05 15:14:34 -07:00
Yifan Hong
27fb95dedf libsnapshot: Refactor: add {Create,Map,Unmap}CowImage
Move operations on image manager in *Snapshot functions to their own
functions for finer granularity in control. *Snapshot functions only
changes snapshot state and snapshot devices, but not the supporting
devices.

Now, MapSnapshot can take a customized COW device path. We will have
a more complicated stack for the COW device in upcomming CLs.

Also, Change SnapshotManager::CreateSnapshot's signature to
accept a SnapshotStatus struct that includes all sizes, so that
cow_partition_size and cow_file_size can also be written to the snapshot
status file.

Test: libsnapshot_test

Change-Id: I388ecd4bcfbfcc3f379ecb6993615234f4fbcb4e
2019-09-05 15:14:34 -07:00
Mark Salyzyn
6ef0081fc6 fs_mgr: overlayfs: README merge in public document
A public facing document rewrite merged into the AOSP-facing README.

Test: none
Bug: 118781664
Change-Id: I6b9d3c4e274edc700dd587f2adc51147382d6b3f
2019-09-05 14:07:57 -07:00
Yifan Hong
dccd6b3389 Merge "libsnapshot: Refactor CreateLogicalAndSnapshotPartitions" 2019-09-04 21:55:39 +00:00
Yifan Hong
d6c426c0f5 Merge "liblp: Indicate usable regions in ResizePartition" 2019-09-04 21:53:57 +00:00
Treehugger Robot
2dc25441e2 Merge "CreateSnapshot: reduce zero-filling for CoW images" 2019-09-04 21:19:44 +00:00
Yifan Hong
48eb2ee176 Merge "libsnapshot: Don't accidentally delete snapshot when collapsing" 2019-09-04 18:53:23 +00:00
Bowgo Tsai
79f875f240 Moving skip_mount.cfg to system_ext partition
We're now adding a core CSI system.img that is common across different
targets. So GSI-specific files should be moved to system_ext partition.

Note that GSI is a system.img that includes the contents of product and
system_ext partitions, under /system/product and /system/system_ext,
respectively. That's why it's under /system/system_ext instead of
/system or /system_ext.

Bug: 137711197
Test: boot GSI on crosshatch
Change-Id: I1ee54c1a434f2d7cf34b4ea2cbda9c732ec932a1
2019-09-04 22:28:47 +08:00
Yifan Hong
a93ac6b9c8 libsnapshot: Allow multiple calls to FinishedSnapshotWrites
If update_engine crashes at certain point, it may
call FinishedSnapshotWrites multiple times. Allow
this to happen.

Test: apply OTA
Change-Id: Ib36cfe38328fd5d6ca408c97024c81569a4bd440
2019-09-03 21:07:30 -07:00
Yifan Hong
d65020d05e liblp: Indicate usable regions in ResizePartition
- Open up GetFreeRegions() API.
- Add a new argument, free_region_hint, to ResizePartitions(). It
  indicates the regions that extents can be allocated to.
- Expose Interval::Intersect functions. libsnapshot needs the algorithm
  to intersect extents to find out the free regions for COW partitions.

Test: liblp_test

Change-Id: I0c079c0e919aa7b0627eb76a071a7cc2a281d692
2019-09-03 20:08:28 -07:00
Yifan Hong
31fda7e476 libsnapshot: Refactor CreateLogicalAndSnapshotPartitions
Create SnapshotManager::MapPartitionWithSnapshot that deals
with one partition so that the code can be re-used later.

Test: libsnapshot_test

Change-Id: Iba36e6c05f8541397323361f7ceb62a6472d2e1c
2019-09-03 19:45:22 -07:00
Yifan Hong
27139799ff libsnapshot: Don't accidentally delete snapshot when collapsing
Test: libsnapshot_test
Change-Id: Ie84546f90f7f186bda88f19f7d29ad437ce97aa8
2019-09-03 16:07:31 -07:00
Yifan Hong
a9ffef27e1 Merge "libsnapshot: Record COW size(s) in snapshot status file" 2019-09-03 16:43:49 +00:00
Alessio Balsini
396a4f279e CreateSnapshot: reduce zero-filling for CoW images
Instead of zero-filling the whole image, only clean the dm-snapshot
metadata header of the CoW file.

Zero-filling a large image may take a long time, and this is a safe, but
not necessary operation if the operation is intended to prepare an
initial CoW file that will be used with a dm-snapshot device.
According to the Linux kernel code, only the first 32 bits of the CoW
file is used to detect if the dm-snapshot device under creation will be
at an initial state or the continuation of a previous dm-snapshot.
CREATE_IMAGE_RESET_SNAP is a flag that reduces the zeroing of the image
file to the first file chunk.

Change-Id: Ibc9fb3b6d19666a92125c988687ff8dacfad47be
Depends-On: I242e57a9a622fbb738bf82f8a260af1b13810069
Bug: 139378014
Test: manual, snapshot_test
Signed-off-by: Alessio Balsini <balsini@google.com>
2019-09-02 11:24:33 +00:00
Treehugger Robot
cc746b5193 Merge "libsnapshot: GetSnapshotExtraDeviceName" 2019-08-29 23:48:18 +00:00
Yifan Hong
65db930d5a libsnapshot: Record COW size(s) in snapshot status file
The COW device consist of two parts: empty space in
super and the loop device. Record it in the snapshot
status file so that MapSnapshot can determine which
devices needs to be mapped.

Test: builds
Change-Id: I08fb051c080802842d2f5b43c2adeed4f04b7c5f
2019-08-29 14:53:50 -07:00
Yifan Hong
9fca50f356 libsnapshot: GetSnapshotExtraDeviceName
Return the name with -inner suffix.

Test: builds
Test: libsnashot_test
Change-Id: Idd66d0c3f1a625af3d73b7a8bdff32b69e501737
2019-08-29 14:49:11 -07:00
Yifan Hong
c205f585e7 liblp: convert more functions to use string_view (#3)
Test: builds

Change-Id: Ia448993ff61d46ae2280e3d3c02b5e84dd51f37c
2019-08-29 14:48:40 -07:00
Mark Salyzyn
6f07a8af11 fs_mgr: overlayfs: test: noatime on rw mounts only
Regression from commit 808763e825
("fs_mgr: overlayfs: noatime").  Test fails on some targets
that do not follow our advise to mount noatime for all partitions.
blueline mounts the system partitions relatime.  In the grand
scheme this is not a problem because we never remount these
partitions read-write because of the overlayfs integration.

Filter out read-only mounts from those enumerated for noatime,
those failures do not concern us in the context of this test.

This test expected to fail if CtsOsTestCases EnvironmentTest fails.

Test: adb-remount-test.sh
Bug: 140253722
Bug: 140014856
Change-Id: I778e7f539ac3f3d6495867e264767f2af25e11c7
2019-08-29 14:12:39 -07:00
Treehugger Robot
a133690fc4 Merge "fs_mgr: overlayfs: noatime" 2019-08-29 16:49:04 +00:00
Treehugger Robot
50ca448402 Merge "fs_mgr: Allow to set defaults for CreateLogicalPartitionParams" 2019-08-29 02:56:40 +00:00
Yifan Hong
1ada55e42a fs_mgr: Allow to set defaults for CreateLogicalPartitionParams
Test: boots
Change-Id: I6a969a19b9d29e682f50872bd3e9027eaca41512
2019-08-28 13:40:15 -07:00
Yifan Hong
2b3a9b2aa8 liblp: convert more functions to use string_view
Test: builds
Change-Id: I464e36b7b909adb3642b82bd98e2c05e11c9287a
2019-08-28 20:17:54 +00:00
Mark Salyzyn
808763e825 fs_mgr: overlayfs: noatime
Should not have used relatime for overlayfs and backing storage
mounts, noatime reduces the wear and tear on the flash device.

This should prevent triggering a falure in the CtsOsTestCases that
tries to spot mounts attached to block storage from updating atime
and resulting in churn.  This statement does not mean we support
performing Cts tests with devices that have deployed
'adb disable-verity' or 'adb remount'.

Post-mortem: relatime was not a 'problem' for overlayfs, copy and
             paste for those flags resulted in the use of relatime
	     for scratch backing storage.

Test: adb-remount-test.sh
Bug: 140014856
Change-Id: Ic97b98ceacaa265094364487ea70cc99b6aee92f
2019-08-28 13:13:13 -07:00
David Anderson
956c204f1e Merge changes I3872dc51,I3b185f68,I37a25ca7
* changes:
  libsnapshot: Improve how devices are collapsed after merging.
  fastbootd: Cancel snapshots when modifying partitions.
  Clean up update state when snapshots are interrupted or cancelled.
2019-08-28 17:51:20 +00:00
David Anderson
f5960a1ac7 Merge "libsnapshot: Do not map snapshots for partitions that were reflashed." 2019-08-28 02:51:43 +00:00
David Anderson
74a6f4cf57 Merge "liblp: Add an attribute to indicate that a partition has been updated." 2019-08-28 02:51:43 +00:00
David Anderson
1145c01e8e libsnapshot: Improve how devices are collapsed after merging.
Currently, we replace snapshot-merge with a linear device wrapping the
base device. This is not efficient. This patch reads LpMetadata for the
underlying partition, and duplicates its table into the snapshot-merge
device. This removes a layer of stacking and also allows removing the
base device.

Note that snapshot_test is growing a bit unwiedly, because it's starting
to implement pieces of libsnapshot that will be filled in later for
update_engine. (MapUpdatePartitions is a good example of this.) When
those pieces land in libsnapshot, snapshot_test will be cleaned up to
remove much of this manual fiddling.

Bug: 139090440
Test: libsnapshot_test gtest
Change-Id: I3872dc51d9e5980803303806f42a5c7e74b0b78a
2019-08-27 15:09:56 -07:00
David Anderson
ad970fc055 fastbootd: Cancel snapshots when modifying partitions.
When flashing or resizing partitions, remove the
LP_PARTITION_ATTR_UPDATED flag. This will cause first-stage init to skip
any snapshots for that partition, and the backing storage (if any)
will later be reclaimed.

Bug: 139155473
Test: manual test
Change-Id: I3b185f68dfecb5a93636af0b5ae289ead1363fd0
2019-08-27 14:03:44 -07:00
David Anderson
c058df7b12 libsnapshot: Do not map snapshots for partitions that were reflashed.
Bug: 139154795
Test: libsnapshot_test gtests
Change-Id: Ic4bb663d4286040adec4c0141373c66a487c91ff
2019-08-27 13:43:18 -07:00
David Anderson
a6e288a178 Clean up update state when snapshots are interrupted or cancelled.
This patch addresses two edge cases with interrupted updates:
 (1) If the device reverts to the old slot *before* merging, snapshots
     must be removed.
 (2) If during a merge, a snapshot is detected to be invalid (due to
     flashing), the snapshot must be removed.

To encapsulate this logic, WaitForMerge has been renamed to
ProcessUpdateState. It is still intended to be called after /data is
mounted, or after a merge is initiated.

Bug: 139154795
Test: libsnapshot_test gtest
Change-Id: I37a25ca722f30ae9548894dcfbd70cb64330e416
2019-08-27 13:43:18 -07:00
Treehugger Robot
f8d80b7d56 Merge "libsnapshot: use DeleteDeviceIfExists() when required" 2019-08-27 19:59:13 +00:00
Paul Lawrence
8c8ce02e2e Merge "Fix ext4/metadata/udc issue" 2019-08-27 15:21:58 +00:00
Yifan Hong
ee9b49d729 Merge "libsnapshot: Add helper for first-stage init mounting" 2019-08-27 01:56:41 +00:00
David Anderson
e0b2b759fa Merge "liblp: Expose MockPropertyFetcher for liblp-dependent tests." 2019-08-26 19:13:49 +00:00
David Anderson
cd7a635dc7 liblp: Add an attribute to indicate that a partition has been updated.
When the bootloader (or fastbootd) flashes the super partition, we need
to make sure that init doesn't re-map any snapshot or snapshot-merge targets.
A simple way to do this is to introduce an attribute that is only added
by update_engine. When this flag is present, we know the partition has
not been flashed.

This bumps the minor version of LpMetadata. To make this as uninvasive
as possible, the new minor version is only used when MetadataBuilder
detects the new attribute. The new liblp can read older metadata, but will
reject it if it contains an illegal attribute set.

Bug: 139154795
Test: liblp_test gtest
Change-Id: I5ae15d11219b41575a9f71d7dbdb43cbf07a3529
2019-08-23 14:44:52 -07:00
David Anderson
9966699af7 liblp: Expose MockPropertyFetcher for liblp-dependent tests.
Bug: 139154795
Test: liblp_test_static builds
Change-Id: I22b15523a32272ed67a8fe16e386f41b3fcf0bb0
2019-08-23 14:44:51 -07:00
David Anderson
9267c3f5c4 libsnapshot: Add helper for first-stage init mounting
With this patch, init can mount snapshots in the first stage by
performing the following operations:
 1. First, check if SnapshotManager::kBootIndicatorPath exists.
 2. If so, call SnapshotManager::NewForFirstStageMount.
 3. If NeedSnapshotsInFirstStageMount returns true,
 4. Call CreateLogicalAndSnapshotPartitions().

When called, this replaces any calls to CreateLogicalPartitions().

Rather than split this into multiple functions (to generate uevents as
needed), we instead use major:minor strings for device-mapper tables.
This means we don't have to wait for paths to resolve.

Bug: 139204329
Test: libsnapshot_test gtest
Change-Id: Ia7ec196a62e51748d6f01a66fe4e9eef25f2898f
2019-08-23 14:42:05 -07:00