Commit graph

1474 commits

Author SHA1 Message Date
Yifan Hong
26a2e6eb7b libsnapshot: cancel ok if unverified and old slot
Allow canceling an update if the update has been applied
but not rebooted into it.

Test: libsnapshot_test
Change-Id: I694d74e200908ec622855074ab811e3029328f43
2019-09-09 14:41:57 -07:00
Yifan Hong
233c32a454 Merge changes from topic "libsnapshot_prep"
* changes:
  libsnapshot: MapPartitionWithSnapshot cleanup itself if failed
  libsnapshot: MapPartitionWithSnapshot: fix timeout
  libsnapshot: require ex lock for {Create}{CowImage,Snapshot} / DeleteSnapshot
  libsnapshot: Refactor: add {Create,Map,Unmap}CowImage
2019-09-06 19:28:09 +00:00
Treehugger Robot
0b18c6735f Merge "fs_mgr: overlayfs: README merge in public document" 2019-09-06 03:28:42 +00:00
Treehugger Robot
95773aa4b6 Merge "Moving skip_mount.cfg to system_ext partition" 2019-09-06 03:17:22 +00:00
Yifan Hong
f35687df6e libsnapshot: MapPartitionWithSnapshot cleanup itself if failed
When we run MapPartitionWithSnapshot, intermediate devices aren't
cleaned up if the call fails. Hence, record these intermediate devices
we have created along the way using the new AutoDevices class. Upon
failure, the AutoDevices object will be destroyed, and all the
intermediate devices will be deleted from device mapper or image
manager. Upon success, AutoDevices::Release() makes sure the
intermediate devices aren't deleted.

Test: libsnapshot_test

Change-Id: Iff4c1297528288a27765c0224b67254b68c89776
2019-09-05 15:14:34 -07:00
Yifan Hong
5576f7cc13 libsnapshot: MapPartitionWithSnapshot: fix timeout
The MapPartitionWithSnapshot call doesn't respect params.timeout.
Fix it.

Test: libsnapshot_test
Change-Id: I2c5a2889e4687449319bb2018e39405682b458a6
2019-09-05 15:14:34 -07:00
Yifan Hong
caaf9a617a libsnapshot: require ex lock for {Create}{CowImage,Snapshot} / DeleteSnapshot
Create / Deleting the COW image / snapshot changes states, so it makes
sense to require an exclusive lock before doing so. If caller doesn't hold
an exclusive lock, parallel calls to MapCowImage / MapSnapshot / UnmapCowImage /
UnmapSnapshot may have weird results.

Test: libsnapshot_test

Change-Id: I4be660df1059ec24144f8baf43a1c8c05d9e372b
2019-09-05 15:14:34 -07:00
Yifan Hong
27fb95dedf libsnapshot: Refactor: add {Create,Map,Unmap}CowImage
Move operations on image manager in *Snapshot functions to their own
functions for finer granularity in control. *Snapshot functions only
changes snapshot state and snapshot devices, but not the supporting
devices.

Now, MapSnapshot can take a customized COW device path. We will have
a more complicated stack for the COW device in upcomming CLs.

Also, Change SnapshotManager::CreateSnapshot's signature to
accept a SnapshotStatus struct that includes all sizes, so that
cow_partition_size and cow_file_size can also be written to the snapshot
status file.

Test: libsnapshot_test

Change-Id: I388ecd4bcfbfcc3f379ecb6993615234f4fbcb4e
2019-09-05 15:14:34 -07:00
Mark Salyzyn
6ef0081fc6 fs_mgr: overlayfs: README merge in public document
A public facing document rewrite merged into the AOSP-facing README.

Test: none
Bug: 118781664
Change-Id: I6b9d3c4e274edc700dd587f2adc51147382d6b3f
2019-09-05 14:07:57 -07:00
Yifan Hong
dccd6b3389 Merge "libsnapshot: Refactor CreateLogicalAndSnapshotPartitions" 2019-09-04 21:55:39 +00:00
Yifan Hong
d6c426c0f5 Merge "liblp: Indicate usable regions in ResizePartition" 2019-09-04 21:53:57 +00:00
Treehugger Robot
2dc25441e2 Merge "CreateSnapshot: reduce zero-filling for CoW images" 2019-09-04 21:19:44 +00:00
Yifan Hong
48eb2ee176 Merge "libsnapshot: Don't accidentally delete snapshot when collapsing" 2019-09-04 18:53:23 +00:00
Bowgo Tsai
79f875f240 Moving skip_mount.cfg to system_ext partition
We're now adding a core CSI system.img that is common across different
targets. So GSI-specific files should be moved to system_ext partition.

Note that GSI is a system.img that includes the contents of product and
system_ext partitions, under /system/product and /system/system_ext,
respectively. That's why it's under /system/system_ext instead of
/system or /system_ext.

Bug: 137711197
Test: boot GSI on crosshatch
Change-Id: I1ee54c1a434f2d7cf34b4ea2cbda9c732ec932a1
2019-09-04 22:28:47 +08:00
Yifan Hong
a93ac6b9c8 libsnapshot: Allow multiple calls to FinishedSnapshotWrites
If update_engine crashes at certain point, it may
call FinishedSnapshotWrites multiple times. Allow
this to happen.

Test: apply OTA
Change-Id: Ib36cfe38328fd5d6ca408c97024c81569a4bd440
2019-09-03 21:07:30 -07:00
Yifan Hong
d65020d05e liblp: Indicate usable regions in ResizePartition
- Open up GetFreeRegions() API.
- Add a new argument, free_region_hint, to ResizePartitions(). It
  indicates the regions that extents can be allocated to.
- Expose Interval::Intersect functions. libsnapshot needs the algorithm
  to intersect extents to find out the free regions for COW partitions.

Test: liblp_test

Change-Id: I0c079c0e919aa7b0627eb76a071a7cc2a281d692
2019-09-03 20:08:28 -07:00
Yifan Hong
31fda7e476 libsnapshot: Refactor CreateLogicalAndSnapshotPartitions
Create SnapshotManager::MapPartitionWithSnapshot that deals
with one partition so that the code can be re-used later.

Test: libsnapshot_test

Change-Id: Iba36e6c05f8541397323361f7ceb62a6472d2e1c
2019-09-03 19:45:22 -07:00
Yifan Hong
27139799ff libsnapshot: Don't accidentally delete snapshot when collapsing
Test: libsnapshot_test
Change-Id: Ie84546f90f7f186bda88f19f7d29ad437ce97aa8
2019-09-03 16:07:31 -07:00
Yifan Hong
a9ffef27e1 Merge "libsnapshot: Record COW size(s) in snapshot status file" 2019-09-03 16:43:49 +00:00
Alessio Balsini
396a4f279e CreateSnapshot: reduce zero-filling for CoW images
Instead of zero-filling the whole image, only clean the dm-snapshot
metadata header of the CoW file.

Zero-filling a large image may take a long time, and this is a safe, but
not necessary operation if the operation is intended to prepare an
initial CoW file that will be used with a dm-snapshot device.
According to the Linux kernel code, only the first 32 bits of the CoW
file is used to detect if the dm-snapshot device under creation will be
at an initial state or the continuation of a previous dm-snapshot.
CREATE_IMAGE_RESET_SNAP is a flag that reduces the zeroing of the image
file to the first file chunk.

Change-Id: Ibc9fb3b6d19666a92125c988687ff8dacfad47be
Depends-On: I242e57a9a622fbb738bf82f8a260af1b13810069
Bug: 139378014
Test: manual, snapshot_test
Signed-off-by: Alessio Balsini <balsini@google.com>
2019-09-02 11:24:33 +00:00
Treehugger Robot
cc746b5193 Merge "libsnapshot: GetSnapshotExtraDeviceName" 2019-08-29 23:48:18 +00:00
Yifan Hong
65db930d5a libsnapshot: Record COW size(s) in snapshot status file
The COW device consist of two parts: empty space in
super and the loop device. Record it in the snapshot
status file so that MapSnapshot can determine which
devices needs to be mapped.

Test: builds
Change-Id: I08fb051c080802842d2f5b43c2adeed4f04b7c5f
2019-08-29 14:53:50 -07:00
Yifan Hong
9fca50f356 libsnapshot: GetSnapshotExtraDeviceName
Return the name with -inner suffix.

Test: builds
Test: libsnashot_test
Change-Id: Idd66d0c3f1a625af3d73b7a8bdff32b69e501737
2019-08-29 14:49:11 -07:00
Yifan Hong
c205f585e7 liblp: convert more functions to use string_view (#3)
Test: builds

Change-Id: Ia448993ff61d46ae2280e3d3c02b5e84dd51f37c
2019-08-29 14:48:40 -07:00
Mark Salyzyn
6f07a8af11 fs_mgr: overlayfs: test: noatime on rw mounts only
Regression from commit 808763e825
("fs_mgr: overlayfs: noatime").  Test fails on some targets
that do not follow our advise to mount noatime for all partitions.
blueline mounts the system partitions relatime.  In the grand
scheme this is not a problem because we never remount these
partitions read-write because of the overlayfs integration.

Filter out read-only mounts from those enumerated for noatime,
those failures do not concern us in the context of this test.

This test expected to fail if CtsOsTestCases EnvironmentTest fails.

Test: adb-remount-test.sh
Bug: 140253722
Bug: 140014856
Change-Id: I778e7f539ac3f3d6495867e264767f2af25e11c7
2019-08-29 14:12:39 -07:00
Treehugger Robot
a133690fc4 Merge "fs_mgr: overlayfs: noatime" 2019-08-29 16:49:04 +00:00
Treehugger Robot
50ca448402 Merge "fs_mgr: Allow to set defaults for CreateLogicalPartitionParams" 2019-08-29 02:56:40 +00:00
Yifan Hong
1ada55e42a fs_mgr: Allow to set defaults for CreateLogicalPartitionParams
Test: boots
Change-Id: I6a969a19b9d29e682f50872bd3e9027eaca41512
2019-08-28 13:40:15 -07:00
Yifan Hong
2b3a9b2aa8 liblp: convert more functions to use string_view
Test: builds
Change-Id: I464e36b7b909adb3642b82bd98e2c05e11c9287a
2019-08-28 20:17:54 +00:00
Mark Salyzyn
808763e825 fs_mgr: overlayfs: noatime
Should not have used relatime for overlayfs and backing storage
mounts, noatime reduces the wear and tear on the flash device.

This should prevent triggering a falure in the CtsOsTestCases that
tries to spot mounts attached to block storage from updating atime
and resulting in churn.  This statement does not mean we support
performing Cts tests with devices that have deployed
'adb disable-verity' or 'adb remount'.

Post-mortem: relatime was not a 'problem' for overlayfs, copy and
             paste for those flags resulted in the use of relatime
	     for scratch backing storage.

Test: adb-remount-test.sh
Bug: 140014856
Change-Id: Ic97b98ceacaa265094364487ea70cc99b6aee92f
2019-08-28 13:13:13 -07:00
David Anderson
956c204f1e Merge changes I3872dc51,I3b185f68,I37a25ca7
* changes:
  libsnapshot: Improve how devices are collapsed after merging.
  fastbootd: Cancel snapshots when modifying partitions.
  Clean up update state when snapshots are interrupted or cancelled.
2019-08-28 17:51:20 +00:00
David Anderson
f5960a1ac7 Merge "libsnapshot: Do not map snapshots for partitions that were reflashed." 2019-08-28 02:51:43 +00:00
David Anderson
74a6f4cf57 Merge "liblp: Add an attribute to indicate that a partition has been updated." 2019-08-28 02:51:43 +00:00
David Anderson
1145c01e8e libsnapshot: Improve how devices are collapsed after merging.
Currently, we replace snapshot-merge with a linear device wrapping the
base device. This is not efficient. This patch reads LpMetadata for the
underlying partition, and duplicates its table into the snapshot-merge
device. This removes a layer of stacking and also allows removing the
base device.

Note that snapshot_test is growing a bit unwiedly, because it's starting
to implement pieces of libsnapshot that will be filled in later for
update_engine. (MapUpdatePartitions is a good example of this.) When
those pieces land in libsnapshot, snapshot_test will be cleaned up to
remove much of this manual fiddling.

Bug: 139090440
Test: libsnapshot_test gtest
Change-Id: I3872dc51d9e5980803303806f42a5c7e74b0b78a
2019-08-27 15:09:56 -07:00
David Anderson
ad970fc055 fastbootd: Cancel snapshots when modifying partitions.
When flashing or resizing partitions, remove the
LP_PARTITION_ATTR_UPDATED flag. This will cause first-stage init to skip
any snapshots for that partition, and the backing storage (if any)
will later be reclaimed.

Bug: 139155473
Test: manual test
Change-Id: I3b185f68dfecb5a93636af0b5ae289ead1363fd0
2019-08-27 14:03:44 -07:00
David Anderson
c058df7b12 libsnapshot: Do not map snapshots for partitions that were reflashed.
Bug: 139154795
Test: libsnapshot_test gtests
Change-Id: Ic4bb663d4286040adec4c0141373c66a487c91ff
2019-08-27 13:43:18 -07:00
David Anderson
a6e288a178 Clean up update state when snapshots are interrupted or cancelled.
This patch addresses two edge cases with interrupted updates:
 (1) If the device reverts to the old slot *before* merging, snapshots
     must be removed.
 (2) If during a merge, a snapshot is detected to be invalid (due to
     flashing), the snapshot must be removed.

To encapsulate this logic, WaitForMerge has been renamed to
ProcessUpdateState. It is still intended to be called after /data is
mounted, or after a merge is initiated.

Bug: 139154795
Test: libsnapshot_test gtest
Change-Id: I37a25ca722f30ae9548894dcfbd70cb64330e416
2019-08-27 13:43:18 -07:00
Treehugger Robot
f8d80b7d56 Merge "libsnapshot: use DeleteDeviceIfExists() when required" 2019-08-27 19:59:13 +00:00
Paul Lawrence
8c8ce02e2e Merge "Fix ext4/metadata/udc issue" 2019-08-27 15:21:58 +00:00
Yifan Hong
ee9b49d729 Merge "libsnapshot: Add helper for first-stage init mounting" 2019-08-27 01:56:41 +00:00
David Anderson
e0b2b759fa Merge "liblp: Expose MockPropertyFetcher for liblp-dependent tests." 2019-08-26 19:13:49 +00:00
David Anderson
cd7a635dc7 liblp: Add an attribute to indicate that a partition has been updated.
When the bootloader (or fastbootd) flashes the super partition, we need
to make sure that init doesn't re-map any snapshot or snapshot-merge targets.
A simple way to do this is to introduce an attribute that is only added
by update_engine. When this flag is present, we know the partition has
not been flashed.

This bumps the minor version of LpMetadata. To make this as uninvasive
as possible, the new minor version is only used when MetadataBuilder
detects the new attribute. The new liblp can read older metadata, but will
reject it if it contains an illegal attribute set.

Bug: 139154795
Test: liblp_test gtest
Change-Id: I5ae15d11219b41575a9f71d7dbdb43cbf07a3529
2019-08-23 14:44:52 -07:00
David Anderson
9966699af7 liblp: Expose MockPropertyFetcher for liblp-dependent tests.
Bug: 139154795
Test: liblp_test_static builds
Change-Id: I22b15523a32272ed67a8fe16e386f41b3fcf0bb0
2019-08-23 14:44:51 -07:00
David Anderson
9267c3f5c4 libsnapshot: Add helper for first-stage init mounting
With this patch, init can mount snapshots in the first stage by
performing the following operations:
 1. First, check if SnapshotManager::kBootIndicatorPath exists.
 2. If so, call SnapshotManager::NewForFirstStageMount.
 3. If NeedSnapshotsInFirstStageMount returns true,
 4. Call CreateLogicalAndSnapshotPartitions().

When called, this replaces any calls to CreateLogicalPartitions().

Rather than split this into multiple functions (to generate uevents as
needed), we instead use major:minor strings for device-mapper tables.
This means we don't have to wait for paths to resolve.

Bug: 139204329
Test: libsnapshot_test gtest
Change-Id: Ia7ec196a62e51748d6f01a66fe4e9eef25f2898f
2019-08-23 14:42:05 -07:00
Yifan Hong
3a136e58e3 rename liblp_test_static to vts_kernel_liblp_test.
Test: builds
Change-Id: I708ecef257b806c29a7f1e6108d91f52b2ed4241
2019-08-23 13:58:47 -07:00
Yifan Hong
fd659bdfed Add liblp_test to presubmit.
Create a new test binary, liblp_test, that can run on presubmit.

Bug: 138328907
Test: TH
Test: atest liblp_test

Change-Id: I968ff84078524e5733da9b009d9a7222aad299bd
2019-08-23 13:58:47 -07:00
Treehugger Robot
f2c7dd666e Merge "liblp_test_static: fix test" 2019-08-23 02:51:58 +00:00
Yifan Hong
607e4dfa1c liblp_test_static: fix test
- Fix test so that it skips compliance test on non-DAP devices
  and retrofit devices as well. Move these device-dependent tests
  into the DeviceTest test suite and skip them on non-DAP devices.
- Also add "ResetPropertyFetcher" in some tests in io_test.cpp.
- Also change some ASSERT to EXPECT.

Note that VtsKernelLiblpTest runs this test. VtsKernelDynamicPartitionsTest
enforces ro.boot.dynamic_partitions to be set.

Bug: 138328907
Test: run it

Change-Id: Ic5cd3921033142f38aae071585f6c0fa563ae362
2019-08-22 14:59:10 -07:00
David Anderson
9612e2767e Merge "libsnapshot: Use a real fake super partition for tests." 2019-08-22 20:39:59 +00:00
David Anderson
7e5bae88b8 libsnapshot: Use a real fake super partition for tests.
We are currently creating test partitions via ImageManager, which
unfortunately leads to a lot of hacks in my local tree to test
first-stage init. ImageManager devices look nothing like real partitions
in super, and it's a lot of work to massage it to act the same.

Instead, let's create an actual super partition. We can do this via
ImageManager, and give it its very own partition table. To make this
work, we need a special IPartitionOpener that will redirect requests for
"super" to our temporary device. A new .cpp file has been added to house
this. A few other tidbits have moved there as well.

This makes setup code a bit more complicated, but now our tests will
behave much closer to a real device.

Bug: 139204329
Test: libsnapshot_test gtest
Change-Id: I2f40109cd3c1d6343d6f52979789e25f9afd533a
2019-08-22 13:28:29 -07:00