Commit graph

1686 commits

Author SHA1 Message Date
Howard Chen
62f35129d4 Merge "Refine the first stage mount for DSU" 2020-01-16 05:33:09 +00:00
Tom Cherry
f633b62b63 Merge "Require liblog for libbase in all configurations" 2020-01-15 23:20:07 +00:00
Treehugger Robot
5c63dc9322 Merge "Refactor update status management as protobuf" 2020-01-15 14:04:25 +00:00
Nikita Ioffe
07892ea610 Merge "Reduce log spam when unmounting /data" 2020-01-14 10:05:33 +00:00
Yifan Hong
b5c0ad7b21 Merge "Merge VtsFiemapWriterTest into fiemap_writer_test" 2020-01-14 00:11:14 +00:00
Yifan Hong
d90c32fbe2 Merge VtsFiemapWriterTest into fiemap_writer_test
Delete AndroidTest.xml for VtsFiemapWriterTest and use auto-generated
test config for fiemap_writer_test.

Also add fiemap_writer_test to vts-core and device-tests.

Also require_root for fiemap_writer_test.

Test: atest fiemap_writer_test
Fixes: 144300653
Change-Id: Ifc3808ea56331809b58f0c02de2caac250d78352
2020-01-13 13:53:17 -08:00
Nikita Ioffe
b03e0cffd4 Reduce log spam when unmounting /data
We are only interested in unmount that failed.

Test: /system/bin/vdc checkpoint startCheckpoint 1
Test: adb reboot userspace
Bug: 135984674
Bug: 147584366
Bug: 143970043
Change-Id: I9e0b174785fbaf76a95b13457a80765bfa856518
2020-01-13 16:05:07 +00:00
Alessio Balsini
824b77d24b Refactor update status management as protobuf
Convert UpdateState and introduce SnapshotUpdateStatus to protobuf to
simplify its access.
In addition, SnapshotUpdateStatus also stores the sum of all the
snapshot sector information. This additional data is used to improve the
merge progress progress.

Bug: 139088917
Test: manual OTA
Change-Id: Ic777d50244c1afa1cdd75fe9b2ffc6dd9ba19ade
Signed-off-by: Alessio Balsini <balsini@google.com>
2020-01-12 18:22:44 +00:00
Nikita Ioffe
cb0c92e66d Add more logging to fs_mgr_remount_userdata_into_checkpoint
Sometimes unmounting userdata fails, this should help in debugging.
Ideally this information should be persisted somewhere so that we can
add it to the bugreport, but so far I don't have solution better than
writing it to a file in /metadata partition.

Test: adb reboot userspace
Bug: 135984674
Bug: 143970043
Change-Id: I92489600bf31f7f47b8b87de7cd4a882be21910e
2020-01-10 17:41:54 +00:00
Yifan Hong
84ef7f65e8 Merge "Add balsini@ as libsnapshot owner" 2020-01-10 10:28:42 +00:00
Tom Cherry
992163021d Require liblog for libbase in all configurations
liblog will soon be required for all of libbase's logging.  This
change proactively requires liblog in all configurations instead of
just Android.

Bug: 119867234
Test: build
Change-Id: I696162fbebc78d4ef23c6032412101ac51d397a4
2020-01-09 10:43:26 -08:00
Alessio Balsini
b0b9c714d3 Add balsini@ as libsnapshot owner
Add balsini@ as libsnapshot owner to cover EMEA timezone.

Bug: n/a
Test: n/a
Change-Id: I2d5c900c62f202db2432df87f7847b0133872443
Signed-off-by: Alessio Balsini <balsini@google.com>
2020-01-09 10:11:45 +00:00
Howard Chen
a841bd48f8 Refine the first stage mount for DSU
* Open the ImageManager according to the saved DSU slot.
* Load the saved install_dir when it presents. This is required
  when installing to an external storage and the current code
  seems to break it.
* Remove an unnecessary C++ using.

Test: gsi_tool install
Bug: 144247097

Change-Id: I5b7be25787f235518a25abacefa612c33861f6ef
2020-01-09 07:47:35 +00:00
Yifan Hong
cd232f9734 SnapshotManager expose no space error
When there is not enough space on /userdata, CreateUpdateSnapshot
returns SnapshotManager::Return with ErrorCode::NO_SPACE.

Test: libsnapshot_test
Bug: 138808058

Change-Id: If2effe63f6a4324eff8d05d4db4ce98be8190262
2020-01-07 14:20:44 -08:00
Yifan Hong
e4b44fc501 ImageManager returns FiemapStatus.
IImageManager::CreateBackingImage and ZeroFillNewImage now returns
FiemapStatus, which is an error code and provides more information
about the reason of error.

In particular, the error code is NO_SPACE if disk space is full during
allocation / writing.

Bug: 138808058
Test: libsnapshot_test

Change-Id: I500a3d9f55a2d7e60438b5b4ae70f8b5fed162fa
2020-01-07 13:01:58 -08:00
Yifan Hong
6170c7a02f libfiemap: Create/Open returns FiemapStatus
Add variants to SplitFiemap::Create and FiemapWriter::Open that
returns a FiemapStatus and take a unique_ptr<T>* out parameter to
get more information on the reason of the failure.

Test: fiemap_writer_test
Test: fiemap_image_test
Bug: 138808058
Change-Id: Iea77ac9810f98c2202e038b7aede15dd2d137b2b
2020-01-07 13:01:58 -08:00
Treehugger Robot
7cd37ba4dc Merge "Helper function to mount snapshot devices in recovery" 2020-01-07 20:26:43 +00:00
Bookatz
685e19dc97 CtsFsMgrTestCases secondary_user
Marks this CTS test module as supporting secondary_user, meaning that
the tests are eligible to be run from a regular Android user other
than the system user (i.e. other than user 0).

Note that 'user' here refers to the concept of users in Android
Multiuser; it is NOT the same as a uid.

Bug: 141773058
Test: module passed with secondary_user params
 (--enable-optional-parameterization --module-parameter secondary_user)

Change-Id: I1829a4297a49cd38b4b8a2558c92941acab8ab9a
2020-01-06 09:49:17 -08:00
David Anderson
81fbcbdbd4 Merge "overlayfs: Refactor how the scratch device is deduced, again." 2019-12-31 00:24:35 +00:00
Yifan Hong
0ede9b43e5 libfiemap: delete unuseful check
Just let fallocate fails. It also doesn't check for the delta between
the old file and the new file.

Test: unit tests
Change-Id: I05e12b097a973d9fe7fe696cc472bd7ec2d180c7
2019-12-27 15:36:41 -08:00
David Anderson
4e5b505751 Merge "libfiemap: Add a MapAllImages() helper." 2019-12-27 21:13:51 +00:00
David Anderson
31a35038a0 Merge "libfiemap: Add a way to get the block device path of a mapped image." 2019-12-27 21:13:33 +00:00
David Anderson
8edafa2393 Merge "libfiemap: Add helpers to remove images from recovery." 2019-12-27 20:28:48 +00:00
David Anderson
69def12f08 overlayfs: Refactor how the scratch device is deduced, again.
The previous refactoring did not uncover the full breadth of issues that
arise when trying to use /data for adb remount. In fact, there are a few
distinct use cases for the scratch device, and one function cannot
sensibly provide them all.

(1) First-stage init needs to know if there are dependent devices. This
    would be userdata, super_<other>, or system_<other>. This knowledge
    is dependent on the state in /metadata, fstab, and the kernel
    command-line.

(2) First-stage init and fastbootd need to know where the scratch
    partition is. If it's not in super_<other> or system_<other>, and
    there is no indicator on /metadata, then it might be in a dynamic
    partition.

(3) "adb remount" needs to find a place to put scratch, which
    effectively amounts to the first writable space it can find.
    However, for Virtual A/B, devices, scratch wants to be stored in
    /data, which requires more complex checks and binder calls.

Trying to encapsulate all of this into one function is too difficult, so
instead, this patch breaks GetScratchStrategy into separate functions:
one to return a physical location if a candidate exists, and another to
deduce the "boot" scratch device.

"adb remount" no longer calls GetScratchDevice, since it only needs to
know whether or not a physical candidate was possible.

fs_mgr_overlayfs_teardown calls GetBootScratchDevice, but now only
attempts to make a dynamic "scratch" partition if one definitely exists.
This makes the functionality clearer and reduces fastbootd uart spam.

Bug: 134949511
Test: adb_remount_test.sh on coral
      adb_remount_test.sh on crosshatch
      adb_remount_test.sh on walleye
Change-Id: I5f6a3677bc6adcaaf54c8ab3594643e4f285c04e
2019-12-25 22:32:31 -08:00
David Anderson
d5745cce70 libfiemap: Add a MapAllImages() helper.
This adds a helper for first-stage init to easily map partitions backed
by /data. This can be used for the scratch partition as well as DSU
partitions.

Bug: 134949511
Test: fiemap_image_test
Change-Id: I46246b41ce19442d1476b9959e34df0e1bff58c3
2019-12-25 22:32:31 -08:00
David Anderson
e1c879c816 Merge "fs_mgr: Move libfiemap into libfs_mgr, and introduce libfs_mgr_binder." 2019-12-24 04:37:05 +00:00
Treehugger Robot
5821125694 Merge "fs_mgr: Fix return in error case" 2019-12-21 06:59:02 +00:00
Greg Kaiser
d8fdf60367 fs_mgr: Fix return in error case
We were doing "return -1" in an error case from a function that
returns bool.  Since it's non-zero, this gets implicitly cast to
'true', which almost certainly isn't what we wanted here.

Test: TreeHugger
Change-Id: I76c188c22d907fc425cf4d5b0b1575f640a78ee3
2019-12-20 19:39:16 -08:00
Treehugger Robot
1451e0f14c Merge "libsnapshot: no underscores in test names" 2019-12-21 00:54:25 +00:00
David Anderson
734047a231 libfiemap: Add a way to get the block device path of a mapped image.
This is needed for adb remount, to find the scratch device if it is
already mapped. Note that on devices without metadata encryption, this
cannot be done by querying device-mapper, since scratch will be a loop
device.

Bug: 134949511
Test: fiemap_image_test
Change-Id: Ia25d279c6f8a4838be32a8c01aefc67b5ec1e002
2019-12-20 16:41:35 -08:00
David Anderson
f41c7bbb96 libfiemap: Add helpers to remove images from recovery.
ImageManager can map images in recovery, but not delete them, because
/data is not mounted. libsnapshot handles this by storing extra state
files, but this is complex to manage and inconvenient for
fs_mgr_overlayfs.

Instead, this patch introduces two new calls:

 - DisableImage(), which indicates the image should not be used. This is
   implemented by adding a new DISABLED attribute to
   LpPartitionMetadata. CreateLogicalPartitions ignores this flag, and
   thus recovery/fastbootd can disable the scratch partition and
   communicate that it can be deleted. This cannot be called from binder
   since it is intended for recovery/first-stage init only.

 - RemoveDisabledImages(), which walks the images for a given folder on
   /metadata and deletes any that are disabled. This can be called from
   binder.

Note that there is no metadata version bump for this flag. It's
considered to be included in the flag list for minor version 1, and
currently is not used for the actual super partition.

Bug: 134949511
Test: adb remount, fastboot flash system
Test: fiemap_image_test
Change-Id: Iaeca2d1eddb5637dd9a20202cafd11ae60b4d0e3
2019-12-20 16:41:35 -08:00
David Anderson
43482de3f9 fs_mgr: Move libfiemap into libfs_mgr, and introduce libfs_mgr_binder.
This eliminates libfiemap as a standalone library and instead bundles it
into libfs_mgr. All consumers of libfiemap need libfs_mgr, and having
them tightly coupled reduces linkage complexity.

When to use libfs_mgr: In recovery, first-stage-init, or when libfiemap
or overlayfs is not used.

When to use libfs_mgr_binder: When not in recovery/first-stage init, and
overlayfs or libfiemap is needed. In this case, IImageManager will proxy
over binder to gsid.

Bug: 134949511
Test: builds, fiemap_writer/image_test passes.
Change-Id: I4c1a5698afdbf9b3bf87accec98254c1fb6f4e0e
2019-12-20 16:41:34 -08:00
Nikita Ioffe
3d6a5fcb87 Now for real unmount bind-mounts on top of /data
Approach in
https://android-review.googlesource.com/c/platform/system/core/+/1191457
depended on ordering of entries. Instead fs_mgr now tries to unmount
everything with the same block device as /data until timeout is reached.
This is very similar to what is done in UmountPartitions in reboot.cpp,
but I'm a little bit reluctant to merge them together, since the later
unmount all read-write partitions, while here we are only interested in
/data.

Test: on both coral and blueline:
Test: adb shell /system/bin/vdc checkpoint startCheckpoint 1
Test: adb shell setprop sys.init.userdata_remount.force_umount_f2fs 1
Test: adb reboot userspace
Test: verified that userdata was remounted
Bug: 135984674
Change-Id: I929a058fa2f9639f0a2b73e9ea269cf1e07db889
2019-12-19 21:18:42 +00:00
Yifan Hong
be230af10b libsnapshot: no underscores in test names
This is required for gtest.
Test: run it

Change-Id: I9b0d7ee651c283c00035104d4257d8e8addf4db0
2019-12-19 10:45:54 -08:00
David Anderson
43ad6362da Merge "liblp: Add a VIRTUAL_AB_DEVICE flag for LpMetadataHeader." 2019-12-18 15:42:44 +00:00
David Anderson
71b287bfd7 libfiemap: Fix fallocate() usage.
This fixes two bugs in how we use fallocate(). First, using ZERO_RANGE
is a mechanism to allocate holes, not blocks. Instead we should be
passing no flags.

The reason this code worked is because of our explicit writes to the
file, which is intended to defeat lazy block allocation. This is not
necessary for F2FS with proper file pinning support, so we now skip this
when possible.

Together, these fixes improve the speed of "adb remount" when backed
by libfiemap.

Bug: 134949511
Test: adb remount when backed by /data
Change-Id: I746053d183e0c83d56979d9c3fa6d96b40b5e478
2019-12-17 15:04:08 -08:00
Yifan Hong
18c6248ffe Merge changes from topic "wait_for_merge"
* changes:
  libsnapshot: add WaitForMerge
  libsnapshot: SnapshotUpdateTest::AddOperation
  libsnapshot: add GetCurrentSlot
2019-12-17 23:01:59 +00:00
David Anderson
1526a46e92 Merge "fs_mgr: Move libfiemap back to fs_mgr from system/gsid." 2019-12-17 17:32:09 +00:00
David Anderson
4fe398018b fs_mgr: Move libfiemap back to fs_mgr from system/gsid.
Bug: 134949511
Test: builds
Change-Id: I07943acce5d4671975afef6d8da1bdbeef43c39d
2019-12-16 20:10:26 -08:00
Nikita Ioffe
0dda423dd0 Unmount bind-mounts on top of /data before unmounting /data
Test: adb shell setprop sys.init.userdata_remount.force_umount_f2fs 1
Test: adb shell /system/bin/vdc checkpoint startCheckpoint 1
Test: adb reboot userspace
Bug: 135984674
Bug: 143970043
Change-Id: Icd7f3ad1c42d9b21bf4eacf1dec60355c9883254
2019-12-16 23:22:55 +00:00
David Anderson
4ba68b6451 liblp: Add a VIRTUAL_AB_DEVICE flag for LpMetadataHeader.
Bug: 134949511
Test: liblp_test gtests
Change-Id: I8b376d0980e1372086ed15b980209f3726ef651a
2019-12-16 22:20:40 +00:00
Alessio Balsini
bef67f9fd4 Helper function to mount snapshot devices in recovery
In recovery the user has the option to mount /system, but when a device
is updating with Virtual A/B, the _b partition may be a snapshot.
This patch introduces a helper function that satisfies the dependency of
setting up the snapshot devices required to mount /system.

Bug: 139157327
Test: manual /system partition mount on VAB device during OTA
Change-Id: I7337bdd38d7016d12d3ee42be1c7893b10e9116d
Signed-off-by: Alessio Balsini <balsini@google.com>
2019-12-16 21:24:25 +00:00
David Anderson
18bb214b3f Merge "liblp: Expand the metadata header for future use." 2019-12-16 17:31:57 +00:00
Yifan Hong
1af515b57c libsnapshot: add WaitForMerge
Add an API that does not initiate the merge, but only waits for it to
finish. It is different from ProcessUpdateState API in that it also
blocks when state == UNVERIFIED and booting from the new slot.
(ProcessUpdateState immediately returns in this case).

This is useful for android.os.UpdateEngine.CleanupSuccessfulUpdate().
Bug: 138808328
Test: libsnapshot_test

Change-Id: I7cc59fcaf69616e7ec7ebe6101991b5106845b65
2019-12-13 19:52:31 -08:00
Yifan Hong
c810f1091a libsnapshot: SnapshotUpdateTest::AddOperation
Reusable test code.
Test: libsnapshot_test

Change-Id: I00a1a460d84c01b0dbd24b668293df87962e8b00
2019-12-13 19:52:31 -08:00
Yifan Hong
c64cefc027 libsnapshot: add GetCurrentSlot
Factor out obscure logic that reads boot indicator path.
GetCurrentSlot() returns three states:
- None (read failure)
- Old (before reboot)
- New (after reboot)
Use these "logical" slot values for simpler code.

Test: libsnapshot_test
Change-Id: I1fa2ce4916c5a1652d25682ec1f11e101c858822
2019-12-13 19:52:31 -08:00
Treehugger Robot
babc293140 Merge "libsnapshot: snaity check overflow bit at the end of update" 2019-12-14 03:05:19 +00:00
David Anderson
9f75098c60 liblp: Expand the metadata header for future use.
A few times we have wanted to stash small bits of information in the
super header, but we haven't had any bits to do so. This patch addresses
future needs in two ways:

  1. A "flags" field has been added for miscellanious bits that do not
     need a version bump.
  2. The header struct has been padded to 256 bytes to allow for future
     expansion without complicating the struct-parsing code.

This is the first time we've materially changed the format, so this
patch needs some extra explanation.

In all the places we rely on sizeof(LpMetadataHeader), we now need to
use the |header_size| field instead. To make newer versions of liblp
compatible with older headers, we read the minimum required header size
and fill in the extra bytes as needed. To make the validation and
reading logic more clear, it is now combined into a single function,
ReadMetdataHeader.

MetadataBuilder will still emit 1.0-compatible headers, to avoid
changing the on-disk format of existing devices. The new header will
only be emitted as-needed.

Bug: 134949511
Test: liblp_test gtest
      retrofit DAP device boots
      launch DAP device boots

Change-Id: I6221123768ff0057a73967ecb2ff9b006c17af88
2019-12-14 00:35:58 +00:00
David Anderson
2d79a4365e remount: Remove fs_mgr_overlayfs_scratch_device()
This is no longer needed now that GetScratchDevice exists. The cache can
go away too, since it only existed to avoid libdm spam. The spam is
avoided by checking GetState before calling GetDmDevicePathByName.

Bug: 134949511
Test: adb remount and adb_remount_test.sh
Change-Id: I9d94e2523af9de394a811d0b398fe20415708a6b
2019-12-12 14:13:40 -08:00
David Anderson
43d9f1833f remount: Refactor fs_mgr_overlayfs_teardown
This pulls code for mapping the scratch device into a separate function.
It also avoids implicitly failing by passing an empty device string.
Finally, it calls GetScratchDevice, to remove a caller of the deprecated
method fs_mgr_overlayfs_scratch_device().

Bug: 134949511
Test: adb remount and adb_remount_test.sh
Change-Id: If4a543d3fa26af3f8578ec8b236859c8e4d9bfd8
2019-12-12 14:13:40 -08:00