Commit graph

1376 commits

Author SHA1 Message Date
Vic Yang
f3a7988e30 liblp: Make kDefaultGroup a string_view
Changing the type of kDefaultGroup from std::string to std::string_view
allows us to make it a constexpr object.  Since kDefaultGroup is the
only dirty object in liblp .bss at runtime, this change turns the .bss
page clean and thus saves 4KB per library load.

Bug: 138856262
Test: Boot cuttlefish and check liblp bss is clean for all 5 processes
      that are using it.
Change-Id: I7d7c0992e0ab769f070807f24e1275ffed424b5b
2019-08-05 13:02:50 -07:00
David Anderson
23a87716b5 Merge "Implement basic libsnapshot functionality." 2019-08-03 01:30:49 +00:00
David Anderson
456e50193b Implement basic libsnapshot functionality.
This CL implements some of the libsnapshot internals necessary to work
with update_engine. In particular it implements snapshot and update
state, as well as creating and mapping snapshot devices. It does not
implement anything related to merging, nor does it implement the full
update_engine flow.

Update state is stored in /metadata/ota/state. To synchronize callers of
libsnapshot, we always flock() this file at the top of public functions
in SnapshotManager. Internal functions are only called while the lock is
held, and a "LockedFile" guard object is always passed through to
indicate proof-of-lock.

Low-level functions, such as snapshot management, have been moved to
private methods. Higher-level methods designed for update_engine will
ultimately call into these.

This CL also adds some functional tests for SnapshotManager. Test state
is stored in /metadata/ota/test to avoid conflicts with the rest of the
system.

Bug: 136678799
Test: libsnapshot_test gtest
Change-Id: I78c769ed33b307d5214ee386bb13648e35db6cc6
2019-08-02 13:30:08 -07:00
Alessio Balsini
81d4ea79e2 dmctl: add replace command
The replace command is used to substitute the current table of a
device-mapper with the provided one.
The new table will be available as soon as the command returns
successfully.

Bug: 137759376
Test: manual
Change-Id: Iaf258d9043fab3a8770702f4e1c8c0e583c81519
Signed-off-by: Alessio Balsini <balsini@google.com>
2019-08-02 03:27:00 +01:00
Treehugger Robot
ab80f54949 Merge "libdm: compute percentage of snapshot-merge" 2019-08-01 23:32:16 +00:00
Treehugger Robot
1553cf829a Merge "libdm: improve ParseStatusText() and test it" 2019-08-01 02:42:23 +00:00
Alessio Balsini
3565e31b42 libdm: compute percentage of snapshot-merge
Computes the merge completion percentage from the sector information.

Provided test for the function.

Change-Id: I64d83baa0478f9e6969636ee067174910d9b8e03
Bug: N/A
Test: dm_test
Signed-off-by: Alessio Balsini <balsini@google.com>
2019-08-01 02:22:24 +01:00
Treehugger Robot
1be681cf50 Merge "dmctl: return detailed device information" 2019-07-31 23:37:29 +00:00
Alessio Balsini
4560856e33 libdm: improve ParseStatusText() and test it
Simplify the argument parsing of DmTargetSnapshot::ParseStatusText() and
improve its robustness when dealing with wrong imputs.

Add test for DmTargetSnapshot::ParseStatusText().

Change-Id: I7f078c9ecacb402e71db49e3e7072e37cffbc234
Test: dm_test
Signed-off-by: Alessio Balsini <balsini@google.com>
2019-07-31 21:23:58 +01:00
Treehugger Robot
ae898442d1 Merge "Revert "fs_mgt: Fix remount failure in case of non-overlayfs system"" 2019-07-31 18:44:38 +00:00
Mark Salyzyn
3d85fc6954 Merge changes I96d1acb3,Ief00af72,Idf8f4a0e
* changes:
  adb-remount-test: add /system/priv-app
  adb-remount-test: support devices that do not have verity
  adb-remount-test add --no-wait-screen option
2019-07-31 15:49:39 +00:00
Ryan Savitski
814cbfa10e Revert "fs_mgt: Fix remount failure in case of non-overlayfs system"
This reverts commit 588fe9e8af.

Reason for revert: definite cause for test failure

Test: adb-remount-test.sh
Bug: 138407617
Fixes: 138622072
Change-Id: If2fee8873dd7d4360e7ff5572a0481247beffbca
2019-07-31 15:03:45 +00:00
Alessio Balsini
712140ee82 dmctl: return detailed device information
Return detailed information for devices, i.e.:
- active
- access
- activeTable
- inactiveTable
- bufferFull

Bug: 137759376
Test: manual test
Change-Id: I6aa4ae319adbb5c4ca98823f24010e68e6a0b719
Signed-off-by: Alessio Balsini <balsini@google.com>
2019-07-31 02:54:14 +00:00
Alessio Balsini
1b80a23c57 libdm: fetch and present device status flags
Fetch detailed information for devices through its flags and store in
helper Info class, i.e.:
- active
- access
- activeTable
- inactiveTable
- bufferFull

Change-Id: I3241c5bca00e038d19f99390f40710ca7cff8456
Bug: 137759376
Test: manual test
Signed-off-by: Alessio Balsini <balsini@google.com>
2019-07-31 00:14:30 +01:00
Mark Salyzyn
9daf79f43f adb-remount-test: add /system/priv-app
In 4.19+ kernels not yet patched, /system/priv-app/<file> can
cause EPERM on fstat of the directory, resulting in failure to
access content.

Test: adb-remount-test.sh
Bug: 138649540
Change-Id: I96d1acb3cdb80824c9d2ebce415ea6e1253f9c5b
2019-07-30 09:31:45 -07:00
Mark Salyzyn
f6d8a631fa adb-remount-test: support devices that do not have verity
This allows the test to run to completion on cuttlefish

Test: adb-remount-test.sh
Bug: 138649540
Change-Id: Ief00af72af94bf14c1d69499b12ece8cfce2f552
2019-07-30 09:31:37 -07:00
Mark Salyzyn
91eb2c0a25 adb-remount-test add --no-wait-screen option
If the device screen drivers or frameworks are compromised, permit
remount testing without waiting for full boot complete to occur with
the --no-wait-screen option.  In the same vein to support development,
add --wait-adb and --wait-fastboot options to adjust the timeout.

For TreeHugger testing where other (previous) tests demonstrate
framework or screen regressions, it _may_ be advised to switch to the
--no-wait-screen option so that we do not register a false signal.
It should be noted that some of the past issues with overlayfs and
adb remount could introduce boot up, framework or screen regressions,
so it is advised to not use the option if that interlocking logic can
not be set up.

Test: adb-remount-test.sh
Bug: 138649540
Change-Id: Idf8f4a0eb6d7c9139bd4f2c600d14a70dc71902f
2019-07-30 09:31:29 -07:00
Treehugger Robot
e1d4e39064 Merge "dmctl: suspend and resume devices" 2019-07-30 03:32:01 +00:00
Treehugger Robot
b6c32c9051 Merge "libdm: suspend and resume devices" 2019-07-30 00:09:31 +00:00
David Anderson
687f1c4e8e Merge "liblp: Fix typo in IsRetrofitDynamicPartitionsDevice." 2019-07-29 23:01:18 +00:00
Alessio Balsini
79338406e0 dmctl: suspend and resume devices
Export suspend and resume functionalities of libdm to command line
through dmctl.

Change-Id: I8e1dd7d67d8814631e4174d3ba169e705efc1df6
Bug: 137759376
Test: manual
Signed-off-by: Alessio Balsini <balsini@google.com>
2019-07-29 20:48:55 +01:00
Alessio Balsini
bdc65bf083 libdm: suspend and resume devices
In some cases it would be required to suspend the device to succesfully
complete some operations. An example is the suspension of the origin
device that is necessary to avoid data corruption when merging a
snapshot.

Introduce suspend and resume ioctls in libdm.

Bug: 137759376
Test: libdm_test
Change-Id: Id2ff34e930a8b32e570cb9f49da9cc3f65cb499c
Signed-off-by: Alessio Balsini <balsini@google.com>
2019-07-29 20:35:56 +01:00
David Anderson
f01faf2620 liblp: Fix typo in IsRetrofitDynamicPartitionsDevice.
Bug: 138590940
Test: manual test
Change-Id: I58c639e9308d1866d5423c79c00669d27f125d64
2019-07-29 12:20:13 -07:00
Mark Salyzyn
588fe9e8af fs_mgt: Fix remount failure in case of non-overlayfs system
$ adb remount
W DM_DEV_STATUS failed for scratch: No such device or address
E Can not mount overlayfs for partitions: No such file or directory
/system/bin/remount exited with status 8
remount failed

Bug: 138407617
Test: $ adb remount

Change-Id: I0f359071234b57324ad2324905b5239dbfa44d01
Signed-off-by: Dmitry Shmidt <dimitrysh@google.com>
2019-07-26 13:46:41 -07:00
Yifan Hong
37602ed83e Merge changes Ifc4c4540,I4636de85
* changes:
  MetadataBuilder::IsABDevice: make static / use std::optional
  MetadataBuilder::NewForUpdate: check sysprop instead
2019-07-25 22:43:35 +00:00
Yifan Hong
7bf1e9f459 fs_mgr: liblp test: give ownership of environment to gtest
From gtest doc:
Note that googletest takes ownership of the registered environment objects. Therefore do not delete them by yourself.

Test: run it
Change-Id: I275884ddd63a17097c3ea4d9d6bb2b5291dc8c90
2019-07-25 00:20:25 +00:00
Yifan Hong
f0dd120793 MetadataBuilder::IsABDevice: make static / use std::optional
Align with hat sRetrofitDap does.

Test: liblp_test_static
Change-Id: Ifc4c45404369a4eb18c02fc6a0ede938da5809b8
2019-07-24 17:19:06 -07:00
Yifan Hong
9d0e6cdfef MetadataBuilder::NewForUpdate: check sysprop instead
Check the value of ro.boot.dynamic_partitions_retrofit instead of
checking the name of the super partition being 'super' is a more
reliable way of determining retrofit DAP devices.

Some devices launch with DAP (e.g. cuttlefish) doesn't have "super"
as the super partition name. When Virtual A/B is implemented on
cuttlefish, update_engine calls NewForUpdate for the current super
partition metadata. Hence, this code needs to check the retrofit
sysprop instead.

Also, renamed IsRetrofitDevice to IsRetrofitMetadata to avoid the
confusion.

Test: OTA on retrofit DAP device
Test: OTA on launch DAP device
Test: liblp_test_static
Change-Id: I4636de854734df1bb61779d9a955217e89fdb2fd
2019-07-24 17:19:06 -07:00
Mark Salyzyn
917616d47a Update overlayfs patch series reference
Test: none
Bug: 127298877
Change-Id: I0162248953ea4d2c97ccec52bf552a5e29a1b61e
2019-07-24 14:03:55 -07:00
David Anderson
8a0f82fefa Sketch out libsnapshot for update_engine integration.
This is a skeleton API so we can begin implementing both libsnapshot and
the relevant changes in update_engine.

Bug: 136678799
Test: builds
Change-Id: I5dc0fc1f401e94da2b5996cd69ab4076847282a4
2019-07-17 20:30:51 -07:00
David Anderson
eb599db1c5 Merge "Revert "Revert "Remove libfiemap_writer from fs_mgr and move it to gsid.""" 2019-07-16 19:03:09 +00:00
David Anderson
974dadf904 Merge "libdm: Fix race conditions in LoopControl::Attach." 2019-07-16 19:03:09 +00:00
David Anderson
af73ae823f Revert "Revert "Remove libfiemap_writer from fs_mgr and move it to gsid.""
This reverts commit 31656c2c53.

Reason for revert: re-landing

Change-Id: Ibd36267645f24880a7451b63dd22192dfd650f31
2019-07-15 23:56:25 +00:00
David Anderson
31656c2c53 Revert "Remove libfiemap_writer from fs_mgr and move it to gsid."
This reverts commit 20fbf0d672.

Reason for revert: broke build

Change-Id: I6914455ba1f0fd56fbf3572f349242d734281610
2019-07-15 23:52:49 +00:00
David Anderson
20fbf0d672 Remove libfiemap_writer from fs_mgr and move it to gsid.
gsid is the only consumer for this library, and it's unlikely to grow
beyond gsid.

Bug: 134536978
Test: gsid builds
Change-Id: Ib632b1dd8de3e29c02d1fe443a1e409af5f1257d
2019-07-15 22:53:39 +00:00
David Anderson
cd22e3a6f6 Merge "fs_mgr: Remove the timeout parameter to DestroyLogicalPartition." 2019-07-13 18:02:05 +00:00
David Anderson
c9b797ac25 Merge "libdm: Improve the reliability of dm device paths." 2019-07-12 23:03:41 +00:00
Treehugger Robot
bde9fc1787 Merge "fs_mgr: overlay: do not allow sub-mounts to be overridden" 2019-07-12 18:38:39 +00:00
David Anderson
4f9d1b15b4 libdm: Fix race conditions in LoopControl::Attach.
This fixes two race conditions in LoopControl::Attach(). The first is
that after LOOP_CTL_GET_FREE, the path is not be available until it has
been processed by ueventd. This can be fixed by adding a timeout
parameter and a call to WaitForFile().

Second, it is possible (albeit unlikely), given that loop devices are
now being used more aggressively, that two processes race when
attempting LOOP_SET_FD. In this case, one process will win, and the
other will fail with EBUSY. We can handle this case by retrying the
operation while respecting the same timeout parameter.

Bug: 135771280
Test: libdm_test gtest
Change-Id: Icf9facc3ca28fdb6ff5c78612d3dc183fa47b1f3
2019-07-11 15:39:53 -07:00
David Anderson
470fe2b5f0 fs_mgr: Remove the timeout parameter to DestroyLogicalPartition.
This is no longer needed as CreateLogicalPartition() ensures the
obtained path will not race with device deletion.

Bug: 135771280
Test: device builds, flashes
Change-Id: I821290aa08fede99d5c51cd68681c351a1ea97bc
2019-07-11 15:39:53 -07:00
David Anderson
924858cd18 libdm: Improve the reliability of dm device paths.
This fixes a race condition where WaitForFile() after
GetDmDevicePathByName appears to succeed, but a subsequent operation on
the path fails. This can happen when CreateDevice() is called
immediately after a call to DeleteDevice (from any process), and the
path is re-used, enqueuing udev events to remove and re-add the block
device.

The fix for this is to introduce a new variant of CreateDevice() that
has a timeout parameter. When the timeout is positive, CreateDevice()
will wait for a /dev/block/mapper/by-uuid symlink to be created, which
signals that ueventd has finished processing the operation.

ueventd will now create these by-uuid symlinks for device-mapper nodes.
Unfortunately, the uuid is only available during "change" events, so we
have to special case device-mapper symlink creation. And since the uuid
is not available during "remove" events, we simply find matching links
to remove them.

This ensures that callers of CreateDevice() can use the device path
knowing that no asynchronous removals are pending. Code that uses the
old CreateDevice+WaitForFile pattern will be transitioned to the new
method.

Note that it is safe to ignore the timeout, or to use the "unsafe"
CreateDevice, if the caller ensures the path by other means. For example
first-stage init has no device removal, and regenerates uevents until
it has acquired all the paths it needs.

Finally, since libdm now inspects sysfs unconditionally, libdm consumers
need r_dir_file perms for sysfs_dm in their sepolicy. Additionally
linking to libdm now requires linking to libext2_uuid.

Bug: 135771280
Test: libdm_test
      device flashes, boots
Change-Id: If5a7383ea38f32a7fbbcf24842dce6a668050a70
2019-07-11 15:39:52 -07:00
Mark Salyzyn
8cf2103083 fs_mgr: overlay: do not allow sub-mounts to be overridden
Prevent sub-mounts from being candidates for overlayfs overrides.
We move them temporarily to hold the references, then move them back
after overlayfs mount.

There is a race condition that can not be solved where the content
of the sub-mount is temporarily unavailable.  This operation occurs
in first stage init (no problem) and during the first adb remount or
adb disable-verity that causes the backing storage to be setup.
Workaround will be to time the adb remount/disable-verity so that the
device-specific problems do not surface.

Test: adb-remount-test.sh and manual device configs
Bug: 130131892
Bug: 135647457
Change-Id: I8782d09b7b389d899aed07b3e6c528280af2d6c8
2019-07-10 14:32:05 -07:00
Justin Yun
7eaf9b58ec Rename product_services to system_ext
Update ld.config.txt and init.

Bug: 134359158
Test: make && check boot and ld.config.txt file
Change-Id: Ie1dce42e690414fb45b26759506ea7f64d06740d
2019-07-09 23:34:17 +00:00
Tom Cherry
247ffbf314 Fix a few clang-tidy issues and add NOLINT for others
android-base:
* Add NOLINT for expanding namespace std for std::string* ostream
  overload

libdm:
* Fix missing parentesis around macro parameters

init:
* Fix missing CLOEXEC usage and add NOLINT for the intended
  usages.
* Fix missing parentesis around macro parameters
* Fix erase() / remove_if() idiom
* Correctly specific unsigned char when intended
* 'namespace flags' should be signed, since 'flags' it signed for
  clone()
* Add clear to property restore vector<string> to empty after move
* Explicit comparison against 0 for strcmp

Test: build
Change-Id: I8c31dafda2c43ebc5aa50124cbbd6e23ed2c4101
2019-07-09 16:17:36 +00:00
liyong
089941c800 Fix unsigned type sub overflow issue when isntall DSU
writer->size() is block size aligned and could be bigger than remaining_bytes
If remaining_bytes is bigger, set remaining_bytes to 0 to avoid sub overflow error.

Bug: 136727859
Test: Successfully install a DSU
Change-Id: If493b0f206561239caec2ee234f7cfd70bf927a7
2019-07-04 11:29:19 +00:00
Paul Lawrence
323959ea27 Make ext4 userdata checkpoints work with metadata encryption
When both ext4 user data checkpoints and metadata encryption are
enabled, we are creating two stacked dm devices. This had not been
properly thought through or debugged.

Test: Enable metadata encryption on taimen (add
keydirectory=/metadata/vold/metadata_encryption to flags for userdata in
fstab.hardware)
    Unfortunately metadata is not wiped by fastboot -w, so it is
    necessary to rm metadata/vold -rf whenever you wipe data.
    fastboot flashall -w works
    fastboot reboot -w works
    A normal boot works
    Disable checkpoint commits with
    setprop persist.vold.dont_commit_checkpoint 1
    vdc checkpoint startCheckpoint 10
    adb reboot
    wait for device to fully boot then
    adb reboot
    Wait for device to fully boot then
    adb logcat -d | grep Checkpoint shows the rollback in the logs

    This tests encryption on top of checkpoints with commit, encryption
    without checkpoints, and rollback, which seems to be the key cases.

Bug: 135905679
Change-Id: I24387a2943dce28b918c34894f24911b20429be7
2019-06-26 15:09:07 -07:00
David Anderson
87db478ad6 Merge "libfiemap_writer: Improve device unwrapping to support simple linear devices." 2019-06-20 23:59:06 +00:00
David Anderson
1bfcd580c8 libfiemap_writer: Improve device unwrapping to support simple linear devices.
Normally we reject complex dm targets that would invalidate or shift the
block mappings returned via FIEMAP/FIBMAP. Currently the only targets
allowed are crypt, default-key, and bow. This patch adds support for
"linear" as long as there is only one linear target and it targets
sector 0 of the underlying block device.

This is useful for testing gsid, so we can simulate how a
metadata-encrypted device works without having to create a dm-crypt or
dm-default-key node.

Bug: 134536978
Test: manual test
Change-Id: I7c12bc20d95ff4c90402e66bafb4cf2fce7818e2
2019-06-20 13:39:11 -07:00
David Anderson
99f610da3d Merge "liblp: Handle invalid alignment offsets correctly." 2019-06-20 19:09:52 +00:00
David Anderson
0b5c3552cc Merge "Remove fs_mgr_wait_for_file." 2019-06-20 19:09:26 +00:00