Commit graph

64900 commits

Author SHA1 Message Date
David Anderson
3e3159c83e libsnapshot: Move first-stage pid var constant into IDeviceInfo.
This will fail in vts_libsnapshot_test because the pid isn't set (and
even if it was, we don't want to kill any running snapuserd instances).
Rather than start an instance just for the sake of killing it, use
dependency injection instead.

Bug: N/A
Test: vts_libsnapshot_test
Change-Id: I1569f927e018319cf4008673a32d5526573d8d57
2020-11-12 20:41:24 -08:00
Treehugger Robot
13e33f7f80 Merge "Create a pool of 20 measurement files per library for coverage." 2020-11-12 19:57:36 +00:00
Devin Moore
eb85221dcf Merge "libutils: add unit tests for invalid utf-8 and utf-16 strings" 2020-11-12 16:05:38 +00:00
David Anderson
9e6cef7f07 Merge changes Ie79ab525,I3befc71f
* changes:
  libsnapshot: Refactor COW reading/writing of footers.
  libsnapshot: Don't try to truncate block devices
2020-11-12 01:45:19 +00:00
Akilesh Kailash
70d27113d0 Merge "libsnapshot: Retrieve base sector for dm-user device creation" 2020-11-12 00:45:21 +00:00
David Anderson
ea79136740 libsnapshot: Refactor COW reading/writing of footers.
This simplifies the footer and label resume system for COW files.
Previously, a footer could be missing, and we'd try to recover the file.
Any ops we could recover were handed off to the CowWriter, which then
searched up to the correct resume label.

In the new model, a reader is opened with an optional label (resume
mode), or without a label.

When resuming to a label, a footer is optional. The read is terminated
by EOF, discovery of the requested label, or the presence of a footer.
However the requested label must be found, and parsing fails if it is
not found.

When opening a COW without a label, the footer must be present, as this
signals the file was successfully flushed. Parsing fails if no footer is
found.

Bug: N/A
Test: cow_api_test
      full OTA with VABC
Change-Id: Ie79ab5259f532dd16c72f0e42da7568c5c5c4623
2020-11-11 15:51:15 -08:00
Daniel Rosenberg
ce69f2985b libsnapshot: Don't try to truncate block devices
When we're writing to a block device, we can't truncate. Ignore those
commands. Truncate mostly just saves time in the read phase by chopping
off unused data in the event finalize wasn't called.

Bug: 172026020
Test: cow_api_test
Change-Id: I3befc71fa9597edf8243d0f9e17440db91409aea
2020-11-11 15:49:18 -08:00
Oliver Nguyen
ca54a7b7a6 Create a pool of 20 measurement files per library for coverage.
Reduces disk space usage for commonly used libraries by merging coverage
data across processes.

Bug: 171338125
Test: Forrest run of coverage tests
Change-Id: I2b9e94871fc5d66971c5b7e725b296bcd721ccf2
2020-11-11 14:55:58 -08:00
Akilesh Kailash
8ec3fa8aba libsnapshot: Retrieve base sector for dm-user device creation
dm-user device requires sector information which is retrieved
from snapuserd daemon once the metadata read is completed.
Split up the initialization of daemon into two APIs. Once
the metadata read is completed, send the sector information
back to libsnapshot to create the dm-user device.

On a sidenote, remove unused code from snapuserd_client

Bug: 168311203
Test: vts_libsnapshot_test, cow_snapuserd_test

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I0647ebbeea3a4705599966974bfec2318a3e1e4e
2020-11-11 22:40:49 +00:00
George Burgess IV
3d0f195bcc Merge "fs_mgr: fix potential uses of nullptr" 2020-11-11 22:02:29 +00:00
Steven Moreland
99ed2152de Merge "Add @SensitiveData to GateKeeperService" 2020-11-11 16:17:19 +00:00
Akilesh Kailash
d29c2f5dbc Merge "libsnapshot:snapuserd: Snapshot merge support." 2020-11-11 01:04:23 +00:00
Yo Chiang
26f47f8815 Merge "Use default DSU slot name when active DSU is empty" 2020-11-10 10:24:00 +00:00
Jaegeuk Kim
0a71c0e38c Merge "fastboot: add casefold/projid/compress options" 2020-11-10 08:08:17 +00:00
George Burgess IV
1c8d814cea fs_mgr: fix potential uses of nullptr
Clang's static analyzer flagged the following potential null pointer
dereferences. Looks like a transitive caller of this function has

```
if (change) *change = foo;
```

...and no nullness checks between that and these unconditional
assignments to `*change`.

> system/core/fs_mgr/fs_mgr_overlayfs.cpp:1100:13: warning: Dereference
of null pointer (loaded from variable 'change')
[clang-analyzer-core.NullDereference] >
system/core/fs_mgr/fs_mgr_overlayfs.cpp:1167:17: warning: Dereference of
null pointer (loaded from variable 'change')
[clang-analyzer-core.NullDereference]

Bug: None
Test: TreeHugger
Change-Id: I656e3b0f56b18ec3ca3d1db773feb81adf163122
2020-11-09 22:24:37 -08:00
Kiyoung Kim
f81058343d Merge "List up libraries with system dependency in the configuration file" 2020-11-10 05:49:35 +00:00
Akilesh Kailash
218059ff9f libsnapshot:snapuserd: Snapshot merge support.
Handle write IO during snapshot merge. When merge
is completed, dm-snapshot-merge issues WRITE IO
to indicate the completion of merge for a specific
operation. Snapuserd daemon tracks the merge completion
process in the COW header; this is required to handle
resuming merge operation gracefully if there is a crash
during merge.

Bug: 168311203
Test: vts_libsnapshot_test, cow_snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I4fc8cc23a6ec5b26b7ae3339212efdd3b9a367e8
2020-11-10 03:54:52 +00:00
Akilesh Kailash
3e64e2ba25 Merge "libsnapshot:snapuserd: Read the cow_operation in reverse order" 2020-11-10 03:51:05 +00:00
Devin Moore
6bcbeeafba libutils: add unit tests for invalid utf-8 and utf-16 strings
Characters get dropped during the conversionis between 8 and 16.

Test: atest libutils_test
Change-Id: Ie76dd38e97968137555ba2ab7ce188c9122ed06e
2020-11-10 02:54:39 +00:00
Akilesh Kailash
d08195691b libsnapshot:snapuserd: Read the cow_operation in reverse order
This is required for merge IO path. During merge process,
kernel scans the metadata page from backwards when merge
is initiated. Since, merge ordering should follow our internal
COW format, read the COW operations from backwards and populate the metadata.

Additionally, Kernel can merge successive operations if the two chunk IDs
are contiguous. This can be problematic when there is a crash
during merge; specifically when the merge operation has dependency.
These dependencies can only happen during copy operations of the
overlapping regions. To avoid this problem, we make sure that
copy operations are merged individually.

Test: cow_snapuserd_test
Bug: 168311203

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ic9c52b323ce58af897cfa343bf9277c8f1f022e3
2020-11-10 02:12:07 +00:00
Akilesh Kailash
bcf87898d7 Merge "libsnapshot:snapuserd: Fix cow_snapuserd_test." 2020-11-09 21:28:44 +00:00
Jaegeuk Kim
638d05e84e fastboot: add casefold/projid/compress options
fastboot -w --fs-options=casefold
fastboot -w --fs-options=casefold,projid
fastboot format:f2fs --fs-options=casefold
fastboot format:f2fs --fs-options=casefold,projid
fastboot format:f2fs --fs-options=casefold,projid,compress

Bug: 172514669
Bug: 160198145
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: If3fad004fdd25dd754432ce98934b4ae6c8a8706
2020-11-09 10:04:30 -08:00
Nikita Ioffe
f389e419ac Merge "Don't SIGABORT if device-mapper doesn't generate uevent for DeleteDevice" 2020-11-09 13:27:12 +00:00
Yo Chiang
cb1d0d90af Use default DSU slot name when active DSU is empty
If host system is Android Q and guest system is S, then reenable DSU in
the guest system (`adb shell gsi_tool enable -s`) would create an empty
DSU active indicator file.
In this case, use the default slot name "DSU" if DSU active indicator
file is non-existent or empty.

Bug: 172541102
Bug: 168571434
Test: Boot GSI S on Pixel Q with DSU
Test: Verify reenable DSU works
Test: adb shell gsi_tool enable -s && adb reboot
Change-Id: Ie59e4378d1aa4dca085c399a75abd4e662340030
2020-11-09 10:01:34 +00:00
David Anderson
3d152c926b Merge "libsnapshot: Fix broken iterator in CompressedSnapshotReader." 2020-11-07 01:32:06 +00:00
David Anderson
4137dee0e5 libsnapshot: Fix broken iterator in CompressedSnapshotReader.
Bug: NA
Test: apply full OTA with VABC
Change-Id: I9173ac1c6fb601a6f9e0bfb935d7908e0f234f0f
2020-11-06 15:10:41 -08:00
Nikita Ioffe
3b39a9ef6a Don't SIGABORT if device-mapper doesn't generate uevent for DeleteDevice
It looks like in some rare scenarios device-mapper won't create an
uevent while handling DM_DEV_REMOVE ioctl.

Since failure to generate uevent will result in ueventd not cleaning up
/dev paths in user space, it should be ok to just fail DeleteDevice call
without aborting the entire process.

Test: presubmit
Bug: 171266986
Change-Id: Iece28e6e02c35ac4bf33b9be74dfe1bbbb466bd4
2020-11-06 22:38:23 +00:00
Daniel Rosenberg
d021544d04 Merge "libsnapshot: Remove Initialize(fd, APPEND)" 2020-11-06 22:16:43 +00:00
Akilesh Kailash
17b5a4433e libsnapshot:snapuserd: Fix cow_snapuserd_test.
This will test the IO operation
between dm-snapshot/dm-user/snapuserd daemon. This test was broken
after cow format and daemon related changes.

Test: cow_snapuserd_test

Bug: 168311203
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I3e5f435332957a2549fd029c84efe13efb98b909
2020-11-06 22:10:10 +00:00
Treehugger Robot
f7c47f81ba Merge "first_stage_init: run first_stage.sh regardless of console presence" 2020-11-06 21:33:03 +00:00
David Anderson
f54cc3bbfd Merge changes I0b9ce272,I2bb3e55b
* changes:
  libsnapshot: Ignore non-data ops in snapshot_reader.
  libsnapshot: Round compressed COW sizes to the nearest block.
2020-11-06 19:05:42 +00:00
Treehugger Robot
674339d9f9 Merge "init: log 'updatable process' clarification" 2020-11-06 18:58:58 +00:00
Treehugger Robot
3b767c431a Merge "Use default dsu slot name when host is Android Q" 2020-11-06 17:06:12 +00:00
Steven Moreland
abc5f8830e init: log 'updatable process' clarification
-> process with updatable components

Fixes: 172605179
Test: N/A
Change-Id: I0f9353fe65cea623e1d2292f0163cc545bfc909d
2020-11-06 17:01:51 +00:00
Yo Chiang
b445a01263 Use default dsu slot name when host is Android Q
DSU slotting support is added in Android R, so
android::gsi::GetActiveDsu() is guaranteed to fail if DSU guest system
is S and host is Q.
In that case, just assume the the DSU slot to be "dsu".

Bug: 172541102
Bug: 168571434
Test: Boot GSI S on Q pixel with DSU
Change-Id: I96e191907ce2fedf8c22b7887e930a7c455eb593
2020-11-06 09:00:41 +00:00
David Anderson
6404d11a20 libsnapshot: Ignore non-data ops in snapshot_reader.
Certain ops (label, footer) do not have actual block mappings, so ignore
these.

Bug: NA
Test: apply OTA with VABC
Change-Id: I0b9ce27239553d8126c17767f35c2651df951ea0
2020-11-06 00:50:35 -08:00
David Anderson
14ae4d1dee libsnapshot: Round compressed COW sizes to the nearest block.
This is needed to create and stack device-mapper devices. The kernel
complains (or rejects) the table otherwise.

Bug: N/A
Test: manual test
Change-Id: I2bb3e55b7d999522c4c990b4ab7c46bcb78553a8
2020-11-06 00:50:34 -08:00
Daniel Rosenberg
dd6cb0d236 libsnapshot: Remove Initialize(fd, APPEND)
All cases where we want to append are done via label, so we can remove
the far more complicated open mode for append. Since this only leaves
open, we can remove the mode entirely.

Bug: 172026020
Test: cow_api_test
Change-Id: Ibe305e56b434da25952a129764398f606ca58aff
2020-11-05 18:38:58 -08:00
David Anderson
56850e12e3 Merge "libsnapshot: Add an open /dev/null mode for CowWriter." 2020-11-05 07:37:43 +00:00
David Anderson
1b9ab3f544 libsnapshot: Add an open /dev/null mode for CowWriter.
This will be used by payload_generator to estimate the COW size without
using extra storage space.

Bug: 168554689
Test: cow_api_test, ota_from_target_files
Change-Id: I095c809e7d81eff5321b86f4c6bdfb6e9467e84e
2020-11-04 20:38:50 -08:00
Treehugger Robot
f2609b2b0f Merge "libsnapshot: Only sync after labels" 2020-11-05 03:52:04 +00:00
Branden Archer
c02f17c216 Add @SensitiveData to GateKeeperService
Bug: 171501998
Test: Builds and manual evaluation
Change-Id: Ic62ca4b35529b2066cd536c5b414a53e254f808c
2020-11-04 13:51:04 -08:00
David Anderson
c497a1c63c Merge "libsnapshot: Use the COW size from the update manifest." 2020-11-04 20:20:14 +00:00
Treehugger Robot
fded8e696c Merge "Fix unused function error for user build" 2020-11-04 10:47:35 +00:00
Yo Chiang
cc86e967c7 Fix unused function error for user build
```
system/core/fs_mgr/fs_mgr_overlayfs.cpp:83:6: error: unused function
'fs_mgr_is_dsu_running' [-Werror,-Wunused-function]
```

Bug: 172405271
Bug: 165925766
Test: build aosp_bonito-user, no warning/error
Change-Id: I3206e5bd13658ab0b7f5826e8e1c206c8df65684
2020-11-04 07:02:47 +00:00
Rick Yiu
b4a3f080e6 Merge "libprocessgroup: Support write to file feature" 2020-11-04 04:54:42 +00:00
Treehugger Robot
f6d42d45c5 Merge changes I669a03a6,Ibb774d6d
* changes:
  libsnapshot: Fix footer values and verification
  libsnapshot: Fix some minor formatting bugs
2020-11-04 04:50:42 +00:00
Yo Chiang
65f4f4359d Merge changes from topic "dsu-overlayfs"
* changes:
  Refactor fs_mgr_overlayfs_teardown()
  Enable overlayFS on DSU system
2020-11-04 03:30:31 +00:00
Yo Chiang
497bca09c7 Merge "adb-remount-test.sh: Make devt errors warnings instead" 2020-11-04 03:30:31 +00:00
Yo Chiang
5ab4400db1 Merge "TransformFstabForDsu() skips handling special devices" 2020-11-04 03:30:31 +00:00