Commit graph

64806 commits

Author SHA1 Message Date
David Anderson
2eb7b9221b libsnapshot: Implement MapAllSnapshots and UnmapAllSnapshots.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I6809e226741dabcf337c3a5cfaba56afdb9edd64
2020-11-02 13:24:15 -08:00
David Anderson
b031def229 Start snapuserd if needed as soon as possible during second-stage init.
snapuserd is used as a user-space block device implementation during
Virtual A/B Compression-enabled updates. It has to be started in
first-stage init, so that updated partitions can be mounted.

Once init reaches second-stage, and sepolicy is loaded, we want to
re-launch snapuserd at the correct privilege level. We accomplish this
by rebuilding the device-mapper tables of each block device, which
allows us to re-bind the kernel driver to a new instance of snapuserd.
After this, the old daemon can be shut down.

Ideally this transition happens as soon as possible, before any .rc
scripts are run. This minimizes the amount of time the original
snapuserd is running, as well as any ambiguity about which instance of
snapuserd is the correct one.

The original daemon is sent a SIGTERM signal once the transition is
complete. The pid is stored in an environment variable to make this
possible (these details are implemented in libsnapshot).

Bug: 168259959
Test: manual test
Change-Id: Ife9518e502ce02f11ec54e7f3e6adc6f04d94133
2020-11-02 13:24:06 -08:00
David Anderson
f57fd97df3 libsnapshot: Add support for first-to-second-stage transitions of snapuserd.
This patch introduces the fundamentals needed to support booting off
dm-user. First, a method has been added to start snapuserd in
first-stage init. It simply forks and execs, creates a specially named
first-stage socket, then waits for requests.

Next, a new method has been added to SnapshotManager to perform a
second-stage handoff. This works by first launching a second copy of
snapuserd using init's normal service management functionality. The new
snapuserd runs alongside the original, but has correct privileges and a
correct selinux context. Next, we inspect each COW device, and if its
table uses dm-user, we replace the table with a renamed control
device. The new control device is bound to the new snapuserd.

device-mapper guarantees that such a table swap is safe. It flushes I/O
to the old table and then replaces it with the new table. Once the new
table is in place, the old dm-user control devices are automatically
destroyed. Thus, once all dm-user devices has been transitioned, the
first-stage daemon is idle and can gracefully exit.

This patch does not modify init. A few changes will be needed on top of
this patch:

(1) CreateLogicalAndSnapshotPartitions will need further changes to
start the first-stage daemon and track its pid. Additionally, it will
need to ensure the named socket file is deleted, so there is no further
IPC allowed after partitions are completed.
(2) init will need to propagate the pid to second-stage init so the
process can be killed (or signalled).
(3) first-stage snapuserd will need to gracefully exit once it has no
active handler threads.
(4) second-stage init will need to invoke the transition helper on
SnapshotMaanager, ideally as soon as feasible.

Bug: 168259959
Test: manual test
Change-Id: I54dec2edf85ed95f11ab4518eb3d7dbaf0bdcbfd
2020-11-02 13:24:05 -08:00
David Anderson
4e4cff7e46 Merge "libsnapshot: Add a tool for inspecting COW files." 2020-10-29 00:27:28 +00:00
David Anderson
5fea7df825 Merge "libsnapshot: Add a skeleton API for mapping and unmapping all snapshots." 2020-10-29 00:27:19 +00:00
David Anderson
a1a5fdc6ee Merge "libsnapshot: Stop reading ops once we reach a footer." 2020-10-29 00:27:02 +00:00
Yifan Hong
7173ca82a4 Merge "Revert "Always create ramdisk/first_stage_ramdisk."" 2020-10-28 21:21:36 +00:00
David Anderson
49a428b62e Merge changes I2b3f5b33,I464b683b,I8e97c543
* changes:
  libsnapshot: Remove the timeout on client recv().
  libsnapshot: Integrate with snapuserd.
  snapuserd: Add an API call to wait for device deletion.
2020-10-28 01:55:09 +00:00
David Anderson
3fa66c7283 libsnapshot: Add a tool for inspecting COW files.
This simple tool will dump the COW header and included ops to stdout.

Bug: N/A
Test: mm inspect_cow && inspect_cow <file>
Change-Id: I369c4a21a84c95ffc10670bd9eeb2ceccb2a56d6
2020-10-27 18:30:13 -07:00
David Anderson
07ad1b3566 libsnapshot: Add a skeleton API for mapping and unmapping all snapshots.
Bug: 168554689
Test: builds
Change-Id: I0da50275cc4d0a85bd6b176b5c1286659d51a25c
2020-10-27 18:30:13 -07:00
David Anderson
485fe695c1 libsnapshot: Stop reading ops once we reach a footer.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: Id8b5ba33220028c856d3761832fe231fd93e94cd
2020-10-27 18:30:13 -07:00
Baligh Uddin
30d70441e5 Merge "Revert "Revert "Switch "system/core/base" ref to "system/libbase"""" 2020-10-27 19:38:09 +00:00
Baligh Uddin
f0c169a7fd Revert "Revert "Switch "system/core/base" ref to "system/libbase"""
This reverts commit 2f77d1adc8.

Reason for revert: Applying a Fix to DS directly.  No need for merged-In, since the topic is already landed in DS branch

Change-Id: I86cba9b20efebc9e700522e1697bc8f893c43089
2020-10-27 19:37:59 +00:00
David Anderson
0cbc6e8119 Merge changes I48e62f25,Ib04e80e8,I3878abfd
* changes:
  snapuserd: Fix infinite loop when reading ops.
  libsnapshot: Add Initialize and InitializeAppend methods to ISnapshotWriter.
  libsnapshot: Implement OpenReader for CompressedSnapshotWriter.
2020-10-27 16:48:12 +00:00
Elliott Hughes
52dac2f8b7 Merge "Remove unused utf8_length()." 2020-10-27 15:31:03 +00:00
Joanne Chung
ad5f9c3820 Merge "Revert "Switch "system/core/base" ref to "system/libbase""" 2020-10-27 09:40:29 +00:00
Joanne Chung
2f77d1adc8 Revert "Switch "system/core/base" ref to "system/libbase""
This reverts commit e960b47673.

Reason for revert: Looks to have broken sc-d1-dev b/171770441

Change-Id: I958a95c3b57e46cab2ce17c4a4f9e80b45f2ad71
2020-10-27 09:14:21 +00:00
Woody Lin
9b4debbc27 Merge "init/service_parser: Add arguments window' and target' for `critical'" 2020-10-27 08:22:14 +00:00
David Anderson
0a03a5a8ec snapuserd: Fix infinite loop when reading ops.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I48e62f258ef2b4c368e8237a132c802a03d7020b
2020-10-26 22:30:21 -07:00
David Anderson
a2b5d9978a libsnapshot: Add Initialize and InitializeAppend methods to ISnapshotWriter.
This is so update engine can resume from the correct label.

Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: Ib04e80e8219f954f105d5a85f86efa7bb9097579
2020-10-26 22:30:21 -07:00
David Anderson
0544f91a9a libsnapshot: Implement OpenReader for CompressedSnapshotWriter.
Bug: 168554689
Test: vts_libsnapshot_test
Test: full OTA with update_device.py
Test: incremental OTA with update_device.py
Change-Id: I3878abfd767d2e47cf8486bc2c06233da2f1ef08
2020-10-26 22:30:21 -07:00
Baligh Uddin
25254eb315 Merge "Switch "system/core/base" ref to "system/libbase"" 2020-10-27 03:38:20 +00:00
Elliott Hughes
939e43e66e Remove unused utf8_length().
Test: treehugger
Change-Id: Idcebc4ae1dcad102873d50f199f5e8745e589da4
2020-10-26 13:14:47 -07:00
Treehugger Robot
df17454a84 Merge "Fix lseek() check" 2020-10-26 17:24:31 +00:00
Greg Kaiser
74b8449f58 Fix lseek() check
Since 'pos' is unsigned, we need to tweak the check.

Test: TreeHugger
Change-Id: I97dcb50fb86196b09b11ba725593e6c83c5586d9
2020-10-26 06:37:32 -07:00
Yo Chiang
346e6792b4 Merge "Call GetDsuMetadataKeyDir() for DSU metadata encryption dir" 2020-10-26 05:27:36 +00:00
Woody Lin
45215ae6e5 init/service_parser: Add arguments window' and target' for `critical'
The critical services can now using the interface `critical
[window=<fatal crash window mins>] [target=<fatal reboot target>]` to
setup the timing window that when there are more than 4 crashes in it,
the init will regard it as a fatal system error and reboot the system.

Config `window=${zygote.critical_window.minute:-off}' and
`target=zygote-fatal' for all system-server services, so platform that
configures ro.boot.zygote_critical_window can escape the system-server
crash-loop via init fatal handler.

Bug: 146818493
Change-Id: Ib2dc253616be6935ab9ab52184a1b6394665e813
2020-10-26 11:38:01 +08:00
Baligh Uddin
e960b47673 Switch "system/core/base" ref to "system/libbase"
BUG: 148941208
Test: TH
Change-Id: Ifa87b06ad7cfce94ed9b5be6b1ee0c358e52aba1
2020-10-24 02:31:13 +00:00
David Anderson
45662c8941 Merge "snapuserd: Refactor client to allow persistent connections." 2020-10-23 22:01:42 +00:00
David Anderson
8425f6e6e6 Merge "snapuserd: Refactor daemon/server." 2020-10-23 18:33:15 +00:00
Colin Cross
b527e66392 Merge "Make the connection between implementation and llndk_library explicit" 2020-10-23 18:09:42 +00:00
Primiano Tucci
24a21867e7 Merge "Introduce security.lower_kptr_restrict property" 2020-10-23 18:03:30 +00:00
Treehugger Robot
d0ce499873 Merge "fs_mgr.cpp: fix the problem that the casefold feature not enabled for the data partition" 2020-10-23 16:47:43 +00:00
Baligh Uddin
3f52272883 Merge "Migrate system/core/adb to packages/modules/adb" 2020-10-23 16:16:32 +00:00
Daniel Rosenberg
323060840a Merge changes I3339d552,I126e1583,Iccc6580a
* changes:
  Add GetLastLabel and InitializeAppend
  Switch up Cow Format to be resumable
  reland: Rename Flush to Finalize
2020-10-23 11:08:46 +00:00
Yongqin Liu
c007c43335 fs_mgr.cpp: fix the problem that the casefold feature not enabled for the data partition
The value of entry.mount_point for data partition is "/data"

Fixes: 5ba5b90cd6 ("fs_mgr: try tune2fs for casefolding on /data only")

Test: got "Can't mount with encoding and encryption" problem reported
      by the db845c build with the default 5.4.38 prebuilt kernel

Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
Change-Id: I226a2275f5f2ee18503c5a3863ef5a1d2c2ed7be
2020-10-23 13:34:49 +08:00
David Anderson
6494a8ca97 libsnapshot: Remove the timeout on client recv().
Two seconds is a bit aggressive - considering this is analagous to a
synchronous binder call, let's drop the timeout entirely.

Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I2b3f5b33f79575d72b15ed314dbcc0ad20ebd9a8
2020-10-22 22:07:25 -07:00
David Anderson
8e46846501 libsnapshot: Integrate with snapuserd.
This integrates libsnapshot with dm-user and snapuserd. Tests progress
significantly further now. Tests involving merging still fail as
snapuserd doesn't support this yet.

Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I464b683b464fe29a646f0f2823b7f4434a878614
2020-10-22 22:07:24 -07:00
David Anderson
6a1e9ca279 snapuserd: Add an API call to wait for device deletion.
This adds a new message to the daemon protocol, which waits for a device
to be deleted. The caller must ensure that the corresponding control
device is actually going away (eg, the device containing the dm-user
table entry has been deleted). Otherwise, this will hang.

This will allow libsnapshot to safely delete the cow since any
outstanding references will be closed.

This also refactors DmUserHandler so that it's freed (and removed from
the handler list) if its corresponding thread exits of its own accord.

Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I8e97c543eec84874c88795a493470e992dc476fc
2020-10-22 22:07:24 -07:00
David Anderson
fe7585a8b0 snapuserd: Refactor client to allow persistent connections.
This refactors SnapuserdClient so it retains a connection for its
lifetime. This allows SnapshotManager to ensure the daemon is running
and hold a connection open across all of its operations.

The main impetus of this change is to remove the ambiguity between first
and second-stage sockets. SnapshotManager should only ever connect to
the first-stage socket during first-stage init, or, to initiate the
"transition" step during second-stage init.

The transition steps are roughly:
 (1) Start second-stage daemon.
 (2) Load new device-mapper tables.
 (3) Connect second-stage daemon to new dm-user devices.
 (4) Activate the new tables, flushing IO to the first-stage daemon.
 (5) Send a signal to the first-stage daemon to exit.

This patch makes it easier to hold these two separate connections.

Bug: 168554689
Test: manual test
Change-Id: I51cb9adecffb19143ed685e0c33456177ec3d81f
2020-10-22 22:07:23 -07:00
David Anderson
cff35f67ad snapuserd: Refactor daemon/server.
This is in preparation for moving to a traditional client/server model
where clients stay connected and the server multiplexes multiple
connections.

Client has been renamed to DmUserClient to differentiate it from local
socket clients.

poll() responsibilities have been moved into SnapuserdServer. In
addition, the server now tracks all open clients and polls them
together with the listen socket.

SnapuserDaemon is now only responsible for signal masking. These two
classes can probably be merged together - I didn't do that here because
the patch was already large.

Bug: 168554689
Test: manual test
Change-Id: Ibc06f6287d49e832a8e25dd936ec07747a1b0555
2020-10-22 22:00:17 -07:00
Daniel Rosenberg
3d17cb9a57 Add GetLastLabel and InitializeAppend
GetLastLabel returns the last Label that a reader is confident about.
InitializeAppend starts a writer up to append data after the last given
label, assuming all later labels are not relevant data.

Change-Id: I3339d5527bae833d9293cbbc63126136b94bd976
Bug: 168829493
Test: cow_api_test
2020-10-22 21:26:27 -07:00
Daniel Rosenberg
2d2fd72502 Switch up Cow Format to be resumable
This switches up the format to alternate ops with data, followed by a
footer containing additional meta information. This allows the file to
be resumed at arbitrary points if writing gets interrupted by power
loss.
Also adds a label op, which allows labeling future ops as connected.
If the footer is missing, Append will treat the last label as possibly
corrupt, and ignore it.

Change-Id: I126e15837d710776f9396e7afc9b0cd595e26b59
Bug: 168829493
Test: cow_api_test
2020-10-22 21:26:27 -07:00
Baligh Uddin
795c2c222c Migrate system/core/adb to packages/modules/adb
BUG: 167963357
Test: TH
Merged-In: Ie1f82db2fb14e1bdd183bf8d3d93d5e9f974be5d
Change-Id: I810a109116247af2af9a8628680620cb683d48a9
2020-10-23 03:43:43 +00:00
Yifan Hong
586be305a4 Revert "Always create ramdisk/first_stage_ramdisk."
This reverts commit 42c55f5ce9.

Reason for revert: b/171512004 It should be created at runtime.

Bug: 171512004
Change-Id: If9277f078cb343fbad825f0e8d1348d50f4b759a
2020-10-22 21:58:35 +00:00
David Anderson
7d55df2895 Merge "libsnapshot: Add .rc file for snapuserd." 2020-10-22 18:21:18 +00:00
Primiano Tucci
253289fe07 Introduce security.lower_kptr_restrict property
This is to allow the tracing service to temporarily
lower kptr_restrict for the time it takes to build
its internal symbolization map (~200ms), only on
userdebug/eng builds.
kptr_restrict unfortunately cannot be lowered by
the tracing service itself. The main reason for that
is the fact that the kernel enforces a CAP_SYS_ADMIN
capability check at write() time, so the usual pattern
of opening the file in init and passing the FD to the
service won't work.

For more details see the design doc go/perfetto-kallsyms.
Bug: 136133013
Test: perfetto_integrationtests --gtest_filter=PerfettoTest.KernelAddressSymbolization in r.android.com/1454882

Change-Id: Ib2a8c69ed5348cc436223ff5e3eb8fd8df4ab860
2020-10-22 16:18:41 +01:00
Daniel Rosenberg
b4a81ccd5a reland: Rename Flush to Finalize
As we change to a more resumable format, flush mostly writes the final
parts of the file that are needed, which would write extra data that is
not needed to continue writing, and would immediately be overwritten.

Additionally, in the next patch we will fsync the file after adding an
op, making the flush built in, and the Finalize name more appropriate.

Bug: 168829493
Test: builds
Change-Id: Iccc6580ac72ff066cfeeb32e3cdaf69c5ba615fc
2020-10-22 02:34:26 -07:00
Treehugger Robot
1ef094a890 Merge "Use the "marketing names" for Q and R." 2020-10-22 01:00:54 +00:00
Christopher Ferris
758e52551f Merge "Update for new kernel headers." 2020-10-22 00:54:16 +00:00