Commit graph

2936 commits

Author SHA1 Message Date
Treehugger Robot
04288337c3 Merge "Rename FstabEntry::metadata_encryption to metadata_encryption_options" 2022-03-19 04:12:05 +00:00
Treehugger Robot
5d2c849c73 Merge "libsnapshot: Fix vts_libsnapshot_test" 2022-03-19 00:14:27 +00:00
Florian Mayer
92c1ef9e97 Add liblp to hwasan-postsubmit
Test: Run tests under HWASan.
Bug: 193568145
Change-Id: I6514e9f170d03ee2d2673d115a9c8337ac6941b0
2022-03-18 21:23:51 +00:00
Akilesh Kailash
44343d4416 libsnapshot: Fix vts_libsnapshot_test
Fix SnapshotUpdateTest.QueryStatusError which
was failing on targets where userspace-snapshots are not
yet enabled.

Bug: 224586316
Test: vts_libsnapshot_test -force_config dmsnap --gtest_filter=SnapshotUpdateTest.QueryStatusError
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ibaacff9b03eafe0bfa537d0f9cab98b7caceb37e
2022-03-18 20:11:17 +00:00
Eric Biggers
20a5f92a33 Rename FstabEntry::metadata_encryption to metadata_encryption_options
There have been two bugs where people use !metadata_encryption.empty()
to check whether metadata encryption is enabled.  It should actually be
!metadata_key_dir.empty(), since 'metadata_encryption' is the encryption
options, which can be empty if the defaults are sufficient.

Rename the field in FstabEntry appropriately.

To avoid breaking fstab files, don't rename the flag in the fstab file
itself.  So, now the fstab flags map to FstabEntry fields as follows:

    keydirectory => metadata_key_dir
    metadata_encryption => metadata_encryption_options

Change-Id: I5bf673047c99e077bd6e1ac006d80e7e16bc814b
2022-03-17 23:22:50 +00:00
Akilesh Kailash
6e43216340 Revert "init: Add diagnostics for snapuserd hangs"
This reverts commit 471643a909.

Reason for revert: Given https://r.android.com/1960063, it is safe to revert this diagnostics patch

Change-Id: Ib3600c1982ee10a0204ac0fdbc3e160c2833ed07
2022-03-14 20:49:53 +00:00
Shikha Malhotra
927d9d1c68 Merge "Adding project_id ranges for internal app and cache folders." 2022-03-10 21:58:40 +00:00
David Anderson
b738b7d23a Merge "fs_mgr: Add force_super_partition bootconfig option to avoid slot suffixing alternate super partition names." 2022-03-10 04:42:20 +00:00
Akilesh Kailash
c08037b2c2 If there an error, cqe field is not allocated by io_uring_wait_cqe(); return
the error immediately.

Bug: 223308671
Test: OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ib485bb767ad58e6a0849fb2431b563e6e7f04cfd
2022-03-08 06:14:22 +00:00
Yi-yo Chiang
fb81cf765a Merge "fs_mgr: Fix misleading comment text regarding __ANDROID_RECOVERY__" 2022-03-03 06:23:14 +00:00
David Anderson
390851dd29 Merge "fs_mgr: Restrict dt fstabs to < Q devices." 2022-03-03 05:05:19 +00:00
Treehugger Robot
d297b88e3e Merge "inspect_cow: Print OTA ops for debugging" 2022-03-03 03:35:04 +00:00
Treehugger Robot
4618fa3e2a Merge "libsnapshot: Edit OWNERS file" 2022-03-03 03:15:04 +00:00
Akilesh Kailash
d49321a6fc libsnapshot: Edit OWNERS file
Bug: 220991038
Test: Presubmit
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: If1f3529b731fb19193857e4009de42dc081edbf9
2022-03-03 01:18:07 +00:00
David Anderson
0a9c083b87 vts_fs_test: Statically link libfs_mgr.
Bug: 222065493
Test: builds
Change-Id: I9ce4aa2594a4e946c79a51f0ed3d2121c7eaee36
2022-03-01 14:12:26 -08:00
David Anderson
dac0476112 fs_mgr: Add force_super_partition bootconfig option to avoid slot suffixing alternate super partition names.
Bug: 171024225
Test: manual test
Change-Id: I279b8fc6878036b3a3bc767f8d51378a2f795e19
2022-02-28 21:44:36 -08:00
Akilesh Kailash
d494442f90 inspect_cow: Print OTA ops for debugging
Bug: 220991038
Test: inspect_cow
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Id77518c7c102aca4979e6eccf45c7cc822f9ccd1
2022-02-28 22:05:21 +00:00
Akilesh Kailash
325e2acbb9 snapuserd: Fallback to synchronous I/O if any errors observed during
async merge.

If there are any I/O errors during async merge, we will
retry the I/O in synchronous I/O path. For this to happen,
we have to reset the iterator so that we replay the blocks
which were partially completed during async merge. Furthermore,
we will disable the async merge and continue to do the I/O
in synchronous path.

Additionally, cut down the queue depth to 8 so that
it will decerease the number of the async offload. We don't
want to have a big queue depth with async offload.

Bug: 220991038
Test: Instrument the code to fail the Async I/O's
randomly and make sure merge is completed. Instrumentation
was done both on readahead and merge code path.

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I0db6d0f46054ca5b8423201a598c726b2c3d21ac
2022-02-24 06:08:26 +00:00
Yi-Yo Chiang
8f73f02e43 fs_mgr: Fix misleading comment text regarding __ANDROID_RECOVERY__
This piece of comment was misleading because it only applies to
BOARD_USES_RECOVERY_AS_BOOT devices. Update the text to give a more
accurate description.

If BOARD_USES_RECOVERY_AS_BOOT is true,
* Recovery ramdisk IS boot ramdisk.
* init_first_stage is actually a symbolic link to
  init_second_stage.recovery, which links libfs_mgr.recovery.

If BOARD_USES_RECOVERY_AS_BOOT is not true,
* init_first_stage is a real binary in the generic ramdisk.
* init_first_stage links libfs_mgr.ramdisk.
* During recovery boot, the '/init' binary could be the
  init_first_stage from the generic ramdisk (A/B), or
  init_second_stage.recovery from the recovery ramdisk (non-A/B;
  standalone recovery partition).

Bug: 219811240
Test: None
Change-Id: Ib395a796f61869c13f1a5f1735ef17c224c26c8c
2022-02-23 16:33:24 +08:00
Yi-yo Chiang
ae0f4d5e45 Merge "TransformFstabForDsu: Insert synthesised entry at end of mount group" 2022-02-20 05:33:41 +00:00
David Anderson
692c6d0f74 Merge "Add a VTS test for Virtual A/B." 2022-02-19 01:28:59 +00:00
David Anderson
7a5379d0e1 Merge "fs_mgr: Fix __ANDROID_RECOVERY__ usage in libfiemap." 2022-02-18 23:29:27 +00:00
David Anderson
05ffcf2d9c fs_mgr: Restrict dt fstabs to < Q devices.
Bug: 214857162
Test: vts_fs_test
Change-Id: Iee3c657297fa76b1131d03a638ff7f5620cea414
2022-02-18 15:22:02 -08:00
David Anderson
36a4784681 Add a VTS test for Virtual A/B.
Bug: N/A
Test: vts_ota_config_test
Change-Id: Ibcee72dcbd9b8d157c3cc989d46fba6277115207
2022-02-18 14:16:44 -08:00
Yi-Yo Chiang
6bcabc03d3 TransformFstabForDsu: Insert synthesised entry at end of mount group
The intention of inserting this synthesised mount entry is for the
FirstStageMount() procedure to eventually fallback to this entry if all
previous mount lines failed.
In order for FirstStageMount() to retry mount, the mount lines have to
be grouped together.
This change ensures that would happen and add test.

Bug: 220074274
Test: atest CtsFsMgrTestCases
Change-Id: Id042a6a6738d27c06a397ef7f4e0977907371c05
2022-02-18 23:00:25 +08:00
David Anderson
c2b2bf3539 Merge "remount: Create a bigger scratch partition on /data." 2022-02-18 01:56:27 +00:00
David Anderson
460886c175 Merge "fs_mgr: Don't allow dt fstabs in new devices." 2022-02-18 01:56:09 +00:00
David Anderson
fb921468ab fs_mgr: Fix __ANDROID_RECOVERY__ usage in libfiemap.
Certain code in libfiemap should be unreachable in recovery/ramdisk
builds, but whether intentionally or not, __ANDROID_RECOVERY__ is not
working in vendor_boot.

To make sure this is doing exactly what we want, use __ANDROID_RAMDISK__
instead.

Bug: 205987817
Test: build boot, vendor_boot, recovery
Change-Id: Id419f7d9068e791427a896c22569ae4543548df7
2022-02-17 13:06:13 -08:00
David Anderson
c1c05daac1 remount: Create a bigger scratch partition on /data.
When creating a scratch partition on /data, use either the size of
super, or half the free space on /data - whichever is smaller.

Bug: 213617068
Test: adb remount
Change-Id: If42fe9e1dc53d5a0137ace917a440232f70947fa
2022-02-17 11:11:07 -08:00
David Anderson
18c6157902 fs_mgr: Don't allow dt fstabs in new devices.
Bug: 214857162
Test: vts_fs_test
Change-Id: I69870fa632b01fdb411371cbb8c5b17271631395
2022-02-17 10:30:59 -08:00
David Anderson
982c3410c7 fastbootd: Remove all scratch partitions on update-super.
This ensures that the dynamic "scratch" partition is removed when doing
a flashall operation.

If "scratch" is on /data, disable the partition. Add IsImageDisabled to
ImageManager so EnsureScratchMapped can skip mapping it. Also, fix
"scratch" not getting unmapped if on /data.

Bug: 205987817
Test: adb remount
      adb sync
      adb reboot fastboot
      fastboot flashall --skip-reboot # no errors
Test: adb-remount-test.sh
Change-Id: I4b9702e1dac15fb663635506fb50a8274e1e10d1
2022-02-17 04:16:53 +00:00
Akilesh Kailash
0cc213be87 Disable Async merge.
I/O failures seen on one specific device intermittently.

Bug: 219642530
Test: OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I5b6fbfb13e40b15b25324b70efef11aede916596
2022-02-16 04:20:54 +00:00
Akilesh Kailash
1a74a82d23 Disable async merge writes until we figure out the intermittent
I/O errors.

Bug: 219642530
Test: OTA

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ica8e95394b92e6893c944676d28e05af2bde9a83
2022-02-15 23:22:04 +00:00
Bob Badour
c9605e6ae1 Add LOCAL_LICENSE_KINDS to system/core
Added SPDX-license-identifier-Apache-2.0 to:
  fastboot/fuzzer/Android.bp
  fs_mgr/fuzz/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Change-Id: I3bb21e1af846d175a345c2a3a632dc47ecdd62de
2022-02-10 11:12:20 -08:00
Akilesh Kailash
1aad41c3e7 Merge changes I5458127b,Ib36bf44c,I481548e8,I449de193
* changes:
  libsnapshot: Propagate io_uring enablement to daemon
  snapuserd: Cut down worker threads
  snapuserd: Terminate daemon spin up during first stage init
  snapuserd: Avoid checking system properties when daemon launched from fist stage init and during selinux transition
2022-02-10 07:34:48 +00:00
Akilesh Kailash
a0049a122d libsnapshot: Propagate io_uring enablement to daemon
During selinux init transition, system properties
are not yet enabled. Hence, store the io_uring feature
in snapshot.proto and propagate the same snapuserd
daemon

Bug: 214340811
Test: OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I5458127b00946b5254d5c68d407ab525a0075cb9
2022-02-09 22:46:49 +00:00
Akilesh Kailash
1ab17cc2a1 snapuserd: Cut down worker threads
We don't need 4 threads per partition all the time.
Cut down to 1 thread for all purposes except during boot time.

During boot, we need multiple worker threads to serve
I/O's to speed up the booting process. For all other
purposes, single thread is sufficient. It will cut
down memory usage ~25MB.

Bug: 214340811
Test: OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ib36bf44c63676d3f3169fbda04b7fc3d1760cfbf
2022-02-09 21:26:27 +00:00
Akilesh Kailash
b7f0a04201 snapuserd: Terminate daemon spin up during first stage init
Bug: 214340811
Test: OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I481548e878ff7ee5fdf7343fab961a1ded391634
2022-02-09 21:26:11 +00:00
Akilesh Kailash
d5f320193c snapuserd: Avoid checking system properties when daemon launched
from fist stage init and during selinux transition

Bug: 207298357
Test: OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I449de1939531dbf3cc35e05c80e51237c4dc2b99
2022-02-09 21:25:59 +00:00
Treehugger Robot
45df56880c Merge "system_dlkm: Allow overlayfs mount for user build" 2022-02-08 06:50:14 +00:00
Ramji Jiyani
023d897961 system_dlkm: Allow overlayfs mount for user build
Bug: 200082547
Test: Verification of overlayfs entries with below
      1. adb root && adb remount && adb reboot
      2. adb shell mount | grep system_dlkm
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
Change-Id: I3fae1fcc6bba81795d6646c10d9fb9552ead8370
2022-02-08 02:13:33 +00:00
David Anderson
4b7400756c Merge "fs_mgr: Add VTS tests for EROFS." 2022-02-07 18:38:40 +00:00
Shikha Malhotra
bc9b847e0f Adding project_id ranges for internal app and cache folders.
Test: atest StorageHostTest
Test: atest installd/tests/installd_service_test.cpp
Change-Id: I4dbda6f2f0d74cb4f4890555742b26c81a6b0146
2022-02-04 15:37:36 +00:00
David Anderson
3de5c2c25d fs_mgr: Add VTS tests for EROFS.
Bug: 211775434
Test: vts_fs_test
Change-Id: I438efe110bb388c28b47753501168449fcb8fbe5
2022-02-03 14:47:47 -08:00
David Anderson
ed54a2f2be Merge "libfiemap: Add a helper to verify image fiemaps." 2022-02-01 22:43:46 +00:00
Akilesh Kailash
fd5562b0a5 init: Wait for snapuserd before starting second stage
This is a race between init process and bionic libc initialization of
snapuserd.

init->fork() ----------------> SecondStageMain() -> PropertyInit()
       |
       |
       v
      execveat ---> __libc_init_common() -> __system_properties_init()
     (snapuserd)

When init process calls PropertyInit(), /dev/__properties__ directory
is created. When bionic libc of snapuserd daemon invokes __system_properties_init
_after_ init process PropertyInit() function is invoked, libc will
try to initialize the property by reading
/system/etc/selinux/plat_property_contexts. Since any reads on /system
has to be served by snapuserd, this specific read from libc cannot be
serviced leading to deadlock.

Reproduce the race by inducing a sleep of 1500ms just before execveat()
so that init process calls PropertyInit() before bionic libc
initialization. This leads to deadlock
immediately and with additional kernel instrumentation with debug
logs confirms the failure:

======================================================
init: Relaunched snapuserd with pid: 428
ext4_file_open: SNAPUSERD: path /system/etc/selinux/plat_property_contexts - Pid: 428 comm 8
ext4_file_read_iter: SNAPUSERD for path: /system/etc/selinux/plat_property_contexts pid: 428 comm 8

[   25.418043][  T428]  ext4_file_read_iter+0x3dc/0x3e0
[   25.423000][  T428]  vfs_read+0x2e0/0x354
[   25.426986][  T428]  ksys_read+0x7c/0xec
[   25.430894][  T428]  __arm64_sys_read+0x20/0x30
[   25.435419][  T428]  el0_svc_common.llvm.17612735770287389485+0xd0/0x1e0
[   25.442095][  T428]  do_el0_svc+0x28/0xa0
[   25.446100][  T428]  el0_svc+0x14/0x24
[   25.449825][  T428]  el0_sync_handler+0x88/0xec
[   25.454343][  T428]  el0_sync+0x1c0/0x200

=====================================================

Fix:

Before starting init second stage, we will wait
for snapuserd daemon to be up and running. We do a simple probe by
reading system partition. This read will eventually be serviced by
daemon confirming that daemon is up and running. Furthermore,
we are still in the kernel domain and sepolicy has not been enforced yet.
Thus, access to these device mapper block devices are ok even though
we may see audit logs.

Note that daemon will re-initialize the __system_property_init()
as part of WaitForSocket() call. This is subtle but important; since
bionic libc initialized had failed silently, it is important
that this re-initialization is done.

Bug: 207298357
Test: Induce the failure by explicitly delaying the call of execveat().
      With fix, no issues observed.
      Tested incremental OTA on pixel ~15 times.
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I86c2de977de052bfe9dcdc002dcbd9026601d0f3
2022-01-25 08:30:08 +00:00
David Anderson
471643a909 init: Add diagnostics for snapuserd hangs
This patch attempts to diagnose snapuserd hangs by performing reads
immediately after entering second-stage init. This is done by spawning
two threads: one to perform the reads, and another to wait for the read
thread to finish. If any aspect of the read fails, or the read thread
does not complete in 10 seconds, then a list of snapuserd's open file
descriptors are logged.

Bug: 207298357
Test: apply working OTA, check logcat for success
      apply broken OTA, check logcat for fd map
Change-Id: I549e07b7d576fcdaca9b2d6ff33e0924c3812c07
2022-01-21 17:35:33 +00:00
Akilesh Kailash
3991ce6c30 Merge changes from topic "snapuserd-async-merge"
* changes:
  Add a new property to disable io_uring and run vts and snapuserd_test
  snapuserd: Async I/O for block verification
  snapuserd: Use io_uring api's for snapshot merge
2022-01-14 00:06:23 +00:00
Akilesh Kailash
0616cb17c2 libsnapshot: Terminate snapuserd daemon after merge completion
Bug: 214340811
Test: OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Icaed87feb7760074d743229856f23f3599e03c62
2022-01-13 23:51:02 +00:00
Akilesh Kailash
401a46a597 Add a new property to disable io_uring and run
vts and snapuserd_test

This should be run on cuttlefish

Bug: 202784286
Test: vts_libnspahost_test, snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I2c28e98f04beca770b8a6efa9474d602fe26f514
2022-01-13 07:18:00 +00:00