Commit graph

2554 commits

Author SHA1 Message Date
Yi-Yo Chiang
67e3bbc7f6 Explicitly cast to 64bit integer when calculating filesystem size
Else the result may overflow on platforms that have 32bit long.

Bug: 165925766
Test: Presubmit
Change-Id: I3018f0bd9846651848bd9b3645f2eeaa5b61c646
2021-02-18 08:24:51 +00:00
Yi-Yo Chiang
d1ff1bcfbc fs_mgr: Add OverlayMountPoints() to get the list of overlayfs backing storage
Remove kOverlayMountPoints; add OverlayMountPoints().

Fix a regression where if host system didn't install a scratch_gsi
partition and physical /cache partition is mounted, then DSU guest
system could use /cache as its overlayfs backing storage. This is
generally unwanted as the /cache partition could be shared between host
and guest.
Dynamically return the list of overlayfs backing storage candidates, so
we don't accidentally fall back to use /cache storage within a DSU
system.

Bug: 165925766
Bug: 179980369
Test: 1. Prepare a DUT that have a physical cache partition, such as
  cuttlefish.
  2. Install a DSU system, and unsure that scratch_gsi is not installed.
  3. Reboot into DSU, verify that adb remount fails and overlayfs
     scratch is not created under /cache.
Change-Id: I1815ac5367c0aac8614aeaabebe0e2cb91cbe161
2021-02-18 04:36:52 +00:00
Jaegeuk Kim
286e5aba63 Merge "fiemap: support FIEMAP_EXTENT_MERGED" 2021-02-17 01:42:23 +00:00
Treehugger Robot
ca474f994d Merge "Revert "[LSC] Add LOCAL_LICENSE_KINDS to system/core"" 2021-02-16 23:25:31 +00:00
Akilesh Kailash
51858a0494 Merge "libsnapshot: Print merge completion ops" 2021-02-16 22:03:30 +00:00
Jaegeuk Kim
0724e48f9f fiemap: support FIEMAP_EXTENT_MERGED
F2FS starts to give a hint, FIEMAP_EXTENT_MERGED, in fiemap, so please allow it.

Bug: 180137825
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: Ic4e3f573e385d248627da1a2ad64fbbd4a74e679
2021-02-16 13:22:13 -08:00
Elliott Hughes
c3a206ccda Revert "[LSC] Add LOCAL_LICENSE_KINDS to system/core"
This reverts commit 187b7d1950.

Reason for revert: system/core is multiple projects, not one.

Change-Id: I790ea41741f8cd9b8b6db2f59a49e71fb0958fd6
2021-02-16 20:01:20 +00:00
Akilesh Kailash
1ce6a7e40c libsnapshot: Print merge completion ops
Once the daemon is terminated, print merge completion
ops and the total ops present in the COW file. This
will help to know if the merge operation was interrupted
and how many pending operations were done during
each reboot until merge is completed

Bug: 167409187
Test: Incremental and full OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ie33c840e80aaeca86f51adc8085cb4e306dca110
2021-02-16 19:19:40 +00:00
Bob Badour
187b7d1950 [LSC] Add LOCAL_LICENSE_KINDS to system/core
Added SPDX-license-identifier-Apache-2.0 to:
  bootstat/Android.bp
  cli-test/Android.bp
  code_coverage/Android.bp
  cpio/Android.bp
  debuggerd/crasher/Android.bp
  debuggerd/proto/Android.bp
  diagnose_usb/Android.bp
  fs_mgr/libdm/Android.bp
  fs_mgr/libfiemap/Android.bp
  fs_mgr/liblp/Android.bp
  fs_mgr/libsnapshot/Android.bp
  fs_mgr/libstorage_literals/Android.bp
  fs_mgr/libvbmeta/Android.bp
  fs_mgr/tests/Android.bp
  fs_mgr/tools/Android.bp
  gatekeeperd/Android.bp
  healthd/Android.bp
  healthd/testdata/Android.bp
  init/Android.bp
  init/Android.mk
  init/sysprop/Android.bp
  init/test_kill_services/Android.bp
  init/test_service/Android.bp
  libappfuse/Android.bp
  libasyncio/Android.bp
  libbinderwrapper/Android.bp
  libcrypto_utils/Android.bp
  libcrypto_utils/tests/Android.bp
  libdiskconfig/Android.bp
  libgrallocusage/Android.bp
  libkeyutils/mini_keyctl/Android.bp
  libmodprobe/Android.bp
  libnetutils/Android.bp
  libpackagelistparser/Android.bp
  libprocessgroup/Android.bp
  libprocessgroup/cgrouprc/Android.bp
  libprocessgroup/cgrouprc_format/Android.bp
  libprocessgroup/profiles/Android.bp
  libprocessgroup/setup/Android.bp
  libqtaguid/Android.bp
  libsparse/Android.bp
  libstats/push_compat/Android.bp
  libsuspend/Android.bp
  libsync/Android.bp
  libsystem/Android.bp
  libsysutils/Android.bp
  libusbhost/Android.bp
  libutils/Android.bp
  libvndksupport/Android.bp
  libvndksupport/tests/Android.bp
  llkd/Android.bp
  llkd/tests/Android.bp
  property_service/libpropertyinfoparser/Android.bp
  property_service/libpropertyinfoserializer/Android.bp
  property_service/property_info_checker/Android.bp
  qemu_pipe/Android.bp
  reboot/Android.bp
  rootdir/Android.bp
  rootdir/Android.mk
  rootdir/avb/Android.bp
  rootdir/avb/Android.mk
  run-as/Android.bp
  sdcard/Android.bp
  set-verity-state/Android.bp
  shell_and_utilities/Android.bp
  storaged/Android.bp
  toolbox/Android.bp
  trusty/apploader/Android.bp
  trusty/confirmationui/Android.bp
  trusty/confirmationui/fuzz/Android.bp
  trusty/coverage/Android.bp
  trusty/fuzz/Android.bp
  trusty/fuzz/test/Android.bp
  trusty/gatekeeper/Android.bp
  trusty/gatekeeper/fuzz/Android.bp
  trusty/keymaster/Android.bp
  trusty/keymaster/fuzz/Android.bp
  trusty/libtrusty/Android.bp
  trusty/libtrusty/tipc-test/Android.bp
  trusty/secure_dpu/Android.bp
  trusty/storage/interface/Android.bp
  trusty/storage/lib/Android.bp
  trusty/storage/proxy/Android.bp
  trusty/storage/tests/Android.bp
  trusty/utils/spiproxyd/Android.bp
  trusty/utils/trusty-ut-ctrl/Android.bp
  usbd/Android.bp
  watchdogd/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD to:
  debuggerd/Android.bp
  fastboot/Android.bp
  libkeyutils/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD
    SPDX-license-identifier-MIT
to:
  Android.bp
  libcutils/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-MIT
to:
  fs_mgr/Android.bp
  fs_mgr/libfs_avb/Android.bp
  trusty/utils/rpmb_dev/Android.bp

Added SPDX-license-identifier-BSD
to:
  fastboot/fuzzy_fastboot/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work
Change-Id: I5bd81adb5cdcf2b4dd4141b204eb430ff526af8f
2021-02-16 04:10:03 -08:00
Akilesh Kailash
7cd1c03178 Merge "libsnapshot:snapuserd: Batch merge copy operation" 2021-02-10 05:08:48 +00:00
Akilesh Kailash
b5f0a3b73b Merge "libsnapshot: No transition of snapuserd during second stage init" 2021-02-10 00:35:23 +00:00
Akilesh Kailash
66914e2b2e libsnapshot:snapuserd: Batch merge copy operation
Allow batch merge of copy operations during merge.
When metadata is read from COW device, assign
the chunk-id by validating there is no overlap
of copy operations. Furthermore, detect the blocks
which are contiguous and batch merge them.

No regression in merge time for full OTA (~35-40 seconds)

Merge time for incremental OTA of ~200M takes about 2 minutes
as compared to 15-20+ minutes without this change.

Add unit test to test ReadMetadata() functionality.

Multiple incremental OTA and full OTA test done on pixel.
adb reboot during merge and validate the merge resume operations.

Bug: 179629624
Test: incremental OTA and full OTA on pixel,
      cow_snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I4cd84e4923e42afacc796b8cec01738b1bb1f420
2021-02-09 23:53:49 +00:00
Akilesh Kailash
03e803455e libsnapshot: No transition of snapuserd during second stage init
When there is a transition of daemon from selinux stage, we observe
intermittent hangs during OTA. This is a workaround wherein
we don't do the transition and allow the daemon to continue which
was spawned during selinux stage.

Bug: 179331261
Test: Incremental OTA, full OTA on pixel
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I622a0ed8afcd404bac4919b1de00728de2c12eaf
2021-02-09 22:45:29 +00:00
Treehugger Robot
dc0ea9d65e Merge "adb-remount-test.sh: Fix failure due to missing host tool hexdump" 2021-02-09 21:52:19 +00:00
Yi-Yo Chiang
fa405414d7 adb-remount-test.sh: Fix failure due to missing host tool hexdump
As we are just using it to generate gibberish, we might as well just
hardcode the gibberish in the script.

Also fix unhandled `grep` failure.

Bug: 179752308
Test: Forrest
Change-Id: I534c7cacdb12a104f26d380fe3a571332091490e
2021-02-09 16:11:23 +08:00
Treehugger Robot
c9924a763d Merge "libfs_avb: verifying vbmeta digest early" 2021-02-09 01:20:20 +00:00
Yifan Hong
2277c27fcc Merge "Add compression to dump" 2021-02-08 22:47:05 +00:00
Treehugger Robot
a24aef614a Merge "libsnapshot: Disable the update space swap test on non-VABC." 2021-02-08 21:31:13 +00:00
Treehugger Robot
20bf5a8aff Merge "Add support for compressed snapshot merges in fastboot." 2021-02-08 21:30:11 +00:00
Yifan Hong
3e6b93ca11 Merge "libsnapshot: Communicate compression status to update_engine." 2021-02-08 21:13:10 +00:00
Bowgo Tsai
ec10d3cf6e libfs_avb: verifying vbmeta digest early
We should check FLAGS_VERIFICATION_DISABLED is set or not
after verifying the vbmeta digest against `androidboot.vbmeta.digest`
from bootloader. This is to ensure the /vbmeta content is not
changed since the bootloader has verified it.

We still allow vbmeta digest verification error if the device is
unlocked. Note that this change will introduce a limitation that
the device will not boot if:

  1. The image is signed with FLAGS_VERIFICATION_DISABLED is set
  2. The device state is locked

However, it should not be a concern as we shouldn't boot a locked
device without verification.

Bug: 179452884
Test: build image with BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --flag 2,
      boot the device, then `adb shell touch /metadata/gsi/dsu/avb_enforce`.
      Reboot the device, checks the device does not boot because
      `androidboot.vbmeta.digest` is empty but AVB is enforced.

Change-Id: Id15a25403d16b36d528dc3b8998910807e801ad2
2021-02-08 21:36:15 +08:00
David Anderson
565577f926 Add support for compressed snapshot merges in fastboot.
SnapshotManager::New() is now preferred in recovery. Previously we used
NewForFirstStageMount(), which is technically incorrect as that enables
code paths specifically for first-stage init.

We also explicitly label the snapuserd context, since rootfs in recovery
has unlabelled files.

Finally, we add a timeout to internal calls to
CreateSnapshotsAndLogicalPartitions. Without this, WaitForDevice() calls
will terminate immediately, which breaks VABC given the more complex
device stacking that is created.

Bug: 168258606
Test: fastboot snapshot-update merge
Change-Id: I3a663b95c0b1eabaf14e6fde409c6902653c3c5e
2021-02-04 20:21:18 -08:00
Yo Chiang
df33cb98ef Merge "fs_mgr: Harden adb-remount-test.sh by ERR trap handler" 2021-02-05 03:41:32 +00:00
Yifan Hong
0badb7ca4c Add compression to dump
Add compression flag to SnapshotManager::Dump.

Test: snapshotctl dump

Change-Id: Ie1f6562736d4e83ef7e70d79c761ee42194d7fc5
2021-02-04 18:44:08 -08:00
David Anderson
114a2c0916 libsnapshot: Communicate compression status to update_engine.
Bug: 178732971
Test: manual with statsd_testdrive

Change-Id: If480e5ef1f6fcdde7ccf6f5060f12d021f75b9a1
2021-02-04 18:44:08 -08:00
David Anderson
c68fe7efec libsnapshot: Disable the update space swap test on non-VABC.
Bug: 179111359
Change-Id: I5d435cef51bc3d902cc16a0a12766206907de3ee
2021-02-04 14:09:00 -08:00
Akilesh Kailash
392dbeeb07 Remove invalid error message in the merge path.
Bug: 167409187
Test: Compile snapuserd
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I7a494627b001cb5be20ac6b1bbded64574c3cec0
2021-02-03 18:39:01 +00:00
Treehugger Robot
9e082d4952 Merge "fs_mgr: Harden adb-remount-test.sh on read-only filesystem" 2021-02-03 08:04:00 +00:00
Yo Chiang
f0b8ebc9e9 fs_mgr: Harden adb-remount-test.sh by ERR trap handler
Exit immediately if any unexpected error.

Bug: 165925766
Test: adb-remount-test.sh
Change-Id: I5e9e796f61be21d4b130640949432c1f531de5a4
2021-01-30 00:03:03 +08:00
Yo Chiang
41b6171741 fs_mgr: Harden adb-remount-test.sh on read-only filesystem
I'm getting this error:

  [  FAILED  ] /cache/overlay wipe

If /cache is not mounted, in which case it would be RO and empty, then
don't try to do `rm -rf /cache/overlay` as it would fail.
If /cache is RO and /cache/overlay doesn't exist, then `rm -f` would
fail, albeit the "-f" flag, this is a toybox bug.
Check file path existence before `rm`-ing so we don't trigger this error.

Change `ls <type> <path>` to `test <type> <path>` for robust checking.

Bug: 178683776
Bug: 165925766
Bug: 178771232
Test: adb-remount-test.sh on GSI + Pixel
Change-Id: Ie95de690d96146892127ab11a461d80cb6cb56fa
2021-01-29 14:12:43 +00:00
Tianjie Xu
b9c7459fba Merge "Set hashtree algorithm for verity partitions" 2021-01-28 03:03:21 +00:00
Julien Desprez
1d72b4f363 Ensure compile first to make avbtool work
Test: make host-unit-tests
Bug: 178591423
Change-Id: Ibac59430a1dff69117b16c44787c177501825986
2021-01-27 14:06:50 -08:00
David Anderson
10b755f1d7 libsnapshot: Add a test for when partitions shrink and grow simultaneously.
Bug: 177935716
Test: vts_libsnapshot_test
Change-Id: Ie9415411b8450147d44c6e3b62f413c5aac993cc
2021-01-26 14:48:09 -08:00
Julien Desprez
a28b67e9b6 Merge "Attempt to make libavb test working in CI" 2021-01-26 02:00:03 +00:00
David Anderson
ea1f0fa006 Merge changes I2db0e626,Ic69fc2f5
* changes:
  libsnapshot: Only mount snapshots in MapAllSnapshots().
  libsnapshot: Do not attempt compression features in recovery.
2021-01-26 00:35:58 +00:00
Akilesh Kailash
1d9a7adc67 Merge "libsnapshot: Add unit test to simulate merge-resume" 2021-01-26 00:09:36 +00:00
David Anderson
5283ae4a86 libsnapshot: Only mount snapshots in MapAllSnapshots().
By accident, this was mounting partitions as well, which caused
conflicts in partial updates where some partitions don't have snapshots.

Test: update_device.py with partial OTA
Change-Id: I2db0e6269f0a02cbe8164fa2a72b887c352f56d8
2021-01-25 13:55:59 -08:00
David Anderson
71ab9dbeba libsnapshot: Do not attempt compression features in recovery.
Test: adb sideload
Change-Id: Ic69fc2f5c37a5d4fc0fcd0819ebf58e0249f7773
2021-01-25 13:54:45 -08:00
David Anderson
291a5058cb Merge "libsnapshot: Only enable compression when the corresponding bit is set in DeltaArchiveManifest." 2021-01-25 21:00:40 +00:00
Julien Desprez
f8d9765e0a Attempt to make libavb test working in CI
Test: presubmit
Bug: 178215452
Change-Id: I82d847e05a2fddf3037a8f97793e24a593230ba7
2021-01-25 18:02:12 +00:00
Bowgo Tsai
38de3ba65f Merge "libfs_avb: Fix unit test failures" 2021-01-25 10:13:25 +00:00
Bowgo Tsai
8d84d08da6 Merge "Mark libvbmeta_test as not unit tests" 2021-01-25 08:43:18 +00:00
Bowgo Tsai
708d962795 libfs_avb: Fix unit test failures
There more output lines of `avbtool` now, adjusts the expected
output of `avbtool info_image --image test.img` in the test
cases.

Bug: 178215452
Test: atest libfs_avb_test
Test: atest libfs_avb_internal_test
Change-Id: I924d6d97ef0a4c19c93017c2491bf251dfc51cae
2021-01-25 16:19:03 +08:00
Akilesh Kailash
21afdd4d7e libsnapshot: Add unit test to simulate merge-resume
Simulate merge interruption and merge restart and
validate the data once entire merge is completed.

Bug: 167409187
Test: cow_snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ia940d5fbd2426bdf13347ffb6637d753b2228de6
2021-01-25 01:18:05 +00:00
Julien Desprez
3123fd8930 Mark libvbmeta_test as not unit tests
Test: presubmit
Bug: 177906739
Change-Id: I6b49a0af19a652fbe507df2a3f82502a70bec988
2021-01-22 09:04:42 -08:00
Yo Chiang
d542910553 adb remount -R should enable DSU if suitable
If executing `adb remount -R` and DSU is running but disabled, then
enable the DSU (one-shot mode) so that the reboot afterwards would stay
within the DSU guest system.
Normally reboot within a DSU guest system would bring the device back to
the host system. However when doing adb remount -R, we actually doesn't
want to exit DSU, but wish to reboot back into DSU guest system again
with remount machinery (overlayfs) properly set up.

Also sort the header include order.

Bug: 165925766
Test: Within a DSU guest system, DSU disabled, adb remount -R
  => After reboot, system is DSU and overlayfs is mounted
Test: adb-remount-test.sh within DSU guest system
Change-Id: I72a7a568e985b183d357ae6e1a7d0113e9921200
2021-01-22 22:15:10 +08:00
Yo Chiang
ec8f9dd1f1 adb-remount-test.sh: Filter out administrative mount: securityfs
securityfs /sys/kernel/security securityfs rw,relatime 0 0

is causing the noatime check in adb-remount-test.sh to fail.

Bug: 165925766
Test: Create an aosp_cf_x86_phone-userdebug AVD && adb-remount-test.sh
Test: Use DSU to install GSI on the AVD && adb-remount-test.sh
Change-Id: Ibae0d4bbbbc78fb74f4ad82f2313251598c77f72
2021-01-22 03:15:10 +00:00
David Anderson
3ee2475061 libsnapshot: Only enable compression when the corresponding bit is set in DeltaArchiveManifest.
Test: vts_libsnapshot_test
Change-Id: I43bab4a823d61c56596e1fd5b1522de08b9674f2
2021-01-21 15:15:40 -08:00
David Anderson
4787c1fa28 Merge changes I06043f8e,I872f271c
* changes:
  libsnapshot: Use a two-phase merge for VABC.
  libsnapshot: Pass the correct source device to snapuserd.
2021-01-21 21:10:38 +00:00
David Anderson
531e15e38c Merge "libsnapshot: Removed the unused "linear" optimization." 2021-01-21 20:49:42 +00:00