Commit graph

73431 commits

Author SHA1 Message Date
Keith Mok
7e87807d16 fuzzy_fastboot: Add malformed download command test
Add a malformed download command test.
And add a sparse file test with very large block size

Bug: 215236564
Test: bootloader fastboot
Change-Id: I1072ba189ac15b2e1eb8f13ffd754f93c967e2d5
2022-01-19 00:02:51 +00:00
Shikha Panwar
f6017f8508 Merge "Make libstatspull_bindgen available to virt apex." 2022-01-17 09:34:30 +00:00
Elliott Hughes
9453199683 Merge "debuggerd: add the PAC keys to the tombstones." 2022-01-14 18:52:01 +00:00
Shikha Panwar
03ac66e640 Make libstatspull_bindgen available to virt apex.
The library is required to log the atoms from virt apex. See bug for more details
Test: m succeeded. The whole topic is tested with statsd_testdrive 409
as mentioned in go/westworld-create-atom#step-3-test-your-atom

Change-Id: If8b13c9d1878265bfcb8e09fc1bd8e78e968f71f
2022-01-14 10:29:51 +00:00
Keith Mok
4915f04165 Merge "Add checking for sparse file format" 2022-01-14 00:38:35 +00:00
Keith Mok
a3b72067de Add checking for sparse file format
Sparse file can come from an untrusted source.
Need more checking to ensure that it is not a malformed
file and would not cause any OOB read access.

Update fuzz test for decoding also.

Test: adb reboot fastboot
      fuzzy_fastboot --gtest_filter=Fuzz.Sparse*
      fuzzy_fastboot --gtest_filter=Conformance.Sparse*
      sparse_fuzzer
Bug: 212705418
Change-Id: I7622df307bb00e59faaba8bb2c67cb474cffed8e
2022-01-14 00:37:48 +00:00
Christopher Ferris
c54ddc2b72 Merge "Update for MapInfo objects in frame data." 2022-01-14 00:15:19 +00:00
Akilesh Kailash
3991ce6c30 Merge changes from topic "snapuserd-async-merge"
* changes:
  Add a new property to disable io_uring and run vts and snapuserd_test
  snapuserd: Async I/O for block verification
  snapuserd: Use io_uring api's for snapshot merge
2022-01-14 00:06:23 +00:00
Akilesh Kailash
91ff0285fd Merge "libsnapshot: Terminate snapuserd daemon after merge completion" 2022-01-14 00:06:04 +00:00
Akilesh Kailash
0616cb17c2 libsnapshot: Terminate snapuserd daemon after merge completion
Bug: 214340811
Test: OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Icaed87feb7760074d743229856f23f3599e03c62
2022-01-13 23:51:02 +00:00
Colin Cross
6a79e8113a Merge "Use 0 instead of NULL for comparison to char16_t" 2022-01-13 23:29:42 +00:00
Elliott Hughes
d13ea523e1 debuggerd: add the PAC keys to the tombstones.
Hard to get otherwise if you're trying to debug PAC issues.

Bug: http://b/214314197
Test: treehugger
Change-Id: I2e5502809f84579bf287364e59d6e7ff67770919
2022-01-13 15:03:19 -08:00
Colin Cross
02a3f958db Use 0 instead of NULL for comparison to char16_t
musl libc defines NULL as nullptr, which is explicitly allowed by
C++11.  nullptr_t cannot be implicitly cast to an integral type.
Use 0 instead.

Bug: 190084016
Test: m USE_HOST_MUSL=true host-native
Change-Id: I0c3b6c94cd69262f574414bf52494333f2f2645a
2022-01-13 13:41:59 -08:00
Akilesh Kailash
401a46a597 Add a new property to disable io_uring and run
vts and snapuserd_test

This should be run on cuttlefish

Bug: 202784286
Test: vts_libnspahost_test, snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I2c28e98f04beca770b8a6efa9474d602fe26f514
2022-01-13 07:18:00 +00:00
Akilesh Kailash
e88af0f6ba snapuserd: Async I/O for block verification
Boot time improvements:

==================================

Incremental OTA of 300M between two git_master branches on Pixel 6:

Android S (with dm-snapshot):

BootComplete                  : 39.379 seconds

Android T (with io_uring):

BootComplete                  : 32.001 seconds

Time taken for each partition:

snapuserd: ReadBlockAsync complete: 2055 ms Block-device: /dev/block/dm-21 Partition-name: system_ext Size: 399302656
snapuserd: ReadBlockAsync complete: 2956 ms Block-device: /dev/block/dm-23 Partition-name: vendor Size: 650084352
snapuserd: ReadBlockAsync complete: 3534 ms Block-device: /dev/block/dm-20 Partition-name: system Size: 859746304
snapuserd: ReadBlockAsync complete: 7808 ms Block-device: /dev/block/dm-22 Partition-name: product Size: 3030687744

====================================

Bug: 202784286
Test: Full/Incremental OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I615f9f4fde4e565aa1d611a2d6bbf6a6f62fa3f1
2022-01-13 07:17:15 +00:00
Akilesh Kailash
3596046590 snapuserd: Use io_uring api's for snapshot merge
using io_uring READ/WRITE opcodes for snapshot merge.
Specifically, it is used only for readahead and ordered ops
code path.

Snapshot merge perf:

===========================================================

Incremental OTA of 300M between two git_master branches on Pixel 6:

===========================================================

On Android S (with dm-snapshot): ~15 minutes:

update_engine: [INFO:cleanup_previous_update_action.cc(330)] Merge finished with state MergeCompleted.
update_engine: [INFO:cleanup_previous_update_action.cc(130)] Stopping/suspending/completing CleanupPreviousUpdateAction
update_engine: [INFO:cleanup_previous_update_action.cc(501)] Reporting merge stats: MergeCompleted in 926508ms (resumed 0 times), using 0 bytes of COW image.

===========================================================

On Android T (with io_uring: ~38 seconds):

update_engine: [INFO:cleanup_previous_update_action.cc(330)] Merge finished with state MergeCompleted.
update_engine: [INFO:cleanup_previous_update_action.cc(130)] Stopping/suspending/completing CleanupPreviousUpdateAction
update_engine: [INFO:cleanup_previous_update_action.cc(501)] Reporting merge stats: MergeCompleted in 38868ms (resumed 0 times), using 0 bytes of COW image.

===========================================================

Bug: 202784286
Test: Full/Incremental OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I24ed3ae16730d0a18be0350c162dc67e1a7b74e1
2022-01-13 06:31:49 +00:00
Jiyong Park
e060580c31 Merge changes from topic "or_return"
* changes:
  Sort exported headers of libutils
  OR_RETURN supports status_t
2022-01-12 07:27:41 +00:00
Treehugger Robot
da0756b575 Merge "healthd: remove hridya@ from OWNERS file" 2022-01-12 02:34:11 +00:00
Jiyong Park
8b196a7336 Sort exported headers of libutils
Bug: N/A
Test: build

Change-Id: I658155371ca32a7e9ba7e82f8e43780c515df77b
2022-01-12 09:23:46 +09:00
Jiyong Park
705abe2d30 OR_RETURN supports status_t
This change provide a specialization of android::base::OkOrFail for
status_t. As a result, a statement whose type is status_t can be used
with OR_RETURN.

The specialization also provides conversion operators to Result<T,
StatusT> where StatusT is a wrapper type for status_t. This allows
OR_RETURN macro to be used in newer functions that returns Result<T,
StatusT>.

Example usage:

\#include <utils/ErrorsMacros.h>

status_t legacy_inner();

status_t legacy_outer() {
  OR_RETURN(legacy_inner());
  return OK;
}

Result<T, StatusT> new_outer() {
  OR_RETURN(legacy_inner()); // the same macro
  return T{...};
}

Bug: 209929099
Test: atest libutils_test
Change-Id: I0def0e84ce3f0c4ff6d508c202bd51902dfc9618
2022-01-12 09:20:27 +09:00
Ken Chen
67b528f521 Merge "Delete legacy library libqtaguid" 2022-01-11 17:30:41 +00:00
Yi-yo Chiang
1eeee3db1a Merge "Remove unused rootdir/avb/*-gsi.avbpubkey modules" 2022-01-11 06:57:25 +00:00
Yi-yo Chiang
c126ad8214 Merge "libfs_avb: Export more utilities to facilitate VTS" 2022-01-11 05:36:25 +00:00
Bowgo Tsai
f452a42db2 Merge "libfs_avb: updating run_tests.sh" 2022-01-11 04:01:39 +00:00
Treehugger Robot
0b5bfdce7d Merge "fastboot: make init_boot.img optional" 2022-01-11 03:54:31 +00:00
Jaegeuk Kim
8b7e76962e Merge "Allow to run fsck.f2fs in first_stage_ramdisk" 2022-01-11 02:30:14 +00:00
Devin Moore
60bd33ffd5 fastboot: make init_boot.img optional
Older devices don't build this image because they don't have a partition
for it.

Bug: N/A
Test: m
Change-Id: I56b58c1bc137ce25e426aa4ec6d2303fa7d77ccc
2022-01-11 02:02:59 +00:00
Treehugger Robot
e0fa2905d0 Merge changes from topic "adb-remount-f2fs"
* changes:
  Mount /mnt/scracth with -o sync,nodiscard
  Run check_fs only /data is mounted
  Don't use FSCK_LOG_FILE used for ext4 only
  Don't try to mount if the disk has no FS magic
  Retry to unmount /data
2022-01-11 00:27:35 +00:00
Pirama Arumuga Nainar
f9a54a86ec Merge "Enable continuous coverage: use '%c' specifier in LLVM_PROFILE_FILE" 2022-01-10 22:36:57 +00:00
Jaegeuk Kim
b783310e71 Allow to run fsck.f2fs in first_stage_ramdisk
[   23.065933][    T1] init: [libfs_mgr]Running /system/bin/fsck.f2fs -a -c 10000 --debug-cache /dev/block/sda8
[   23.067470][    T1] logwrapper: Cannot log to file /dev/fscklogs/log
[   23.067829][    T1] logwrapper:
[   23.068997][  T332] logwrapper: executing /system/bin/fsck.f2fs failed: Permission denied
[   23.069759][    T1] fsck.f2fs: executing /system/bin/fsck.f2fs failed: Permission denied
[   23.071659][  T332] logwrapper:
[   23.083283][    T1] fsck.f2fs: fsck.f2fs terminated by exit(255)
[   23.083283][    T1]
[   23.111166][    T1] F2FS-fs (sda8): Found nat_bits in checkpoint
[   23.121242][    T1] F2FS-fs (sda8): Mounted with checkpoint version = 6a65cb64
[   23.121831][    T1] init: [libfs_mgr]__mount(source=/dev/block/platform/14700000.ufs/by-name/metadata,target=/metadata,type=f2fs)=0: Success

Bug: 210589189
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: I720e2aa4b1ab845af96610cd0d9c6e3c9b4cae03
2022-01-10 21:35:21 +00:00
Jaegeuk Kim
bbad87aeee Mount /mnt/scracth with -o sync,nodiscard
"-o discard" issues UNMAP commands to loopback resulting in punch_hole
on the pinned file in /data. That will break the pinned block map.

Bug: 210589189
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: Ia927c43fc75164ce5929173f5740737eac4de484
2022-01-10 21:34:00 +00:00
Jaegeuk Kim
06db803f32 Run check_fs only /data is mounted
After switching to /system from ramdisk, first_stage_init has no required
libraries for check_fs.

[   20.838811][    T1] init: [libfs_mgr]Created logical partition scratch on device /dev/block/dm-6
[   20.972704][    T1] init: [libfs_mgr]Running /system/bin/fsck.f2fs -a -c 10000 --debug-cache /dev/block/dm-6
[   20.977879][  T345] logwrapper: executing /system/bin/fsck.f2fs failed: No such file or directory
[   20.978470][    T1] fsck.f2fs: executing /system/bin/fsck.f2fs failed: No such file or directory
[   20.981137][    T1] fsck.f2fs: fsck.f2fs terminated by exit(255)
[   21.002958][    T1] init: [libfs_mgr]__mount(source=/dev/block/dm-6,target=/mnt/scratch,type=f2fs)=0: Success
[   21.017748][    T1] init: [libfs_mgr]umount(/mnt/scratch)
[   21.021028][    T1] init: [libfs_mgr]Running /system/bin/fsck.f2fs -a -c 10000 --debug-cache /dev/block/dm-6
[   21.028759][  T347] logwrapper: executing /system/bin/fsck.f2fs failed: No such file or directory
[   21.028793][    T1] fsck.f2fs: executing /system/bin/fsck.f2fs failed: No such file or directory
[   21.049100][    T1] fsck.f2fs: fsck.f2fs terminated by exit(255)
[   21.068101][    T1] init: [libfs_mgr]__mount(source=/dev/block/dm-6,target=/mnt/scratch,type=f2fs)=0: Success

Bug: 210589189
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: Ie41cba4e7553860fdb48996d9b58a34093f0b723
2022-01-10 21:33:50 +00:00
Jaegeuk Kim
d595de5139 Don't use FSCK_LOG_FILE used for ext4 only
Bug: 210589189
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: Iaf7a61c6c9c6cd557f953b1a665e7a1640d357ae
2022-01-10 21:31:46 +00:00
Treehugger Robot
afd4ccd70b Merge "Remove win_sdk" 2022-01-10 20:18:16 +00:00
Jaegeuk Kim
9547726f4a Don't try to mount if the disk has no FS magic
This avoids:
[   22.932529][    T1] init: [libfs_mgr]Invalid f2fs superblock on '/dev/block/platform/14700000.ufs/by-name/metadata'
[   22.934609][    T1] F2FS-fs (sda8): Magic Mismatch, valid(0xf2f52010) - read(0x0)
[   22.935061][    T1] F2FS-fs (sda8): Can't find valid F2FS filesystem in 1th superblock
[   22.937306][    T1] F2FS-fs (sda8): Magic Mismatch, valid(0xf2f52010) - read(0x0)
[   22.943700][    T1] F2FS-fs (sda8): Can't find valid F2FS filesystem in 2th superblock
[   22.951937][    T1] init: [libfs_mgr]__mount(source=/dev/block/platform/14700000.ufs/by-name/metadata,target=/metadata,type=f2fs)=-1: Invalid argument

Bug: 210589189
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: I27989b25769eae83eb06ac86146f27baf288b7e1
2022-01-10 18:02:03 +00:00
Jaegeuk Kim
bdb981d113 Retry to unmount /data
If we fail to umount /data, device won't boot up at all.

Bug: 208161227
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: I92d34a253039eb677d4df9fad8a0821fbc684f57
2022-01-10 09:50:23 -08:00
Yi-Yo Chiang
0c99f93351 Remove unused rootdir/avb/*-gsi.avbpubkey modules
They were moved to test/vts-testcase

Bug: 176869478
Test: presubmit
Change-Id: I4f4f1c50e624125c839161d83e28f051099e6ec0
2022-01-10 14:25:42 +00:00
Yi-Yo Chiang
d543bebbf7 libfs_avb: Export more utilities to facilitate VTS
Change should be no-op, it just export some private methods.

Bug: 211741246
Test: ./run_tests.sh
Test: presubmit
Change-Id: Ia4ac739018f188a0154625ec574fc4bebf7c62ee
2022-01-10 16:36:51 +08:00
Bowgo Tsai
68c97406a2 libfs_avb: updating run_tests.sh
Adding `--host` for host-side test cases with atest, so we
still can run the test when there is no device connected.

Otherwise, TradeFed will emit the follow error:
  TradeFed subprocess exited early with exit code=7(NO_DEVICE_ALLOCATED).

Bug: None
Test: ./system/core/fs_mgr/libfs_avb/run_tests.sh
Change-Id: I0a593e6dc42861c2dc9cfff308cd7fd76b39ef2c
2022-01-10 16:26:58 +08:00
Yi-yo Chiang
28d7b767a9 Merge "Move *-gsi_avbpubkey filegroup to vts directory" 2022-01-10 08:19:43 +00:00
Cole Faust
eea1dadba8 Remove win_sdk
win_sdk is deprecated, if given on the command line
sdk will be build instead.

Bug: 212724080
Test: Presubmits
Change-Id: Ia92b879fe08f0a1d9ebefc90e369adf1d4aa4112
2022-01-07 14:55:54 -08:00
Devin Moore
f56e0dbf33 Merge "fastboot: make copy_boot_avb_footer more generic" 2022-01-07 22:13:19 +00:00
Treehugger Robot
e664e8fb72 Merge "Use insert() with ranges instead of reserve" 2022-01-07 20:25:00 +00:00
Devin Moore
cb5098f547 fastboot: make copy_boot_avb_footer more generic
The funciton is now used for init_boot and boot partitions, so the name
and the error messages should be more generic.

Bug: 203698939
Test: m
Change-Id: I168fdcb08accb11b2c9cfc402168744274f59665
2022-01-07 18:03:33 +00:00
Kelvin Zhang
500ed210e0 Use insert() with ranges instead of reserve
STL implementation of insert() is generally smart enough to reserve
storage before hand. No need to do reserve() followed by for each loop

Test: th
Change-Id: I36e88c121868b97ea525b292a67b36983c530e56
2022-01-07 09:55:38 -08:00
Devin Moore
12e5f038a9 Merge changes Ib5a63a7b,If00f581a
* changes:
  fastboot: copy AVB footer from init_boot.img to end of partition
  fastboot: Support flashing init_boot.img into init_boot partition
2022-01-07 16:24:13 +00:00
Hridya Valsaraju
cecdf78a1f Merge "fastboot: Add elsk@ to OWNERS file" 2022-01-07 13:15:41 +00:00
Yi-Yo Chiang
88e0f4abdb Move *-gsi_avbpubkey filegroup to vts directory
Bug: 176869478
Test: presubmit
Change-Id: I41cf85bb2f3b5270fcb848fc07e9145d26adcc18
2022-01-07 18:27:15 +08:00
Ken Chen
c3904fd372 Delete legacy library libqtaguid
The library has been replaced by BPF solution. No any valid user is
using this.

Bug: N/A
Test: m
Change-Id: If14ff29859b06feb5cd7d4ebed9f83ab146ea9e4
2022-01-07 18:26:25 +08:00
Treehugger Robot
836eed37fb Merge "healthd: remove hridya@ from OWNERS file" 2022-01-07 03:13:44 +00:00