Commit graph

3707 commits

Author SHA1 Message Date
Daniel Zheng
a1014852a1 libsnapshot: Implement CowWriterV3::EmitCopy
Add copy operation to v3 writer

Test: cow_api_test
Bug: 307452468
Change-Id: I5956084bfadff4536bad11f3683d639408a79892
2023-11-02 14:40:23 -07:00
Daniel Zheng
805d10fa16 libsnapshot: Implement CowWriterV3:EmitBlocks
Add in replace ops to WriterV3 without compression support (will be
added in later). CowReader also has to be changed since we no longer
have a CowFooter (our checks need to be updated correspondingly).
We need next_data_pos_ to be pointed to after the operation buffer. This
section is appended to incrementally as we write replace + xor
operations

Test: cow_api_test
Change-Id: Ie979c72f842edd04337d900fd43dac8031207517
2023-11-02 14:38:37 -07:00
David Anderson
53bd5585e9 libsnapshot: Implement CowWriterV3::EmitzeroBlocks.
Add zero block operation to writer v3. We currently write this operation
to after the scratch space. Once resume point support is added we may
need to change this to write after the resume point. This CL only
supports zero blocks, so writeoperation does not take in data yet.

Bug: 307452468
Test: cow_api_test
Change-Id: I659b2e2e4f6e0d96e374ed29012318cc34b4158d
2023-11-02 13:38:37 -07:00
Daniel Zheng
ee35a8184c libsnapshot: Parser v3 implementation
Also adds a test for CowWriterV3 + CowParserV3.

Bug: 307452468
Test: read a header written by v3 writer
Change-Id: I77cf048604c82a010cfdbfb38d0f8beef597d112
2023-11-02 13:38:37 -07:00
Daniel Zheng
875df3436d libsnapshot: Add Parser v3 + stub
previously upload patch broke build -> bug id: 308868619

This patch updates the unimplemnted pure virtual method data_loc()
Adding in parser v3 plus stub implementation.

Bug: 307452468
Test: th
Change-Id: I815a741b01c46a11a2e60f59305843ce8329a065
2023-11-02 10:20:48 -07:00
David Anderson
e5a343b1af libsnapshot: Add Parser base class
Add in Parser base class and refactor code to work with it. Base class
will have ops() method which returns a vector of v3 operations.
v2_parser translates it's v2 operations to v3 operations with this
method.

Bug: 307452468
Test: Test with critical OTA paths?
Change-Id: I52d0d0554973714189a6e1013d026f96503238b6
2023-11-02 10:19:31 -07:00
Hsiu-Chang Chen
77aa824423 Merge changes from topic "revert-2781730-YHMBOJWFHT" into main
* changes:
  Revert "libsnapshot: Add Parser base class"
  Revert "libsnapshot: Add Parser v3 + stub"
2023-11-02 04:24:16 +00:00
Hsiu-Chang Chen
e2ce495aa2 Revert "libsnapshot: Add Parser base class"
Revert submission 2781730

Reason for revert: It breaks build

Reverted changes: /q/submissionid:2781730

Bug: 308868619
Change-Id: I255263d14a1ced57b80efd373dd22eb22bb6efc1
2023-11-02 03:25:24 +00:00
Hsiu-Chang Chen
ad37351bd1 Revert "libsnapshot: Add Parser v3 + stub"
Revert submission 2781730

Reason for revert: It breaks build

Reverted changes: /q/submissionid:2781730

Bug: 308868619
Change-Id: I53b6b619635f150455cdba00237d130cae5df347
2023-11-02 03:25:24 +00:00
Daniel Zheng
a4c2d78763 Merge changes Ie01f94d0,Iccc9755a into main
* changes:
  libsnapshot: Add Parser v3 + stub
  libsnapshot: Add Parser base class
2023-11-02 01:05:42 +00:00
Daniel Zheng
4ca81f26d8 libsnapshot: Add Parser v3 + stub
Adding in parser v3 plus stub implementation.

Bug: 307452468
Test: th
Change-Id: Ie01f94d036c2062e34dbdf32456dc0a4612492d3
2023-11-01 13:06:55 -07:00
David Anderson
c165e84b15 libsnapshot: Add Parser base class
Add in Parser base class and refactor code to work with it. Base class
will have ops() method which returns a vector of v3 operations.
v2_parser translates it's v2 operations to v3 operations with this
method.

Bug: 307452468
Test: Test with critical OTA paths?
Change-Id: Iccc9755a892911a1638f5b62d7d6a2402c68ab20
2023-11-01 13:06:54 -07:00
Akilesh Kailash
8fb49bc0a2 libsnapshot: Wait for COW path
After mapping COW device, wait for the device to appear
before initializing it.

Bug: 305829996
Test: OTA on Pixel
Change-Id: If9e0e1ce684aa15bc9f3fd5a01e4f0421de87e81
Signed-off-by: Akilesh Kailash <akailash@google.com>
2023-10-30 12:14:15 -07:00
Kiyoung Kim
260e48f07e Deprecate Vendor Overlay when VNDK is deprecated
Vendor overlay was introduced to enable system-only update with small
changes on the vendor, but this is no longer required with same reason
of VNDK deprecation - more frequent Vendor update than expected, and
makes system-vendor interface unstable. This change adds more comments
to explain that Vendor overlay will be deprecated along with VNDK, and
skip test if VNDK is deprecated.

Bug: 307925435
Bug: 307902290
Test: fs_mgr_vendor_overlay_test passed with CF trunk-staging device
Change-Id: I7b9359a5754e8740e749c48f6265a1b0f92f13af
2023-10-30 13:19:12 +09:00
Treehugger Robot
54f2e06cf5 Merge "vts_fs_test: Do not check /metadata for automotive" into android14-tests-dev am: 76a9eb4374
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2800753

Change-Id: I403f0915937a0c8b24a54b5b265b70b1ce529ff2
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-10-26 21:26:48 +00:00
Christopher Ferris
268c408310 Merge "Fix indices in fuzzer." into main 2023-10-26 18:37:06 +00:00
Christopher Ferris
14e997fe43 Fix indices in fuzzer.
There are a few places where an index is randomly generated, but the
max is incorrectly set to the size of the value not one less than
the size.

All of these occurrences have been fixed.

Bug: 306230574

Test: Ran fuzzer instance that failed, no longer fails.
Change-Id: I20dfbc19a0df9cb160f8c861e072f083e24a4fab
2023-10-24 14:38:49 -07:00
Yi-Yo Chiang
50dca4a77b remount: If checkpoint is in progress, wait for it to complete
Instead of having the user to retry the `remount` command until success
(manual poll). Just block the remount command until checkpoint is
complete, and then continue.

Bug: 252989722
Test: adb remount on a cold device, remount command would block for a
  few dozens of seconds and then continue.
Test: adb remount on a warm device, remount command would process
  immediately. Do not show "force end checkpointing" instructions.
Change-Id: I65f3a5ade4c9538a55892345c75b45ac3a1755fd
2023-10-24 13:15:03 +00:00
Philip Chen
35817eeb55 vts_fs_test: Do not check /metadata for automotive
This aligns with GAS req 4.0.

Bug: 307215040
Test: build
Change-Id: I2115e147d484d066f892ea50eeca626e64beebfc
2023-10-23 21:43:23 +00:00
David Anderson
d1e286f12f Merge "Refactor off COW header v3" into main 2023-10-23 18:25:46 +00:00
Yi-Yo Chiang
05d3adb713 fs_mgr_overlayfs: Remove temp dir under /dev when aborting
When aborting from fs_mgr_overlayfs_mount, we should remove any temp dir
we created under /dev before aborting the program.

Bug: 306124139
Test: adb-remount-test
Change-Id: I6debf84935e39d884f2f5b3e9720fdfd4332465c
2023-10-23 07:57:28 +00:00
Yi-Yo Chiang
c78df87fad fs_mgr_overlayfs: Support remounting submounts
The move mount motions logic are built on top of heuristics and allow
non-fatal failures.
If the overlay was setup and mounted without reboot (this could happen
if AVB is disabled), the mount state (especially the MS_SHARED and
MS_PRIVATE flags) can still get skewed somehow, due to unforeseen errors
or resource race.
It is always advised to reboot after initial overlay setup, as the
overlay mount logic is less likely to require moving submounts when
executed by `init`, this provides the greatest chance of success.

Below is an example of the expected outcome of remounting submounts:
[precondition]
* AVB is already disabled, so overlay is setup and mounted w/o reboot
* No existing overlay; is initial overlay setup

0. /proc/self/mountinfo would contain something like:
<id> <parent id> <mountpoint> <shared/private> <filesystem>
2    1           /product     shared:2         erofs
3    2           /product/app shared:3         erofs

1. adb remount /product/app
(note how the mount flag of <3> changes to private so <30> can be
MS_MOVE later)
2    1           /product     shared:2         erofs
3    2           /product/app private          erofs
30   3           /product/app shared:30        overlay

2. adb remount /product
(note how the parent of <30> changes to <40> as the result of MS_MOVE)
(note that <3> is _not_ moved)
2    1           /product     private          erofs
3    2           /product/app private          erofs
30   40          /product/app shared:30        overlay
40   2           /product     shared:40        overlay

Bug: 306124139
Test: adb-remount-test
Test: Verified with a remount submount scenario by editing the fstab.
Test: remount /system/bin then /system and verify the submount overlay
  (/system/bin) is moved under the parent mount overlay (/system).
Change-Id: I6cdbe8c52d826a6f03fd363c909ebb0005446b96
2023-10-23 07:57:28 +00:00
Yi-Yo Chiang
f2e7170cba fs_mgr_overlayfs: Don't try to mount scratch if already mounted
This eliminates spammy logs.

Bug: 306124139
Test: adb remount -v && check the output logs
Change-Id: I3143f5865b5292ba581009179c7e7141afa8fda5
2023-10-20 10:19:52 +00:00
Yi-Yo Chiang
aba8c1ee36 Merge "fs_mgr_overlayfs: Setup remount scratch dir with encoded mountpoint name" into main 2023-10-19 04:25:51 +00:00
Yi-Yo Chiang
9032b39ea5 fs_mgr_overlayfs: Setup remount scratch dir with encoded mountpoint name
Instead of using the basename(mounpoint) as partition scratch dir name,
which could lead to name collision, use the normalized and encoded
mountpoint as scratch dir name.

Bug: 243503963
Bug: 306124139
Test: adb-remount-test
Change-Id: I5a64f17bc3b88f0ce42bd0c5779c8dd23a07917f
2023-10-18 19:41:46 +08:00
Daniel Zheng
a16e0ba64e Merge "V3 writer header" into main 2023-10-18 08:41:20 +00:00
Treehugger Robot
e9741b25e5 Merge changes I9830baf7,I01da0a68,I05d0a6c7 into main
* changes:
  Added liblp_apis_fuzzer
  Added TestPartitionOpener_group
  Added liblp_builder_fuzzer and liblp_super_layout_builder_fuzzer
2023-10-18 04:58:25 +00:00
Daniel Zheng
7a9e007a01 Refactor off COW header v3
Cow reader will store header v3 in memory. ReadCowHeader can remain
mostly unchanged since the cow prefix is the same.
header->prefix.header_size will then tell us if we're reading a v3 or v2
header. v3 header is strictly a superset of v2 header so we can read a
v2 header into a v3 struct.

Added a test case to test_v3 where we write a header using v2_writer and
ensure CowReader is able to read it.

Test: cow_api_test
Change-Id: I142f18d871322930b7dc341c342c8b63a481341c
2023-10-17 16:45:06 -07:00
Daniel Zheng
f2270d2bb6 V3 writer header
V3 writer set up header. Anything related to cluster ops can be set to
zero, as V3 cow format will no longer support this. All footer related
code can also be dropped as the footer isn't being used.

Test: cow_api_test
Change-Id: I39ece8eaa0813c5991d53b5e0ff9271f27e0662a
2023-10-17 14:59:49 -07:00
Daniel Zheng
f8c50df1c7 Remove v2_writer lseek call
Removing lseek call as writefully() should be sufficient in ensuring our
fd is moved to the correct spot

Test: cow_api_test
Change-Id: I404b192335ab1672ed61de426e17e63ab9d51ece
2023-10-17 14:59:49 -07:00
Daniel Zheng
94a732e026 Remove duplicate parser v2 code
Remove a duplicate check

Test: th
Change-Id: Ia9c3c8ff1ee524005f95aa57dc3d6da9641ce2e6
2023-10-17 14:59:49 -07:00
Daniel Zheng
f0a28dae28 Add write_cow functionality
Adding gflags + some basic functionality to this binary. We should be
able to use this to write v2 or v3 cow, and then use inspect_cow to
inspect the contents

Test: write_cow
Change-Id: I44bf5a8c807c4c4759e54624c51e966f2ea18ce9
2023-10-17 12:47:19 -07:00
Daniel Zheng
06c086dd73 Rename basic_v2_writer
Gonna add to this binary options to write v2 + v3 cow, + some additional
configurations (for testing purposes). I think this binary can serve as
something to use in parallel with inspect_cow

Test: m write_cow
Change-Id: I93c703b246baa5b400044436a94b33a32e03ea68
2023-10-17 09:39:40 -07:00
Akilesh Kailash
57b34ae34c Merge "libsnapshot: Remove consistency check after merge" into main 2023-10-17 01:31:07 +00:00
David Anderson
f7d07db679 snapuserd: Test both async and sync paths.
Rather than split this into two binaries as we did for libsnapshot, this
uses test parameterization. I went this route because even for async
builds, the sync path must always work as a fallback.

Bug: N/A
Test: snapuserd_test
Change-Id: Ib68c8fd831c345f41a4e14a99fd0a0b8b1d47b6d
2023-10-16 20:23:46 +00:00
Akilesh Kailash
e02b831aef libsnapshot: Remove consistency check after merge
Merge thread will mark merge-completion after msync on header is
complete. This should be definitive enough to track the completion
status.

Bug: 305187301
Test: OTA on Pixel
Change-Id: I366dc5052fa91a6eacf394a1970200cdebc0e135
Signed-off-by: Akilesh Kailash <akailash@google.com>
2023-10-16 11:32:54 -07:00
Daniel Zheng
452f176ca8 Merge "Add v3 to CreateCowWriter" into main 2023-10-16 15:20:47 +00:00
Yi-Yo Chiang
c0281978b0 Merge "fs_mgr: Don't run clean_scratch_files on non-dynamic devices" into main 2023-10-16 10:24:55 +00:00
Kelvin Zhang
9c949326bd Merge "Make partition metadata write atomic" into main 2023-10-14 00:01:41 +00:00
Kelvin Zhang
78d5352289 Make partition metadata write atomic
We have seen multiple OTA failures with invalid geometry magic
signature. Make partition metadata write atomic by writing to a tmpfile
first and then do a rename.

Test: th
Bug: 303770065
Bug: 298149189
Change-Id: Id1d565de73439b95b665144c2f02fc97273d341c
2023-10-13 15:46:28 -07:00
David Anderson
7e407514dd Remove unused variable in snapuserd_core.
Bug: N/A
Test: builds
Change-Id: I8fc3bd2dd8eb3cfa3c6bfe513f054a4d6a674de1
2023-10-13 21:34:12 +00:00
Daniel Zheng
cab8646a07 Add v3 to CreateCowWriter
Adding v3 writer as a return type to CreateCowWriter

Test: write_cow
Change-Id: Ia68ddf17b9f5f9fae27da4d6ab5fb150078d0c38
2023-10-12 14:10:50 -07:00
Kelvin Zhang
d6fe3032a9 Merge "Revert "Shove type into source_info"" into main 2023-10-11 20:50:02 +00:00
Kelvin Zhang
b6153adde1 Revert "Shove type into source_info"
This reverts commit e90a39aee2.

Reason for revert: 304602386

Change-Id: Ib9d31bdadc7511e67f4c7e5731d18e7bf0162bfc
2023-10-11 19:44:38 +00:00
Nolen Johnson
cb33fb4008 fs_mgr: Don't run clean_scratch_files on non-dynamic devices
* This results in a metric ton of denials on some devices and
  eats up valuable resources on boot, plus there's 0 need for it,
  so kill it.

Change-Id: Ic52d5b3f06724430e9505345024cf0041b37ca49
2023-10-11 13:58:40 -04:00
Edward Liaw
da3caeab43 Merge "liblp_test: Add to kernel-presubmit" into main 2023-10-11 16:53:26 +00:00
Edward Liaw
f5736cb42a Merge "libdm_test: Add to kernel-presubmit" into main 2023-10-11 16:53:10 +00:00
Edward Liaw
eafffc0c9c liblp_test: Add to kernel-presubmit
Bug: 284307085
Test: atest :kernel-presubmit
Change-Id: I2a138cf55a2af07b5c231f163196266f8b3dd524
Signed-off-by: Edward Liaw <edliaw@google.com>
2023-10-10 23:13:30 +00:00
Edward Liaw
942a31e7be libdm_test: Add to kernel-presubmit
Bug: 284307085
Test: atest :kernel-presubmit
Change-Id: I13ae25f17c943825bdf5ee67c00b7d297cf6e319
Signed-off-by: Edward Liaw <edliaw@google.com>
2023-10-10 23:05:09 +00:00
Daniel Zheng
bb45742ba1 Move Cow Header child writer
v2 writer and v3 writer will write different versions of the header.
It's better to have each writer hold it's own header.

Test: th
Change-Id: Ibe310d58b830950ad556aabcd0c1009483fc8d86
2023-10-10 15:06:26 -07:00