Commit graph

1248 commits

Author SHA1 Message Date
Treehugger Robot
66b0c31f60 Merge "Increase adb-remount timeout" 2019-04-08 15:13:47 +00:00
Treehugger Robot
8e5bd5a247 Merge "Update GSI keys for dynamic system image" 2019-04-08 13:41:47 +00:00
Yifan Hong
a482da1d76 Allow to build non-sparse super image.
- Rename SparseBuilder to ImageBuilder, WriteSplitSparseFiles
  to WriteSplitImageFiles, WriteToSparseFile to WriteToImageFile, etc.
- Add sparsify argument to ImageBuilder, WriteToImageFile, and
  WriteSplitImageFiles

This allows lpmake to write non-sparse super image directly. Virtual
devices needs non-sparse images to work on.

Test: lpmake without --sparse
Test: liblp_test
Bug: 120041578

Change-Id: I76ee09efd02df7caaa8abd37a55ae0bebf9cfa29
2019-04-04 11:33:38 -07:00
Treehugger Robot
73bb342956 Merge "libfiemap_writer: test pinned file bit" 2019-04-03 21:31:18 +00:00
Julien Desprez
c25a9da325 Increase adb-remount timeout
Unblocked the previous error we are now reaching the
default 5min timeout which is too short for this test.
Increase it to 1h.

Test: None
Bug: 117670584
Change-Id: I7fe40e54a7fb978392ee919226c0f05413e7349e
2019-04-03 08:27:29 -07:00
Bowgo Tsai
b9b9a68f49 Update GSI keys for dynamic system image
Bug: 112293933
Test: tree hugger
Change-Id: Ia5fd08410fd680b8fd3676cfc4daae1caf1ff425
2019-04-03 17:54:01 +08:00
Treehugger Robot
cd68bda4ec Merge "Replace reboot-fastboot by reboot fastboot" 2019-04-03 08:24:34 +00:00
Jaegeuk Kim
a9b3c3d605 libfiemap_writer: test pinned file bit
The pinned file should have NOCOW_FL flag.

Bug: 129803146
Change-Id: Idaf5bdbbdcce1c3805cd16ddb0c1c7f1df87dcfb
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2019-04-03 04:35:33 +00:00
Julien Desprez
74239ea1b1 Replace reboot-fastboot by reboot fastboot
Older host adb client do not recognize reboot-fastboot,
switch it for "adb reboot fastboot" which should be the
same.

Test: None
Bug: 117670584
Change-Id: Iec5230ca66ec18fe7d7c0ebd3f9ab9596a6e7b3c
2019-04-02 17:04:39 -07:00
Treehugger Robot
2b8e96bd64 Merge "fs_mgr: remount: system is root" 2019-04-02 21:54:34 +00:00
Mark Salyzyn
b18aba0a9f fs_mgr: remount: system is root
For devices without overlayfs and system is root (Hikey, Hikey960)
correct "/system" to "/" if that is the mount point.

Test: adb-remount-test.sh
Bug: 129720614
Change-Id: I3bdbd5ea7b70d03477055e60521d8132b486c904
2019-04-02 21:22:27 +00:00
David Anderson
97d1fccd5b Merge "Fix device mapper name clashes on non-A/B devices." 2019-04-02 20:16:36 +00:00
Julien Desprez
61dea3d154 Add the basic config to run through atest
Add basic config that can run through atest the remount
script.
Very first step before being able to run in infra.

Test: atest adb-remount-sh
Bug: 117670584
Change-Id: I399f79fb7d7cd1b8a832be23efb3b625be693f7e
2019-03-28 10:21:02 -07:00
David Anderson
5aa37dc3dc Merge "init: Support booting off GSIs installed to non-userdata block devices." 2019-03-28 09:46:44 +00:00
Yifan Hong
f959fffc1c Merge changes from topic "lpdumpd"
* changes:
  libbase: realpath is wrapped with TEMP_FAILURE_RETRY
  liblp: Replace open with GetControlFileOrOpen
  init: expand prop in 'file'
  libcutils: android_get_control_file uses realpath.
2019-03-26 20:35:36 +00:00
Nicolas Geoffray
7f03e4eea1 Allow makefiles to be parsed in a trimmed down manifest.
Test: m
Change-Id: Id0e52eacc8e8b5fa75af2050a72be6a8bdadc984
Exempt-From-Owner-Approval: Unbreaks ART team
2019-03-25 09:14:33 +00:00
Treehugger Robot
dddf40a7f5 Merge changes from topic "b/129002747"
* changes:
  Make VTS run libdm_test
  Create a VTS test from liblp_test
2019-03-25 05:09:36 +00:00
Hridya Valsaraju
1a9a02c9a6 Make VTS run libdm_test
Bug: 129002747
Test: vts-tradefed run commandAndExit vts-kernel --primary-abi-only --module VtsKernelLibdmTest

Change-Id: If0fb06bde561ae839d3aff96a73eb395082b4653
2019-03-22 15:11:49 -07:00
Hridya Valsaraju
4e3b6332ab Create a VTS test from liblp_test
Bug: 129002747
Test: vts-tradefed run commandAndExit vts-kernel --primary-abi-only --module VtsKernelLiblpTest

Change-Id: I6497428ec1ce93f4c1814c3185c490d3701f1140
2019-03-22 15:11:49 -07:00
Yifan Hong
26328e80b1 liblp: Replace open with GetControlFileOrOpen
... so that fds from init can be used.

Bug: 126233777
Test: unit tests

Change-Id: Ife652e61305ef4fb6a02edfa765a91b5959d1b4b
2019-03-22 14:26:25 -07:00
David Anderson
7082f68808 Fix device mapper name clashes on non-A/B devices.
Bug: 123666267
Test: sysfs reports -verity suffix for verity node names
Change-Id: I07fe428ce86209825e39a0ebf7b4d622b93cc81b
2019-03-22 13:47:27 -07:00
Mark Salyzyn
b43448941a fs_mgr: overlay: test is alarming on legacy eng builds.
On taimen-eng build, the test reports:

  [  WARNING ] user fastboot missing required to invalidate, ignoring a failure
  ERROR: expected "cat: /vendor/hello: No such file or directory"
         got "Hello World! Fri Mar 22 08:56:32 PDT 2019"
  [  FAILED  ] vendor content after flash vendor

Which is a result of a corner case problem on devices that need to
use overlayfs to support adb remount, but do not have fastbootd as
required by Dynamic Android Partitions (DAP).  These legacy non-DAP
devices we consider this a Known Issue.

The message however is too alarming when reporting this Known Issue
that the test has notified you it accepts.  Use standard notification
output format, and change the result to a series of WARNING instead.

The output would then look like:

  [  WARNING ] user fastboot missing required to invalidate, ignoring a failure
  [  WARNING ] expected "cat: /vendor/hello: No such file or directory"
  [  WARNING ] got "Hello World! Fri Mar 22 08:56:32 PDT 2019"
  [  WARNING ] vendor content after flash vendor

Which still tells us there is an issue, but does not cause a state
of extreme alarm when looking at the test output.

Test: adb-remount-test.sh
Bug: 120448575
Bug: 128876541
Bug: 123079041
Change-Id: I1d8d60f23f7670ade7eae1be629264f7507e0cfd
2019-03-22 09:41:20 -07:00
David Anderson
ee725f610f init: Support booting off GSIs installed to non-userdata block devices.
Bug: 126230649
Bug: 127953521
Test: GSI boots when installed to sdcard on hikey960
      GSI boots when installed to /data/gsi
Change-Id: Id59926ebe363939c8c5745bb1bf5bd13722dae7e
2019-03-20 18:22:00 -07:00
Mark Salyzyn
a497ab9b55 fs_mgr: remount: manufacture a missing / entry for default fstab
Adjust adb-remount-test.sh to run on cuttlefish.

Test: adb-remount-test.sh
Bug: 128033396
Change-Id: I86825a519fcc821f7da67b0570d7eb2043ae4b42
2019-03-19 16:45:12 -07:00
Treehugger Robot
0eadef4076 Merge "fs_mgr: overlay: suppress noise associated with test mounting scratch partition" 2019-03-19 18:58:41 +00:00
David Anderson
e2d977bffd Merge changes I88689889,Ibfd66734
* changes:
  liblp: Reclaim wasted space from unaligned partitions, v2.
  Revert "liblp: Reclaim wasted space from unaligned partitions."
2019-03-18 20:09:44 +00:00
Mark Salyzyn
60a76f330c fs_mgr: overlay: suppress noise associated with test mounting scratch partition
Add internal fs_mgr_is_ext4 and fs_mgr_is_f2fs to get heads up on
mount failures and thus bypass trying.

Test: adb-remount-test.sh
Bug: 109821005
Change-Id: Ieb1f8c19ced930b6fe2d1791ef710ce528da7e37
2019-03-18 12:50:00 -07:00
Treehugger Robot
4cd0914048 Merge changes from topic "apex_earlymount_no_bionic_bindmount"
* changes:
  /bionic path is gone
  Revert "Handle adb sync with Bionic under /bionic"
  Don't bind-mount bionic files
2019-03-15 09:02:18 +00:00
David Anderson
71fc3f358b liblp: Reclaim wasted space from unaligned partitions, v2.
When allocating a partition with a size that is unaligned (to the
optimal alignment), the remaining sectors are wasted since they are
never reallocated. This is because the free list is guaranteed to only
contain optimally-aligned regions. Unfortunately this means when a
partition is resized, we are wasting a small amount of space each time.
On a non-A/B device, this could wind up being significant.

For example, with an alignment of 512KiB, a 4KiB partition at offset 0
will waste 508KiB of space. The next extent to be allocated by any
partition will start at the next 512KiB.

To address this, we check if the last extent for a partition can be
extended to cover the difference between its last sector and the next
optimally aligned sector. We also verify that this region was not
allocated to any other partition, and does not appear in the free list,
to make sure we're not stealing space that will be used somewhere else.

Bug: 120434950
Test: liblp_test gtest
Change-Id: I88689889d44a4d2c51e659241918aaf2c064e049
2019-03-14 12:09:09 -07:00
David Anderson
483ce1d496 libfiemap_writer: Calculate FIBMAP blocks correctly.
FIBMAP blocks are returned in FIGETBSZ units, which means the number of
blocks also needs to be determined by FIGETBSZ. Using the stat blocksize
is incorrect. On VFAT, FIGETBSZ returns 512 whereas st_blksize is 32768.

Bug: 126230649
Test: fiemap_writer_test gtest
Change-Id: Id0a667936ff9c0b60b1e8f72920cf62ceece1657
2019-03-13 19:01:13 -07:00
Jiyong Park
192fdeb495 /bionic path is gone
The path no longer exists, and thus the path doesn't need to be
unmounted upon remounting.

Bug: 125549215
Test: system/core/fs_mgr/tests/adb-remount-test.sh
Change-Id: I7b263c755ad7eeaa63a00ad9795a134707698625
2019-03-14 07:35:54 +09:00
David Anderson
5df0092353 libfiemap_writer: Fix the progress bar on VFAT partitions.
We allocate VFAT file space by seeking and writing a single byte. It
turns out this is not fast enough to avoid invoking the progress
callback, so instead, we should seek in increments.

Bug: 126230649
Test: fiemap_writer_test gtest
Change-Id: I2b363af4ebde873411e8a3acd22ca68d175afe3f
2019-03-13 13:58:12 -07:00
David Anderson
b8e1711980 Revert "liblp: Reclaim wasted space from unaligned partitions."
When this patch landed, it was thought that over-allocating partitions
had no undesired side effects. However there are cases where it is now
causing problems. When flashing a GSI for VTS, the vbmeta is appended to
the end of the partition. If libfs_avb cannot find this vbmeta by
looking at the end of the partition, verification will fail. Similar use
cases with verifying other dynamic partitions are expected to be
problematic, so for now we revert to block-size alignment.

This reverts commit dccfdca1e1.

Bug: 125113249
Test: liblp_test gtest
Change-Id: Ibfd66734b63265e9fbb2ccf6b670c168c36190bc
2019-03-13 12:58:21 -07:00
David Anderson
f9134aa1d5 libfiemap_writer: Fix bugs in vfat handling.
FIBMAP returns block numbers, whereas fiemap_writer was treating them as
byte offsets.

Bug: 126230649
Test: gsi_tool install to vfat works
Change-Id: Ice80128213425aa9601e381d0fbe53f9581bf596
2019-03-11 19:17:01 -07:00
Mark Salyzyn
27c5047313 fs_mgr: overlay: preserve errno android::gsi::IsGsiRunning()
Resulted in remount reporting an mount_all failure.

Test: adb-remount-test.sh
Bug: 126256072
Change-Id: Ib9567748e5c12b047e38869c07755eb741308df4
2019-03-07 13:07:17 -08:00
David Anderson
6b4b265abf Add write support to SplitFiemap.
This adds few new methods to introduce write support to SplitFiemap:

(1) Write(), which has an internal cursor to stream data into the split
    files. As the end of one file is reached, the next is opened.
(2) Flush(), which calls fsync() on each internal FiemapWriter.
(3) HasPinnedExtents(), which calls the same on each internal
    FiemapWriter.

Included are some tests for edge cases in Write().

Bug: 126230649
Test: fiemap_writer_test gtest
Change-Id: I9fd509215975dbbb20a44b020315d3c1b287d1a0
2019-03-06 19:32:20 -08:00
Mark Salyzyn
6a26b4b74d fs_mgr: remount: automatically disable verity
Disable verity even if -R flag is not supplied.

Test: adb-remount-test.sh
Bug: 127494168
Change-Id: I3506c8e7d5525ca1d0685004c1fb7e26448fa648
2019-03-06 08:25:36 -08:00
Mark Salyzyn
3babd8488b fs_mgr: remount: pull root device from fstab
For devices that do not support overlayfs that are system as root,
need to pull /dev/root device from / mount point in fstab.

Test: adb-remount-test.sh
Bug: 126256072
Change-Id: I198c0e8c60770508737318afde95db2966294ff5
2019-03-06 08:25:36 -08:00
Mark Salyzyn
448fc9783c fs_mgr: test: track device active slot throughout
Report any discrepancy in the active slot.

Fix a problem with problematic error propagation for adb_cat()

Test: adb-remount-test.sh
Bug: 126256072
Change-Id: I8a5d4e364945c5e60d252333886987b8dca0cfb3
2019-03-06 08:25:36 -08:00
David Anderson
925c78b7e8 Merge "Support FiemapWriters that extend across multiple files." 2019-03-05 23:06:19 +00:00
David Anderson
79a43cf135 Merge "Add more filesystem support to libfiemap_writer." 2019-03-05 18:14:53 +00:00
Mark Salyzyn
f4e7422c65 fs_mgr: remount: add partition argument list
Can supply a specific partition to remount.  Partitions can be
specified by name or mount point.  Some extra work to differentiate
an unknown partition, invalid partition, or one that is covered by
overlayfs.

Test: adb-remount-test.sh
Bug: 122602260
Change-Id: Iab6f51c2b5ebe01f1cea3fb235445d5e2f495365
2019-03-04 11:13:05 -08:00
David Anderson
f344d63222 Support FiemapWriters that extend across multiple files.
This introduces a new SplitFiemap class that will divide an allocation
request across multiple FiemapWriters. This is primarily useful on
filesystems that have onerous restrictions on maximum file sizes. Vfat,
for example, supports a maximum of 4GiB, which is too small to satisfy
larger userdata size requests.

Bug: 126230649
Test: fiemap_writer_test gtest
Change-Id: I3c95d341e4e94e0c44bbf0e8553c34ccfdcd155b
2019-03-04 10:08:55 -08:00
David Anderson
4d4db8c09e Add more filesystem support to libfiemap_writer.
This adds support for vfat filesystems.

Bug: 126230649
Test: fiemap_writer_test gtest
Change-Id: I028cc7d95c313dd3ed826bc44cc3d0ffdcb58597
2019-03-02 00:31:15 -08:00
David Anderson
42294631a7 Merge "Fix double-accounting bug in liblp." 2019-03-02 03:16:10 +00:00
David Anderson
88e36c1514 Fix double-accounting bug in liblp.
When flashing in fastbootd, we create a new MetadataBuilder using the
given super_empty.img and attempt to import the existing partition
table. This will fail if there is some incompatibility in the partition
layout or partition quotas.

This import code was accidentally double-accounting partitions when
determining if they could fit within the group quota, preventing
"fastboot flashall" once partitions reached a certain size.

Bug: 126930319
Test: liblp_test gtest
Change-Id: I89a69cba110b62719197c9a4885cfc5bcf8f009f
2019-03-01 16:17:27 -08:00
David Anderson
b1f97f9d1b Merge "Remove the block size alignment restriction." 2019-03-01 22:48:25 +00:00
David Anderson
47f7420366 Remove the block size alignment restriction.
Rather than require block-size alignment, instead bump the requested
file size to the necessary alignment. This ensures that the final block
is usable without placing onerous restrictions on the caller to figure
out the file system's block size.

This will require callers (namely, gsid) to track the actual desired
image size separately from the flie size.

This patch also updates tests to use the actual filesize of the
filesystem, rather than hardcoded 4096.

Bug: 126230649
Test: fiemap_writer_test gtest
Change-Id: I000cca274718c3ceac526d7c3392fe3a23bb42bc
2019-03-01 09:52:52 -08:00
Treehugger Robot
c8964edb7e Merge "fs_mgr: fix fs_mgr avb_keys parsing problem" 2019-03-01 15:31:47 +00:00
Greg Kaiser
4d6f4cc4fe libfs_avb: Pass const string by reference
Minor performance improvement for ValidatePublicKeyBlob().

Test: TreeHugger
Change-Id: I3498b529abedb8169d2488a2dc04cbd42cdb6f7d
2019-02-28 06:34:25 -08:00