Commit graph

700 commits

Author SHA1 Message Date
David Anderson
4d71d85805 liblp: Allow flashing to readonly logical partitions.
By default, logical partitions with the readonly flag are created with a
readonly device. This change allows callers of CreateLogicalPartition
to create writable devices so they can still be flashed.

Bug: 78793464
Test: fastboot flash product_services works with fastbootd
Change-Id: Ia8d2761a3067e3b62815acbf0b6fb7f033072ba2
2018-08-06 15:47:26 -07:00
Treehugger Robot
b0c9318493 Merge "liblp: Try to merge extents during partition resizes." 2018-08-03 20:02:41 +00:00
David Anderson
1aab3909f7 liblp: Try to merge extents during partition resizes.
When adding extents to partitions, if the previous extent and new extent
are contiguous, merge them together to avoid allocating unnecessary
device-mapper targets.

Bug: 79173901
Test: liblp_test gtest
Change-Id: I80087df9aea8141c5e16f8d4cdb3dd7da02aee8c
2018-08-03 11:02:25 -07:00
David Anderson
b9f734c936 liblp: Add more logging for logical partition operations.
This logs when partition tables update, when partitions resize, and when
partitions are unmapped from device mapper.

Bug: N/A
Test: N/A
Change-Id: I1125332c79fccc3ebc556b3b48856901e2503c47
2018-08-03 10:59:43 -07:00
David Anderson
4d9c7459c4 liblp: Require block-aligned partition sizes.
DM_TABLE_LOAD will reject dm-linear entries if their size is not a
multiple of the backing device's logical block size. For example, a
partition of 10GiB+512 bytes will fail to map in device-mapper if the
logical block size is 4096 bytes. To address this, this patch adds a
few changes to liblp:

The block size given to lpmake is now recorded in LpGeometryMetadata.
The block size must be a multiple of the sector size. In addition,
partiton sizes are now aligned to the block size, and the super
partition must have enough free space to allocate at least one block (in
addition to storing metadata).

GrowPartition now has multiple checks that the block-size invariant is not
violated, to ensure that no invalid partition tables will be created.

Bug: 79173901
Test: liblp_test gtest
Change-Id: I484aac1f9b90ebd92dc1c89ce1e09cd89bbb441e
2018-08-02 10:33:41 -07:00
David Anderson
de1daa72aa liblp: Simplify GrowPartition().
The partition resize algorithm duplicates a lot of logic because it
handles the final free interval separately from other free intervals.
This is unnecessary and makes it harder to change the actual algorithm.

This change makes GrowPartition() treat the final free space region the
same as free gaps in between partitions. It does this by converting the
extent list into a gap list, and then adds a final gap for the remainder
of the free space. The resize function no longer has to treat the end of
the disk separately.

This patch does not change the way partitions are allocated, it is
purely a refactoring.

Bug: 79173901
Test: liblp_test gtest
Change-Id: I4780f20b23fe021eac62de874b061857712c04fe
2018-08-02 09:14:47 -07:00
Treehugger Robot
ddcee93c01 Merge "liblp: Add a ResizePartition helper to MetadataBuilder." 2018-08-02 16:01:24 +00:00
David Anderson
2e755e38af liblp: Add a ResizePartition helper to MetadataBuilder.
This change is to assist with implementing the fastbootd "resize-partition"
command. The GrowPartition and ShrinkPartition functions are now
private.

Bug: 78793464
Test: N/A
Change-Id: Ic66a3052359e2558663272ff6e014704206b197e
2018-08-01 17:41:29 -07:00
Treehugger Robot
8f2a34a89b Merge "fs_mgr: Always mount tmpfs with MS_NOEXEC" 2018-08-01 22:41:56 +00:00
Tom Cherry
6e05552e35 Merge "init: run fsck for early mount partitions" 2018-08-01 21:02:35 +00:00
Tom Cherry
81ae07509d init: run fsck for early mount partitions
Bug: 111883560
Test: fsck runs successfully during early mount
Change-Id: I19bcd2610f87f5331f94c228fcbec8f9379fc5e8
2018-07-31 13:58:17 -07:00
David Anderson
f792e4163d liblp: Allow building liblp as a shared library.
Bug: 79173901
Test: system image, partition_tools both build
Change-Id: I70f45d4cebfbddc39d03aa3fb3a56b6c48a20d7e
2018-07-27 19:34:56 -07:00
Treehugger Robot
0937a84521 Merge "Move recovery to /system/bin" 2018-07-27 20:25:53 +00:00
Tao Bao
9ff0669ac1 Merge "fs_mgr: Build and use shared libs." 2018-07-26 04:48:46 +00:00
Treehugger Robot
56f76ec018 Merge changes Ica6bf5c6,I3f626433
* changes:
  fs_mgr: libdm: add support android-verity target.
  dmctl: Do not skip argument if not matched with '-ro'.
2018-07-26 02:29:12 +00:00
Tao Bao
a9fad6cd15 fs_mgr: Build and use shared libs.
Bug: 78793464
Test: `m dist`
Change-Id: I21ed5de72217c5cfa37e0217b0e79ed0a6f82031
2018-07-25 13:42:39 -07:00
Jerry Zhang
5de2be5cd5 Move recovery to /system/bin
Executables should be in /system/bin
rather than sbin. Change lookup paths
in a few places to reflect this.

Test: recovery is in system/bin
Bug: 78793464
Change-Id: Ibcdff7abf1f21e2566e02cad8bdb3c26ee500534
2018-07-24 21:42:23 -07:00
Sandeep Patil
efc5479085 fs_mgr: libdm: add support android-verity target.
The support for android-verity makes it possible for us to test
raw verified filesystem images to be attached to android-verity
target and have it verified by the kernel.

This makes the testing of android-verity device mapper target
much easier as it doesn't _have_ to be used for root mount.

Bug: 72722987
Test:
 $ losetup /dev/block/loop0 /data/local/tmp/verity_fs.img
 $ dmctl create verity-fs android-verity 0 4200 \
   Android:7e4333f9bba00adfe0ede979e28ed1920492b40f 7:0

Change-Id: Ica6bf5c6e1fd758fdb4005fc8a09755f369a8a0f
Signed-off-by: Sandeep Patil <sspatil@google.com>
2018-07-23 15:24:55 -07:00
Sandeep Patil
0d469de9a0 dmctl: Do not skip argument if not matched with '-ro'.
'dmctl create' command would fail if '-ro' flag is not
used. This happend because the argument in place of '-ro'
is skipped unconditionally.

Bug: 72722987
Test:
 $ losetup /dev/block/loop0 fs.img
 $ dmctl create FOO linear 0 25600 /dev/block/loop0 0

Change-Id: I3f6264336b9c9a1b5de76f3a1d29e6045453f9cb
Signed-off-by: Sandeep Patil <sspatil@google.com>
2018-07-23 14:50:26 -07:00
David Anderson
23de22d54b fs_mgr: Add a method for creating one-off partition instances.
fastbootd needs to temporarily map and unmap logical partitions in order
to flash them. To make this easier, this change introduces fs_mgr
methods to create and destroy a single logical partition by name.

Bug: 79173901
Test: N/A
Change-Id: Ic085f2175c26a5edf5f18544012dcfe6d8088f09
2018-07-23 11:43:27 -07:00
David Anderson
e0384bbc69 fs_mgr: Remove early prototype code.
This early code is no longer needed now that logical partitions can be
created and flashed normally.

Bug: 79173901
Test: N/A
Change-Id: I393ef23b3d3ce1cd9c80833358051838d0e9f333
2018-07-19 14:38:33 -07:00
David Anderson
9555f3dcf6 liblp: Allow packing partition data into super images.
This change allows partition data to be included in the sparse image for
the super partition. The data is specified as a map from partition name
to partition image file. The image file can either be a normal file or a
sparse image. If it is sparse, it is unpacked into a temporary file and
then re-packed into the super image.

Bug: 79173901
Test: (1) lpmake -S with a partition image flashes to walleye.
      (2) The image can be mounted if it contains a filesystem.
      (3) simg2img will successfully unpack the image.

Change-Id: I98e9af3bfec9863143c1b95a1712b1d435942542
2018-07-18 16:53:42 -07:00
David Anderson
ce483b6f06 liblp: Allow specifying a custom block size when building sparse images.
liblp currently creates sparse files with a block size of 512 bytes.
Unfortunately devices can reject sparse files that don't have a specific
block size. To accomodate this, SparseBuilder now requires that
partition metadata be built with an explicit block size, and the device
alignment must be a multiple of that block size.

This change also modifies SerializeGeometry to automatically include
0-padding to 4096 bytes.

Bug: 79173901
Test: lpmake -S can be flashed with fastboot to walleye
Change-Id: Iab04f6f55bb373e21767963335c85ad1119a65a7
2018-07-17 18:37:46 -07:00
David Anderson
b36db4ba0e liblp: Add a method to create sparse images of the super partition.
This change enables flashing of the super partition without using
lpflash or adding explicit fastboot support for the liblp image format.

Bug: 79173901
Test: image can be flashed to device and read by liblp
Change-Id: Id6c5e595ba831644364b7335b58cf6a43d5833c3
2018-07-13 18:17:56 -07:00
David Anderson
62cd1e06ab liblp: Move image reading/writing functions to a new cpp file.
This is in preparation for adding sparse image support, to avoid
cluttering the more critical reading and writing logic.

Bug: 79173901
Test: N/A
Change-Id: Icdddb849aebba4adf18a3e63ffbd3f36adda812d
2018-07-13 18:17:56 -07:00
David Anderson
8cf1e38b21 liblp: Force more sizes to be a multiple of the sector size.
This makes offset calculations and library interactions much easier.

Bug: 79173901
Test: liblp_test gtest
Change-Id: I595c5435bd6bc166693a434ecdcd2d098185f963
2018-07-13 18:17:56 -07:00
David Anderson
4813823e73 liblp: Add block device size to LpMetadataGeometry.
This makes it easier to validate the device and make assumptions about
the device layout without having actual access.

Bug: 79173901
Test: liblp_test gtest
Change-Id: I53d24684020ec9210c8a17d155e738b3b2405806
2018-07-13 18:17:55 -07:00
David Anderson
b3fc16c5f3 liblp: Rename the metadata partition name to "super".
Bug: 79173901
Test: N/A
Change-Id: I99ccb095030c82cbc4bb89b40bff333decd35532
2018-07-12 14:08:42 -07:00
David Anderson
9c3c29466a liblp: Clean up public headers.
This changes reader.h and writer.h to be private includes. A new liblp.h
header now contains the public API surface of those two files, as well
as some miscellanious functions previously in metadata_format.h.

Bug: 79173901
Test: N/A
Change-Id: I40c5dda0c8e5765f8bccfd5c17b4c800b41be77b
2018-07-12 14:08:42 -07:00
David Anderson
a5cb671e94 liblp: Improve reliability of UpdatePartitionTable.
If UpdatePartitionTable is interrupted while writing metadata, the next
update becomes much more risky, as only one valid copy may exist. If
that subsequent update is interrupted, all metadata copies may be
corrupt.

To alleviate this, we now synchronize both metadata copies before each
update. If the backup copy is corrupted, we replace it with the primary
copy, and vice versa. Similarly if the primary and backup metadata do
not match, we sync the backup to contain the same data as the primary.

If for some reason this synchronization process fails, we do not proceed
to write a new partition table.

Bug: 79173901
Test: liblp_test gtest
Change-Id: Ic80cf9a5dc6336ff532e069ca5c8c76371550cb9
2018-07-11 17:14:12 -07:00
Treehugger Robot
c457df7d34 Merge "liblp: Make it easier to test UpdatePartitionTable." 2018-07-12 00:08:52 +00:00
Treehugger Robot
32960e54b4 Merge "liblp: Refactor the partition table update API." 2018-07-11 23:44:13 +00:00
David Anderson
451694e29d liblp: Make it easier to test UpdatePartitionTable.
This change makes the internal UpdatePartitionTable function more
testable by parameterizing its write functions. It also adds two tests,
one of which exposes a flaw in the current implementation.

Bug: 79173901
Test: liblp_test gtest
Change-Id: I3c4112794b97d577a27f035baeac2d42ac75f552
2018-07-11 13:37:52 -07:00
David Anderson
21671eda3e liblp: Refactor the partition table update API.
This change separates the WritePartitionTable function into two separate
functions: FlashPartitionTable and UpdatePartitionTable. This will be
cleaner than having extra unnecessary parameters and makes it clearer
when to call each one.

The motivation for this change is to improve the reliability of
UpdatePartitionTable() and to be able to simulate I/O failures for
testing.

Bug: 79173901
Test: liblp_test gtest
Change-Id: Iee5eb2e99fb76aef0b93a65bf85a89907e7cd9bf
2018-07-11 13:05:09 -07:00
Pawin Vongmasa
2b67f0b85b Fix build breakage on non-linux
Bug: 79173901

Test: liblp_test gtest
Change-Id: I846e99f636ed05f97e44d05a617dc17b5f127d2e
2018-07-10 16:46:44 -07:00
David Anderson
2332afb31f liblp: Implement support for request queue alignment.
Block devices in the Linux kernel have a "minimum I/O request" size. The
minimum size is usually acquired by the block driver and can change
from device to device. When stacking devices (such as with
device-mapper), the kernel goes through great lengths to make sure this
alignment is respected for optimal I/O. In device-mapper's case,
misalignment can lead to kernel warnings and performance issues.

While this is unlikely to matter with a few targets, it could become
problematic on a large number of targets, and so we would prefer to
align all partition extents to the minimum I/O size.

We now support two new properties in the partition table geometry: an
"alignment", which is the minimum I/O size, and an "alignment offset",
which is an offset that when applied to sector 0, causes the sector to
be properly aligned within its parent device (for example, if a
physical partition is misaligned). All partition extents now begin on a
sector that respects this alignment.

One major caveat is that it is difficult for the initial partition table
to have the correct alignment without build system and/or flash tool
support. To accomodate this, all alignment is optional, and the lpmake
tool will support a default alignment of 1MiB as a failsafe.

Bug: 79173901
Test: liblp_test gtest
Change-Id: I5bc41b90aa085f4f30393951af0d2b37c4ac2a72
2018-07-09 18:50:28 -07:00
David Anderson
839a818221 liblp: Fix incorrect get_block_device_size usage.
Bug: 79173901
Test: N/A
Change-Id: I22a3e86153830782453a47e460800f3cb4c26958
2018-07-06 16:01:27 -07:00
Treehugger Robot
2bbbd06a8a Merge "fs_mgr: fs_mgr_update_verity_state take std::function" 2018-06-28 17:26:34 +00:00
Mark Salyzyn
b5065fcb28 fs_mgr: fs_mgr_update_verity_state take std::function
By changing to a std:function from a plain function pointer as an
argument to fs_mgr_update_verity_state(), a lambda with captured
objects can be executed providing thread safety.

Test: compile and boot
Bug: 109821005
Change-Id: I2699a08cc2c5216625dcbf84eeac0c460f4ae36a
2018-06-28 08:43:35 -07:00
Mark Salyzyn
eba4706a2e fs_mgr: Add support in fs_mgr_read_fstab_file for /proc/mounts
Generate fstab from /proc/mounts.  Basically like any regular fstab
minus the requirement for the Android fs_mgr_flags.  Added a unit
test to confirm that fs_mgr_read_fstab("/proc/mounts") matches the
setmntent("/proc/mounts")+getmntent() content.

As a result, discovered a bug in parse_flags that would report "ro"
if the "rootcontext=..." flag was referenced.

Test: fs_mgr_unit_test
Bug: 109821005
Change-Id: I635ee979e11ee089b09adbedb5f42b6ba78026de
2018-06-27 14:49:06 -07:00
David Anderson
b5acb1ac4a fs_mgr: remove fs_mgr_dm_ioctl
Bug: 110035986
Test: N/A
Change-Id: Ia35a45415f1b2bc476784890d838b44e6854d5b9
2018-06-27 12:48:29 -07:00
David Anderson
40b594827f fs_mgr: replace DM_TABLE_STATUS use with libdm.
This change introduces a new GetTableStatus method on DeviceMapper,
which returns a vector of information about each target in a device's
table. Some target types (such as verity) can also return additional
information as a string.

Support for this call has also been added to the "dmctl" tool via a
"table" command. Examples:

    $ dmctl create blah zero 0 8000 linear 8000 1000 /dev/block/sdd1 0
    $ dmctl table blah
    Targets in the device-mapper table for blah:
    0-8000: zero
    8000-9000: linear

    For verity:
    sailfish:/ # dmctl table system
    Targets in the device-mapper table for system:
    0-4128792: android-verity, V

Bug: 110035986
Test: libdm_test gtest; AVB1 device still boots
Change-Id: Iaf13450d3b32e2264c7c399a8af8d6bade260592
2018-06-27 12:48:28 -07:00
David Anderson
9803fbf30b fs_mgr: Use libdm for fs_mgr_verity.
This change uses libdm to replace the hand-coded ioctl code in fs_mgr_verity.
Since AVB2 is now preferred to AVB1, this is a surgical change to allow
us to remove fs_mgr_dm_ioctl. Rather than refactor the target string
construction code, we instead provide a DmTargetVerityString class which
passes a raw parameter string directly to DmTable.

Bug: 110035986
Test: AVB1 device still boots
Change-Id: Iad88993a98abb388a12cec9e6d4543dfb4157414
2018-06-26 12:41:47 -07:00
David Anderson
1d17339560 libdm: Add DmTargetVerity and use it in fs_mgr_avb.
This change adds a DmTargetVerity class and replaces the existing verity
table builder in fs_mgr_avb.cpp. DmTargetVerity must be constructed with
all the required parameters for dm-verity. Additional arguments can be
appended via helper methods.

Bug: 110035986
Test: libdm_test gtest; boots with verity on AVB devices
Change-Id: I4fc2f7ef122c841e8b4ec4156177a6224ffbac30
2018-06-25 17:56:12 -07:00
David Anderson
4b52440812 fs_mgr: Use libdm for dm-linear devices.
This removes the custom ioctl code in fs_mgr_dm_linear.cpp in favor of
using libdm.

Bug: 110035986
Test: logical partitions are still created in first-stage init
Change-Id: I941a27ce865aa1bc712b91d8d4c874462e0d11df
2018-06-25 12:56:15 -07:00
David Anderson
2d3e79f5d5 libdm: Add unit tests for dm-linear.
Bug: 110035986
Test: libdm_test gtest
Change-Id: I83fb978931aa36b83880c4d44745cc45ee516fc4
2018-06-25 12:56:14 -07:00
David Anderson
1f428ea054 libdm: Add helper classes for loop control.
This change adds a LoopControl class for interacting with the loop
control device. In addition it provides a LoopDevice class for creating
temporary loop devices. This is aimed at being a building block for libdm
tests, so the current functionality is limited to attaching and
detaching file descriptors and finding free loop devices.

Bug: 110035986
Test: libdm_test gtest
Change-Id: Ice2891e3a44a037aff7b81c63ac793815640d582
2018-06-25 11:37:51 -07:00
Treehugger Robot
a67b00a322 Merge "libdm: Implement GetDmDevicePathByName()." 2018-06-23 00:00:51 +00:00
Treehugger Robot
245b03248f Merge "liblp: Add unit tests for flashing and readback." 2018-06-22 23:28:02 +00:00
David Anderson
5e9e74be61 libdm: Implement GetDmDevicePathByName().
This change implements the ability to get the path of a block device
given a device-mapper device name. In addition, dmctl now has a
"getpath" command to perform this query, as a shortcut for searching
through /sys/block/*/dm/name.

Bug: 110035986
Test: N/A
Change-Id: I9ebd824fa800004f591fc02fc1b1950e0c7fba65
2018-06-22 14:23:31 -07:00