Even though build_image uses logger but there was no way to configure it
and even -v (verbose) didn't work.
common.ParseOptions() parses common arguments (e.g. -v -h --logfile).
Bug: n/a
Test: m
Test: build_image -v ... should print logs
Change-Id: I45a61ada9c6a5a616f05d3214e185d08b0f1c152
EROFS images by default will only work on newer kernels. Old versions of
5.4 and 4.19 need an extra flag to mkfs.erofs.
Add BUILD_EROFS_USE_LEGACY_COMPRESSION to force this flag.
Bug: 222373106
Test: build with BUILD_EROFS_USE_LEGACY_COMPRESSION enabled
Change-Id: I3818e14356beaa11073c6b9fea6a6ad999bba4aa
These wrapper scripts are difficult to maintain and there's not really
any value add. The argument strings don't match and every new build flag
has to be connected in an extra place.
Bug: 222715577
Test: make with EROFS enabled
Change-Id: Ie0e51cc30aa08b004d5d8345a2a2d885e193ffa9
build_image.py has been handling fsverity metadata generation in the
packing step, but it can cause issues because the metadata files are
missing in the $OUT directory, and they only exist in result system.img.
This change moves the generation logic into Makefile, and makes the
metadata tracked by ninja graph.
Bug: 206326351
Test: PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true and build
Change-Id: I1f910d8ac6e2cc3c54f35916871733c632f18e44
It's not guaranteed that the requested image size to mkfs is precisely
respected, due to metadata alignment and such. For accurate care maps
use the real image size rather than requested.
Bug: 205541521
Test: smartsync to 7892270, check that care_map.pb has the right block
count
Change-Id: I60fe64f720db13d3c3c4f1d8968341d7293217c9
fsverity digest manifest stores a map from files to fsverity digests.
The manifest is installed as a serialized protobuf file, to a signed apk
system/etc/security/fsverity/BuildManifest.apk.
Bug: 193113311
Test: build with PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true
Change-Id: I55fc10400206b8ce0d5f198faea08fe3930b362c
Using fsverity tool, fsverity metadata for specific artifacts in system
mage can be generated. Users can do that by setting a makefile variable
PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA to true.
If set to true, the following artifacts will be signed.
- system/framework/*.jar
- system/framework/oat/<arch>/*.{oat,vdex,art}
- system/etc/boot-image.prof
- system/etc/dirty-image-objects
One fsverity metadata container file per one input file will be
generated in system.img, with a suffix ".fsv_meta". e.g. a container
file for "system/framework/foo.jar" will be
"system/framework/foo.jar.fsv_meta".
Bug: 193113311
Test: build with PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true
Change-Id: Ib70d591a72d23286b5debcb05fbad799dfd79b94
This adds BOARD_xIMAGE_EROFS_PCLUSTER_SIZE and
BOARD_EROFS_PCLUSTER_SIZE, which set the "pcluster size" of erofs images
for individual images or all erofs images respectively. The pcluster
size affects the maximum size of a physical compressed block.
This also adds BOARD_EROFS_SHARE_DUP_BLOCKS, which turns on chunk
support in EROFS.
Bug: 201685920
Test: manual test
Change-Id: I27ec0899f89890562796dd9fa567fc74182fbefb
This is another code block where lots of stuff is duplicated, making it
hard to add new partitions or partition features.
Bug: N/A
Test: m otapackage, treehugger
Change-Id: I4c71275303a9246b37c03b24f531925b90d26fc8
This is another code block where lots of stuff is duplicated, making it
hard to add new partitions or partition features.
Bug: N/A
Test: m otapackage, misc_info.txt is identical
Change-Id: I435987605fc66b78bfaf454556f1eaa498e3728f
This adds BOARD_EROFS_COMPRESSOR to change the compression algorithm
globally, and BOARD_{x}IMAGE_EROFS_COMPRESSOR to change it for
individual partitions.
Bug: N/A
Test: manual test
Change-Id: I2ef831558242a4070ee96269140c33b66c689351
After the erofs image is built, it won't be modified again. The reserve
room is only used for the hash footer. There is no need to set an
excessive margin.
Bug: 199248596
Test: erofs image size is smaller than before
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
Change-Id: If2535b9ad2c2d60bea74feeca576a168387ff341
The image size of erofs is only affected by the content of the input
file, so it doesn't care about the input image_size. In addition, the
build time of erofs image is longer than ext4 due to compression. Only
execute mkfs once to reduce unnecessary time consumption.
Bug: 199248596
Test: erofs image build faster than before
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
Change-Id: I23a4e640d5066485de3fea0f808998045e716d1f
When we introduced Dynamic Partitions, we stopped giving readonly
partitions fixed sizes. In addition we introduced deduplication for
ext4. These two factors greatly reduce the impact of sparse images,
since there aren't many fill blocks to optimize.
This patch disables sparsing for images that are rightsized and do not
explicitly specify extra reserved space. This makes the images a little
easier to work with from an engineering perspective. They no longer have
to be unsparsed to interact with any tooling. It also eases a potential
source of bugs, as b/184225422 is not reproducible with sparsing off.
On Pixel, the difference between the sparsed partitions and unsparsed is
12M (out of roughly 4G).
Bug: 198001223
Test: make, treehugger, make target-files-package
dynamic partitions are no longer sparse images
Change-Id: I74459f8abe74a15a24ba5a40cf701e6af2db8179
This option is to reduce system partition size.
Bug: 171942852
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: Idc849cfce33ac0badb2b9b7953bb821c46a24472
build_image adds additional parameters (uuid, hash_seed) if
prop_dict["ext_mkuserimg"] is set to "mkuserimg_mke2fs".
The comparison does not take paths into consideration, so passing a
full path to mkuserimg_mke2fs would cause the parameters to not
be included.
This is currently not an issue for aosp builds, but could cause problems
for customized build systems.
Bug: 187742822
Test: Manual, using vendor build system, also executed 'm droid'
Change-Id: I7a8973dd0c4d8a39aea5aafcfe1aa69750fb1449
If dynamic partitioning is enabled and the partition size is not set,
we will get a KeyError before image_size or partition_size is calculated
when we try to catch exception in BuildImageMkfs.
Bug: 186704243
Test: build_image.py can correctly throw exception
Change-Id: I3d8c143ad5603d07fe94afb8bb911ead244f0bf7
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
The current inode usage estimate applies a factor (1.04x) to account for
differences between the directory and file count and what mkfs.ext4
produces. Bump this to 1.06x to allow more leniency.
This increases bramble partition sizes by 32KB total.
Bug: 182365132
Test: manual test
Change-Id: I9ade82cb89422e31ae8c5df9fff31157e88278c5
Bug:180682930
Test: erofs image can be set to input timestamp and UUID
Change-Id: I71e455350db51096e18ab5a03123500d56ebe252
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
* Prerequisites: external/f2fs-tools: sload compression support
* Must work with corresponding changes in system/extras repository
* If Board config does not change, it falls back to old behavior,
i.e. no compression for the system partition
* Kernel f2fs compression support is a prerequisite if the Board config
enables the compression (see below)
* Necessary board config change
(e.g. device/<vendor>/<product>/BoardConfig-common.mk)
BOARD_SYSTEMIMAGE_FILE_SYSTEM_TYPE := f2fs
BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE := 200000000
BOARD_SYSTEMIMAGE_FILE_SYSTEM_COMPRESS := true
BOARD_SYSTEMIMAGE_F2FS_SLOAD_COMPRESS_FLAGS := <sload compress sub-options>
Setting BOARD_SYSTEMIMAGE_FILE_SYSTEM_COMPRESS to true enables both the
compression support when the initial empty file system be made (mkfs.f2fs)
and the compression flag (-c) when the system image files be side-loaded
by sload.
Sload compress sub-options (i.e. options other than -c) will be provided
by BOARD_SYSTEMIMAGE_F2FS_SLOAD_COMPRESS_FLAGS. If it is not given, or
is empty, the default sub-options will be used
Please refer to the sload.f2fs manual page.
Setting BOARD_SYSTEMIMAGE_FILE_SYSTEM_TYPE to f2fs is trivially necessary.
* File system table (fstab), notably the file 'fstab.hardware', should
also changed:
- The file type must be changed to f2fs
- Perhaps also other f2fs-specific options
Bug: 170918499
Test: Pixel4a userdebug build (from build id 6918751)
Signed-off-by: Robin Hsu <robinhsu@google.com>
Change-Id: Id9d67b5cb35dc806e06ff1320e89114abc996a28
If dynamic partitioning is enabled and the partition size is not set,
du will be used to calculate the partition size.
For compressed file systems, it's better to use the compressed size
to avoid wasting space.
Bug:174816929
Test: erofs image size is smaller than the original file
Change-Id: I1deda85d312c19620680531223fffcfb815e5fd4
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
Modules partition is a dynamic read-write partition.
- AVB is not enabled on the partition
- OTA is file-based; see follow up CL for details
- No build prop files; in particular, no build fingerprint
- No fs_config
- No notice files; notice files are included in individual APEXes
Test: build on CF
Bug: 163543381
Change-Id: Ie397b9ec61dfd1c158450d050196024604854d4d
This reverts commit 6022545272.
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.
Bug: 152167826
Change-Id: I5c952cd2a976ba1a09ddc66d56c2b8b55a61986b
Test: unittests pass
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.
Bug: 152167826
Test: unittests pass
Change-Id: I2fe7e93a2f4de8e55f5f8051b000b96b5efdc85a
* When $OUT is on a compressed disk,
`du -k -s` might not return the correct sizes.
* Adding `-b` gets us close to the actual size.
toybox commit: https://github.com/landley/toybox/pull/177
Test: m, builds and boots.
Devices: crosshatch blueline bonito sargo (anything with dynamic partitions)
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.
Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
`zip -d out/dist/target_files.zip IMAGES/\*`
`add_img_to_target_files -a out/dist/target_files.zip`.
Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
the partial target-files.zip IMAGES are identical to the merged
target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
Regardless of the value of "ext4_share_dup_blocks" "-c" flag is passed
unconditionally. The correct behavior is to check if
"ext4_share_dup_blocks" is set to "true".
Test: 1. build bonito with "BOARD_EXT4_SHARE_DUP_BLOCKS := false"
2. check for "shared_blocks" feature of system.img
./out/host/linux-x86/bin/tune2fs -l \
./out/target/product/bonito/system.img | \
grep -o shared_blocks
Change-Id: I8c0e5606658dd0f47b71cfc6c453a77a57da4f42
This adds the ability to format f2fs and ext4 in a way that supports
casefolding and project quotas. Project quotas require wider inodes, and
thus cannot be enable apart from at build time. Casefolding can be
configured with f2fs.fsck/tune2fs later.
Both features are enabled via inheriting emulated_storage.mk in the
device.mk
Bug: 138322712
Bug: 138321217
Test: Enable on device with related patches, view fs configuration on /data
Change-Id: Ib1ab0589006ff1b3fb7478b7e9b0a8aad71e4b09
system_other is not OTA'd (outside of factory OTA), and its block map differs greatly from
system.img, so do not attempt to re-use it.
Bug: 139201772
Test: make with base_fs file for system
Change-Id: I1ed2091da381ebdb93522a081e5d2f7c18c9d069
This enables mixed builds to use the file_contexts.bin from the system
build when regenerating images that come from system target files, and
similarly for file_contexts.bin from the other build when regenerating
images from other target files.
In monolithic (non-mixed) builds all image-specific selinux_fc props
point to the same file_contexts.
Bug: 132108151
Test: Built and booted mixed build devices.
Change-Id: Id51ed6d96ea6337879f1ab21d47c93c67bc25312
assert-max-image-size doesn't make sense for
dynamic partitions, as build_image.py always find the
right size for the output image. Hence:
- build_image.py no longer need to write generated_*_info.txt
(which contains the size of the image).
- assert-max-image-size on the static BOARD_*IMAGE_PARTITION_SIZE. If
a partition is dynamic, that variable isn't set, and
assert-max-image-size becomes a no-op. If the partition is static,
assert-max-image-size checks the static partition size as it used
to be.
- Fix read-size-of-partitions to use the size of the partition by
reading the image directly (instead of using generated_*_info.txt).
For devices without AVB, with DAP enabled, and does not have
RESERVED_SIZE for partitions, because of right sizing, the original
code always warns about approaching size limits. Since such checks
doesn't make sense for dynamic partitions, remove them.
Test: builds on device with dynamic partitions
Test: builds on cuttlefish with DAP enabled (without AVB), no
more size limit warnings:
WARNING: out/target/product/vsoc_x86/vendor.img approaching size limit (X now; limit X)
Fixes: 122377935
Change-Id: I75e1b8322197cb18cf397d02aefd49d777bb6405
