Commit graph

131 commits

Author SHA1 Message Date
Jaekyun Seok
b7735d8105 Support /product partition
This CL adds the following build flags to support building product partition
from Android build system.
- BOARD_AVB_PRODUCT_ADD_HASHTREE_FOOTER_ARGS
- BOARD_AVB_PRODUCT_KEY_PATH
- BOARD_AVB_PRODUCT_ALGORITHM
- BOARD_AVB_PRODUCT_ROLLBACK_INDEX_LOCATION
- BOARD_PREBUILT_PRODUCTIMAGE
- BOARD_PRODUCTIMAGE_EXTFS_INODE_COUNT
- BOARD_PRODUCTIMAGE_FILE_SYSTEM_TYPE
- BOARD_PRODUCTIMAGE_PARTITION_SIZE
- BOARD_PRODUCTIMAGE_SQUASHFS_BLOCK_SIZE
- BOARD_PRODUCTIMAGE_SQUASHFS_COMPRESSOR
- BOARD_PRODUCTIMAGE_SQUASHFS_COMPRESSOR_OPT
- BOARD_PRODUCTIMAGE_SQUASHFS_DISABLE_4K_ALIGN
- BOARD_USES_PRODUCTIMAGE
- LOCAL_PRODUCT_MODULE
- PRODUCT_FOOTER_ARGS
- PRODUCT_PRODUCT_BASE_FS_PATH
- PRODUCT_PRODUCT_VERITY_PARTITION
- TARGET_COPY_OUT_PRODUCT
- TARGET_OUT_PRODUCT*

And PRODUCT_PRODUCT_PROPERTIES is added to save product-specific
properties in /product/build.prop.

Bug: 64195575
Test: succeeded building product.img with BOARD_PRODUCTIMAGE_PARTITION_SIZE,
BOARD_PRODUCTIMAGE_FILE_SYSTEM_TYPE and PRODUCT_PRODUCT_VERITY_PARTITION.

Change-Id: I774e6a38003734421591e51bed103802ff84f432
2018-01-25 09:33:58 +09:00
Jin Qian
fde9f7906b Add BOARD_EXT4_SHARE_DUP_BLOCKS to share duplicated blocks on ext4
This option applies to system.img and vendor.img.

Bug: 64109868
Change-Id: Ib4c7384049dca08e57441ef71cc81c2fba4462dd
2018-01-22 17:21:11 -08:00
Ng Zhi An
9446c1df98 Fix order of args passed to mksquashfsimage.sh
Bug: 72234456
Test: make systemimage
Change-Id: I1b4b07f0a488eade2f297fea12d02b6e8ccba7d8
2018-01-19 15:55:39 -08:00
Tao Bao
cd53a89599 releasetools: Dump mkfs/fsck/simg2img outputs on error.
The output (stdout and stderr combined) would be helpful in understanding
the cause of the failure.

Not changing other occurrences in build_image.py yet, since it's less
obvious if a failed call should be considered fatal (thus whether to
dump such outputs). For example, GetVeritySize() simply returns 0 on
failed calls. They deserve further clean-ups in later CLs (with unit
tests).

Bug: 71864688
Test: Inject errors into mke2fs/e2fsck/simg2img. `m snod` prints outputs.
Test: `m snod` remains quiet on successful runs.
Change-Id: I172403f6cd05bce93767265dbcb110271a68e2a8
2018-01-19 11:15:42 -08:00
Tao Bao
c72727ac3f releasetools: Clean up build_image.py.
Mostly cosmetic changes, such as moving print statement to print
function. The only functional change is to wrap the file opening in
Append() with try statement, which would dump the error message as
needed.

Test: `m dist`
Test: python -m unittest test_build_image
Change-Id: I4e6c593517cf737bfcbe51bc533b22a247b10e44
2018-01-03 11:52:28 -08:00
Tao Bao
d8a953d7ab releasetools: Move assertions into CheckHeadroom().
Assert the input arguments inside CheckHeadroom() function, and add
matching tests. Also use common.MakeTemp{File,Dir} in tests.

Test: `m dist`
Test: python -m unittest test_build_image
Change-Id: Ib45f24651af9591a9cfdd72aac3558f6c9a80f44
2018-01-03 11:52:16 -08:00
Tao Bao
1c830bfbaa releasetools: Add common.MakeTempDir().
This simplifies the work to track mkdtemp()'d dirs. This CL also cleans
up the temporary file creation in build_image.py.

Test: `m dist`
Test: python -m unittest test_common
Change-Id: Ied7d86126714b9b26e5df7bcca1dfdea21cfcf60
2018-01-02 12:22:01 -08:00
Treehugger Robot
b379762089 Merge "build_image: Factor out CheckHeadroom() and add tests." 2017-12-19 00:00:41 +00:00
Treehugger Robot
d30b75190d Merge "Remove the support for BOARD_HAS_EXT4_RESERVED_BLOCKS." 2017-12-18 22:17:46 +00:00
Tao Bao
d4349f2106 build_image: Factor out CheckHeadroom() and add tests.
The test is mostly trivial, but it ensures the result parsing from
actual calls to mke2fs.

Test: python -m unittest test_build_image
Test: `m dist` on aosp_marlin-userdebug (w/ and w/o defining
      PRODUCT_SYSTEM_HEADROOM respectively).
Change-Id: I8b9964213950e76f6d7d5518414a1bab888b4706
2017-12-07 23:39:56 -08:00
Jaegeuk Kim
10c9f5d9d3 Merge "core, release: add sload.f2fs for otatools.zip" 2017-12-08 00:57:42 +00:00
Tao Bao
79d52f8798 Remove the support for BOARD_HAS_EXT4_RESERVED_BLOCKS.
This is only used in file-based OTA, where we may reserve space on
/system partition. With the deprecation of file-based OTA since O, the
code has become obsolete.

Test: `m dist`
Change-Id: I2bd686b292f93b566fe28a2bcd74c564ffc75dac
2017-12-07 14:36:21 -08:00
Elliott Hughes
73ff57f48a Explain why BuildImage failed.
BuildImage can return false for lots of reasons. Log two of the more
plausible ones. You'll still need to edit this file and pass True to
RunCommand to work out *what* went wrong, but at least this logging will
tell you *where* things went wrong.

Bug: http://b/63142920
Test: debugged my actual problem
Change-Id: If84427058108dcac0a1b5e2216b8bb11eef522ca
2017-12-06 12:16:39 -08:00
Jaegeuk Kim
2ea1eba98c core, release: add sload.f2fs for otatools.zip
This also updates mkf2fsuserimg.sh to call sload.f2fs.

Change-Id: I245f6b3dfcd966fbc6783329d781375879fcd065
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-12-05 10:59:50 -08:00
Tomasz Wasilczyk
29ec06b7ba Silence superfluous build messages.
Bug: 69315492
Test: it builds
Change-Id: Ic02484b80fa27c53786204402930ed5f33ec7fda
2017-11-15 10:34:01 -08:00
Treehugger Robot
404af2ca4b Merge "Build repeatable system images with mke2fs." 2017-10-18 17:20:00 +00:00
Tao Bao
d86e3114fc Build repeatable system images with mke2fs.
We have added support in mkuserimg_mke2fs.sh that allows specifying
filesystem UUID and hash seed UUID. This CL generates and passes the
UUIDs based on the image name and build fingerprint. This way we can
rebuild and get identical images.

Note that this only applies to images generated with `m dist` and with
signing tools (sign_target_files_apks.py and
add_img_to_target_files.py). Images under $OUT (built with `make`) are
not affected.

Bug: 64994964
Test: lunch marlin-userdebug; run `m dist` twice (after deleting the
      intermediate files under $OUT/obj/PACKAGING), and compare the
      generated images.
Change-Id: I41cf4e5869582bb930af2f35a8e9c79bff43b2a2
(cherry picked from commit 3aa21e6bb9)
2017-10-16 22:48:33 -07:00
Bowgo Tsai
6ceeb1a8bf Adding Android verified boot 1.0 metadata into ENG builds
Adding verified boot metadata with a "disable magic". The resulting
metadata at the end of each image (e.g., system.img, vendor.img) will
be the same as triggering an "adb disable-verity" on an USERDEBUG image.

This can help simplify the code on fs_mgr, which won't have to check if
current image is an ENG build or not.

Bug: 63056044
Test: boot sailfish eng/userdebug builds
Change-Id: I95d23ac7b76c04d6d4483c9c4dc1de16bf0d9c3a
2017-10-12 10:08:44 +08:00
Tao Bao
2b6dfd6689 Use computed salt for AVB-signed images.
We want the generated images being identical for the same source files.
Currently the generated ext4 image (either from make_ext4fs or mke2fs)
is reproducible, but the AVB footer added by avbtool contain changes
because of the random salt being used.

This CL changes the avbtool invocation to specify "--salt <hexstring>"
(already supported by avbtool) to use reproducible salt that's computed
based on fingerprints (or thumbprints if applicable).

Bug: 67023482
Test: Regenerate images from the same source as follows:
  Use a target_files.zip from an AVB-enabled target.
    $ zip -d target_files.zip IMAGES/\*
    $ ./build/make/tools/releasetools/add_img_to_target_files.py \
          -v target_files.zip
  Repeat the above commands and compare the generated images.
Change-Id: Id9db17ae0132ca3a820b4be5a5ef06ca3fef71ed
(cherry picked from commit 8f05cca1d9)
2017-09-29 10:47:40 -07:00
Tianjie Xu
149b7fbf08 Add a verbose option to RunCommand in build_image.py
Make the build log less verbose; also print the result for
AdjustPartitionSizeForVerity().

Bug: 65292710
Test: Run add_img_to_target_files with and without "-v" and check
the logs.

Change-Id: Icdc86197ce69db036658ea9e77cc4ebe1750463b
2017-09-09 03:42:31 +00:00
Bowgo Tsai
3e599ead66 AVB: support chain partition signing
Current build system will include AVB metadata from each partition and
store them into /vbmeta partiton when BOARD_AVB_ENABLE is set, which makes
each partition tightly-coupled.

Add the support for 'chain partition':
  - The vbmeta of each partition is stored on the same partition itself.
  - The public key used to verify each partition is stored in /vbmeta.

For example, the following build variables are required to enable chain
partition for system partition:
  - BOARD_AVB_SYSTEM_KEY_PATH := path/to/system_private_key
  - BOARD_AVB_SYSTEM_ALGORITHM := SHA512_RSA8192
  - BOARD_AVB_SYSTEM_ROLLBACK_INDEX := 1
  - BOARD_AVB_SYSTEM_ROLLBACK_INDEX_LOCATION := 2

The corresponding settings will be added into META/misc_info.txt for
build_image.py and/or add_img_to_target_files.py:
  - avb_system_key_path=path/to/system_private_key
  - avb_system_algorithm=SHA512_RSA8192
  - avb_system_add_hashtree_footer_args=--rollback_index 1
  - avb_system_rollback_index_location=2

To enable chain partition for other partitions, just replace SYSTEM with
BOOT, VENDOR and/or DTBO in the build variables.

Also switch from  `avbtool make_vbmeta_image --setup_rootfs_from_kernel system.img ...`
to `avbtool add_hashtree_footer --image system.img --setup_as_rootfs_from_kernel...`
when BOARD_BUILD_SYSTEM_ROOT_IMAGE is true. This works for both chained
and no-chained:
  - chained: `avbtool add_hashtree_footer --setup_as_rootfs_from_kernel` will
    add dm-verity kernel cmdline descriptor to system.img
  - no-chained: `avbtool make_vbmeta_image --include_descriptors_from_image
    system.img` will include the kernel cmdline descriptor from system.img into
    vbmeta.img

Bug: 38399657
Test: `make` pass, flash images from $OUT and boot device without chain partitions
Test: `make` pass, flash images from $OUT and boot device with chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device without chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device with chain partitions
Test: follow the same steps in
      https://android-review.googlesource.com/#/c/407572/

Change-Id: I344f79290743d7d47b5e7441b3a21df812a69099
2017-06-13 17:45:10 +08:00
Tianjie Xu
65b98cd3b1 Merge "Generate care_map.txt when AVB is enabled" 2017-06-02 18:09:58 +00:00
Tianjie Xu
6b2e155ee6 Generate care_map.txt when AVB is enabled
When AVB is enabled, generate care_map.txt and add it to the target
files. Also copy it into the OTA package where it will later be used
by the update_verifier.

Bug: 62208947
Test: \
1. Run add_img_to_target_files on the TF of a new pixel device,
and care_map.txt generates successfully.

2. Make dist in oc-dr1-release and find care_map.txt in the OTA package.

3. update_verifier succeeds in reading all the blocks on the care_map,
and fails to read out-of-bound blocks.

Change-Id: I2881711e6f87789cb7de150dbeca18b756fed68a
2017-06-01 21:36:43 -07:00
Bowgo Tsai
7ea994b21c Support signing custom images with AVB HASH or AVB HASHTREE
`make custom_images` supports to build different kinds of *non-droid* images,
e.g., odm.img. Adding the support of signing them with either AVB HASH footer
or AVB HASHTREE footer. The user can use HASH for small images and
HASHTREE for large images.

Sample signing configurations:
 * AVB HASH footer:
   - CUSTOM_IMAGE_AVB_HASH_ENABLE := true
   - CUSTOM_IMAGE_AVB_ADD_HASH_FOOTER_ARGS := --append_to_release_string my_odm_image

 * AVB HASHTREE footer:
   - CUSTOM_IMAGE_AVB_HASHTREE_ENABLE := true
   - CUSTOM_IMAGE_AVB_ADD_HASHTREE_FOOTER_ARGS := --fec_num_roots 8

 * Using custom signing key:
   - CUSTOM_IMAGE_AVB_ALGORITHM := SHA256_RSA2048
   - CUSTOM_IMAGE_AVB_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem

Bug: 36701014
Test: `make custom_images` with AVB HASH footer
Test: `make custom_images` with AVB HASHTREE footer
Test: `make droid` to check system.img is still properly signed with AVB HASHTREE
Test: `make droid` to check vendor.img is still properly signed with AVB HASHTREE

Change-Id: I8dc420e12e37e9a631345c0cd883339db05d489f
2017-05-24 14:21:03 +08:00
Julius D'souza
001c676b81 Add the option to reserve headroom for partition images.
This is useful for devices with low disk space with different
build variants.

Bug: 37469715
Test: Regular image builds successfully, errors occur when
the headroom size is greater than available partition space.

Change-Id: I526cdd0f84981bbd16e3afcfe1cd7fc43dce98ef
2017-05-08 11:59:25 -07:00
Tianjie Xu
e3ad41bb2f Switch command to subprocess in build_image.py
Bug: 36012162
Test: image builds successfully for bullhead and sailfish

Change-Id: Iaa83034b39d392ff8a2154c7b32b21bf33ef552f
2017-03-10 15:16:24 -08:00
Connor O'Brien
20f08c3e16 Pass flash erase & logical block size to mke2fs
This information can be used to tune ext4 stripe and stride in the
userdata partition for better performance

Test: Build & flash userdata, confirm correct stripe & stride values
Bug: 33243520
Merged-In: Ia97cdd2d0239c3484b895fce49299f692ef911d8
Change-Id: Ia97cdd2d0239c3484b895fce49299f692ef911d8
Signed-off-by: Connor O'Brien <connoro@google.com>
2017-01-13 12:17:05 -08:00
Patrick Tjin
a1900841bb build_image: add support for specifying number of inodes
Add support for specifying number of inodes when creating
system, vendor, oem partitions.  These are all read-only
and have no use for extra inodes.  Removing extra inodes
saves a lot of space.

Bug: 32246383
Change-Id: I13f1d4614b64a4abc752c42a1c65d3d151481c21
(cherry picked from commit b59eca3586)
2016-12-06 21:07:23 -08:00
The Android Automerger
ada51f3b8f Manually merge commit 'c13da91' into stage-aosp-master
* commit 'c13da91':
  Add support for creating ext4 images with mke2fs

BUG:33304034
2016-12-03 04:02:09 +00:00
Adrien Schildknecht
9a072cc0b0 Add support for creating ext4 images with mke2fs
We are investigating replacing make_ext4fs with the upstream tool mke2fs.
To mitigate the trouble that may arise if the new tool behave differently
compared to the old one, there will be a transition period.

Devices that want to use the new way of creating ext4 images can set the
variable "TARGET_USES_MKE2FS" to true in their BoardConfig.mk
By default, the build system will choose the old tool 'make_ext4fs'.

Test: m otapackage with TARGET_USES_MKE2FS={,false,true}
Change-Id: I282bcb9efe335a86c53986283090ca947d65c7f8
2016-11-18 17:06:29 -08:00
Tao Bao
f4a88279ed Merge "releasetools: Support verity signer args."
am: 02d2d68aec

Change-Id: Ib1aed8ff11eeef2fd64a6520e92cfb30678dadc7
2016-10-24 18:53:33 +00:00
Tao Bao
458104266f releasetools: Support verity signer args.
system/extras/verity/build_verity_metadata.py now accepts
"--signer_args" to specify verity signer args.

Also remove the duplicate "--verity_signer_args" in
add_img_to_target_files.py, as we already have that in common.py.

Bug: 31500665
Test: Building and signing work w/ and w/o --signer_args.

Change-Id: I02f59c50a1ebf15c5505e9fffd5b9bbbbaa785be
2016-10-17 16:25:22 -07:00
David Zeuthen
4e9c89ab87 resolve merge conflicts of 86180eb to nyc-mr1-dev-plus-aosp
Change-Id: Ia4bfb2571bac509cebe22f70ff43e6696f832423
2016-10-04 18:53:34 -04:00
David Zeuthen
4014a9daa7 Make room for AVB hashtree and metadata.
While the system.img images currently built with AVB support verify
correctly, mounting the filesystem content fails. This is because
'avbtool add_hashtree_footer' used to claim some of the unused /
DONT_CARE space for stashing the verity tables and this resulting in the
mapped device ending up being smaller causing the mount failure.

Fix this by leaving enough room for AVB hashtree and metadata before
building the image. This is achieved by moving the AVB hashtree support
into build_image.py and using a just added '--calc_max_image_size'
option to 'avbtool add_hashtree_footer' to figure out how much space to
leave out.

This depends on https://android-review.googlesource.com/#/c/281821/

Bug: 31264226
Test: Mounting dm-verity set up from system.img now works.

Merged-In: I4c5de1004c1059f8c582e76b3b8517d427aa1a87

Change-Id: I945a5f1f6782791736cd319f216cfa6b448fb04d
2016-10-04 17:32:16 -04:00
Sami Tolvanen
3d207c8889 Merge "build_image: add padding to match partition size" am: ef5751c1ec am: 5875285ecf
am: e261a2dc20

Change-Id: Ib396abb9f6707c984b2888b8b5a265d8ebcf76be
2016-09-02 18:24:43 +00:00
Sami Tolvanen
433905f50e build_image: add padding to match partition size
For some partition sizes, we currently build an image that's 1-2
blocks smaller than the actual partition, which causes fs_mgr to
not find metadata. This change adds padding to FEC metadata that
correctly positions the metadata header at the end.

Bug: 28865197
Change-Id: Ie0e044715a9c5ae8ba395e7d2ff9fbd7cffc0b4c
2016-09-02 10:20:02 -07:00
Alex Light
a3e8e9c6f3 resolve merge conflicts of 3f7c348 to nyc-mr1-dev-plus-aosp
Bug: 29278988

Change-Id: Icffb9a280311c7d3c45f096243ddf18f7ebecd9d
2016-06-29 15:43:54 -07:00
Alex Light
4e358ab2c3 Add system_other partition, install odex files
For AB devices, support flashing two system partitions for factory use.
The normal system image on one partition, but without dex preopt. And a
system_other image that just contains the odex files. The dex files will
not be stripped out of the system image, in case the second system
partition is wiped.

Setting BOARD_USES_SYSTEM_OTHER_ODEX := true in the BoardConfig.mk
enables this behavior.

One can control which directories are placed in system_other by the
SYSTEM_OTHER_ODEX_FILTER configuration variable. Currently we default
to only copying only app and priv-app odexs.

Bug: 29278988
Change-Id: I7f4e87da919e7dc6a89fd8c668193cd4e98631bc
2016-06-24 11:07:15 -07:00
Elliott Hughes
2d208075c6 Merge \\\"Remove obsolete MTD support from the releasetools scripts.\\\" am: 63fc2b6766 am: ece6ff869c
am: f66d3e61f8

Change-Id: Ib754bc3e9b1c895d431e3b7be9e5435bb6b05fb5
2016-06-17 05:05:08 +00:00
Elliott Hughes
f66d3e61f8 Merge \\"Remove obsolete MTD support from the releasetools scripts.\\" am: 63fc2b6766
am: ece6ff869c

Change-Id: I4b5f5b6a622a2fbb996861e8832e1fc1d12e6bf4
2016-06-17 05:02:05 +00:00
Elliott Hughes
ece6ff869c Merge \"Remove obsolete MTD support from the releasetools scripts.\"
am: 63fc2b6766

Change-Id: I26ed21479ab52d535bcc99b13e55a1476a02f76c
2016-06-17 04:59:26 +00:00
Elliott Hughes
305b088729 Remove obsolete MTD support from the releasetools scripts.
Bug: http://b/29250988
Change-Id: I653dc306485c6b35411840b53211d42eb6d19e34
2016-06-16 20:48:47 -07:00
Mohamad Ayyash
44dc2a973f resolve merge conflicts of 8ebbc4e to nyc-mr1-dev
am: 423862500b

Change-Id: I8f8dd48e52c1ab7cb3462725569cf7c1e0443d38
2016-06-16 20:22:30 +00:00
Mohamad Ayyash
423862500b resolve merge conflicts of 8ebbc4e to nyc-mr1-dev
Change-Id: I2349bb3fede317487227b16cb7c7b093a93a3876
2016-06-16 13:11:45 -07:00
Mohamad Ayyash
3336f9ae67 Merge \"squashfs: make disable-4k-align a parameter\" into nyc-dev
am: 8ebbc4e7e2

Change-Id: Ia5c59b7bfc7f2b25fa9b13c2ce52c1bd75ccecce
2016-06-16 16:39:39 +00:00
Mohamad Ayyash
1b6d3485fc squashfs: make disable-4k-align a parameter
BUG: 29388879
Change-Id: I121d2b76db5fb269e2f7152abb2e78098437a0d5
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2016-06-15 16:37:21 -07:00
Mohamad Ayyash
c3216ac077 Merge \\"Pass block_list param to mksquashfs\\" into nyc-dev am: 3bddf74752
am: 52cef0d4f0

Change-Id: I0fe16c6b9728acf7b46f412d705fa0aacd2b4655
2016-06-15 17:02:47 +00:00
Mohamad Ayyash
f80e55b821 Merge \"Pass block_list param to mksquashfs\" into nyc-dev
am: 3bddf74752

Change-Id: I713c6a60d10ef1e85475e462bb34c89751f74e7d
2016-06-15 16:58:36 +00:00
Mohamad Ayyash
52cef0d4f0 Merge \"Pass block_list param to mksquashfs\" into nyc-dev
am: 3bddf74752

Change-Id: I1679c255b297a36bc10b50ff9d6214c92d763489
2016-06-15 16:12:55 +00:00
Mohamad Ayyash
c3484f7f64 Pass block_list param to mksquashfs
Needed to generate image.map and to optimize the OTA size

BUG: 22322817
Change-Id: I7802f4dbb9116a94ea99a00b68c3a7ff180ef08a
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2016-06-13 09:48:05 -07:00