Commit graph

79 commits

Author SHA1 Message Date
Mohamad Ayyash
423862500b resolve merge conflicts of 8ebbc4e to nyc-mr1-dev
Change-Id: I2349bb3fede317487227b16cb7c7b093a93a3876
2016-06-16 13:11:45 -07:00
Mohamad Ayyash
1b6d3485fc squashfs: make disable-4k-align a parameter
BUG: 29388879
Change-Id: I121d2b76db5fb269e2f7152abb2e78098437a0d5
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2016-06-15 16:37:21 -07:00
Mohamad Ayyash
f80e55b821 Merge \"Pass block_list param to mksquashfs\" into nyc-dev
am: 3bddf74752

Change-Id: I713c6a60d10ef1e85475e462bb34c89751f74e7d
2016-06-15 16:58:36 +00:00
Mohamad Ayyash
c3484f7f64 Pass block_list param to mksquashfs
Needed to generate image.map and to optimize the OTA size

BUG: 22322817
Change-Id: I7802f4dbb9116a94ea99a00b68c3a7ff180ef08a
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2016-06-13 09:48:05 -07:00
Mohamad Ayyash
dfec815882 Pass squashfs block size as a parameter
BUG: 28920352
Change-Id: Ifed02bd1565131814047991ec87fa61691bfe240
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2016-05-24 20:55:13 +00:00
Mohamad Ayyash
8837882fb9 Use fs_config files for mksquashfs
BUG: 27467028
Change-Id: I7648030ad4b31d70a5d439e9552fd2cbfe288b74
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2016-04-07 22:10:51 -07:00
Mohamad Ayyash
f876555893 Hook incremental ext4 image generation into build
Allows passing the *.base_fs or *.map files for system and vendor in
INTERNAL_SYSTEM_BASE_FS_PATH and INTERNAL_VENDOR_BASE_FS_PATH variables
respectively.

Internal Design Doc: go/incremental-ext4
BUG: 26839493
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>

Change-Id: Ie57ef6dbfa6084268b6535fe0a673a3b4aaa6e2f
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2016-03-03 12:07:21 -08:00
Sami Tolvanen
405e71dcd3 Fix metadata location when file system doesn't span the partition
Pad the sparse image with a zero fill chunk to correctly position
verity and FEC metadata at the end of the partition.

Bug: 27073791
Change-Id: I9f70d579a42e5007d50e9c02a98a608d2815f0ed
(cherry picked from commit 6a8781a251)
2016-02-10 10:53:47 -08:00
Patrick Tjin
e11aa50bb2 Add compressor options for squashfs vendor images
Change-Id: I4f9f5d62e24654cb37d436cb73dd5893aed30fa8
2016-02-09 15:41:07 -08:00
Sami Tolvanen
4a0600406e build_image: don't call append2simg for FEC
append2simg can be slow, so generate error correction data before verity
data is appended, and append all metadata to the image at the same time.

Needs matching changes from
  I09349368925e5e2f2a7961a712a1e933eb0b9ec8

Bug: 26251929
Change-Id: Iea8cee0aee353b4d029d9581e8fbd68ec7b7dd4b
2015-12-18 15:50:25 +00:00
Sami Tolvanen
ff914f5dd0 build_image: remove an unnecessary call to append2simg
append2simg on a large image is fairly slow, so append verity metadata to
the hash tree to avoid two calls.

Bug: 26251929
Change-Id: I5ec84dcd6e8500e990af13b1f9cef5e809b41605
2015-12-18 13:28:25 +00:00
Todd Poynor
b2a555e6fc Make squashfs sparse image generation optional
Allow non-sparse squashfs image generation, a la ext filesystems.

Bug: 26218766
Change-Id: Ifd596af16411e0cf69a2e7e05d725fd9442dc79c
2015-12-15 20:11:12 -08:00
Sami Tolvanen
f99b53143d Error correction: Append codes to verified partitions
Append error-correcting codes to verified partitions provided that
PRODUCT_SUPPORTS_VERITY_FEC is true.

This moves verity metadata to be after the hash tree, and requires
matching changes from
  Ide48f581bbba77aed6132f77b309db71630d81ed

Bug: 21893453
Change-Id: I6945cbab99e214566a1f9d3702333f2dbbc35816
2015-10-14 18:06:20 +01:00
Tao Bao
ce9635ec7a am 2124d7e1: Merge "Only use a fixed timestamp when packaging."
* commit '2124d7e1e2dec91ce4ab15c6a215d61f7693a8d6':
  Only use a fixed timestamp when packaging.
2015-09-30 23:50:49 +00:00
Tao Bao
822f5840c0 Only use a fixed timestamp when packaging.
[1] changed to use a fixed timestamp for all the generated images
(either the one under $ANDROID_PRODUCT_OUT/ or the one added into the
target_files zip). It makes 'adb sync' after 'fastboot flashall' to
resync all the files. This CL changes to use a fixed timestamp only when
packaging into the target_files zip.

[1]: commit 052ae3542b

Bug: 24377993
Bug: 24536411
Change-Id: I209318966109fd4902468dfe96caf777464f9fe1
2015-09-30 16:01:14 -07:00
Tao Bao
9a5afa5242 am 5e23069b: Merge "build_image.py: Use a fixed timestamp in the generated image."
* commit '5e23069bceebfbf17f67c709bad495d707b194dc':
  build_image.py: Use a fixed timestamp in the generated image.
2015-09-28 21:16:34 +00:00
Tao Bao
052ae3542b build_image.py: Use a fixed timestamp in the generated image.
Bug: 24377993
Change-Id: I9b1433c8c877dc2a391615bb016b8970426e9f70
2015-09-28 13:44:13 -07:00
Simon Wilson
37700c9ad1 Merge "Add compressor options for squashfs system images" 2015-07-29 21:35:57 +00:00
Tao Bao
1358123688 am a04e6b89: am dd68e68f: Merge "Override properties that point to out/ directory."
* commit 'a04e6b89ec262f4dd38c9d80ee05ce313bab9347':
  Override properties that point to out/ directory.
2015-07-21 18:25:36 +00:00
Tao Bao
84e7568b1f Override properties that point to out/ directory.
A few properties are stored as links to the files in the out/
directory, which are no longer available when (re)packing from
target_files zip. Redirect those properties to the actual files in
the unzipped directory when repacking.

Change-Id: I18e930cbea224ab9c6db8d4e721b1c266a671db6
2015-07-19 08:48:09 -07:00
Thierry Strudel
74a81e6633 fix device specific fs_config on build server
fs_config function from libcutils, in order to properly set the xattr FS
used to define device specific uid, gid, permissions and capabilities,
now needs TARGET_OUT make variable to be passed explicitly to all tools
using it:
  fs_config
  fs_get_stats used by mktarball.sh
  make_ext4fs used by mkuserimg.sh
  mksquashfs used by mksquashfsimage.sh

Bug: 21989305
Bug: 22048934
Change-Id: I6caf9cf870882fce2ead93027767092c29b75ded
Signed-off-by: Thierry Strudel <tstrudel@google.com>
2015-07-10 10:55:54 -07:00
Simon Wilson
011ea06db7 Add compressor options for squashfs system images
Change-Id: I2a75f8754273f6e00b96ba6aca6741c4388f5da8
(cherry picked from commit f86e7eeabc)
2015-07-07 11:41:20 -07:00
Tao Bao
4e66343b2b Retry adjusting the size computation for reserved blocks.
Due to the change in https://lwn.net/Articles/546473/, kernel reserves a
few extra blocks (lesser of 2% and 4096 blocks) on ext4 FS which leads to
OTA update failures. Adjust the size computation if the device has
BOARD_HAS_EXT4_RESERVED_BLOCKS := true.

It amends the last attemp in [1]. Now it computes the used blocks from the
make_ext4fs output, instead of altering its argument.

[1]: commit efbb5d2e69.

Bug: 21522719
Bug: 22023465
Bug: 22174684
Change-Id: Iaae6507f6de68a5892f2e3035d330039287b4492
(cherry picked from commit c7a6f1e4f8)
2015-06-30 12:17:28 -07:00
Tao Bao
c7a6f1e4f8 Retry adjusting the size computation for reserved blocks.
Due to the change in https://lwn.net/Articles/546473/, kernel reserves a
few extra blocks (lesser of 2% and 4096 blocks) on ext4 FS which leads to
OTA update failures. Adjust the size computation if the device has
BOARD_HAS_EXT4_RESERVED_BLOCKS := true.

It amends the last attemp in [1]. Now it computes the used blocks from the
make_ext4fs output, instead of altering its argument.

[1]: commit efbb5d2e69.

Bug: 21522719
Bug: 22023465
Bug: 22174684
Change-Id: I9783a51abe6581ff5c75db81e78ac606d0f32c4c
2015-06-29 20:45:53 -07:00
Tao Bao
2384d01c1c Revert "Adjust the size computation for reserved blocks"
We should not change the parameter to mkfs_ext4. Bug filed at
b/22174684. Will come up with a new CL.

This reverts commit efbb5d2e69.

Change-Id: I15f7a9a07a65a936ba186d2ce672ec4832ed3e17
2015-06-29 12:55:38 -07:00
Ying Wang
8f180b904f Merge "Create userdata.img with real data when SANITIZE_TARGET=address." 2015-06-25 21:29:52 +00:00
Ying Wang
2a04839aec Create userdata.img with real data when SANITIZE_TARGET=address.
Bug: 21785137
Change-Id: Ie0c36988759fe07419ad29bab5a71cdd0d992d2a
2015-06-25 14:29:04 -07:00
Mohamad Ayyash
07a7590f08 Merge "build_image.py: Make sparse squashfs images" into mnc-dev 2015-06-24 23:15:56 +00:00
Simon Wilson
6d5ddc32ab resolved conflicts for merge of ecfc97f7 to mnc-dev
Change-Id: Ic7661c4176dc87cb2796f9cbb61bdd64f89cdc01
2015-06-24 16:06:34 -07:00
Simon Wilson
ecfc97f772 resolved conflicts for merge of f86e7eea to lmp-mr1-ub-dev
Change-Id: I746021aa57edd5f6fae6e57cec0908951a659664
2015-06-24 15:42:27 -07:00
Mohamad Ayyash
042e851999 Merge "build_image.py: Make sparse squashfs images" 2015-06-24 22:01:00 +00:00
Mohamad Ayyash
2cd51cc711 build_image.py: Make sparse squashfs images
Change-Id: Iea0399868a9406f9929d8176d61d5698a2043697
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2015-06-24 11:08:00 -07:00
Mohamad Ayyash
fa6c8a9b79 build_image.py: Make sparse squashfs images
Change-Id: Iea0399868a9406f9929d8176d61d5698a2043697
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2015-06-24 10:44:52 -07:00
Tao Bao
efbb5d2e69 Adjust the size computation for reserved blocks
Due to the change in https://lwn.net/Articles/546473/, kernel reserves a
few extra blocks (lesser of 2% and 4096 blocks) on ext4 FS which leads to
OTA update failures. Adjust the size computation if the device has
BOARD_HAS_EXT4_RESERVED_BLOCKS := true.

Bug: 21522719
Bug: 22023465
Change-Id: I49f16adbf2dedc5279fbb8622bf99ef71dcc494f
2015-06-23 14:53:28 -07:00
Simon Wilson
f86e7eeabc Add compressor options for squashfs system images
Change-Id: I2a75f8754273f6e00b96ba6aca6741c4388f5da8
2015-06-22 15:42:52 -07:00
Mohamad Ayyash
cade49ddee Allow flashing a squashfs system image
Change-Id: Ic51d11274784a8d2e9d57d5f0b821ebc89680e5a
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2015-06-16 13:11:21 -07:00
Baligh Uddin
601ddeae06 Allow custom boot_signer and verity_signer.
Bug: 21701163
Change-Id: I85e75fd3a166a9f6a7859ebf81dee73322dc2798
2015-06-09 15:48:14 -07:00
Ying Wang
4540a85dd4 Support to configure and build multiple custom images.
Build additional images requested by the product makefile.
This script gives the ability to build multiple additional images and
you can configure what modules/files to include in each image.
1. Define PRODUCT_CUSTOM_IMAGE_MAKEFILES in your product makefile.
   PRODUCT_CUSTOM_IMAGE_MAKEFILES is a list of makefiles.
   Each makefile configures an image.
   For image configuration makefile foo/bar/xyz.mk, the built image
   file name
   will be xyz.img. So make sure they won't conflict.
2. In each image's configuration makefile, you can define variables:
  - CUSTOM_IMAGE_MOUNT_POINT, the mount point, such as "oem", "odm"
    etc.
  - CUSTOM_IMAGE_PARTITION_SIZE
  - CUSTOM_IMAGE_FILE_SYSTEM_TYPE
  - CUSTOM_IMAGE_DICT_FILE, a text file defining a dictionary
    accepted by BuildImage() in tools/releasetools/build_image.py.
  - CUSTOM_IMAGE_MODULES, a list of module names you want to include
    in the image; Not only the module itself will be installed to proper
    path in the image, you can also piggyback additional files/directories
    with the module's LOCAL_PICKUP_FILES.
  - CUSTOM_IMAGE_COPY_FILES, a list of "<src>:<dest>" to be copied to
    the image. <dest> is relativ to the root of the image.

To build all those images, run "make custom_images".

Bug: 19609718
Change-Id: Ic73587e08503a251be27797c7b00329716051927
(cherry picked from commit 5fcf1094f9)
2015-06-03 09:56:29 -07:00
Tao Bao
f3282b4a7f Wrap zipfile.write(), writestr() and close()
In order to work around the zip 2GiB limit, we need to wrap the related
functions in zipfile. Calls to those functions should always be replaced
with calls to the wrappers instead.

Bug: 18015246
Change-Id: I499574cee51ec4804bc10cbefe0b17940afed918
(cherry picked from commit 2ed665a033)
2015-05-08 15:53:27 -07:00
Tao Bao
2ed665a033 Wrap zipfile.write(), writestr() and close()
In order to work around the zip 2GiB limit, we need to wrap the related
functions in zipfile. Calls to those functions should always be replaced
with calls to the wrappers instead.

Bug: 18015246
Change-Id: Ice494371ca6654e88ded2ae0eb680f51082effcb
2015-05-08 13:51:12 -07:00
Mohamad Ayyash
e71b55606e am 347195da: am 6ddc3600: Merge "build_image.py: Verity support for flashing non partition spanning fs"
* commit '347195da2ea801db95cf201fa7516bf392dc005a':
  build_image.py: Verity support for flashing non partition spanning fs
2015-04-02 18:37:05 +00:00
Mohamad Ayyash
dd063528ed build_image.py: Verity support for flashing non partition spanning fs
Change-Id: Ic5f7b256ed9a31e99ebaebdf8a8a19ba9adcced3
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2015-04-01 00:50:39 +00:00
Dan Albert
eecf00db59 resolved conflicts for merge of 4c32aa3d to lmp-mr1-dev-plus-aosp
Change-Id: I32a06c88416e68ce628f642e0d025d1df5e227d7
2015-03-27 14:37:14 -07:00
Dan Albert
99e22a57d0 Merge "Make releasetools pylint clean." 2015-03-27 18:40:02 +00:00
Ying Wang
52ea2066a1 am abc748c1: am 6ff1b035: Merge "Support to build image of root file system with /system and ramdisk combined."
* commit 'abc748c11cbba71a99f306d318c74b5336b28730':
  Support to build image of root file system with /system and ramdisk combined.
2015-03-26 17:57:22 +00:00
Ying Wang
a2292c94d1 Support to build image of root file system with /system and ramdisk combined.
Added support to build system.img that combines contents of /system and
the ramdisk, and can be mounted at the root of the file system.
To enable this feature, define BoardConfig.mk variable:
BOARD_BUILD_SYSTEM_ROOT_IMAGE := true

Ideally we would just change TARGET_OUT (the path of the staging system
directory) to under TARGET_ROOT_OUT. But at this point many places in
the build system assume TARGET_OUT is independent of TARGET_ROOT_OUT and
we can't make it easily configurable.
Instead this implementation takes the least intrusive approach:
We don't change TARGET_OUT or TARGET_ROOT_OUT. We just assemble a
temporary staging directory that contains contents of both TARGET_OUT
and TARGET_ROOT_OUT, in build_image.BuildImage() of
tools/releasetools/build_image.py.
When build_image.py is directly called from the makefile, we pass in the
parameters from the global dictionary; when build_image.BuildImage() is
called from add_img_to_target_files.py, we need to override values to
point to files extracted from the target_files zip file.
We need to combine the fs_config files of both /system and ramdisk,
when fs_config is enabled.

Also this change refactored build_image.BuildImage() by moving the extra
parameters to the image property dictionary.

(cherry-picked from commit 0eabd4f2c5)

Bug:19868522
Change-Id: Iafc467a0e3427b0d6ad3b575abcc98ddcc9ea0f1
2015-03-26 10:35:00 -07:00
Dan Albert
8b72aefb5a Make releasetools pylint clean.
This caught a few bugs/syntax errors (a few character classes were not
escaped properly in regex patterns, some indentation was illegal,
etc).

Change-Id: I50637607524e68c4fb9cad7167f58a46b8d26b2c
2015-03-24 11:05:16 -07:00
Ying Wang
d4ca3b38d9 am 4cbc4b39: am ae61f50a: Support to configure and build multiple custom images.
* commit '4cbc4b392da57c34626af38a4ea0fe4dc115af57':
  Support to configure and build multiple custom images.
2015-03-14 00:31:44 +00:00
Ying Wang
ae61f50a68 Support to configure and build multiple custom images.
Build additional images requested by the product makefile.
This script gives the ability to build multiple additional images and
you can configure what modules/files to include in each image.
1. Define PRODUCT_CUSTOM_IMAGE_MAKEFILES in your product makefile.
   PRODUCT_CUSTOM_IMAGE_MAKEFILES is a list of makefiles.
   Each makefile configures an image.
   For image configuration makefile foo/bar/xyz.mk, the built image
   file name
   will be xyz.img. So make sure they won't conflict.
2. In each image's configuration makefile, you can define variables:
  - CUSTOM_IMAGE_MOUNT_POINT, the mount point, such as "oem", "odm"
    etc.
  - CUSTOM_IMAGE_PARTITION_SIZE
  - CUSTOM_IMAGE_FILE_SYSTEM_TYPE
  - CUSTOM_IMAGE_DICT_FILE, a text file defining a dictionary
    accepted by BuildImage() in tools/releasetools/build_image.py.
  - CUSTOM_IMAGE_MODULES, a list of module names you want to include
    in the image; Not only the module itself will be installed to proper
    path in the image, you can also piggyback additional files/directories
    with the module's LOCAL_PICKUP_FILES.
  - CUSTOM_IMAGE_COPY_FILES, a list of "<src>:<dest>" to be copied to
    the image. <dest> is relativ to the root of the image.

To build all those images, run "make custom_images".

Bug: 19609718
Change-Id: Ic73587e08503a251be27797c7b00329716051927
(cherry picked from commit 5fcf1094f9)
2015-03-14 00:06:54 +00:00
Mohamad Ayyash
9b19382f0e resolved conflicts for merge of fc759204 to stage-aosp-master
Change-Id: Icf58e881286a66cb3ca4cb122fa8c11ec217fbab
2015-03-04 15:43:45 -08:00