Commit graph

5061 commits

Author SHA1 Message Date
Tao Bao
4a5a1e15a1 Merge "Consider prebuilt vendor.img as having vendor partition."
am: 8a7b0dee6b

Change-Id: I24451a308365873432eafd3fdc30403a9514ef7e
2017-09-14 00:25:48 +00:00
Tao Bao
b22afea0f0 Consider prebuilt vendor.img as having vendor partition.
vendor.img could be built from the source - in which case we have
unpacked files under VENDOR/ - or dropped in as a prebuilt binary blob.
We should consider either of them as target having a vendor partition.
Because we need to add its AVB hashtree info into vbmeta.img if target
is using AVB. Otherwise libfs_mgr would refuse to mount this
"AVB-enabled" vendor.img.

For targets not using AVB, this change is no-op.

Bug: 65462819
Test: Having vendor.img as prebuilt, `make dist`. Check that the
      generated vbmeta.img contains the info from vendor.img.
Test: Build, flash and boot the above image.zip.
Change-Id: Iaeb30e2059cb33fb39f23e5ffd28f338d00ccbfc
2017-09-13 11:05:15 -07:00
Tianjie Xu
b34a5ce36e Merge "Add a verbose option to RunCommand in build_image.py"
am: ab61b6b895

Change-Id: I7ebc5a522a76134576ab3f7ed61e2f6916d179da
2017-09-11 20:04:00 +00:00
Tianjie Xu
ab61b6b895 Merge "Add a verbose option to RunCommand in build_image.py" 2017-09-11 19:22:38 +00:00
Dan Willemsen
8c3849a15d Merge "Convert to Android.bp"
am: 65631cf079

Change-Id: I22e6fecccd465e5f2a962168ee78c241ecb31aa2
2017-09-09 22:42:58 +00:00
Dan Willemsen
c0a0f2dc1f Convert to Android.bp
Test: m atree fs_config fs_config_generate_test fs_config-unit-tests fs_get_stats zipalign
Test: cd out/host/linux-x86/nativetest64/fs_config-unit-tests ./fs_config-unit-tests
Change-Id: I3c5403a7f1cf221715eb52a41c640af9afcee175
2017-09-09 13:47:34 -07:00
Tianjie Xu
149b7fbf08 Add a verbose option to RunCommand in build_image.py
Make the build log less verbose; also print the result for
AdjustPartitionSizeForVerity().

Bug: 65292710
Test: Run add_img_to_target_files with and without "-v" and check
the logs.

Change-Id: Icdc86197ce69db036658ea9e77cc4ebe1750463b
2017-09-09 03:42:31 +00:00
Chih-Hung Hsieh
6dfe4bc16a Merge "Accept alternative email addresses."
am: 0b14739e0f

Change-Id: I32c6c40c84796601dbd7392c14175c3bda7aa920
2017-09-07 22:32:09 +00:00
Chih-Hung Hsieh
2b1efe63cf Accept alternative email addresses.
Some OWNERS files use secondary email addresses, so we only
check existence of _account_id and email attributes.

Test: run against existing OWNERS files
Change-Id: I6f0804a8a90b7a23783c3fe45686d4d973876ed1
2017-09-07 13:07:26 -07:00
Tao Bao
6b8eeb0bf4 Merge "Skip checking files that have less blocks in block map."
am: aa599f3916

Change-Id: I11ea419d275b5af8f0b095377fbdb19fee465c1d
2017-09-07 17:56:50 +00:00
Tao Bao
aa599f3916 Merge "Skip checking files that have less blocks in block map." 2017-09-07 17:50:51 +00:00
Tao Bao
d5ec31f8a9 Merge "releasetools: Validate A/B OTA payload signatures."
am: 631b3a031c

Change-Id: I3abd410e0709cd44b2e2b74471e3cc023dba7cd5
2017-09-06 20:48:14 +00:00
Tao Bao
b418c30e3a Skip checking files that have less blocks in block map.
When creating ext4 images with mke2fs, it may skip allocating some
blocks if they contain all zeros. As a result, there could be less
blocks listed in the block map than the actual file length.

For example, for a file with a length of 112200-byte (27+ blocks),
the listed blocks in block.map could be '43665-43688' (24 blocks).
Because some all-zero blocks are not taking actual space.

The generated ext4 images are perfectly valid - kernel will figure out
that data block is not allocated and writes all zeros into user buffer.
However, we can't fully reconstruct a file from its block list in our
Python script. Ideally this can be avoided by mounting or parsing an
ext4 image directly, which is yet to be supported in our script.

This CL skips checking for such files to avoid failing
validate_target_files.py.

Bug: 65213616
Test: validate_target_files.py passes on targets with mke2fs generated
      images (e.g. marlin).
Change-Id: Id9cc59e345b9283844044ef94ceb5702f0ca0526
2017-09-05 15:53:24 -07:00
Tao Bao
a198b1e964 releasetools: Validate A/B OTA payload signatures.
$ PYTHONPATH=$PYTHONPATH:system/update_engine/scripts \
    ./build/make/tools/releasetools/check_ota_package_signature.py \
    build/target/product/security/testkey.x509.pem \
    out/dist/aosp_marlin-ota-eng.zip

Package: out/dist/aosp_marlin-ota-eng.zip
Certificate: build/target/product/security/testkey.x509.pem
...
Whole package signature VERIFIED

Verifying A/B OTA payload signatures...
...
Payload signatures VERIFIED

Bug: 65261072
Test: Signed a package and its payload with the right keys; ran the
      command above.
Test: Signed the payload with a different key; ran the command above and
      observed the reported verification failure.
Change-Id: If626ecb327a9826cd0956eef94914c939068a7d1
2017-09-01 14:33:15 -07:00
Tao Bao
7313d99f00 Merge "Add compatibility metadata for Treble-enabled non-A/B OTA packages."
am: 02b158d7a4

Change-Id: I54e64fcd3fbff266f8082a4347432941a8c6e8fe
2017-08-29 19:20:08 +00:00
Tao Bao
bcd1d161e8 Add compatibility metadata for Treble-enabled non-A/B OTA packages.
Commit 21803d3574 only added compatibility
metadata for A/B OTA packages, because by that time we didn't use
'ro.treble.enabled' property to determine if a target was Treble-enabled.

This CL uses 'ro.treble.enabled' to guard the packing of compatibility
metadata for both of A/B and non-A/B OTA packages. It also switches to
checking the system/vendor fingerprints to determine if there's an
update to the partition (previously it was computing the SHA-1 of the
images, which may have unintentionally changed due to issues that give
non-repetitive builds).

Bug: 64339310
Test: Generate OTA packages (full and incremental) on Treble-enabled
      targets (sailfish, as well as a non-A/B angler target with
      Treble-enabled locally); check that the compatibility.zip entry
      exists.
Test: Generate OTA packages on Treble-unenabled non-A/B target (angler);
      check that the compatibility.zip entry doesn't exist.
Test: Generate OTA packages on Treble-unenabled A/B target; check that
      the compatibility.zip entry doesn't exist.

Change-Id: I2a1fcf612439d849ba8ccea217a0faf5d5ba8e14
2017-08-28 15:19:44 -07:00
Tao Bao
f7b5d5fbaa Merge "Remove file_contexts.bin from non-A/B OTA package."
am: 7a1ed9edef

Change-Id: I8e3a230473e442b5c4d42eb77f2f09457e6ddabd
2017-08-25 23:14:47 +00:00
Tao Bao
ac7b7db12c Remove file_contexts.bin from non-A/B OTA package.
This entry only exists in non-A/B full OTA path. However, this file has
never been loaded by recovery - recovery was only reading the one from
the recovery image (which also has been removed by a recent change in
[1]).

[1]: commit e35926e1aff2e6b9b54656bd59c8178e295a1b7e in
platform/bootable/recovery.

Bug: 63538434
Test: `m dist` on angler. file_contexts.bin is gone from full OTA
      package. Sideloading the generated full OTA on angler.
Change-Id: Iec2e2d36599fa1ed499516a3e74f06cd36ade494
2017-08-25 14:14:53 -07:00
Jeff Gaston
5b767956fd Merge "Remove the need to find Android.mk files via directory symlinks" am: 6ab53d7211 am: 03e7e81837
am: aed14c5515

Change-Id: I84c8258ebb89eff647cdd52663f812c5600145e1
2017-08-08 00:19:21 +00:00
Jeff Gaston
aed14c5515 Merge "Remove the need to find Android.mk files via directory symlinks" am: 6ab53d7211
am: 03e7e81837

Change-Id: Ib453be719ce27c3f354b1353e45939cee2102aea
2017-08-08 00:16:53 +00:00
Jeff Gaston
03e7e81837 Merge "Remove the need to find Android.mk files via directory symlinks"
am: 6ab53d7211

Change-Id: I0495d45001821d095531d5be7c049fc2836888f7
2017-08-08 00:14:51 +00:00
Jeff Gaston
5c1f3fd068 Remove the need to find Android.mk files via directory symlinks
Bug: 64397960
Test: m -j
Change-Id: I5b77dd654dfced63900913c447b5448b632158d7
2017-08-07 14:21:20 -07:00
Tao Bao
ccce786735 Merge "Look for non-existent files listed in avb_vbmeta_args." am: 5180c3b97b
am: d91be38c5e

Change-Id: Ib9f1508aeb75da3682f2f50341b2ac12a6926a13
2017-07-21 20:54:51 +00:00
Tao Bao
d91be38c5e Merge "Look for non-existent files listed in avb_vbmeta_args."
am: 5180c3b97b

Change-Id: I6a035f46f09f70f11c271693158392f98927aa0f
2017-07-21 20:52:24 +00:00
Tao Bao
9a5f419ecf Look for non-existent files listed in avb_vbmeta_args.
In BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS, if we have defined
"--include_descriptors_from_image" with an image file whose path points
to source tree, add_img_to_target_files.py or sign_target_files_apks.py
may fail to find the file. Because these scripts may run without a
source tree, by taking target_files.zip as the only input.

This CL scans additional locations in the input target_files.zip to find
those missing files in avb_vbmeta_args. As long as the files are included
in the target_files.zip, they get a second chance to be found.

Bug: 63910867
Test: As follows:
 1. Setup BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS with a local file path;
 2. Remove the local file;
 3. sign_target_files_apks.py fails without this CL;
 4. sign_target_files_apks.py works.
Change-Id: I3c58f80a5535db02b74cfe40d0c0beff72587cf8
(cherry picked from commit 1dc5d47653)
2017-07-21 11:57:00 -07:00
Tao Bao
f691e0a869 Merge "Look for non-existent files listed in avb_vbmeta_args." into oc-dr1-dev
am: 9c77abd6f2

Change-Id: I86b3d6fa39a08014d97397322349dc1c389346d5
2017-07-21 17:18:00 +00:00
Tao Bao
1dc5d47653 Look for non-existent files listed in avb_vbmeta_args.
In BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS, if we have defined
"--include_descriptors_from_image" with an image file whose path points
to source tree, add_img_to_target_files.py or sign_target_files_apks.py
may fail to find the file. Because these scripts may run without a
source tree, by taking target_files.zip as the only input.

This CL scans additional locations in the input target_files.zip to find
those missing files in avb_vbmeta_args. As long as the files are included
in the target_files.zip, they get a second chance to be found.

Bug: 63910867
Test: As follows:
 1. Setup BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS with a local file path;
 2. Remove the local file;
 3. sign_target_files_apks.py fails without this CL;
 4. sign_target_files_apks.py works.
Change-Id: I3c58f80a5535db02b74cfe40d0c0beff72587cf8
2017-07-21 01:20:14 -07:00
Tianjie Xu
a4caab580c Merge "Generate brotli compressed {}.new.dat for full OTA" am: 1b28eca1a1 am: a07a04c737
am: 4b53a17756

Change-Id: If03919b9012f4a693e895b02a0c148eeb0a4da5e
2017-07-14 23:34:39 +00:00
Tianjie Xu
4b53a17756 Merge "Generate brotli compressed {}.new.dat for full OTA" am: 1b28eca1a1
am: a07a04c737

Change-Id: I5f2b92791a759def118c6aa70a3e0777a9e8da85
2017-07-14 23:30:32 +00:00
Tianjie Xu
a07a04c737 Merge "Generate brotli compressed {}.new.dat for full OTA"
am: 1b28eca1a1

Change-Id: I3474e8efb634513fbc4876e5c272cf56b3443e12
2017-07-14 23:27:21 +00:00
Tianjie Xu
1b28eca1a1 Merge "Generate brotli compressed {}.new.dat for full OTA" 2017-07-14 23:21:54 +00:00
Tianjie Xu
b0a29ad8c2 Generate brotli compressed {}.new.dat for full OTA
Brotli has a better compression ratio than deflation. So for non-AB
full OTA, we can compress the new.dat with brotli first and store it
in the zip package.
This increase the OTA generation time by ~5 minutes for a full OTA
(measured based on 2.0G system.new.dat)

Bug: 34220646
Test: Generate a full OTA for bullhead
Change-Id: I9c0550af8eafcfa68711f74290c8e2d05a96648f
2017-07-14 10:53:21 -07:00
Tao Bao
f7f27b9002 Merge "Add the missing 'import stat' line." into oc-dr1-dev
am: d43ab80c75

Change-Id: I5d2712c53e9284b96a80ca66cc2b8361e27d3176
2017-07-12 22:21:55 +00:00
Tao Bao
9fdd00f682 Add the missing 'import stat' line.
Accidentally broken by the cherry-pick in commit
f829b40c48 - the original CL in oc-dev
doesn't require the 'import stat' line.

Bug: 63629728
Test: `pylint --rcfile=pylintrc sign_target_files_apks.py`
Test: Successfully sign a build with sign_target_files_apks.py.
Change-Id: I94be613fb2219597148c4339ac33fc93d0cdae47
2017-07-12 12:01:43 -07:00
Tao Bao
55625564c6 Merge "Track the AVB property name change." am: c45161536f
am: fcc2cf6c1e

Change-Id: I9bbed4c45961f2d3fa775b8b977ae58bbaae33ff
2017-07-07 23:35:43 +00:00
Tao Bao
fcc2cf6c1e Merge "Track the AVB property name change."
am: c45161536f

Change-Id: If20f1f0880680b8da185d846a9e0f17139389b50
2017-07-07 23:33:41 +00:00
Tao Bao
48ae05ea77 Track the AVB property name change.
system_avb_hashtree_enable has been renamed to
avb_system_hashtree_enable in commit
3e599ead66. 'care_map.txt' is missing due
to the change.

Bug: 63142730
Test: `m dist` and check IMAGES/care_map.txt exists in target_files zip.
Change-Id: I60c269b41df844b50353d357bf67c20f15548642
(cherry picked from commit 3f72176da0)
2017-07-07 15:02:37 -07:00
Tao Bao
3f72176da0 Track the AVB property name change.
system_avb_hashtree_enable has been renamed to
avb_system_hashtree_enable in commit
3e599ead66. 'care_map.txt' is missing due
to the change.

Bug: 63142730
Test: `m dist` and check IMAGES/care_map.txt exists in target_files zip.
Change-Id: I60c269b41df844b50353d357bf67c20f15548642
2017-06-29 15:19:09 -07:00
Tao Bao
1612f02ff2 Merge "Allow packing custom radio images into target_files zip." am: 9ea6a8b817
am: ce10b6e2f3

Change-Id: I3b6f7ade42f58baa722e2ee3df81efac48a5b653
2017-06-29 03:19:02 +00:00
Tao Bao
ce10b6e2f3 Merge "Allow packing custom radio images into target_files zip."
am: 9ea6a8b817

Change-Id: Ifdcdfa2214e4fa75bb0c7a5df1cb1796814fa7bc
2017-06-29 03:16:17 +00:00
Tao Bao
9ea6a8b817 Merge "Allow packing custom radio images into target_files zip." 2017-06-28 22:03:19 +00:00
Tao Bao
118b3243f8 Merge "Support replacing AVB signing keys." am: 682849976d
am: b5363b2dbc

Change-Id: I8388e8c3c3bf801d7a9d50f111e91555b933a4d7
2017-06-28 19:44:54 +00:00
Tao Bao
b5363b2dbc Merge "Support replacing AVB signing keys."
am: 682849976d

Change-Id: Iedbd4ce27ffdfeb3d88406f75636221739427add
2017-06-28 19:41:25 +00:00
Tao Bao
c218a470fb Support replacing AVB signing keys.
sign_target_files_apks.py now supports signing targets using verified
boot 2.0 (aka AVB). It allows replacing AVB signing keys for both of
chained and non-chained partitions.

An example command line looks as follows.
 $ ./build/tools/releasetools/sign_target_files_apks.py \
     --avb_vbmeta_key external/avb/test/data/testkey_rsa2048.pem \
     --avb_vbmeta_algorithm SHA256_RSA2048 \
     --avb_vbmeta_extra_args \
         "--signing_helper_with_files ./signing-helper.sh" \
     --avb_system_key external/avb/test/data/testkey_rsa4096.pem \
     --avb_system_algorithm SHA256_RSA4096 \
     --avb_system_extra_args \
         "--signing_helper_with_files ./signing-helper.sh" \
     product-target_files.eng.zip signed-product-target_files.zip

To verify the signed images:
 $ unzip signed-product-target_files.zip IMAGES/\*
 $ avbtool verify_image --image IMAGES/vbmeta.img \
     --expected_chain_partition system:1:testkey_rsa4096_pub.pem \
     --key external/avb/test/data/testkey_rsa2048.pem

Bug: 38315721
Test: sign_target_files_apks.py on AVB-enabled target w/ and w/o chained
      partitions respectively. Check the signing command lines; validate
      the signed images with 'avbtool verify_image'.
Change-Id: Ia009555b16ddb6d8ba6a0858d5ca7d983bbab887
(cherry picked from commit 639118ff4d)
2017-06-28 12:38:32 -07:00
Tao Bao
7b8a7936dd Merge "Delay the write-back of misc_info in sign_target_files_apks.py." into oc-dr1-dev 2017-06-28 19:36:51 +00:00
Tao Bao
57ae9a2234 Delay the write-back of misc_info in sign_target_files_apks.py.
Currently we're writing META/misc_info.txt to the new TF.zip during
ReplaceVerityPrivateKey(). We should delay that until we have replaced
everything in need. Otherwise we won't be able to replace/overwrite
that zip entry (unless `zip -d` first).

This CL also cleans up the return value of ReplaceVerityPublicKey() and
ReplaceVerityKeyId(), since the caller no longer needs the values.

Test: sign_target_files_apks.py and check the generated signed TF.zip.
Change-Id: I9fbd7182247728281519e5e3971557f6b018ad65
(cherry picked from commit 46a5999a02)
2017-06-28 11:07:34 -07:00
TreeHugger Robot
3b7460eda9 Merge "Move /default.prop to /system/etc/prop.default" into oc-dr1-dev 2017-06-28 06:28:10 +00:00
Tao Bao
95a95c3a5e Allow packing custom radio images into target_files zip.
Radio images are added to INSTALLED_RADIOIMAGE_TARGET, which by default
will be packed under RADIO/ in a target_files zip. This CL introduces
BOARD_PACK_RADIOIMAGES that allows additionally copying them into
IMAGES/, which will then be included into <product>-img.zip.

Bug: 62195105
Test: Define BOARD_PACK_RADIOIMAGES and `m dist`. Check the generated
      target_files zip and <product>-img.zip.
Change-Id: I3deafd2dfecd1d4dbfdfc2d002fc40ef22fb60ea
2017-06-27 13:50:12 -07:00
Tao Bao
639118ff4d Support replacing AVB signing keys.
sign_target_files_apks.py now supports signing targets using verified
boot 2.0 (aka AVB). It allows replacing AVB signing keys for both of
chained and non-chained partitions.

An example command line looks as follows.
 $ ./build/tools/releasetools/sign_target_files_apks.py \
     --avb_vbmeta_key external/avb/test/data/testkey_rsa2048.pem \
     --avb_vbmeta_algorithm SHA256_RSA2048 \
     --avb_vbmeta_extra_args \
         "--signing_helper_with_files ./signing-helper.sh" \
     --avb_system_key external/avb/test/data/testkey_rsa4096.pem \
     --avb_system_algorithm SHA256_RSA4096 \
     --avb_system_extra_args \
         "--signing_helper_with_files ./signing-helper.sh" \
     product-target_files.eng.zip signed-product-target_files.zip

To verify the signed images:
 $ unzip signed-product-target_files.zip IMAGES/\*
 $ avbtool verify_image --image IMAGES/vbmeta.img \
     --expected_chain_partition system:1:testkey_rsa4096_pub.pem \
     --key external/avb/test/data/testkey_rsa2048.pem

Bug: 38315721
Test: sign_target_files_apks.py on AVB-enabled target w/ and w/o chained
      partitions respectively. Check the signing command lines; validate
      the signed images with 'avbtool verify_image'.
Change-Id: Ia009555b16ddb6d8ba6a0858d5ca7d983bbab887
2017-06-27 10:50:46 -07:00
Hung-ying Tyan
7eb6a926ad Move /default.prop to /system/etc/prop.default
This patch tries to fix the problem where the default properties need
to go with the system image especially on non-AB devices where
/default.prop is on the ramdisk image. A symlink is created at
/default.prop for backward compatibility.

Bug: 37815285
Test: Tested with ag/2416542. Booted pixel phones, checked the location
      of prop.default, verified the symlink, checked a few properties
      (via adb shell getprop) and manually tested a few apps (Camera,
      Maps etc).
      sign_target_files_apks.py was tested with:
      sign_target_files_apks -o -e DynamiteLoader.apk= -e DynamiteModulesA.apk= \
          -e DynamiteModulesB.apk= -e DynamiteModulesC.apk= -e DynamiteModulesD.apk= \
          -e GoogleCertificates.apk= out/dist/*-target_files-*.zip signed-target_files.zip
      Booted to recovery and ran 'adb sideload' successfully.
Change-Id: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
Merged-In: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
(cherry-picked from 4fbbe4578bb10d54292d9b243edf4999fddf1c93)
2017-06-27 15:10:32 +08:00