Commit graph

1301 commits

Author SHA1 Message Date
Tao Bao
5ef78033d5 Merge "releasetools: Clean up test_sign_target_files_apks.py." 2018-02-21 02:06:00 +00:00
Tao Bao
a0f55ee9eb Merge "releasetools: Add a testcase for common.ParseCertificate()." 2018-02-21 00:04:24 +00:00
Tao Bao
66472637ad releasetools: Check for duplicate entries in ReplaceCerts().
testdata/{media,platform}.x509.pem files are copied from
build/target/product/security/.

Fixes: 69479366
Test: python -m unittest test_sign_target_files_apks
Change-Id: I8ba42b6f5b5432ee4b8786b241daff11db792c14
2018-02-20 10:22:30 -08:00
Tao Bao
de1d479ace releasetools: Clean up test_sign_target_files_apks.py.
Since we have been carrying test certificates in testdata/ for other
tests, do the same for test_sign_target_files_apks.py. Copy
verity.x509.pem from build/target/product/security/ to testdata/ for
that purpose.

Also capture the stderr output in ReplaceVerityKeyId().

Test: python -m unittest test_sign_target_files_apks
Change-Id: Ie11e042086952e8a4a5a63950cb0b16cc436b7e6
2018-02-20 10:12:15 -08:00
Tao Bao
17e4e61691 releasetools: Add a testcase for common.ParseCertificate().
Also minor clean up to the function, to suppress the following lint
warning.
R:1813, 2: Redefinition of cert type from list to unicode (redefined-variable-type)

Test: python -m unittest -v test_common.CommonApkUtilsTest
Change-Id: Ib4b1dfa8b19e505fc70684b648efc36171c73bbf
2018-02-16 17:14:23 -08:00
Tao Bao
fc7e0e06b6 releasetools: Add unittest for common.GetSparseImage().
Add construct_sparse_image() to test_utils.py, which is a util function
to create sparse images. The new tests also partially cover the recent
changes that add 'incomplete' and 'uses_shard_blocks' tags.

Test: python -m unittest test_common
Change-Id: Ia15f5c4ad12423691216ebbad2c28f95c8427d7e
2018-02-14 15:12:52 -08:00
Tao Bao
9452d84b97 Merge "releasetools: Allow generating BBOTA for images with shared blocks." 2018-02-13 21:53:25 +00:00
Tao Bao
6392e05813 Merge "releasetools: Remove the unconditional fallback to bsdiff." 2018-02-13 21:53:17 +00:00
Tao Bao
4f0b725439 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload." 2018-02-13 20:20:30 +00:00
Tao Bao
e709b094e4 releasetools: Allow generating BBOTA for images with shared blocks.
When target defines 'BOARD_EXT4_SHARE_DUP_BLOCKS := true', the generated
system/vendor images may contain shared blocks (i.e. some blocks will
show up in multiple files' block list), which violates the current
assumptions in BBOTA script.

This CL allows generating BBOTAs by considering the first occurrence as
the "owner" of the shared blocks. All the later users of the shared
blocks will have an incomplete block list, whose RangeSet's will be
tagged with 'uses_shared_blocks'.

Files with 'uses_shared_blocks' tag will not be diff'd with imgdiff,
potentially with patch size penalty. Such files will be accounted for in
imgdiff stats report, where we can revisit for a better solution.

Bug: 64109868
Test: Generate BBOTA full and incremental package with targets defining
      'BOARD_EXT4_SHARE_DUP_BLOCKS := true'.
Change-Id: I87fbc22eef7fafe2a470a03fdcfa1babf088ea8d
2018-02-13 11:14:23 -08:00
Tao Bao
4ccea8549e releasetools: Remove the unconditional fallback to bsdiff.
This CL uses the 'incomplete' tag to skip applying imgdiff to files with
incomplete block list. It's not the ideal fix to address the holes in
ext4 images, but would unhide other imgdiff issues covered by the
unconditional fallback.

Bug: 68016761
Test: Generate an incremental OTA package from images with incomplete
      block list. Check the imgdiff stats report.
Test: `python -m unittest test_blockimgdiff`
Change-Id: Ice77686414e70f5e42de35c1757fb31cf02e4fd4
2018-02-13 11:06:20 -08:00
Tao Bao
508b087943 releasetools: Make blockimgdiff.py pylint-clean.
************* Module blockimgdiff
C:433, 0: Unnecessary parens after 'if' keyword (superfluous-parens)
C:687, 0: Wrong hanging indentation (add 4 spaces).
            max_stashed_blocks, self._max_stashed_size, max_allowed,
            ^   | (bad-continuation)
C:688, 0: Wrong hanging indentation (add 4 spaces).
            self._max_stashed_size * 100.0 / max_allowed))
            ^   | (bad-continuation)
C:691, 0: Wrong hanging indentation (remove 2 spaces).
            max_stashed_blocks, self._max_stashed_size))
          | ^ (bad-continuation)
C:898, 0: Wrong hanging indentation (add 4 spaces).
                  "imgdiff" if imgdiff else "bsdiff",
                  ^   | (bad-continuation)
C:899, 0: Wrong hanging indentation (add 4 spaces).
                  xf.tgt_name if xf.tgt_name == xf.src_name else
                  ^   | (bad-continuation)
C:901, 0: Wrong hanging indentation (add 4 spaces).
                  xf.tgt_ranges, xf.src_ranges, e.message))
                  ^   | (bad-continuation)
C:909, 0: Wrong hanging indentation (add 4 spaces).
                      xf.tgt_name,))
                      ^   | (bad-continuation)
C:917, 0: Wrong hanging indentation (add 4 spaces).
                      xf.tgt_name, e.message))
                      ^   | (bad-continuation)
C:961, 0: Wrong hanging indentation (remove 2 spaces).
                xf.patch_len, tgt_size, xf.patch_len * 100.0 / tgt_size,
              | ^ (bad-continuation)
C:962, 0: Wrong hanging indentation (remove 2 spaces).
                xf.style,
              | ^ (bad-continuation)
C:963, 0: Wrong hanging indentation (remove 2 spaces).
                xf.tgt_name if xf.tgt_name == xf.src_name else (
              | ^ (bad-continuation)
C:965, 0: Wrong hanging indentation (remove 2 spaces).
                xf.tgt_ranges, xf.src_ranges))
              | ^ (bad-continuation)
C:1422, 0: Wrong continued indentation (add 28 spaces).
                tgt_skipped.size() * 100.0 / tgt_size, tgt_name))
                ^                           | (bad-continuation)
C:1560, 0: Wrong continued indentation (add 8 spaces).
            split_src_ranges) in enumerate(split_info_list):
            ^       | (bad-continuation)
R:566, 6: Redefinition of src_str type from list to str (redefined-variable-type)
C:1198,25: More than one statement on a single line (multiple-statements)
C:1211,25: More than one statement on a single line (multiple-statements)
C:1220,16: More than one statement on a single line (multiple-statements)
C:1277,38: More than one statement on a single line (multiple-statements)
C:1284,19: More than one statement on a single line (multiple-statements)
C: 19, 0: standard import "import copy" comes before "import common" (wrong-import-order)
C: 20, 0: standard import "import functools" comes before "import common" (wrong-import-order)
C: 21, 0: standard import "import heapq" comes before "import common" (wrong-import-order)
C: 22, 0: standard import "import itertools" comes before "import common" (wrong-import-order)
C: 23, 0: standard import "import multiprocessing" comes before "import common" (wrong-import-order)
C: 24, 0: standard import "import os" comes before "import common" (wrong-import-order)
C: 25, 0: standard import "import os.path" comes before "import common" (wrong-import-order)
C: 26, 0: standard import "import re" comes before "import common" (wrong-import-order)
C: 27, 0: standard import "import subprocess" comes before "import common" (wrong-import-order)
C: 28, 0: standard import "import sys" comes before "import common" (wrong-import-order)
C: 29, 0: standard import "import threading" comes before "import common" (wrong-import-order)
C: 31, 0: standard import "from collections import deque, OrderedDict" comes before "import common" (wrong-import-order)
C: 32, 0: standard import "from hashlib import sha1" comes before "import common" (wrong-import-order)

Test: `pylint --rcfile=pylintrc blockimgdiff.py`
Test: Generate an incremental BBOTA package.
Change-Id: Ia7ecd0e1fa48daf4e43251bdcdfcd08fb316015d
2018-02-12 13:01:34 -08:00
Tao Bao
9739514769 releasetools: Remove the global diff_done in blockimgdiff.py.
pylint complains about undefined `diff_done`:

W:754, 8: Global variable 'diff_done' undefined at the module level (global-variable-undefined)
W:820,14: Global variable 'diff_done' undefined at the module level (global-variable-undefined)

It would still warn about using global statement after adding the
definition.

W:859, 8: Using the global statement (global-statement)
W:925,14: Using the global statement (global-statement)

This CL computes 'diff_done' via 'len(diff_queue)' instead. It also
moves the progress reporting _before_ the diff work. This way it avoids
showing 100% progress with still changing filenames (because multiple
workers could see an empty queue simultaneously upon finishing their own
works).

There're possible alternatives, such as using the 'nonlocal' keyword in
Python 3 (which we're not there yet), or by using mutable object instead
(e.g. 'diff_done = [0]'). This CL looks cleaner, since it just kills the
var.

Test: Generate a BBOTA incremental. Check the on-screen progress
      report.
Test: `pylint --rcfile=pylintrc blockimgdiff.py` no longer complains
      about the global diff_done.
Change-Id: I339824735527e1f794b5b1dc99ff3fdb2da85744
2018-02-12 12:55:25 -08:00
Tao Bao
667ff57272 releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload.
The secondary payload should always be applied with
SWITCH_SLOT_ON_REBOOT=0. This CL moves the 'secondary' parameter from
Payload.WriteToZip() to Payload.__init__(). So it can append the flag to
secondary/payload_properties.txt.

Bug: 35724498
Test: Generate an A/B OTA with --include_secondary. Check
      secondary/payload_properties.txt entry in the generated ZIP.
Test: `python -m unittest test_ota_from_target_files`
Change-Id: I816c07ab57a1c8a52eff785801634b8b1cb134d4
2018-02-10 00:13:07 -08:00
Tao Bao
294651d7b4 releasetools: Add an ImgdiffStats class that reports imgdiff stats.
We have a couple of active imgdiff workarounds (and likely with one more
inbounding that allows having shared blocks in ext4 image). Most of
these workarounds need extending imgdiff's capability. While us not
getting there anytime soon, collect the stats to better understand the
impact of each kind so we can prioritize accordingly.

A sample report is as follows.

  Imgdiff Stats Report
========================

 APK files diff'd with imgdiff (count: 88)
-------------------------------------------

  /system/priv-app/Shell/Shell.apk
  ...

 Large APK files split and diff'd with imgdiff (count: 4)
----------------------------------------------------------

  /system/priv-app/Settings/Settings.apk
  ...

Bug: 68016761
Test: Generate an incremental BBOTA package. Check the stats report.
Test: python -m unittest test_blockimgdiff
Change-Id: I27ad862cde472ab2806db877632ce5a0607420f2
2018-02-09 23:02:32 -08:00
Tao Bao
cb73aed1f0 releasetools: Refactor the condition checking for using imgdiff.
In Transfer class, unbundle 'intact' with the monotonicity of the input
ranges. Negate the logic of 'intact', and thus rename it to 'trimmed'.
Move this property from an attribute of Transfer class as the one in
RangeSet.extra. 'trimmed' indicates whether the source / target ranges
have been modified after creating the Transfer() instance.

The logic that determines whether we can apply imgdiff has been
refactored and consolidated into BlockImageDiff.CanUseImgdiff(). Now
both of the two paths call this single copy, i.e. the one that detects
large APKs (before creating Transfer()'s), and the other that's about to
generate the patch for a given Transfer instance.

Bug: 68016761
Test: python -m unittest test_blockimgdiff
Test: Generate an incremental BBOTA package.
Change-Id: Id07195f63f1fa6c3af6e9091940d251cf09fa104
2018-02-09 12:46:01 -08:00
Tao Bao
acb3cecc46 Merge "releasetools: RangeSet.monotonic is not an optional attribute." 2018-02-08 04:45:44 +00:00
Tao Bao
67ba60029d Merge "releasetools: Support packaging secondary payload." 2018-02-07 05:24:46 +00:00
Tao Bao
fe97dbd4ce releasetools: RangeSet.monotonic is not an optional attribute.
'monotonic' has been non-optional since [1] (L-MR1). Fix the comment in
RangeSet.parse(), as well as the use in blockimgdiff.py.

[1] commit 8b72aefb5a.

Test: Generate an incremental BBOTA package.
Change-Id: I7f95231683473b4f0f07f9c83fccc0e36a1340cb
2018-02-06 16:00:52 -08:00
Tao Bao
3f15ada51b releasetools: Capture stderr output when calling delta_generator.
Prior to this CL, the call to delta_generator in
check_ota_package_signature.VerifyAbOtaPayload() didn't redirect stderr.
The logs (mostly INFO) on successful verification added noise to the
normal output, which also upset the unittest result parser.

This CL captures stderr outputs from delta_generator, and will only dump
them on error.

Bug: 72884343
Test: `python -m unittest -v test_ota_from_target_files > /dev/null`
      gives clean output.
Test: Inject error into delta_generator. The call to
      check_ota_package_signature correctly dumps both of stdout and
      stderr outputs.
Change-Id: I014a4b21bf758dcf0a4b9963259d6019851935ee
2018-02-05 23:39:23 -08:00
Tao Bao
f7140c0f8c releasetools: Support packaging secondary payload.
By default, an A/B OTA package doesn't contain the images for the
secondary slot (e.g. system_other.img). Specifying
"--include_secondary" that's introduced in this CL allows generating
a separate payload that will install secondary slot images. Both
payloads will be added to the generated A/B OTA package.

An example A/B OTA package with secondary payload
  |
  +-- payload.bin
  |
  +-- payload_properties.txt
  |
  +-- secondary/payload.bin
  |
  +-- secondary/payload_properties.txt
  |
  +-- ...

Such a package needs to be applied in a two-stage manner. During the
first stage, the updater applies the primary payload only. Upon
finishing, it reboots the device into the newly updated slot. It then
continues to install the secondary payload to the inactive slot, but
without switching the active slot at the end (needs the matching support
in update_engine, i.e. SWITCH_SLOT_ON_REBOOT flag).

Due to the special install procedure, the secondary payload will be
always generated as a full payload.

Bug: 35724498
Test: Generate full and incremental OTAs with --include_secondary. Check
      the generated OTAs.
Test: python -m unittest test_ota_from_target_files
Change-Id: I975e826bec492e86eb400f99de0c355a32420127
2018-02-05 13:28:52 -08:00
Treehugger Robot
ca2ffed06c Merge "releasetools: Fix an issue with pubkey extraction." 2018-02-05 21:25:47 +00:00
Treehugger Robot
993e1d2634 Merge "releasetools: Add sdk and security patch level info to metadata." 2018-02-05 17:25:16 +00:00
Tao Bao
04e1f012dd releasetools: Fix an issue with pubkey extraction.
When calling 'openssl x509 -pubkey' to extract the public key from a
certificate, openssl 1.0 and 1.1 handle the '-out' parameter
differently. openssl 1.0 doesn't write the output into the specified
filename, which leads to the payload verification failure in
check_ota_package_signature.VerifyAbOtaPayload(). This CL addresses
the issue by always collecting the output from stdout instead.

It also refactors the two copies into common.ExtractPublicKey(), and
adds unittest. get_testdata_dir() is moved into test_utils.py that holds
common utils for running the unittests.

Bug: 72884343
Test: python -m unittest test_common
Test: python -m unittest test_ota_from_target_files
Test: Run sign_target_files_apks with '--replace_ota_keys' on marlin
      target_files zip. Check the payload pubkey replacement.
Test: Trigger the tests with forrest, and tests no longer fail on
      machines with openssl 1.0.1.
Change-Id: Ib0389b360f064053e9aa7cc0546d718e7b23003b
2018-02-04 13:59:52 -08:00
Tao Bao
93db955964 Merge "releasetools: Detect incomplete block ranges." 2018-02-03 04:32:13 +00:00
Tao Bao
c7b403a2e8 releasetools: Add Payload class.
This breaks down the current WriteABOTAPackageWithBrilloScript() into
smaller and testable units, which also prepares for the work in
b/35724498.

Bug: 35724498
Test: python -m unittest test_ota_from_target_files
Test: Get identical A/B OTA packages w/ and w/o the CL.
Change-Id: I2ea45ce98e2d2baa58e94fb829b7242f6fe685a7
Merged-In: I2ea45ce98e2d2baa58e94fb829b7242f6fe685a7
(cherry picked from commit 036d721812)
2018-02-02 16:18:07 -08:00
Tao Bao
24604cc5b9 releasetools: Clean up two no-op lines that set 'ota-wipe'.
The same thing has been set in GetPackageMetadata(), so they are no-op.

Also replace an occurrence of 'OPTIONS.source_info_dict' with
source_info, which has been missed from previous clean-up CLs.

Test: Generate an A/B OTA package, and an incremental BBOTA with
      --wipe_user_data. Check the metadata.
Test: Generate a two-step incremental BBOTA successfully.
Change-Id: I4bb491cac9064d93fb86d12e617c8f38f040e01e
2018-02-01 16:30:42 -08:00
Tao Bao
35dc255180 releasetools: Add sdk and security patch level info to metadata.
The info comes from the build.prop file of the target build (thus no
backward compatibility concerns). OTA server and client can optionally
use these info to understand the expected behavior of an OTA package.

Bug: 72751683
Test: python -m unittest test_ota_from_target_files
Test: `m dist`, then check the metadata in the generated OTA package.
Change-Id: I5935f67684d2486bb5f00d67ce4bc756589a56ed
2018-02-01 15:58:20 -08:00
Tao Bao
c765cca38b releasetools: Detect incomplete block ranges.
This CL detects incomplete block ranges (e.g. due to the holes in
mke2fs created images). Such block ranges will be tagged, so we won't
attempt to imgdiff those files. Note that the change to blockimgdiff.py,
which uses the tag info, will come in a separate CL.

An 'extra' attribute is added to RangeSet class, which defaults to an
empty dict. An 'incomplete' tag will be added into the dict by the
caller of the class. Not adding this tag as an immediate attribute,
because it is not a property regarding the ranges being represented, but
rather some storage space for the caller.

This CL also refactors GetSparseImage and RoundUpTo4K into common.py, so
the same code can be called from both of ota_from_target_files.py and
validate_target_files.py. Not able to add unittests for
GetSparseImage(), as SparseImage requires data in specific format.

Bug: 68016761
Test: Run validate_target_files.py on target-files.zip. It skips
      validating files with missing holes as before.
Test: Run ota_from_target_files.py on angler target-files.zip. It gives
      identical packages w/ and w/o the CL.
Test: pylint on changed files. There're warnings with common.py, but
      unrelated to this change.
Change-Id: I126ccfea13c0d5ebcc8c1b4ff1a4f9200e97423a
2018-02-01 12:35:48 -08:00
Tao Bao
bb20e8c5f2 releasetools: Make validate_target_files.py pylint clean.
C: 73, 0: Wrong hanging indentation (add 4 spaces).
      file_name, actual_sha1, expected_sha1)
      ^   | (bad-continuation)
C:171, 0: Wrong continued indentation (add 20 spaces).
        'SYSTEM/etc/recovery.img', expected_recovery_sha1)
        ^                   | (bad-continuation)
C:185, 0: Wrong continued indentation (add 20 spaces).
        file_path='IMAGES/boot.img', expected_sha1=boot_info[3])
        ^                   | (bad-continuation)
C:191, 0: Wrong continued indentation (add 20 spaces).
        file_path='IMAGES/recovery.img',
        ^                   | (bad-continuation)
C:192, 0: Wrong continued indentation (add 20 spaces).
        expected_sha1=expected_recovery_sha1)
        ^                   | (bad-continuation)
W: 67,15: Use % formatting in logging functions and pass the % parameters as arguments (logging-format-interpolation)
W:150,17: Use % formatting in logging functions and pass the % parameters as arguments (logging-format-interpolation)
W:153,15: Use % formatting in logging functions and pass the % parameters as arguments (logging-format-interpolation)
W:194,15: Use % formatting in logging functions and pass the % parameters as arguments (logging-format-interpolation)
C: 27, 0: standard import "import logging" comes before "import common" (wrong-import-order)
C: 28, 0: standard import "import os.path" comes before "import common" (wrong-import-order)
C: 29, 0: standard import "import re" comes before "import common" (wrong-import-order)
C: 31, 0: standard import "import sys" comes before "import common" (wrong-import-order)

Test: pylint --rcfile=pylintrc validate_target_files.py
Test: Run validate_target_files.py with a target-files.zip.
Change-Id: Ie64acdb4cee4326938c4ad5a34b575d7b82478c0
2018-02-01 12:03:17 -08:00
Tao Bao
12d87fc174 releasetools: Move the AVB salt setup into common.LoadInfoDict().
We used to do this in add_img_to_target_files.AddImagesToTargetFiles(),
which didn't cover the path when calling from make_recovery_patch. As a
result, /system/bin/install-recovery.sh contains different SHA values
from the actual images.

Test: Set up aosp_bullhead to use AVB. `m dist`, then run the following
      command to verify the generated install-recovery.sh.

  $ ./build/make/tools/releasetools/validate_target_files.py \
      out/dist/aosp_bullhead-target_files-eng.zip

Change-Id: Id7be8fb17072252fcd4d08db2057b8c4af053376
2018-01-31 12:25:10 -08:00
Tianjie Xu
df1166e92f Protect SparseImage._GetRangeData() with lock
The generator function is not thread safe and is prone to race
conditions. This CL uses a lock to protect this generator and loose the
locks elsewhere, e.g. 'WriteRangeDataToFd()'.

Bug: 71908713
Test: Generate an incremental package several times for angler 4208095 to 4442250.
Change-Id: I9e6f0a182a1ba7904a597f403f2b12fe05016513
2018-01-29 11:52:10 -08:00
Tianjie Xu
8a7ed9f771 Double check the sha1 for ranges during package generation
Check that the Sha1 for src&tgt ranges are correct before computing
patches. This adds ~6 seconds overhead for ~2400 commands.

Bug: 71908713
Test: Generate an incremental package from angler 4208095 to 4442250.
Change-Id: I8cf8ce132fb09a22f7d6689274ddb4a27770be76
2018-01-28 01:54:52 +00:00
Jaekyun Seok
b7735d8105 Support /product partition
This CL adds the following build flags to support building product partition
from Android build system.
- BOARD_AVB_PRODUCT_ADD_HASHTREE_FOOTER_ARGS
- BOARD_AVB_PRODUCT_KEY_PATH
- BOARD_AVB_PRODUCT_ALGORITHM
- BOARD_AVB_PRODUCT_ROLLBACK_INDEX_LOCATION
- BOARD_PREBUILT_PRODUCTIMAGE
- BOARD_PRODUCTIMAGE_EXTFS_INODE_COUNT
- BOARD_PRODUCTIMAGE_FILE_SYSTEM_TYPE
- BOARD_PRODUCTIMAGE_PARTITION_SIZE
- BOARD_PRODUCTIMAGE_SQUASHFS_BLOCK_SIZE
- BOARD_PRODUCTIMAGE_SQUASHFS_COMPRESSOR
- BOARD_PRODUCTIMAGE_SQUASHFS_COMPRESSOR_OPT
- BOARD_PRODUCTIMAGE_SQUASHFS_DISABLE_4K_ALIGN
- BOARD_USES_PRODUCTIMAGE
- LOCAL_PRODUCT_MODULE
- PRODUCT_FOOTER_ARGS
- PRODUCT_PRODUCT_BASE_FS_PATH
- PRODUCT_PRODUCT_VERITY_PARTITION
- TARGET_COPY_OUT_PRODUCT
- TARGET_OUT_PRODUCT*

And PRODUCT_PRODUCT_PROPERTIES is added to save product-specific
properties in /product/build.prop.

Bug: 64195575
Test: succeeded building product.img with BOARD_PRODUCTIMAGE_PARTITION_SIZE,
BOARD_PRODUCTIMAGE_FILE_SYSTEM_TYPE and PRODUCT_PRODUCT_VERITY_PARTITION.

Change-Id: I774e6a38003734421591e51bed103802ff84f432
2018-01-25 09:33:58 +09:00
Treehugger Robot
082837365d Merge "Add BOARD_EXT4_SHARE_DUP_BLOCKS to share duplicated blocks on ext4" 2018-01-24 04:10:42 +00:00
Tianjie Xu
07a4299182 Merge "Don't check the stderr output from bsdiff/imgdiff" 2018-01-23 06:10:46 +00:00
Tianjie Xu
a2a9f9990f Don't check the stderr output from bsdiff/imgdiff
bsdiff/imgdiff returns non-zero values correctly in case of failures.
So we don't need to check the stderr anymore. This avoids some false
reports of the diff program.

Bug: 72335938
Bug: 71505046
Test: Check the exit value of bsdiff/imgdiff in code search,
      generate a package for angler.

Change-Id: I18f0c3882a40a5288d6aee715713a05270e0db2b
2018-01-23 01:29:56 +00:00
Jin Qian
fde9f7906b Add BOARD_EXT4_SHARE_DUP_BLOCKS to share duplicated blocks on ext4
This option applies to system.img and vendor.img.

Bug: 64109868
Change-Id: Ib4c7384049dca08e57441ef71cc81c2fba4462dd
2018-01-22 17:21:11 -08:00
Treehugger Robot
8413824f4b Merge "releasetools: Add PayloadSigner class." 2018-01-22 18:54:57 +00:00
Ng Zhi An
9446c1df98 Fix order of args passed to mksquashfsimage.sh
Bug: 72234456
Test: make systemimage
Change-Id: I1b4b07f0a488eade2f297fea12d02b6e8ccba7d8
2018-01-19 15:55:39 -08:00
Treehugger Robot
3221f0f387 Merge "releasetools: Dump mkfs/fsck/simg2img outputs on error." 2018-01-19 22:35:47 +00:00
Tao Bao
fabe08352c releasetools: Add PayloadSigner class.
Create a wrapper class that handles the payload signing, which unifies
the paths with and without external signer. Also add tests for the newly
added class.

The test keys under testdata/ are created with the script in
development/tools/make_key. testdata/testkey_with_passwd.pk8 uses
password "foo".

Test: python -m unittest test_ota_from_target_files
Test: Get identical A/B OTA packages w/ and w/o the CL.
Change-Id: Ic770aec726498a3babb88ec509985e7f1210fb18
2018-01-19 13:25:04 -08:00
Tao Bao
cd53a89599 releasetools: Dump mkfs/fsck/simg2img outputs on error.
The output (stdout and stderr combined) would be helpful in understanding
the cause of the failure.

Not changing other occurrences in build_image.py yet, since it's less
obvious if a failed call should be considered fatal (thus whether to
dump such outputs). For example, GetVeritySize() simply returns 0 on
failed calls. They deserve further clean-ups in later CLs (with unit
tests).

Bug: 71864688
Test: Inject errors into mke2fs/e2fsck/simg2img. `m snod` prints outputs.
Test: `m snod` remains quiet on successful runs.
Change-Id: I172403f6cd05bce93767265dbcb110271a68e2a8
2018-01-19 11:15:42 -08:00
Tao Bao
a2ff4c9b06 releasetools: Clean up three functions in add_img_to_target_files.py.
This CL makes the following changes:
 (a) cleans up the similar codes in AddCareMapTxtForAbOta() that handle
     'system' and 'vendor' partitions;
 (b) fixes an issue with the arcname in AddPackRadioImages() and
     AddRadioImagesForAbOta(), where forward slash should always be used
     in zip entry names;
 (c) refactors the branching statements in AddRadioImagesForAbOta() to
     reduce indentation levels.

Test: python -m unittest test_add_img_to_target_files
Test: `m dist` with aosp_marlin-userdebug. Check META/care_map.txt in the
      generated target_files.zip.
Change-Id: I3d6f794962d0c68390fbd18eb13c2622acab3ff5
2018-01-18 23:37:29 -08:00
Tao Bao
bea20ac722 releasetools: Refactor AddImagesToTargetFiles().
Separate three functions out of AddImagesToTargetFiles(), into
AddCareMapTxtForAbOta(), AddRadioImagesForAbOta() and
AddPackRadioImages() respectively. This CL tries to apply minimal
changes for the refactoring purpose.

Also add tests for AddRadioImagesForAbOta() and AddPackRadioImages().
The tests for AddCareMapTxtForAbOta() require better testing support to
mock sparse_img.SparseImage, which will be added in later CLs.

Test: python -m unittest test_add_img_to_target_files
Test: `m dist` with aosp_marlin-userdebug. Check META/care_map.txt in
      the generated target_files.zip.
Change-Id: I1bb723c15237ff721f165cfce0ce996008ce9948
2018-01-18 23:37:23 -08:00
Treehugger Robot
13f228ebde Merge "releasetools: Remove the reloading of target info dict." 2018-01-18 22:48:15 +00:00
Tao Bao
d32e78fd3c releasetools: Add a note for the ZipInfo._decodeExtra() workaround.
Test: N/A
Change-Id: I9e84fc27230182f17c2fda0df57a2c149281dcda
2018-01-17 10:10:00 -08:00
Tao Bao
df3a48b43e releasetools: Consolidate the codes that generate metadata dict.
Now they (A/B, non-A/B full and non-A/B incremental) share the same
code. Also add tests for the newly added function.

Test: python -m unittest test_ota_from_target_files
Test: Build A/B and non-A/B packages respectively.
Change-Id: I5499dd9c683ac1a18d8a24f21ff8845fe69f343e
2018-01-16 15:34:27 -08:00
Tianjie Xu
a7229a08b7 Merge "Iterate through the sorted file map when finding transfers" 2018-01-13 00:01:03 +00:00
Tianjie Xu
20a86cdd8d Iterate through the sorted file map when finding transfers
This helps to generate a deterministic package.

Bug: 71770360
Test: Generate a incremental package and transfers are added by file name.
Change-Id: I7562a200b97a1babbb09a77801324cc9408cc01f
2018-01-12 12:21:00 -08:00
Tao Bao
2db1385db5 releasetools: Remove the reloading of target info dict.
In non-A/B OTA path, we've been loading the info dict for the target
build twice (once to have a peek at the OTA type, with a reload after
unzipping the target-files zip). Remove the reloading.

This CL moves the loading of the source info dict up a bit to avoid
having two copies in the A/B and non-A/B paths. It also moves up the
sanity checks of the loaded info dicts before unzipping the target-files
zips.

Test: Generate full and incremental packages for bullhead and marlin.
Change-Id: Iccb953feb0b6ecf62e6f73d6e9ceea00077d098d
2018-01-11 21:59:30 -08:00
Treehugger Robot
896fa043bc Merge "releasetools: Remove the fallback-to-full logic." 2018-01-11 22:24:36 +00:00
Treehugger Robot
155a140c82 Merge "releasetools: Remove two dead functions in ota_from_target_files.py." 2018-01-11 22:23:18 +00:00
Tianjie Xu
c56071b21b Merge "Sort the split transfers to generate a determinate package" 2018-01-11 04:15:28 +00:00
Tao Bao
175df38e6c releasetools: Remove two dead functions in ota_from_target_files.py.
GetBuildProp() and GetVendorBuildProp() have been factored into
BuildInfo class. These two functions were meant to be deleted in commit
481bab8d48.

Test: python -m unittest test_ota_from_target_files
Test: No users of these two functions.
Change-Id: Id590a46d597939973ef2339cdfea4f5163548d4d
2018-01-10 19:57:57 -08:00
Tao Bao
32b80dca9f releasetools: Remove the fallback-to-full logic.
The fallback-to-full logic doesn't look beneficial, but may hide away
real bugs, especially when it's on by default.

Test: Generate an incremental non-A/B OTA successfully.
Test: Inject bugs into script; observe that incremental OTA
      generation errors out instead of silently falling back.
Test: No internal users of the flag from code search.
Change-Id: Ifc627d898552c0bd72463e8c107e36610421608e
2018-01-10 14:03:16 -08:00
Tianjie Xu
d3bf67e1d3 Sort the split transfers to generate a determinate package
We split large apks and generated patches for them in parallel,
resulting in nondeterminate packages between different runs. This CL
sort the split transfers by target name first; and then add them
sequentially to the final transfer list.

Also fix a side effect where we may generate a wrong sha1 for split
ranges due to synchronization error.

Bug: 71770360
Bug: 71759418

Test: Generate the package several times, compare the log and the transfer list.
Change-Id: I2a49e22594d59ffaa98b11edc776be4e3c4c561f
2018-01-10 12:23:57 -08:00
Tao Bao
481bab8d48 releasetools: Add a BuildInfo class.
Prior to this CL, we have similar codes in different paths
(WriteFullOTAPackage / WriteBlockIncrementalOTAPackage /
WriteABOTAPackageWithBrilloScript). This CL factors out the common codes
that deal with OEM-specific properties into BuildInfo class, and adds
tests for the new class.

Test: python -m unittest test_ota_from_target_files
Test: Create an incremental package that uses and doesn't use OEM
      properties respectively.
Test: Create a full package that uses and doesn't use OEM properties
      respectively.
Change-Id: Ida914cace12803d55396fa503bafcac2db2a520e
2018-01-09 15:08:23 -08:00
Tao Bao
818ddf5e7e releasetools: Add tests for common.ReadApkCerts().
Test: python -m unittest test_common
Test: Run sign_target_files_apks.py on a target with compressed APKs.
Change-Id: I107a8b8f2f0f82e2d1947f14c8a8b3778f633b11
2018-01-05 11:53:55 -08:00
Tao Bao
c72727ac3f releasetools: Clean up build_image.py.
Mostly cosmetic changes, such as moving print statement to print
function. The only functional change is to wrap the file opening in
Append() with try statement, which would dump the error message as
needed.

Test: `m dist`
Test: python -m unittest test_build_image
Change-Id: I4e6c593517cf737bfcbe51bc533b22a247b10e44
2018-01-03 11:52:28 -08:00
Tao Bao
d8a953d7ab releasetools: Move assertions into CheckHeadroom().
Assert the input arguments inside CheckHeadroom() function, and add
matching tests. Also use common.MakeTemp{File,Dir} in tests.

Test: `m dist`
Test: python -m unittest test_build_image
Change-Id: Ib45f24651af9591a9cfdd72aac3558f6c9a80f44
2018-01-03 11:52:16 -08:00
Tao Bao
e838d1446c releasetools: Clean up ReplaceVerityKeyId and add tests.
Test: python -m unittest test_sign_target_files_apks
Test: Run sign_target_files_apks.py on marlin target_files.zip.
Change-Id: Ic3c3f4f14c73f8f8e48a8341e024e0861e665989
2018-01-02 12:29:31 -08:00
Tao Bao
1c830bfbaa releasetools: Add common.MakeTempDir().
This simplifies the work to track mkdtemp()'d dirs. This CL also cleans
up the temporary file creation in build_image.py.

Test: `m dist`
Test: python -m unittest test_common
Change-Id: Ied7d86126714b9b26e5df7bcca1dfdea21cfcf60
2018-01-02 12:22:01 -08:00
Treehugger Robot
7d100ce03b Merge "releasetools: Clean up sign_target_files_apks.py." 2018-01-02 19:56:03 +00:00
Tao Bao
0c28d2d02d releasetools: Clean up sign_target_files_apks.py.
Mostly cosmetic changes, such as replacing print statement with print
function. Also change 'import cStringIO' to optionally look for the one
in io module, to allow Python 2/3 compatibility.

Test: pylint --rcfile=pylintrc sign_target_files_apks.py
Test: Run sign_target_files_apks.py on marlin target_files.zip.
Change-Id: I4dc98b01da6f89e624114bbca5522f659901c1f2
2018-01-02 09:57:29 -08:00
Tao Bao
186ec99eb9 releasetools: Fix a bug in blockimgdiff.HeapItem.
HeapItem defines __bool__(), which contains a logical error that should
return the opposite value.

Note that the bug only manifests while using Python 3, which calls
__bool__(). With Python 2, `if x:` or bool(x) actually calls
x.__nonzero__() or x.__len__(). If a class defines neither __len__() nor
__nonzero__(), as the case in HeapItem, it always returns True.

Test: python -m unittest test_blockimgdiff
Test: python3 -m unittest test_blockimgdiff
Test: Generate an incremental non-A/B OTA package successfully.
Change-Id: Ibe8430e0b495a7d2f430cfffb716d2536ffb53d2
2018-01-02 09:54:32 -08:00
Tao Bao
0582cb639f releasetools: Remove the dead code for blockimgdiff_version <= 2.
The code is on infeasible path since we already have assertions in
common.BlockDifference().

Also remove the dead code that checks for OPTIONS.info_dict, as we
already set that in ota_from_target_files.main(), for both of A/B and
non-A/B.

Test: Generate incremental OTAs w/ and w/o the CL, and get identical
      packages.
Change-Id: Ifb8fc101e78f5ce58c60c8e49028b66ce0d20246
2017-12-21 11:53:27 -08:00
Tao Bao
2f5358d38b releasetools: Fix the bug that skips loading OEM dict.
The CL in [1] unintentionally breaks the OEM dict loading logic in the
incremental BBOTA path. We should always require and load the OEM
property dict if _either_ of the source and target builds uses OEM
properties. Otherwise with the current "and" operator, it skips loading
the OEM property dict and thus fails to generate an OTA package that has
OEM property changes (e.g. updating from build with fingerprint to
another one using thumbprint).

The CL in [1] actually makes the right change in the file-based OTA
path, but introduces the bug in the block-based OTA path.

This CL also cleans up the line that reads recovery_mount_options.

[1] commit 7f804ba71f ("releasetools:
allow for multiple OEM property values.").

Test: Genearte an OTA that has OEM property changes successfully.
Change-Id: Idce4ad59825d432618535ce09ab22bd7ddc524f2
2017-12-20 23:06:02 -08:00
Treehugger Robot
b379762089 Merge "build_image: Factor out CheckHeadroom() and add tests." 2017-12-19 00:00:41 +00:00
Treehugger Robot
b08567cbb6 Merge "releasetools: Fix the tag replacement for ro.build.vendor.fingerprint." 2017-12-18 22:18:14 +00:00
Treehugger Robot
d30b75190d Merge "Remove the support for BOARD_HAS_EXT4_RESERVED_BLOCKS." 2017-12-18 22:17:46 +00:00
Treehugger Robot
e58d385df1 Merge "releasetools: Use delta_generator to verify payload signatures." 2017-12-15 23:38:12 +00:00
Tao Bao
750385e455 releasetools: Use delta_generator to verify payload signatures.
We used to take a hard approach by parsing the payload with Python
script. This can be done by calling deleta_generator directly, which
also avoids the dependency on protobuf.

- Passing case
$ ./build/make/tools/releasetools/check_ota_package_signature.py \
    build/target/product/security/testkey.x509.pem \
    out/dist/aosp_marlin-ota-eng.zip

Package: out/dist/aosp_marlin-ota-eng.zip
Certificate: build/target/product/security/testkey.x509.pem
  ...
Whole package signature VERIFIED

Verifying A/B OTA payload signatures...
[1215/122842:INFO:generate_delta_main.cc(171)] Verifying signed payload.
[1215/122845:INFO:payload_verifier.cc(93)] signature blob size = 264
[1215/122845:INFO:payload_verifier.cc(112)] Verified correct signature 1 out of 1 signatures.
[1215/122845:INFO:payload_verifier.cc(93)] signature blob size = 264
[1215/122845:INFO:payload_verifier.cc(112)] Verified correct signature 1 out of 1 signatures.
[1215/122845:INFO:generate_delta_main.cc(181)] Done verifying signed payload.

Payload signatures VERIFIED

$ echo $?
0

- Failing case
Sign the whole package file with a different key, but leaving payload entries intact.

$ ./build/make/tools/releasetools/check_ota_package_signature.py \
    testkey2.x509.pem \
    marlin-ota-mismatching.zip
Package: marlin-ota-mismatching.zip
Certificate: testkey2.x509.pem
  ...
Whole package signature VERIFIED

Verifying A/B OTA payload signatures...
[1215/123054:INFO:generate_delta_main.cc(171)] Verifying signed payload.
[1215/123056:INFO:payload_verifier.cc(93)] signature blob size = 264
[1215/123056:ERROR:payload_verifier.cc(118)] None of the 1 signatures is correct. Expected:
[1215/123056:INFO:utils.cc(444)] Logging array of length: 256
[1215/123056:INFO:utils.cc(461)] 0x00000000 : 00 01 ff ff ff ff ff ff ff ff ff ff ff ff ff ff
  ...
[1215/123056:ERROR:payload_verifier.cc(121)] But found decrypted hashes:
[1215/123056:INFO:utils.cc(444)] Logging array of length: 256
[1215/123056:INFO:utils.cc(461)] 0x00000000 : 52 68 78 36 f6 9e cd 2d 5e 9f 31 d5 26 03 c9 aa
  ...
[1215/123056:ERROR:payload_signer.cc(333)] PayloadVerifier::VerifySignature( signature_blob, public_key_path, payload_hash) failed.
[1215/123056:INFO:generate_delta_main.cc(177)] VerifySignedPayload failed

    ERROR: Failed to verify payload with delta_generator: marlin-ota-mismatching.zip

$ echo $?
1

Bug: 65261072
Test: See above.
Change-Id: Id2e065655ec49b80dd2b13c6a859f41913be055b
2017-12-15 12:45:09 -08:00
Tao Bao
0c6a414fff releasetools: Remove the support for generating verify package.
We used to have "--gen_verify" that would generate a verify package for
non-A/B devices. Since a) we don't have active users; b) it works with
non-A/B only; c) we have better alternatives (e.g. using fastboot to
compute the partition checksum), this CL removes the support.

Test: `m dist`
Change-Id: Ib4a2c8c2a0394b54e66c106089aa52a0b900f034
2017-12-15 10:20:52 -08:00
Tao Bao
23f7ea9e70 Merge "releasetools: Add common.ZipDelete()." 2017-12-15 16:36:55 +00:00
Tao Bao
89d7ab23e8 releasetools: Add common.ZipDelete().
We have been shelling out to 'zip -d' to delete existing ZIP entries in
add_img_to_target_files.py. This CL moves the function into common.py,
and calls that for the similar work in ota_from_target_files.py. This CL
also adds unittests for the newly added function.

Test: `m dist`
Test: python -m unittest test_common
Test: ota_from_target_files.py generates identical packages w/ and w/o
      the CL (so we know the streaming property computation is intact).
Test: Run 'add_img_to_target_files.py -a' that triggers a call to
      ReplaceUpdatedFiles().
Change-Id: Icaa6c3ea3ee2166023f78fa79275295f837ea842
2017-12-14 17:17:04 -08:00
Treehugger Robot
78b5724622 Merge "releasetools: Remove '-w' from ota_from_target_files options." 2017-12-14 22:18:56 +00:00
Tao Bao
a7054eede1 releasetools: Fix the tag replacement for ro.build.vendor.fingerprint.
For devices using derived fingerprint (i.e. /system/build.prop doesn't
contain ro.build.fingerprint, but has ro.build.thumbprint instead), the
current code (in android.os.Build) doesn't have a matching logic to do
the same for ro.vendor.build.fingerprint. This means we will see
ro.build.thumbprint in /system/build.prop, while there's no matching
ro.vendor.build.thumbprint in /vendor/build.prop.

From signing script point of view, it should just apply the tag
replacement (e.g. test-keys -> release-keys) for whatever it sees when
signing a target_files.zip.

This CL also adds unit tests for EditTags() and RewriteProps().

Fixes: 27950003
Test: Use 'sign_target_files_apks.py' to sign a target that uses derived
      fingerprint and vendor partition. Check VENDOR/build.prop.
Test: python -m unittest test_sign_target_files_apks
Change-Id: I09019da970840cd82f54b68a32b4e94984bc1d8d
2017-12-08 15:19:47 -08:00
Tao Bao
d4349f2106 build_image: Factor out CheckHeadroom() and add tests.
The test is mostly trivial, but it ensures the result parsing from
actual calls to mke2fs.

Test: python -m unittest test_build_image
Test: `m dist` on aosp_marlin-userdebug (w/ and w/o defining
      PRODUCT_SYSTEM_HEADROOM respectively).
Change-Id: I8b9964213950e76f6d7d5518414a1bab888b4706
2017-12-07 23:39:56 -08:00
Jaegeuk Kim
10c9f5d9d3 Merge "core, release: add sload.f2fs for otatools.zip" 2017-12-08 00:57:42 +00:00
Tao Bao
79d52f8798 Remove the support for BOARD_HAS_EXT4_RESERVED_BLOCKS.
This is only used in file-based OTA, where we may reserve space on
/system partition. With the deprecation of file-based OTA since O, the
code has become obsolete.

Test: `m dist`
Change-Id: I2bd686b292f93b566fe28a2bcd74c564ffc75dac
2017-12-07 14:36:21 -08:00
Treehugger Robot
9a33b96103 Merge "Explain why BuildImage failed." 2017-12-07 21:45:59 +00:00
Tao Bao
337633f2b9 releasetools: Remove '-w' from ota_from_target_files options.
'-w' is a short option for '--wipe_user_data', which happens to be the
same as the whole-file signing flag for signapk.

This CL removes the short version to make it less error-prone. Users
should use '--wipe_user_data' to generate packages with enforced data
wipe.

Bug: 70259309
Test: ota_from_target_files.py generates packages with and without
      '--wipe_user_data'.
Test: Not seeing active users that uses 'ota_from_target_files -w'.
Change-Id: Ib8c138663a87aafc6a0b5b6f7c286fb74ef6134b
2017-12-06 15:47:03 -08:00
Xin Li
5f03da0404 Merge "DO NOT MERGE: Merge Oreo MR1 into master" 2017-12-06 23:18:30 +00:00
Treehugger Robot
7b2129f101 Merge "AVB: If building recovery.img, add AVB hash footer." 2017-12-06 22:12:25 +00:00
Elliott Hughes
73ff57f48a Explain why BuildImage failed.
BuildImage can return false for lots of reasons. Log two of the more
plausible ones. You'll still need to edit this file and pass True to
RunCommand to work out *what* went wrong, but at least this logging will
tell you *where* things went wrong.

Bug: http://b/63142920
Test: debugged my actual problem
Change-Id: If84427058108dcac0a1b5e2216b8bb11eef522ca
2017-12-06 12:16:39 -08:00
Tao Bao
9ad68d6e69 Merge "releasetools: Address pylint warnings in add_img_to_target_files.py."
am: 32c6c4a11e

Change-Id: I4e1d8ad1c6982f969b759f26cf1cb59862bd236f
2017-12-05 22:02:40 +00:00
Jaegeuk Kim
2ea1eba98c core, release: add sload.f2fs for otatools.zip
This also updates mkf2fsuserimg.sh to call sload.f2fs.

Change-Id: I245f6b3dfcd966fbc6783329d781375879fcd065
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-12-05 10:59:50 -08:00
Tao Bao
ead6281575 Merge "releasetools: Remove deprecated pylint options."
am: 3694e104b7

Change-Id: I87a9d6b07dc281bacc99e2935f0835958c3d67b5
2017-12-05 18:45:56 +00:00
Tao Bao
6b9fef5243 releasetools: Address pylint warnings in add_img_to_target_files.py.
C:661, 0: Wrong hanging indentation (add 2 spaces).
        OPTIONS.input_tmp, "VENDOR_IMAGES")
        ^ | (bad-continuation)
C:673, 0: Wrong continued indentation (add 14 spaces).
                os.path.join("IMAGES", img_name))
                ^             | (bad-continuation)
C: 54, 0: Import "import datetime" should be placed at the top of the module (wrong-import-position)
C: 55, 0: Import "import hashlib" should be placed at the top of the module (wrong-import-position)
C: 56, 0: Import "import os" should be placed at the top of the module (wrong-import-position)
C: 57, 0: Import "import shlex" should be placed at the top of the module (wrong-import-position)
C: 58, 0: Import "import shutil" should be placed at the top of the module (wrong-import-position)
C: 59, 0: Import "import subprocess" should be placed at the top of the module (wrong-import-position)
C: 60, 0: Import "import tempfile" should be placed at the top of the module (wrong-import-position)
C: 61, 0: Import "import uuid" should be placed at the top of the module (wrong-import-position)
C: 62, 0: Import "import zipfile" should be placed at the top of the module (wrong-import-position)
C: 64, 0: Import "import build_image" should be placed at the top of the module (wrong-import-position)
C: 65, 0: Import "import common" should be placed at the top of the module (wrong-import-position)
C: 66, 0: Import "import rangelib" should be placed at the top of the module (wrong-import-position)
C: 67, 0: Import "import sparse_img" should be placed at the top of the module (wrong-import-position)
W:391,12: Redefining built-in 'dir' (redefined-builtin)

After this CL, it still gives the warning below, which is to be
addressed with follow-ups.

R:635, 2: Too many nested blocks (6/5) (too-many-nested-blocks)

Test: `m dist`
Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Change-Id: I64da184b6b69e93449dbfc989a5d7f46d5223f42
2017-12-04 12:15:35 -08:00
Tao Bao
dac0c88d9d releasetools: Remove deprecated pylint options.
This option u'required-attributes' will be removed in Pylint 2.0
This option u'ignore-iface-methods' will be removed in Pylint 2.0

Test: pylint (1.6.5) no longer gives above warnings.
Change-Id: Ib07cd5aef87407260140c1fca0b11c439cbdf390
2017-12-04 12:15:23 -08:00
David Zeuthen
8fecb289ee AVB: If building recovery.img, add AVB hash footer.
In some non-A/B setups, recovery.img is still being used. If AVB is
enabled, we currently don't add a hash footer to recovery.img nor do
we include the hash digest in vbmeta.img. This CL fixes that.

This was tested on a build with the following settings

 TARGET_NO_RECOVERY := false
 BOARD_USES_RECOVERY_AS_BOOT := false
 BOARD_BUILD_SYSTEM_ROOT_IMAGE := false
 BOARD_RECOVERYIMAGE_PARTITION_SIZE := 33554432
 BOARD_AVB_RECOVERY_ADD_HASH_FOOTER_ARGS := --prop foo:bar

and then it was verified using 'avbtool info_image' that recovery.img
has a hash footer and a 'foo' property with the value 'bar'. This was
also checked successfully for vbmeta.img.

Test: See above.
Bug: None
Change-Id: I98124d5661ea768411416fa8d2a2ae6cc664fdc8
2017-12-04 13:54:38 -05:00
Tianjie Xu
4b57c28908 Merge "Improve the comment in AddSplitTransfers"
am: cca4a4c993

Change-Id: I853cd62997a964b91324d079e6940926c889ba6d
2017-11-29 21:57:52 +00:00
Tianjie Xu
41390d4e1e Improve the comment in AddSplitTransfers
Bug: 69624507
Test: build an incremental package
Change-Id: I54dac0a2ddcc19b722e352fb19d53ff20444fa42
2017-11-22 11:35:18 -08:00
Tianjie Xu
ff9efd3290 Merge "Fall back to normal split if imgdiff fails when splitting large apks"
am: 520c9f0f5f

Change-Id: I0db7558c996cc515a79a99ce1d27525728a041dc
2017-11-22 18:46:04 +00:00
Tianjie Xu
520c9f0f5f Merge "Fall back to normal split if imgdiff fails when splitting large apks" 2017-11-22 18:28:31 +00:00
Tianjie Xu
f68b50f963 Fall back to normal split if imgdiff fails when splitting large apks
Imgdiff expects the input files to be valid zip archives. During the
split of large apks, imgdiff may fail when there's a hole in the input
file; potentially due to the blocks allocation of mke2fs. This CL solves
the issue by falling back to normal split in case of the imgdiff split
failure. The split transfers will then use bsdiff instead.

Bug: 69624507
Test: generate the incremental package for the failed targets and check the
transfers.
Change-Id: I4882452378123e60ad3434053b57f33e53ac4b82
2017-11-21 23:37:36 -08:00
Tao Bao
51f5d3360e Merge "check_ota_package_signature.py: Set up PYTHONPATH."
am: 9f686938da

Change-Id: Ie84a386dd6acc44fd1fba987e8f0a5703248cd59
2017-11-21 21:40:13 +00:00
Tao Bao
d2ae0b0613 check_ota_package_signature.py: Set up PYTHONPATH.
... to include system/update_engine/scripts if ANDROID_BUILD_TOP is set
(e.g. after lunching a target).

Test: ./build/make/tools/releasetools/check_ota_package_signature.py
      works without manually setting PYTHONPATH.
Change-Id: I0c5101fab2dd69cb6a598b909aa04bc4d5f45284
2017-11-21 11:30:27 -08:00
Tao Bao
4d1764b0be Merge "releasetools: Allow building AVB-enabled recovery.img."
am: 37066490a5

Change-Id: Ie8393a7e81c4f056a12a0be5ed2cd07ed54ef76b
2017-11-21 17:07:57 +00:00