bsdiff/imgdiff returns non-zero values correctly in case of failures.
So we don't need to check the stderr anymore. This avoids some false
reports of the diff program.
Bug: 72335938
Bug: 71505046
Test: Check the exit value of bsdiff/imgdiff in code search,
generate a package for angler.
Change-Id: I18f0c3882a40a5288d6aee715713a05270e0db2b
Now they (A/B, non-A/B full and non-A/B incremental) share the same
code. Also add tests for the newly added function.
Test: python -m unittest test_ota_from_target_files
Test: Build A/B and non-A/B packages respectively.
Change-Id: I5499dd9c683ac1a18d8a24f21ff8845fe69f343e
This helps to generate a deterministic package.
Bug: 71770360
Test: Generate a incremental package and transfers are added by file name.
Change-Id: I7562a200b97a1babbb09a77801324cc9408cc01f
In non-A/B OTA path, we've been loading the info dict for the target
build twice (once to have a peek at the OTA type, with a reload after
unzipping the target-files zip). Remove the reloading.
This CL moves the loading of the source info dict up a bit to avoid
having two copies in the A/B and non-A/B paths. It also moves up the
sanity checks of the loaded info dicts before unzipping the target-files
zips.
Test: Generate full and incremental packages for bullhead and marlin.
Change-Id: Iccb953feb0b6ecf62e6f73d6e9ceea00077d098d
GetBuildProp() and GetVendorBuildProp() have been factored into
BuildInfo class. These two functions were meant to be deleted in commit
481bab8d48.
Test: python -m unittest test_ota_from_target_files
Test: No users of these two functions.
Change-Id: Id590a46d597939973ef2339cdfea4f5163548d4d
The fallback-to-full logic doesn't look beneficial, but may hide away
real bugs, especially when it's on by default.
Test: Generate an incremental non-A/B OTA successfully.
Test: Inject bugs into script; observe that incremental OTA
generation errors out instead of silently falling back.
Test: No internal users of the flag from code search.
Change-Id: Ifc627d898552c0bd72463e8c107e36610421608e
We split large apks and generated patches for them in parallel,
resulting in nondeterminate packages between different runs. This CL
sort the split transfers by target name first; and then add them
sequentially to the final transfer list.
Also fix a side effect where we may generate a wrong sha1 for split
ranges due to synchronization error.
Bug: 71770360
Bug: 71759418
Test: Generate the package several times, compare the log and the transfer list.
Change-Id: I2a49e22594d59ffaa98b11edc776be4e3c4c561f
Prior to this CL, we have similar codes in different paths
(WriteFullOTAPackage / WriteBlockIncrementalOTAPackage /
WriteABOTAPackageWithBrilloScript). This CL factors out the common codes
that deal with OEM-specific properties into BuildInfo class, and adds
tests for the new class.
Test: python -m unittest test_ota_from_target_files
Test: Create an incremental package that uses and doesn't use OEM
properties respectively.
Test: Create a full package that uses and doesn't use OEM properties
respectively.
Change-Id: Ida914cace12803d55396fa503bafcac2db2a520e
Test: python -m unittest test_common
Test: Run sign_target_files_apks.py on a target with compressed APKs.
Change-Id: I107a8b8f2f0f82e2d1947f14c8a8b3778f633b11
Mostly cosmetic changes, such as moving print statement to print
function. The only functional change is to wrap the file opening in
Append() with try statement, which would dump the error message as
needed.
Test: `m dist`
Test: python -m unittest test_build_image
Change-Id: I4e6c593517cf737bfcbe51bc533b22a247b10e44
This simplifies the work to track mkdtemp()'d dirs. This CL also cleans
up the temporary file creation in build_image.py.
Test: `m dist`
Test: python -m unittest test_common
Change-Id: Ied7d86126714b9b26e5df7bcca1dfdea21cfcf60
Mostly cosmetic changes, such as replacing print statement with print
function. Also change 'import cStringIO' to optionally look for the one
in io module, to allow Python 2/3 compatibility.
Test: pylint --rcfile=pylintrc sign_target_files_apks.py
Test: Run sign_target_files_apks.py on marlin target_files.zip.
Change-Id: I4dc98b01da6f89e624114bbca5522f659901c1f2
HeapItem defines __bool__(), which contains a logical error that should
return the opposite value.
Note that the bug only manifests while using Python 3, which calls
__bool__(). With Python 2, `if x:` or bool(x) actually calls
x.__nonzero__() or x.__len__(). If a class defines neither __len__() nor
__nonzero__(), as the case in HeapItem, it always returns True.
Test: python -m unittest test_blockimgdiff
Test: python3 -m unittest test_blockimgdiff
Test: Generate an incremental non-A/B OTA package successfully.
Change-Id: Ibe8430e0b495a7d2f430cfffb716d2536ffb53d2
The code is on infeasible path since we already have assertions in
common.BlockDifference().
Also remove the dead code that checks for OPTIONS.info_dict, as we
already set that in ota_from_target_files.main(), for both of A/B and
non-A/B.
Test: Generate incremental OTAs w/ and w/o the CL, and get identical
packages.
Change-Id: Ifb8fc101e78f5ce58c60c8e49028b66ce0d20246
The CL in [1] unintentionally breaks the OEM dict loading logic in the
incremental BBOTA path. We should always require and load the OEM
property dict if _either_ of the source and target builds uses OEM
properties. Otherwise with the current "and" operator, it skips loading
the OEM property dict and thus fails to generate an OTA package that has
OEM property changes (e.g. updating from build with fingerprint to
another one using thumbprint).
The CL in [1] actually makes the right change in the file-based OTA
path, but introduces the bug in the block-based OTA path.
This CL also cleans up the line that reads recovery_mount_options.
[1] commit 7f804ba71f ("releasetools:
allow for multiple OEM property values.").
Test: Genearte an OTA that has OEM property changes successfully.
Change-Id: Idce4ad59825d432618535ce09ab22bd7ddc524f2
We used to have "--gen_verify" that would generate a verify package for
non-A/B devices. Since a) we don't have active users; b) it works with
non-A/B only; c) we have better alternatives (e.g. using fastboot to
compute the partition checksum), this CL removes the support.
Test: `m dist`
Change-Id: Ib4a2c8c2a0394b54e66c106089aa52a0b900f034
We have been shelling out to 'zip -d' to delete existing ZIP entries in
add_img_to_target_files.py. This CL moves the function into common.py,
and calls that for the similar work in ota_from_target_files.py. This CL
also adds unittests for the newly added function.
Test: `m dist`
Test: python -m unittest test_common
Test: ota_from_target_files.py generates identical packages w/ and w/o
the CL (so we know the streaming property computation is intact).
Test: Run 'add_img_to_target_files.py -a' that triggers a call to
ReplaceUpdatedFiles().
Change-Id: Icaa6c3ea3ee2166023f78fa79275295f837ea842
For devices using derived fingerprint (i.e. /system/build.prop doesn't
contain ro.build.fingerprint, but has ro.build.thumbprint instead), the
current code (in android.os.Build) doesn't have a matching logic to do
the same for ro.vendor.build.fingerprint. This means we will see
ro.build.thumbprint in /system/build.prop, while there's no matching
ro.vendor.build.thumbprint in /vendor/build.prop.
From signing script point of view, it should just apply the tag
replacement (e.g. test-keys -> release-keys) for whatever it sees when
signing a target_files.zip.
This CL also adds unit tests for EditTags() and RewriteProps().
Fixes: 27950003
Test: Use 'sign_target_files_apks.py' to sign a target that uses derived
fingerprint and vendor partition. Check VENDOR/build.prop.
Test: python -m unittest test_sign_target_files_apks
Change-Id: I09019da970840cd82f54b68a32b4e94984bc1d8d
The test is mostly trivial, but it ensures the result parsing from
actual calls to mke2fs.
Test: python -m unittest test_build_image
Test: `m dist` on aosp_marlin-userdebug (w/ and w/o defining
PRODUCT_SYSTEM_HEADROOM respectively).
Change-Id: I8b9964213950e76f6d7d5518414a1bab888b4706
This is only used in file-based OTA, where we may reserve space on
/system partition. With the deprecation of file-based OTA since O, the
code has become obsolete.
Test: `m dist`
Change-Id: I2bd686b292f93b566fe28a2bcd74c564ffc75dac
'-w' is a short option for '--wipe_user_data', which happens to be the
same as the whole-file signing flag for signapk.
This CL removes the short version to make it less error-prone. Users
should use '--wipe_user_data' to generate packages with enforced data
wipe.
Bug: 70259309
Test: ota_from_target_files.py generates packages with and without
'--wipe_user_data'.
Test: Not seeing active users that uses 'ota_from_target_files -w'.
Change-Id: Ib8c138663a87aafc6a0b5b6f7c286fb74ef6134b
BuildImage can return false for lots of reasons. Log two of the more
plausible ones. You'll still need to edit this file and pass True to
RunCommand to work out *what* went wrong, but at least this logging will
tell you *where* things went wrong.
Bug: http://b/63142920
Test: debugged my actual problem
Change-Id: If84427058108dcac0a1b5e2216b8bb11eef522ca
This also updates mkf2fsuserimg.sh to call sload.f2fs.
Change-Id: I245f6b3dfcd966fbc6783329d781375879fcd065
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
C:661, 0: Wrong hanging indentation (add 2 spaces).
OPTIONS.input_tmp, "VENDOR_IMAGES")
^ | (bad-continuation)
C:673, 0: Wrong continued indentation (add 14 spaces).
os.path.join("IMAGES", img_name))
^ | (bad-continuation)
C: 54, 0: Import "import datetime" should be placed at the top of the module (wrong-import-position)
C: 55, 0: Import "import hashlib" should be placed at the top of the module (wrong-import-position)
C: 56, 0: Import "import os" should be placed at the top of the module (wrong-import-position)
C: 57, 0: Import "import shlex" should be placed at the top of the module (wrong-import-position)
C: 58, 0: Import "import shutil" should be placed at the top of the module (wrong-import-position)
C: 59, 0: Import "import subprocess" should be placed at the top of the module (wrong-import-position)
C: 60, 0: Import "import tempfile" should be placed at the top of the module (wrong-import-position)
C: 61, 0: Import "import uuid" should be placed at the top of the module (wrong-import-position)
C: 62, 0: Import "import zipfile" should be placed at the top of the module (wrong-import-position)
C: 64, 0: Import "import build_image" should be placed at the top of the module (wrong-import-position)
C: 65, 0: Import "import common" should be placed at the top of the module (wrong-import-position)
C: 66, 0: Import "import rangelib" should be placed at the top of the module (wrong-import-position)
C: 67, 0: Import "import sparse_img" should be placed at the top of the module (wrong-import-position)
W:391,12: Redefining built-in 'dir' (redefined-builtin)
After this CL, it still gives the warning below, which is to be
addressed with follow-ups.
R:635, 2: Too many nested blocks (6/5) (too-many-nested-blocks)
Test: `m dist`
Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Change-Id: I64da184b6b69e93449dbfc989a5d7f46d5223f42
This option u'required-attributes' will be removed in Pylint 2.0
This option u'ignore-iface-methods' will be removed in Pylint 2.0
Test: pylint (1.6.5) no longer gives above warnings.
Change-Id: Ib07cd5aef87407260140c1fca0b11c439cbdf390
In some non-A/B setups, recovery.img is still being used. If AVB is
enabled, we currently don't add a hash footer to recovery.img nor do
we include the hash digest in vbmeta.img. This CL fixes that.
This was tested on a build with the following settings
TARGET_NO_RECOVERY := false
BOARD_USES_RECOVERY_AS_BOOT := false
BOARD_BUILD_SYSTEM_ROOT_IMAGE := false
BOARD_RECOVERYIMAGE_PARTITION_SIZE := 33554432
BOARD_AVB_RECOVERY_ADD_HASH_FOOTER_ARGS := --prop foo:bar
and then it was verified using 'avbtool info_image' that recovery.img
has a hash footer and a 'foo' property with the value 'bar'. This was
also checked successfully for vbmeta.img.
Test: See above.
Bug: None
Change-Id: I98124d5661ea768411416fa8d2a2ae6cc664fdc8
