This CL refactors the non-A/B OTA generation code into a function,
so it allows adding common post-generation code for both flows.
It moves the code for target files diffing to common path, and fixes the
diffing functionality (broken due to the use of unzip pattern).
Test: Run ota_from_target_files with `--log_diff` flag to generate
non-A/B and A/B OTAs respectively.
Change-Id: I92e5549a58344fc56a151a709c06534913d265d5
The dependency was overlooked as the module was conditionally imported.
Bug: 63866463
Test: `m -j ota_from_target_files`; Run ota_from_target_files with
`--log_diff` flag (on a non-A/B target).
Change-Id: I71eff2e55cc28fd649f0ca1b38da53907dac6597
These two tools are only used within build system, so not adding them
to otatools.zip.
Bug: 63866463
Test: TreeHugger
Test: Build a non-A/B target that calls make_recovery_patch.
Change-Id: Iaeddc98a2c4a27c6b49d91047500029e460db9e3
The flag would otherwise be turned on unconditionally even when
importing merge_target_files as a module. As a result, `atest
releasetools_test` was broken / flaky prior to this change, as
PythonUnitTestResultParser doesn't like unexpected outputs in stderr.
Test: `atest releasetools_test releasetools_py3_test`
Change-Id: Ifd614c797d2cee3b33e4c2a7ad8e598850e9d60c
The lib will be shared by more users (e.g. add_img_to_target_files) in
follow-up changes.
Test: TreeHugger
Test: atest releasetools_test releasetools_py3_test
Change-Id: Iaa48bfc8c1dada41362d974773c124a001471ef8
Bug: 63866463
Test: TreeHugger
Test: m -j otapackage
Test: `m -j otatools-package`. Check bin/ota_from_target_files is
available. Build an OTA package with standalone otatools.
Change-Id: I533dee1c6997d233601e7f4eae637ff5743ed637
Taking an extracted target_files dir input is only needed for
add_img_to_target_files.py, in particular for the call from build
system. For the case of img_from_target_files.py, it goes much faster
with zip2zip (3m vs 7s in my local test). This CL drops the unneeded
code path in preparation for the switch to zip2zip.
Test: `python build/make/tools/releasetools/img_from_target_files.py \
-v aosp_taimen-target_files.zip aosp_taimen-img-test.zip`
Change-Id: I7f85d7d629d6fd1621a082f94a11cac7c3e6bbf2
The change in [1] allows brillo_update_payload to write to stdout/stderr
directly without buffering, to ensure dumping progress to screen
continously. However, it now unconditionally writes to stdout/stderr
even if caller doesn't specify `-v`. The behavior upsets
PythonUnitTestResultParser in [2], which doesn't want any unexpected
output from stderr for result parsing.
This CL updates the logic to only give continous outputs in verbose
mode.
[1] commit 2f7e11ef27
[2] https://android.googlesource.com/platform/tools/tradefederation/+/refs/heads/master/test_framework/com/android/tradefed/testtype/PythonUnitTestResultParser.java
Test: Run ota_from_target_files.py with `-v`. Check there's continous
output from brillo_update_payload.
Test: `python -m unittest -v test_ota_from_target_files > /dev/null`
doesn't show outputs from brillo_update_payload.
Change-Id: I3b851203ef011a5b4d982948cf52793f6fb6ad2d
Bug: 137853921
Bug: 138671115
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Test: Ran 'merge_builds --build_vbmeta' for two devices, one with the
vbmeta struct on system.img and another with vbmeta_system.img. Flashed
the regenerated vbmeta.img files on devices, devices boot.
Change-Id: I8d7585c7af468be3d242d8aceeed6d27e6fc6d96
This allows test runner to take care of the external tools setup, then
to start the test without a lunch'd environment. This is needed before
having supports like python_test_helper that packs dependents into
releasetools_test module.
Bug: 138791766
Test: Don't lunch any target. Use `FORCE_RUN_RELEASETOOLS=1 python -m
unittest -v test_add_img_to_target_files` and check there's no
skipped testcases.
Change-Id: Iafeaba54fc228d8e1fc5ff3c91f7ffebef18ad7f
Now that mm/ONE_SHOT_MAKEFILE have been removed, we can expect to know
about all of our dependencies at the end of the build.
This removes 19k nodes from our build graph (aosp-master
aosp_arm64-eng), though in a default build, only 3k of those are used.
Test: ALLOW_MISSING_DEPENDENCIES=true, then trigger a missing dependency
Test: treehugger
Test: create link_type files, then apply CleanSpec.mk, ensure they're removed
Change-Id: I9506331e4a9911d2f26e59a2f72a97aef1644073
The build servers are no longer using this. It's hard to search for
others using it -- I've cleaned them up as I've run across them, but
we'll just have to see who breaks once this goes in.
This was the last user of makeparallel, so we can remove it.
Test: make
Test: treehugger
Change-Id: If6df3f1a67d6a6df36afaa4b07cd88a48a364fe1
I've switched all the build server configs to using TARGET_PRODUCT /
TARGET_BUILD_VARIANT and explicit goals instead.
Remove tools/check_builds.sh which relied on this, but hasn't been
touched in a long time.
Test: m PRODUCT-test
Test: treehugger
Change-Id: If5f8c714567b33aeb38223c176ca24ea649eb57d
It used to be packed at IMAGES/recovery-two-step.img, but to serve OTA
purpose only.
Test: `m dist` with a non-A/B target. Check the file in the generated
target_files.zip.
Test: Create two-step package. Check that recovery-two-step.img is used.
Change-Id: Iec6a73c682e0f844cd8c0b758c9470fa35dd15d8
Based on the actual semantics, it actually wants an in-memory _bytes_
buffer (io.BytesIO), especially when running with Python 3. This CL
fixes the issue and adds a unittest.
Bug: 131631303
Test: python -m unittest test_sign_target_files_apks
Test: python3 -m unittest test_sign_target_files_apks
Change-Id: I3fb067acc26713f1842e831225607779fd0d1b7e
Bug: 73062966
Test: can load AIDs from partitions other than vendor
Test: bionic-unit-tests
Merged-In: Ia85abbeefe5a945369970f2aef42692e07ab8c09
Change-Id: Ia85abbeefe5a945369970f2aef42692e07ab8c09
(cherry picked from commit 8a54ec8482)
Bug: 73062966
Test: can load AIDs from partitions other than vendor
Test: bionic-unit-tests
Merged-In: Ia85abbeefe5a945369970f2aef42692e07ab8c09
Change-Id: Ia85abbeefe5a945369970f2aef42692e07ab8c09
aapt does not support parsing some referencing attributes.
Switch to latest tool, aapt2, which works better.
Test: aapt2 dump badging test.apk and returned 0.
Test: Run check_target_files_signatures.py with a target_files.zip.
Test: Run sign_target_files_apks.py with a target_files.zip.
Change-Id: Ib3a4740506f29ebab6930767f2aa8a0b5c4ba053
- Removes functions that can be replaced with one-line external methods
- Moves read_config_list to common alongside similar methods LoadDictionaryFrom*
- Runs pyformat on merge_target_files.py
Bug: 137853921
Test: python -m unittest test_merge_target_files
Test: Using merge_target_files.py to create a merged build, & booting.
Change-Id: I833c1086db41e1374057cc7447fc50d1915734a7
This helps to break the circular dependency between common and
blockimgdiff.
Bug: 32379627
Test: unit tests pass
Change-Id: I90b5ff34782acbfac86f36265bd96c207d898bf6
This allows sharing the same signing config on different target_files
zips. Nonexistent APEX will be ignored with a warning.
Bug: 137249701
Test: Run sign_target_files_apks with APEX overrides.
Change-Id: I2bad0f5c00753ed36ec5ae3431c7dc2ff1fc3e9c
Merged-In: I2bad0f5c00753ed36ec5ae3431c7dc2ff1fc3e9c
(cherry picked from commit b369c7226a)
For very long processes, we might want to keep stdout and stderr
by default to None.
So no redirection will occur in the child process as explained in:
https://docs.python.org/2/library/subprocess.html
That will result in the child process stdin and stderr to be same
than in common.py and avoid to have the logs blocked during the
child process execution and flushed only when child process terminates.
Since the logs are continously displayed, it allows to easily confirm
that the process is not blocked.
Bug: 133380588
Test: generate iota & Check that the logs are not blocked.
Change-Id: I6d6cb56547bf3a4a4334dfa22b6b2b05d2c36a5e
Signed-off-by: Regnier, Philippe <philippe.regnier@intel.com>
While in theory this could incur memory overhead for Python 2, the
impact is low for the existing use cases (plus we're moving away from
Python 2).
Bug: 131631303
Test: No additional occurrence of iteritems.
Test: Build with Python 3.
Change-Id: I0205c9edf25f46e3d85967c7dd2c1af035757741
This extends the original pin list generation to support specifying a
range within a file. If any part of a file is pinned, its local file
header in the APK file is pinned as well.
Test: Build cuttlefish.
Bug: 136040313
Bug: 135953430
Change-Id: I6de8c2b4c2f35c6bf257f61414c6bce761233e51
If a kernel happens to decode to gibberish (including
non-ascii bytes), catch the decoding error when
searching for the kernel version.
Bug: 137041171
Merged-In: Ic035b3a5c8c80025cb3cede7b0fdcf8a2e5a35fd
Change-Id: Ic035b3a5c8c80025cb3cede7b0fdcf8a2e5a35fd
And a few minor clean-ups to the styling.
Bug: 131631303
Test: python -m unittest test_merge_target_files
Test: python3 -m unittest test_merge_target_files
Test: Use `python merge_target_files` to merge two target_files zips.
Test: Use `python3 merge_target_files` to merge two target_files zips.
Change-Id: I8502dfb243408f658d022e8d5e5fbb60066e4ff0
Split merge_target_files function into several steps, so we can increate
readability and add conditional flows for other *SSI mixed build with less effort
Test: m -j & atest passed
Change-Id: I558f9dd5bca31b132a09cb36d9dfcd30c92efbc9
This CL changes the condition for building super_empty.img from
PRODUCT_BUILD_SUPER_PARTITION to PRODUCT_USE_DYNAMIC_PARTITIONS, as a
follow-up to the change in [1].
With the CL in [1], it skips building super.img and super_empty.img both
when turning off PRODUCT_BUILD_SUPER_PARTITION. However, the latter
should be mandatory whenever dynamic partitions is enabled. Because
fastboot relies on this file to properly flash dynamic partitions. Plus,
the cost for building super_empty.img is much lower than the one for
super.img.
As part of the change, it'll write group info into target_files when
building with PRODUCT_BUILD_SUPER_PARTITION == false. It's the work for
target_files merging script to determine the values to be picked up. The
current logic in merge_target_files.py always uses the one from vendor
target_files. This CL adds a testcase to ensure the behavior.
[1] https://android-review.googlesource.com/c/platform/build/+/928756
Bug: 135752763
Test: `m dist` with a target that sets PRODUCT_BUILD_SUPER_PARTITION to
false. Check the built artifacts contain super_empty.img. Verify
that the build can be flashed properly.
Change-Id: I277f087eab45663a6c3b33333d16e9e576c1c25c
This allows a consistent logic in using the avbtool which could be
board-specific.
Test: `atest releasetools_test`
Test: Run sign_target_files_apks.py on a target_files.zip.
Change-Id: I8cd93b8e71146985734f85c31f4662f5e2e9534c
This ensures a matching interface between sign_apex and apex_utils.
The test apex `testdata/foo.apex` is generated by running
`system/apex/apexer/runtests.sh`.
Test: python -m unittest test_sign_apex
Test: atest releasetools_test
Change-Id: I7c14b1df2a3038ad206aa3e5aac084c47baaa00b
