Commit graph

2946 commits

Author SHA1 Message Date
Kelvin Zhang
b5661d6ef5 Add per partition timestamp to 16K boot OTA
update_engine requiers per partition timestamp if this partition's build
prop includes build.date.utc.

Test: th
Bug: 293313353
Change-Id: I6bf3de8e855f0830b26bdbe2cbdc78ba891662a7
2023-09-06 13:02:19 -07:00
Kelvin Zhang
dfa0c85705 Merge "Use seek() instead of writing 0s" into main 2023-09-01 17:00:52 +00:00
Treehugger Robot
145ca3034f Merge "Mark boot OTAs as partial OTAs" into main 2023-09-01 16:58:26 +00:00
Kelvin Zhang
8aa6525ad7 Use seek() instead of writing 0s
test_ZipWrite_large_file need to write a 4GB file to zipfile for
testing. Instead of writing sequence of 0s to the test file, use seek().

Test: th
Bug: 298138109
Change-Id: I886bfa0c3f21664e529ba7de9688909518148f61
2023-08-31 12:17:14 -07:00
Kelvin Zhang
c4f737406a Mark boot OTAs as partial OTAs
Boot partition only OTA must be marked as "partial". This is required
for update_engine to correctly copy non-updated partitions to the target
slot.

Test: th
Bug: 293313353
Change-Id: Ic89d03f214e12fa43f686d37854d6fed1d78a6a3
2023-08-31 12:17:14 -07:00
Treehugger Robot
c20ef7ca4a Merge "Avoid calling MergeUpdateEngineConfig for non-ab target" into main 2023-08-31 14:30:17 +00:00
Kelvin Zhang
506a30851e Add timestamp to 16K boot OTA
Test: th
Bug: 297536338
Change-Id: Id4919d2437079e36899cd4c56da011a29aa0b470
2023-08-29 19:38:19 -07:00
Jooyung Han
ccec6ad2ec ota_from_target_files requires apexd_host
apexd_host is invoked in check_target_files_vintf.py
to prepare the apex directory before invoking checkvintf. ota_from_target_files uses check_target_files_vintf.py via (python_library) releasetools_check_target_files_vintf.

Since soong doesn't inherit "required:" property from library to binary, adding the dependency manually to ota_from_target_files.

Bug: n/a
Test: $ m ota_from_target_files # should build apexd_host
Change-Id: Ie7e6c3acf376c23960f242294f5a9bf99fec26aa
2023-08-29 01:28:40 +00:00
Kelvin Zhang
f6fe0a9c9c Reduce OTA tools's dependency on brillo_update_payload
brillo_update_payload is thin bash wrapper for delta_generator, and
OTA tools are wrappers around brillo_update_payload. This is
inconveinent, as adding a new flag requires changing 3 places. Since
brillo_update_payload is really thin, skip it and invoke delta_generator
directly.

Test: th
Bug: 293313353
Change-Id: I2fd5af95c571ea45aea62f9eec475f0f4e864953
2023-08-25 13:49:09 -07:00
Kelvin Zhang
c7441e5907 Add a tool to generate OTA from images
During build, we will need to generate an OTA for boot partition using a
16K boot image. Typically, OTA is generated from target_files.zip . To
avoid relying on target_files.zip as a dependency for 16K OTA, add a
tool to generate OTA directly from a raw image.

Test: th, ota_from_raw_img --partition_name boot --output ota.zip $OUT/boot_16k.img
Bug: 293313353
Change-Id: I2076332faf2a8dc573450597efd481e285a49545
2023-08-23 12:37:29 -07:00
Himanshu Jakhmola
21ef2c600b Avoid calling MergeUpdateEngineConfig for non-ab target
super image generation is failing for non-ab target as it is not able to find update_engine_config.txt while calling MergeUpdateEngineConfig.

META/update_engine_config.txt exists when AB_OTA_UPDATER is true.
Avoid calling MergeUpdateEngineConfig for non-ab target.

Bug: 290811619
Change-Id: Ie88b31adce15df87f73031e1f5bff0d825601e0c
2023-08-22 04:08:01 +00:00
Treehugger Robot
ef9ba4d22b Merge "Check device's launch API level before using new compresison algo" into main 2023-08-17 00:35:36 +00:00
Kelvin Zhang
8f83000732 Check device's launch API level before using new compresison algo
If a T launch device later changes to LZ4(which is supported since U),
we will fallback to older compression algorithms, because if the full
OTA is applied on an T build, the update_engine on device won't support
lz4.

Bug: 295989519
Test: generate full OTA for T launched devices, make sure lz4 is not
used

Change-Id: I82fdf788e47e2a6daeaa4479bfecf317d8ebb5d5
2023-08-16 13:16:48 -07:00
David Iserovich
babdafe97e [config] Dist supports custom super image script
Some boards need to supply a custom super image script, and this
is supported via the BOARD_CUSTOM_BUILD_SUPER_IMAGE variable.

However, this variable is not used by img_from_target_files.py.

Change img_from_target_files.py to call the supplied script.

Bug: None
Change-Id: I6bca937534f58459d3419139ee7a37a10138ab8f
2023-08-10 18:12:44 +00:00
Daniel Zheng
31694e3f83 Merge "Adding option to configure compression level" into main 2023-08-09 20:03:44 +00:00
Daniel Zheng
74955f8883 Adding option to configure compression level
Adding option to ota_from_target_files to configure compression level.
This option can be configured via the --vabc_compression_param flag.
e.g. --vabc_compression_param=gz,9 specifies gz compression algorithm
using level 9 compression

Test: ota_from_target_files
Change-Id: Ifc851faccbb3fba466d45c9695aaab322a362081
2023-08-08 08:09:15 -07:00
Dennis Song
a380533f13 Remove system_dlkm from _FRAMEWORK_PARTITIONS
system_dlkm partition is related to GKI, so it should be
in the same category as boot image.

Test: merge_target_files \
        --path otatools \
        --framework-target-files framework-target_files.zip \
	--vendor-target-files vendor-target_files.zip \
	--output-target-files merged-target_files.zip
Bug: 294482075
Change-Id: Ie8f60d72f44d31e1bc3b556735b92ed0a2fda662
2023-08-04 07:08:17 +00:00
Daniel Zheng
3bff72fbc6 Adding supported compression methods
Adding in supported compression methods to help

Test: m ota_from_target_files
Change-Id: I1c3101f983eb8b38d43752530bd9f61a85fddba4
2023-08-03 15:32:29 -07:00
James Wu
c5e321a06e Revise minSdkVersion to fix failed mainline_modules_bundles-userdebug
Test: abtd tests
Bug: 293456031
Change-Id: I674c50690bd04b7e4867c4aceb31d5bdcb53be84
2023-08-02 19:25:58 +00:00
Treehugger Robot
092719a471 Merge "Remove OPTIONS.host_tools" into main 2023-08-02 01:52:27 +00:00
Cole Faust
6833d7d47a Remove OPTIONS.host_tools
It appears to be unused.

Test: Presubmits
Change-Id: Idd7f9dda996458b93da77c783a546cdb3d1e0047
2023-08-01 18:00:37 -07:00
Treehugger Robot
d5474ac3da Merge "Add the option for custom_image to be AVB or NONAVB" into main 2023-08-01 19:08:06 +00:00
Kelvin Zhang
5b9ca88501 Support generating secondary OTAs from directories
Test: ota_from_target_files --include_secondary
Change-Id: I1004defdb4fe601a6ec733a7ceb213de7d540174
2023-08-01 10:04:08 -07:00
Kelvin Zhang
456c6d9a67 Adding an option to disable puffdiff
This is handy for quick OTA generation during development.

Change-Id: I878f7cb3479d6338c04aca44d16e48e83270252c
2023-08-01 10:03:54 -07:00
Ray-cy.lee
e97e0cb16b Add the option for custom_image to be AVB or NONAVB
Check BOARD_AVB_$(call to-upper,$(partition))_KEY_PATH to decide whether custom_image should sign AVB or not. If key path isn't set, the custom image will be excluded from AVB and copied to /IMAGES in target-files directly. This allows vendor to use custom_images flow packing unsigned image.
And to every non-avb custom partition, one image whose name is partition name must be added in its BOARD_<CUSTOM_PARTITION>_IMAGE_LIST.

BOARD_CUSTOMIMAGES_PARTITION_LIST := tvconfig
BOARD_TVCONFIG_IMAGE_LIST := \
    device/xxxx/yyyy/tvconfig.img \
    device/xxxx/yyyy/tvconfig_custom1.img

Test:
1) Build image, target-files, OTA package by m and m dist
2) Sign images by sign_target_files_apk.py

Fix: 285227850
Change-Id: I7477dafe023e4b168f0f08fb7aedd9e511a60e1b
2023-07-26 20:21:29 +08:00
Kelvin Zhang
43df0805c2 Fix test failures in releasetools
new_path used before assignment, use `path` instead.

Bug: 280425458
Change-Id: I55086f92cf109af67032614cefde60e6478bb149
2023-07-24 13:16:03 -07:00
Justin Yun
22ce9479be Disable copying images for target_files.zip
For the OTA build, target_files.zip requires setting fixed timestamp
for images. However, it caused regression on 'adb sync'.
Restoring image build process for target_files.zip instead of copying
them from the PRODUCT_OUT, only the images in target_files.zip set the
fixed timestamp.
By setting COPY_IMAGES_FOR_TARGET_FILES_ZIP flag, we still may copy
the images for target_files.zip for future use.

Bug: 287534409
Test: m; flashall; adb sync
Test: m dist; flash images from target_files.zip and check the timestamp
Change-Id: I4135eb721c33581f191621847e6595833263d707
2023-07-17 07:46:48 +09:00
Cole Faust
f5350e965b Merge "Remove BOARD_BPT* variables and the bpt parition table image" into main 2023-07-12 01:26:51 +00:00
Cole Faust
4f3dc8759f Remove BOARD_BPT* variables and the bpt parition table image
These variables appear to never be set.

Test: Presubmits
Change-Id: If4b89376f73204ad5780dff421f1216da255c3eb
2023-07-11 16:58:06 -07:00
Kelvin Zhang
f3f6d0569b Merge "Mark extracted files as executable" into main 2023-07-11 17:39:31 +00:00
Elliott Hughes
c1577ff58b Merge "Switch from minigzip to gzip." into main 2023-07-11 15:52:27 +00:00
Kelvin Zhang
4cb28f6f80 Mark extracted files as executable
Python's zipfile doens't restore file permission by default, so we need
to manually restore permission.

Test: th
Bug: 290643514
Change-Id: I89c1e2ee178b534fa7e3f02afd04d170100d37e7
2023-07-10 13:03:07 -07:00
Treehugger Robot
0dc5eeaaca Merge "Make test more deterministic by setting search_path to None" 2023-06-28 22:28:28 +00:00
Jooyung Han
c269ec3538 Merge changes from topic "apexd_host"
* changes:
  Use apexd_host for host-side APEX extraction
  Deprecate flattened apexes
2023-06-28 17:36:15 +00:00
Kelvin Zhang
03dc6ee96f Make test more deterministic by setting search_path to None
Test: th
Bug: 2640751
Change-Id: I9779dfa40c1e8319cb67ae0928c977f5a10cda5f
2023-06-28 10:10:39 -07:00
Treehugger Robot
59f99b31eb Merge "Fix test flake" 2023-06-28 01:18:55 +00:00
Elliott Hughes
97ad120adb Switch from minigzip to gzip.
Bug: http://b/288169261
Test: treehugger
Change-Id: I404cbcb850734ed8291c2215a4329fb372806d63
2023-06-27 15:39:05 -07:00
Kelvin Zhang
97a5afe0f3 Fix test flake
Test: th
Bug: 289013326
Change-Id: Ib233978ff861b010bdc3bd1a02ea9baefe6a3489
2023-06-27 10:30:48 -07:00
Jooyung Han
c9542abbbd Use apexd_host for host-side APEX extraction
Host-side simulation of APEX activation is done by a new tool
'apexd_host'. This simplies checkvintf invocation for local builds and
for target-files.

For local builds, checkvintf no londer depends on $OUT/apex, a flattened
view of APEXes. In fact, the build system doesn't need to install
$OUT/apex. They are installed for now only for its side-effect of
installing symbol files. We'd better not rely on $OUT/apex.

For target-files, scanning/activating apexes are extracted and moved to
the new tool. Now check_target_files_vintf is not more efficient because
it doesn't copy .apex files to a temporary directory.

Bug: 260358957
Bug: 288826922
Test: m (running checkvintf) # for local builds
Test: m target-files-package && check_garget_files_vintf target-files.zip
Test: atest releasetools_test
Change-Id: Iba23f429d96f9ec31814196aa14bdbb800649218
2023-06-27 17:00:06 +09:00
Jooyung Han
2ac1f2fabc Deprecate flattened apexes
Flattened apexes are deprecated. mege_dexopt does not need to care about
flattened apexes.

Bug: 278826656
Test: m otapackage
Change-Id: I527eed64c5f1a7466e2c48d73360f47f6cfa0911
2023-06-27 13:01:56 +09:00
Xin Li
24c928cda8 Merge "Merge Android 13 QPR3" 2023-06-26 22:29:55 +00:00
Treehugger Robot
0e5e1749bb Merge "Handle symlinks when extracting zipfiles" 2023-06-22 01:27:35 +00:00
Kelvin Zhang
e473ce9e80 Handle symlinks when extracting zipfiles
python3.11's zipfile implementation does not handle symlinks. This
causes important symlinks in ramdisk to be broken, and later causing a
boo failure.

Test: unzip a target files with symlinks, make sure symlinks are created
Bug: 287896098

Change-Id: Ia7d6ac8ffb03807680a36ff648aa11afafb7f481
2023-06-21 13:31:26 -07:00
Treehugger Robot
5c3a34b358 Merge "Handle zip64 extra fields better" am: 57c1aa7b16
Original change: https://android-review.googlesource.com/c/platform/build/+/2628431

Change-Id: Ieefcc8ee08c2d91f447fc5209967d43db7454c75
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-17 23:04:21 +00:00
Treehugger Robot
57c1aa7b16 Merge "Handle zip64 extra fields better" 2023-06-17 22:12:26 +00:00
Kelvin Zhang
1e774245a4 Handle zip64 extra fields better
Test: check_target_files_signatures
Bug: 283033491
Change-Id: I7da89f8389c09cc99201cff342483c158bd7e9c1
2023-06-17 09:21:46 -07:00
Jooyung Han
4941f6fd47 Merge "Remove --blkid_path argument" am: e4635b3e04
Original change: https://android-review.googlesource.com/c/platform/build/+/2624574

Change-Id: Icb064093d587479713e7da5bd670d4cb36f977a3
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-16 07:30:28 +00:00
Jooyung Han
e4635b3e04 Merge "Remove --blkid_path argument" 2023-06-16 06:48:57 +00:00
Treehugger Robot
8ac8ce2b16 Merge changes I9c1a5346,I0a9d2c58 am: 5c73eb33db
Original change: https://android-review.googlesource.com/c/platform/build/+/2627031

Change-Id: Ied665b5cbd2d1b207c184458eef0ef49259df129
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-15 16:36:23 +00:00
Kelvin Zhang
38d0c373ac Fix python3.11's support for zip64
Bug: 283033491
Test: check_target_files_signatures -v -l
Change-Id: I9c1a5346e3a5f3920242dc9a5268d999f50a4937
2023-06-14 12:54:57 -07:00