Commit graph

2906 commits

Author SHA1 Message Date
Xin Li
24c928cda8 Merge "Merge Android 13 QPR3" 2023-06-26 22:29:55 +00:00
Treehugger Robot
0e5e1749bb Merge "Handle symlinks when extracting zipfiles" 2023-06-22 01:27:35 +00:00
Kelvin Zhang
e473ce9e80 Handle symlinks when extracting zipfiles
python3.11's zipfile implementation does not handle symlinks. This
causes important symlinks in ramdisk to be broken, and later causing a
boo failure.

Test: unzip a target files with symlinks, make sure symlinks are created
Bug: 287896098

Change-Id: Ia7d6ac8ffb03807680a36ff648aa11afafb7f481
2023-06-21 13:31:26 -07:00
Treehugger Robot
5c3a34b358 Merge "Handle zip64 extra fields better" am: 57c1aa7b16
Original change: https://android-review.googlesource.com/c/platform/build/+/2628431

Change-Id: Ieefcc8ee08c2d91f447fc5209967d43db7454c75
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-17 23:04:21 +00:00
Treehugger Robot
57c1aa7b16 Merge "Handle zip64 extra fields better" 2023-06-17 22:12:26 +00:00
Kelvin Zhang
1e774245a4 Handle zip64 extra fields better
Test: check_target_files_signatures
Bug: 283033491
Change-Id: I7da89f8389c09cc99201cff342483c158bd7e9c1
2023-06-17 09:21:46 -07:00
Jooyung Han
4941f6fd47 Merge "Remove --blkid_path argument" am: e4635b3e04
Original change: https://android-review.googlesource.com/c/platform/build/+/2624574

Change-Id: Icb064093d587479713e7da5bd670d4cb36f977a3
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-16 07:30:28 +00:00
Jooyung Han
e4635b3e04 Merge "Remove --blkid_path argument" 2023-06-16 06:48:57 +00:00
Treehugger Robot
8ac8ce2b16 Merge changes I9c1a5346,I0a9d2c58 am: 5c73eb33db
Original change: https://android-review.googlesource.com/c/platform/build/+/2627031

Change-Id: Ied665b5cbd2d1b207c184458eef0ef49259df129
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-15 16:36:23 +00:00
Kelvin Zhang
38d0c373ac Fix python3.11's support for zip64
Bug: 283033491
Test: check_target_files_signatures -v -l
Change-Id: I9c1a5346e3a5f3920242dc9a5268d999f50a4937
2023-06-14 12:54:57 -07:00
Kelvin Zhang
b789e84499 Search for partition maps in IMAGES dir as well
Partition images are allowed to be in either IMAGES/ or RADIO/ dir of a
target_files zip, so when searching for .map files we should look in
both dirs.

Test: th
Bug: 227848550
Change-Id: I0a9d2c582d8f5d570237434902fac012513c9aad
2023-06-14 10:01:21 -07:00
Jooyung Han
629490292a Remove --blkid_path argument
deapexer doesn't need it.

Bug: 279858383
Test: presubmit
Change-Id: If0ec42b5edd4642f07c96ba641030c4dd6fb4660
2023-06-14 15:16:34 +09:00
Treehugger Robot
b2dec242f8 Merge "Supply apex_info.pb as parameter to delta_generator if file exists" am: df2af7c840
Original change: https://android-review.googlesource.com/c/platform/build/+/2617618

Change-Id: I9751859fb7114e3cc216860bb2cb951fcefe74fa
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-09 20:04:17 +00:00
Håkan Kvist
ddb968dce0 Supply apex_info.pb as parameter to delta_generator if file exists
Supply generator with parameter --apex_info_file=META/apex_info.pb
if the file exists.

This ensures that apex_info is included in payload header.

This is identical to the behaviour of brillo_update_payload
which is not being used since:
Invoke delta_generator directly
fcd731e3d6

Issue: 286253576
Test: Manual, confirm that apex_info is included in payload header
Change-Id: Ic096c5f8966beec8686f918aba462c955290a6c5
2023-06-09 12:46:58 +02:00
Treehugger Robot
980458ae17 Merge "fixup! Allow ota_from_target_file to work entirely on directories" am: d60dbf9773
Original change: https://android-review.googlesource.com/c/platform/build/+/2617616

Change-Id: I093dccc54174e68f282d9a0f79e6a5ba7f2555ea
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-08 23:08:30 +00:00
Håkan Kvist
b8982fc538 fixup! Allow ota_from_target_file to work entirely on directories
The following log message would always be shown:
WARNING : Cannot find care map file in target_file package

Break out of the care map copying loop as soon a file has been
copied. This ensures that else statement is only executed if no
care map file exists.

Test: Manual. Run ota_from_target_files with target-zip with and
    without care map files.
Change-Id: Ia196aa182ed81f21424317a7005f5634866b4b99
2023-06-08 15:17:57 +02:00
Treehugger Robot
976a36fb8e Merge "Use python based unzip function for portability" am: 3cda7834aa
Original change: https://android-review.googlesource.com/c/platform/build/+/2615433

Change-Id: Id10787e68ac4a0a6ea0ada47a24f46dd7e128915
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-06 19:08:34 +00:00
Treehugger Robot
4c357806a3 Merge "Always set a avb salt for hermetic build" am: fff48d788f
Original change: https://android-review.googlesource.com/c/platform/build/+/2613794

Change-Id: I3feffd1c65f8e92e7c6a6c6e0481e206b28ec8dd
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-06 19:07:16 +00:00
Kelvin Zhang
7c9205b008 Use python based unzip function for portability
Bug: 283033491
Test: th
Change-Id: Ief86b55c1d4a14220a0fb593c583a721d59cf86c
2023-06-05 15:54:12 -07:00
Kelvin Zhang
c819b29f46 Always set a avb salt for hermetic build
When building images via `m` , build_image.py is invoked directly
without going through add_img_to_target_files. To ensure images built in
either way are identical, move uuid/salt computation to build_image.py,
so that the same uuid/salt will be used.

Bug: 281960439
Test: m installclean && m && m target-files-dir , maks sure images in
$OUT and $OUT/obj/PACKING/target_files_intermediates are identical

Change-Id: Icdab29df84f5a0ec7c080f99f9fdbdc3c9b10b90
2023-06-05 09:59:54 -07:00
Treehugger Robot
8f26a9ca1d Merge "Adding option to disable fastboot_info" am: ecf635f507
Original change: https://android-review.googlesource.com/c/platform/build/+/2612133

Change-Id: I45872eab4df9b1044ff6a6fb1c582f5cd316a365
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-03 02:42:53 +00:00
Treehugger Robot
ecf635f507 Merge "Adding option to disable fastboot_info" 2023-06-03 01:56:41 +00:00
Treehugger Robot
b5535214ad Merge "Open ab_partitions.txt with just read access" am: f9a6f5f06b
Original change: https://android-review.googlesource.com/c/platform/build/+/2611511

Change-Id: I075bf60c1586dd0af5d39cc23db97e9ae5c5532d
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-01 22:42:53 +00:00
Daniel Zheng
ee10d07057 Adding option to disable fastboot_info
Fastboot_info can be disabled if use_fastboot_info is set to false.
Adding this flag as fastboot-info.txt is currently broken

Test: m updatepackage -> inspect contents
Bug: 284263071
Change-Id: I3e0ca13968ba9747cc39284ea6798981d22ad5e5
2023-06-01 14:21:58 -07:00
Kelvin Zhang
89b87f6aa9 Open ab_partitions.txt with just read access
We don't actually need write permission, so going with least privilege
principle. We have observed some mysterious permission denied errors on server environments. Without detailed logs or access to the server it's hard to pinpoint what the root cause is. This is an attempt/hypothesis to fix the permission denied error.

Test: th
Bug: 283033491
Change-Id: I52dc360d593aab57c749109994bf3e1e3625d0ce
2023-06-01 17:28:25 +00:00
Treehugger Robot
592ad2cb13 Merge "Fix merge_target_files in GRF case" am: b3adc24301
Original change: https://android-review.googlesource.com/c/platform/build/+/2564310

Change-Id: I564bb6d43bfa5b9cb79463f0ba50038d03ef6abb
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-25 11:37:45 +00:00
Treehugger Robot
b3adc24301 Merge "Fix merge_target_files in GRF case" 2023-05-25 10:46:32 +00:00
Treehugger Robot
74ad39125c Merge "Unsparse images before generating OTA" am: b4406d06f0
Original change: https://android-review.googlesource.com/c/platform/build/+/2598008

Change-Id: I40ac20dd7b30d11e1eda77688718566af4c1c416
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 21:22:54 +00:00
Treehugger Robot
b4406d06f0 Merge "Unsparse images before generating OTA" 2023-05-23 20:51:10 +00:00
Treehugger Robot
f6d61bd3c1 Merge "Add 16K artifacts to PREBUILT_IMAGES/ dir of target_files" am: 8485145bd7
Original change: https://android-review.googlesource.com/c/platform/build/+/2595883

Change-Id: I36815cfa877645e242d40a3b4b4cadd0e50a6ef5
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 03:30:56 +00:00
Kelvin Zhang
2268091521 Unsparse images before generating OTA
Test: th
Bug: 283172692
Change-Id: Ie6d3dc704fd9a8c107e2888222e4c2bf804dad3e
2023-05-22 10:22:59 -07:00
Kelvin Zhang
decee4adb7 Add 16K artifacts to PREBUILT_IMAGES/ dir of target_files
Test: th
Bug: 283506035
Change-Id: I9be3982b5a4324ec839937d10c33266c7d27c525
2023-05-19 14:31:00 -07:00
Treehugger Robot
e1623bb140 Merge "Allow source build to be an extracted target_files" am: fce28c1b97
Original change: https://android-review.googlesource.com/c/platform/build/+/2589846

Change-Id: I0a4ebf623140c02b39c7d3cb2e9bd060b7ea4090
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-16 23:14:19 +00:00
Treehugger Robot
fce28c1b97 Merge "Allow source build to be an extracted target_files" 2023-05-16 22:56:29 +00:00
Treehugger Robot
a2ebcf5e03 Merge "Support custom payload signer in merge_ota" am: bb404f2b27
Original change: https://android-review.googlesource.com/c/platform/build/+/2587127

Change-Id: Idd3456e86e1785b5611810f71a42042b66d5610c
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-16 22:31:54 +00:00
Satoshi Futenma
1f93ce20c5 Support custom payload signer in merge_ota
Add options to handle custom payload signer as it is required in
merge_ota.py as well if the original OTA packages are signed by
the signer.

If input is only one OTA, clone apex_info.pb to the target.

Use common.ZipWriteStr instead of zipfile.writestr, this ensures
that the same permission for the entries as done by
ota_from_target_files.

Bug: 282189563
Test: manual. pass single OTA to merge_ota, with same signing
    parameters as originally used. Confirm that output zip is
    binary identical to input.
Change-Id: I3b926b8cd69bc74dff6ccf8b5ccc72bedffcac6f
2023-05-16 10:21:05 +02:00
Kelvin Zhang
b97416e09e Allow source build to be an extracted target_files
common.LoadInfoDict() already supports loading from extracted
directories, just use it.

Test: generate an incremental OTA where both inputs are directories
Bug: 227848550
Bug: 277028723
Change-Id: Iedba831bb4d65d971df6b2ac95279e3234a02e2f
2023-05-15 11:13:26 -07:00
Treehugger Robot
531125c397 Merge "parallelize add_partition_calls only if output_zip is None" am: 8d5d2257d9
Original change: https://android-review.googlesource.com/c/platform/build/+/2584696

Change-Id: Ida95b894dc799345b02aa64b58f4d6392db92d7f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-12 11:39:56 +00:00
Treehugger Robot
8d5d2257d9 Merge "parallelize add_partition_calls only if output_zip is None" 2023-05-12 10:56:23 +00:00
Jeongik Cha
fa9090946e parallelize add_partition_calls only if output_zip is None
if output_zip isn't None, writing into zip file in parallel is not
thread-safe.

Bug: 281960217
Test: m dist
Change-Id: I10d68a4bb779cee244f40410ec95d38ca6040306
2023-05-12 15:14:58 +09:00
Treehugger Robot
d5dc0c17fe Merge "parallelize add_partition_calls" am: 96b51b54b6
Original change: https://android-review.googlesource.com/c/platform/build/+/2585397

Change-Id: I58e1b516a5109eaed483af1c8b9979f838e0b96f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-12 05:19:41 +00:00
Daniel Zheng
6b558b3fe4 Merge "Add fastboot-info to updatepackage" am: 717591dd4e
Original change: https://android-review.googlesource.com/c/platform/build/+/2583539

Change-Id: I92a70b845d04e2f251eed2b96fbcfe58d57ffeea
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-12 05:19:25 +00:00
Treehugger Robot
96b51b54b6 Merge "parallelize add_partition_calls" 2023-05-12 04:54:11 +00:00
Daniel Zheng
717591dd4e Merge "Add fastboot-info to updatepackage" 2023-05-12 04:31:55 +00:00
Jeongik Cha
03d014a03f parallelize add_partition_calls
it made total time equal to the longest image build

1m10s->30s in local build

Bug: 281960217
Test: m dist
Change-Id: I13d4f45d9b46b39292a014e3b4e1913365d89b7a
2023-05-12 08:43:44 +09:00
Kelvin Zhang
06417aaa9e Merge "Fix non-AB ota generation for directories" am: ea70695d92
Original change: https://android-review.googlesource.com/c/platform/build/+/2585207

Change-Id: I7a6472946360238393a734640e27557f7623446f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-10 04:46:20 +00:00
Kelvin Zhang
b036007592 Fix non-AB ota generation for directories
Test: ota_from_target_files -v --block yukawa-target_files-10097950.zip
test.zip

Change-Id: I4291010d66cc28bdb6b6b334cdbed06daf3de837
2023-05-09 20:30:53 -07:00
Daniel Zheng
380cb8065a Add fastboot-info to updatepackage
Modifying img_from_target_files to also add fastboot-info to
updatepackage

Test: m updatepackage
Bug: 194686221
Change-Id: I2e08c4269f0d83417b9d7079633bc28796d1cdd6
2023-05-09 13:44:22 -07:00
Treehugger Robot
5e4f59f9ca Merge "Support generating partial OTAs from extracted target_files" am: c3216a321b
Original change: https://android-review.googlesource.com/c/platform/build/+/2577071

Change-Id: I79fa834772dc12fe26678238bcf80ecffd2a34ce
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-04 00:54:58 +00:00
Kelvin Zhang
6b10e15acc Support generating partial OTAs from extracted target_files
This allows the build system to potentially paralleize generation of OTA
package and zipping of target files

Bug: 262185376
Bug: 227848550
Change-Id: I90b6c25761683ebe3803b22fc8e23540a5282c66
2023-05-02 20:48:14 -07:00