Commit graph

7949 commits

Author SHA1 Message Date
Ibrahim Kanouche
cf066f01f3 Merge "Add deps tracking for SBOMs" 2023-01-18 16:11:49 +00:00
Ibrahim Kanouche
649b4d741c Add deps tracking for SBOMs
Test: m compliance_sbom

Bug: 242220547
Change-Id: Ife93e0e1cdc43490818798922d574ae1b1d441d1
2023-01-16 16:44:22 +00:00
Treehugger Robot
b80d9d6c33 Merge "Allow vendors to add custom vbmeta partitions" 2023-01-14 00:23:02 +00:00
Treehugger Robot
1ecedfc353 Merge "Allow otatools to deduce partial partition list from input" 2023-01-13 18:00:34 +00:00
Elliott Hughes
12d8a59513 Merge "zipalign/ziptime: use ftello()/fseeko()." 2023-01-13 14:52:05 +00:00
Kelvin Zhang
b81b4e3621 Allow vendors to add custom vbmeta partitions
Currently we only have vbmeta_system and vbmeta_vendor. Add
BOARD_AVB_VBMETA_CUSTOM_PARTITIONS , which allows OEMs to define
their own vbmeta_*.img partitions. example: aosp/2381823

Bug: 263568805
Change-Id: Id671e2c3aee9ada90256381cce432927df03169b
2023-01-12 17:34:42 -08:00
Elliott Hughes
04434739a2 zipalign/ziptime: use ftello()/fseeko().
Even if we were shipping 64-bit Windows host tools (which we still
aren't), this would still be a bug. Windows is LLP64 instead of LP64,
so the long that ftell()/fseek() uses isn't big enough for a 64-bit off_t.

Test: treehugger
Change-Id: I4e24afe811ff9b7d5696887cc5ee92e54e4a3b76
2023-01-13 00:41:38 +00:00
Kelvin Zhang
30669e6c29 Ovewrite build props in all ramdisk fragments
Under some build configuration, there could be build prop stored under
path VENDOR_BOOT/RAMDISK_FRAGMENTS/recovery/RAMDISK/default.prop .
When signing, we must overwrite all build props, or device would display
incorrect build fingerprint.

Bug: 264853953
Change-Id: Id3b176b4ce4efa348ecfb3c9848f72273c5ccca8
2023-01-10 21:09:14 -08:00
Chih-hung Hsieh
380635abec Merge "Fix uninitialized value warnings." 2023-01-10 18:46:09 +00:00
Chih-hung Hsieh
951f8c3130 Merge "Make DeprecatedOrUnsafeBufferHandling not high severity" 2023-01-10 02:15:35 +00:00
Chih-Hung Hsieh
fed64e390f Fix uninitialized value warnings.
Bug: 263274255
Test: presubmit; make tidy-build-make_subset
Change-Id: I33c8b0b006732e4351a15774b50036d10ffd26b5
2023-01-10 00:52:02 +00:00
Chih-Hung Hsieh
1a1f90e49a Make DeprecatedOrUnsafeBufferHandling not high severity
This tidy check has been disabled by default globally,
in build/soong/cc/config/tidy.go.
Local projects can enable it to find/fix such issues,
but if not fixed, it should not be treated as a
high severity one and not to alarm daily builds.

Bug: 227361697
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: Ifa65a788ddc6a67ed3250d5bed737c6e3b153617
2023-01-09 14:19:11 -08:00
Kelvin Zhang
a10528c7a8 Allow otatools to deduce partial partition list from input
Different devices/builds might have different constraints on what
partitions should be updated together in a partial OTA. Add a convenient
feature to auto detect.

Test: generate ota with --partial , make sure output OTA contains only 4
partitions

Change-Id: Id79e4bc4c5fb9d1225cb2aa7c1823afccc0b61f2
2023-01-03 16:04:49 -08:00
Bob Badour
f9d23a9f51 Fix concurrency bug in bottom-up resolution walk.
Concurrency is hard, and now that we no longer track origin it doesn't
improve performance in any meaningful way.

Bug: 261787132

Test: m droid dist compliance_dumpgraph compliance_dumpresolutions \
        compliance_sbom compliance_listshare compliance_rtrace \
        compliance_checkshare xmlnotice textnotice htmlnotice \
        compliancenotice_shippedlibs compliancenotice_bom

Test: m compliance_checkshare cts && \
        out/host/linux-x86/bin/compliance_checkshare out/host/linux-x86/gen/META/lic_intermediates/out/host/linux-x86/cts/android-cts.zip.meta_lic

Test: similar command as above for gts on internal

Test: m compliance_checksare droid dist && \
        out/host/linux-x86/bin/compliance_checkshare out/target/product/sunfish/gen/META/lic_intermediates/out/target/product/sunfish/obj/PACKAGING/systemimage_intermediates/system.img.meta_lic

Change-Id: I57a75927bf879c3ce6603049d8d583211dc0ce29
2023-01-04 00:02:02 +00:00
Bob Badour
3fe369c271 An ActionSet doesn't need to walk the graph.
Resolutions map back to the root, but actions do not. An iteration
works just fine.

Simplify TargetNodeSet so that it is directly iterable.

Bug: 261787132

Test: m droid dist compliance_dumpgraph compliance_dumpresolutions \
        compliance_sbom compliance_listshare compliance_rtrace \
        compliance_checkshare xmlnotice textnotice htmlnotice \
        compliancenotice_shippedlibs compliancenotice_bom

Test: m compliance_checkshare cts && \
        out/host/linux-x86/bin/compliance_checkshare out/host/linux-x86/gen/META/lic_intermediates/out/host/linux-x86/cts/android-cts.zip.meta_lic

Change-Id: Ic5a2d809b5a9a47b5d85f61e3a4a790dbe8f5fd2
2023-01-04 00:01:49 +00:00
Bob Badour
42b02efd05 Create regression test to catch concurrency error
The bottom-up walk to resolve conditions treats "already started" as
"alreay finished".

Bug: 261787132

Test: m compliance_checkshare
Change-Id: Ibb548bd4f5464b47682633878d475dfe011df3e1
2023-01-04 00:01:36 +00:00
Bob Badour
36f4a72aa7 Create regression test to catch WalkAction error.
Bug: 261787132

Test: m compliance_checkshare

Change-Id: I9f6a39a7f1cf50f9a0134b16f68d33f171cf7f13
2023-01-04 00:01:26 +00:00
Bob Badour
a6ee6d5511 Fix tests so they can fail.
Expected values were overwriting actual values.

Origin no longer tracked.

Bug: 261787132

Test: m droid dist

Test: m compliance_checkshare

Change-Id: Ie1fbc3d596cb08c6f0935a79441d8b00a4d1eb97
2023-01-04 00:01:03 +00:00
Chih-hung Hsieh
cef5fbb2e9 Merge "Fix uninitialized value warnings." 2022-12-22 00:06:15 +00:00
Chih-Hung Hsieh
442e08ffa7 Fix uninitialized value warnings.
Bug: 263274255
Test: presubmit; make tidy-build-make_subset
Change-Id: Ib6ce849596312f56e8d640c5f78e0e70f7ca54e3
2022-12-20 16:51:53 -08:00
Kelvin Zhang
37bc304a3d Allow build_image.py to build vbmeta partition
Cuttlefish needs to rebuild vbmeta at runtime, instead of duplicating
logic in cuttlefish tooling, re-use these logic in release tools.

Test: build_image $OUT misc_info.txt /tmp/vbmeta.img /tmp/otatols
Bug: 149866755
Change-Id: I2050f9e2dd7eed9b81a49a8442ec199147041d20
2022-12-20 10:59:58 -08:00
Jooyung Han
a4dc4a3ac7 Merge "Refactor: remove --apex-info-file arg to checkvintf" 2022-12-01 00:05:16 +00:00
Bob Badour
cac8a3cf09 Change condition to match go/thirdpartylicenses
Policy uses restricted_if_statically_linked at
https://opensource.google/documentation/reference/thirdparty/licenses#RestrictedIfStaticallyLinked

Test: m droid
Change-Id: I9b18d3133ae72de1bede99f46b530298313bc2e6
2022-11-30 10:52:41 -08:00
Jooyung Han
8af44a923f Refactor: remove --apex-info-file arg to checkvintf
Now, checkvintf finds input file from dirmaps. We don't need to pass
around --apex-info-file.

dump_apex_info also generates files under /apex without an explicit
--out_file argument.

This makes "make" know less about coupling between dump_apex_info
and checkvintf.

Bug: 260310732
Test: manually run check_target_files_vintf -v target-files
Change-Id: Ibfe31978e983e13df35c32521913d5cbc89dc1d1
2022-11-28 15:01:30 +09:00
Treehugger Robot
9e0472e9c7 Merge "Fix recovery OTA error on merged targets" 2022-11-15 18:33:53 +00:00
Treehugger Robot
5cf8eef25e Merge "Fix 'expected str instance, bytes found' error when compute patches" 2022-11-14 19:14:22 +00:00
luoqiangwei1
3e3456bb72 Fix 'expected str instance, bytes found' error when compute patches
When building an incremental OTA package, if the compress_target=True
parameter is passed in when calculating a block, an exception will occur
in the subsequent join operation.

The reason is that Python 3 no longer allows str type join bytes type.

Bug: None
Test: Build Incremental OTA package

Change-Id: I6d556f9905b7ab75b70d3785334d71d5a6e5479b
Signed-off-by: luoqiangwei1 <luoqiangwei1@xiaomi.com>
2022-11-11 00:26:19 +08:00
Ibrahim Kanouche
f0c6cf501c Merge "Optimized project selection for getProjectMetadata" 2022-11-10 15:28:12 +00:00
Dennis Shen
da42e6fedb Merge "Update deapexer call to explictly use blkid" 2022-11-10 14:48:41 +00:00
Kelvin Zhang
fa40c04cba Fix recovery OTA error on merged targets
Currently, when merging target files, system side's update_engine
version info is copied into the merged target_files. This causes
recovery OTA failure, because update_engine_sideload daemon in recovery
is from vendor side. To fix it, use the older version of update_engine
config when merging target files.

Test: th
Bug: 257426678
Change-Id: I5d3304366749995c99a61ca0645a22ee047f1465
2022-11-09 14:43:17 -08:00
Cole Faust
7a48527b95 Fix crash in check_elf_file.py
`lib` is already a string and doesn't need to be decoded.

Bug: 258395719
Test: On internal master: build/soong/soong_ui.bash --make-mode --bazel-mode-staging TARGET_PRODUCT=aosp_coral TARGET_BUILD_VARIANT=userdebug droid dist device-tests platform_tests
Change-Id: I83ff35339bea9f01c61d953e251788e0894005dc
2022-11-09 12:50:35 -08:00
Dennis Shen
a8d1143beb Update deapexer call to explictly use blkid
To support erofs apex extract via deapexer, we need blkid to tell the filesystem type of underlying payload image. If it is ext4, debugfs_static will be used, if it is erofs, then we should use fsck.erofs. Thus we now need explicit blkid input.

Also, remove dependency on fsck.erofs for other deapexer calls. Only extract deapexer call needs blkid and fsck.erofs.

BUG: b/255963179, b/240288941
Change-Id: I8cea0f2def664f9cabf8b14c9a7ecc47bbddfbdd
2022-11-09 18:22:14 +00:00
Treehugger Robot
8f6474af5a Merge "Support chained init_boot partition signing" 2022-11-08 02:41:53 +00:00
Hongguang Chen
0d6b727e03 Support chained init_boot partition signing
Bug: 256048561
Bug: 256237041
Test: sign_target_files_apks -d certs --avb_init_boot_algorithm \
        SHA256_RSA4096 --avb_init_boot_key init_boot_rsa4096.pem \
	xxx.zip signed.zip (Check signed.zip/META/misc_info.txt)
Change-Id: I65fc7913089ae318d90df55a533d3e2ebd93b029
2022-11-07 15:48:23 -08:00
Dennis Shen
f58e548943 add EROFS apex support.
Signing code uses deapexer to expand payload image, deapexer currently
only supports ext4 payload image expansion using debugfs or
debugfs_static. To support EROFS, need to supply deapexer with
fsck.erofs native binary for EROFS payload expansion.

cherrypick of ag/20154950BUG: b/195515562, b/240288941
Change-Id: Ibf707989f2502bfcf112202a3a6fdb74b6a8dbbc
2022-11-07 20:40:41 +00:00
Treehugger Robot
e32ad5e482 Merge "Convert soong_to_convert.py to python 3" 2022-11-05 03:34:03 +00:00
Cole Faust
5865420eb1 Convert soong_to_convert.py to python 3
Bug: 203436762
Test: m out/target/product/emulator_x86_64/soong_to_convert.txt
Change-Id: Ibf8d80a160315ed03a11fbeef66a630b224bea0a
2022-11-04 17:36:13 -07:00
Bob Badour
8eeff2b0b8 Merge "Fix final nits from noticeindex change." 2022-11-04 16:57:01 +00:00
Dennis Shen
44b714b7ca Update deapexer extract call with more input
BUG: b/255963179, b/240288941
Change-Id: Id7f5eb20e4c6f3c3e793860389f4be48f4f3938f
2022-11-03 17:39:04 +00:00
Ibrahim Kanouche
a68ed086ab Optimized project selection for getProjectMetadata
Test: m compliance_sbom

Change-Id: I9e60b7e138a4fca74170954d8bb83862677e750f
2022-11-03 16:01:13 +00:00
Bob Badour
3afc3004f2 Fix final nits from noticeindex change.
Test: m droid dist cts alllicensemetadata
Change-Id: Ie446e5693a09f4ae1c936cdae6f1e668105541f0
2022-11-02 10:24:04 -07:00
Bob Badour
88b02afbf0 Merge "Switch noticeindex from regex to proto." 2022-11-02 15:46:38 +00:00
Maciej Żenczykowski
458f90272c sbom_test - use UTC
Test: TreeHugger, m compliance_sbom
Signed-off-by: Maciej Żenczykowski <maze@google.com>
Change-Id: Id2abec8537a1dfa32d6816da4c34c86f46313086
2022-11-01 19:49:05 -07:00
Ibrahim Kanouche
1bb27903e8 Merge "Added SBOM generator module to implement the spdx utility bill of material" 2022-11-01 23:36:22 +00:00
Ibrahim Kanouche
bedf1a8a86 Added SBOM generator module to implement the spdx
utility bill of material

Added methods to graph.go to extract the annotation types

Test: m compliance_sbom

Test: tested against real targets, see go/paste/4932131502424064

Bug: 241591242
Change-Id: I344c15d64cd034d72076f9d9a677e593e288e3a6
2022-11-01 20:42:01 +00:00
Jihoon Kang
eb35b48df7 Merge "Create custom diff tool to compare stub contents" 2022-11-01 20:32:20 +00:00
Sam Delmerico
9fac7000ec Merge "move logtags rule from //build/make to //build/bazel" 2022-11-01 13:50:10 +00:00
Treehugger Robot
5667fe0a74 Merge "Allow RO build only for readonly mount points" 2022-10-31 22:58:32 +00:00
Bob Badour
5c12c66769 Replace nil-able *sync.Waitgroup with sync.Once
Simplifies synchronization and eliminates lock for nil waitroup.

Test: m droid

Test: m out/soong/.intermediates/packages/modules/StatsD/apex/com.android.os.statsd/android_common_com.android.os.statsd_image/NOTICE.html.gz

Change-Id: I381ee79e142214e7331241071f076db2f7960ba6
2022-10-29 22:48:13 -07:00
Bob Badour
d2c28ba897 Fix top-down resolve re-walking graph too much.
Firing off multiple concurrent walks of the same sub-tree with the same
conditions. Data race meant every walk would proceed. Move the logic to
entry of walk function and compare under lock.

Bug: 255526010

Test: m droid

Test: m out/soong/.intermediates/packages/modules/StatsD/apex/com.android.os.statsd/android_common_com.android.os.statsd_image/NOTICE.html.gz

Change-Id: Ie30edbb2ac9eaa9aa55badfc518d51eaadbb6be6
2022-10-29 22:27:49 -07:00