Commit graph

8063 commits

Author SHA1 Message Date
Treehugger Robot
fad3925959 Merge "Update generateSPDXNamespace to generate a unique spdx doc namespace" 2023-04-21 16:50:00 +00:00
Treehugger Robot
269248add6 Merge "Cleanup the list of Soong prebuilt module types." 2023-04-21 01:40:01 +00:00
Wei Li
6f407ba0a8 Cleanup the list of Soong prebuilt module types.
Bug: 272356622
Test: CIs
Change-Id: I4031f0a73f5af1bbf1eb374bcd7d3a19b9609a46
2023-04-20 14:13:58 -07:00
Ibrahim Kanouche
397a9cf430 Update generateSPDXNamespace to generate a unique spdx doc namespace
Test: m compliance_sbom
Change-Id: Icc1e5aec0e1758ee32c43cfa79b5b49b3d1d9d30
2023-04-20 17:43:12 +00:00
Treehugger Robot
ece71703f8 Merge "Allow ota_from_target_file to work entirely on directories" 2023-04-20 15:17:50 +00:00
Bob Badour
ffba21bbae [LSC] Add LOCAL_LICENSE_KINDS to build/make
Added SPDX-license-identifier-Apache-2.0 to:
  tools/sbom/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Change-Id: I1f8e5fd8e787a4c53bf4caf80a4000d46d0291e4
2023-04-19 09:48:01 -07:00
Kelvin Zhang
9dbe2ce40f Allow ota_from_target_file to work entirely on directories
When building android, build system will first zip target_file directory
into a .zip file, and then invoke ota_from_target_files. We
ota_from_target_files can work directly on top of directories, we can
parallelize OTA generation and target_file zipping, making builds
faster.

Bug: 227848550
Bug: 277028723
Test: th
Change-Id: Id72bd7cca708af0e1bae2db754f39b27777cd601
2023-04-18 17:47:29 -07:00
Wei Li
49af939a23 Copy SBOM files of unbundled APKs to dist directory.
Bug: 266726655
Test: CIs
Test: tapas Browser2 Calendar arm userdebug && m dist
Change-Id: I82328dc4916451a99ad0bf6d7bb945c9b98d0bea
2023-04-18 15:57:34 -07:00
Treehugger Robot
67ca3c05c7 Merge "Fix duplicate super_empty.img" 2023-04-17 21:19:13 +00:00
Treehugger Robot
b7a45ea13d Merge "Try with search_path for some avb path args" 2023-04-17 20:43:12 +00:00
Kelvin Zhang
aecdc18222 Fix duplicate super_empty.img
When calling add_img_to_target_files twice, super_empty.img will be
written twice, and resulting zip would contain duplicate entries.

Test: th
Bug: 278522974
Change-Id: I2419d0c4f8b04f876a8cc69a1517f59cab94d75d
2023-04-17 09:52:39 -07:00
Alex Buynytskyy
2ba40b5be6 Merge "Finalization script fixes." 2023-04-17 15:26:16 +00:00
Alex Buynytskyy
f7a5e2fa4b Finalization script fixes.
- actually do the first step instead of just logging a message,
- make sure extension SDK bump gets picked up by the topic,
- fix the resources_sdk_int in first step, and unfix in the second.

Bug: 278246368
Fixes: 278246368
Test: local run
Change-Id: I0d0f4e2ec7f0b584b9fc14a545cfec73a9fcbdea
2023-04-16 21:15:18 -07:00
Treehugger Robot
3fb8d2bad8 Merge "Fix the following issues mentioned in Pixel SBOM review." 2023-04-15 20:34:52 +00:00
Wei Li
5290825452 Fix the following issues mentioned in Pixel SBOM review.
1) PackageSupplier should be NOASSERTION if there is no homepage
   information in METADATA file of source packages
2) PackageDownloadLocation of upstream packages should be NOASSERTION if
   there is no code repository URL in METADATA file of source packages

Test: CIs
Test: atest --host sbom_writers_test
Change-Id: I8a0298b7bacc2f96555f9d7dde0d21ada8c6b564
2023-04-15 06:05:32 +00:00
David Anderson
23c2df71bc Merge "ota_from_target_files: Disable XOR on <T vendor." 2023-04-15 04:52:46 +00:00
Treehugger Robot
7b640806da Merge "Revert "Remove all ZIP64LIMIT hack"" 2023-04-15 01:48:58 +00:00
Dennis Song
5da012adb0 Merge "Include META/* in the vendor item list when rebuild-sepolicy" 2023-04-15 01:15:44 +00:00
David Anderson
1c59617820 ota_from_target_files: Disable XOR on <T vendor.
Bug: N/A
Test: ota_from_target_files
Change-Id: Ia9093a3c3f79d88f98f0777e8a0f8c72741055b8
2023-04-14 16:01:55 -07:00
Kelvin Zhang
f92f7f046a Revert "Remove all ZIP64LIMIT hack"
This reverts commit 37a4290909.

Reason for revert: b/278156419

Change-Id: I67ea667619a9623be849d911993010ef0f0bfd88
2023-04-14 21:32:54 +00:00
Treehugger Robot
ebf41e9a91 Merge "Create separate python libraries for the following logic and refactor SBOM generation script accordingly." 2023-04-14 20:43:58 +00:00
Treehugger Robot
6fb4c8dcee Merge "Order entries in apex_info.pb" 2023-04-14 20:33:18 +00:00
Håkan Kvist
01e381954b Order entries in apex_info.pb
Adding the entries in apex_info.pb in sorted order makes it easier to
compare ota packages generated between the same source and target on
different computers/environments, ideally the generated ota zips should
be identical.

os.listdir() was used to find the apex files which should be
included in apex_info.pb. listdir() does not guarantee any order,
solution is to sort the result to ensure consistent order.

Bug: 278095305
Test: Manual. Confirm that apex_info.pb is generated sorted
    protoc --decode_raw < apex_info.pb
Change-Id: I2851d207f828a6ac31e63e20c9025965ac4b0eba
2023-04-14 10:36:04 +02:00
Dennis Song
09f5c5dc0b Include META/* in the vendor item list when rebuild-sepolicy
To ensure that `AddImagesToTargetFiles` can still be used with
vendor item list that do not specify all of the required META/
files, those files should be included by default.

Test: atest --host releasetools_test
Bug: 278133178
Change-Id: I5a04bfc43d405d6838028f2535ced2124dbd43aa
2023-04-14 12:13:42 +08:00
zhangyongpeng
707569711b Try with search_path for some avb path args
If we can't find relative signing_helper path, try with the OPTIONS.search_path dir prefix, and integrate the avb_*_key_path for the same logic.

Test: Build && releasetools_test

Signed-off-by: zhangyongpeng <zhangyongpeng@xiaomi.com>
Change-Id: Ifb1096ddea90693668f3344eb242bf9725113d11
2023-04-14 12:12:47 +08:00
Wei Li
dec97b1462 Create separate python libraries for the following logic and refactor SBOM generation script accordingly.
1) writer classes of generating SBOM in different SPDX formats
2) data classes to model the SBOM structure in SPDX

Bug: 272358880
Test: CIs
Test: build/soong/tests/sbom_test.sh
Test: atest --host sbom_writers_test

Change-Id: I1175cf0d99864bc4304559a59484ef0ba401cd64
2023-04-13 13:34:57 -07:00
Ibrahim Kanouche
8c745a436b Added validation to SBOM generator
Test: m compliance_sbom
Change-Id: I1ff2dfbc48361cfb785c64306112bc687ca51057
2023-04-13 16:10:12 +00:00
Treehugger Robot
64bfc41359 Merge "Fix s to t OTA" 2023-04-13 01:01:57 +00:00
Kelvin Zhang
010bbea401 Merge "Fix releasetools timeouts" 2023-04-12 15:37:59 +00:00
Kelvin Zhang
629bc8dc1c Fix s to t OTA
Android S update_engine has a smaller minor version, we should use
update_engine version number from source build when generating an
incremental OTA.

Test: th
Bug: 277837157
Change-Id: Ic1ccdccd26c19bf9fb13d146a9151e699e481201
2023-04-11 21:30:42 -07:00
Treehugger Robot
84031f7b49 Merge "Invoke delta_generator directly" 2023-04-12 01:00:43 +00:00
Treehugger Robot
b2be62551c Merge "Script updates." 2023-04-11 21:52:43 +00:00
Kelvin Zhang
ea84d42199 Fix releasetools timeouts
Release tools test uses sleep(5) to make sure mtime on files change in a
visible way. Use hardcoded mtime instead of sleeps.

Improvement: 145.6s -> 65s

Test: atest releasetools_test
Bug: 277782284
Fixes: 277782284

Change-Id: I4d7b04707f3b6c5843cde883f43e95a1e7a69879
2023-04-11 13:53:05 -07:00
Kelvin Zhang
fcd731e3d6 Invoke delta_generator directly
Currently, ota_from_target_files(a python script) calls
brillo_update_payload(a bash script), which then calls delta_generator(a
C++ binary) to do the actual diffing. Having brillo_update_payload in
the middle does not offer any additional flexibility, but it makes
maintaince more difficult. Bash code is less readable and harder to
debug. Further more, everytime we want to add a new flag to
delta_generator, we have to add the flag in three places:
ota_from_target_files, brillo_update_payload, delta_generator.

Historically, brillo_update_payload was there because we inherited from
ChromeOS side. This bash scripts extracts target_files.zip and invoke
delta_generator to generate the actual OTA. Any customizations we want
on OTA must be implemented by modifying the input target_files.zip ,
Manipuating big zip files is slow and inefficient.

To make thing simpler/faster, remove the intermdiary call to
brill_update_payload. ota_from_target_files will now extract target
files and call delta_generator directly.

Test: th
Bug: 227848550
Change-Id: I44b296e43bf7921bcf48ef6a1e4021a12669739c
2023-04-11 13:53:05 -07:00
Alex Buynytskyy
d98c43c8dd Script updates.
- add a mode to disable finalization steps if they were already merged,
- add the extension version into commit message.

Bug: 243966946
Test: local run
Change-Id: I51505e5d5715b56f3b385d714e1aabaae06f58ed
2023-04-11 16:51:03 +00:00
Dennis Song
a0ffed1fa1 Merge "Support merging target files from directory" 2023-04-11 05:40:05 +00:00
Treehugger Robot
52fdcf5457 Merge "Allow setting future api level before REL" 2023-04-11 00:09:23 +00:00
Dennis Song
5bfa43e5eb Support merging target files from directory
Expand `merge_target_files.py` API capabilities so that
`--framework-target-files` and `--vendor-target-files`
can be either zip archives or directories.

Test: Create a merged package by vendor target files folder
Test: atest --host releasetools_test
Bug: 276068400
Change-Id: I200be2a458ae59a61e05bfd7c78ab66093db32eb
2023-04-10 17:07:03 +08:00
Justin Yun
870ea2e188 Allow setting future api level before REL
At the dev stage, devices may set ro.board.(first_)api_level to the
future API level

Bug: 276927022
Test: test_post_process_props.py
Change-Id: I85c29af74ed8daa780278f64b023480bb6659781
2023-04-10 15:37:19 +09:00
Wei Li
3bcd0bca6f Check if an installed file exists and report error if not.
So far there is only one case happened in b/277039235 that an installed file in that product is moved to another directory in LOCAL_POST_INSTALL_CMD. The team agreed to convert to Android.bp, but better to add a check and report the error.

Resend this to check symlinks using os.path.islink(), which doesn't follow the link. os.path.isfile() follows symlinks and returns false since the symlinks are for devices and could not be resolved on host file systems.

Bug: 277039235
Test: m sbom
Test: build/soong/tests/sbom_test.sh
Change-Id: Ia9f1cd24dc974a3e41487bc17c9c76a26d419395
2023-04-07 16:36:30 -07:00
Wei Li
0bb86d0515 Merge "Revert "Check if an installed file exists and report error if not."" 2023-04-07 21:40:09 +00:00
Wei Li
fd559f89b5 Revert "Check if an installed file exists and report error if not."
This reverts commit f049389e2e.

Reason for revert: the check includes more files that should not be excluded.

Change-Id: I7f7d260218826cc8fd033400b2cfaf40e4c280d0
2023-04-07 21:35:13 +00:00
Treehugger Robot
d8e23f9519 Merge "Add comments explaining how secondary OTA works" 2023-04-07 20:57:36 +00:00
Kelvin Zhang
bd111ccae2 Add comments explaining how secondary OTA works
Test: th
Change-Id: I72f890e3dcb6f8369dea8071da040b4c27877a2b
2023-04-07 09:47:04 -07:00
Wei Li
f049389e2e Check if an installed file exists and report error if not.
So far there is only one case happened in b/277039235 that an installed file in that product is moved to another directory in LOCAL_POST_INSTALL_CMD. The team agreed to convert to Android.bp, but better to add a check and report the error.

Bug: 277039235
Test: m sbom
Test: build/soong/tests/sbom_test.sh
Change-Id: I9a2302ced6dd605e60e54c49a8778426444dcc4f
2023-04-07 07:33:11 +00:00
Alex Buynytskyy
0fce469f2d Merge "Minor renaming and documentation." 2023-04-06 23:38:33 +00:00
Alex Buynytskyy
307d9bcfdf Minor renaming and documentation.
Bug: 276752264
Test: local run
Change-Id: I75814313a81ca041fa1461e48733bd40a020b777
2023-04-06 15:28:39 -07:00
Alex Buynytskyy
f5b6dd80ef Merge "Copy ABI dumps from current/64 to FINAL_PLATFORM_SDK_VERSION/64" 2023-04-06 19:52:00 +00:00
Jihoon Kang
0c76cab34e Merge "Specify list_files to be built with embedded launcher" 2023-04-06 16:54:02 +00:00
Hsin-Yi Chen
ebce0d9501 Copy ABI dumps from current/64 to FINAL_PLATFORM_SDK_VERSION/64
Because prebuilts/abi-dumps/ndk/current contains additional ABI dumps
that are not included in the finalization process, the script copies
only the subdirectory to the finalized ABI dump directory.

Test: ./finalize-sdk-rel.sh
Bug: 276503801
Change-Id: I76287cf110323baf40035a957564a1e8f39e8015
2023-04-06 14:01:07 +00:00