Commit graph

7887 commits

Author SHA1 Message Date
Spandan Das
03b846ff37 Merge "Migrate check_elf_file.py to python3" 2022-10-27 16:46:18 +00:00
Spandan Das
60b8195c83 Migrate check_elf_file.py to python3
Also, create a python_binary_host so that it runs using the hermetic
python toolchain.

Test: m check-elf-files
Bug: 203436762
Change-Id: I964342a27f6b6c9dcdbbe910d4dc6c9ec00c2213
2022-10-27 02:08:23 +00:00
Treehugger Robot
1a6c0c5d86 Merge "Delete compare_fileslist.py" 2022-10-25 21:22:19 +00:00
Treehugger Robot
fe7277fdde Merge "The --why parameter in whichgit should be optional" 2022-10-25 20:08:59 +00:00
Cole Faust
29c032d260 Delete compare_fileslist.py
It was written in python 2, and is unused.

Bug: 203436762
Test: Presubmits
Change-Id: I3298ec06727ba1b72337753f52e3d09519fe0b39
2022-10-25 11:34:42 -07:00
Ibrahim Kanouche
bfa6c8e553 Merge "Added functions to projectmetadata to retrieve additional project info" 2022-10-25 17:25:11 +00:00
Cole Faust
3b9af315d2 Merge "Delete java-layers.py" 2022-10-25 17:00:52 +00:00
Cole Faust
c22b90c5d4 Merge "Delete parsedeps.py" 2022-10-25 17:00:11 +00:00
Treehugger Robot
f99edd9f67 Merge "Convert fileslist_util.py to python 3" 2022-10-25 15:58:19 +00:00
Treehugger Robot
aa3b322971 Merge "Convert findleaves.py to python 3" 2022-10-25 09:17:27 +00:00
Cole Faust
e0fab08a33 Convert fileslist_util.py to python 3
Also update it from getopt to argparse.

Bug: 203436762
Test: m $OUT/installed-files-root.txt
Change-Id: I006a36feab01cbbd76dc7d7c1c0675197848212a
2022-10-24 18:46:03 -07:00
Cole Faust
6a75c4f782 Delete java-layers.py
There are no usages of LOCAL_JAVA_LAYERS_FILE,
and java-layers.py was written in python 2.

Bug: 203436762
Test: Presubmits
Change-Id: I1be0f2547757a853e20811f28529479487037e94
2022-10-24 18:15:54 -07:00
Cole Faust
e995769d44 Delete parsedeps.py
This script is only ever manually run, and it's written
in python 2. Just delete it instead of updating to python
3 because it's unlikely anyone uses it anymore.

Bug: 203436762
Test: Presubmits
Change-Id: Ib55b3bf43a8066822308f49d1a948d27c520c5b2
2022-10-24 18:02:23 -07:00
Cole Faust
407d214b0d Convert findleaves.py to python 3
Kati special-cases findleaves.py to an internal
C++ implementation, so I don't think this python
script is even used. Update it anyways, and
test by manually running findleaves.py.

Bug: 203436762
Test: ./build/make/tools/findleaves.py --mindepth=2 --dir=build/make/target Android.mk
Change-Id: If27cc37d8f673388c0e31b9491e792b98ca59da4
2022-10-24 17:57:39 -07:00
Spandan Das
ce69e9c812 Convert check_radio_versions to python3
Additionaly,
1. Run it as a python_binary_host so that it uses the hermetic python
toolchain
2. Fix an existing bug where if `board_info_txt` was empty, $^ would
   evaluate to its own path, and it would try to parse the relevant
   configuration from the wrong file

Bug: 196300985
Test: m out/target/product/vsoc_arm64/android-info.txt
Change-Id: Ie1e51a93041591829da4351dd57f37106e15d764
2022-10-24 21:02:00 +00:00
Joe Onorato
8e908b221f The --why parameter in whichgit should be optional
Test: ./build/make/tools/whichgit --modules out/target/product/coral/system.img
Change-Id: Ibc6e208d75c8e68c8130dc9ba3fc7af51390691b
2022-10-24 08:03:20 -07:00
Ibrahim Kanouche
776ad80e68 Added functions to projectmetadata to retrieve
additional project info

Test: m droid

Bug: 254901942
Change-Id: I3de4bc528bd321c76900d277295bb10709035a9c
2022-10-22 01:22:53 +00:00
Treehugger Robot
e9d0a7f5d9 Merge "Convert fs_config_generator.py to python 3" 2022-10-21 21:17:23 +00:00
Treehugger Robot
775187f921 Merge "Fix ota build error when using boot variable files" 2022-10-21 20:43:21 +00:00
Cole Faust
216978e7c4 Convert fs_config_generator.py to python 3
Bug: 203436762
Test: ./test_fs_config_generator.py and m libc
Change-Id: I16ce5f5e273ade064acce8889a9eb3f2707910ae
2022-10-21 11:16:18 -07:00
Kelvin Zhang
eb147e051e Fix ota build error when using boot variable files
When performing a deep copy of info dict, input_file is serialized as
filename. Make ExtractFromInputFile support reading from filename of a
ZipFile.

Test: build OTA with partner provided target_files
Bug: 253549364
Change-Id: I366a076c32d638d61b83a1df9bf864807cf5257c
2022-10-21 10:53:21 -07:00
Treehugger Robot
e7bf5958b9 Merge "Fix broken fs_config_generator tests" 2022-10-21 01:33:41 +00:00
Kelvin Zhang
08b7e3aacb Merge "add a test case to validate deepcopy on PartitionBuildProps" 2022-10-20 21:46:07 +00:00
Cole Faust
f08ab3d098 Fix broken fs_config_generator tests
These tests were failing at TOT. Also add a
python_test_host target so the tests will be
run on CI in the future.

Bug: 203436762
Test: m libc, prebuilts/build-tools/linux-x86/bin/py2-cmd build/make/tools/fs_config/test_fs_config_generator.py
Change-Id: I6dda841023a2f5e76d59360d08626fc1a8842ffe
2022-10-20 13:40:07 -07:00
Treehugger Robot
de764d7d82 Merge "Update normalize_path.py to Python 3." 2022-10-20 20:27:06 +00:00
GeQi
3fa9e32563 add a test case to validate deepcopy on PartitionBuildProps
Test: atest --host releasetools_test
Bug: 253549364
Change-Id: I187554ae33041ed090ee352167ef8deb51d754bc
2022-10-20 18:39:08 +08:00
Lukacs T. Berki
740641571c Update normalize_path.py to Python 3.
Test: presubmits.
Change-Id: I2db4fedcbb658a7ca4efa3a4557a216b8682735d
2022-10-20 08:11:18 +00:00
Treehugger Robot
eb5f9e2bef Merge "check_target_files_vintf: Support flattened apexes" 2022-10-20 03:19:24 +00:00
Kelvin Zhang
5ef25191f7 Fix picle error on ota_from_target_files
When trying to make a deep copy of PartitionBuildProps, execution fails
with picle error because PartitionBuildProps stores a ZipFile object,
which stores a lock object, which doesn't support pickling.

To fix it, pickle ZipFile object as a str to its path.

Test: generate ota with boot variable files
Bug: 253549364
Change-Id: I92c5d78ce65045495203c85862922d33d886e9ea
2022-10-19 11:25:22 -07:00
Bob Badour
e790bc6f0a Merge "Refactor projectmetadata into separate package." 2022-10-19 02:04:50 +00:00
Bob Badour
dc62de4760 Refactor projectmetadata into separate package.
Replace regular expressions to extract fields from a text proto with
and actual parsed protobuf.

Refactor TestFS into its own package, and implement StatFS.

Test: m droid dist cts alllicensemetadata

Test: repo forall -c 'echo -n "$REPO_PATH  " && $ANDROID_BUILD_TOP/out/host/linux-x86/bin/compliance_checkmetadata . 2>&1' | fgrep -v PASS

Change-Id: Icd17a6a2b6a4e2b6ffded48e964b9c9d6e4d64d6
2022-10-18 16:55:47 -07:00
Yi-yo Chiang
163ece5b55 Merge "Remove deprecated BOARD_BUILD_SYSTEM_ROOT_IMAGE" 2022-10-18 06:57:24 +00:00
Jooyung Han
eb1182115b check_target_files_vintf: Support flattened apexes
Flattened apexes are extracted and copytree()'d to /apex directory to
simulate "activation" on device.

Bug: 242314000
Bug: 239055765
Test: m otapackage # target with "flattened" apexe
Change-Id: I90cc37b5f64ebad1e15f6fc5fa245d0f5955095d

Change-Id: I7e8c3fdf8e4620344d23cee0de509e574206ad9b
2022-10-18 07:46:14 +09:00
Kelvin Zhang
2e1ff6e262 Remove unnecesasry use of ZipClose()
ZipClose() was introduced to work with large zip without switching to
ZIP64. It is a hacky patch. Now we have moved to python3 and zip64
completely, no need to keep using it.

Test: th
Bug: 246504616
Change-Id: I1ff15171bb26887b819f655e32817627ecad1132
2022-10-17 10:04:45 -07:00
Jooyung Han
62283b9649 Skip flattened apexes while check_target_files_vintf
Flattened apexes should be handled differently. For now just skip
flattened apexes to fix the build breakage.

Bug: 253669548
Bug: 253700117
Bug: 239055765
Test: m otapackage for targets with "flattened apexes"
Change-Id: I102a961fc1365f42881ca9ee6546e71246792d2a
2022-10-17 10:38:41 +09:00
Wei Li
ddfc1ebf14 Merge "Change function ZipDelete to use Python module zipinfo instead of command 'zip'." 2022-10-16 05:55:00 +00:00
Wei Li
8895f9ecea Change function ZipDelete to use Python module zipinfo instead of command 'zip'.
'zip' is a host command, and it can not be provided as a dependency when some release tools are used in Bazel rules.

Test: atest --host releasetools_test
Bug: 243748589

Change-Id: Ie5f42eadbd316ccd018b19194c466a908971af82
2022-10-14 18:31:56 -07:00
Rob Seymour
0a00cd27d2 Merge "Add APEX data to checkvintf for compatibility check." 2022-10-14 18:04:46 +00:00
Cole Faust
43d2d682a8 Remove extra semicolons from event log tag generated files
To avoid an errorprone error.

Test: Presubmits
Change-Id: I5d5eefd5e54e6e862755894dd0b2f34e9e7fad89
2022-10-13 17:47:42 -07:00
Yi-Yo Chiang
06c6273073 Remove deprecated BOARD_BUILD_SYSTEM_ROOT_IMAGE
This variable is always false or empty now, so clean up remaining
references of the variable.

Remove "buildvariant=" kernel commandline as it was used by
system-as-root kernels to determine the system build variant. It is
unsupported now.

Bug: 241346584
Test: Presubmit
Change-Id: I0f97b6e0bf8b03429f65366119ff78f9c7983516
2022-10-13 16:08:53 +08:00
Treehugger Robot
1c5f813bf0 Merge "Zipalign: Don't align directory entries" 2022-10-12 23:28:34 +00:00
Fabien Sanglard
8163cfa5f4 Zipalign: Don't align directory entries
Directories are entries with uncompressed size zero and ending with
character '/' or '\' are allowed in apks since b/204425803. These
entries should not be considered for alignment since they are not
mmap by the framework.

Test: align_test.cpp
Bug: 250872480
Change-Id: I964aad118a82839f9ed230acc4c2c76f51888c67
2022-10-12 13:57:16 -07:00
Kelvin Zhang
1caead09f9 Use AVB footer to determine caremap
Care maps need to store the original image size, which excludes bytes
used by hash tree or FEC code.

We used to propagate original image size using the global OPTIONS
dictionary. This is bad coding practice, and also fragile because we
have to make sure what's stored in dictionary and what's on disk are
consistent. Instead, let's read the content of images on disk, and parse
the AVB footer. The AVB footer contains the ground truth original image
size.

Test: build OTA, make sure the care maps have valid range
Bug: 246504616
Change-Id: I9250b478ab34dda60578a6b8c23ae6d7a9385788
2022-10-10 13:12:50 -07:00
Kelvin Zhang
f294c87cfe Move GetCareMap from common.py to add_img_to_target_files.py
common.py is too big, break it down if possible

Test: th
Bug: 246504616
Change-Id: Ic1c1f11b53673419558137dffc2e233eeafff2c1
2022-10-07 10:42:00 -07:00
Rob Seymour
dc4e0f2ee0 Update check_target_files_vintf to check the last line of stdout.
More output is now produced on stdout by checkvintf.
Update logic to compare against the last line for incompatible.

Fixes the following tests:
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_kernel_incompat
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_matrix_incompat
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_sku_incompat

Bug: 251221029
Test: atest releasetools_test
Change-Id: I592acf9a3868e2a60e203b2201e7fadcbb0e7f04
2022-10-05 19:52:54 +00:00
Tim Zimmermann
a06f8330f6
releasetools: Support erofs on non-dynamic partitions
Change-Id: Ibeb228132d3c0cc1d7407c2a40498b72580c81d5
2022-10-01 13:03:36 +02:00
Kelvin Zhang
137807d69d Support container password for apex signing
Some partners need the ability to sign apexes with passwords enabled.

Test: th
Bug: 206007131
Change-Id: I6abb0775031a4c6bf8aaae679f5c7ad8f4cffe46
2022-09-30 09:19:57 -07:00
Joe Onorato
cccf2ca337 Add a tool that will tell which git projects are used for a build target
See build/make/tools/whichgit -h for the options

Test: build/make/tools/whichgit --modules com.google.android.tzdata
Test: build/make/tools/whichgit --modules framework
Test: TARGET_BUILD_APPS=com.google.android.tzdata4 build/make/tools/whichgit --products mainline_modules_arm64 --variants eng --modules $TARGET_BUILD_APPS --why "*"
Change-Id: I05d91b85575a920b3ec9f5959e2ce463a45e63a3
2022-09-28 23:46:18 -07:00
Colin Cross
4b54525b2b Fix nondeterminisim in xmlnotice
SafePathPrefixes contains "prebuilts/" which is a prefix of another
entry "prebuilts/module_sdk" which can both match the same path.
SafePathPrefixes is a map, so the iteration order is nondeterminisitic.
Move both SafePathPrefixes and SafePrebuiltPrefixes into lists that
will always have a deterministic iteration order.

Bug: 230357391
Test: build NOTICE.xml.gz multiple times
Change-Id: Ibfcd6715b70f26164e0ef4d59f73b240f47f8db7
2022-09-28 15:40:20 -07:00
Ibrahim Kanouche
8550b25d0a Merge "Added a Breadth-first top down walk function to policy_walk." 2022-09-27 05:14:52 +00:00