Commit graph

7871 commits

Author SHA1 Message Date
Cole Faust
407d214b0d Convert findleaves.py to python 3
Kati special-cases findleaves.py to an internal
C++ implementation, so I don't think this python
script is even used. Update it anyways, and
test by manually running findleaves.py.

Bug: 203436762
Test: ./build/make/tools/findleaves.py --mindepth=2 --dir=build/make/target Android.mk
Change-Id: If27cc37d8f673388c0e31b9491e792b98ca59da4
2022-10-24 17:57:39 -07:00
Treehugger Robot
e9d0a7f5d9 Merge "Convert fs_config_generator.py to python 3" 2022-10-21 21:17:23 +00:00
Treehugger Robot
775187f921 Merge "Fix ota build error when using boot variable files" 2022-10-21 20:43:21 +00:00
Cole Faust
216978e7c4 Convert fs_config_generator.py to python 3
Bug: 203436762
Test: ./test_fs_config_generator.py and m libc
Change-Id: I16ce5f5e273ade064acce8889a9eb3f2707910ae
2022-10-21 11:16:18 -07:00
Kelvin Zhang
eb147e051e Fix ota build error when using boot variable files
When performing a deep copy of info dict, input_file is serialized as
filename. Make ExtractFromInputFile support reading from filename of a
ZipFile.

Test: build OTA with partner provided target_files
Bug: 253549364
Change-Id: I366a076c32d638d61b83a1df9bf864807cf5257c
2022-10-21 10:53:21 -07:00
Treehugger Robot
e7bf5958b9 Merge "Fix broken fs_config_generator tests" 2022-10-21 01:33:41 +00:00
Kelvin Zhang
08b7e3aacb Merge "add a test case to validate deepcopy on PartitionBuildProps" 2022-10-20 21:46:07 +00:00
Cole Faust
f08ab3d098 Fix broken fs_config_generator tests
These tests were failing at TOT. Also add a
python_test_host target so the tests will be
run on CI in the future.

Bug: 203436762
Test: m libc, prebuilts/build-tools/linux-x86/bin/py2-cmd build/make/tools/fs_config/test_fs_config_generator.py
Change-Id: I6dda841023a2f5e76d59360d08626fc1a8842ffe
2022-10-20 13:40:07 -07:00
Treehugger Robot
de764d7d82 Merge "Update normalize_path.py to Python 3." 2022-10-20 20:27:06 +00:00
GeQi
3fa9e32563 add a test case to validate deepcopy on PartitionBuildProps
Test: atest --host releasetools_test
Bug: 253549364
Change-Id: I187554ae33041ed090ee352167ef8deb51d754bc
2022-10-20 18:39:08 +08:00
Lukacs T. Berki
740641571c Update normalize_path.py to Python 3.
Test: presubmits.
Change-Id: I2db4fedcbb658a7ca4efa3a4557a216b8682735d
2022-10-20 08:11:18 +00:00
Treehugger Robot
eb5f9e2bef Merge "check_target_files_vintf: Support flattened apexes" 2022-10-20 03:19:24 +00:00
Kelvin Zhang
5ef25191f7 Fix picle error on ota_from_target_files
When trying to make a deep copy of PartitionBuildProps, execution fails
with picle error because PartitionBuildProps stores a ZipFile object,
which stores a lock object, which doesn't support pickling.

To fix it, pickle ZipFile object as a str to its path.

Test: generate ota with boot variable files
Bug: 253549364
Change-Id: I92c5d78ce65045495203c85862922d33d886e9ea
2022-10-19 11:25:22 -07:00
Bob Badour
e790bc6f0a Merge "Refactor projectmetadata into separate package." 2022-10-19 02:04:50 +00:00
Bob Badour
dc62de4760 Refactor projectmetadata into separate package.
Replace regular expressions to extract fields from a text proto with
and actual parsed protobuf.

Refactor TestFS into its own package, and implement StatFS.

Test: m droid dist cts alllicensemetadata

Test: repo forall -c 'echo -n "$REPO_PATH  " && $ANDROID_BUILD_TOP/out/host/linux-x86/bin/compliance_checkmetadata . 2>&1' | fgrep -v PASS

Change-Id: Icd17a6a2b6a4e2b6ffded48e964b9c9d6e4d64d6
2022-10-18 16:55:47 -07:00
Yi-yo Chiang
163ece5b55 Merge "Remove deprecated BOARD_BUILD_SYSTEM_ROOT_IMAGE" 2022-10-18 06:57:24 +00:00
Jooyung Han
eb1182115b check_target_files_vintf: Support flattened apexes
Flattened apexes are extracted and copytree()'d to /apex directory to
simulate "activation" on device.

Bug: 242314000
Bug: 239055765
Test: m otapackage # target with "flattened" apexe
Change-Id: I90cc37b5f64ebad1e15f6fc5fa245d0f5955095d

Change-Id: I7e8c3fdf8e4620344d23cee0de509e574206ad9b
2022-10-18 07:46:14 +09:00
Kelvin Zhang
2e1ff6e262 Remove unnecesasry use of ZipClose()
ZipClose() was introduced to work with large zip without switching to
ZIP64. It is a hacky patch. Now we have moved to python3 and zip64
completely, no need to keep using it.

Test: th
Bug: 246504616
Change-Id: I1ff15171bb26887b819f655e32817627ecad1132
2022-10-17 10:04:45 -07:00
Jooyung Han
62283b9649 Skip flattened apexes while check_target_files_vintf
Flattened apexes should be handled differently. For now just skip
flattened apexes to fix the build breakage.

Bug: 253669548
Bug: 253700117
Bug: 239055765
Test: m otapackage for targets with "flattened apexes"
Change-Id: I102a961fc1365f42881ca9ee6546e71246792d2a
2022-10-17 10:38:41 +09:00
Wei Li
ddfc1ebf14 Merge "Change function ZipDelete to use Python module zipinfo instead of command 'zip'." 2022-10-16 05:55:00 +00:00
Wei Li
8895f9ecea Change function ZipDelete to use Python module zipinfo instead of command 'zip'.
'zip' is a host command, and it can not be provided as a dependency when some release tools are used in Bazel rules.

Test: atest --host releasetools_test
Bug: 243748589

Change-Id: Ie5f42eadbd316ccd018b19194c466a908971af82
2022-10-14 18:31:56 -07:00
Rob Seymour
0a00cd27d2 Merge "Add APEX data to checkvintf for compatibility check." 2022-10-14 18:04:46 +00:00
Cole Faust
43d2d682a8 Remove extra semicolons from event log tag generated files
To avoid an errorprone error.

Test: Presubmits
Change-Id: I5d5eefd5e54e6e862755894dd0b2f34e9e7fad89
2022-10-13 17:47:42 -07:00
Yi-Yo Chiang
06c6273073 Remove deprecated BOARD_BUILD_SYSTEM_ROOT_IMAGE
This variable is always false or empty now, so clean up remaining
references of the variable.

Remove "buildvariant=" kernel commandline as it was used by
system-as-root kernels to determine the system build variant. It is
unsupported now.

Bug: 241346584
Test: Presubmit
Change-Id: I0f97b6e0bf8b03429f65366119ff78f9c7983516
2022-10-13 16:08:53 +08:00
Treehugger Robot
1c5f813bf0 Merge "Zipalign: Don't align directory entries" 2022-10-12 23:28:34 +00:00
Fabien Sanglard
8163cfa5f4 Zipalign: Don't align directory entries
Directories are entries with uncompressed size zero and ending with
character '/' or '\' are allowed in apks since b/204425803. These
entries should not be considered for alignment since they are not
mmap by the framework.

Test: align_test.cpp
Bug: 250872480
Change-Id: I964aad118a82839f9ed230acc4c2c76f51888c67
2022-10-12 13:57:16 -07:00
Kelvin Zhang
1caead09f9 Use AVB footer to determine caremap
Care maps need to store the original image size, which excludes bytes
used by hash tree or FEC code.

We used to propagate original image size using the global OPTIONS
dictionary. This is bad coding practice, and also fragile because we
have to make sure what's stored in dictionary and what's on disk are
consistent. Instead, let's read the content of images on disk, and parse
the AVB footer. The AVB footer contains the ground truth original image
size.

Test: build OTA, make sure the care maps have valid range
Bug: 246504616
Change-Id: I9250b478ab34dda60578a6b8c23ae6d7a9385788
2022-10-10 13:12:50 -07:00
Kelvin Zhang
f294c87cfe Move GetCareMap from common.py to add_img_to_target_files.py
common.py is too big, break it down if possible

Test: th
Bug: 246504616
Change-Id: Ic1c1f11b53673419558137dffc2e233eeafff2c1
2022-10-07 10:42:00 -07:00
Rob Seymour
dc4e0f2ee0 Update check_target_files_vintf to check the last line of stdout.
More output is now produced on stdout by checkvintf.
Update logic to compare against the last line for incompatible.

Fixes the following tests:
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_kernel_incompat
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_matrix_incompat
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_sku_incompat

Bug: 251221029
Test: atest releasetools_test
Change-Id: I592acf9a3868e2a60e203b2201e7fadcbb0e7f04
2022-10-05 19:52:54 +00:00
Tim Zimmermann
a06f8330f6
releasetools: Support erofs on non-dynamic partitions
Change-Id: Ibeb228132d3c0cc1d7407c2a40498b72580c81d5
2022-10-01 13:03:36 +02:00
Kelvin Zhang
137807d69d Support container password for apex signing
Some partners need the ability to sign apexes with passwords enabled.

Test: th
Bug: 206007131
Change-Id: I6abb0775031a4c6bf8aaae679f5c7ad8f4cffe46
2022-09-30 09:19:57 -07:00
Joe Onorato
cccf2ca337 Add a tool that will tell which git projects are used for a build target
See build/make/tools/whichgit -h for the options

Test: build/make/tools/whichgit --modules com.google.android.tzdata
Test: build/make/tools/whichgit --modules framework
Test: TARGET_BUILD_APPS=com.google.android.tzdata4 build/make/tools/whichgit --products mainline_modules_arm64 --variants eng --modules $TARGET_BUILD_APPS --why "*"
Change-Id: I05d91b85575a920b3ec9f5959e2ce463a45e63a3
2022-09-28 23:46:18 -07:00
Colin Cross
4b54525b2b Fix nondeterminisim in xmlnotice
SafePathPrefixes contains "prebuilts/" which is a prefix of another
entry "prebuilts/module_sdk" which can both match the same path.
SafePathPrefixes is a map, so the iteration order is nondeterminisitic.
Move both SafePathPrefixes and SafePrebuiltPrefixes into lists that
will always have a deterministic iteration order.

Bug: 230357391
Test: build NOTICE.xml.gz multiple times
Change-Id: Ibfcd6715b70f26164e0ef4d59f73b240f47f8db7
2022-09-28 15:40:20 -07:00
Ibrahim Kanouche
8550b25d0a Merge "Added a Breadth-first top down walk function to policy_walk." 2022-09-27 05:14:52 +00:00
Treehugger Robot
0cf4107cbb Merge "Update ota_metadata_pb2.py" 2022-09-26 20:55:57 +00:00
Ibrahim Kanouche
e7c33de179 Added a Breadth-first top down walk function
to policy_walk.

Test: m droid dist
Change-Id: I678d2a2402c7c3ab446e8533c9f862cd8f54f889
2022-09-26 19:04:51 +00:00
Bob Badour
113c92b0a3 Make notice order more deterministic.
Bug: 230357391

Test: m droid dist
Change-Id: Ib3f771f9fd16743fca3f6c80c2ff85c9e42772a9
2022-09-23 11:27:24 -07:00
Kelvin Zhang
67c69d14f0 Update ota_metadata_pb2.py
Automatically created with aprotoc ota_metadata.pb --python_out=.

Test: th
Change-Id: I5b9522845ba781c6919b893f71e28393b8a505b7
2022-09-23 10:07:04 -07:00
Rob Seymour
3f1c9577e8 Add APEX data to checkvintf for compatibility check.
As part of extending libvintf to support VINTF data inside of APEXes:
  Create apex-info-list.xml as part of build to pass into checkvintf.
  Include the /apex data to dirmap

  Extend check_target_files_vintf.py to include APEX data:
    Unzip APEX from partions
    Extract APEX data
    Create apex-info-list.xml

Bug: 239055387
Bug: 242314000
Test: m
Test: m dist
Change-Id: I4b4e159051bacb46dc43b83e006ca0f0eb58d772
2022-09-22 23:01:44 +00:00
Bob Badour
085a2c23e7 Policy clarified: No need to share a "distribution medium"
Including code built from restricted sources in a distribution medium
does not require sharing the code for building the distribution medium.

Test: m cts dist

Test: m cts dist gts (requires cherry-pick to internal)

Change-Id: I7fcd889b11a97f8deaf4de9d72fdadd09deebe30
2022-09-22 03:23:23 +00:00
Bob Badour
10f5c48b23 Policy change: GPL+CE is permissive.
Bug: 210546823

Test: m cts compliance_checkshare compliancenotice_bom \
    compliancenotice_shippedlibs compliance_listshare \
    compliance_dumpgraph compliance_dumpresolutions htmlnotice \
    compliance_rtrace textnotice xmlnotice

Change-Id: I7da36972ffbb1494e33fd63db8e5ec851d47704c
2022-09-20 21:50:08 -07:00
jiangxu5
2d8a4cbf84 [Bugfix] Fix compilation problems
vbmeta_system does not need to be rebuilt with vendor side

Signed-off-by: jiangxu5 <jiangxu5@xiaomi.com>
Change-Id: Ieb2076055aa3342a6071ec2ed5c976e22f59acd1
2022-09-18 12:38:51 +00:00
Treehugger Robot
e98964339e Merge "Fix incorrect input for path and entry" 2022-09-15 01:54:29 +00:00
Sorin Basca
05085835ea Increase javac heap to 4096M
Bug: 246361502
Test: TH
Change-Id: Ic2d6f991ad65d13a83319af07944069d1f511783
2022-09-14 11:33:22 +01:00
Zhou Xuezan
d0d49f5b42 Fix incorrect input for path and entry
Change-Id: Idff184d8fb8b328f42fc4f96cdde2663f6581864
2022-09-14 16:26:55 +08:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
fcf791f787 Merge "Add check_elf_file to BUILD.bazel" 2022-09-08 14:00:42 +00:00
Baligh Uddin
968b961fb6 Allow passing in payload signer for ota signing.
BUG: 244632678
Test: TH
Change-Id: I12c45ac378a0d4c58b146115bc69b8e6d85a3075
2022-09-08 02:17:58 +00:00
Treehugger Robot
c0b090d805 Merge "Add a tool to merge two partial OTAs" 2022-09-07 23:13:23 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
1004dcaf82 Add check_elf_file to BUILD.bazel
In supporting cc_prebuilt_binary building with Bazel, we
need to enable Bazel to be able to introspect and build
the check_elf_files.py tool, originally built via make

Bug: 241415823
Test: b build '//build/make/tools:check_elf_file'
Change-Id: I4eeae86f683c678809e84edc2fee88f9d2436f9f
2022-09-07 22:24:33 +00:00
Treehugger Robot
bb56d32071 Merge "Buildify make/tools/BUILD" 2022-09-07 21:13:20 +00:00