ZipClose() was introduced to work with large zip without switching to
ZIP64. It is a hacky patch. Now we have moved to python3 and zip64
completely, no need to keep using it.
Test: th
Bug: 246504616
Change-Id: I1ff15171bb26887b819f655e32817627ecad1132
Flattened apexes should be handled differently. For now just skip
flattened apexes to fix the build breakage.
Bug: 253669548
Bug: 253700117
Bug: 239055765
Test: m otapackage for targets with "flattened apexes"
Change-Id: I102a961fc1365f42881ca9ee6546e71246792d2a
'zip' is a host command, and it can not be provided as a dependency when some release tools are used in Bazel rules.
Test: atest --host releasetools_test
Bug: 243748589
Change-Id: Ie5f42eadbd316ccd018b19194c466a908971af82
This variable is always false or empty now, so clean up remaining
references of the variable.
Remove "buildvariant=" kernel commandline as it was used by
system-as-root kernels to determine the system build variant. It is
unsupported now.
Bug: 241346584
Test: Presubmit
Change-Id: I0f97b6e0bf8b03429f65366119ff78f9c7983516
Directories are entries with uncompressed size zero and ending with
character '/' or '\' are allowed in apks since b/204425803. These
entries should not be considered for alignment since they are not
mmap by the framework.
Test: align_test.cpp
Bug: 250872480
Change-Id: I964aad118a82839f9ed230acc4c2c76f51888c67
Care maps need to store the original image size, which excludes bytes
used by hash tree or FEC code.
We used to propagate original image size using the global OPTIONS
dictionary. This is bad coding practice, and also fragile because we
have to make sure what's stored in dictionary and what's on disk are
consistent. Instead, let's read the content of images on disk, and parse
the AVB footer. The AVB footer contains the ground truth original image
size.
Test: build OTA, make sure the care maps have valid range
Bug: 246504616
Change-Id: I9250b478ab34dda60578a6b8c23ae6d7a9385788
More output is now produced on stdout by checkvintf.
Update logic to compare against the last line for incompatible.
Fixes the following tests:
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_kernel_incompat
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_matrix_incompat
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_sku_incompat
Bug: 251221029
Test: atest releasetools_test
Change-Id: I592acf9a3868e2a60e203b2201e7fadcbb0e7f04
SafePathPrefixes contains "prebuilts/" which is a prefix of another
entry "prebuilts/module_sdk" which can both match the same path.
SafePathPrefixes is a map, so the iteration order is nondeterminisitic.
Move both SafePathPrefixes and SafePrebuiltPrefixes into lists that
will always have a deterministic iteration order.
Bug: 230357391
Test: build NOTICE.xml.gz multiple times
Change-Id: Ibfcd6715b70f26164e0ef4d59f73b240f47f8db7
As part of extending libvintf to support VINTF data inside of APEXes:
Create apex-info-list.xml as part of build to pass into checkvintf.
Include the /apex data to dirmap
Extend check_target_files_vintf.py to include APEX data:
Unzip APEX from partions
Extract APEX data
Create apex-info-list.xml
Bug: 239055387
Bug: 242314000
Test: m
Test: m dist
Change-Id: I4b4e159051bacb46dc43b83e006ca0f0eb58d772
Including code built from restricted sources in a distribution medium
does not require sharing the code for building the distribution medium.
Test: m cts dist
Test: m cts dist gts (requires cherry-pick to internal)
Change-Id: I7fcd889b11a97f8deaf4de9d72fdadd09deebe30
vbmeta_system does not need to be rebuilt with vendor side
Signed-off-by: jiangxu5 <jiangxu5@xiaomi.com>
Change-Id: Ieb2076055aa3342a6071ec2ed5c976e22f59acd1
In supporting cc_prebuilt_binary building with Bazel, we
need to enable Bazel to be able to introspect and build
the check_elf_files.py tool, originally built via make
Bug: 241415823
Test: b build '//build/make/tools:check_elf_file'
Change-Id: I4eeae86f683c678809e84edc2fee88f9d2436f9f
Some partners have large number of products that share common
images. Generating OTAs for all these products waste a lot of CPU
cycles, because we waste time diffing the same images multiple times.
To mitigate this, add a tool for merging partial OTAs. Partners can
generate a partial OTA for common images, and re-use the generated OTA
by merging with each product specific OTA.
Bug: 227848550
Test: Run merge_otas.py on 4 OTAs, install the generated OTA
Change-Id: Ie38d522cdc9b89b1123b3d8190516c3648f99788
Move some depency on global OPTIONS to parameters. This makes it easier
for other modules to re-use these functions w/o having to modify OPTIONS
variables before/after calling.
Test: th
Bug: 227536004
Change-Id: I3bab292f65a4222f0c9502bcf04fa7a85f8124c6
Bazel doesn't allow a module and file with the same name.
Delete the symlinks as they don't appear to be used.
Bug: 198619163
Test: Presubmits
Change-Id: I50eac56aa2b87708d1762e504b3bfd86c07a6bfb
There's also a Payload class in system/update_engine/scripts. To avoid
name collision, rename the one in releasetools to PayloadGenerator.
No functional changes in this CL.
Bug: 227848550
Test: th
Change-Id: Ib7d4c7ad9839d99416d965e3a3661b8cee7b7693
This allows other modules to import these classes w/o bring in tons of
depedency. No functional changes.
Test: th
Bug: 227848550
Change-Id: I98139b45c02eddefa8a26d032e759fa11cc4c694
* changes:
Remove HashTreeInfo from verity_utils.py
Remove verity keys from info_dict
Remove replace verity key args
Remove unused args from common.py
Clean up unused code for VB in verity_utils.py
Remove verity related props from build_image.py
Test: generate an OTA with --security_patch_level flag, install it, make
sure data wipe happened.
Bug: 242812845
Change-Id: Ic33c5c468dca050a966653e9233388371d8e9b1e
We are removing VB related functionalities from release tools. This
change removes the hashtreeinfo and generator in verity_utils.py. It
also clears the function args using it.
Bug: 241044073
Test: atest under build/make
Change-Id: Icc298256be6917ffbd40cf4735f8e8092b541c9f
We are removing VB support from release tools. This change aims to
remove verity keys in info_dict generated by common.LOADINFODICT
Bug: 242672222
Test: atest under build/make
Change-Id: I9268cb113c2c0bd11ce85b8485e5a4605bdbdca0
We are removing VB support from release tools. This change aims to
remove the args related to replace verity key.
Bug: 242672222
Test: atest under build/make
Change-Id: I446a0b16e482c43542a1c0e41b24e80eb9fbc8e6
This makes it easier for other otatools to re-use these logic without
having to pull in lots of dependencies.
Test: th
Bug: 227848550
Change-Id: I81ed01c5cea4b934a074650731b6f89752221de9
We are removing support of VB from release tools. This change aims to
remove args that no longer be used.
Bug: 242672222
Test: atest under build/make
Change-Id: I347447ebf1e589f8d3c4c8254f2940911c0be7f1
We are removeing support of VB from release tools. This change aims to
remove code that are unreachable after removeing Version1ImageBuilder.
Bug: 242672222
Test: atest under build/make
Change-Id: Iaae39e4c49045f6c34351140cd43087dabd63e9c
We are removing VB related functionalities from release tools. This
change remove the verity related props in build_image.py, and also
remove VB1.0 bootimage builder, which will be unreachable without these
props.
Bug: 241044073
Test: atest under build/make
Change-Id: Ib3f1c100c2c6c9e63a64f49269619b76074a27fd
BUG:242954151
Test: make target-files-package, then check the two sha1 are same:
1. sha1 of IMAGES/boot.img
2. expected sha1 of boot in VENDOR/bin/install-recovery.sh
Test: make target-files-package; unzip target_files.zip IMAGES/\*`;
`zip -d arget_files.zip IMAGES/\*`
`add_img_to_target_files -a target_files.zip`.
Verify that the rebuilt images are identical to the deleted ones.
Change-Id: I5906ac1696b6403674fd8330b1569aed53510945
Caller expects output_file to contain the final OTA, so we must copy
data into that location. Modifying loca variables isn't enough.
Test: make ota with --no_signing
Change-Id: Id4aaefbc53b4250b7308701493d151f9408f47f1
Currently, if we want to override all apex/apk key, we must specify all apk/apex names exhaustively. This causes CLI arguments to be very long and sometimes exceeds OS's limit.
Test: run signing test locally on build 8862875 on git_master
Bug: 239991438
Change-Id: Idbd3b7eef9b2a27236f4ca7a27449fcdf0d2dd3f
The intention was to make extra_apex_payload_key take a comma separted
list of keys. Iterating over a string will only iterate over each
characters individually. Must split by "," to iterate over keys.
Test: Run signing test locally using tradefed.sh
Bug: 239991438
Change-Id: I51bfed53a43b1b0fc1e33e0dc3d418a727e6ff15
When running tests, we might need to add many
extra_payload_keys. Currently we have to add --extra_apex_payload_key
for each key pair we pass, resulting in extremely long argument list.
Test: th
Bug: 239991438
Change-Id: I3e5f9d76c7f45822fb986b603dc089407c35b76b
Let the fsverity_manifest_generator logic be reused and just keep the
bits to generate the BuildManifest.apk. Since this can all be acheived
with a series of shell commands, remove the python script and just do it
directly in the Makefile.
Bug: 237384936
Test: TH
Change-Id: I168dc1cea0be72b5098f99e4183e080a687fb03e
`info->symlink_target` is a `std::string`. This `strdup` therefore leaks
memory. Just use `std::string::operator=(const char *)` instead.
Bug: 206470603
Test: TreeHugger
Change-Id: I71bcaec583c7d429b4d09522cc5eeb2ced009007
Currently the only way to do this is to enable deduplication.
Bug: N/A
Test: BOARD_EROFS_COMPRESSOR := none
Change-Id: I7f8b99fbcefe7bbc15dfab7062b321aa3e2cf1ca
SymlinkTree is one of the action types Blaze returns in response to
`aquery` (action query). build-runfiles is a tool copied from Bazel
to implement this action (see src/main/tools/build-runfiles.cc in
the Bazel Git repo). It creates a symlink farm (i.e., a directory
hiearchy consisting of symlinks) from a given manifest. Mixed builds
need this tool the mixed builds (in particular, every Python applicaion
has a runtime symlink farm).
Bug: 232085015
Test: m USE_BAZEL_ANALYSIS=1 com.android.adbd
Change-Id: I9cfcb33cb7d0f63bd36ffd2b4101f53cfc6a42fc
Uniform the split() function
str.split() will return a list split all spaces in str,
while str.split(' ') will return a list might contain ''
which might have potential issues.
Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
Change-Id: I0961659b140f800bdbe285f63bb4f02b8459ff8b
Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
Test: mma and run sign_apex
This reverts commit 8fe9e3a2c7.
Reason for revert: Updateable SEPolicy is punted.
Change-Id: I28ae2afbb1083e215f4e0a27a9164133cd73dd76
Merged-In: I28ae2afbb1083e215f4e0a27a9164133cd73dd76
* Dump current date/time to the output .html file
to distinguish files from different build dates.
* Guess android_root from the path of warn_common.py,
but only if it contains build/make and build/soong.
* Do not depend on the TOP variable,
which is no longer emitted by build/soong make.
* Do not depend on the USE_RBE variables,
which is not emitted in local makes.
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Bug: 198657613
Change-Id: Ib75c60216f9e06f9a768e63b536495fb9658843f
Make it possible to provide a prebuilt hints file to specify different compression strategies on a file basis for all partitions using:
BOARD_EROFS_COMPRESS_HINTS := <path>/erofs-hints.txt
or separately for every partition using:
BOARD_SYSTEMIMAGE_EROFS_COMPRESS_HINTS := <path>/system-partition-erofs-hints.txt
BOARD_VENDORIMAGE_EROFS_COMPRESS_HINTS := <path>/vendor-partition-erofs-hints.txt
Hints format: Each line is defined by tokens separated by spaces in the following form:
<pcluster-in-bytes> <match-pattern>
pcluster-in-bytes can be specified as 0 which mean file will not be compressed
Bug: 231934752
Test: used dump.erofs -S system.img to verify the system image isn't compressed after specifying BOARD_SYSTEMIMAGE_EROFS_COMPRESS_HINTS := no-compress.txt, where no-compress.txt contains 0 .*
Test: used dump.erofs --nid=<inode-for-all-apk-files> vendor.img to verify apk files are not compressed (the rest of the partition content is compressed)
Test: used dump.erofs -S <all-images> to verify all images are not compressed after specifying BOARD_EROFS_COMPRESS_HINTS := <path>/no-compress.txt
Change-Id: I64054e26af5ea5781c45d2183b7796a9ad2b7927
* Use android_root or __file__ path to find
the source tree root and its sub directories.
* Use the sub directory list to clean up RBE-returned
warning lines, removing all mixed prefix before
the top sub directories.
* Change some re.match rules and order to save
unnecessary comparisons.
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Bug: 198657613
Change-Id: I78cc17d04b8ab9e12935ef04797f3272298d5267
* The index was a directory or file path.
This change adds additional entries that have
warning type prefix in the index.
* In the "Directories/Files with at least 1% warnings"
section, the list will include number of warnings for
(1) "all warnings in a directory/file"
(2) "warnings of a type in a directory/file"
(3) "warnings of a type in all directories/files"
Examples:
4273 (8.7%) frameworks/av/...
4130 (8.4%) packages/...
2427 (5.0%) [google-explicit-constructor] */...
1628 (3.3%) [cert-err34-c] */...
1099 (2.2%) [google-runtime-int] bionic/tests/math_data/...
608 (1.2%) [cert-err34-c] external/...
Bug: 231245501
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I13bb54c846ad514334f78c5a71e994a131a92963
signapk currently accepts a Provider class that can be instantiated
and inserted before the signing. This commit adds support to specify
a -providerArg parameter that can be used to configure the Provider.
Prior to JDK 9 a Provider would accept a providerArg in a constructor
accepting a String; in JDK 9+ a Provider should first be instantiated
with the zero-arg constructor, then the configure method should be
called with the providerArg.
Bug: 142334653
Bug: 190974913
Fixes: 232134730
Test: Manually verified new Provider can be inserted with pre- and
post-JDK 9 behavior.
Change-Id: I96f027640c59d3357e8dcf656626d1601bfef861
Downgrade OTA was broken because generic_ramdisk was removed from boot
partition in android T. Neither does O6 have init_boot. O6 also doesn't
include generic ramdisk in vendor boot, so no way for otatools to locate
contents of generic ramdisk. As a hack, omit boot partition timestamp if
we can't find ramdisk.
Test: generate T->S downgrade OTA for O6
Bug: 231656318
Merged-In: I9f5359462332aadadc324348873a3a4b2b126068
Change-Id: I9f5359462332aadadc324348873a3a4b2b126068
Downgrade OTA was broken because generic_ramdisk was removed from boot
partition in android T. Neither does O6 have init_boot. O6 also doesn't
include generic ramdisk in vendor boot, so no way for otatools to locate
contents of generic ramdisk. As a hack, omit boot partition timestamp if
we can't find ramdisk.
Test: generate T->S downgrade OTA for O6
Bug: 231656318
Change-Id: I9f5359462332aadadc324348873a3a4b2b126068
We might add new values to dynamic_partition_info.txt, so some kind of
generic fallback mechanism is needed. If keys are different, we need to
decided on a case-by-case basis which side takes precedence. For
example, Virtual AB requires vendor support, so vendor side takes
precedence. VABC on T+ devices are implemented entirely in system, so
system/framework side takes precedence.
Bug: 230876542
Test: th
Change-Id: I67747368547d3ef3e29ad64f8f818ef4c5896246
(cherry picked from commit 6a683ce02b)
Merged-In: I67747368547d3ef3e29ad64f8f818ef4c5896246
Only enable zucchini if configuration file zucchini_config.txt exits
in target files and version in source and target is the same.
Without this patch zucchini would be enabled if configuration file
was missing from both target files.
Bug: 231204699
Test: Run ota package generation with zucchini_config.txt missing from
both source and target zips. Confirm that zucchini is disabled.
Change-Id: Ia34998fd911d5860b38fe49fa5a88056a22d661e
Some OTA files may have additional OTA props; when this is being computed within
ota_utils.py -> ComputeRuntimeBuildInfos, the ramdisk type needs to be passed into
PartitionBuildProps.FromInputFile as not all ramdisk may still use gzip. This check
is already defined in common.py, so this change also exposes the function GetRamdiskFormat.
Bug: 231075507
Signed-off-by: TJ Rhoades <tjr@microsoft.com>
Change-Id: If1b93b887990f0d90df2c6003122821c1e66a1ac
We might add new values to dynamic_partition_info.txt, so some kind of
generic fallback mechanism is needed. If keys are different, we need to
decided on a case-by-case basis which side takes precedence. For
example, Virtual AB requires vendor support, so vendor side takes
precedence. VABC on T+ devices are implemented entirely in system, so
system/framework side takes precedence.
Bug: 230876542
Test: th
Change-Id: I67747368547d3ef3e29ad64f8f818ef4c5896246
Rather than hardcoding a list of allowed vendor partitions, we accept
anything in the vendor target files that is not a framework partition.
Also extend support for inferred misc_info keys when the device uses
SYSTEM/product or SYSTEM/system_ext.
Test: test --host releasetools_test
Test: Use to merge a device with a custom IMAGES/*.img in the vendor
build, and SYSTEM/system_ext in the system build.
Bug: 225902565
Change-Id: I638c0f9c019357150516ea6c208ecd60c03c450f
Kati's $(wildcard) results are guaranteed to be sorted,
but go's filepath.Glob() is not.
Remove the file existence functions because they can
just be replaced with wildcards.
Bug: 226974242
Test: go test
Change-Id: I02fb6292b932bc28cd856ec3c7cb9ed9e96ca630
Original change: https://android-review.googlesource.com/c/platform/build/+/2059633
Change-Id: If57c75832a7eca9ab5b2399b946b16eb80abf23a
Ignore-AOSP-First: this is an automerge
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
Original change: https://android-review.googlesource.com/c/platform/build/+/2071827
Change-Id: Ic9750494da0dcd8a0956495da0818f796350ac5c
Ignore-AOSP-First: this is an automerge
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
Even though build_image uses logger but there was no way to configure it
and even -v (verbose) didn't work.
common.ParseOptions() parses common arguments (e.g. -v -h --logfile).
Bug: n/a
Test: m
Test: build_image -v ... should print logs
Change-Id: I45a61ada9c6a5a616f05d3214e185d08b0f1c152
add_img_to_target_files.py will re-make image and build a new
signed vbmeta.img with new images in target-file.
We need to add vendor_kernel_boot into AVB signing list and make it
support that image.
Test: make dist -j110 and check vbmeta.img
Bug: 214409109
Signed-off-by: Lucas Wei <lucaswei@google.com>
Change-Id: Id07433f3dc33f95e2edd49de890f1e098cb9ef31
GKI targets, e.g., `gki_arm64` or `gki_x86_64` have only
boot-*.img files without a ramdisk. It has no other images,
e.g., init_boot.img, system.img, etc.
The current build system assumes that the ramdisk is either
in a boot.img or in a init_boot.img, which is not true for
those GKI targets. Adding a new flag to support building
boot-*.img without a ramdisk while not building an init_boot.img.
Bug: 220834917
Test: build and `unpack_bootimg`
Change-Id: I789343c3e3d9ff0c36c0e19680a9792bd31a1c9f
The long-form variables (PRODUCTS.<makefile>.<variable>)
are used to get information about multiple products.
However, they've never really worked correctly, and so
importing multiple products is deprecated behavior.
Remove as many usages of the long-form variables and
multi-product imports as possible.
Bug: 228518445
Test: Manually
Change-Id: I0b67f16360ff8bdcdb39638de739440472bccf76
1. Filtered out the Android.mk file if the Make modules on them are
not in the PRODUCT_PACKAGES or the correspoding deps;
2. Merged the Android.mk file list by partitions for all lunch targets.
Bug: 225256154
Test: lunch aosp_coral-eng; m
out/target/product/coral/mk2bp_remaining.html
Test: python3 build/make/tools/mk2bp_partition.py
Change-Id: I40919e81ab3c6bd516379f0acd72932e8db50088
This rule originally used an android_library because it needed access to
SDK methods, but we can instead just provide Java sources and expect the
dependee to link the SDK methods.
Test: enabled QuickSearchBox and build
Change-Id: I7f1cdbea49e100ecc0944f8caa39683fb73016fb
As a first step to making .rbc files compatible with bazel,
remove regex support since bazel doesn't have it.
Fixes: 227384703
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I8b946c20cc42897a47a5516a167732f4e16b6158
Add vendor_kernel_boot image for vendors whose bootloader support
extra first stage booting kernel modules ramdisks. This benefit
kernel repo to build kernel-artifacts only image without Andorid
artifacts dependency.
Bug: 214409109
Signed-off-by: Lucas Wei <lucaswei@google.com>
Change-Id: If07218b86a7751b3d452a172610af960f5f9ec74
Right now they're named ro.bootimage.* which is really confusing.
Change it so that if generic ramdisk is in boot.img, then name the
properties as ro.bootimage.*, otherwise if generic ramdisk is in
init_boot.img, then name the prop as ro.init_boot.*.
Bug: 211823573
Test: m dist
Change-Id: Icff3c7be04b9a3c6490fbf49ba4f05565b029890
gki_arm64 is a special target to build multiple GKI boot-*.img
files. It has neither system.img nor vbmeta.img. Skips checking
system.img and vbmeta.img in this case.
Bug: 223078405
Test: validate_target_files signed-gki_arm64-target_files-8277156.zip
Change-Id: I9d3c102c11787cf70c1e1fc8e6a552f8fd19ff72
This will be used in the following change to split
merge_target_files into a collection of smaller
more-focused scripts.
Bug: 221858722
Test: m otatools; Use to create merged builds
Change-Id: Ie01dac81c5f9f28f1e0fe037259eabd2478e60b6
