Commit graph

7111 commits

Author SHA1 Message Date
Joe Onorato
841c3e3ea6 Add skeleton for product-config tool.
There's also a bunch of boilerplate error handling and option parsing
stuff that I wish someone would put into a library that's available to
the android tree.

Test: atest product-config-test
Change-Id: Ieebcc7bd47a8644d1374fb02c146e9038859f4a2
2021-01-21 12:38:04 -08:00
Treehugger Robot
d041e4babf Merge "Make ro.product.cpu.abilist.* to be fetched dynamically" 2021-01-14 18:34:22 +00:00
Yifan Hong
7a27574e5f Merge "GetBootImageBuildProp returns None if file does not exist" 2021-01-13 19:40:13 +00:00
Nikita Ioffe
015b689271 Merge "Refactor apex signing logic in preparation for compressed apexes" 2021-01-13 14:54:49 +00:00
Tianjie Xu
2de3459680 Merge "Fix errors from validate_target_files" 2021-01-13 00:02:36 +00:00
Nikita Ioffe
6068e8dd52 Refactor apex signing logic in preparation for compressed apexes
Logic for signing compressed apex will be added in the follow-up cl.

Test: python -m unittest test_sign_apex
Bug: 172912232
Change-Id: I8dc1d334c17e11f9eed8fe0a575b6dfcf337ab5d
2021-01-12 22:29:58 +00:00
Tianjie
2e0b835d28 Fix errors from validate_target_files
We should skip lines that don't contain '=', e.g. empty lines.
Also, pop a warning instead of an error if a prop is defined
multiple times with the same value

Bug: 177240467
Test: unittest, run validate_target_files
Change-Id: Ifc9eadb91e5dda7170a19d875016e5a47e8fc592
2021-01-12 14:04:58 -08:00
Yifan Hong
14a8712c4d Add missing dependencies to releasetools_common and binary defaults.
When GetBootImageBuildProp is moved to common, its dependencies
(toybox, lz4 and unpack_bootimg) aren't moved accordingly.
Copy the dependencies over.

Test: pass
Change-Id: Iaa8f41ae0109e8eb1c058ecd7dd854bb7de9391a
2021-01-12 14:04:19 -08:00
Yifan Hong
7dc5117ffd GetBootImageBuildProp returns None if file does not exist
If system/etc/ramdisk/build.prop does not exist in the ramdisk,
GetBootImageBuildProp should return None instead of a path
to a non-existant file.

Test: pass
Bug: 177171706
Change-Id: I34e447bc315d203e136e8c0d59d00961f81301e3
2021-01-12 11:27:55 -08:00
Nikita Ioffe
c3fdfed11b Fix test_sign_apex
Test: python -m unittest test_sign_apex
Change-Id: I2a27a394b3de32fa29e487bea0b7721502a8bd98
2021-01-11 23:50:31 +00:00
Yifan Hong
cb09afc04a Merge changes from topic "common.GetBootImageTimestamp"
* changes:
  Load boot build props to info_dict properly.
  Add GetBootImageBuildProp.
  Split PARTITIONS_WITH_BUILD_PROP from *_WITH_CARE_MAP
  Move GetBootImageTimestamp to common.
2021-01-08 22:18:51 +00:00
Colin Cross
383e482c22 zipalign: log more I/O errors
b/177069043 is a flake that happens in zipalign during I/O, but
the error doesn't contain enough information to determine what the
problem is.  Add more logging on I/O errors.

Test: atest --host zipalign_tests
Bug: 177069043
Change-Id: I89573e5b3867abe05d855c837bb2aa5b5c630aa7
2021-01-08 11:11:40 -08:00
Yifan Hong
10482a2a56 Load boot build props to info_dict properly.
Add boot to PARTITION_WITH_BUILD_PROP. Let PartitionBuildProps handle
boot partitions by extracting build.prop file.

Test: generate OTA and call
      `payload_info.py payload.bin`, check that boot image timestamp is
      set
Bug: 162623577

Change-Id: I71955e4ee494ffbb5d111c3d02a129c0f879b6d1
2021-01-07 17:20:22 -08:00
Yifan Hong
85ac5012aa Add GetBootImageBuildProp.
Add a function that extracts build.prop from boot image.

Test: TH
Bug: 162623577
Change-Id: I673ccd8e7c4846ca4c41a86349a388b2e800724e
2021-01-07 17:20:22 -08:00
Yifan Hong
5057b95572 Split PARTITIONS_WITH_BUILD_PROP from *_WITH_CARE_MAP
Right now they are the same content. In the future, boot will be added
to PARTITIONS_WITH_BUILD_PROP, but it is not added to
PARTITIONS_WITH_CARE_MAP. Boot partition has a cpio filesystem in the
ramdisk, so it contains a build.prop file, but it doesn't make sense to
create care map from it.

Test: TH
Bug: 162623577

Change-Id: I9b5a20fe2d774b52cf7d5eae9deecbc75122a3dd
2021-01-07 17:20:22 -08:00
Yifan Hong
c65a0545dd Move GetBootImageTimestamp to common.
Test: TH
Bug: 162623577

Change-Id: Icbd8b07dce91006a1abca1f0dc40726de8a10219
2021-01-07 17:20:22 -08:00
Bob Badour
7a95820907 Revert^2 "Rough-in license metadata support to make."
Changed shell script to /bin/sh syntax for portability to mac_os etc.

54066744f0

Change-Id: Ibd472fc1bedfdc228a704108f0280b13ab1c9333
2021-01-06 20:52:43 -08:00
Jerome Gaillard
54066744f0 Revert "Rough-in license metadata support to make."
Revert "Export soong license data to make."

Revert "Add ability to declare licenses in soong."

Revert submission 1377717-metalics

Reason for revert: This has broken renderscript_mac target for aosp-master, see b/176909442

Reverted Changes:
I26ac54ca9:Define the standard license_kind rules.
I656486070:Export soong license data to make.
If9d661dfc:Export soong license data to make.
I97943de53:Add ability to declare licenses in soong.
Icaff40171:Rough-in license metadata support to make.
Ib8e538bd0:Add variables for notice deps, license kinds etc.

Change-Id: Iaf2243cba16f6191b022371a14f81ef8194a47d1
2021-01-06 19:00:05 +00:00
Bob Badour
20fc1b3719 Rough-in license metadata support to make.
Change-Id: Icaff40171b94538219a8caa697b182dbdcc8229a
2021-01-05 08:42:48 -08:00
SzuWei Lin
baf5c8114b Make ro.product.cpu.abilist.* to be fetched dynamically
The patch rename the original ro.product.cpu.abilist* to
ro.system.product.cpu.abilist*. The property service will
define ro.product.cpu.abilist* by the intersect ABIs of
ro.{partition}.product.cpu.abilist*.

Bug: 176520383
Test: build aosp_arm64-userdebug and check the system/build.prop
Change-Id: I26e404632b28e9f6abd462920e0084ad2022a763
2020-12-31 16:59:27 +08:00
Treehugger Robot
fdb2f2c5db Merge "Swich TEST_MAPPING config to unit_tests:true" 2020-12-22 21:32:31 +00:00
Julien Desprez
dc0663f66f Merge "Remove the zipalign_test mapping file" 2020-12-22 00:25:05 +00:00
Julien Desprez
6d9e78fd5d Swich TEST_MAPPING config to unit_tests:true
All unit_tests:true are run in presubmit which avoids
the explicit definition of TEST_MAPPING.

Test: presubmit runs all those tests
Bug: 175408655
Change-Id: Ibfad3fe2dd8d2fa1ebc6f6bcbd8ab34c2a0069dc
2020-12-21 13:56:05 -08:00
Julien Desprez
2bbc159e97 Remove the zipalign_test mapping file
Not needed as the host-unit-tests suite runs in
presbumit now and cover this

Test: None
Bug: 175408655
Change-Id: Iea3e3cd7f696f43b6c686ef9cbee1194f514e589
2020-12-21 09:58:17 -08:00
Yo Chiang
d21e7dc833 Add support for packing multiple vendor ramdisks in vendor_boot
Add support for partitioning the vendor_boot kernel modules into
multiple vendor ramdisk fragments. The partition granularity is kernel
module directory. This mechanism builds upon the existing
BOARD_KERNEL_MODULE_DIRS mechanism. For example, say we have three
kernel module directories:

BOARD_KERNEL_MODULE_DIRS := foo bar baz

We can then define a vendor ramdisk fragment:

BOARD_MKBOOTIMG_ARGS += --header_version 4
BOARD_VENDOR_RAMDISK_FRAGMENTS := dlkm_foobar

And let said ramdisk to contain the DLKM directories "foo" and "bar":

BOARD_VENDOR_RAMDISK_FRAGMENT.dlkm_foobar.KERNEL_MODULE_DIRS := foo bar
BOARD_VENDOR_RAMDISK_FRAGMENT.dlkm_foobar.MKBOOTIMG_ARGS := <mkbootimg args>

The built vendor_boot image would contain two ramdisks.
The first one being the "default" ramdisk, which contains DLKM directory
"baz" and the rest of the files that get's installed to
$(TARGET_VENDOR_RAMDISK_OUT).
The second one is the "dlkm_foobar" ramdisk, which contains the two DLKM
directories.

Design doc: go/vendor-boot-v4

Bug: 162864255
Test: Modify BoardConfig.mk to have a product build v4 vendor_boot
Test: Use unpack_bootimg to verify the vendor_boot image
Test: Teach a bootloader how to handle v4 boot image, flash boot &
  vendor_boot and boot device
Change-Id: Ibb1bbd7ebe36430c55ec6c4818c1d3888a319089
2020-12-17 11:07:04 +08:00
Julien Desprez
31839fe1cd Merge "mark zipalign_test as unit tests" 2020-12-14 16:11:41 +00:00
Elliott Hughes
27de935d83 Update OWNERS.
Test: treehugger
Change-Id: I5f9ddd03d5cd34fb398b842c933dc6a365be4b39
2020-12-11 12:56:27 -08:00
Julien Desprez
74b8e09094 mark zipalign_test as unit tests
Test: make
Bug: 175408655
Change-Id: I03a7a0bb19ea408830acbf1ebe12ffb25012af91
2020-12-11 11:52:45 -08:00
Jaegeuk Kim
74e57f6482 Merge "Enable system partition compression" 2020-12-10 00:41:04 +00:00
Treehugger Robot
70c61bc6e9 Merge changes I1deda85d,I43d7b2c6
* changes:
  calculation partition size use compressed image size
  add erofs to non partition spanning fs
2020-12-09 06:50:56 +00:00
Robin Hsu
3e51f4234c Enable system partition compression
* Prerequisites: external/f2fs-tools: sload compression support
* Must work with corresponding changes in system/extras repository
* If Board config does not change, it falls back to old behavior,
  i.e. no compression for the system partition
* Kernel f2fs compression support is a prerequisite if the Board config
  enables the compression (see below)
* Necessary board config change
 (e.g. device/<vendor>/<product>/BoardConfig-common.mk)

BOARD_SYSTEMIMAGE_FILE_SYSTEM_TYPE := f2fs
BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE := 200000000
BOARD_SYSTEMIMAGE_FILE_SYSTEM_COMPRESS := true
BOARD_SYSTEMIMAGE_F2FS_SLOAD_COMPRESS_FLAGS := <sload compress sub-options>

Setting BOARD_SYSTEMIMAGE_FILE_SYSTEM_COMPRESS to true enables both the
compression support when the initial empty file system be made (mkfs.f2fs)
and the compression flag (-c) when the system image files be side-loaded
by sload.

Sload compress sub-options (i.e. options other than -c) will be provided
by BOARD_SYSTEMIMAGE_F2FS_SLOAD_COMPRESS_FLAGS.   If it is not given, or
is empty, the default sub-options will be used

Please refer to the sload.f2fs manual page.

Setting BOARD_SYSTEMIMAGE_FILE_SYSTEM_TYPE to f2fs is trivially necessary.

* File system table (fstab), notably the file 'fstab.hardware', should
  also changed:
  - The file type must be changed to f2fs
  - Perhaps also other f2fs-specific options

Bug: 170918499
Test: Pixel4a userdebug build (from build id 6918751)
Signed-off-by: Robin Hsu <robinhsu@google.com>
Change-Id: Id9d67b5cb35dc806e06ff1320e89114abc996a28
2020-12-09 07:24:55 +08:00
Rupert Shuttleworth
729427460b Ensure file permissions are writable before attempting to sign.
Test: Manually using USE_BAZEL=1 use_rbe m dist (fixes a build error).
Change-Id: I1aac185f12ac430514c975103547ce211ea6c99e
2020-12-08 10:03:34 +00:00
Huang Jianan
35f015ea5e calculation partition size use compressed image size
If dynamic partitioning is enabled and the partition size is not set,
du will be used to calculate the partition size.

For compressed file systems, it's better to use the compressed size
to avoid wasting space.

Bug:174816929
Test: erofs image size is smaller than the original file
Change-Id: I1deda85d312c19620680531223fffcfb815e5fd4
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
2020-12-07 07:09:28 +00:00
Huang Jianan
62d926e2fe add erofs to non partition spanning fs
Bug:174816929
Test: sparse image size will be fixed after rebuilding erofs image
Change-Id: I43d7b2c61c57301935981f6f1282afb685cce331
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
2020-12-04 09:08:00 +00:00
Colin Cross
bbc94b0d83 Use jni_libs instead of required for signapk conscrypt JNI
Using jni_libs tells Soong to install the JNI library into
a genrule sandbox alongside the java binary.

Bug: 124313442
Test: m checkbuild
Change-Id: Ia550102c7aa483497f95ac2ea8bb01bce89ae9c1
2020-12-02 11:37:21 -08:00
Colin Cross
40c9513f7f Follow argument changes to RuleBuilder
Pass pctx and ctx to NewRuleBuilder instead of RuleBuilder.Build,
and don't pass ctx to RuleBuilderCommand.BuiltTool.  Follows the
changes in I63e6597e19167393876dc2259d6f521363b7dabc.

Test: m checkbuild
Change-Id: I5d11e07ae1a24a967e379c151f4f5d7d188c6cc0
2020-12-01 16:21:51 -08:00
Daniel Norman
e9fdfb9fea Merge "Runs host_init_verifier on merged target files packages." 2020-11-25 18:13:30 +00:00
Chih-hung Hsieh
5364ffafa9 Merge "Dump a new CSV with warning messages" 2020-11-25 17:38:21 +00:00
Daniel Norman
21c34f78e8 Runs host_init_verifier on merged target files packages.
This verifies the init rc files in the merged result.

Bug: 163089173
Test: test_common.py
Test: Run merge_target_files.py to merge two target-files packages where
      one has init_rc errors. Observe script failure.
Test: Run merge_target_files.py on two good target-files packages,
      observe no failure.
Change-Id: I86c8e5a2bc07c2c1896ac40afd32bc1d055447ee
2020-11-24 17:26:27 +00:00
Saeid Farivar Asanjan
75dc8d2dab Dump a new CSV with warning messages
Bug: 171068870
Test: manual
Change-Id: Ibb528940daf9a6cd99ece8e5a63d67aacdfc131c
2020-11-24 05:46:33 +00:00
Treehugger Robot
84e9bf1a94 Merge "Try to find host tools in the same directory as the caller" 2020-11-24 02:05:39 +00:00
Treehugger Robot
bda655cead Merge "build: make fs_config_generator.py python2 explicitly" 2020-11-23 23:08:22 +00:00
Kelvin Zhang
788055c00b Merge "Add --disable_verity_computation option to ota script" 2020-11-23 19:43:24 +00:00
Kelvin Zhang
caf7bbc39e Add --disable_verity_computation option to ota script
Test: treehugger, generate an OTA with --disable_verity_computation, serve it to
VABC device. Make sure it works

Change-Id: I0a5f3c018727cb3eed3edd004d60d09d13e34039
2020-11-23 13:24:34 -05:00
Jiyong Park
c8c94ac38e Try to find host tools in the same directory as the caller
common.py is used by many python-based host tools. When a host tool
invokes another host tool, it's usually done with the name of the tool,
not the absolute path of the tool. For example, ["cmd", "arg"] instead
of ["out/soong/host/linux-x86/bin/cmd", "arg"].

Previously, the caller of the tool has to teach all the locations of the
sub-tools that the tool will internally use. But this isn't ideal;

1) It's against the abstraction. The existence of the sub tools and their
names are implementation detail of the containing tool. It's also
subject to change.

2) This isn't scalable. Sometimes a host tool invokes a large number of
host tools. Furthermore, the sub tools might invoke other sub-sub tools.
Then the location of the sub-sub-tools had to be known to the top-level
tool.

The idea here is to make use of the fact that

a) dependencies to the sub (and sub-sub) tools are already described in
the Android.bp and

b) the dependencies are guaranteed to be up-to-date and installed to the
host tool directory (out/soong/host/linux-x86/bin) by Soong.

Then by the time a host tool is invoked, all of its sub tools should
be found in the host tool directory. So, when "cmd" is about to be
invoked, common.py first tries to find it from the user-given paths. If
not found there, it falls back to search the directory where the current
tool is located at. Then finally falls back to the original name "cmd"
and expects it to be found under PATH.

Bug: 172414391
Test: m
Change-Id: Id7b44f9021be3bbf0631ddafe382ea3990f7ea74
2020-11-20 10:40:54 +09:00
Daniel Norman
38888d390f Turns on building vbmeta.img when merging if avb_enable=true.
Needed after commit 8218225794
which skips regenerating vbmeta.img if avb_building_vbmeta_image!=true.
Partial builds that disable vbmeta generation do not set this field.

Bug: 169892879
Test: Run merge_target_files.py on builds that have avb_enable=true but
      PRODUCT_BUILD_VBMETA_IMAGE=false. Observe vbmeta.img generation.
Change-Id: Id498dbd28c909e19cd35178b95a0b5dc22a1ac70
2020-11-19 14:51:15 -08:00
Yifan Hong
187e975927 Merge changes from topic "gki_vintf_level"
* changes:
  Pass kernel release file to check_vintf.
  Pass kernel version file to check_vintf.
2020-11-16 23:48:47 +00:00
Treehugger Robot
59dd38cdb0 Merge "Include complete boot image on builds containing gki apex" 2020-11-16 18:04:46 +00:00
Treehugger Robot
fc65231af1 Merge "Support PRODUCT_BUILD_VBMETA_IMAGE for target files" 2020-11-16 13:59:41 +00:00
Kelvin Zhang
072efc92e2 Merge "Allow downgradeable secondary payload" 2020-11-14 03:09:49 +00:00