Commit graph

2286 commits

Author SHA1 Message Date
Yifan Hong
cb09afc04a Merge changes from topic "common.GetBootImageTimestamp"
* changes:
  Load boot build props to info_dict properly.
  Add GetBootImageBuildProp.
  Split PARTITIONS_WITH_BUILD_PROP from *_WITH_CARE_MAP
  Move GetBootImageTimestamp to common.
2021-01-08 22:18:51 +00:00
Yifan Hong
10482a2a56 Load boot build props to info_dict properly.
Add boot to PARTITION_WITH_BUILD_PROP. Let PartitionBuildProps handle
boot partitions by extracting build.prop file.

Test: generate OTA and call
      `payload_info.py payload.bin`, check that boot image timestamp is
      set
Bug: 162623577

Change-Id: I71955e4ee494ffbb5d111c3d02a129c0f879b6d1
2021-01-07 17:20:22 -08:00
Yifan Hong
85ac5012aa Add GetBootImageBuildProp.
Add a function that extracts build.prop from boot image.

Test: TH
Bug: 162623577
Change-Id: I673ccd8e7c4846ca4c41a86349a388b2e800724e
2021-01-07 17:20:22 -08:00
Yifan Hong
5057b95572 Split PARTITIONS_WITH_BUILD_PROP from *_WITH_CARE_MAP
Right now they are the same content. In the future, boot will be added
to PARTITIONS_WITH_BUILD_PROP, but it is not added to
PARTITIONS_WITH_CARE_MAP. Boot partition has a cpio filesystem in the
ramdisk, so it contains a build.prop file, but it doesn't make sense to
create care map from it.

Test: TH
Bug: 162623577

Change-Id: I9b5a20fe2d774b52cf7d5eae9deecbc75122a3dd
2021-01-07 17:20:22 -08:00
Yifan Hong
c65a0545dd Move GetBootImageTimestamp to common.
Test: TH
Bug: 162623577

Change-Id: Icbd8b07dce91006a1abca1f0dc40726de8a10219
2021-01-07 17:20:22 -08:00
Julien Desprez
6d9e78fd5d Swich TEST_MAPPING config to unit_tests:true
All unit_tests:true are run in presubmit which avoids
the explicit definition of TEST_MAPPING.

Test: presubmit runs all those tests
Bug: 175408655
Change-Id: Ibfad3fe2dd8d2fa1ebc6f6bcbd8ab34c2a0069dc
2020-12-21 13:56:05 -08:00
Yo Chiang
d21e7dc833 Add support for packing multiple vendor ramdisks in vendor_boot
Add support for partitioning the vendor_boot kernel modules into
multiple vendor ramdisk fragments. The partition granularity is kernel
module directory. This mechanism builds upon the existing
BOARD_KERNEL_MODULE_DIRS mechanism. For example, say we have three
kernel module directories:

BOARD_KERNEL_MODULE_DIRS := foo bar baz

We can then define a vendor ramdisk fragment:

BOARD_MKBOOTIMG_ARGS += --header_version 4
BOARD_VENDOR_RAMDISK_FRAGMENTS := dlkm_foobar

And let said ramdisk to contain the DLKM directories "foo" and "bar":

BOARD_VENDOR_RAMDISK_FRAGMENT.dlkm_foobar.KERNEL_MODULE_DIRS := foo bar
BOARD_VENDOR_RAMDISK_FRAGMENT.dlkm_foobar.MKBOOTIMG_ARGS := <mkbootimg args>

The built vendor_boot image would contain two ramdisks.
The first one being the "default" ramdisk, which contains DLKM directory
"baz" and the rest of the files that get's installed to
$(TARGET_VENDOR_RAMDISK_OUT).
The second one is the "dlkm_foobar" ramdisk, which contains the two DLKM
directories.

Design doc: go/vendor-boot-v4

Bug: 162864255
Test: Modify BoardConfig.mk to have a product build v4 vendor_boot
Test: Use unpack_bootimg to verify the vendor_boot image
Test: Teach a bootloader how to handle v4 boot image, flash boot &
  vendor_boot and boot device
Change-Id: Ibb1bbd7ebe36430c55ec6c4818c1d3888a319089
2020-12-17 11:07:04 +08:00
Jaegeuk Kim
74e57f6482 Merge "Enable system partition compression" 2020-12-10 00:41:04 +00:00
Treehugger Robot
70c61bc6e9 Merge changes I1deda85d,I43d7b2c6
* changes:
  calculation partition size use compressed image size
  add erofs to non partition spanning fs
2020-12-09 06:50:56 +00:00
Robin Hsu
3e51f4234c Enable system partition compression
* Prerequisites: external/f2fs-tools: sload compression support
* Must work with corresponding changes in system/extras repository
* If Board config does not change, it falls back to old behavior,
  i.e. no compression for the system partition
* Kernel f2fs compression support is a prerequisite if the Board config
  enables the compression (see below)
* Necessary board config change
 (e.g. device/<vendor>/<product>/BoardConfig-common.mk)

BOARD_SYSTEMIMAGE_FILE_SYSTEM_TYPE := f2fs
BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE := 200000000
BOARD_SYSTEMIMAGE_FILE_SYSTEM_COMPRESS := true
BOARD_SYSTEMIMAGE_F2FS_SLOAD_COMPRESS_FLAGS := <sload compress sub-options>

Setting BOARD_SYSTEMIMAGE_FILE_SYSTEM_COMPRESS to true enables both the
compression support when the initial empty file system be made (mkfs.f2fs)
and the compression flag (-c) when the system image files be side-loaded
by sload.

Sload compress sub-options (i.e. options other than -c) will be provided
by BOARD_SYSTEMIMAGE_F2FS_SLOAD_COMPRESS_FLAGS.   If it is not given, or
is empty, the default sub-options will be used

Please refer to the sload.f2fs manual page.

Setting BOARD_SYSTEMIMAGE_FILE_SYSTEM_TYPE to f2fs is trivially necessary.

* File system table (fstab), notably the file 'fstab.hardware', should
  also changed:
  - The file type must be changed to f2fs
  - Perhaps also other f2fs-specific options

Bug: 170918499
Test: Pixel4a userdebug build (from build id 6918751)
Signed-off-by: Robin Hsu <robinhsu@google.com>
Change-Id: Id9d67b5cb35dc806e06ff1320e89114abc996a28
2020-12-09 07:24:55 +08:00
Rupert Shuttleworth
729427460b Ensure file permissions are writable before attempting to sign.
Test: Manually using USE_BAZEL=1 use_rbe m dist (fixes a build error).
Change-Id: I1aac185f12ac430514c975103547ce211ea6c99e
2020-12-08 10:03:34 +00:00
Huang Jianan
35f015ea5e calculation partition size use compressed image size
If dynamic partitioning is enabled and the partition size is not set,
du will be used to calculate the partition size.

For compressed file systems, it's better to use the compressed size
to avoid wasting space.

Bug:174816929
Test: erofs image size is smaller than the original file
Change-Id: I1deda85d312c19620680531223fffcfb815e5fd4
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
2020-12-07 07:09:28 +00:00
Huang Jianan
62d926e2fe add erofs to non partition spanning fs
Bug:174816929
Test: sparse image size will be fixed after rebuilding erofs image
Change-Id: I43d7b2c61c57301935981f6f1282afb685cce331
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
2020-12-04 09:08:00 +00:00
Daniel Norman
21c34f78e8 Runs host_init_verifier on merged target files packages.
This verifies the init rc files in the merged result.

Bug: 163089173
Test: test_common.py
Test: Run merge_target_files.py to merge two target-files packages where
      one has init_rc errors. Observe script failure.
Test: Run merge_target_files.py on two good target-files packages,
      observe no failure.
Change-Id: I86c8e5a2bc07c2c1896ac40afd32bc1d055447ee
2020-11-24 17:26:27 +00:00
Treehugger Robot
84e9bf1a94 Merge "Try to find host tools in the same directory as the caller" 2020-11-24 02:05:39 +00:00
Kelvin Zhang
788055c00b Merge "Add --disable_verity_computation option to ota script" 2020-11-23 19:43:24 +00:00
Kelvin Zhang
caf7bbc39e Add --disable_verity_computation option to ota script
Test: treehugger, generate an OTA with --disable_verity_computation, serve it to
VABC device. Make sure it works

Change-Id: I0a5f3c018727cb3eed3edd004d60d09d13e34039
2020-11-23 13:24:34 -05:00
Jiyong Park
c8c94ac38e Try to find host tools in the same directory as the caller
common.py is used by many python-based host tools. When a host tool
invokes another host tool, it's usually done with the name of the tool,
not the absolute path of the tool. For example, ["cmd", "arg"] instead
of ["out/soong/host/linux-x86/bin/cmd", "arg"].

Previously, the caller of the tool has to teach all the locations of the
sub-tools that the tool will internally use. But this isn't ideal;

1) It's against the abstraction. The existence of the sub tools and their
names are implementation detail of the containing tool. It's also
subject to change.

2) This isn't scalable. Sometimes a host tool invokes a large number of
host tools. Furthermore, the sub tools might invoke other sub-sub tools.
Then the location of the sub-sub-tools had to be known to the top-level
tool.

The idea here is to make use of the fact that

a) dependencies to the sub (and sub-sub) tools are already described in
the Android.bp and

b) the dependencies are guaranteed to be up-to-date and installed to the
host tool directory (out/soong/host/linux-x86/bin) by Soong.

Then by the time a host tool is invoked, all of its sub tools should
be found in the host tool directory. So, when "cmd" is about to be
invoked, common.py first tries to find it from the user-given paths. If
not found there, it falls back to search the directory where the current
tool is located at. Then finally falls back to the original name "cmd"
and expects it to be found under PATH.

Bug: 172414391
Test: m
Change-Id: Id7b44f9021be3bbf0631ddafe382ea3990f7ea74
2020-11-20 10:40:54 +09:00
Daniel Norman
38888d390f Turns on building vbmeta.img when merging if avb_enable=true.
Needed after commit 8218225794
which skips regenerating vbmeta.img if avb_building_vbmeta_image!=true.
Partial builds that disable vbmeta generation do not set this field.

Bug: 169892879
Test: Run merge_target_files.py on builds that have avb_enable=true but
      PRODUCT_BUILD_VBMETA_IMAGE=false. Observe vbmeta.img generation.
Change-Id: Id498dbd28c909e19cd35178b95a0b5dc22a1ac70
2020-11-19 14:51:15 -08:00
Yifan Hong
187e975927 Merge changes from topic "gki_vintf_level"
* changes:
  Pass kernel release file to check_vintf.
  Pass kernel version file to check_vintf.
2020-11-16 23:48:47 +00:00
Treehugger Robot
59dd38cdb0 Merge "Include complete boot image on builds containing gki apex" 2020-11-16 18:04:46 +00:00
Treehugger Robot
fc65231af1 Merge "Support PRODUCT_BUILD_VBMETA_IMAGE for target files" 2020-11-16 13:59:41 +00:00
Yifan Hong
72e78f21b2 Pass kernel version file to check_vintf.
check_vintf now accept a path to a file that contains the kernel
version string. Use it.

Test: m check-vintf-all and manual inspect
Bug: 161317193
Change-Id: I4812c93f352686d73c5832d59e43a297d93e67a2
2020-11-13 18:45:03 -08:00
Bowgo Tsai
8218225794 Support PRODUCT_BUILD_VBMETA_IMAGE for target files
Currently we support skip building vbmeta.img when
PRODUCT_BUILD_VBMETA_IMAGE is set to false, which is used
when generating a system-only or vendor-only build artifacts.

However, when using `make dist` to build target files,
vbmeta.img is still generated. This change fixes this by
passing the related info to misc_info.txt for the release
tool to skip building vbmeta.img when necessary.

To make this happen, a device config should:
 1. Set PRODUCT_BUILD_VBMETA_IMAGE := false
 2. Remove 'vbmeta' from AB_OTA_PARTITIONS:
     AB_OTA_PARTITIONS += \
	 boot \
	 system \
    -    vbmeta \
	 dtbo \
	 product \
	 system_ext \
         vbmeta_system

Bug: 161659370
Test: `make dist` and checks the target file does not include vbmeta.img
Change-Id: Iddca9fe0b171a50a80e5a88aa462e118f18a7e6f
2020-11-13 11:28:17 +08:00
Kelvin Zhang
6cdf39168f Allow downgradeable secondary payload
Test: treehugger
Bug: 172811557
Change-Id: I58f3123b9bc69342427f95250a5ee57ede4ac926
2020-11-11 14:23:00 -05:00
Kelvin Zhang
83ea783146 Filter out ab_partitions in source_info as well
When generating a incremental partial OTA, we need to filter out
partitions in ab_partitions which are not in partial list

Test: Generate a partial incremental OTA
Change-Id: Iff7748ce6181a9a231557de0539004211587232c
2020-11-11 14:23:00 -05:00
Kelvin Zhang
65029a2801 Include complete boot image on builds containing gki apex
Test: treehugger
Bug: 171906589
Change-Id: I231dac425ab14602492346006e331e4666173614
2020-11-11 09:16:32 -05:00
Kelvin Zhang
6f83343e7e Merge "Allow downgradable partial OTA" 2020-11-11 14:05:47 +00:00
Kelvin Zhang
80195724e7 Allow downgradable partial OTA
Cuttlefish doesn't support updatin vendor partition. So in testing we
need to exclude vendor and generate a downgrade partial package.

Test: generate a partial downgrade OTA
Bug: 171999375
Change-Id: I97e450d4063320cf55003a7a5c5ef913c321e067
2020-11-10 09:11:20 -05:00
Treehugger Robot
ba1811bf83 Merge "Checks for APK sharedUserIds that cross partition group boundaries." 2020-11-06 22:29:46 +00:00
Daniel Norman
d33515623a Checks for APK sharedUserIds that cross partition group boundaries.
This check is used when merging target files to ensure that a merged
build does not contain any APKs that share UID across builds.

Bug: 171431774
Test: test_common
Test: Use merge_target_files.py to merge two partial builds,
      observe no failures for inputs without colliding APKs.
Test: Use merge_target_files.py to merge two partial builds,
      observe failure for inputs that have an APK that shares a
      UID across input partition groups.
Change-Id: I9dc57216882741ae46a99cfd7847f34702c75582
2020-11-05 15:17:05 -08:00
Treehugger Robot
50c0f97fe7 Merge "Finds APK shared UID violations when merging target files." 2020-11-04 21:44:40 +00:00
Kelvin Zhang
31233e5922 Disable FEC computation for VABC
Test: treehugger, generate a package and make sure --disable_fec is
passed to delta_generator

Change-Id: If256af90b11a7fa03299c1272466d35ef3798a24
2020-11-03 15:45:30 -05:00
Treehugger Robot
0806152b83 Merge "Fix partition timestamps for secondary payload" 2020-11-02 21:22:17 +00:00
Kelvin Zhang
a59bb27754 Fix partition timestamps for secondary payload
When generating secondary payload, we exclude many partitions. These
partitions should be excluded from partition timestamps as well.

Bug: 172022126
Test: ota_from_target_files --include_secondary
Change-Id: Ia3121919154181dc882c4694ad2591dcb81e719f
2020-10-30 14:54:40 -04:00
Alistair Delva
38d49ca61a Merge "Allow flashing erofs system imgs" 2020-10-30 06:41:25 +00:00
Daniel Norman
b8d52a2fdc Finds APK shared UID violations when merging target files.
This involved moving the find-shareduid-violation.py script to
releasetools to simplify the cross-tool usage. This new location aligns
this script with other similar python host tools.

In a future change this violation file will be used to check for
shared UID violations across the input build partition boundary.

Bug: 171431774
Test: test_merge_target_files
Test: Use merge_target_files.py to merge two partial builds,
      observe shared UID violations file contents in the result.
Test: m dist out/dist/shareduid_violation_modules.json
      (Checking that existing behavior in core/tasks is presereved)
Change-Id: I7deecbe019379c71bfdbedce56edac55e7b27b41
2020-10-29 09:33:26 -07:00
Gao Xiang
961041ac80 Allow flashing erofs system imgs
Bug: 158339527
Test: build & boot
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
Change-Id: Ie74db8ebcd7214ffbec49d42b305b2f3ac2e8c9d
2020-10-28 01:24:39 +08:00
Treehugger Robot
13e76a3e8f Merge "Formatting and general cleanup of merge_target_files." 2020-10-23 21:04:20 +00:00
Daniel Norman
dbbf5a3912 Formatting and general cleanup of merge_target_files.
Also adds a new validation that IMAGES/<partition>.img must come from
the same build that provides <PARTITION>/*.

Bug: 171431774
Test: python3 -m unittest test_merge_target_files
Test: Use merge_target_files to merge a build.
Change-Id: Ia1f3f2e65a0ca90962216fb0c5cdd24c4c150cae
2020-10-22 16:37:13 -07:00
Hongguang Chen
49ab1b90df OTA: Support A/B devices custom images update.
Add a new custom_image option to configure which custom images to use to
update custom partitions in A/B update.

This change also moves oem_settings to common option as A/B update will
use it to set oem properties too.

BUG: 171225290
Test: unittest pass, generate OTAs, flash to devices and check results
Change-Id: I279477d6b2954fb3705d7efede0a8bcd330c108b
2020-10-21 20:20:52 +00:00
Tianjie Xu
ec952237e2 Merge "sign_target_files_apks: Fix password encrypted keys handle" 2020-10-20 21:45:54 +00:00
Daniel Norman
29fda6ce51 Merge "Make merge_target_files more lenient, needed for cross-release merging." 2020-10-20 20:31:17 +00:00
Oleh Cherpak
e555ab1848 sign_target_files_apks: Fix password encrypted keys handle
This patch restores the possibility of using password encrypted
keys for build signing.

Bug: 171221825
Test: 1. Generate password encrypted keys (write non empty passwords):
        $ subject='/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com'
        $ mkdir ~/.android-certs
        $ for x in releasekey platform shared media; do \
            ./development/tools/make_key ~/.android-certs/$x \
            "$subject"; \
          done
      2. Create a file with passwords:
        Example of passwd file:
            [[[ 12345678 ]]] /home/user/.android-certs/releasekey
            [[[ 12345678 ]]] /home/user/.android-certs/platform
            [[[ 12345678 ]]] /home/user/.android-certs/shared
            [[[ 12345678 ]]] /home/user/.android-certs/media
            [[[ 12345678 ]]] /home/user/.android-certs/networkstack
      3. Tell system where to find passwords:
        $ export ANDROID_PW_FILE=/path/to/file/with/passwords
      4. Generate a release image:
        $ make dist
        $ sign_target_files_apks \
        -o --default_key_mappings ~/.android-certs \
        out/dist/*-target_files-*.zip \
        signed-target_files.zip

Signed-off-by: Oleh Cherpak <oleh.cherpak@globallogic.com>
Change-Id: I3e9d5318f69a2c3ac6eec64b36163b6544d49c90
2020-10-20 18:29:35 +00:00
Tianjie Xu
d53d6f7169 Merge "Add ota script support to generate partial updates" 2020-10-19 18:47:12 +00:00
Treehugger Robot
6a2135b6e6 Merge changes from topic "split_ramdisk"
* changes:
  Exclude board cmdline/dtb/pagesize/base in generic boot image
  Add BOARD_MOVE_RECOVERY_RESOURCES_TO_VENDOR_BOOT.
  Add BOARD_EXCLUDE_KERNEL_FROM_RECOVERY_IMAGE.
  Add GKI global variable.
2020-10-17 00:40:54 +00:00
Tianjie Xu
11e5f33e2c Merge "Update ota script to match init's logic" 2020-10-16 23:10:37 +00:00
Tianjie
b37c5be8cc Update ota script to match init's logic
In aosp/1459042, init has updated it's logic to use
ro.build.version.release_or_codename for fingerprint calculation.
The ota script needs the same update for ota targeting to work
correctly.

As a best effort to support pre-S dev stage builds, use
ro.build.version.release for sdk version < 30.

Bug: 170968068
Bug: 158483506
Test: unit tests, generate an OTA for S build
Change-Id: I01ca8a3b7b8b58f94b10f90ed0d27e688a72b866
2020-10-16 11:46:03 -07:00
Jaegeuk Kim
ed754fb339 Add build property to enable fscompression
Bug: 162253911
Bug: 110272315
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: I3cb7d0a055a7f83c9d0321d2b16f11f40a74f7aa
Merged-In: I3cb7d0a055a7f83c9d0321d2b16f11f40a74f7aa
2020-10-15 17:10:25 -07:00
Yifan Hong
38ab4d81ed Add ota script support to generate partial updates
Allow the ota generation script to take a list of partitions;
and thus generating a partial ota update package. This allow the
testing of partial update functionality, e.g. system-only updates.

Bug: 170921953
Test: unit tests, generate and apply a partial update on coral with system, system_ext, vbmeta_system
Change-Id: I0d8f93806dfbf7b781ea911117b02fc5c0971434
2020-10-15 16:30:13 -07:00