Commit graph

1486 commits

Author SHA1 Message Date
Tianjie Xu
ccbae48ac5 Generates the care_map.txt in protobuf format
Call the host binary care_map_generator to generate the care_map in
protobuf format. The parsing part of proto messages has already been
supported in the update_verifier.

Bug: 77867897
Test: unittests pass; run add_image_to_target_files
Change-Id: I40d3184b4b5c48a6dd55203afc84cca73d5765e9
2018-08-17 17:39:00 +00:00
Dario Freni
77c778c040 s/PRODUCT-SERVICES/PRODUCT_SERVICES/g
Attempting to reduce the number of different spellings we have for
"product-services" in the codebase.

Bug: 112431447
Test: m
Change-Id: Iec531122297cf4b86ff8601732d4ab7ccf5c2183
2018-08-17 00:28:17 +01:00
Bowgo Tsai
d624fa63d5 Add support for /odm partition
This CL is largely an adaptation of Change-Id
I5d229f6ac729ea6df9ff1f14cee2e28972cd9b4d

tools/vendor_buildinfo.sh is also renamed to tools/device_buildinfo.sh.
The caller then can invoke device_buildinfo.sh "vendor" | "odm" to
generate properties for vendor.img and odm.img, respectively.

It adds the following variables:

- BOARD_AVB_ODM_KEY_PATH
- BOARD_AVB_ODM_ALGORITHM
- BOARD_AVB_ODM_ROLLBACK_INDEX_LOCATION
- BOARD_ODMIMAGE_FILE_SYSTEM_TYPE
- BOARD_ODMIMAGE_EXTFS_INODE_COUNT
- BOARD_ODMIMAGE_EXTFS_RSV_PCT
- BOARD_ODMIMAGE_PARTITION_SIZE
- BOARD_ODMIMAGE_JOURNAL_SIZE
- BOARD_ODMIMAGE_SQUASHFS_BLOCK_SIZE
- BOARD_ODMIMAGE_SQUASHFS_COMPRESSOR
- BOARD_ODMIMAGE_SQUASHFS_COMPRESSOR_OPT
- BOARD_ODMIMAGE_SQUASHFS_DISABLE_4K_ALIGN
- BOARD_PREBUILT_ODMIMAGE
- BOARD_USES_ODMIMAGE
- LOCAL_ODM_MODULE
- PRODUCT_ODM_BASE_FS_PATH
- PRODUCT_ODM_VERITY_PARTITION
- PRODUCT_ODM_PROPERTIES
- TARGET_COPY_OUT_ODM
- TARGET_OUT_ODM
- TARGET_OUT_ODM_*

Bug: 64195575
Test: boot a Taimen with existing images

Test: `make odmimage` with
      - BOARD_AVB_ENABLE := true
      - BOARD_ODMIMAGE_PARTITION_SIZE := 62914560
      - BOARD_ODMIMAGE_FILE_SYSTEM_TYPE := ext4
      - TARGET_COPY_OUT_ODM := odm
      - PRODUCT_ODM_PROPERTIES += odm.test.build=success

Test: `make odmimage` with
      - BOARD_ODMIMAGE_PARTITION_RESERVED_SIZE := 10485760
      - BOARD_ODMIMAGE_FILE_SYSTEM_TYPE := ext4
      - BOARD_AVB_ENABLE := true
      - TARGET_COPY_OUT_ODM := odm
      - PRODUCT_ODM_PROPERTIES += odm.test.build=success
      - PRODUCT_USE_DYNAMIC_PARTITION_SIZE := true

Change-Id: I4dea7b567ec49a766c7a4683decaf81c7e921d55
2018-08-08 18:59:44 +08:00
Isaac Chen
13d3426f3c Merge "Work around e2fsck issue for building sparse GSIs"
am: a0d3277b8d

Change-Id: I53b45da39145cb10f37bd7bcede55f13cfed801b
2018-08-03 17:07:18 -07:00
Treehugger Robot
a0d3277b8d Merge "Work around e2fsck issue for building sparse GSIs" 2018-08-03 23:29:33 +00:00
Isaac Chen
ec7fa1cd09 Work around e2fsck issue for building sparse GSIs
When trying to support sparse format for GSIs, aosp/720949, a bug in
e2fsck was triggered, and hence failed the build for sparse GSI.

This workaround disables the detection of ASAN ODR so we can build
sparse GSIs successfully like we previously did for the vendors,
some of which can't handle unsparse images.

Bug: 112062612
Test: $ lunch aosp_x86_64-eng
      $ m SANITIZE_HOST=address ASAN_OPTIONS=detect_odr_violation=0 -j
      $ emulator

Change-Id: Ifafd27aa1e57d785b914f78a4d9bfef6a0c33fc9
2018-08-03 11:51:08 +08:00
Yifan Hong
635887d5ac Merge "target_files: copy to PRODUCT-SERVICES instead"
am: 567df6214f

Change-Id: I95bf48eba07daa2b0f6d5b84c45de9b22f7aac0b
2018-08-02 15:31:56 -07:00
Yifan Hong
567df6214f Merge "target_files: copy to PRODUCT-SERVICES instead" 2018-08-02 21:53:57 +00:00
Bryan Henry
a2bf1e5c5c Merge "Fix 2009-01-01 timestamps in releasetools to always be UTC"
am: e118d5bc4a

Change-Id: I12c8cc0664306ba116076b3f6d43e1c01fb9120f
2018-08-01 17:24:41 -07:00
Bryan Henry
e118d5bc4a Merge "Fix 2009-01-01 timestamps in releasetools to always be UTC" 2018-08-02 00:16:06 +00:00
Yifan Hong
4d6b38c47a target_files: copy to PRODUCT-SERVICES instead
The add_img_to_target_files.py script uses upper() to determine
the name of sub-directory under target_files_intermediates. The
name of the image is product-services.img but the sub-directory
name was PRODUCT_SERVICES. Change it to PRODUCT-SERVICES so that
'm dist' works.

Test: m dist
Change-Id: I8cad841c674b5d62c1d803d0383fc3aee14f272a
2018-08-01 16:46:16 -07:00
Yifan Hong
9a7d525849 Merge "Split PRODUCT_USE_LOGICAL_PARTITIONS into sub-flags."
am: d1c308bf40

Change-Id: I834e75dcfc04af0dd8655027bfc251c3d0162815
2018-08-01 11:39:59 -07:00
Bryan Henry
e6d547d53a Fix 2009-01-01 timestamps in releasetools to always be UTC
The usage of datetime.fromtimestamp previously resulted in the build or
signing machine's local timezone affecting the Unix timestamp ultimately
applied to images generated by add_img_to_target_files. The go/ab build
outputs would use 2009-01-01 00:00 UTC, for example, but local builds
and the release signed images (generated through go/ab-sign) would use
2009-01-01 00:00 PST. This change makes the timestamps always use UTC.

Bug: 80600931
Bug: 80093599
Test: 'm -j droid dist' and verified timestamps in resulting
target_files zip.

Change-Id: Ic2a19591519850c249f78254e1464aa6839bfc6c
2018-07-31 18:32:00 -07:00
Yifan Hong
2dae5721d4 Split PRODUCT_USE_LOGICAL_PARTITIONS into sub-flags.
Split the flag into PRODUCT_USE_DYNAMIC_PARTITION_SIZE
and PRODUCT_BUILD_SUPER_PARTITION. More sub-flags can be
added with the same way.

Also change some checks with USE_LOGICAL_PARTITIONS with
the appropriate sub-flags.

This allows easier device bring-up to fulfill the requirements
separately.

This also enables dynamic partition size without logical patition.
Originally, to enable BOARD_*_PARTITION_RESERVED_SIZE must also
enable PRODUCT_USE_LOGICAL_PARTITIONS. The patch fix the rule to
let dynamic partition size is able to be used independently.

Fixes: 111966003

Test: make superimage with PRODUCT_USE_LOGICAL_PARTITIONS
Change-Id: I45b29c87bf94356d8416bbd58a14c8982039c222
2018-08-01 00:39:13 +00:00
Yifan Hong
8883a739b3 Merge changes I1459d62f,I4c2b975e
am: bb937a6b69

Change-Id: Ifdedee1b0251bc9c2a8baf30d28b5d315b5c9e20
2018-07-30 11:35:33 -07:00
Yifan Hong
ebc041af56 productservices => product_services
Make names consistent.
Some things that does not change:
- productservicesimage target and intermediates

Test: build product services image
Bug: 111609632
Change-Id: I4c2b975e194577aad3d51b908b103880625deb5e
2018-07-26 17:19:02 -07:00
Tao Bao
0adc598d82 Merge "releasetools: Fix an issue in image size computation."
am: d578232c95

Change-Id: Idd139fe594e95cd92926da42beb670be239ee113
2018-07-26 12:53:51 -07:00
Tao Bao
4f35d47160 Merge "releasetools: Return the actual image size when building logical partitions."
am: f42fc1a7c3

Change-Id: I714fbd9458c2d6708ade24b36cbfd965a135b840
2018-07-26 10:09:53 -07:00
Tao Bao
c2606eb548 releasetools: Fix an issue in image size computation.
When building a system image with system_root_image enabled, the size
computation should include files under both of in_dir (i.e. /system
files) and root (pointed by 'root_dir'). Because files from both
locations will end up into the built image. The files under root are
usually only a few MiBs, but should be accounted for especially in the
context of logical partitions (where the partition size will be
allocated based on the actual need). Note that we will still need some
"reserved space" (defined via BOARD_*_PARTITION_RESERVED_SIZE) to cover
the cost for filesystem and/or verity metadata.

This CL moves the combination of the two dirs up, before parsing and
computing other properties. This doesn't affect anything for a
successful image building path. It may however increase the time to
error out in certain error path, since it copies the files earlier now.

Test: python -m unittest test_build_image
Test: `make dist`
Test: Setup a target with PRODUCT_USE_LOGICAL_PARTITIONS == true and
      BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE == 20MiB. Build system
      image successfully.
Test: Setup a target with PRODUCT_USE_LOGICAL_PARTITIONS == true and
      BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE == 20MiB. Write a large
      file to root dir (PRODUCT_OUT_ROOT). The image building fails, but
      reporting a size that accounts for both of /system and root.
Change-Id: Idfb26b8e259626ba57ec3bd4f85d357c30e56163
2018-07-26 16:59:57 +00:00
Tao Bao
f42fc1a7c3 Merge "releasetools: Return the actual image size when building logical partitions." 2018-07-26 16:59:36 +00:00
Tao Bao
4251fe97aa releasetools: Return the actual image size when building logical partitions.
When verity is enabled, `partition_size` has the "adjusted" partition
size for holding the filesystem files (i.e. excluding verity hashtree,
and FEC metadata if applicable), whereas `original_partition_size` has
the one for the actual image returned by build_image.py.
`partition_size` must be smaller than `original_partition_size`. As a
result, the later assertion in Makefile (e.g. assert-max-image-size) may
fail to hold when using `partition_size`.

This CL addresses the issue by returning the value in
`original_partition_size` instead, if available. It also changes to
include both values when dumping the size info on image creation
failure.

Fixes: 79106666
Test: `m dist` with aosp_walleye-userdebug
Test: Set up walleye to use logical partitions, with a reserved size of
      20MiB. `m systemimage` no longer fails.
Test: Set up walleye that doesn't use verified boot. Use `m systemimage`
      to build an oversized image. Check the error message.
Change-Id: I41d77bffa85e8eadfbf4a2a7dcd2e798fadfdab7
2018-07-24 11:57:10 -07:00
Tao Bao
fdcb3b5829 Merge "releasetools: Support verifying AVB signed images with chained partitions."
am: 78078646e9

Change-Id: If7656208f92fde69dad358d298cd1c9eb1002ecd
2018-07-24 11:53:57 -07:00
Tao Bao
02a0859b3c releasetools: Support verifying AVB signed images with chained partitions.
For example, verify a target_files.zip that has system AVB-signed as a
chained partition.

  $ build/make/tools/releasetools/validate_target_files.py \
      signed-target_files-4904652.zip \
      --verity_key verifiedboot_pub.pem \
      --avb_system_key_path system_pub.pem

Note that verifiedboot_pub.pem should be the key (either public or
private) to verify vbmeta.img, and 'system_pub.pem' should be the key
(either public or private) for the chained partition of system.

testdata/testkey.key is the private key converted from
testdata/testkey.pk8 for testing purpose (`openssl pkcs8 -in
testdata/testkey.pk8 -inform DER -out testdata/testkey.key -nocrypt`).

Bug: 63706333
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Test: `m dist` on aosp_walleye-userdebug; Run validate_target_files.py
      on the generated target_files.zip.
Test: Set up walleye with chained system partition; `m dist`; Run
      validate_target_files.py on the generated target_files.zip.
Change-Id: I38517ab39baf8a5bc1a6062fab2fe229b68e897d
2018-07-23 15:20:14 -07:00
Tao Bao
f013dc3fed Merge "releasetools: Clean up AppendVBMetaArgsForPartition()."
am: 540a71d2b3

Change-Id: If13ee90c84f75c2452c1e1e8504928414a971ed4
2018-07-23 14:56:22 -07:00
Tao Bao
540a71d2b3 Merge "releasetools: Clean up AppendVBMetaArgsForPartition()." 2018-07-23 21:40:37 +00:00
Yifan Hong
fe421ed263 Merge changes from topic "board_partition_reserved_size"
am: 2d90c246a9

Change-Id: I460ea917b47a61acb0cbb0f17a867d15f61c7d6d
2018-07-23 11:58:52 -07:00
Yifan Hong
2d90c246a9 Merge changes from topic "board_partition_reserved_size"
* changes:
  build_image.py recognize BOARD_PRODUCT_SERVICESIMAGE_PARTITION_RESERVED_SIZE
  build_image.py recognize BOARD_PRODUCTIMAGE_PARTITION_RESERVED_SIZE
2018-07-23 18:38:24 +00:00
Tao Bao
f87ea90d76 Merge "releasetools: Rename the ramdisk_dir in prop dict to root_dir."
am: d094713952

Change-Id: I013c82846a143c5eac050325e66092afc66c958f
2018-07-23 09:41:25 -07:00
Tao Bao
d094713952 Merge "releasetools: Rename the ramdisk_dir in prop dict to root_dir." 2018-07-23 16:33:46 +00:00
Tao Bao
1a3d83d667 Merge "releasetools: Make scripts pylint clean."
am: 23a4070483

Change-Id: I4f4b35565f7d154dd47e4b5b8444e76c6c7227c6
2018-07-23 07:22:59 -07:00
Tao Bao
23a4070483 Merge "releasetools: Make scripts pylint clean." 2018-07-23 14:11:37 +00:00
Tao Bao
3e53ef7b1c releasetools: Clean up AppendVBMetaArgsForPartition().
Drop the parameter of public_key_dir. It's not necessary to stage all
the extracted public keys into the same temporary dir.

Redirect the stderr output to STDOUT while calling avbtool, so that it
can dump the error message accordingly.

Also remove the check for having empty img_path arg. It doesn't look
like a clean logic to allow the function to be called with invalid /
empty arg. The only caller (i.e. AddVBMeta) already asserts img_path
being valid as well.

Test: python -m unittest test_add_img_to_target_files
Test: `m dist` with aosp_walleye-userdebug
Change-Id: Id58c5ae780ac8a22661ffea629144d4836839175
2018-07-22 22:36:27 -07:00
Tao Bao
708ca5f60b Merge "releasetools: Track the change to applypatch arguments."
am: 80810b9d36

Change-Id: I9df643323e5624596b5981e9a4d48fdafb1db2f3
2018-07-20 17:12:39 -07:00
Yifan Hong
9c35a022e1 build_image.py recognize BOARD_PRODUCT_SERVICESIMAGE_PARTITION_RESERVED_SIZE
Test: builds
Bug: 79106666
Change-Id: I00c1a96c09cf7bd9274b65ac0afc47462686171d
2018-07-20 16:51:26 -07:00
Yifan Hong
56a6c3bb34 build_image.py recognize BOARD_PRODUCTIMAGE_PARTITION_RESERVED_SIZE
Test: builds
Bug: 79106666
Change-Id: I26aba694fd3d6a2feed7601524dde52569ba41e2
2018-07-20 16:51:26 -07:00
Tao Bao
8bfd3c7483 releasetools: Rename the ramdisk_dir in prop dict to root_dir.
This property indicates the path to the root directory (i.e.
TARGET_ROOT_OUT at build time, or ROOT/ in a target_files.zip).
build_image.py will only use this property when building a target using
system_root_image. Under such a configuration, files under this dir
never land into a ramdisk, but as part of system.img. This CL renames
the property name to avoid confusion.

Note that with this change releasetools keeps working with older
target_files zips that contain 'ramdisk_dir'. Because common.py doesn't
read the value of that property, but sets it to ROOT/ directly.

Test: `m dist` on aosp_marlin-userdebug.
Test: `m dist` on aosp_angler-userdebug.
Test: Run add_img_to_target_files.py with a new target_files.zip and an
      old target_files.zip respectively.
Change-Id: Ie9e19dd17d8c2abb4c4a97449e436774d5dd2a36
2018-07-20 16:05:36 -07:00
Tao Bao
80810b9d36 Merge "releasetools: Track the change to applypatch arguments." 2018-07-20 23:04:33 +00:00
Dario Freni
5f9ddc95a2 Merge "Add support for /product-services partition"
am: 3cf03b775a

Change-Id: I1b0b3628d1b96967e569093b93b12fbc5acd7ad5
2018-07-20 02:09:37 -07:00
Dario Freni
3cf03b775a Merge "Add support for /product-services partition" 2018-07-20 09:00:25 +00:00
Tao Bao
b1750e2825 Merge "releasetools: Remove the assertion of not allowing thumbprint in Treble world."
am: 805ab25e05

Change-Id: Iee41170414e8b17f05d2defa405b18834bc7e8f3
2018-07-19 15:05:39 -07:00
Tao Bao
76def243ec releasetools: Make scripts pylint clean.
Clean up the following scripts.
  blockimgdiff.py
  common.py
  edify_generator.py
  img_from_target_files.py
  ota_from_target_files.py

Seems we don't have a way to force pylint-ing the scripts using
per-directory pylintrc files (and we don't have pylint tool in AOSP
repo), per
https://android.googlesource.com/platform/tools/repohooks/#todo_limitations.

Test: `m dist`
Test: pylint --rcfile=pylintrc <script.py>
Change-Id: Ia6fd1ddc86f4d84c68e500f225d4a89d0fea8ec7
2018-07-19 10:39:05 -07:00
Tao Bao
4223012099 releasetools: Remove the assertion of not allowing thumbprint in Treble world.
The claim in the original CL doesn't seem to hold. We may still have
devices using both of thumbprint and Treble compatibility check.

Bug: 111640956
Test; python -m unittest test_ota_from_target_files
Test: Build an incremental package for a target using thumbprints.
Change-Id: Ia491d5279901c2385d1c30802abcb99d65d919cc
2018-07-19 10:16:45 -07:00
Dario Freni
5f681e1dd7 Add support for /product-services partition
This CL is largely an adaptation of Change-Id
I774e6a38003734421591e51bed103802ff84f432

It adds the following variables:

- BOARD_AVB_PRODUCT_SERVICES_KEY_PATH
- BOARD_AVB_PRODUCT_SERVICES_ALGORITHM
- BOARD_AVB_PRODUCT_SERVICES_ROLLBACK_INDEX_LOCATION
- BOARD_PRODUCT_SERVICESIMAGE_FILE_SYSTEM_TYPE
- BOARD_PRODUCT_SERVICESIMAGE_EXTFS_INODE_COUNT
- BOARD_PRODUCT_SERVICESIMAGE_EXTFS_RSV_PCT
- BOARD_PRODUCT_SERVICESIMAGE_PARTITION_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_JOURNAL_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_BLOCK_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_COMPRESSOR
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_COMPRESSOR_OPT
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_DISABLE_4K_ALIGN
- BOARD_PREBUILT_PRODUCT_SERVICESIMAGE
- BOARD_USES_PRODUCT_SERVICESIMAGE
- LOCAL_PRODUCT_SERVICES_MODULE
- PRODUCT_PRODUCT_SERVICES_BASE_FS_PATH
- PRODUCT_PRODUCT_SERVICES_VERITY_PARTITION
- PRODUCT_PRODUCT_SERVICES_PROPERTIES
- TARGET_COPY_OUT_PRODUCT_SERVICES
- TARGET_OUT_PRODUCT_SERVICES
- TARGET_OUT_PRODUCT_SERVICES_*

Bug: 80741439
Test: Successfully built product-services.img with one module in it, and flashed
on device. Also successfully built image with /system/product-services directory
and no /product-services partition.
Change-Id: I5d229f6ac729ea6df9ff1f14cee2e28972cd9b4d
2018-07-19 11:15:11 +01:00
Tao Bao
4948aed1a3 releasetools: Track the change to applypatch arguments.
This writes /system/bin/install-recovery.sh with the updated argument
formats. Requires matching changes to bootable/recovery/applypatch.

Bug: 110106408
Test: python -m unittest test_common.InstallRecoveryScriptFormatTest
Test: See additional tests in the other CL.
Change-Id: I2dec6f064819c01b5bb34c3d6fd0e247a5f48f08
2018-07-17 17:58:14 -07:00
Tao Bao
40e1bb6df9 Merge "releasetools: Fix secondary payload generation."
am: c26dea96a2

Change-Id: I64322df5111736b9276103fb71c36e9ec4a9e1d3
2018-07-12 21:13:29 -07:00
Tao Bao
1248980a8b releasetools: Fix secondary payload generation.
The change in [1] removed duplicating radio images from RADIO/ to
IMAGES/. When getting the target_files zip for generating secondary
payload, we need to additionally copy the files under RADIO/ over.

[1] commit 5277d1015f

Test: python -m unittest test_ota_from_target_files
Test: Generate an OTA with secondary payload (--include_secondary).
Change-Id: I096f1642a905fb764e63f5df2edc1396aa6befd8
2018-07-12 14:52:52 -07:00
Tao Bao
ff4434f433 Merge "releasetools: Remove RemoveBackwardEdges()."
am: d4b9e01f39

Change-Id: If887e4f4e453f1ec2b199ac249d844e753129803
2018-07-11 12:13:04 -07:00
Tao Bao
d4b9e01f39 Merge "releasetools: Remove RemoveBackwardEdges()." 2018-07-11 19:00:32 +00:00
Tao Bao
b047778ebd Merge "releasetools: Address two issues with system_root_image file map."
am: 81421ccf38

Change-Id: I171a9a7b8a4b0e1051e8752f519ca5d81e3b0f4f
2018-07-11 10:14:56 -07:00
Tao Bao
5bab0dd1c4 releasetools: Remove RemoveBackwardEdges().
RemoveBackwardEdges() was used only in BBOTA v1 (which has been
deprecated since O). v2+ calls ReverseBackwardEdges() and
ImproveVertexSequence().

Also remove the imgdiff tag of 'trimmed' that would be set through this
function() only.

Test: python -m unittest test_blockimgdiff
Test: Build an incremental non-A/B OTA.
Test: Code search shows no active user.
Change-Id: I3b58ae048a1fbc283269e70fdfa29eb8d184ede7
2018-07-10 22:13:40 -07:00