Commit graph

5712 commits

Author SHA1 Message Date
Tao Bao
f013dc3fed Merge "releasetools: Clean up AppendVBMetaArgsForPartition()."
am: 540a71d2b3

Change-Id: If13ee90c84f75c2452c1e1e8504928414a971ed4
2018-07-23 14:56:22 -07:00
Tao Bao
540a71d2b3 Merge "releasetools: Clean up AppendVBMetaArgsForPartition()." 2018-07-23 21:40:37 +00:00
Yifan Hong
fe421ed263 Merge changes from topic "board_partition_reserved_size"
am: 2d90c246a9

Change-Id: I460ea917b47a61acb0cbb0f17a867d15f61c7d6d
2018-07-23 11:58:52 -07:00
Yifan Hong
2d90c246a9 Merge changes from topic "board_partition_reserved_size"
* changes:
  build_image.py recognize BOARD_PRODUCT_SERVICESIMAGE_PARTITION_RESERVED_SIZE
  build_image.py recognize BOARD_PRODUCTIMAGE_PARTITION_RESERVED_SIZE
2018-07-23 18:38:24 +00:00
Tao Bao
f87ea90d76 Merge "releasetools: Rename the ramdisk_dir in prop dict to root_dir."
am: d094713952

Change-Id: I013c82846a143c5eac050325e66092afc66c958f
2018-07-23 09:41:25 -07:00
Tao Bao
d094713952 Merge "releasetools: Rename the ramdisk_dir in prop dict to root_dir." 2018-07-23 16:33:46 +00:00
Tao Bao
1a3d83d667 Merge "releasetools: Make scripts pylint clean."
am: 23a4070483

Change-Id: I4f4b35565f7d154dd47e4b5b8444e76c6c7227c6
2018-07-23 07:22:59 -07:00
Tao Bao
23a4070483 Merge "releasetools: Make scripts pylint clean." 2018-07-23 14:11:37 +00:00
Tao Bao
3e53ef7b1c releasetools: Clean up AppendVBMetaArgsForPartition().
Drop the parameter of public_key_dir. It's not necessary to stage all
the extracted public keys into the same temporary dir.

Redirect the stderr output to STDOUT while calling avbtool, so that it
can dump the error message accordingly.

Also remove the check for having empty img_path arg. It doesn't look
like a clean logic to allow the function to be called with invalid /
empty arg. The only caller (i.e. AddVBMeta) already asserts img_path
being valid as well.

Test: python -m unittest test_add_img_to_target_files
Test: `m dist` with aosp_walleye-userdebug
Change-Id: Id58c5ae780ac8a22661ffea629144d4836839175
2018-07-22 22:36:27 -07:00
Chih-Hung Hsieh
d83446d789 Merge "Escape special HTML characters in warning messages."
am: a14c131154

Change-Id: Ia8a2afa3d765d7a07a5c7c732af9f66912c1483c
2018-07-20 17:24:14 -07:00
Treehugger Robot
a14c131154 Merge "Escape special HTML characters in warning messages." 2018-07-21 00:20:02 +00:00
Tao Bao
708ca5f60b Merge "releasetools: Track the change to applypatch arguments."
am: 80810b9d36

Change-Id: I9df643323e5624596b5981e9a4d48fdafb1db2f3
2018-07-20 17:12:39 -07:00
Yifan Hong
9c35a022e1 build_image.py recognize BOARD_PRODUCT_SERVICESIMAGE_PARTITION_RESERVED_SIZE
Test: builds
Bug: 79106666
Change-Id: I00c1a96c09cf7bd9274b65ac0afc47462686171d
2018-07-20 16:51:26 -07:00
Yifan Hong
56a6c3bb34 build_image.py recognize BOARD_PRODUCTIMAGE_PARTITION_RESERVED_SIZE
Test: builds
Bug: 79106666
Change-Id: I26aba694fd3d6a2feed7601524dde52569ba41e2
2018-07-20 16:51:26 -07:00
Tao Bao
8bfd3c7483 releasetools: Rename the ramdisk_dir in prop dict to root_dir.
This property indicates the path to the root directory (i.e.
TARGET_ROOT_OUT at build time, or ROOT/ in a target_files.zip).
build_image.py will only use this property when building a target using
system_root_image. Under such a configuration, files under this dir
never land into a ramdisk, but as part of system.img. This CL renames
the property name to avoid confusion.

Note that with this change releasetools keeps working with older
target_files zips that contain 'ramdisk_dir'. Because common.py doesn't
read the value of that property, but sets it to ROOT/ directly.

Test: `m dist` on aosp_marlin-userdebug.
Test: `m dist` on aosp_angler-userdebug.
Test: Run add_img_to_target_files.py with a new target_files.zip and an
      old target_files.zip respectively.
Change-Id: Ie9e19dd17d8c2abb4c4a97449e436774d5dd2a36
2018-07-20 16:05:36 -07:00
Tao Bao
80810b9d36 Merge "releasetools: Track the change to applypatch arguments." 2018-07-20 23:04:33 +00:00
Chih-Hung Hsieh
b2afb63ac9 Escape special HTML characters in warning messages.
Bug: 111699078
Test: run warn.py with a build.log file
Change-Id: I553dad57a6d81ea6cc89289741d9963f49cc3b47
2018-07-20 15:36:26 -07:00
Dario Freni
5f9ddc95a2 Merge "Add support for /product-services partition"
am: 3cf03b775a

Change-Id: I1b0b3628d1b96967e569093b93b12fbc5acd7ad5
2018-07-20 02:09:37 -07:00
Dario Freni
3cf03b775a Merge "Add support for /product-services partition" 2018-07-20 09:00:25 +00:00
Tao Bao
b1750e2825 Merge "releasetools: Remove the assertion of not allowing thumbprint in Treble world."
am: 805ab25e05

Change-Id: Iee41170414e8b17f05d2defa405b18834bc7e8f3
2018-07-19 15:05:39 -07:00
Tao Bao
76def243ec releasetools: Make scripts pylint clean.
Clean up the following scripts.
  blockimgdiff.py
  common.py
  edify_generator.py
  img_from_target_files.py
  ota_from_target_files.py

Seems we don't have a way to force pylint-ing the scripts using
per-directory pylintrc files (and we don't have pylint tool in AOSP
repo), per
https://android.googlesource.com/platform/tools/repohooks/#todo_limitations.

Test: `m dist`
Test: pylint --rcfile=pylintrc <script.py>
Change-Id: Ia6fd1ddc86f4d84c68e500f225d4a89d0fea8ec7
2018-07-19 10:39:05 -07:00
Tao Bao
4223012099 releasetools: Remove the assertion of not allowing thumbprint in Treble world.
The claim in the original CL doesn't seem to hold. We may still have
devices using both of thumbprint and Treble compatibility check.

Bug: 111640956
Test; python -m unittest test_ota_from_target_files
Test: Build an incremental package for a target using thumbprints.
Change-Id: Ia491d5279901c2385d1c30802abcb99d65d919cc
2018-07-19 10:16:45 -07:00
Dario Freni
5f681e1dd7 Add support for /product-services partition
This CL is largely an adaptation of Change-Id
I774e6a38003734421591e51bed103802ff84f432

It adds the following variables:

- BOARD_AVB_PRODUCT_SERVICES_KEY_PATH
- BOARD_AVB_PRODUCT_SERVICES_ALGORITHM
- BOARD_AVB_PRODUCT_SERVICES_ROLLBACK_INDEX_LOCATION
- BOARD_PRODUCT_SERVICESIMAGE_FILE_SYSTEM_TYPE
- BOARD_PRODUCT_SERVICESIMAGE_EXTFS_INODE_COUNT
- BOARD_PRODUCT_SERVICESIMAGE_EXTFS_RSV_PCT
- BOARD_PRODUCT_SERVICESIMAGE_PARTITION_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_JOURNAL_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_BLOCK_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_COMPRESSOR
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_COMPRESSOR_OPT
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_DISABLE_4K_ALIGN
- BOARD_PREBUILT_PRODUCT_SERVICESIMAGE
- BOARD_USES_PRODUCT_SERVICESIMAGE
- LOCAL_PRODUCT_SERVICES_MODULE
- PRODUCT_PRODUCT_SERVICES_BASE_FS_PATH
- PRODUCT_PRODUCT_SERVICES_VERITY_PARTITION
- PRODUCT_PRODUCT_SERVICES_PROPERTIES
- TARGET_COPY_OUT_PRODUCT_SERVICES
- TARGET_OUT_PRODUCT_SERVICES
- TARGET_OUT_PRODUCT_SERVICES_*

Bug: 80741439
Test: Successfully built product-services.img with one module in it, and flashed
on device. Also successfully built image with /system/product-services directory
and no /product-services partition.
Change-Id: I5d229f6ac729ea6df9ff1f14cee2e28972cd9b4d
2018-07-19 11:15:11 +01:00
Tao Bao
4948aed1a3 releasetools: Track the change to applypatch arguments.
This writes /system/bin/install-recovery.sh with the updated argument
formats. Requires matching changes to bootable/recovery/applypatch.

Bug: 110106408
Test: python -m unittest test_common.InstallRecoveryScriptFormatTest
Test: See additional tests in the other CL.
Change-Id: I2dec6f064819c01b5bb34c3d6fd0e247a5f48f08
2018-07-17 17:58:14 -07:00
Tao Bao
40e1bb6df9 Merge "releasetools: Fix secondary payload generation."
am: c26dea96a2

Change-Id: I64322df5111736b9276103fb71c36e9ec4a9e1d3
2018-07-12 21:13:29 -07:00
Tao Bao
1248980a8b releasetools: Fix secondary payload generation.
The change in [1] removed duplicating radio images from RADIO/ to
IMAGES/. When getting the target_files zip for generating secondary
payload, we need to additionally copy the files under RADIO/ over.

[1] commit 5277d1015f

Test: python -m unittest test_ota_from_target_files
Test: Generate an OTA with secondary payload (--include_secondary).
Change-Id: I096f1642a905fb764e63f5df2edc1396aa6befd8
2018-07-12 14:52:52 -07:00
Tao Bao
ff4434f433 Merge "releasetools: Remove RemoveBackwardEdges()."
am: d4b9e01f39

Change-Id: If887e4f4e453f1ec2b199ac249d844e753129803
2018-07-11 12:13:04 -07:00
Tao Bao
d4b9e01f39 Merge "releasetools: Remove RemoveBackwardEdges()." 2018-07-11 19:00:32 +00:00
Tao Bao
b047778ebd Merge "releasetools: Address two issues with system_root_image file map."
am: 81421ccf38

Change-Id: I171a9a7b8a4b0e1051e8752f519ca5d81e3b0f4f
2018-07-11 10:14:56 -07:00
Tao Bao
5bab0dd1c4 releasetools: Remove RemoveBackwardEdges().
RemoveBackwardEdges() was used only in BBOTA v1 (which has been
deprecated since O). v2+ calls ReverseBackwardEdges() and
ImproveVertexSequence().

Also remove the imgdiff tag of 'trimmed' that would be set through this
function() only.

Test: python -m unittest test_blockimgdiff
Test: Build an incremental non-A/B OTA.
Test: Code search shows no active user.
Change-Id: I3b58ae048a1fbc283269e70fdfa29eb8d184ede7
2018-07-10 22:13:40 -07:00
Tao Bao
d3554e628f releasetools: Address two issues with system_root_image file map.
With system_root_image, e2fsdroid writes file map with extra leading
slashes in filenames (e.g. "//system/framework/am.jar"). This breaks the
detection of files with incomplete ranges, and thus fails the patch
generation. This CL addresses the issue by stripping out leading
slashes.

Additionally, non-/system files (e.g "//sbin/charger") are not packed
under SYSTEM/ in a target_files.zip, despite being part of system.img.
We need to look for these files under ROOT/ instead.

This CL also asserts the availability of all files listed on a file map,
to avoid silently missing other edge cases.

Bug: 80380658
Test: python -m unittest test_common
Test: Successfully generated an incremental for a target using
      system_root_image that was previously failing.
Change-Id: I62a2460e882f3930e99add4d2b44291edf7a51a0
2018-07-10 16:05:14 -07:00
Tao Bao
27bc4a9b1d Merge "releasetools: Remove three obsolete functions from EdifyGenerator."
am: e9856a835f

Change-Id: Iaf358557ae4ac4e154894868701c84f350ae7a6e
2018-07-06 16:30:56 -07:00
Tao Bao
a24f711bf3 releasetools: Remove three obsolete functions from EdifyGenerator.
Namely MakeTemporary, Verify and FileCheck. They were once used for
file-based OTA that has been deprecated.

Test: Check that there's no active user, including device-specific
      releasetools script.
Change-Id: Ic73054d73271ad91e860b34b2ba156361c58bc1a
2018-07-06 14:21:26 -07:00
Tao Bao
a85cb404cd Merge "releasetools: Add dict-like setter to BuildInfo."
am: ee36b42beb

Change-Id: I97039b14a1664d5cddd2cf44de600686bbf3f26d
2018-07-06 14:05:23 -07:00
Tao Bao
667c7534ed releasetools: Add dict-like setter to BuildInfo.
BuildInfo already has dict-like getters. This CL adds the matching
setter method to keep it backward compatible with device-specific
scripts that expect a dict.

It also adds items() method that allows iterating the items.

Bug: 111087332
Test: python -m unittest test_ota_from_target_files.BuildInfoTest
Test: Generate an incremental OTA that sets info_dict in device-specific
      releasetools script.
Change-Id: Idd033f98a9186740f9da1a300d4c2bdddd9c1345
2018-07-06 10:15:13 -07:00
Anton Hansson
04750bc03a Merge "Split up fs_config_* modules per partition."
am: 4e0ccc4c8a

Change-Id: I82751b7f53a22bdcccb6667dfb17089d296dd43c
2018-06-27 06:07:11 -07:00
Anton Hansson
e85ec84eef Split up fs_config_* modules per partition.
This was already done for partitions other than the system partition.
Add new, separate "_system" targets, and add new phony modules which
collects _system and _nonsystem targets for now.

Update the base_system.mk file to only depend on the system module, and
put the other ones in base_vendor.mk for now.

Bug: 80410283
Test: make fs_config_dirs[_*] fs_config_files[_*] and verify output
Change-Id: I7843578dbb9f27edf4c1e5b4a795a530f79e0be1
2018-06-27 11:15:59 +01:00
Yifan Hong
1dce9b6943 Merge "build_image.py: remove extra new lines in error message"
am: 674454f223

Change-Id: Iaa6edf6b28c5d2e5cdbc55068ab2631c17138288
2018-06-22 22:06:16 -07:00
Yifan Hong
92fb231105 build_image.py: remove extra new lines in error message
Test: m dist
Change-Id: I7908e4eb0a9bbcb187e1fa6d15f606537d69864d
2018-06-22 15:05:46 -07:00
Yifan Hong
bb42f7d655 Merge changes from topic "board_partition_reserved_size"
am: 3967e0eb99

Change-Id: I767967f1086ec303c63199d0e38ec02dedf09152
2018-06-22 13:25:18 -07:00
Treehugger Robot
3967e0eb99 Merge changes from topic "board_partition_reserved_size"
* changes:
  build_image.py recognize BOARD_VENDORIMAGE_PARTITION_RESERVED_SIZE
  build_image.py recognize BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE
2018-06-22 20:10:47 +00:00
Yifan Hong
749062d379 build_image.py recognize BOARD_VENDORIMAGE_PARTITION_RESERVED_SIZE
Test: `make vendorimage`

Test: `make vendorimage` with the following:
    - install a large file to vendor image
    fails as expected (because _PARTITION_SIZE is exceeded)

Test: `make vendorimage` with the following:
    - set PRODUCT_USE_LOGICAL_PARTITIONS to true
    - set BOARD_VENDORIMAGE_PARTITION_RESERVED_SIZE
    fails as expected (BOARD_VENDORIMAGE_PARTITION_SIZE needs
    to be undefined)

Test: `make vendorimage` with the following:
    - install a large file to vendor image
    - set PRODUCT_USE_LOGICAL_PARTIIONS to true
    - add a small BOARD_VENDORIMAGE_PARTITION_RESERVED_SIZE
    - remove BOARD_VENDORIMAGE_PARTITION_SIZE
    build succeeds.

Bug: 79106666

Change-Id: Ica8fdce64e8f37d91e66e3d49c0c74fadd15a832
2018-06-22 10:25:54 -07:00
Yifan Hong
bbcba1e353 build_image.py recognize BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE
- Copy "use_logical_partitions" to *_image_info.txt before sending
  it to build_image.py, so that the script can use this variable.

- build_image.py emits an additional properties file to inform
  the build system about the system image size.

Test: `make systemimage`

Test: `make systemimage` with the following:
    - install a large file to system image
    fails as expected (because _PARTITION_SIZE is exceeded)

Test: `make systemimage` with the following:
    - set PRODUCT_USE_LOGICAL_PARTITIONS to true
    - set BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE
    fails as expected (BOARD_SYSTEMIMAGE_PARTITION_SIZE needs
    to be undefined)

Test: `make systemimage` with the following:
    - install a large file to system image
    - set PRODUCT_USE_LOGICAL_PARTIIONS to true
    - add a small BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE
    - remove BOARD_SYSTEMIMAGE_PARTITION_SIZE
    build succeeds.

Test: same for systemotherimage

Bug: 79106666

Change-Id: I574062882acd1ecd633ac38c5a8c5351b90a32d8
2018-06-22 10:25:54 -07:00
Tao Bao
8012498f79 Merge "releasetools: Skip signing APKs with given prefixes."
am: 8acf3c302b

Change-Id: Ie8f30d2c4e95c047653ca7120c62571df9f14842
2018-06-20 16:36:02 -07:00
Tao Bao
93c2a01268 releasetools: Skip signing APKs with given prefixes.
We may pack prebuilts that end with ".apk" into target_files zip, via
PRODUCT_COPY_FILES. META/apkcerts.txt won't contain the cert info for
such files, and we want to keep them as is while signing, despite of the
".apk" extension.

This CL adds "--skip_apks_with_path_prefix" option to
sign_target_files_apks.py. APKs with matching prefixes will be copied
verbatim into the signed images. The prefix should match the entry names
in the target_files (e.g. "SYSTEM_OTHER/preloads/"). The option may be
repeated to specify multiple prefixes.

Note that although we may skip signing an APK file with "-e ApkName=".
This would skip *all* the APK files with the matching basename.
"--skip_apks_with_path_prefix" allows matching the exact prefix.

For example:
$ ./build/make/tools/releasetools/sign_target_files_apks.py     \
    --skip_apks_with_path_prefix SYSTEM_OTHER/preloads/         \
    --skip_apks_with_path_prefix PRODUCT/prebuilts/PrebuiltApp1 \
    --skip_apks_with_path_prefix VENDOR/app/PrebuiltApp2.apk    \
    target_files.zip                                            \
    signed-target_files.zip

Bug: 110201128
Test: Run the command above and check the logs.
Test: `python -m unittest test_sign_target_files_apks`
Change-Id: I7bd80b360917cef137cf1e7e8cfa796968831f47
2018-06-20 13:49:57 -07:00
Tao Bao
023b5104d7 Merge "releasetools: Factor out the check for (compressed) APK file."
am: 2ba547a1c7

Change-Id: I0ae458b3c0d8c0335c8175138185ce714d065471
2018-06-20 09:18:56 -07:00
Tao Bao
11f955c506 releasetools: Factor out the check for (compressed) APK file.
Test: Run sign_target_files.py to sign a target_files.zip.
Test: `python -m unittest test_sign_target_files_apks`
Change-Id: Ie795d1bce7bae6af427832283e3d10bfecad80c5
2018-06-19 14:50:23 -07:00
Tom Cherry
814c6d2d46 Merge "Always create /vendor/etc/{passwd,group}"
am: b6b2d49e23

Change-Id: Id6b1002ded8d3e9f199dfef9138d9d076d43cd57
2018-06-15 10:15:11 -07:00
Tom Cherry
b6b2d49e23 Merge "Always create /vendor/etc/{passwd,group}" 2018-06-15 17:04:16 +00:00
Tom Cherry
53940fd2ba Always create /vendor/etc/{passwd,group}
Previously we only created these if the device set
TARGET_FS_CONFIG_GEN, however there are now other targets that want to
depend on these.  Instead of having those targets conditionally depend
on them, we always create them, defaulting to blank contents (by
reading /dev/null for TARGET_FS_CONFIG_GEN).

Test: builds succeed
Change-Id: Ie95286f5a800d891022eb66cd6fefcc967000c2e
2018-06-14 13:29:32 -07:00