Commit graph

2043 commits

Author SHA1 Message Date
David Anderson
117b52f74e releasetools: Pass --virtual-ab to lpmake.
Note that we are currently excluding this flag for retrofit devices,
since it necessitates support for a newer super.img format, which
retrofit devices may not fully support.

Bug: 134949511
Test: m -j, lpdump super_empty.img
Change-Id: Id69a8d755b9f4c4bb5fbd456a416ced1a5f1d135
2019-12-16 22:17:28 +00:00
Treehugger Robot
d6aa90ce9a Merge "sign_target_apks: Add networkstack to default key_map" 2019-12-13 22:22:08 +00:00
Dan Austin
5290364e9b Add AFTL relevant options to OPTIONS in common.py.
Adds AFTL relevant options in preparation for avbtool changes. The
options are:
aftl_server: A string representing the URL of the transparency log.
aftl_key_path: The path to the public key associated with the
transparency log (in PEM format).
aftl_manufacturer_key_path: The path to the key associated with the
manufacturer for AFTL message signing.
aftl_signing_helper: The signing helper for use with AFTL message
signing.

Change-Id: I73d466f24b171c16dbdaa40c589779bed52be551
Test: Ran add_img_to_target_files with all AFTL options.
Bug: 138779249
2019-12-12 15:48:52 -08:00
David Anderson
0ec64ac0aa releasetools: Generalize base_fs_file handling.
Share the code to handle base_fs files rather than duplicate it for
system/vendor. At the same time, add support for system_ext and product
partitions.

Bug: 139201772
Test: manual test
Change-Id: Ib1b7c018c828901e24eefc01e3faead03e77bc2d
2019-12-11 22:32:00 +00:00
Oleh Cherpak
982e608085 sign_target_apks: Add networkstack to default key_map
Test: run cts -m CtsSecurityTestCases -t \
	android.security.cts.PackageSignatureTest#testPackageSignatures
Bug: 145955635
Change-Id: I1a1498562e2b5983010cb98e3edcd03ceb2cce19
Signed-off-by: Oleh Cherpak <oleh.cherpak@globallogic.com>
2019-12-10 15:03:57 +00:00
David Anderson
c55b2f516c Merge "build_image: Do not use base_fs file for system_other." 2019-12-04 21:28:53 +00:00
David Anderson
3269f25476 build_image: Do not use base_fs file for system_other.
system_other is not OTA'd (outside of factory OTA), and its block map differs greatly from
system.img, so do not attempt to re-use it.

Bug: 139201772
Test: make with base_fs file for system
Change-Id: I1ed2091da381ebdb93522a081e5d2f7c18c9d069
2019-12-04 21:28:42 +00:00
Baligh Uddin
ac936fd573 Fix syntax error for SignApexFile
BUG: 145662097
TEST: Local smoke test which invokes sign_apex for local apex files.
2019-12-04 08:30:32 -08:00
Alistair Delva
1cfe41269f Merge "Allow f2fs sparse mode to be disabled" 2019-12-02 21:54:20 +00:00
Treehugger Robot
e7c1f6314c Merge "Removes custom prefix/suffix from MergeDynamicPartitionInfoDicts." 2019-11-27 20:30:34 +00:00
Daniel Norman
55417148f9 Removes custom prefix/suffix from MergeDynamicPartitionInfoDicts.
All callers of this function now always pass the same values, so this
change hardcodes those values within the function body.

Fix: 145008064
Test: python -m unittest test_common
Test: build & boot a merged target that uses DAP

Change-Id: I0051c5ba507983231825edfcaf349e574efa451a
2019-11-26 11:09:17 -08:00
Yifan Hong
874a5aa95e Merge "Move check partition size logs to a file" 2019-11-26 00:32:52 +00:00
Steven Laver
615385ee8b releasetools: merge_target_files.py should handle new contents of dynamic_partitions_info.txt
With a recent change (cb8e2755ff) to the
generation of dynamic_partitions_info.txt,
process_dynamic_partitions_info_txt() needs a corresponding update.

Change-Id: I30ec92176fd9cd737160b579214eb5b3d6a44c10
2019-11-22 09:34:27 -08:00
Donghoon Yu
92420db7dd releasetools: fix avbtool arguments.
The following error occurred when building dist.
Separate these arguments

error log:
avbtool: error: unrecognized arguments: --partition_name vendor_boot

Bug: 144820089

Change-Id: I3891ef3bd9a2f7d2cd1377228b9894b599dc1cc3
Signed-off-by: Donghoon Yu <hoony.yu@samsung.com>
2019-11-21 06:15:02 +00:00
Alistair Delva
91238cce14 Allow f2fs sparse mode to be disabled
Bug: 142424832
Bug: 142803849
Change-Id: Ic112128f15e3f29208332afceea4fbc3fca40b36
2019-11-15 19:05:41 +00:00
Treehugger Robot
e24ef86473 Merge "releasetools: build_super_image.py should unpack */build.prop" 2019-11-15 04:34:02 +00:00
Steven Laver
f43fa9dd99 releasetools: build_super_image.py should unpack */build.prop
build_super_image.py attempts to retrieve the "ro.product.device"
property, which requires parsing the */build.prop files from the target
files package. This fixes the following error:
"ExternalError: couldn't resolve ro.product.device"

Test: ./otatools/releasetools/build_super_image.py -p ./otatools
aosp_arm64-target_files.zip super.img

Change-Id: I592524a27b1a4e7544644b1a344071cff9718942
2019-11-14 23:47:31 +00:00
Yifan Hong
309109351e Move check partition size logs to a file
Don't contaminate the build log.
Test: m check-all-partition-sizes -j (verbose logs stored to file)
Test: m check-all-partition-sizes-nodeps -j (see verbose logs)
Test: manually run the script with[out] -v and with[out] --logfile

Change-Id: I345a340deab3e29bb9cb05d4970a55d8758607a7
2019-11-05 14:06:27 -08:00
Yifan Hong
96527f375f Fix Virtual A/B size checks
Test: build
Test: test_check_partition_sizes
Bug: 143111912
Change-Id: I4e056c25948e4169a0b5b098168141e27c31a0d4
2019-11-05 14:06:23 -08:00
Treehugger Robot
f5ad6e5443 Merge "Support Factory OTA for Virtual A/B devices" 2019-11-05 18:47:01 +00:00
Yifan Hong
fe07343fed Support Factory OTA for Virtual A/B devices
In Factory OTA, secondary payload should be generated without
virtual_ab flag set, so that update package generator generates
a regular A/B OTA (with snapshot_enabled set to false), and
OTA client writes system_b without snapshots.

Test: apply secondary OTA
Fixes: 138733621
Change-Id: I05d534a12d4a7891ce5597f3c29e55f5185e5a8d
2019-11-04 15:14:26 -08:00
Yifan Hong
3a7c2ef7cc Reland "Move partition size checks to python script"
This reverts commit 0141e45b96.

Reason for revert: Fixed in follow-up CL
Test: forrest
Bug: 143734706

Change-Id: I007acf228d4fb4d6a16ae9089e3f04cf33a567bb
2019-11-01 11:41:26 -07:00
Andrei-Valentin Onea
15e75e8807 Merge "Revert "Move partition size checks to python script"" 2019-11-01 15:36:10 +00:00
Raff Tsai
0141e45b96 Revert "Move partition size checks to python script"
This reverts commit a4906cd306.

Reason for revert: b/143734706

Change-Id: I905bd9c0bfcbd123c8f9e78986bfcb5600a9eca9
2019-11-01 05:36:20 +00:00
Yifan Hong
8244f27722 Merge "Move partition size checks to python script" 2019-11-01 01:52:11 +00:00
Yifan Hong
a4906cd306 Move partition size checks to python script
Easier to read and modify its logic.

Test: builds
Test: test_check_partition_sizes
Test: atest releasetools_test
Test: atest releasetools_py3_test
Change-Id: I73204664b64a0bcee7375fa6fca46758d5a2bb72
2019-10-31 11:22:20 -07:00
Tianjie Xu
526b8563a5 Merge "Stop creating update_engine_payload_key" 2019-10-17 20:59:51 +00:00
Treehugger Robot
323a460d5b Merge "releasetools: Fix an issue in common.GetAvbPartitionArg." 2019-10-16 20:52:19 +00:00
Tianjie Xu
2df23d7f41 Stop creating update_engine_payload_key
Now the update_engine is able to read public keysfrom otacerts directly.
So the update_engine_payload_key is no longer needed.

Also remove the key replace in sign_target_files_apks.py. So we should
not use the new script to sign the old target files.

Bug: 116660991
Test: build the system image, unit tests pass
Change-Id: I9dae1f8b397f2b5efafed66a8faac1cb9087c741
2019-10-16 18:27:22 +00:00
Tianjie Xu
3654beb83d Merge "Call delta_generator to get the signature size" 2019-10-15 16:49:15 +00:00
Tao Bao
3612c88ed7 releasetools: Fix an issue in common.GetAvbPartitionArg.
It's a bug introduced in commit 1aeef725a7, which affects unittest only.

Bug: 130351427
Test: Use a lunch'd target. Run
      `atest --host releasetools_test releasetools_py3_test`.
Change-Id: I7ff01a6af47d002e1203bd376d477b60d769cbd1
2019-10-14 17:53:21 -07:00
Tao Bao
21fb765515 Merge "releasetools: Set the search path based on the current executable." 2019-10-14 22:20:02 +00:00
Tao Bao
dc2f0b0abd Merge "Include per-partition fingerprint as AVB prop." 2019-10-14 14:53:46 +00:00
Tianjie Xu
21e6deb647 Call delta_generator to get the signature size
The signature size is needed during payload hashing and signing.
We used to sign the update_engine's payload with RSA keys only. In
this case, the signature size always equals the key size. But the
assumption is no longer true for EC keys, whose DER-encoded signature
size is a variant with a maximum size.

Therefore, we always give the maximum signature size to the delta
generator, who then add paddings to the real signature if necessary.
The maximum signature size is calculated by calling the delta_generator
with the new option '--calculate_signature_size'. For custom payload
signers, we also deprecate the '--payload_signer_key_size' and replace
it with '--payload_signer_maximum_signature_size'.

The EC key in the test is generated with:
The EC key in the unittest is generated with the command:
openssl ecparam -name prime256v1 -genkey -noout -out prime256v1-key.pem
openssl pkey -in prime256v1-key.pem -out testkey_EC.key

Bug: 141244025
Test: sign and verify a payload
Change-Id: Ife6e269d8aa3d870405aca20086330f1795e167f
2019-10-12 00:36:54 +00:00
Tao Bao
afd92a8689 releasetools: Set the search path based on the current executable.
We used to mimic the behavior of build system, to find the default
search path based on OUT_DIR_COMMON_BASE or OUT_DIR. These variables
should be internal to build system.

Since we've switched releasetools script to hermetic Python executables
(e.g. `m -j ota_from_target_files`, then run the binary at
`out/host/linux-x86/bin/ota_from_target_files`), we can set the search
path in relative to the path of the current executable.

Bug: 133126366
Test: TreeHugger
Test: 1. Build aosp_x86, by "lunch aosp_x86; m -j"
      2. Inject errors to the executables under out/host/linux-x86/bin,
         e.g.  to `lpmake`.
      3. Set up OUT_DIR (e.g., to /tmp/out) and build the same product
         again by "export OUT_DIR=/tmp/out; lunch aosp_x86; m -j". Check
         that the second run finishes successfully (with the binaries at
         /tmp/out as opposed to out/; otherwise it would fail the build
         due to the invalid binaries from step 2).
Test: lunch a target;
      `atest --host releasetools_test releasetools_py3_test`
Change-Id: I366099c3dfd5fa4282745ef258a8cf35338e1e42
2019-10-11 08:53:41 -07:00
cfig
1aeef725a7 AVB: decouple vbmeta.img from recovery.img for non-A/B devices
For following cases:
Case 1: A/B devices: no change
Case 2: non-A/B devices, with unsigned recovery image:
    not allowed anymore by mandating BOARD_AVB_RECOVERY_KEY_PATH
Case 3: non-A/B devices, with signed recovery image:
    vbmeta.img should not include ChainPartitionDescriptor of recovery.img,
    otherwise device can not even boot into normal mode if recovery partition
    is damaged

This CL will cause a build break if BOARD_AVB_RECOVERY_KEY_PATH
is not set for non-A/B targets with recovery.img
The following is an example to fix the build break by specifying
AVB signing configs for the recovery.img.

BOARD_AVB_RECOVERY_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem
BOARD_AVB_RECOVERY_ALGORITHM := SHA256_RSA2048
BOARD_AVB_RECOVERY_ROLLBACK_INDEX := $(PLATFORM_SECURITY_PATCH_TIMESTAMP)
BOARD_AVB_RECOVERY_ROLLBACK_INDEX_LOCATION := 2

Also note that libavb in bootloader needs an update to include this
commit Iaa886037edb18c2ff6c60fa2a7f883ab7303ba1a, to support verifying
recovery.img independently (not through vbmeta.img).

Bug: 130351427
Test (Case 3):
  normal   mode: avb_slot_verify(flags=AVB_SLOT_VERIFY_FLAGS_NONE)
  recovery mode: avb_slot_verify(flags=AVB_SLOT_VERIFY_FLAGS_NO_VBMETA_PARTITION)
Test: PYTHONPATH=build/make/tools/releasetools \
    python -m unittest test_validate_target_files
Test: Use a lunch'd target. `atest --host releasetools_test releasetools_py3_test`
Test: validate_target_files.py with Case-3 target files
Change-Id: I2a73252b385fa463b4abd444923a8acc473df0b4
2019-10-10 14:28:34 +00:00
Tao Bao
19b02fe8e5 Include per-partition fingerprint as AVB prop.
This allows querying per-partition fingerprint via libavb (in
particular, avb_property_lookup).

Bug: 80097573
Test: `m dist`; `avbtool info_image --image /path/to/image` to check the
      written prop.
Test: `atest --host releasetools_test releasetools_py3_test`
Test: Run sign_target_files_apks to sign a target_files.zip that's built
      with the change. Check the AVB prop in the signed images.
Change-Id: Id4f06df82c29e77aca128439c8c11f0367fe587e
2019-10-09 21:25:19 -07:00
Tao Bao
3ed35d3d25 releasetools: Fix the support for getting reproducible images.
We used to support getting idempotent images when given the same inputs,
by specifying UUID and AVB salt to the image builders. The support was
broken by Q changes that moved away from build-time fingerprints [1].

This CL addresses the issue by updating the logic in computing UUID and
AVB salt to be based on BuildInfo.fingerprint, as opposed to the
deprecated static build fingerprint.

[1] https://android-review.googlesource.com/c/platform/build/+/892933

Bug: 134525174
Test: TreeHugger
Test: lunch a target; `atest --host releasetools_test releasetools_py3_test`
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
Test: `add_img_to_target_files -a out/dist/target_files.zip`. Verify that
      the rebuilt images are identical to the ones deleted.
Change-Id: I8844fc7f044e9abfcd96a00e8958fa0583c70eaf
2019-10-07 23:32:07 -07:00
Tao Bao
1c320f8573 releasetools: Move BuildInfo into common.
There is no change to module functionalities. Testcases are moved around
accordingly.

Bug: 134525174
Test: TreeHugger
Test: lunch a target; atest --host releasetools_test releasetools_py3_test
Change-Id: I7bc8f49cc239e7c6655fe5e375508f01c1743b94
2019-10-07 20:13:59 -07:00
Tao Bao
e114804150 releasetools: Move MockScriptWriter into test_utils.
Bug: 134525174
Test: TreeHugger
Test: lunch a target; atest --host releasetools_test releasetools_py3_test
Change-Id: I6d30f4d153d59d65227275e1d3285e30dfafd90e
2019-10-07 20:00:34 -07:00
Tao Bao
60f5d67677 releasetools: Update OWNERS.
Test: N/A
Change-Id: Ie476ff08a360f15f769e54dca7428a2d055f1a37
2019-10-07 12:39:55 -07:00
Tao Bao
7b1c49a70d Merge "releasetools: Update the handling of recovery fstab." 2019-10-07 19:33:21 +00:00
Tao Bao
ae3a9a138d Merge "releasetools: Fix releasetools_py3_test." 2019-10-07 19:06:07 +00:00
Tao Bao
615b65d097 releasetools: Fix releasetools_py3_test.
Test: Use a lunch'd target;
      `atest --host releasetools_test releasetools_py3_test`
Change-Id: Ibcced83a7989ef8fc15464728ab1d5fa71ca988e
2019-10-06 23:29:34 -07:00
Tao Bao
a264feffe8 releasetools: Update tests to match SparseImage change.
The previous change in commit 22632cc82c
changed the behavior in loading file map. It now always puts a copy of
the input text in `extra` field. Update the tests accordingly.

Bug: 79951650
Test: Use a lunch'd target; `atest --host releasetools_test`
Change-Id: Iccf06c817c1305bf9946d7759c6f6f6af21fe85e
2019-10-06 22:21:14 -07:00
Tao Bao
99d855db8f Merge "releasetools: Support verifying files with non-monotonic ranges." 2019-10-07 01:33:30 +00:00
Tao Bao
765668fff7 releasetools: Update the handling of recovery fstab.
First, remove the use of info_dict['fstab'] in add_img_to_target_files.
 - info_dict['fstab'] corresponds to recovery fstab
   (`/etc/recovery.fstab`), which may differ from the one used for
   normal boot.
 - When calling build_image.ImagePropFromGlobalDict, we already have the
   desired info from global dict (`META/info_dict.txt`).

Second, common.LoadInfoDict now loads recovery fstab only for non-A/B
devices. Because the info is only meaningful for installing non-A/B OTAs
(under recovery mode).

Fixes: 132458722
Test: TreeHugger
Test: Build non-A/B incremental OTAs.
Change-Id: Id23e7b17264c123319fe00b1663d52bfd9b4a5e2
2019-10-04 22:39:32 -07:00
Tao Bao
22632cc82c releasetools: Support verifying files with non-monotonic ranges.
Fixes: 79951650
Test: Run validate_target_files on target_files.zip with files in
      non-monotonic ranges.
Test: python -m unittest test_validate_target_files
Test: python3 -m unittest test_validate_target_files
Change-Id: I82571d3358598775de4cdeb5e64035689fea6487
2019-10-03 23:21:22 -07:00
Bill Peckham
e868aec14b Moving recovery resources from /system to /vendor
This change is part of a topic that moves the recovery resources from the
system partition to the vendor partition, if it exists, or the vendor directory
on the system partition otherwise. The recovery resources are moving from the
system image to the vendor partition so that a single system image may be used
with either an A/B or a non-A/B vendor image. The topic removes a delta in the
system image that prevented such reuse in the past.

The recovery resources that are moving are involved with updating the recovery
partition after an update. In a non-A/B configuration, the system boots from
the recovery partition, updates the other partitions (system, vendor, etc.)
Then, the next time the system boots normally, a script updates the recovery
partition (if necessary). This script, the executables it invokes, and the data
files that it uses were previously on the system partition. The resources that
are moving include the following.

* install-recovery.sh
* applypatch
* recovery-resource.dat (if present)
* recovery-from-boot.p (if present)

This change includes the platform build system and release tools changes to
move the recovery resources from system to vendor (or /system/vendor). The
release tools need to know where to generate the recovery patch, and they
discover this from misc_info.txt variable board_uses_vendorimage, which the
platform build system generates.

We remove applypatch from PRODUCT_PACKAGES, but it is added back as a required
module in target/product/base_vendor.mk.

Several release tools rely on the misc_info.txt board_uses_vendorimage variable
to know how to generate and detect the recovery patch.

This change partially removes the --rebuild_recovery flag from the
merge_target_files.py script. The flag will be fully removed in a follow-on
change.

Bug: 68319577
Test: Ensure that recovery partition is updated correctly.
Change-Id: Ia4045bd67ffb3d899efa8d20dab4c4299b87ee5f
2019-10-04 00:04:34 +00:00
Treehugger Robot
1623ec97f1 Merge "add vendor_boot image support" 2019-09-27 21:07:24 +00:00