Commit graph

6694 commits

Author SHA1 Message Date
Yifan Hong
28ffd73591 Proper checks for vendor and ODM SKUs
If DEVICE_MANIFEST_FILE / ODM_MANIFEST_FILES is set, also include check
for empty vendor / odm SKU, respectively. Then, do cross product of
vendor SKUs and ODM skus, and check each possible combination.

Test: build with and without vendor skus
Test: m check-vintf-all
Bug: 148601741

Change-Id: I23cc81998e7afc36c43540bf6d615b4d4b288447
2020-03-13 18:32:07 -07:00
Xindong Xu
2a7aaa69ae ota_tool: fix incremental ota error
Problem:
Logical partitions do not update right since commit
f67dd8059d

    Generate block based OTA for product partitions

due to the wrong order of parameters. This is logged in
/tmp/recovery.log:

    skipping 0 blocks already patched to 202

Bug: 149794632
Change-Id: I92f7a8a226a8a96b892254362d362c515bfb6dc6
2020-03-13 14:25:35 +01:00
Daniel Norman
d5fe862628 Uses a per-partition fingerprint for building images and avb_salt.
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.

Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
      `add_img_to_target_files -a out/dist/target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
      the partial target-files.zip IMAGES are identical to the merged
      target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
2020-03-02 19:05:20 +00:00
Chih-Hung Hsieh
3cce2bcc60 Separate html/csv output functions into html_writer.py
* also suppress some pylint warnings

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I497dbb7496ca21470ba33db03eedb27f5e8f1e96
2020-02-27 15:39:18 -08:00
Tianjie Xu
223d055768 Merge "Preserve the monotonic property when subtracting 0 block" 2020-02-27 21:22:23 +00:00
Tianjie Xu
f02ececc7f Preserve the monotonic property when subtracting 0 block
The monotonic property of the range is lost after the recent change to
subtract 0th block from the file range. As a result, we read the block
in the wrong order when pull files from the raw image. This yields a
incorrect hash during validation.

Bug: 150334561
Test: run validate_target_file. unittests
Change-Id: Id4fedaaf273c42c8d21a4549fda4c9a45e22066a
2020-02-26 21:50:47 -08:00
Chih-Hung Hsieh
445ad810c8 Recognize more warning patterns.
* add more Java, Kotlin, and Rust warning patterns
* recognize more clang-tidy cert-* and bugprone-* warnings;
  still keep the catch-all 'cert' and 'bugprone' groups.

Test: warn.py build.log > warnings.html
Change-Id: I909ec09822f42f60493ebe943c055bf0d4b69938
2020-02-26 14:34:21 -08:00
Chih-Hung Hsieh
5ae5519661 Use same warn_common.py and warn.py for Android and Chrome
* add chrome_project_list.py
* use newer Chrome's warn_common.py and severity.py
* modify warn_common.py to work with both python2 and python3,
  to work with and without google3

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I9aeee7031c11988d8ead1733ff4846ae6cb1ee73
2020-02-25 19:47:35 -08:00
Bowgo Tsai
2fe786a528 Support removing AVB public keys
AVB public keys might be installed into the first-stage ramdisk.
This CL supports removing some AVB keys during the signing process,
which can be useful when a key needs to be revoked.

Also correcting the path of force_debuggable detection.

Bug: 150109393
Test: make dist
Test: ./build/tools/releasetools/sign_target_files_apks \
      --remove_avb_public_keys key1.avbpubkey,key2.avbpubkey \
      out/dist/*-target_files-*.zip signed-target_files.zip
Change-Id: I1af95adbbec655f40121f3392965dd1950aca4e5
2020-02-24 17:53:10 +08:00
David Anderson
ab9c7e6d04 Exclude holes from the block map.
e2fsdroid can export holes as a "0" block, in order to make sure logical
and physical block assignments are maintained properly. Exclude these
hole blocks from range sets in LoadFileBlockMap.

Bug: 139201772
Test: m dist on non-A/B
Change-Id: Ic04e869572229678fdeaaa6fbf6f464d6c672051
2020-02-21 17:04:00 +00:00
P.Adarsh Reddy
8b2e7c9c51 Make oemaids_headers available to vendor.
This change makes the oemaids_headers header library
available to vendor side.

Bug: 149665231
Test: Tested compilation, oemaids_headers is available to
a sample vendor binary.

Change-Id: I87369f1db883e40e83d6076defdd4c0d84aac46f
2020-02-18 09:44:23 +00:00
Tamas Petz
c0a8c63935 build_image: check value of "ext4_share_dup_blocks"
Regardless of the value of "ext4_share_dup_blocks" "-c" flag is passed
unconditionally. The correct behavior is to check if
"ext4_share_dup_blocks" is set to "true".

Test: 1. build bonito with "BOARD_EXT4_SHARE_DUP_BLOCKS := false"
      2. check for "shared_blocks" feature of system.img
         ./out/host/linux-x86/bin/tune2fs -l \
         ./out/target/product/bonito/system.img | \
         grep -o shared_blocks

Change-Id: I8c0e5606658dd0f47b71cfc6c453a77a57da4f42
2020-02-10 11:45:06 +01:00
Tianjie Xu
9bd832a012 Remove the key parameter when verifying avb images in validate_target_files
If a key is specified, the avbtool always use the input key to verify
all the chained images. And this will cause failures when the vbmeta &
system use different keys (e.g. RSA 4096 vs RSA2048). Because the public
key to vbmeta will always fail to verify the system image. Remove the
'--key' parameter in the verification command, so the avbtool will use
the embedded public in the image.

Test: validate target-file from sdk_gphone_x86_64
Bug: 148916990
Change-Id: I9d31be0f8c32af605af94fa73d07818f40f51ec4
2020-02-06 13:31:26 -08:00
Chih-Hung Hsieh
ed74896600 More Java/Kotlin warning patterns
* Simplify java warning pattern functions.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I837fa9290acded3b936182877d9e0c00de4cdf4f
2020-02-04 15:34:12 -08:00
Chih-hung Hsieh
9f7a42025d Merge "Add new Java/Kotlin/Make warning patterns" 2020-02-04 18:41:57 +00:00
Chih-Hung Hsieh
a7f5f3fcbb Add new Java/Kotlin/Make warning patterns
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I3195999d60f268f105ead022c193319de4af4e64
2020-01-31 16:14:04 -08:00
Tianjie Xu
0e38d64e49 Add quote to the signing arguments when passing to apexer
We should add quote when passing the signing arguments to apexer. So
the final argument whould look like
apexer ... --signing_args "--signing_helper_with_files=%path"

And the argument parser in avbtool will eventually parse the
"--signing_helper_with_files" correctly.

Bug: 148627666
Test: unittest pass, run a smoke sign_target_files_apk with signing arguments
Change-Id: Ie203d26a508cb98ca94e6c364a2640b681ad79f2
2020-01-31 20:19:18 +00:00
Tianjie Xu
cea6ad1fb0 Pass signing_args to apexer to repack apex
In the build signing environment, we don't necessary have access to the
physical private key. And we work around it by passing an extra
signing_helper to the avbtool instead of a private key. Since we need
to perform signing to add the hashtree footer when rebuilding the apex
image. We should propogate the signing_helper to apexer and eventually
to avbtool.

Bug: 148627666
Test: unit tests pass. Smoke test by pass a public key to apexer and
hardcode the private key in signing helper script.
Change-Id: I56d57485493a5ba2cbe4c49e493ee9604f68088b
2020-01-31 01:37:36 +00:00
Tianjie Xu
ba5de57120 Merge "Assets should be optional in apex repacking" 2020-01-29 22:59:30 +00:00
Tianjie Xu
83bd55c1db Assets should be optional in apex repacking
We should only provide the --assets_dir option when the assets directory
is available in the original apex.

Bug: 148452817
Test: unit tests pass
Change-Id: I44308b80a43ff3f441223f0ecc248991d1d83f31
2020-01-29 11:40:42 -08:00
Chih-Hung Hsieh
b09530bd65 Add new C++ and Java warning patterns
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I11457fdab7abd475e835367582c0b9aab11cfe6f
2020-01-29 11:01:36 -08:00
Tianjie Xu
8fff584906 Merge "Make the error more instructive on missing android_jar_path" 2020-01-29 03:19:23 +00:00
Daniel Rosenberg
dac25a96df Merge "Add build properties to enable casefold and prj_quota" 2020-01-28 23:53:25 +00:00
Tianjie Xu
61a792f323 Make the error more instructive on missing android_jar_path
So when $ANDROID_BUILD_TOP is missing, the error message print out the
path of the missing file; instead of a type error.

Bug: 148452817
Test: check result with missing ANDROID_BUILD_TOP
Change-Id: I5f841f6626eb3843364f20e78b278d3244505396
2020-01-28 10:58:14 -08:00
Daniel Rosenberg
6cc2c81f4d Add build properties to enable casefold and prj_quota
This adds the ability to format f2fs and ext4 in a way that supports
casefolding and project quotas. Project quotas require wider inodes, and
thus cannot be enable apart from at build time. Casefolding can be
configured with f2fs.fsck/tune2fs later.

Both features are enabled via inheriting emulated_storage.mk in the
device.mk

Bug: 138322712
Bug: 138321217
Test: Enable on device with related patches, view fs configuration on /data
Change-Id: Ib1ab0589006ff1b3fb7478b7e9b0a8aad71e4b09
2020-01-27 12:45:47 -08:00
Tianjie Xu
88a759d651 Resign apks contained in apex
Some apex payload images contain apk files. And these apks need to be
signed during the signing processed when sign_target_files_apks is
called. To support the signing, we can extract the payload and repack
the apex file with the (de)apexer tool. Add the signing support in the
apex_util.

Bug: 146508800
Test: unit tests pass, run sign_apex, sign_target_files_apks
Change-Id: If6d58975248709a144b07dbabf47c27916e5695e
2020-01-27 19:48:39 +00:00
Treehugger Robot
5023711e9d Merge "Always check VNDK variant identicalness" 2020-01-17 18:45:41 +00:00
Vic Yang
4873e65c8d Always check VNDK variant identicalness
Previously, we only check VNDK core and vendor variants are identical
when a VNDK library is not declared to have different variants AND the
target has TARGET_VNDK_USE_CORE_VARIANT set.  Therefore, it is fairly
easily to break a TARGET_VNDK_USE_CORE_VARIANT target as it needs to be
tested explicitly.

This change uses the new LOCAL_CHECK_SAME_VNDK_VARIANTS and expands the
check to run regardless of TARGET_VNDK_USE_CORE_VARIANT.  Also adds
support for VNDK-in-product.

Bug: 145157349
Test: Build success for targets with and without
      TARGET_VNDK_USE_CORE_VARIANT.
Test: With the corresponding change in build/make, remove libbinder
      from build/soong/cc/config/vndk.go and check build fails even
      when TARGET_VNDK_USE_CORE_VARIANT is not set.

Change-Id: Iec708b971072e6580f77a03e243b30b89b3b054d
2020-01-14 20:05:49 -08:00
Chih-Hung Hsieh
5392cdbeb1 Call a writer to dump html; new warning patterns.
* HTML emit functions now take a writer parameter.
  This makes warn_common.py one step closer to the ChromeOS version.
* Add new found warning patterns from java and yacc.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I5c446ca767746598f07603591fdf98f7d82cae17
2020-01-14 11:57:12 -08:00
Chih-Hung Hsieh
8724ff72ef Clean up cpp warning patterns
* Remove the useless 'option' key.
  It is only used in some C/C++ warning patterns
  to give a hint of options to turn to -Werror.
  Now the global default is -Werror.
* Factor out common code patterns into high/medium/low functions.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: Ibd3f768b1552ada925eb5afb0f01ab674c968a87
2020-01-13 11:10:54 -08:00
Chih-Hung Hsieh
949205a661 Use new Severity class and update *_warn_patterns
* This new class definition and patterns are
  shared between Android and ChromeOS compiler tools.
* Suppress hard to fix and false positive linter warnings.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: Icb47809100ad30796cb1da82610e989d450194fa
2020-01-10 15:21:20 -08:00
Treehugger Robot
b4484dc33b Merge "Rebuild the vendor install-recovery if necessary." 2020-01-10 13:49:55 +00:00
Treehugger Robot
88ab531831 Merge "Go one directory up looking for releasetools path" 2020-01-10 12:31:32 +00:00
Robin Lee
da427de124 Rebuild the vendor install-recovery if necessary.
Test: Manual on a device with VENDOR/bin/install-recovery.sh
Bug: 146504238
Bug: 68319577
Change-Id: I33c253716bda5e7e655dac5c0636e9995e7b64c1
2020-01-09 00:29:38 +01:00
Treehugger Robot
620a2f2366 Merge "Add system_ext to VINTF search paths." 2020-01-07 23:04:54 +00:00
Chih-Hung Hsieh
a9f7746f29 Add new compiler warning patterns
* Mostly new Java and Kotlin warnings found in nightly builds.
* More clean up of Java and C++ warning patterns will follow.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I8776401d15c3a59535722d52a5eec03f954d3b15
2020-01-06 12:11:44 -08:00
Robin Lee
34ea73940e Go one directory up looking for releasetools path
Change I366099c3dfd5fa4282745ef258a8cf35338e1e42 introduced a more
restricted search where we only look in out/host/x/bin instead of
out/host/x when looking for files.

Because signapk is in out/host/x/framework, and lib64 is in out/host/x,
we need to go one level up for this to work properly.

Fix: 147096688
Test: atest --host releasetools_test releasetools_py3_test
Change-Id: I597e1d95c09eb0e27b2faebb107218761901d9ac
2020-01-02 19:25:20 +00:00
P Adarsh Reddy
7e9b5c4204 Handle virtual A/B defines during dynamic_partitions_info merge.
While merging the Dynamic Partition info Dicts in the case of
split builds, pick the virtual-a/b flags like virtual_ab,
virtual_ab_retrofit from the vendor half.

Bug: 146811936
Test: Merged build contains the virtual a/b defines.
Change-Id: I0171b074094e47cfd12dc6af2565acdcd1b65163
2019-12-24 06:32:03 +00:00
Yifan Hong
dafae5228c Merge "Add /system/vendor/odm for VINTF search path" 2019-12-23 22:26:55 +00:00
Yifan Hong
9cbb624abd Add system_ext to VINTF search paths.
Bug: 140280874
Test: pass

Change-Id: Ifd5b784fc3c4553284f74a1f06cbac0a9bef4c75
2019-12-19 14:00:10 -08:00
Yifan Hong
2870d1e3c1 Add /system/vendor/odm for VINTF search path
Compared with board_config.mk and found this one is missing.

Also add docs to explain where are these values from.

Test: pass

Change-Id: Ic22d75a538657bd286edcae6d95f9ee462911975
2019-12-19 14:00:03 -08:00
Chih-hung Hsieh
9b88d8f8b1 Merge "Split warn.py into multiple files" 2019-12-19 04:30:08 +00:00
Chih-Hung Hsieh
888d143e4c Split warn.py into multiple files
* Split warning patterns into *_warn_patterns.py
* Split project list into android_project_list.py
* Split out the Severity class, to be changed later
* Split core of warn.py into warn_common.py and
  leave only platform dependent code in warn.py.
  This allows the core logic be used with different
  parallel processing libraries.
* Old warn.py just calls -m warn.warn.

Test: path_to_build/tools/warn.py build.log
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I6734e4472a21018cd5ce06d549f6dbca24f4de54
2019-12-17 15:12:31 -08:00
David Anderson
117b52f74e releasetools: Pass --virtual-ab to lpmake.
Note that we are currently excluding this flag for retrofit devices,
since it necessitates support for a newer super.img format, which
retrofit devices may not fully support.

Bug: 134949511
Test: m -j, lpdump super_empty.img
Change-Id: Id69a8d755b9f4c4bb5fbd456a416ced1a5f1d135
2019-12-16 22:17:28 +00:00
Treehugger Robot
d6aa90ce9a Merge "sign_target_apks: Add networkstack to default key_map" 2019-12-13 22:22:08 +00:00
Dan Austin
5290364e9b Add AFTL relevant options to OPTIONS in common.py.
Adds AFTL relevant options in preparation for avbtool changes. The
options are:
aftl_server: A string representing the URL of the transparency log.
aftl_key_path: The path to the public key associated with the
transparency log (in PEM format).
aftl_manufacturer_key_path: The path to the key associated with the
manufacturer for AFTL message signing.
aftl_signing_helper: The signing helper for use with AFTL message
signing.

Change-Id: I73d466f24b171c16dbdaa40c589779bed52be551
Test: Ran add_img_to_target_files with all AFTL options.
Bug: 138779249
2019-12-12 15:48:52 -08:00
David Anderson
0ec64ac0aa releasetools: Generalize base_fs_file handling.
Share the code to handle base_fs files rather than duplicate it for
system/vendor. At the same time, add support for system_ext and product
partitions.

Bug: 139201772
Test: manual test
Change-Id: Ib1b7c018c828901e24eefc01e3faead03e77bc2d
2019-12-11 22:32:00 +00:00
Oleh Cherpak
982e608085 sign_target_apks: Add networkstack to default key_map
Test: run cts -m CtsSecurityTestCases -t \
	android.security.cts.PackageSignatureTest#testPackageSignatures
Bug: 145955635
Change-Id: I1a1498562e2b5983010cb98e3edcd03ceb2cce19
Signed-off-by: Oleh Cherpak <oleh.cherpak@globallogic.com>
2019-12-10 15:03:57 +00:00
Mitchell Wills
a428b092e6 Fix comment to reflect that the value passed to head is actually the size
There is no functional change

Bug: 130257221
Test: generated an archive using the script and ran the result
Change-Id: I946f2827a9519873be2173fc50d9bc1cc46a326f
2019-12-06 19:13:21 -08:00
Mitchell Wills
032de6765b Use bash instead of sh in self extracting archive
`echo -n` and `read` seems to not work the same on mac

Bug: 130257221
Test: generated an archive using the script and ran the result on linux and mac
Change-Id: I129090ccf50781b748bff9a5c7890824d2670ad4
2019-12-06 16:55:24 -08:00