Commit graph

3693 commits

Author SHA1 Message Date
Chih-Hung Hsieh
01530a6ff2 Recognize new warning messages.
* Generalize regular expression to match new warning messages.
* Remove old unused pattern.

Change-Id: I54f2e5bdb255b290cb3751ba56558f5fe8d7245e
Test: Run warn.py with build.log
2016-08-24 20:53:27 +00:00
Tianjie Xu
4f09900e67 Replace META/care_map.txt when signing target files
Do not copy the "META/care_map.txt" from the source zipfile when
signing the target files with sign_target_files_apks. Because we'll
generate a new care_map after rebuilding the system/vendor images;
and we'll write the new "META/care_map.txt" to the signed-target-file.

Change-Id: I6919cfdf8314a4084b5f612a9c89469f391486a4
Test: Run sign_target_files_apks locally, and the entry is updated.
Bug: 30812253
2016-08-11 18:04:27 -07:00
Tao Bao
28e2fa1726 releasetools: Update the path to /default.prop for A/B devices.
On A/B devices (i.e. system_root_image="true"), /default.prop is
packaged at ROOT/default.prop (as opposed to BOOT/RAMDISK/default.prop
for non-A/B devices). Update the path so that we handle properties like
ro.bootimage.build.fingerprint properly.

The one for recovery is not affected, which stays at
BOOT/RAMDISK/default.prop for A/B devices and gets updated correctly.

Bug: 30811237
Test: Verify the property in the generated signed-TF.zip.
Change-Id: Id201a042d7ea988a64f89c6d04f43326a9851e27
2016-08-11 11:28:51 -07:00
Alex Deymo
3750ed3e17 Merge changes from topic 'ab_sideload'
* changes:
  Replace OTA sideload verification key when signing A/B devices.
  Install the update-payload-key in the recovery image as well.
2016-08-10 03:55:24 +00:00
Alex Deymo
b3e8ce6d1d Replace OTA sideload verification key when signing A/B devices.
The update-payload-key is used by update_engine_sideload from recovery
to verify an update payload.

Bug: 27178350
Change-Id: I7a0a307ae565e5e9cbf2c9b58fbcc055e87771ce
2016-08-04 16:06:12 -07:00
Chih-Hung Hsieh
d9cd1fafb5 Issue warning on projects with LOCAL_CLANG set to false.
* LOCAL_CLANG_EXCEPTION_PROJECTS is the list of project prefixes.
* New warning message is detected by warn.py as high severity.

Bug: 30575506
Change-Id: I87a38b2ba6bab1d2e5e7a9d5ec9557f0e609727c
Test: build and find new warning in build.log
2016-08-02 16:10:49 -07:00
Treehugger Robot
240faf5ef0 Merge "Fix clang-tidy performance warnings." 2016-08-01 21:04:20 +00:00
Tao Bao
6a54299fbd Handle multiple APKs from the same package in check_target_files_signatures.py.
We were using the package name as the key to index APKs. APKs from the
same package got messed up and gave wrong signature summary. Switch to
using the package filename as the key, which is identical in a given build.

Also fix the trailing space when printing the signature summary.

Bug: 30418268
Test: Run with a target_files.zip that has multiple APKs from the same package.

Change-Id: I6317e8c05e987c5690915e05c294153d10e2f0ab
2016-07-27 19:52:28 -07:00
Chih-Hung Hsieh
6c083acf2a Fix clang-tidy performance warnings.
* Use const reference parameter type to avoid unnecessary copy.

Bug: 30407689
Change-Id: Iefab05c077367f272abf545036b853e8a295c8cd
Test: build with WITH_TIDY=1
2016-07-27 10:28:33 -07:00
Treehugger Robot
620ce44a7a Merge "Add option to dump HTML warnings per project." 2016-07-26 23:04:29 +00:00
Chih-Hung Hsieh
4354a33874 Add option to dump HTML warnings per project.
* Use --byproject flag to dump warnings in HTML file by projects.
* Add external/google and vendor/google projects, vs other
  external/non-google and vendor/non-google projects.
* Refactor code in dumpcategory to dump warnings by projects
  when --byproject is specified.
* Recognize new compiler -Wvarargs warning pattern.
* No change to default HTML output format.

Change-Id: I2406ad392ab18683357ff0ee0cf6c41f87cf399c
Test: run through current build.log files.
2016-07-26 14:53:28 -07:00
Chih-Hung Hsieh
c868293abc Add performance* to default global tidy checks.
* This enables 5 performance related checks:
    performance-faster-string-find
    performance-for-range-copy
    performance-implicit-cast-in-loop
    performance-unnecessary-copy-initialization
    performance-unnecessary-value-param
* Recognize these new warnings in warn.py.

Bug: http://b/27779618
Change-Id: I7af3fdc262dd44db90d3b62eb37c5eac5a2eaa01
Test: build with WITH_TIDY=1 and run warn.py with build.log
2016-07-26 14:36:54 -07:00
Chih-Hung Hsieh
48a16ba709 Count warnings per project and dump out CSV format.
* No change to html output, except removing unused "PageTop" anchor.
* Add --gencsv flag to output a CSV file instead of html file.
  CSV files are combined to a table for dashboard charts.
* Count warnings from selected source project directories.
* CSV output lines are:
   num_of_warnings,project_name,warning_description
* project_name is empty for sum of warnings from all projects.
* warning_description could be:
  (1) "<severity>: <warning type description>"
      for one warning type, or
  (2) "<severity> warnings"
      for all warning types of the severity.

Change-Id: I1cf976ed0cf142091226b22665b7f28d383608fd
Test: run through current build.log files.
2016-07-21 14:43:02 -07:00
Chih-Hung Hsieh
c21ddbdcce Recognize new clang and clang-tidy warnings.
* Count new clang-tidy warnings:
    google-runtime-references
    google-runtime-operator
* Count new clang warnings:
    -Wconst-conversion similar to -Wliteral-conversion
    -Wundefined-var-template
* Modified static analyzer warning message about
    "Null passed to a callee that requires a non-null ..."
* Disable google-runtime-references by default and
  disable misc-macro-parentheses in vendor projects;
  too many old C/C++ code have this warning.

Change-Id: Ib35e0af85875aada17c92b0237b76cd5ae3bfcc8
Test: build with clang-tidy.
2016-07-20 14:45:01 -07:00
Chih-Hung Hsieh
5ecafd1260 Avoid matching the same warning line twice.
Used to skip only immediately followed identical warning lines.
Now classifywarning is called only once for each unique warning line.

Change-Id: Ie3b77ded70b41dafad91c042dbe15ad2be119e62
Test: run through build.log file
2016-07-18 15:27:02 -07:00
Tao Bao
461a3283d9 Merge "Fix root_filesystem_config.txt generation" 2016-07-13 23:04:10 +00:00
Tao Bao
62a6c464a2 releasetools: Fix the detection of using squashfs.
We should disable using imgdiff if *any* of the source and target
partitions uses squashfs.

Bug: 30004734
Test: Create an incremental with two builds with one of them uses squashfs.
Change-Id: I826cd13d7b852c548e4b45e61f5ae00f6407cac3
(cherry picked from commit f8acad1480)
2016-07-10 23:16:14 -07:00
Tao Bao
b6568cd4be releasetools: Disable using imgdiff for squashfs.
We use imgdiff to handle files in zip format (e.g. jar/zip/apk) for
higher compression ratio.

For system/vendor in squashfs, a) all files are compressed in LZ4
format; b) we use 4096-byte block size in their sparse images, but the
files in squashfs may not be laid out as 4K-aligned. So the blocks for
a given file as listed in block map may not form a valid zip file, which
may fail the patch generation with imgdiff.

Disable using imgdiff for squashfs images, and use bsdiff instead.

Bug: 22322817
Change-Id: Ie76aa4cece5c9d38cb1d1a34c505a4a8f37512d3
(cherry picked from commit 293fd135c7)
2016-07-10 23:15:48 -07:00
Tao Bao
13b6962e8d releasetools: Fix the payload public key replacement.
update_engine expects the extracted public key instead of the
certificate.

Bug: 28701652
Change-Id: I292d39da9e039f96d01a4214226aeb46f8cb881d
(cherry picked from commit afaf295cb8)
2016-07-08 12:52:45 -07:00
Gaelle Nassiet
aa8f2f65a0 Fix root_filesystem_config.txt generation
The first line returned by awk is empty. In consequence, when
fs_config computes its mode it will consider this line as a file (no
trailing slash) and affect the default android_files mode which is
0644. The mode for the root directory should be the default
android_dirs mode 0755.
Add a special case in fs_config to consider empty line as a directory.

Change-Id: I9f33f6fcf4be05c31914db898e65c92b0a611518
Signed-off-by: Gaelle Nassiet <gaellex.nassiet@intel.com>
2016-07-08 10:40:06 +02:00
Treehugger Robot
c78147c650 Merge "Move apksigner library to tools/apksig." 2016-07-07 20:33:05 +00:00
Treehugger Robot
1734523900 Merge "Recognize new C/C++ compiler and static analyzer warnings." 2016-07-07 18:07:21 +00:00
Alex Klyubin
1b09a508ba Move apksigner library to tools/apksig.
This moves build/tools/apksigner/core to its own project tools/apksig.
The move also renames the moved Java packages from
com.android.apksigner.core.* to com.android.apksig.* to reflect the
new name of the library.

Bug: 27461702
Change-Id: Iab812ae2b8f0a741014f842460c78e35bc249d43
2016-07-07 10:53:14 -07:00
Treehugger Robot
d81beca2b2 Merge "APK signer primitive." 2016-07-06 22:59:32 +00:00
Chih-Hung Hsieh
8d14543a9b Recognize new C/C++ compiler and static analyzer warnings.
Change-Id: I5f47c45498c640702922704aa5305a85e9493fcc
Test: run with Android build.log.
2016-07-06 10:22:23 -07:00
Tianjie Xu
aaca421ec1 Skip copying existing vendor images
add_img_to_target_files.py has an option of "-a" to add missing
images only. Under this option, the script should skip copying
the radio images for A/B devices when given image exists already
under "IMAGES/".

Test: Run the command on an A/B device, the existing radio images under "IMAGES/" don't get overwritten; and missing images are added correctly.
Bug: 29608905
Change-Id: Ie034b85a5d777d53e367f99470cea4d19cb1aaaf
2016-06-29 18:32:30 +00:00
Alex Klyubin
819e8485e4 APK signer primitive.
This adds an APK signer primitive which preserves as much of the input
APK as possible. For example, it preserves the order of APK entries
and preserves their contents, including compressed form and alignment
of data.

Bug: 27461702
Change-Id: I51d07c530480182a66379e70a00f680544ff6214
2016-06-27 11:28:08 -07:00
Ethan Xia
37b4a9819b releasetools: Change the default key path for bvbtool
The bvb path has been changed from 'system/bvb' to 'external/bvb'

Change-Id: I8587351b7d702a287883c6bd3c9de1cd3540435b
2016-06-27 17:19:01 +08:00
Chih-hung Hsieh
4837ce8ca6 Merge "Add +/- buttons to expand/collapse warning categories." 2016-06-24 18:07:44 +00:00
Tao Bao
8adcfd1688 Fix the path for verity_key replacement when signing.
system_root_image expects the key at ROOT/verity_key as opposed to
BOOT/verity_key. Also refactor the verity key replacement lines.

Bug: 29397395
Test: 'sign_target_files_apks.py --replace_verity_private_key newkey --replace_verity_public_key newkey.pub target_files.zip signed-target_files.zip' and verify the replaced key in boot.img.
Change-Id: I58a5defff4be008ad55d4b5a5b7148569c3b8d66
(cherry picked from commit e0ee794fa1)
2016-06-23 22:17:46 -07:00
Baligh Uddin
34eb000de5 Add ability to pass in payload_signer args
Bug: 28701652
Change-Id: I110d5fc14446e4a6a0f8e25dcb0d300decdf09a4
(cherry picked from commit 2abbbd0333)
2016-06-23 22:16:46 -07:00
Tao Bao
1a5e1d18a7 releasetools: Support using payload_signer.
For A/B OTAs, by default it calls 'openssl pkeyutl' to sign the payload
and metadata with the package private key. If the private key cannot be
accessed directly, a payload signer that knows how to do that should be
supplied via "--payload_signer <signer>".

The signer will be called with "-inkey <path_to_private_key>",
"-in <input_file>" and "-out <output_file>" parameters.

Test: Use a dummy signer, call 'ota_from_target_files.py --payload_signer <signer> <target_files.zip> <ota.zip>' and verify the signatures in the generated package.
Bug: 28701652
Change-Id: I26cfdd3fdba6fc90799221741b75426988e46fd3
(cherry picked from commit dea0f8bfed)
2016-06-23 22:15:30 -07:00
Badhri Jagan Sridharan
35c9b1243f releasetools: replace verity keyid
Replace verity keyid with the keyid extracted from cert
passed through --replace_verity_keyid. The veritykeyid in the
BOOT/cmdline of input target files is replaced with keyid
extracted from --replace_verity_keyid and written to the
output target files.

BUG: 28384658
Change-Id: Ic683f36f543c4fcd94b6f95e40f01200fbf45ee1
(cherry picked from commit b58d23fe00)
2016-06-23 22:13:12 -07:00
Tao Bao
a80ed221b8 Replace OTA keys when signing for A/B devices.
It replaces the package verification key (change of path due to
system_root_image flag), as well as the payload verification key.

Bug: 29397395
Change-Id: I10435072aaf4356f2d8b5e1b6e82eb9cead7ad62
(cherry picked from commit 24a7206430)
2016-06-23 22:12:31 -07:00
Chih-Hung Hsieh
465b610183 Add +/- buttons to expand/collapse warning categories.
* Add expand/collapse-all buttons to expand/collapse all warnings.
* Use HTML styles to reduce output file size.

Change-Id: Ica188cc4f123ce0ab8547f88315325c3e0560a39
Test: Checked output html file with Chrome browser.
2016-06-23 13:18:58 -07:00
Alex Klyubin
9a41c93f41 Use Builder pattern for ApkVerifier parameters.
This should make it easier to add parameters/options without breaking
existing clients.

Bug: 27461702
Change-Id: Ia4577f78d703a6b91828dd08492c78d5e9afb110
2016-06-22 14:59:12 -07:00
Elliott Hughes
d8a52f9785 Remove --no_prereq flag from OTA script.
Bug: http://b/29393071
Test: aosp_flounder "make dist"
Change-Id: I9ec85210e118f7e525291e31ab4081a2bd10f998
2016-06-20 14:44:18 -07:00
Alex Klyubin
05f87de7a3 Let caller handle NoSuchAlgorithmException.
This surfaces relevant NoSuchAlgorithmExceptions to the caller instead
of rethrowing as other exception types. Some setups need to be able to
distringuish issues due to their own misconfiguration
(required crypto algorithm mising -- NoSuchAlgorithmException) from
issues with the APK being signed or verified.

Bug: 27461702
Change-Id: I993f73edb29b2cd4cc485734a89a924ec357ef19
2016-06-17 12:15:32 -07:00
Alex Klyubin
f31ced2e10 maxSdkVersion can be specified for APK verification.
This enables verification of APKs which are served to a specific
range of Android platform versions, or to replicate behavior of
particular platform versions.

Bug: 27461702
Change-Id: I44ab4c99419eb97d72c4ccd109137fe1efda577d
2016-06-17 10:02:47 -07:00
Alex Klyubin
21213cff9c Merge "Reject PKCS#7 SignerInfo with unsupported parameters." 2016-06-17 15:56:51 +00:00
Treehugger Robot
63fc2b6766 Merge "Remove obsolete MTD support from the releasetools scripts." 2016-06-17 04:47:47 +00:00
Elliott Hughes
305b088729 Remove obsolete MTD support from the releasetools scripts.
Bug: http://b/29250988
Change-Id: I653dc306485c6b35411840b53211d42eb6d19e34
2016-06-16 20:48:47 -07:00
Dan Willemsen
eb4431595c Remove build/libs
There's only a single library, libhost, and it's only used by acp and
atree in build/tools, move it there.

Bug: 28001743
Change-Id: Ie404d2793710de4e265a6fa95d462c32d4042623
2016-06-16 14:52:47 -07:00
Alex Klyubin
d7236da153 Reject PKCS#7 SignerInfo with unsupported parameters.
This addresses the TODO to mimic the behavior of Android when
verifying APK JAR signatures. Unfortunately, the behavior of Android
kept changing in interesting ways between different platform versions.
This is hard-coded as a big lookup.

Bug: 27461702
Change-Id: I49bc181ee05f774ef8ee041af870385b35212c23
2016-06-16 12:44:36 -07:00
Tao Bao
7c5dc578b3 Support wiping userdata for A/B OTA packages.
update_engine now accepts POWERWASH=1 to schedule a factory reset in
the post-install phase. Hook up with the --wipe_user_data flag in the
OTA script.

Bug: 28700985
Change-Id: Ie73876a61db90d124d2af588d674757376e9aabc
(cherry picked from commit 38ca0be399)
2016-06-15 23:19:52 -07:00
Alex Klyubin
1fb96c3ff5 Merge "Don't depend on Bouncy Castle." 2016-06-15 20:32:35 +00:00
Treehugger Robot
98b4f07dfc Merge "Faster and cleaner way to obtain UTF-8 encoded form." 2016-06-14 22:37:10 +00:00
Treehugger Robot
6c0c720389 Merge "Use more prebuilt build-tools" 2016-06-14 22:14:07 +00:00
Alex Klyubin
d4761a19b8 Faster and cleaner way to obtain UTF-8 encoded form.
Instead of specifying character encoding by name, the faster, cleaner,
and safer way is to use StandardCharsets.UTF_8.

Bug: 27461702
Change-Id: I897284d3ceeb44a21cc74de09a9b25f6aec8c205
2016-06-14 14:18:21 -07:00
Alex Klyubin
cf89865b95 Merge "APK JAR signature verifier." 2016-06-14 20:56:28 +00:00