Commit graph

73 commits

Author SHA1 Message Date
Stephen Hines
aa94811b2a Merge "Guess source tree root from common file prefix." 2019-10-18 17:59:23 +00:00
Chih-Hung Hsieh
83980501d9 Fix severity of analyzer warnings; add new warnings.
* The following were high severity warnings but mistakenly
  grouped into analyzer warnings in a recent change.
        clang-analyzer-unix.Malloc
        clang-analyzer-cplusplus.NewDeleteLeaks
        clang-analyzer-cplusplus.NewDelete
* Add new Protoc and Kotlin warnings about unused imports and variables.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Ied1f9ace97f29fba58a40dff2346a0391ab22f06
2019-09-26 12:14:32 -07:00
Chih-Hung Hsieh
790020437d Factor out common code in Java warn patterns.
* Some comments are kept to reduce diff artifacts.
* Warning patterns will be separated by tools later,
  to make it easier to share code with Chromium and
  other build tools.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I6b589478cf5ae73940b4712d9faabd0909546f02
2019-09-25 13:39:41 -07:00
Chih-Hung Hsieh
d591a5e012 Add new compiler warning patterns
* Add Kotlin, NDK, Protoc, Asm, and other new patterns.
* Put all clang-analyzer warnings to the ANALYZER severity group.
  Some security and null pointer warnings still have HIGH severity.
* Comment out old C/C++ warnings that overlap with analyzer
  warnings and do not appear in current build.log.
* Other old fixed warning patterns should be commented out later.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Idc7b09e5e4eefa127d41e9f37da1f566ed1ede1f
2019-09-24 14:52:09 -07:00
Chih-Hung Hsieh
f5db8985f5 Guess source tree root from common file prefix.
* When processing a downloaded build.log file,
  find_android_root is very slow because the
  android source tree is not found.
  Change it to look for warn.py only up to 100 times,
  and then set android_root to the longest common path prefix.

Bug: 141524232
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I8cc2e208d439267313aaa6cdbc00f40320409c63
2019-09-24 18:19:17 +00:00
Chih-Hung Hsieh
a1187072c5 Handle rustc warning messages.
* Combine the first two lines of rustc warning messages.
* Give some rustc warnings HIGH and MEDIUM severity.

Bug: 141311294
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Ia3a206d68d9e92934296487b006a7b424044a822
2019-09-20 17:14:22 -07:00
Elliott Hughes
36d86c1670 Remove perfprofd reference.
Dead and gone.

Test: treehugger
Change-Id: I8575fd9efaa371d61fac5204ee4d982e985a9a56
2019-07-17 08:16:29 -07:00
Chih-Hung Hsieh
9018ea49d8 Fix pylint warnings and now prefer python3
* Read/write warning messages as utf-8 strings.
* No more embedded raw utf-8 code in file.

Test: ./warn.py --csvpath warnings.csv build.log > warnings.html
Change-Id: Ie51700721a32bede1d3b250f4a42fd8facd6af75
2019-06-24 15:24:39 -07:00
Andreas Gampe
161ec1ecc1 Build: update warn.py for Errorprone 2.3.2
Bug: 114129741
Test: manual
Change-Id: I7fb34881a623ffb115a92bfffb108b503d4645ad
2018-10-15 08:56:58 -07:00
Chih-Hung Hsieh
e1672866a7 Recognize -Wimplicit-fallthrough.
Test: warn.py build.log
Change-Id: Ie55248f7348b20d09909a856fe7581a0476b9d9b
2018-08-31 16:19:19 -07:00
Christopher Ferris
e027dd4aef Remove micro_bench.
micro_bench has been deprecated, so remove it from the list of executables
to be built by default.

Bug: 26928775

Test: Builds.
Change-Id: I701b3077ec00c7e75145aeb52463f47bf8ff2f09
2018-08-07 19:57:56 +00:00
Chih-Hung Hsieh
b2afb63ac9 Escape special HTML characters in warning messages.
Bug: 111699078
Test: run warn.py with a build.log file
Change-Id: I553dad57a6d81ea6cc89289741d9963f49cc3b47
2018-07-20 15:36:26 -07:00
Andreas Gampe
b9dc23a4d8 Tools: Update warn.py for Errorprone 2.3.1
Note: this uses the default severity in Errorprone, modulo
forced-ERROR settings in Android.

Bug: 72971960
Test: manual
Change-Id: I6d8b855b88289b763b2a98e8685627bf1163989d
2018-06-08 10:24:35 -07:00
Andreas Gampe
2e987af407 Tools: Sort Errorprone warn.py entries by name
To improve updateability.

Bug: 72971960
Test: none
Change-Id: I7ab7cdfe53518b29d036f418fe66aef69b5e409a
2018-06-08 10:22:59 -07:00
Chih-Hung Hsieh
9be27764c0 Fix gpylint warnings.
* This file should be gpylint clean so it can be easily copied
  and pass presubmit checks in google repository.

Test: gpylint warn.py
Change-Id: I5391f19c7a0141bc211f5b3924f48b345bcb03ab
2018-06-07 10:51:47 -07:00
Treehugger Robot
991bbccf95 Merge "Remove references to common_time" 2018-06-05 10:33:13 +00:00
Chih-Hung Hsieh
9e9355dac6 Recognize more clang-tidy compiler warnings.
* Some are from clang compiler and Android makefile checker.

Test: run with build.log
Change-Id: I371f08f3ae956e9425515b2cd67c5af3d0425b89
2018-06-04 11:32:32 -07:00
Neil Fuller
ce483225bf Remove references to common_time
The code is unused and is being removed.

Bug: 80462439
Test: build / boot
Change-Id: I812bff4fdc061b060633949b7087ab9c951fa7e0
2018-06-04 15:38:21 +01:00
Andreas Gampe
cf528ca79d Warn.py: Update errorprone categories
From current tip-of-tree.

Bug: 72451126
Test: build/make/tools/warn.py errorprone.log
Change-Id: I1d632fb8801b74af695b4f7f464a99792e46637a
2018-01-25 12:25:20 -08:00
Andreas Gampe
7f494c121f Warn: Change GuardedBy regular expressions
This was modified in commits 20a8664 "Change canonical name for
GuardedByChecker to GuardedBy" and 1f6dc3dc "Merge GuardedByValidator
into GuardedByChecker."

Test: m
Test: manual
Change-Id: I07305e59694a955e52bd69c1cbe53346808b78aa
2018-01-17 21:28:21 -08:00
Colin Cross
fdea893833 Replace build/tools with build/make/tools
Test: m checkbuild
Change-Id: I8db6c27a6d2372f142d10568fe92d6f2e9a59252
2017-12-06 16:12:40 -08:00
Chih-Hung Hsieh
2cd467b36c Recognize new clang-tidy android-* warnings.
Test: warn.py build.log
Change-Id: I9bcd1fffcb567346a3d959f98821e2cf58e87ba2
2017-11-16 15:42:11 -08:00
Chih-Hung Hsieh
655c542dc8 Find target keywords in the first 100 lines.
Bug: 62416199
Test: warn.py build.log
Change-Id: I8c5ef310758fdd44519aeb1efe2f0bd92fe1ac87
2017-06-07 23:17:13 +00:00
Chih-Hung Hsieh
8ef1aae6d3 Recognize new clang and make warnings.
Test: warn.py build.log
Change-Id: I078b3cc35a00590312208cb90af0a1f2431abf09
2017-05-18 14:29:59 -07:00
Chih-Hung Hsieh
ef21d1458b Get Android TOP from build.log file.
find_android_root works only when warn.py is found
so it works for build bot, but not when warn.py is
run against a stand alone build.log file.
Now get the TOP directory path from build.log
if it is found.

Bug: 37725427
Test: warn.py --gencsv build.log
Change-Id: I7b776c2e71a8d8d769f0600b08abe37acbd6ce7c
2017-04-27 10:25:37 -07:00
George Burgess IV
169f5f1c6c warn.py: have URLs open in new tabs
Bug: None
Test: Ran warn.py on a build log. Code links in resultant HTML open in
new tabs when clicked.

Change-Id: Ia3def5ea14fe216700cb2ffabe4a39b98e69d55a
2017-04-21 15:19:36 -07:00
Sam Saccone
03aaa7e2d8 Add the ability to save off the CSV to a file.
While we are generating the html file also expose a way for a user to
save off a csv file. This prevents having to run the tool twice.

* Add a new flag called csvpath that takes a file path for the csv dump.
* Use csv.writer to simply print formats.
* Modify the dump_csv and count_severity to take a csv writer.

Test: run with a build.log file, compare before and after html/csv dumps.
Change-Id: I27ed730625a352fdc3fb2928433eb5e8b2ea229c
2017-04-12 18:58:53 -07:00
Nick Glorioso
8a7b25c736 Update warn.py for new Error Prone diagnostics
Test: Local build, tee'd to file, ran warn.py and checked output HTML
Change-Id: Iee3386e88c73e4b8e0a1fcc60105ee5fb86e3cb0
2016-11-22 10:35:07 -08:00
Chih-Hung Hsieh
76d0065ef8 Separate parallel_classify_warnings out of parse_input_file.
* Allow other callers of this module to:
  * pass any input stream to parse_input_file,
  * pass any warning_lines to parallel_classify_warnings,
  * call dump_csv or dump_html to get output.
* No output change.
* Capture and ignore signal.SIGTERM at the end,
  to avoid bad warning/error messages from the exit clean-up process.

Test: run warn.py build.log.
Change-Id: I1414797a536c0ee622e2a34c226578621be1ddab
2016-11-09 18:19:05 -08:00
Chih-Hung Hsieh
63de300094 Do not use multiprocessing.Pool when --processes=1.
Purpose of this change is not to save some execution time,
but to avoid forking another process. In other applications
that use a wrapper to call this python script, it is difficult
to get overwritten file I/O functions work in a subprocess.
So the wrapper will call warn.py with --processes=1.

Test: run "warn.py --processes=1 build.log"
Change-Id: I5998d5c70d81a456c86eb4002f444a4a60135477
2016-10-28 10:53:34 -07:00
Treehugger Robot
92396e1747 Merge "Use parallel subprocesses to classify warnings." 2016-10-18 03:49:49 +00:00
Chih-Hung Hsieh
a6bd044ac1 Use parallel subprocesses to classify warnings.
* Add a --processes flag to specify number of parallel processes,
  with default multiprocessing.cpu_count().
* Wrap long line to suppress pylint warning.

Test: run warn.py with a large build.log file.
Change-Id: I9a93a9324bc531c1bce741367013051ce40a67fa
2016-10-17 17:47:20 -07:00
Stephen Hines
d0aec89d43 Split apart frameworks/compile into sub-projects.
These projects are all used by RenderScript, but it would be good to
differentiate warnings for each.

Change-Id: I983279d590630175a36075b01e8eec9100c07d26
Test: Ran warn.py
2016-10-17 15:40:38 -07:00
Chih-Hung Hsieh
144864eed0 Add more patterns for a finer project warning list.
Bug: 31377083
Test: run warn.py --byproject build.log
Change-Id: I3cad8e24bc97a84f067a162ff728e30fde3c7197
2016-10-14 12:27:17 -07:00
Treehugger Robot
6949e7ecba Merge "Better pattern matching and dump of warning messages." 2016-10-12 16:35:34 +00:00
Chih-Hung Hsieh
5722f9247b Better pattern matching and dump of warning messages.
* Warning messages must start with source file path.
  This will reject lines containing only 'warning:'
  but not a source file path prefix.
* Escape both backslash and quotation marks in the
  dump of strings to JavaScript string literals.

Bug: 32060052
Test: run through build.log files
Change-Id: Ib064768b1ba2954f974604ea054f7a6d2ad15ae7
2016-10-11 15:33:19 -07:00
Chih-Hung Hsieh
a45c5c1cf9 Normalize warning source file path to relative path.
Remove Android root directory path from long
absolute source file path.

Test: run warn.py with build.log
Bug: 32059187
Change-Id: Ib630cebd8c56a9fef443139ca502d97e59c7b42e
2016-10-11 15:30:26 -07:00
Chih-Hung Hsieh
1eabb0ecb2 Add a new ANALYZER severity group.
* Separate clang-tidy's clang-analyzer-* warnings from
  other clang-tidy warnings.

Bug: 31559947
Test: build with WITH_TIDY=1 and DEFAULT_GLOBAL_TIDY_CHECKS=*,-readability-*,-google-readability-*
Change-Id: If03c421c9319741ef6588bc0d803b6da0cc4433a
2016-10-05 11:53:20 -07:00
Treehugger Robot
312c310f7c Merge "Only run warn.py's main function if it is the main program." 2016-09-29 03:58:07 +00:00
Meike Baumgärtner
ac9d5df349 Only run warn.py's main function if it is the main program.
Test: run warn.py --byproject build.log
Change-Id: I155cce8c952d6d8a02c8e1809bf61a46c847c26e
2016-09-28 17:59:08 -07:00
Chih-Hung Hsieh
9f766239f0 Fix pylint warning and refactor project_list.
Bug: 31377083
Test: run warn.py --byproject build.log
Change-Id: Id442cf6608bf5c14784063c05f5ff0b731f17486
2016-09-27 21:13:52 -07:00
Chih-Hung Hsieh
b426c54e8a Fix Google pylint warnings.
* Rename 'severity' to 'Severity'.
* Fix bad line indentations and space.

Test: run warn.py --byproject build.log
Change-Id: I33d701cca408b5ca4715be8d44c890942b7d132a
2016-09-27 18:15:40 -07:00
Chih-Hung Hsieh
2a38c37d57 Add buttons to group warning by project or severity.
* Add more project patterns.
* Add more top level comments for global variables and functions.
* Resequence severity numbers to match the dump order.
* Emit warning messages and tables to static HTML JavaScript arrays.
* Replace old static HTML table dumper functions with
  new dynamic HTML JavaScript to generate sections of warnings.
* Warning messages are grouped into sections by severity or projects.
* Better descriptions for SKIP warning patterns.
* Replace output function with print.

Bug: 31377083
Test: run warn.py --byproject build.log
Change-Id: I7b44ef6223d5b2f1aa31655a5a47d854f9a1dedc
2016-09-26 16:02:12 -07:00
Treehugger Robot
bcd229277a Merge "Fix Google pylint warnings." 2016-09-26 21:18:08 +00:00
Chih-Hung Hsieh
6c0fdbb84d Fix Google pylint warnings.
* Rename variables and functions to follow Google coding style.
* Fix line indentations to follow Google coding style.
* Add module and function docstrings.
* Change shebang line to avoid pylint warning.
* Suppress some pylint warnings to avoid too many changes at once.
* Fix typos.

Test: run warn.py --byproject build.log
Change-Id: I569961981b562e35cb5609a96cd5ebb40a80829c
2016-09-26 12:24:54 -07:00
Chih-Hung Hsieh
0a19207f7b Recognize new warning messages.
* -Wunnamed-type-template-args and -W#pragma-messages

Test: run warn.py with build.log.
Change-Id: I36277e001081604f92a4d9c92f9a0b5d3f0ee5e5
2016-09-22 17:44:38 -07:00
Chih-Hung Hsieh
99459fc0d0 Simplify warn.py and output of warning count table and tablerow.
* Use a loop to initialize empty 'members' and 'option' of warnpatterns.
* Create severity.{color,columnheader,header} from an attributes list.
* Compute totalbyproject and totalbyseverity from a constructor.
* Skip all-zero rows and columns in the warning count table.
* Remove redundant spaces and newlines in tablerow output.

Test: run through build.log files
Change-Id: I4d3fa4ecd92e1afab91d85b7535d03f8696e83d8
2016-09-22 15:15:43 -07:00
Dan Willemsen
121e284b46 Fix link_type checking
This was printing "error:", but not actually triggering an error.
Instead of trying to write a single line bash script to handle this,
move the actual check into python. This allows us to print all of the
errors for a single module before triggering the failure.

Also updates the warning format and the warn.py script to properly parse
these warning. Many of the java:sdk -> java:platform warnings are false
positives due to the lack of LOCAL_SDK_VERSION markings on prebuilts.

Individual tags can be marked as warnings now, which lets us check for
system libraries linking against vendor libraries (which won't work on
AOSP). I'm not sure this is a completely valid check, which one reason
that it's just a warning.

Test: m all_link_types (with some missing libs commented out)
Change-Id: I333e418c9a4511b7c7e826891ae481da08fbf6f9
2016-09-15 14:40:39 -07:00
Chih-Hung Hsieh
e41c99b721 Emit a table of warnings by project and severity.
* Use 't1' CSS class for main warning tables,
  and default table style for new warning count table.
* Count total skipped warnings.
* Append count of warnings in each severity header.
* Fix some skipped warning patterns and count/dump skipped warnings.
* Add dumpskippedanchors function to fix expand_collapse
  error when --byproject flag is used and severity.SKIP
  is not emitted.
* Minor coding style changes:
  * Loop through severity.kinds.
  * Remove unused import.
  * Add spaces around assignment operators.
  * Adjust indentation.

Bug: 31377083
Test: run warn.py with build.log
Change-Id: I68949edc4601b10ea2f7ac84e48e6c3da4ab1e4f
2016-09-13 10:18:01 -07:00
Chih-Hung Hsieh
0862c48770 Recognize new clang-tidy and Java warnings.
Test: run warn.py with build.log
Change-Id: I11ff844bcfd7fb13aef8909307bc7eb1f5ee9b2a
2016-09-07 05:27:16 +00:00