Commit graph

76 commits

Author SHA1 Message Date
Chih-Hung Hsieh
98b285dafb Fix/suppress most pylint and gpylint warnings
* Add missing function doc strings.
  Suppress this warning on trivial functions in *_warn_patterns.py.
* Remove unused g-importing-memeber, g-complex-comprehension.
* Suppress pylint warning on unrecognized g-* options.
* Suppress too-few-public-methods warnings on simple classes.
* Suppress too-many-arguments and missing-function-docstring in
  html_writer.py, which will be refactored later.
* Fix bad naming, long lines and line breaks, and bad quotes.

Test: compare output for build.log
Change-Id: Icdb34f014a10ec1e642c2cfe8003fc3ae245b507
2021-04-28 14:57:56 -07:00
Chih-Hung Hsieh
7cc0e15e4a Use python3
Test: warn.py build.log > warnings.html
Change-Id: I781ea7bfe023ec5af829757c417bc70aba80af0d
2021-04-26 17:09:36 -07:00
Chih-Hung Hsieh
888d143e4c Split warn.py into multiple files
* Split warning patterns into *_warn_patterns.py
* Split project list into android_project_list.py
* Split out the Severity class, to be changed later
* Split core of warn.py into warn_common.py and
  leave only platform dependent code in warn.py.
  This allows the core logic be used with different
  parallel processing libraries.
* Old warn.py just calls -m warn.warn.

Test: path_to_build/tools/warn.py build.log
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I6734e4472a21018cd5ce06d549f6dbca24f4de54
2019-12-17 15:12:31 -08:00
Stephen Hines
aa94811b2a Merge "Guess source tree root from common file prefix." 2019-10-18 17:59:23 +00:00
Chih-Hung Hsieh
83980501d9 Fix severity of analyzer warnings; add new warnings.
* The following were high severity warnings but mistakenly
  grouped into analyzer warnings in a recent change.
        clang-analyzer-unix.Malloc
        clang-analyzer-cplusplus.NewDeleteLeaks
        clang-analyzer-cplusplus.NewDelete
* Add new Protoc and Kotlin warnings about unused imports and variables.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Ied1f9ace97f29fba58a40dff2346a0391ab22f06
2019-09-26 12:14:32 -07:00
Chih-Hung Hsieh
790020437d Factor out common code in Java warn patterns.
* Some comments are kept to reduce diff artifacts.
* Warning patterns will be separated by tools later,
  to make it easier to share code with Chromium and
  other build tools.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I6b589478cf5ae73940b4712d9faabd0909546f02
2019-09-25 13:39:41 -07:00
Chih-Hung Hsieh
d591a5e012 Add new compiler warning patterns
* Add Kotlin, NDK, Protoc, Asm, and other new patterns.
* Put all clang-analyzer warnings to the ANALYZER severity group.
  Some security and null pointer warnings still have HIGH severity.
* Comment out old C/C++ warnings that overlap with analyzer
  warnings and do not appear in current build.log.
* Other old fixed warning patterns should be commented out later.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Idc7b09e5e4eefa127d41e9f37da1f566ed1ede1f
2019-09-24 14:52:09 -07:00
Chih-Hung Hsieh
f5db8985f5 Guess source tree root from common file prefix.
* When processing a downloaded build.log file,
  find_android_root is very slow because the
  android source tree is not found.
  Change it to look for warn.py only up to 100 times,
  and then set android_root to the longest common path prefix.

Bug: 141524232
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I8cc2e208d439267313aaa6cdbc00f40320409c63
2019-09-24 18:19:17 +00:00
Chih-Hung Hsieh
a1187072c5 Handle rustc warning messages.
* Combine the first two lines of rustc warning messages.
* Give some rustc warnings HIGH and MEDIUM severity.

Bug: 141311294
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Ia3a206d68d9e92934296487b006a7b424044a822
2019-09-20 17:14:22 -07:00
Elliott Hughes
36d86c1670 Remove perfprofd reference.
Dead and gone.

Test: treehugger
Change-Id: I8575fd9efaa371d61fac5204ee4d982e985a9a56
2019-07-17 08:16:29 -07:00
Chih-Hung Hsieh
9018ea49d8 Fix pylint warnings and now prefer python3
* Read/write warning messages as utf-8 strings.
* No more embedded raw utf-8 code in file.

Test: ./warn.py --csvpath warnings.csv build.log > warnings.html
Change-Id: Ie51700721a32bede1d3b250f4a42fd8facd6af75
2019-06-24 15:24:39 -07:00
Andreas Gampe
161ec1ecc1 Build: update warn.py for Errorprone 2.3.2
Bug: 114129741
Test: manual
Change-Id: I7fb34881a623ffb115a92bfffb108b503d4645ad
2018-10-15 08:56:58 -07:00
Chih-Hung Hsieh
e1672866a7 Recognize -Wimplicit-fallthrough.
Test: warn.py build.log
Change-Id: Ie55248f7348b20d09909a856fe7581a0476b9d9b
2018-08-31 16:19:19 -07:00
Christopher Ferris
e027dd4aef Remove micro_bench.
micro_bench has been deprecated, so remove it from the list of executables
to be built by default.

Bug: 26928775

Test: Builds.
Change-Id: I701b3077ec00c7e75145aeb52463f47bf8ff2f09
2018-08-07 19:57:56 +00:00
Chih-Hung Hsieh
b2afb63ac9 Escape special HTML characters in warning messages.
Bug: 111699078
Test: run warn.py with a build.log file
Change-Id: I553dad57a6d81ea6cc89289741d9963f49cc3b47
2018-07-20 15:36:26 -07:00
Andreas Gampe
b9dc23a4d8 Tools: Update warn.py for Errorprone 2.3.1
Note: this uses the default severity in Errorprone, modulo
forced-ERROR settings in Android.

Bug: 72971960
Test: manual
Change-Id: I6d8b855b88289b763b2a98e8685627bf1163989d
2018-06-08 10:24:35 -07:00
Andreas Gampe
2e987af407 Tools: Sort Errorprone warn.py entries by name
To improve updateability.

Bug: 72971960
Test: none
Change-Id: I7ab7cdfe53518b29d036f418fe66aef69b5e409a
2018-06-08 10:22:59 -07:00
Chih-Hung Hsieh
9be27764c0 Fix gpylint warnings.
* This file should be gpylint clean so it can be easily copied
  and pass presubmit checks in google repository.

Test: gpylint warn.py
Change-Id: I5391f19c7a0141bc211f5b3924f48b345bcb03ab
2018-06-07 10:51:47 -07:00
Treehugger Robot
991bbccf95 Merge "Remove references to common_time" 2018-06-05 10:33:13 +00:00
Chih-Hung Hsieh
9e9355dac6 Recognize more clang-tidy compiler warnings.
* Some are from clang compiler and Android makefile checker.

Test: run with build.log
Change-Id: I371f08f3ae956e9425515b2cd67c5af3d0425b89
2018-06-04 11:32:32 -07:00
Neil Fuller
ce483225bf Remove references to common_time
The code is unused and is being removed.

Bug: 80462439
Test: build / boot
Change-Id: I812bff4fdc061b060633949b7087ab9c951fa7e0
2018-06-04 15:38:21 +01:00
Andreas Gampe
cf528ca79d Warn.py: Update errorprone categories
From current tip-of-tree.

Bug: 72451126
Test: build/make/tools/warn.py errorprone.log
Change-Id: I1d632fb8801b74af695b4f7f464a99792e46637a
2018-01-25 12:25:20 -08:00
Andreas Gampe
7f494c121f Warn: Change GuardedBy regular expressions
This was modified in commits 20a8664 "Change canonical name for
GuardedByChecker to GuardedBy" and 1f6dc3dc "Merge GuardedByValidator
into GuardedByChecker."

Test: m
Test: manual
Change-Id: I07305e59694a955e52bd69c1cbe53346808b78aa
2018-01-17 21:28:21 -08:00
Colin Cross
fdea893833 Replace build/tools with build/make/tools
Test: m checkbuild
Change-Id: I8db6c27a6d2372f142d10568fe92d6f2e9a59252
2017-12-06 16:12:40 -08:00
Chih-Hung Hsieh
2cd467b36c Recognize new clang-tidy android-* warnings.
Test: warn.py build.log
Change-Id: I9bcd1fffcb567346a3d959f98821e2cf58e87ba2
2017-11-16 15:42:11 -08:00
Chih-Hung Hsieh
655c542dc8 Find target keywords in the first 100 lines.
Bug: 62416199
Test: warn.py build.log
Change-Id: I8c5ef310758fdd44519aeb1efe2f0bd92fe1ac87
2017-06-07 23:17:13 +00:00
Chih-Hung Hsieh
8ef1aae6d3 Recognize new clang and make warnings.
Test: warn.py build.log
Change-Id: I078b3cc35a00590312208cb90af0a1f2431abf09
2017-05-18 14:29:59 -07:00
Chih-Hung Hsieh
ef21d1458b Get Android TOP from build.log file.
find_android_root works only when warn.py is found
so it works for build bot, but not when warn.py is
run against a stand alone build.log file.
Now get the TOP directory path from build.log
if it is found.

Bug: 37725427
Test: warn.py --gencsv build.log
Change-Id: I7b776c2e71a8d8d769f0600b08abe37acbd6ce7c
2017-04-27 10:25:37 -07:00
George Burgess IV
169f5f1c6c warn.py: have URLs open in new tabs
Bug: None
Test: Ran warn.py on a build log. Code links in resultant HTML open in
new tabs when clicked.

Change-Id: Ia3def5ea14fe216700cb2ffabe4a39b98e69d55a
2017-04-21 15:19:36 -07:00
Sam Saccone
03aaa7e2d8 Add the ability to save off the CSV to a file.
While we are generating the html file also expose a way for a user to
save off a csv file. This prevents having to run the tool twice.

* Add a new flag called csvpath that takes a file path for the csv dump.
* Use csv.writer to simply print formats.
* Modify the dump_csv and count_severity to take a csv writer.

Test: run with a build.log file, compare before and after html/csv dumps.
Change-Id: I27ed730625a352fdc3fb2928433eb5e8b2ea229c
2017-04-12 18:58:53 -07:00
Nick Glorioso
8a7b25c736 Update warn.py for new Error Prone diagnostics
Test: Local build, tee'd to file, ran warn.py and checked output HTML
Change-Id: Iee3386e88c73e4b8e0a1fcc60105ee5fb86e3cb0
2016-11-22 10:35:07 -08:00
Chih-Hung Hsieh
76d0065ef8 Separate parallel_classify_warnings out of parse_input_file.
* Allow other callers of this module to:
  * pass any input stream to parse_input_file,
  * pass any warning_lines to parallel_classify_warnings,
  * call dump_csv or dump_html to get output.
* No output change.
* Capture and ignore signal.SIGTERM at the end,
  to avoid bad warning/error messages from the exit clean-up process.

Test: run warn.py build.log.
Change-Id: I1414797a536c0ee622e2a34c226578621be1ddab
2016-11-09 18:19:05 -08:00
Chih-Hung Hsieh
63de300094 Do not use multiprocessing.Pool when --processes=1.
Purpose of this change is not to save some execution time,
but to avoid forking another process. In other applications
that use a wrapper to call this python script, it is difficult
to get overwritten file I/O functions work in a subprocess.
So the wrapper will call warn.py with --processes=1.

Test: run "warn.py --processes=1 build.log"
Change-Id: I5998d5c70d81a456c86eb4002f444a4a60135477
2016-10-28 10:53:34 -07:00
Treehugger Robot
92396e1747 Merge "Use parallel subprocesses to classify warnings." 2016-10-18 03:49:49 +00:00
Chih-Hung Hsieh
a6bd044ac1 Use parallel subprocesses to classify warnings.
* Add a --processes flag to specify number of parallel processes,
  with default multiprocessing.cpu_count().
* Wrap long line to suppress pylint warning.

Test: run warn.py with a large build.log file.
Change-Id: I9a93a9324bc531c1bce741367013051ce40a67fa
2016-10-17 17:47:20 -07:00
Stephen Hines
d0aec89d43 Split apart frameworks/compile into sub-projects.
These projects are all used by RenderScript, but it would be good to
differentiate warnings for each.

Change-Id: I983279d590630175a36075b01e8eec9100c07d26
Test: Ran warn.py
2016-10-17 15:40:38 -07:00
Chih-Hung Hsieh
144864eed0 Add more patterns for a finer project warning list.
Bug: 31377083
Test: run warn.py --byproject build.log
Change-Id: I3cad8e24bc97a84f067a162ff728e30fde3c7197
2016-10-14 12:27:17 -07:00
Treehugger Robot
6949e7ecba Merge "Better pattern matching and dump of warning messages." 2016-10-12 16:35:34 +00:00
Chih-Hung Hsieh
5722f9247b Better pattern matching and dump of warning messages.
* Warning messages must start with source file path.
  This will reject lines containing only 'warning:'
  but not a source file path prefix.
* Escape both backslash and quotation marks in the
  dump of strings to JavaScript string literals.

Bug: 32060052
Test: run through build.log files
Change-Id: Ib064768b1ba2954f974604ea054f7a6d2ad15ae7
2016-10-11 15:33:19 -07:00
Chih-Hung Hsieh
a45c5c1cf9 Normalize warning source file path to relative path.
Remove Android root directory path from long
absolute source file path.

Test: run warn.py with build.log
Bug: 32059187
Change-Id: Ib630cebd8c56a9fef443139ca502d97e59c7b42e
2016-10-11 15:30:26 -07:00
Chih-Hung Hsieh
1eabb0ecb2 Add a new ANALYZER severity group.
* Separate clang-tidy's clang-analyzer-* warnings from
  other clang-tidy warnings.

Bug: 31559947
Test: build with WITH_TIDY=1 and DEFAULT_GLOBAL_TIDY_CHECKS=*,-readability-*,-google-readability-*
Change-Id: If03c421c9319741ef6588bc0d803b6da0cc4433a
2016-10-05 11:53:20 -07:00
Treehugger Robot
312c310f7c Merge "Only run warn.py's main function if it is the main program." 2016-09-29 03:58:07 +00:00
Meike Baumgärtner
ac9d5df349 Only run warn.py's main function if it is the main program.
Test: run warn.py --byproject build.log
Change-Id: I155cce8c952d6d8a02c8e1809bf61a46c847c26e
2016-09-28 17:59:08 -07:00
Chih-Hung Hsieh
9f766239f0 Fix pylint warning and refactor project_list.
Bug: 31377083
Test: run warn.py --byproject build.log
Change-Id: Id442cf6608bf5c14784063c05f5ff0b731f17486
2016-09-27 21:13:52 -07:00
Chih-Hung Hsieh
b426c54e8a Fix Google pylint warnings.
* Rename 'severity' to 'Severity'.
* Fix bad line indentations and space.

Test: run warn.py --byproject build.log
Change-Id: I33d701cca408b5ca4715be8d44c890942b7d132a
2016-09-27 18:15:40 -07:00
Chih-Hung Hsieh
2a38c37d57 Add buttons to group warning by project or severity.
* Add more project patterns.
* Add more top level comments for global variables and functions.
* Resequence severity numbers to match the dump order.
* Emit warning messages and tables to static HTML JavaScript arrays.
* Replace old static HTML table dumper functions with
  new dynamic HTML JavaScript to generate sections of warnings.
* Warning messages are grouped into sections by severity or projects.
* Better descriptions for SKIP warning patterns.
* Replace output function with print.

Bug: 31377083
Test: run warn.py --byproject build.log
Change-Id: I7b44ef6223d5b2f1aa31655a5a47d854f9a1dedc
2016-09-26 16:02:12 -07:00
Treehugger Robot
bcd229277a Merge "Fix Google pylint warnings." 2016-09-26 21:18:08 +00:00
Chih-Hung Hsieh
6c0fdbb84d Fix Google pylint warnings.
* Rename variables and functions to follow Google coding style.
* Fix line indentations to follow Google coding style.
* Add module and function docstrings.
* Change shebang line to avoid pylint warning.
* Suppress some pylint warnings to avoid too many changes at once.
* Fix typos.

Test: run warn.py --byproject build.log
Change-Id: I569961981b562e35cb5609a96cd5ebb40a80829c
2016-09-26 12:24:54 -07:00
Chih-Hung Hsieh
0a19207f7b Recognize new warning messages.
* -Wunnamed-type-template-args and -W#pragma-messages

Test: run warn.py with build.log.
Change-Id: I36277e001081604f92a4d9c92f9a0b5d3f0ee5e5
2016-09-22 17:44:38 -07:00
Chih-Hung Hsieh
99459fc0d0 Simplify warn.py and output of warning count table and tablerow.
* Use a loop to initialize empty 'members' and 'option' of warnpatterns.
* Create severity.{color,columnheader,header} from an attributes list.
* Compute totalbyproject and totalbyseverity from a constructor.
* Skip all-zero rows and columns in the warning count table.
* Remove redundant spaces and newlines in tablerow output.

Test: run through build.log files
Change-Id: I4d3fa4ecd92e1afab91d85b7535d03f8696e83d8
2016-09-22 15:15:43 -07:00