Commit graph

94 commits

Author SHA1 Message Date
Josh Gao
9f9200c2ee Merge "versioner: extract strict warnings out of -v into its own flag." 2016-11-22 00:51:14 +00:00
Josh Gao
2796894c98 Merge "versioner: use a single work queue." 2016-11-22 00:50:59 +00:00
Josh Gao
acc3d80edf versioner: extract strict warnings out of -v into its own flag.
Bug: None
Test: versioner -v; versioner -s
Change-Id: I8f6ac7064c6f05de0811677f7fcfce929a2568c1
2016-11-17 18:12:21 -08:00
Josh Gao
338cf12963 versioner: use a single work queue.
Previously, each thread was assigned a fixed list of work, and the main
thread would block until every thread was finished, leading to most
cores sitting idle for the last few hundred milliseconds while a few
particularly long running threads would keep working. Use a single work
queue to evenly distribute load across the threads.

Bug: http://b/32748936
Test: python run_tests.py
Change-Id: I80e231ece3b95e2721a32f658905841b89a8dc3b
2016-11-17 18:11:57 -08:00
Josh Gao
3091f5a06e versioner: ignore non-header files.
Don't try to compile non-headers (vim swap files, etc.) that happen to
be in the header directory.

Bug: none
Test: manually created a file in libc/include
Change-Id: I2c1647ec7174dc617898dc2f07c9a80a08b69c72
2016-11-16 17:01:57 -08:00
Treehugger Robot
b3321c9614 Merge changes I72d37b7c,I7d254a10,I2e967acf
* changes:
  versioner: use a virtual filesystem for input files.
  versioner: cache -cc1 flags generated by the clang Driver.
  versioner: refactor to use CompilerInstance directly.
2016-11-15 00:54:09 +00:00
Josh Gao
78b8a1430d versioner: use a virtual filesystem for input files.
Use an InMemoryFileSystem to store and share input files across
compilations.

This improves the result of `time versioner` further, from:
    versioner  109.12s user 17.43s system 2433% cpu 5.201 total
to:
    versioner  112.20s user 1.38s system 2416% cpu 4.700 total

Bug: http://b/32748936
Test: python run_tests.py
Change-Id: I72d37b7c30850b8399cc40338247700fe3e7b2f9
2016-11-09 18:12:10 -08:00
Josh Gao
b5c496346f versioner: cache -cc1 flags generated by the clang Driver.
Profiling showed that majority of time spent by versioner was being
spent in the x86 and x86_64 driver statting random files in /usr/lib,
looking for a toolchain. Hardcode a list of per-target flags which
correspond to a subset of the ones generated by clang, and use those
instead of calling out to Driver.

This changes the result of `time versioner` from:
    versioner  156.57s user 1180.08s system 4109% cpu 32.525 total
to:
    versioner  109.12s user 17.43s system 2433% cpu 5.201 total

Bug: http://b/32748936
Test: python run_tests.py
Change-Id: I7d254a105bf1a283cbba0546321b04e308e703d1
2016-11-09 18:10:15 -08:00
Josh Gao
16016df79f versioner: refactor to use CompilerInstance directly.
This will make it easier to switch over to a virtual filesystem,
which should drastically improve performance.

This also fixes an issue with warning/error reporting.

Bug: http://b/32748936
Test: python run_tests.py
Change-Id: I2e967acf75db29c95f3a03f4f94cccd22c367ad5
2016-11-09 13:50:07 -08:00
Treehugger Robot
84e269d5af Merge "versioner: don't always hide future decls." 2016-11-09 02:07:17 +00:00
Josh Gao
9ae2b9a521 versioner: don't always hide future decls.
Bug: http://b/32748732
Test: python run_tests.py
Change-Id: I8f799015ebb1017f5e7d1324fb4c724e12a268bc
2016-11-08 16:28:07 -08:00
Treehugger Robot
147fdb4724 Merge "Clean up update_headers.sh." 2016-11-05 20:09:12 +00:00
Josh Gao
1a176de2f1 versioner: allow static inlines that don't conflict.
Bug: http://b/32664285
Test: python run_test.py
Change-Id: I9d050e545390eccd82661325c0ec58055f5f28a3
2016-11-04 13:16:29 -07:00
Josh Gao
a00e672361 versioner: fix output.
Bug: http://b/32664285
Test: python run_test.py
Change-Id: Ic487036a17cc6778493e316146e54ead8ce3e65b
2016-11-04 13:16:16 -07:00
Dan Albert
b89b7a5022 Clean up update_headers.sh.
Clean up some output, check for errors from the versioner.

Test: tools/update_headers.sh
Bug: None
Change-Id: Ib398b534a9a8ee6dc6a2b4d90747f628aa911bab
2016-11-04 11:41:14 -07:00
Dan Albert
ae50e9d85c Copy the NOTICE file when updating headers.
Test: ./update_headers.sh
Bug: None
Change-Id: Ibde1e6d7ef1750d8369b65b3125b55023efb5b2a
2016-10-31 16:18:29 -07:00
Pirama Arumuga Nainar
079be165a6 Make versioner build with LLVM r275480
Bug: http://b/31320715

Test: tools/versioner/preupload.sh
Change-Id: Icdff19b214b2b26c60288e49d26a88f04df68ef0
2016-10-25 09:35:20 -07:00
Elliott Hughes
dfb74c5f59 Minor copyright header unification.
Regenerating the NOTICE file the other day left me scratching my head at
various "how do they differ?" cases, resolved by this patch.

Test: N/A
Change-Id: I4230bfa1d7ec842a4b9fb8c73dd3b39443d74054
2016-10-24 12:57:55 -07:00
Dan Albert
c30862fdbc One NOTICE file to rule them all.
Generate a single NOTICE file rather than one per library. All the
headers come from libc these days anyway.

Test: tools/update_notice.sh
Bug: None
Change-Id: I127da185fdabc2815042e19aea74c507ec108f46
2016-10-20 11:48:55 -07:00
Dan Albert
32d5592a69 Update our notice files as a pre-upload step.
Also updates the license files, since apparently they're out of date.

Test: repo upload --cbr .
Bug: None
Change-Id: Ic8d855a7ee5185c4933da699292868e02ef79531
2016-10-20 11:27:50 -07:00
Josh Gao
0173379610 Add versioner to preupload hook.
Bug: http://b/31676510
Change-Id: Ic7cfad1cb214c9f711a985bace484a20db1a89b6
Test: submitted this patch, and it failed
2016-09-23 14:34:03 -07:00
Josh Gao
ac3e564142 versioner: turn on -Wundef.
Poking around with -Wundef exposed a pretty horrifying bug, so we
should probably keep it turned on.

Bug: http://b/31496165
Change-Id: Iada279eec91ded0d59ff3841d5a3ce1a36e34c3a
2016-09-15 14:21:28 -07:00
Josh Gao
bb96628f48 versioner: turn on -Wall -Wextra -Werror.
Change-Id: Ica47d0fa671934a1ea193869265de8986b012a58
Test: mma && versioner
2016-09-14 14:22:27 -07:00
Josh Gao
fff29fe17a versioner: add support for __VERSIONER_NO_GUARD.
Add an attribute that tells the preprocessor not to guard a
declaration, primarily for use with symbols that get reexported by
libc++ of the form `namespace std { using ::wctrans; }`.

Bug: http://b/28178111
Change-Id: I08c8751214797e37e8f26e7f7416a19e81c2bb4c
2016-09-07 18:34:39 -07:00
Treehugger Robot
061446c2df Merge "Add a script to update the bionic headers." 2016-08-16 17:08:46 +00:00
Josh Gao
a77b3a9464 versioner: handle _FILE_OFFSET_BITS=64.
Compile headers with both -D_FILE_OFFSET_BITS=32 and -D_FILE_OFFSET_BITS=64.

Bug: http://b/30170081
Change-Id: I92651e075cc69bdc1a2581f99892c9a7fdcdb35b
Test: python run_tests.py
2016-08-15 16:43:13 -07:00
Josh Gao
9ab148c450 versioner: add idempotence test that uses the bionic headers.
Bug: http://b/30170081
Change-Id: Iaf798077518a1a82c626c3b1524633c6df578409
Test: python run_tests.py
2016-08-15 14:28:25 -07:00
Josh Gao
9f7ce3d27f versioner: fix off-by-one error in header guard calculation.
This was causing redundant guards to be emitted in cases where
declarations were already guarded by another macro.

Bug: http://b/30170081
Change-Id: I9080838fbea5a56225df4b26d0918657fb6c4f21
Test: python run_tests.py
2016-08-15 14:09:53 -07:00
Dan Albert
8420be9bcc Add a script to update the bionic headers.
Test: ./update_headers.sh
Bug: http://b/30465923
Change-Id: I888a1ec0c4f26134845583a03bbaad88fa17afa0
2016-08-15 13:55:22 -07:00
Josh Gao
1b4b3d7a3c versioner: improve preprocessor tests.
Make sure everything works with relative/absolute paths, and with and
without trailing slashes.

Bug: http://b/30170081
Change-Id: Ia13a8ae2fa44590e835d0f4b006d4c31e9eaea83
Test: python run_tests.py
2016-08-15 13:42:36 -07:00
Josh Gao
a6b8c4e163 versioner: always copy over source headers when preprocessing.
Also, be less noisy when preprocessing.

Bug: http://b/30170081
Change-Id: If0d81b0a3a49756164e4945a20703afeb302fd4d
Test: python run_tests.py
2016-08-15 13:41:57 -07:00
Josh Gao
b0af100297 versioner: add test output directories to gitignore.
Bug: http://b/30170081
Change-Id: Ie493d8c321bb0e8d308234dec2d42ac8306c21c6
Test: python run_tests.py
2016-08-15 13:41:47 -07:00
Andreas Gampe
64da006063 versioner: Ignore if Clang is missing
This happens in some branches. Work around for now.

Change-Id: I75c01a3ae52ce89f6c56a19728903bf86db0a309
2016-08-12 19:10:21 -07:00
Josh Gao
79786347db versioner: copy unmodified headers when preprocessing.
Change-Id: I2517f560fd44fb519d6ee7e8d0a485036d3ecd87
2016-08-12 16:24:04 -07:00
Josh Gao
f8592a3ff3 versioner: add support for preprocessing headers.
Bug: http://b/30170081
Change-Id: I9b662ca9e0fa3a1f9c0211594e851f5f9bef7266
2016-08-08 16:39:07 -07:00
Josh Gao
8feba83242 versioner: fix makefiles.
Move the Android.mk to src/ so that mm in that directory works.
Also, remove the dependency on FORCE_BUILD_LLVM_COMPONENTS.

Change-Id: Iad7f3fc96f5f26f9535141d752ba865b40d5ff7e
2016-08-08 16:39:07 -07:00
Josh Gao
566735df35 versioner: purge iostreams.
Change-Id: I1b16a4b5c4a8a1333f05636c8c67890d8ce1a090
2016-08-08 16:39:07 -07:00
Josh Gao
16057886d7 versioner: assorted fixes.
Zero initialize the contents of ArchMap, keep track of the symbol name
in Declaration, remove unnecessary copies in loops.

Change-Id: I47b07755846f252b83ffc4c89547a34b2f7ab868
2016-08-08 16:39:07 -07:00
Josh Gao
bfb6bae9fb versioner: replace availability attributes with annotate.
Major refactor to use __attribute__((annotate)) to be able to keep
track of the semantic differences between __INTRODUCED_IN(x) and
__INTRODUCED_IN_X86(x), for use in the upcoming preprocessor.

Bug: http://b/30170081
Change-Id: I6496a8c40ba7f4553de9a2be0bbddcf37c813937
2016-08-08 16:39:07 -07:00
Josh Gao
681f6b59d2 versioner: improve the test script.
Make it actually run under python3, and add checking for unexpected
success.

Change-Id: I9aac0e8cc1526fd63db21e57554f3aa5a9b4091d
2016-08-08 16:39:07 -07:00
Josh Gao
f9128bc7d4 versioner: remove unnecessary kernel/common symlink.
The target was moved by commit 5956b4e, but none of the headers
contained were actually necessary to compile any bionic headers.

Bug: http://b/30170081
Change-Id: I8c1a7464de0992c20af194340c0e36d131a836e7
2016-08-02 14:32:42 -07:00
Josh Gao
180c883f09 versioner: add symlink for kernel/android/uapi.
The android-only kernel headers were moved to their own directory by
commit d6e8b8c. Add a new symlink to match.

Change-Id: I06ce7c3167fbb1e058e4a137fae9375bb5792a46
2016-08-02 14:32:42 -07:00
Colin Cross
0e25b3425b Fix clean mma in bionic
Change-Id: If2410c5ff9c08372098e5ec831e33e4da0e04b30
2016-07-14 16:53:18 -07:00
Colin Cross
d3b6bc7624 Move relocation_packer from Android.mk to Android.bp
Change-Id: I3d24f8513fe3ca19900705d4861ab797464e0930
2016-07-14 09:07:13 -07:00
Josh Gao
d8c77257ea versioner: whitelist atexit, turn on symbol checking by default.
Change-Id: I32e726c74ee618ace3a4329d46408a42732a8d9d
2016-06-03 15:01:19 -07:00
Josh Gao
4af829acb7 versioner: add missing test.
Change-Id: I1dc9a708b53dbb46af9e4b8ab69bf8ed46ab045f
2016-06-03 15:01:19 -07:00
Josh Gao
958f3b31c4 versioner: fix false positive with functions only available as inlines.
Change-Id: I09cc335b4006c6ceafcbd1bec9e50161f8262942
2016-06-03 15:01:19 -07:00
Josh Gao
173e7c0753 versioner: improve error output slightly.
Print [introduced = 9, deprecated = 10, obsoleted = 11] instead of
[9,10,11].

Change-Id: Ifb8a66abbcec92aa13086d220af7ee6fa17b0897
2016-06-03 15:01:19 -07:00
Josh Gao
658dbd920d versioner: merge stdout and stderr in the test runner.
Some of the error messages emitted by versioner (the ones where it was
invoked incorrectly) go to stderr, which meant that the test runner
ignored them. Merge stdout and stderr, and switch from testing for
exact equality to endswith, because of the compilation errors test.

Change-Id: I0e2c25bcc9dea4c12ea82a6a05b29e561a61a902
2016-06-03 15:01:18 -07:00
Josh Gao
80d909bbfb versioner: clean up tests, test runner.
Git doesn't track empty directories, so most of the tests would fail on
a fresh checkout because of dependencies/common being missing. Remove
the use of dependencies from all of the non-dependency related tests.

Change-Id: I09cc5765aae1576914c1c5d7dfa3fb666eab4a3f
2016-06-03 15:01:18 -07:00