Commit graph

65 commits

Author SHA1 Message Date
Mark Salyzyn
01bdb04be6 liblog: allow event tags to include some punctuations
event_log_tag parser complains about a period (.) in the name,
we would consider such an enhancement to the tag names possible.
I expect we would want to be able to support alphanumerics,
underscore (_), period (.), minus (-), at (@) and comma (,) for
starters as they are present in the other text log buffer tags.

We introduce a local endOfTag function that is used during parsing and
during android_lookupEventTagNum for submitting new tags.  This
function caused us to enforce const char more closely.  By filtering
in both places we resolve an issue that could have plagued us if
garbage requests were made.

Test: gTest liblog-unit-tests, logd-unit-tests & logcat-unit-tests
Bug: 31456426
Change-Id: I596b8706e843719ddac07ec40e1cd2875c214bed
2017-04-14 12:56:06 -07:00
Mark Salyzyn
5768d3d976 liblog: logprint supports number of seconds time event field
Add s to report time in seconds.  The time could be a period, duration
or monotonic, expanded to seconds, minutes, hours and days.  gTest has
to acquire a dynamic tag allocation as there are no users of this
feature yet.

Looking to the future, audio media logging has binary content similar
to the binary events structures Android logging uses and they have
a definition of a duration field in their internal binary logging, so
may be of use when we unify the logs.

Test: gTest logcat-unit-tests --gtest_filter=*.descriptive
Bug: 31456426
Change-Id: I262c03775983b3bc7b1b00227ce2bb2b0f357bec
2017-04-14 12:54:25 -07:00
Mark Salyzyn
46186a7238 logcat: test: standardize rest() to let logs land when injecting
Add an internal rest() function to set the standard for how long we
should wait for a log message to land end to end into the logs.  Add
a retry on -EBUSY for all descriptive tests along with a rest() after
log injection so the tests can try to survive heavy Denial Of Service
(DOS) loads.  Some of the resting was done at a coarse 1 second, when
200ms will do just fine.

Fix a few pesky issues surrounding test reliability and correctness:
- stdint.h include missing.
- missing a logcat_executable, means logcatd.descriptive testing was
  testing logcat instead.
- count of interfering tests is 3 now: logcat, liblogcat and logcatd, if
  they all run in the same PID, we have to not fail.
- Added a missed opportunity to add uniqueness to logcat.descriptive sync
  test.

Test: gTest logcat-unit-tests
Bug: 31456426
Change-Id: Iaf2bee97878957f654613bb2e78e32379bb8d1b7
2017-04-14 12:50:59 -07:00
Mark Salyzyn
a073c39616 logcatd: introduce logcatd executable
logcatd is the same as logcat, except that the -L flag, if supplied,
runs once, then the command re-runs itself without the -L flag with
the same argument set.  By introducing a logcatd daemon executable
we can solve the problem of the longish reads from pstore that
sometimes occur when the system is excessively busy, starving this
background cgroup daemon as we absorb the delay in a backgrounded
init "service", rather than in a forgrounded init "exec".  This would
not have been efficiently possible without the introduction of
liblogcat.

There are no doubt many flags that make no sense to run twice with,
and without, the -L flag.  In the general sense we expect the caller
to perform the correct set of operations and not pick these nonsense
operations.  logcatd is only supplied on engineering and debug builds
for logpersist, and is only an automated aid to triage.

Test: gTest logcat-unit-tests
Test: manual confirm logpersist functions as expected, required reboot
Bug: 28788401
Bug: 30041146
Bug: 30612424
Bug: 35326290
Change-Id: I53ba31970749daf37eef42636f039f485932416f
2017-04-04 08:01:50 -07:00
Mark Salyzyn
e74e51ded7 logcat: Add -h and --help flags
If -h or --help argument was supplied, logcat would report the help
message but also return an error.  Officially add -h and --help as
recognized flags, report all help with a (zero) success error code.
Adding this, and the associated test, was split off as a stepping
stone to resolving an issue with logcat -L or --last flag operations.

Test: gTest logcat-unit-tests --gtest_filter=*.help
Bug: 28788401
Bug: 30041146
Bug: 30612424
Bug: 35326290
Change-Id: I948e7fa4e92bd23f52717758ffd96bbd068c53d4
2017-04-04 08:01:11 -07:00
Mark Salyzyn
fbc0fe429b logcat: test: run 256 simultaneous logcats
For logd daemon, heavy reader stress. For system popen fork and
execute of logcat measure baseline against liblogcat. For liblogcat
local concurrent thread, locking, argument parsing and context scaling
for popen-style pair of android_logcat_run_command_thread* functions.

NB: 1000 logcat executables ran, but did not scale well on time
    blocked for more than a minute. With 343 contexts of
    android_logcat_run_command_thread ran out of local resources
    to even return a file descriptor, and the per-context event tag
    mappings coincidentally ran out at 340 when threads ran, although
    that path was consistently 15% faster than the popen test.

Test: gtest logcat-unit-tests --gtest_filter=*.End_to_End_multitude
Bug: 35326290
Change-Id: I0e1a5d4f8ffbd77fa8db13d53f4d328973731895
2017-03-27 11:57:53 -07:00
Mark Salyzyn
d70f246f8a logcat: test: tail_# occasional failures
When PID becomes 5 digits, fgetLongTime fails

Test: gTest logcat-unit-tests --gtest_filter=*.tail_*
Bug: 27319267
Change-Id: I9cc636ed9955a317eb3b0ff0a820a50b800b7e0d
2017-03-14 15:35:15 -07:00
Mark Salyzyn
210095136d logcat: test -v epoch tail time
Test: gTest logcat-unit-tests
Bug: 35373582
Change-Id: I57d84af2eb5d1fce8753539edf3133bfe0bb7213
2017-03-13 16:29:04 -07:00
Mark Salyzyn
71919f4f5d liblogcat: measure performance
Existing dummy benchmark converted to a single-pass benchmark.

Add support for ">[filename]" interpretation in liblogcat, helpful to
provide more consistent benchmark results when sending output to
/dev/null.

NB: results are not consistent as there is a roundtrip to logd
    service, but they are indicative of better performance for
    liblogcat.

Benchmark                                Time           CPU Iterations
----------------------------------------------------------------------
BM_logcat_stat_popen_libc         35221677 ns     603314 ns        100
BM_logcat_stat_popen_liblogcat    15871805 ns     579398 ns       1287
BM_logcat_stat_system_libc        39037825 ns     319882 ns        100
BM_logcat_stat_system_liblogcat    9896755 ns     762610 ns       1310
BM_logcat_dump_popen_libc        776206254 ns  131448772 ns          5
BM_logcat_dump_popen_liblogcat   610674905 ns  118709624 ns          6
BM_logcat_dump_system_libc       512191254 ns     388724 ns         10
BM_logcat_dump_system_liblogcat  443897570 ns  377215696 ns          2

Test: logcat-benchmarks --benchmark_filter='BM_logcat_stat*|BM_logcat_dump*'
Bug: 35326290
Change-Id: Ie94900a3070487f45742f6c0789b5b3cf036df34
2017-02-22 16:06:24 -08:00
Mark Salyzyn
eb40816fb8 liblogcat: LOCAL_C_INCLUDE_DIRS correction
Test: compile
Bug: 35326290
Change-Id: I62b46443a6b9149577e85d4f512c3946b467a1fb
2017-02-21 08:48:29 -08:00
Mark Salyzyn
6dabc81a0b liblogcat: add android_logcat_popen and android_logcat_system
Supply a wrapper to the logcat API that provides some analogous
functionality to popen and system libc calls with some bits of
KISS shell-like parsing for environment, quotes and error
redirection handling.

Test: gTest logcat-unit-tests
Bug: 35326290
Change-Id: I9494ce71267ad2b2bec7fcccfc7d4beddae9aea6
2017-02-17 14:42:14 -08:00
Mark Salyzyn
c0cf90d1fb logcat: Create liblogcat
Try to leverage as much of logcat as-is and produce a viable
library API that others can use for their own logcat execution.

Added a test to check ANDROID_PRINTF_LOG environment variable.

Test: gTest logcat-unit-tests
Bug: 35326290
Change-Id: I30de692ea9d83e6fd6e5d9e7cf93d31401a88a40
2017-02-17 14:42:14 -08:00
Mark Salyzyn
5f606605ed logcat: Add coding style
- Android coding standard compliance with an eye to reducing merge
  impact.
- resolve a few misbehaviors in logcat_test.

SideEffects: none
Test: gTest logcat-unit-tests
Bug: 35326290
Change-Id: I63d0667ad34c0df11086a6ffe94b7030430b865b
2017-02-17 14:42:14 -08:00
Mark Salyzyn
9b4d7e1c38 logcat: test: inject() tuning
Add a hidden -v nsec flag to logcat (actually logprint in liblog.so)
so that we can do more exacting tests of logcat.tail_time.  Halve the
spam pressure of logcat.tail_* in inject() routine, and give us a few
more retries at the higher counts.  Add instrumentation to
logcat.logrotate failures.  Add inject for logcat.year test.

Test: gTest logcat-unit-tests
Bug: 34454772
Change-Id: If6f3bd21892c8a2b9ccee8c8bbf592a1ae0b2a57
2017-01-30 15:42:56 -08:00
Mark Salyzyn
e7e21ea67f logcat: test: increase retry on fgetLongTimestamp to 4, and sleep to settle to 3
Test: gTest logcat-unit-tests
Bug: 34454772
Change-Id: I8557b0f7f25c4aa160a23a2428c9b53c8e3b26e1
2017-01-25 15:29:29 -08:00
Mark Salyzyn
15e585853f logcat: test: inject messages if necessary for tail test requirements
For the gTests logcat.tz, logcat.tail_3, logcat.tail_10,
logcat.tail_100, logcat.tail_1000 and logcat.tail_time tests inject
messages if we come up short should the background logging not be
sufficient to feed the test requirements.  Test frames run the
tests right after reboot requiring injection.  Will have radio
silence, so we also add kernel logs if available to help add to the
background logging activity level.

We also will inject a radio message for logcat.buckets so that test
can survive no radio content.

Test: set device to airplane mode (to stress this problem) then:
      adb logcat -b all -c && adb shell su root \
          /data/nativetest/logcat-unit-tests/logcat-unit-tests \
          --gtest_filter=logcat.tail_1000 (or others in this set)
Bug: 34454772
Change-Id: I5ec246552f3ab1fc9c5864ed69d63b851fdf538d
2017-01-23 13:33:02 -08:00
Mark Salyzyn
26a1facfbf logcat: do not report security buffer errors
Do not report security buffer errors if not specifically
named in the buffer list.

Test: gTest logcat-unit-test --gtest_filter=logcat.security
Bug: 34511645
Change-Id: I028d51abad0329fcf42e467b135d035b06c1d2e3
2017-01-20 15:30:34 -08:00
Mark Salyzyn
a46326a51c logcat: unit test produces log stutter
logd now counts multiple identical messages, causing messages to be
held back.  Stop doing that in the tests.  When that filter appeared,
the logcat-unit-tests went from 23 seconds to a couple of minutes of
runtime breakign the automated testing.

Test: gTest logcat-unit-tests total time less than 1 minute.
Bug: 33845198
Bug: 33535908
Change-Id: I6145bae89b6d5e5024ef4f8fe66440c9ae53f7c6
2016-12-27 19:13:00 +00:00
Mark Salyzyn
472245d962 liblog: move android_log_event_context class to log/log_event_list.h
rename class from android_log_event_context to android_log_event_list

Test: gTest logcat-unit-tests
Bug: 31992412
Bug: 31456426
Change-Id: Ib61cbca7d453837d64959c56b0e11f8c5edbfbdd
2016-11-21 09:46:34 -08:00
Mark Salyzyn
1a57ae3a7d liblog: logprint: report truncated event log contents if error
We need to accept that a log tag can contain no payload.  For those
that are corrupted, and to aid debugging, report what we did manage
to interpret.  Report last character as a ! for corruption, and ^
for truncation.  Fix a few Android Coding standard issues.

Test: gTest logcat-unit-tests
Bug: 32903864
Change-Id: Id11bef3a7b6569305c51701dd66c45d2038d6628
2016-11-15 14:46:34 -08:00
Mark Salyzyn
4fd0507b20 liblog: logcat: logprint support -v descriptive
Expand logprint feature to pull out the log tag description
fields, parse them and merge into the logging content.  Add
-v descriptive, -v colour(british, hidden) and -v help. Also
added a unit test for the descriptive format borrowing from
event tags that have been unchanged since 2009.

Had to add -v help because we have too many undocumented
formats and format adverbs.

Test: gTest logcat-unit-tests --gtest_filter=logcat.descriptive
Bug: 31456426
Change-Id: I93a1c003b7a3f4c332544946fdedb7277919cec3
2016-11-04 15:22:52 -07:00
Treehugger Robot
9fa133cc28 Merge "fix regression from android_lookupEventTag_len()" 2016-10-24 23:08:47 +00:00
Mark Salyzyn
9f53cac1b4 fix regression from android_lookupEventTag_len()
Commit 807e40ecc9 'liblog: logd: Add
android_lookupEventTag_len()' which addressed a Dirty Shared memory
leak resulted in a regression. Most notably logcat <tag> stopped
working for the events log buffer.

AndroidLogEntry::tag also requires callers to check out
AndroidLogEntry::tagLen as tag is no longer guaranteed to be
nul terminated.

Test: logcat-unit-tests --gtest_filter=logcat.event_tag_filter
Bug: 31456426
Change-Id: Ibe5236131b640eb5b7e3df0ab4b5f3e25b85ad45
2016-10-24 13:39:19 -07:00
Mark Salyzyn
aeaaf81c2c liblog: logd: logcat: Split out log/logger.h into public and private.
log/logger.h pieces moved into log/log.h.  Correct for some
minor Android Coding standards.

Test: gTests liblog-unit-tests, logd-unit-tests and logcat-unit-tests
Bug: 19235719
Bug: 26552300
Bug: 31289077
Bug: 31456426
Change-Id: I0a19fd8788eec20a582e72e4c62c04534bdb1b9a
2016-10-24 11:12:49 -07:00
Treehugger Robot
955648a915 Merge "Rely on the platform -std default." 2016-10-14 02:28:38 +00:00
Mark Salyzyn
56cddcd72e logcat: test: report system(command) details
Provide more details regarding test failures

Test: manual, forced a failure, reports actual command issued
Bug: 30566487
Change-Id: I2431dbd335685b5eaef54ba6bd688b8588018aa9
2016-10-12 12:10:08 -07:00
Elliott Hughes
36e0d390a2 Rely on the platform -std default.
Bug: http://b/32019064
Test: builds
Change-Id: I18a1d816d63b64601485045070851f32d44e85eb
2016-10-10 14:31:12 -07:00
Mark Salyzyn
0dd4431072 logd: logcat: Replace log/log.h with android/log.h
Should use android/log.h instead of log/log.h as a good example
to all others.

Test: Compile
Bug: 26552300
Bug: 31289077
Change-Id: If4c9711eb57267d4707b03d54a932af9de241b13
2016-09-30 12:47:05 -07:00
Mark Salyzyn
004cd3c55d liblog: logd: logcat: deprecate log/log_read.h
Always used in combination with log/logger.h except in log_time.cpp,
and not used externally.  As a result liblog has to support stl, a
small price to pay since goal is to convert liblog to C++ internally.

Test: compile
Bug: 31456426
Bug: 26552300
Bug: 31289077
Change-Id: I72828ec807d0a2c8e40bbdebd7a69f147a7ca5a9
2016-09-30 12:47:05 -07:00
Mark Salyzyn
02687e75b0 logcat: add --id=<id> for -f option
Clear file rotate content if id signature changes. This allows
logcatd to detect change of property ro.build.id to reset the
recorded data. The id is stored in the the -f filespec as
<filespec>.id.

If the id file is missing or unreadable, do not clear. Allow an
API where chmod 0 to the id file blocks this feature. Does not
work for root.

Add logcat.logrotate_id gTest.

Bug: 30591615
Change-Id: I895749ca7c6d4af57ec57f058a29d8e3af2f8b27
2016-08-08 10:19:48 -07:00
Mark Salyzyn
b7d059bb2f logcat: clear when specifying file output
If -c and -f are both specified, clear the files rather
than notifying the logger to clear its data. The -f then
acts like a switch between clearing the in-memory log
data, or the on-disk logrotate data.

Bug: 28936216
Change-Id: Ib1d1fb46ea09f81a2fd9bebb6c8f0f9f2355f6e8
2016-06-10 18:10:01 +00:00
Mark Salyzyn
1325ebfab2 Revert "logcat: expand -n, -r and -b"
This reverts commit 33c262513f.

Bug: 28120456
Bug: 22654233
Change-Id: I2a13f009d9e08dc2389b9872e45e95e0563af22f
2016-06-08 08:22:19 -07:00
Mark Salyzyn
e7e5f1b6ae logcat: confirm -g result after -c
Bug: 28451229
Change-Id: I43a1721ae991cc1316335ba4b3b868b904926d42
2016-04-29 11:33:26 -07:00
Chih-Hung Hsieh
1cc82ce95c Fix google-explicit-constructor warnings.
Bug: 28341362
Change-Id: I2cf746fd07addc6e6c22e98e2a5bb8a8ce47c885
2016-04-25 13:49:46 -07:00
Mark Salyzyn
33c262513f logcat: expand -n, -r and -b
- Add property expansion to the -n/--rotate-count,
  -r/--rotate-kbytes and -b/--buffer parameters.
  e.g. -r '${logcat.rotate-count:-6}' will expand the property
  logcat.rotate-count, and if not present will default to 6
- Add gtest to confirm mid-stream expansion
- No longer support ",:;<whitespace>" for buffer tokenization,
  settling on only supporting ",".

Bug: 28120456
Change-Id: I7ebb6146f72047631536c457952fa50d0abb98ab
2016-04-13 11:43:03 -07:00
Mark Salyzyn
4517773296 logcat: allow comma-separate list of buffers
- Add parsing to support comma+ separated list of buffers.
- Move get_size test into a standalone get_groups function
- add some additional tests for this feature, leveraging
  get_groups to confirm buffer selection.

Bug: 28120456
Change-Id: I0b42736c08cf4b2a435cb74cda540dc163a26bd1
2016-04-13 11:42:57 -07:00
Mark Salyzyn
3842b1a5f6 logcat: apct test failures
Bug: 27241615
Change-Id: I100a5071c059fe9fa85bbdc33db552b16d7bdf60
2016-04-12 22:36:04 +00:00
Mark Salyzyn
c18c2130d0 logcat: -f <non-existent-directory>/<filename> segfaults
- Check if the result of opendir is NULL in lastLogTime
- Cleanup: alphabetically sorted long options, reserved
  an alias for --regex
- Add a unit test, non existent directory should return
  gracefully with an exit(1) and not SIGSEGV.

NB: This failure was with eng/debug feature logpersist
    turned on, /data/misc/logd/ directory was missing,
    deleted, or temporarily inaccessible.

Bug: 27954627
Change-Id: I60246a53b02fdd7e5490fe458b02ad7b14843584
2016-04-01 09:42:32 -07:00
Casey Dahlin
6ac498d4bb logcat: Add --max-count option
This new option causes logcat to quit after it prints N messages.

Test: New unit test passes
Bug: 27706081
Change-Id: Ie52ddf300160a041e68a6bac0ae7ade68bb28a7c
2016-03-25 13:55:29 -07:00
Casey Dahlin
dc42a8734e logcat: Add --regex option
You can now filter log output by a regex on the messages.

Test: New unit test passes
Bug: 27706081
Change-Id: Idfa637f62a25fb1b92e9b49b89572dff0fec6e08
2016-03-25 13:55:29 -07:00
James Hawkins
588a2cad7f system/core: Cleanup direct calls to opendir by containing in a
std::unique_ptr.

Bug: 26643633
Change-Id: Ia3491fdbff086558da694ae949cf08e4c89d0307
2016-02-18 14:52:46 -08:00
Mark Salyzyn
8beb0d3ad7 logcat: test: rotated logs can exceed size during crash
Some crash messages can be as large as 4K, so allow
the rotated logs to have a larger size.

Change-Id: I20ef8d526ad2d5eca128d1f99f4ea2d2da426403
2015-12-18 10:35:20 -08:00
Mark Salyzyn
ba7a9a016b logd: liblog: logcat: switch to android_log_clockid() (2)
android_log_timestamp returns the property leading letter,
it is better to return a clockid_t with android_log_clockid()

Bug: 23668800
Change-Id: I38dee773bf3844177826b03a26b03215c79a5359
2015-12-07 18:45:31 +00:00
Mark Salyzyn
9e18cdcebd Revert "logd: liblog: logcat: switch to android_log_clockid()"
This reverts commit 77b5696b1d.

Change-Id: I7711bf1a7e3f72ed29a2498d7287b725a0e624bd
2015-12-07 18:30:58 +00:00
Mark Salyzyn
77b5696b1d logd: liblog: logcat: switch to android_log_clockid()
android_log_timestamp returns the property leading letter,
it is better to return a clockid_t with android_log_clockid()

Bug: 23668800
Change-Id: I3c4e3e6b87f6676950797f1f0e203b44c542ed43
2015-12-04 14:49:19 -08:00
Mark Salyzyn
b6bee33182 liblog: logd: support logd.timestamp = monotonic
if ro.logd.timestamp or persist.logd.timestamp are set to the value
monotonic then liblog writer, liblog printing and logd all switch to
recording/printing monotonic time rather than realtime. If reinit
detects a change for presist.logd.timestamp, correct the older entry
timestamps in place.

ToDo: A corner case condition where new log entries in monotonic time
      occur before logd reinit detects persist.logd.timestamp, there
      will be a few out-of-order entries, but with accurate
      timestamps. This problem does not happen for ro.logd.timestamp
      as it is set before logd starts.

NB: This offers a nano second time accuracy on all log entries
    that may be more suitable for merging with other system
    activities, such as systrace, that also use monotonic time. This
    feature is for debugging.

Bug: 23668800
Change-Id: Iee6dab7140061b1a6627254921411f61b01aa5c2
2015-11-03 15:15:51 -08:00
Mark Salyzyn
9812fc4bd0 logcat: continue where we left off
Issue introduced as part of new logcatd functionality in
commit f3555d9427

Faulty logic, add a gTest to confirm.

Bug: 19608716
Change-Id: Ic1b97def25e03e69faae4398a3dff2ff0f88545e
2015-10-06 12:46:54 -07:00
Mark Salyzyn
f28f6a9ba2 liblog: logcat: Add year and zone to date on each line
- '-v year' modifier adds the four digit year prefix
- '-v <timezone>' modifier sets and prints the timezone suffix
- Only promise in logcat to support UTC as a timezone since
  all others are based on the configured environment
- '-v zone' modifier toggles the timezone suffix on or off
- '-T YYYY-MM-DD HH:MM:SS.mmm...' format is added

Bug: 23668700
Change-Id: I7521c1efb7455dc9a25434fce72a69a65dcf72ea
2015-09-24 14:49:55 -07:00
Elliott Hughes
bcc2b5f44a Remove LOCAL_ADDITIONAL_DEPENDENCIES in cases where it's not needed.
Change-Id: I720b8ef1050da45a7833adef8219b6acb2cf3a38
2015-04-02 14:31:07 -07:00
Mark Salyzyn
9c51243d8b logcat: test: logcat.logrotate fails
Test did not handle all possible multiples of block sizes.

Bug: 19604106
Change-Id: I693358be9598926389631baf21c5883e0f2fd735
2015-03-12 20:11:36 +00:00