Commit graph

30 commits

Author SHA1 Message Date
Mark Salyzyn
2c9d909a00 logd: better drop message merging
- Former algorithm anlo coalesced adjacent records
- New algorithm maintains a hash list of all drop
  records and coalesces them all.

Bug: 20334069
Bug: 20370119
Change-Id: Idc15ce31fc1087c2cfa39da60c62feade8b88761
2015-04-20 12:58:54 -07:00
Mark Salyzyn
202e153f94 logd: propagate ::log status
Add a return value for the ::log() methods, this allows
us to optimize the wakeup for the readers to only occur
when the log message is actually placed.

This is for a future where we may dedupe identical log
messages, filter out log messages, and certainly if we
filter the messages out with an internal logd check of
__android_log_is_loggable().

Change-Id: I763b2a7c29502ab7fa0a5d5022c7b60244fcfde4
2015-04-15 19:43:28 -07:00
Mark Salyzyn
ae76923839 logd: disable worst uid prune for events buffer
There is some usage statistics that would be hurt by pruning by UID,
since _all_ usage statistics come from system_server. In other words
we expect it to be chatty. Until we formulate and evaluate a better
(eg: per-tag?) filtration mechanism, lets hold off pruning by UID.

Bug: 19608965
Change-Id: Iddd45a671e13bdcf3394c20919ad1f2e4ef36616
2015-04-10 15:45:10 -07:00
Mark Salyzyn
ab0dcf6828 logd: annotate worst-UID pruned entries
- internal dropped entries are associated by prune by worst UID
  and are applied by UID and by PID
- track dropped entries by rewriting them in place
- merge similar dropped entries together for same UID(implied),
  PID and TID so that blame can more clearly be placed
- allow aging of dropped entries by the general backgound pruning
- report individual dropped entries formatted to reader
- add statistics to track dropped entries by UID, the combination
  of statistics and dropped logging can track over-the-top Chattiest
  clients.

Bug: 19608965
Change-Id: Ibc68480df0c69c55703270cd70c6b26aea165853
2015-04-10 15:45:08 -07:00
Mark Salyzyn
720f6d1d55 logd: Add Pid statistics
- Optional class of statistics for PID
- Enhance pidToName
- Enhanced uidToName
- Enhance pidToUid
- template sort and iteration

Bug: 19608965
Change-Id: I04a1f02e9851b62987f9b176908134e455f22d1d
2015-04-08 14:37:32 -07:00
Mark Salyzyn
97c1c2beee logd: optimize statistics
- Go back to basic requirements
- Simplify
- use hash tables to minimize memory impact

Bug: 19608965
Change-Id: If7becb34354d6415e5c387ecea7d4109a15259c8
2015-03-20 12:31:23 -07:00
Mark Salyzyn
f7c0f75275 logd: replace internal CLOCK_MONOTONIC use with sequence numbers
- switch to simpler and faster internal sequence number, drops
  a syscall overhead on 32-bit platforms.
- add ability to break-out of flushTo loop with filter return -1
  allowing in reduction in reader overhead.

Change-Id: Ic5cb2b9afa4d9470153971fc9197b07279e2b79d
2015-03-18 12:43:23 -07:00
Mark Salyzyn
11e55cb9c1 logd: add reinit command
- respond to SIGHUP
- respond to logd command "reinit"
- respond to logd --reinit
- reopens files on /data, eg: re-read of persistent properties

Bug: 19681572
Change-Id: Iadac58e6653f027cb7355497bd675eef376ce0a8
2015-03-12 12:24:47 -07:00
Mark Salyzyn
4ed16b4381 Revert "logd: Add minimum time bucket statistics"
This forward port reverts
    commit e457b74ce6

No longer as necessary once we add
    liblog: Instrument logging of logd write drops
Although this provided an indication of how close statistically we
were to overloading logd it is simpler to understand fails thus to
hunt and peck a corrected value for /proc/sys/net/unix/max_dgram_qlen

Change-Id: I2b30e0fc30625a48fd11a12c2d2cc6a41f26226f
2015-03-04 13:21:41 -08:00
Dragoslav Mitrinovic
8e8e8db549 logd: fix persistent blocking reader performance
logd suffers major performance degradation when persistent (blocking)
client reader connects to it (e.g. logcat). The root cause of the
degradation is that each time when reader is notified of the arrival
of new log entries, the reader commences its search for the new entries
from the beginning of the linked list (oldest entries first).

This commit alters the search to start from the end of the linked list
and work backwards. This dramatically decreases logd CPU consumption
when blocking reader is connected, and increases the maximum logging
throughput (before the logs start getting lost) by a factor ~ 20.

Change-Id: Ib60955ce05544e52a8b24acc3dcf5863e1e39c5c
2015-01-24 00:10:34 +00:00
TraianX Schiau
da6495d06f logd: Fix pruning
In a scenario in which an on-line (blocking) client is running and
a clean is attempted (logcat -c), the following can be observed:

1) the on-line logger seems to freeze
2) any other clear attempt will have no effect

What is actually happening:

In this case prune function will "instruct" the oldest timeEntry
to skip a huge number (very close to ULONG_MAX) of messages, this
being the cause of 1.

Since the consumer thread will skip all the log entries, mStart
updating will also be skipped. So a new cleaning attempt will have
the same oldest entry, nothing will be done.

Fix description:
a. keep a separated skipAhead count for individual log buffers (log_id_t)
      LogTimeEntry::LogTimeEntry
      LogTimeEntry::FilterSecondPass
      LogTimeEntry::skipAhead
      LogTimeEntry::riggerSkip_Locked

b. update LogTimeEntry::mStart even if the current message is skipped
      LogTimeEntry::FilterSecondPass

c. while pruning, only take into account the LogTimeEntrys that are monitoring
   the log_id in question, and provide a public method of checking this.
      LogTimeEntry::isWatching
      LogTimeEntry::FilterFirstPass
      LogTimeEntry::FilterSecondPass

d. Reset the skip cont befor the client thtread starts to sleep, at this point
   we should be up to date.
      LogTimeEntry::cleanSkip_Locked
      LogTimeEntry::threadStart

Change-Id: I1b369dc5b02476e633e52578266a644e37e188a5
Signed-off-by: TraianX Schiau <traianx.schiau@intel.com>
2015-01-14 19:12:47 +00:00
Mark Salyzyn
eae155e1ce logd: insert order for first entry
Change-Id: I39e8e6d32407a1796a0c3a121f9dc4dc5402c3df
2014-10-14 23:57:13 +00:00
Mark Salyzyn
e72c6e4366 logd: cleanup
- simplify access and control exposure to class list
- indent
- compile warning
- Follow standard naming convention for variables and methods
- merge common fragments
- Side Effects: none

Change-Id: I74796043ac34753c6dd10018719ebc0bcd94e010
2014-10-06 22:42:07 +00:00
Mark Salyzyn
1a240b4790 logd: Allow apps to clear their UID-specific data
Bug: 13501501
Change-Id: Ia72e25fc19430ce63fb359cd9b3f0523d41f5aa8
2014-06-13 08:06:14 -07:00
Mark Salyzyn
ab4b7308ec logd: used before set warning
Change-Id: I30c078f8b22ebe4dcb48d57adf88b3e51b4fa202
2014-05-23 09:48:09 -07:00
Mark Salyzyn
57a0af9313 init: logd: Allow Developer settings to adjust logd size
- AID_SYSTEM can set persist.logd.size
- AID_SYSTEM can issue command to /dev/socket/logd to
  change the runtime global log sizes.
- Add support for ro.logd.size.* as populated by BoardConfig.mk
- Limit size to maximum ~3% of physical memory.

Bug: 14563261
Bug: 14627052
Change-Id: I606420db2e9d9d032296e71149e4a5b20cbd1137
2014-05-13 07:42:15 -07:00
Mark Salyzyn
671e343c7d logd: logcat: Add persist.logd.size group of properties
- logd Add persist.logd.size (global), persist.logd.size.<logid>
- logcat report a more flexible multiplier in -g command.

Bug: 14563261
Bug: 14469172
Change-Id: Ie389caa14ad3ae3d4a3a3cc463425bb9dbc5e483
2014-05-06 14:58:41 -07:00
Mark Salyzyn
c8a576c637 logd: Statistics improvements
- Drop in-place sorting.
- Add fast-track for uid sizes and elements.
- Add sort() for per-uid and per-pid statistics.

Change-Id: Ib8655d4cc0b3bd4e87534522987bcc7fc7a5251a
2014-04-25 10:46:27 -07:00
Mark Salyzyn
99f47a9e7c logd: liblog: logcat: debuggerd: Add LOG_ID_CRASH
Change-Id: Iea453764a1a4fc1661f461f10c641c30150e4d20
2014-04-16 15:38:50 -07:00
Mark Salyzyn
1c95047939 logd: liblog: logcat: enable prune features for user
- Enable whitelist, blacklist and logsize tuneables for user

Change-Id: Id0c283844c71c5263a8cfbebf2e550f7ac415858
2014-04-02 13:12:04 -07:00
Mark Salyzyn
ca4c63d799 logd: build breakage in user
build breakage in branch: git_klp-dev-gpl @ 1101951

Change-Id: I4c9681bf4c97368ec19f5a79fef134aba937f337
2014-04-01 16:57:29 -07:00
Mark Salyzyn
e457b74ce6 logd: Add minimum time bucket statistics
* Only enabled for dev builds because halves performance.
- Used to establish if /proc/sys/net/unix/max_dgram_qlen
  is appropriate for the platform
  (see logd/LogStatistics.cpp comments)
- enabled if logd.dgram_qlen.statistics is not zero/empty

Change-Id: Ib8ecfeb75b6f0f954c099a6b1032b8afb5efcbd4
2014-04-01 14:38:52 -07:00
Mark Salyzyn
b992d0d7d3 logd: liblog: Thread IDs missing from logcat -v thread
- stuff caller's thread id into the packet.

Bug: 13568206
Change-Id: I02d0cdf9b1d9e839ff8969f591db42dfe6e4cc95
2014-03-21 10:37:44 -07:00
Mark Salyzyn
dfa7a07f5b logd: liblog: logcat: Add LogWhiteBlackList
- liblog android_logger_get_log_size and android_logger_get_readable_size
  adjusted to return long instead of int because of -G flag extending range

NB: ifdef'd only for userdebug and eng builds

- liblog Add android_logger_[sg]et_prune_list and android_logger_set_log_size
- logcat Add -P, -p and -G flags
- logd Add LogWhiteBlackList and configurable log size

(cherry picked from commit 18a5432158)

Change-Id: I1572338c1b34bd968ad7867857ef708156ec3b6a
2014-03-13 14:48:07 -07:00
Mark Salyzyn
64d6fe9362 logd: prune by worst offending UID
(cherry picked from commit 3c4919e474)

Change-Id: I39965007569123ff5eebe01b5bfa555bbcb2dfe7
2014-03-13 14:48:02 -07:00
Mark Salyzyn
34facab86b logd: liblog: logcat: Add Statistics
- logd add statistical collection and formatting
- liblog add android_logger_get_statistics call
- logcat add -S flag
- logcat add -b all

(cherry picked from commit 51a29c8dc4)

Change-Id: I521753b1969ecd4590c956aeeb1557d101059d67
2014-03-13 14:47:58 -07:00
Mark Salyzyn
7e2f83c0bc logd: liblog: 64-bit issues
- structure packing
- move towards log_time from struct timespec
- extend log_time to cover differences between
  log_time and struct timespec

Change-Id: I106ed0b609917306d170044054b5b32645f2a295
2014-03-05 16:12:28 -08:00
Mark Salyzyn
c03e72cc1c logd: Adjust to match defacto coding style
(cherry picked from commit c46f77bd2a)

Change-Id: I80685cdc7116e10c5a5a77abe856fd96804f9117
2014-02-26 09:54:23 -08:00
Mark Salyzyn
740f9b4f97 logd: prune more aggressively when over the top
(cherry picked from commit 63c15d5061)

Change-Id: I929dddc7da048c032fb791c7af23f215f8856bf3
2014-02-26 09:53:55 -08:00
Mark Salyzyn
0175b0747a logd: initial checkin.
* Create a new userspace log daemon for handling logging messages.

Original-Change-Id: I75267df16359684490121e6c31cca48614d79856
Signed-off-by: Nick Kralevich <nnk@google.com>

* Merge conflicts
* rename new syslog daemon to logd to prevent confusion with bionic syslog
* replace racy getGroups call with KISS call to client->getGid()
* Timestamps are filed at logging source
* insert entries into list in timestamp order
* Added LogTimeEntry tail filtration handling
* Added region locking around LogWriter list
* separate threads for each writer
* /dev/socket/logd* permissions

Signed-off-by: Mark Salyzyn <salyzyn@google.com>

(cherry picked from commit 3e76e0a497)

Author: Nick Kralevich <nnk@google.com>
Change-Id: Ice88b1412d8f9daa7f9119b2b5aaf684a5e28098
2014-02-26 09:52:35 -08:00