Commit graph

36 commits

Author SHA1 Message Date
Mark Salyzyn
8fa8896d2e logd: security buffer only AID_SYSTEM reader
- limit AID_SYSTEM uid or gid to read security buffer messages
- adjust liblog tests to reflect the reality of this adjustment

To fully test all security buffer paths and modes

$ su 0,0,0 /data/nativetest/liblog-unit-tests/liblog-unit-tests --gtest_filter=liblog.__security*
$ su 1000,1000,1000 /data/nativetest/liblog-unit-tests/liblog-unit-tests --gtest_filter=liblog.__security*
$ su 2000,2000,2000 /data/nativetest/liblog-unit-tests/liblog-unit-tests --gtest_filter=liblog.__security*

ToDo: Integrate the above individually into the gTest Q/A testing

Bug: 26029733
Change-Id: Idcf5492db78fa6934ef6fb43f3ef861052675651
2016-02-01 13:29:06 -08:00
Mark Salyzyn
507eb9fec2 logd: prune maintain per-id watermark
Without this change LogBuffer::prune and LogBuffer::erase
contributes 16.7% and 1.79% respectively. With this change,
they contributes 3.06 and 2.33% respectively. Pruning is
performed roughly 1 in every 255 log entries, a periodic
tamer latency spike.

Bug: 23685592
Change-Id: I6ae1cf9f3559bca4cf448efe8bcb2b96a1914c54
2016-01-11 14:46:51 -08:00
Mark Salyzyn
10b82b6834 logd: isMonotonic improvements
Use 1972 as a right delineation. Otherwise use half way point
between the monotonic and realtime. Treat correction factor as
unsigned, ensure that any wrapping to a negative value is
dropped or set to EPOCH. Acknowledge that we can get a more
accurate time track by acquiring the time rather than relying on
healthd timestamp.

Bug: 26331432
Change-Id: I09075fca58676a30cf7d87baf2d4b0f53795abaa
2015-12-29 11:34:29 -08:00
Mark Salyzyn
bec3c3def9 logd: Add worst pid of system filter
- Add a new statistic that reports per pid and log_id for AID_SYSTEM
- Add a new pruning filter ~1000/! boolean
- Use this new statistic to prune on worst pid within AID_SYSTEM

Bug: 26029733
Bug: 21615139
Bug: 22855208
Change-Id: Iab5dd28f807dcf03d276372853883f3b6afa8294
2015-12-29 09:32:35 -08:00
Mark Salyzyn
ee3b838e13 logd: statistics per-pid filter
Primarily gives access to the Chattiest TIDs and TAGs
associated with a pid.

Has a secondary effect of allowing us to pull out the
command line, comm and in some cases the associated
PACKAGE for a specific pid while the logs are still
present even if the executable is gone.

Bug: 26029733
Bug: 21615139
Change-Id: I1ea63165a680a9318360579b70b1512078ed5682
2015-12-18 13:17:37 -08:00
Mark Salyzyn
b6bee33182 liblog: logd: support logd.timestamp = monotonic
if ro.logd.timestamp or persist.logd.timestamp are set to the value
monotonic then liblog writer, liblog printing and logd all switch to
recording/printing monotonic time rather than realtime. If reinit
detects a change for presist.logd.timestamp, correct the older entry
timestamps in place.

ToDo: A corner case condition where new log entries in monotonic time
      occur before logd reinit detects persist.logd.timestamp, there
      will be a few out-of-order entries, but with accurate
      timestamps. This problem does not happen for ro.logd.timestamp
      as it is set before logd starts.

NB: This offers a nano second time accuracy on all log entries
    that may be more suitable for merging with other system
    activities, such as systrace, that also use monotonic time. This
    feature is for debugging.

Bug: 23668800
Change-Id: Iee6dab7140061b1a6627254921411f61b01aa5c2
2015-11-03 15:15:51 -08:00
Mark Salyzyn
58b8be8906 logd: correct for number of elements in prune
Chatty logs would distort the average log size by elevating the
elements, but not the size. Add statistical collection for the
number of elements that report chatty, and subtract that from
the number of elements to improve the pruning estimate. Pick
minElements as 1% rather than 10% of the total with this more
accurate number of elements, to a minumum of 4.

Bug: 24511000
Change-Id: I3f36558138aa0b2a50e4fac6440c3a8505d95276
2015-10-13 13:43:16 -07:00
Mark Salyzyn
9b3a2784b9 Merge "logd: use coalesce instead of merge (cleanup)" 2015-10-13 16:47:15 +00:00
Mark Salyzyn
1c04253af2 Merge "logd: object layer format statistics" 2015-10-12 22:10:30 +00:00
Mark Salyzyn
aaad42f47c logd: use coalesce instead of merge (cleanup)
- switch to coalesce instead of merge in naming of functions
  and variables. Confusing since we also to merge-sorts and
  other activities in the logger.
- define maxPrune rather than using a number in the code path.

Bug: 24511000
2015-10-09 15:00:44 -07:00
Mark Salyzyn
c5dc970edc logd: clear return and deal with busy if readers locked
- Propagate to caller the clearing errors, busy blocked by reader.
- For clear, perform retries within logd with a one second lul each,
  telling readers to skip, but on final retry to kill all readers if
  problem still persists due to block reader (or high volume logspammer).

Bug: 23711431
Change-Id: Ie4c46bc9480a7f49b96a81fae25a95c603270c33
2015-10-07 16:23:57 -07:00
Mark Salyzyn
758058ffd8 logd: object layer format statistics
Simplify table generation by placing the line and header formatting
into each type's (UID, PID, TID) object. Switch to const return
values for the ownership passing functions (*ToName() functions
and methods). Use longer variable names to reduce confusion.

Switch from LINES To NUM for pruned column as that more accurately
reflects what is dropped since one entry can contain several lines.

Bug: 22855208
Change-Id: Ib110dce98a68cf5f844eb30f8a192a1f691eeba2
2015-10-05 09:27:57 -07:00
Mark Salyzyn
831aa29730 logd: worst uid record watermark part four
With part deux we caused an apparent regression by not checking for
stale recorded iterators. This checking was on-purpose bypassesed
when leading prune entries were to be deleted without touching the
statistics engine due to an in-place merge.

Part deux had us leaving iterators we were not focussed on untouched
which in turn because they were left behind, had a much higher
likelihood of being deleted without touching the statistics engine.

Perform the check every delete.

Bug: 23789348
Change-Id: Idc6cc23d1f9e3b6cd9a083139a0de59479fbfe08
2015-09-03 17:13:45 -07:00
Mark Salyzyn
73160acc5c logd: switch asprintf to std::string
Bug: 23350706
Change-Id: I715cdd4563a09de3680081947a3439f0cac623be
2015-08-20 10:32:16 -07:00
Mark Salyzyn
c892ea3fa8 logd: worst uid record watermark
Hold on to last worst uid watermark and bypass a spike to O(n*n*x)
(n=samples, x=number of spammers) wrt chatty trimming.

Bug: 23327476
Change-Id: I9f21ce95e969b67e576417a760f75c4d86acf364
2015-08-20 10:25:57 -07:00
Mark Salyzyn
94a89c42fe logd: log buffer switch to std::list
Bug: 23350706
Bug: 23327476
Change-Id: Ib0530b9dd5842c6d05c84d7a66f2531c97461067
2015-08-20 08:36:13 -07:00
Mark Salyzyn
ed777e9eec logd: serialize accesses to stats helpers
Quick low-risk to resolve possible hash table corruption.
Resolved an unlikely path memory leak.

ToDo: replace lock with nested lock so no lock
      helpers are required.

Bug: 22068332
Change-Id: I303ab06608502c7d61d42f111a9c43366f184d0c
2015-06-25 07:39:24 -07:00
Mark Salyzyn
21fb7e0b75 logd: improve details on chatty records
- Report applications UID, TID/PID by name.
- change wording to have an accurate connotation
- drop privilege check since filtered upstream

Bug: 19608965
Bug: 20334069
Bug: 20370119
Change-Id: I2b1c26580b4c2de293874214ff5ae745546f3cca
2015-04-30 13:14:51 -07:00
Mark Salyzyn
202e153f94 logd: propagate ::log status
Add a return value for the ::log() methods, this allows
us to optimize the wakeup for the readers to only occur
when the log message is actually placed.

This is for a future where we may dedupe identical log
messages, filter out log messages, and certainly if we
filter the messages out with an internal logd check of
__android_log_is_loggable().

Change-Id: I763b2a7c29502ab7fa0a5d5022c7b60244fcfde4
2015-04-15 19:43:28 -07:00
Mark Salyzyn
97c1c2beee logd: optimize statistics
- Go back to basic requirements
- Simplify
- use hash tables to minimize memory impact

Bug: 19608965
Change-Id: If7becb34354d6415e5c387ecea7d4109a15259c8
2015-03-20 12:31:23 -07:00
Mark Salyzyn
f7c0f75275 logd: replace internal CLOCK_MONOTONIC use with sequence numbers
- switch to simpler and faster internal sequence number, drops
  a syscall overhead on 32-bit platforms.
- add ability to break-out of flushTo loop with filter return -1
  allowing in reduction in reader overhead.

Change-Id: Ic5cb2b9afa4d9470153971fc9197b07279e2b79d
2015-03-18 12:43:23 -07:00
Mark Salyzyn
11e55cb9c1 logd: add reinit command
- respond to SIGHUP
- respond to logd command "reinit"
- respond to logd --reinit
- reopens files on /data, eg: re-read of persistent properties

Bug: 19681572
Change-Id: Iadac58e6653f027cb7355497bd675eef376ce0a8
2015-03-12 12:24:47 -07:00
Mark Salyzyn
4ed16b4381 Revert "logd: Add minimum time bucket statistics"
This forward port reverts
    commit e457b74ce6

No longer as necessary once we add
    liblog: Instrument logging of logd write drops
Although this provided an indication of how close statistically we
were to overloading logd it is simpler to understand fails thus to
hunt and peck a corrected value for /proc/sys/net/unix/max_dgram_qlen

Change-Id: I2b30e0fc30625a48fd11a12c2d2cc6a41f26226f
2015-03-04 13:21:41 -08:00
Mark Salyzyn
f5fc509589 logd: Add control statistics enable/disable.
- ro.build.type=user turn off statistics
- ro.config.low_ram=true turn off statistics
- logd.statistics override

Bug: 17526159
Bug: 17526187
Change-Id: I74796043ac34753c6dd10018719ebc0bcd94e00f
2014-10-06 22:42:47 +00:00
Mark Salyzyn
e72c6e4366 logd: cleanup
- simplify access and control exposure to class list
- indent
- compile warning
- Follow standard naming convention for variables and methods
- merge common fragments
- Side Effects: none

Change-Id: I74796043ac34753c6dd10018719ebc0bcd94e010
2014-10-06 22:42:07 +00:00
Mark Salyzyn
1a240b4790 logd: Allow apps to clear their UID-specific data
Bug: 13501501
Change-Id: Ia72e25fc19430ce63fb359cd9b3f0523d41f5aa8
2014-06-13 08:06:14 -07:00
Mark Salyzyn
472e04f2b4 Merge "logd: liblog: logcat: enable prune features for user" 2014-04-09 23:33:28 +00:00
Mark Salyzyn
4ba0387af5 logd: Add pidToUid helper
Change-Id: I23ebae1957c027bff6cbc2573a227bf0c44c08a2
2014-04-07 10:51:00 -07:00
Mark Salyzyn
9a03863e88 logd: Add pidToName helper
Change-Id: Idd8e804ab65feb8dc432150ae701464de1ad5302
2014-04-07 10:50:55 -07:00
Mark Salyzyn
1c95047939 logd: liblog: logcat: enable prune features for user
- Enable whitelist, blacklist and logsize tuneables for user

Change-Id: Id0c283844c71c5263a8cfbebf2e550f7ac415858
2014-04-02 13:12:04 -07:00
Mark Salyzyn
e457b74ce6 logd: Add minimum time bucket statistics
* Only enabled for dev builds because halves performance.
- Used to establish if /proc/sys/net/unix/max_dgram_qlen
  is appropriate for the platform
  (see logd/LogStatistics.cpp comments)
- enabled if logd.dgram_qlen.statistics is not zero/empty

Change-Id: Ib8ecfeb75b6f0f954c099a6b1032b8afb5efcbd4
2014-04-01 14:38:52 -07:00
Mark Salyzyn
b992d0d7d3 logd: liblog: Thread IDs missing from logcat -v thread
- stuff caller's thread id into the packet.

Bug: 13568206
Change-Id: I02d0cdf9b1d9e839ff8969f591db42dfe6e4cc95
2014-03-21 10:37:44 -07:00
Mark Salyzyn
dfa7a07f5b logd: liblog: logcat: Add LogWhiteBlackList
- liblog android_logger_get_log_size and android_logger_get_readable_size
  adjusted to return long instead of int because of -G flag extending range

NB: ifdef'd only for userdebug and eng builds

- liblog Add android_logger_[sg]et_prune_list and android_logger_set_log_size
- logcat Add -P, -p and -G flags
- logd Add LogWhiteBlackList and configurable log size

(cherry picked from commit 18a5432158)

Change-Id: I1572338c1b34bd968ad7867857ef708156ec3b6a
2014-03-13 14:48:07 -07:00
Mark Salyzyn
34facab86b logd: liblog: logcat: Add Statistics
- logd add statistical collection and formatting
- liblog add android_logger_get_statistics call
- logcat add -S flag
- logcat add -b all

(cherry picked from commit 51a29c8dc4)

Change-Id: I521753b1969ecd4590c956aeeb1557d101059d67
2014-03-13 14:47:58 -07:00
Mark Salyzyn
7e2f83c0bc logd: liblog: 64-bit issues
- structure packing
- move towards log_time from struct timespec
- extend log_time to cover differences between
  log_time and struct timespec

Change-Id: I106ed0b609917306d170044054b5b32645f2a295
2014-03-05 16:12:28 -08:00
Mark Salyzyn
0175b0747a logd: initial checkin.
* Create a new userspace log daemon for handling logging messages.

Original-Change-Id: I75267df16359684490121e6c31cca48614d79856
Signed-off-by: Nick Kralevich <nnk@google.com>

* Merge conflicts
* rename new syslog daemon to logd to prevent confusion with bionic syslog
* replace racy getGroups call with KISS call to client->getGid()
* Timestamps are filed at logging source
* insert entries into list in timestamp order
* Added LogTimeEntry tail filtration handling
* Added region locking around LogWriter list
* separate threads for each writer
* /dev/socket/logd* permissions

Signed-off-by: Mark Salyzyn <salyzyn@google.com>

(cherry picked from commit 3e76e0a497)

Author: Nick Kralevich <nnk@google.com>
Change-Id: Ice88b1412d8f9daa7f9119b2b5aaf684a5e28098
2014-02-26 09:52:35 -08:00