Commit graph

376 commits

Author SHA1 Message Date
Nick Kralevich
33ee84f871 logd: Don't trigger an integrity failure on permissive SELinux denials
Only trigger an integrity failure if a policy is reloaded or
SELinux is disabled. Don't trigger the integrity failure if
we see a permissive=1 denial, which could occur if an SELinux
domain is in permissive mode.

Bug: 27313768
Bug: 26902605
Change-Id: Ib85a2799eb6378ae8acdb965b1812d691183fdd3
2016-02-26 17:00:15 -08:00
Mark Salyzyn
d26b2cacad logd: sock_alloc_send_pskb starves pruning
(cherry pick from commit 5c77ad55d0)

Allow socket send syscall to terminate after 32 seconds if reader
stalled because of memory pressure allocating new network buffers

Add a gTest to catch regressions, add security buffer to log_dump

Bug: 27242723
Change-Id: Idaa6699d9d284e7f5f723ae0e76b3d6aa3371489
2016-02-25 12:43:12 -08:00
Mark Salyzyn
317bfb923c logd: Allow (some) headers to be individually importable
(cherry pick from commit 2ad0bd0a9b)

LogReader.h needs to be individually importable.

Fix a few others, drop includes of local includes, let them be
included in source instead and allow headers to be included
alphabetically. Was not a complete audit since goal was to
separate LogReader.h out from the pack.

Bug: 27242723
Change-Id: Ic7759ef90995e5bd285810706af33550c73cf5b5
2016-02-25 12:42:31 -08:00
Mark Salyzyn
5c77ad55d0 logd: sock_alloc_send_pskb starves pruning
Allow socket send syscall to terminate after 32 seconds if reader
stalled because of memory pressure allocating new network buffers

Add a gTest to catch regressions, add security buffer to log_dump

Bug: 27242723
Change-Id: Idaa6699d9d284e7f5f723ae0e76b3d6aa3371489
2016-02-24 10:15:32 -08:00
Mark Salyzyn
2ad0bd0a9b logd: Allow (some) headers to be individually importable
LogReader.h needs to be individually importable.

Fix a few others, drop includes of local includes, let them be
included in source instead and allow headers to be included
alphabetically. Was not a complete audit since goal was to
separate LogReader.h out from the pack.

Bug: 27242723
Change-Id: Ic7759ef90995e5bd285810706af33550c73cf5b5
2016-02-24 10:15:22 -08:00
Sami Tolvanen
0bdad0f231 logd: enforce policy integrity
If a SELinux policy change or a switch to permissive mode is detected
on a user build, restart the device into safe mode, and keep it there
until an OTA is applied or user data is wiped.

This change deprecates the ro.logd.auditd property.

Needs matching changes from
  I781c3059ea8d4fb2f0c923e4488b1932d69678d3
  Ica825cf2af74f5624cf4091544bd24bb5482dbe7
  Id3ca7889ede30b54b7af73dd50653ca1a20d59aa

Bug: 26902605
Change-Id: Idcdc5bff133f13c1267f0ec0a75cc8cf1ddbda0d
(cherry picked from commit d122ee65b6)
2016-02-22 14:00:43 -08:00
Sami Tolvanen
d122ee65b6 logd: enforce policy integrity
If a SELinux policy change or a switch to permissive mode is detected
on a user build, restart the device into safe mode, and keep it there
until an OTA is applied or user data is wiped.

This change deprecates the ro.logd.auditd property.

Needs matching changes from
  I781c3059ea8d4fb2f0c923e4488b1932d69678d3
  Ica825cf2af74f5624cf4091544bd24bb5482dbe7
  Id3ca7889ede30b54b7af73dd50653ca1a20d59aa

Bug: 26902605
Change-Id: Idcdc5bff133f13c1267f0ec0a75cc8cf1ddbda0d
2016-02-16 12:54:54 -08:00
Mark Salyzyn
d1f41d606b logd: use ro.debuggable instead of ro.build.type
Change-Id: Ic644f1f78e68094f700834675f4acc63386a6b92
2016-02-10 10:23:03 -08:00
Mark Salyzyn
d048f113a3 logd: add internal prdebug function
Usage: android::prdebug(const char *fmt, ...) __printflike(1, 2);

Will add logd prefix tag, log as debug priority in kernel logs,
and will suffix a newline if one is not supplied. To be used to
aid debugging of the logger only.

Change-Id: I217326ef34dc4eb0ea076bacb7a7a8b564c931c3
2016-02-09 07:46:39 -08:00
Mark Salyzyn
0604f6fbae init.rc: too many start logd
Bug: 26934873
Change-Id: Ia00da6253a50bedc8ba825df1cf641b86cdebeed
2016-02-02 16:01:17 -08:00
Mark Salyzyn
8fa8896d2e logd: security buffer only AID_SYSTEM reader
- limit AID_SYSTEM uid or gid to read security buffer messages
- adjust liblog tests to reflect the reality of this adjustment

To fully test all security buffer paths and modes

$ su 0,0,0 /data/nativetest/liblog-unit-tests/liblog-unit-tests --gtest_filter=liblog.__security*
$ su 1000,1000,1000 /data/nativetest/liblog-unit-tests/liblog-unit-tests --gtest_filter=liblog.__security*
$ su 2000,2000,2000 /data/nativetest/liblog-unit-tests/liblog-unit-tests --gtest_filter=liblog.__security*

ToDo: Integrate the above individually into the gTest Q/A testing

Bug: 26029733
Change-Id: Idcf5492db78fa6934ef6fb43f3ef861052675651
2016-02-01 13:29:06 -08:00
Mark Salyzyn
c85ce9ba8e Merge "Revert "logd: security buffer only AID_SYSTEM reader"" 2016-01-26 22:38:58 +00:00
Mark Salyzyn
674ce6ef58 Revert "logd: security buffer only AID_SYSTEM reader"
This reverts commit 756332e88b.

Change-Id: Ib8607ac758b7e88a9db716ec4274ec0e4e3dc596
2016-01-26 21:47:35 +00:00
Mark Salyzyn
69ce7c5483 Merge "logd: security buffer only AID_SYSTEM reader" 2016-01-26 20:41:51 +00:00
Mark Salyzyn
756332e88b logd: security buffer only AID_SYSTEM reader
Bug: 26029733
Change-Id: I140e5eb8bb39271fa674744651127e3fc545cbe8
2016-01-26 07:20:29 -08:00
Mark Salyzyn
ef4e4f3f07 logd: test wrap functionality
Behavior is modified now, so react by fortifying the tests

Bug: 26447386
Change-Id: I6295783a0bc75282499aeba503a86bdb846908b5
2016-01-21 11:56:23 -08:00
Mark Salyzyn
7fd6c5c6f5 logd: prune maintain per-id watermark (part deux)
iterator corruption as we allow mLast to slip through the FIFO

Bug: 23685592
Bug: 26646087
Change-Id: Ifcbaecf390ee47b195c3d823d080c66c15db4530
2016-01-19 16:14:39 -08:00
Mark Salyzyn
7b7b2da6b8 Merge "logd: prune maintain per-id watermark" 2016-01-13 20:23:28 +00:00
Mark Salyzyn
ae6aa1f1ed Merge "logd: wrap timed entry too early, timeout immediately" 2016-01-12 21:57:50 +00:00
Mark Salyzyn
4690640760 logd: unaligned access on security buffer
Bug: 26485626
Change-Id: I990447ca59982bb2de15decda575fd0e2838b5ed
2016-01-12 10:09:31 -08:00
Mark Salyzyn
507eb9fec2 logd: prune maintain per-id watermark
Without this change LogBuffer::prune and LogBuffer::erase
contributes 16.7% and 1.79% respectively. With this change,
they contributes 3.06 and 2.33% respectively. Pruning is
performed roughly 1 in every 255 log entries, a periodic
tamer latency spike.

Bug: 23685592
Change-Id: I6ae1cf9f3559bca4cf448efe8bcb2b96a1914c54
2016-01-11 14:46:51 -08:00
Mark Salyzyn
0157097845 logd: wrap timed entry too early, timeout immediately
Bug: 26447386
Change-Id: I8d5588831f558061ef21b2a5aeedc865e9ae4cc7
2016-01-07 15:13:14 -08:00
Mark Salyzyn
1a5bac2069 Merge "Revert "logd: liblog: whitelist "snet_event_log""" 2016-01-06 21:22:04 +00:00
Mark Salyzyn
0ee8de3c2d Revert "logd: liblog: whitelist "snet_event_log""
Callers will not guarantee that they can or will ratelimit, we need to
retain the ability to blacklist snet_event_log as a result.

This reverts commit 6aa21b225d.

Bug: 26178938
Change-Id: Ibf47d2e23a84c56f5f72d02312c698df7ff2b601
2016-01-06 21:19:23 +00:00
Mark Salyzyn
66607ebc0e logd: document ro.logd.size
Bad comment advise in LogBuffer.cpp results in partners failing to
considering using ro.logd.size to set the platform buffer size
default.

NB: It is not good practice to increase the log buffer size to deal
with logspam, as increases will result in logd scale issues getting
closer to hitting the background cgroup cpu cap. Once we hit that
cap, logd spirals, pruning old entries slower than the incoming log
entries. logd.writer will take 100% cpu.

Change-Id: If4a7a74f300d078eeaed0ffd3eb3fd77d1f9fe90
2016-01-05 09:09:18 -08:00
Mark Salyzyn
ff8b8e838b logd: build breakage aosp-brillo-master @ 2508494
Change-Id: I38e506bc997c6e9759a064b517f9372a27d0e510
2015-12-30 13:46:07 -08:00
Mark Salyzyn
554630f94e Merge "logd: isMonotonic improvements" 2015-12-30 21:31:58 +00:00
Mark Salyzyn
6aa21b225d logd: liblog: whitelist "snet_event_log"
Dangerous bridge to cross to whitelist, who is special, who is not?
Rationalized as these events are used to catch exploits on platform.
As it stands no one should be allowed to block any messages in the
security context, not even for development purposes.

Bug: 26178938
Change-Id: Ibdc76bc0fe29ba05be168b623af1c9f41d7edbd2
2015-12-30 10:07:19 -08:00
Mark Salyzyn
10b82b6834 logd: isMonotonic improvements
Use 1972 as a right delineation. Otherwise use half way point
between the monotonic and realtime. Treat correction factor as
unsigned, ensure that any wrapping to a negative value is
dropped or set to EPOCH. Acknowledge that we can get a more
accurate time track by acquiring the time rather than relying on
healthd timestamp.

Bug: 26331432
Change-Id: I09075fca58676a30cf7d87baf2d4b0f53795abaa
2015-12-29 11:34:29 -08:00
Mark Salyzyn
bec3c3def9 logd: Add worst pid of system filter
- Add a new statistic that reports per pid and log_id for AID_SYSTEM
- Add a new pruning filter ~1000/! boolean
- Use this new statistic to prune on worst pid within AID_SYSTEM

Bug: 26029733
Bug: 21615139
Bug: 22855208
Change-Id: Iab5dd28f807dcf03d276372853883f3b6afa8294
2015-12-29 09:32:35 -08:00
Mark Salyzyn
9c66a58f21 logd: Allow flags "eng" and "svelte" in boolean
- enhance property_get_bool, drop property_get_bool_svelte
- enhance base properties with ro and persist variants
- update and fortify README.property
- primarily move auditd and kernel logger into a realm where
  they can be controlled by build properties.
- Move logd.klogd to logd.kernel, and add ro.logd.kernel
  and persist.logd.kernel.
- Add ro.logd.auditd and persist.logd.auditd.
- Document persist.logd.security
- Document log.tag and persist.logd.tag properties.
- Document ro.logd.size, persist.logd.size and logd.size
  properties.

Bug: 26178938
Bug: 26029733
Bug: 17760225
Change-Id: Ibc1a497e45d69db5cf52133937f7ba6fb1d6cd21
2015-12-22 07:44:31 -08:00
Mark Salyzyn
ee3b838e13 logd: statistics per-pid filter
Primarily gives access to the Chattiest TIDs and TAGs
associated with a pid.

Has a secondary effect of allowing us to pull out the
command line, comm and in some cases the associated
PACKAGE for a specific pid while the logs are still
present even if the executable is gone.

Bug: 26029733
Bug: 21615139
Change-Id: I1ea63165a680a9318360579b70b1512078ed5682
2015-12-18 13:17:37 -08:00
Mark Salyzyn
7b4a2049b0 logd: test drop __unused
Change-Id: I1ccf49809b8a0a125601bdb52834c463e2d94165
2015-12-18 10:35:00 -08:00
Mark Salyzyn
236fecd923 Merge "logd: ro.config.low_ram set buffer size to 64K" 2015-12-17 23:47:14 +00:00
Mark Salyzyn
1d9c7e728f logd: simpleperf inspired optimizations
memset is hot on log writes with too much overhead, not required.

Bug: 24444530
Bug: 23685592
Change-Id: Ibe3c83abca6774fc79a3be461f2f1585f4a9b1ff
2015-12-15 14:24:04 -08:00
Mark Salyzyn
cdda62b2c1 logd: ro.config.low_ram set buffer size to 64K
Bug: 25792367
Change-Id: Icae85a7de70b3a71f89b2bf8c80866649f727177
2015-12-14 14:37:07 -08:00
Mark Salyzyn
86052a5d4f Merge "logd: liblog: logcat: Add LOG_ID_SECURITY" 2015-12-11 17:33:33 +00:00
Mark Salyzyn
30edbdce9b Merge changes I598c8c5c,I672b0f4d,Id882978b,Idff5e080,Ib124eca1, ...
* changes:
  logger.h: reduce maximum payload so sum is page size
  liblog: test for maximum payload can not survive change
  liblog: logprint use uid name if length less then 5
  liblog: logprint add uid format modifier
  liblog: readlog apps get logger_entry_v4 for pstore
  logd: readlog apps get logger_entry_v4
  logger.h: Add definition for logger_entry_v4
2015-12-11 16:42:28 +00:00
Mark Salyzyn
ad9dac1c27 Merge "logd: best 2/3 filter for timezone glitches" 2015-12-10 23:38:03 +00:00
Mark Salyzyn
b06247d9a0 logd: best 2/3 filter for timezone glitches
klogd is sensitive to changes in timezone resulting in glitches
surrounding conversion to local realtime logging. logger manages
a map from monotonic to realtime, but the process is racey since
the system can change the timezone at any time, this catches those
cases where it glitches.

Bug: 21868540
Change-Id: I03de6675fcd04f18ba7306a24dc3d9e750d86976
2015-12-10 13:20:48 -08:00
Mark Salyzyn
f011a335f2 logd: logd-reinit exits with 194
Bug: 26115803
Change-Id: I6e8843ce9766756a40e6d176e0a822f62045c2af
2015-12-10 12:26:08 -08:00
Mark Salyzyn
7b87365ecf logd: readlog apps get logger_entry_v4
Adds the uid field to outgoing content for readlog applications.
AID_LOG, AID_ROOT and AID_SYSTEM gain access to the information.

Bug: 25996918
Change-Id: I0254303c19d174cbf5e722c38844be5c54410c85
2015-12-09 08:12:07 -08:00
Mark Salyzyn
083b037c07 logd: liblog: logcat: Add LOG_ID_SECURITY
- Largish commit, buffer and access controls done together
- Add LOG_ID_SECURITY binary content log
- Add "default" meta buffer
- allow LOG_ID_SECURITY only from AID_SYSTEM and AID_ROOT UID & GID
- Use __android_log_security() to gate logging
- Add __android_log_security_bwrite() native access to security
  logging.
- Add liblog.__security_buffer end-to-end gTest

Bug: 26029733
Change-Id: Ibcf5b4660c17c1aa6902c0d93f8ffd29c93d9a93
2015-12-08 16:46:29 -08:00
Mark Salyzyn
fc9f5b8af4 logd: test wakeup on wrap timeout
Change-Id: I8dd605452f1fef7706a627eedc251e39bb2e34cc
2015-12-07 14:24:02 -08:00
Mark Salyzyn
b75cce0389 logd: wakeup on wrap or timeout
If a timeout is specified for the reader, then go to sleep
with the socket open. If the start time is about to get
pruned in the specified log buffers, then wakeup and dump
the logs; or wakeup on timeout, whichever comes first.

Bug: 25929746
Change-Id: I7d2421c2c5083b33747b84f74d9a560d3ba645df
2015-12-07 14:24:02 -08:00
Mark Salyzyn
3ea02e8eca Merge "logd: test: leniency towards background cgroup" 2015-12-07 20:59:34 +00:00
Mark Salyzyn
2e2d2edc59 Merge "logd: test: statistics report chatty effective percentage" 2015-12-07 19:30:51 +00:00
Mark Salyzyn
074f542476 logd: test: leniency towards background cgroup
Accepting a new lower standard of performance due to
the realities of the background cgroup.

Change-Id: Icd85050ffbf7f0129dd4c053323faedd0ff74048
2015-12-07 11:26:12 -08:00
Mark Salyzyn
f33657da4d logd: test: statistics report chatty effective percentage
Add parsing to recognize optional chatty effective
percentage field as reported in the logger statistics.

Bug: 22855208
Change-Id: Id9c5e4a907ed0f9319beb9ddbfa27f4844bffc7d
2015-12-07 11:10:04 -08:00
Mark Salyzyn
ba7a9a016b logd: liblog: logcat: switch to android_log_clockid() (2)
android_log_timestamp returns the property leading letter,
it is better to return a clockid_t with android_log_clockid()

Bug: 23668800
Change-Id: I38dee773bf3844177826b03a26b03215c79a5359
2015-12-07 18:45:31 +00:00
Mark Salyzyn
b566435b02 Merge "Revert "logd: liblog: logcat: switch to android_log_clockid()"" 2015-12-07 18:31:44 +00:00
Mark Salyzyn
9e18cdcebd Revert "logd: liblog: logcat: switch to android_log_clockid()"
This reverts commit 77b5696b1d.

Change-Id: I7711bf1a7e3f72ed29a2498d7287b725a0e624bd
2015-12-07 18:30:58 +00:00
Mark Salyzyn
9eca0e775b Merge "logd: liblog: logcat: switch to android_log_clockid()" 2015-12-07 18:16:29 +00:00
Elliott Hughes
4f71319df0 Track rename of base/ to android-base/.
Change-Id: Idf9444fece4aa89c93e15640de59a91f6e758ccf
2015-12-04 22:00:26 -08:00
Mark Salyzyn
77b5696b1d logd: liblog: logcat: switch to android_log_clockid()
android_log_timestamp returns the property leading letter,
it is better to return a clockid_t with android_log_clockid()

Bug: 23668800
Change-Id: I3c4e3e6b87f6676950797f1f0e203b44c542ed43
2015-12-04 14:49:19 -08:00
Mark Salyzyn
e6cf1c6372 Merge "Revert "liblog: resolve deadlocks"" 2015-11-16 19:07:47 +00:00
Mark Salyzyn
c2e7d4965f Revert "liblog: resolve deadlocks"
This reverts commit 7a2a307192.

Bug: 25693940
Change-Id: I9a7c926289e972f80c03c92e33535e1dedaa7381
2015-11-16 19:00:41 +00:00
Mark Salyzyn
eb29808615 logd: klogd: parse error
Resolve three areas missing taglen checking. Add some additional
limit-checking paranoia. Problem started when p was allowed to
go beyond the size of the incoming buffer in some blind
p = cp + 1 fragments, placed the check for that after them all
before harm could be done, rather than in each location.

Bug: 25620123
Change-Id: Ib5687fd30ef0cd3ba3bc0df310b436ad675ccabc
2015-11-13 07:58:05 -08:00
Mark Salyzyn
7a2a307192 liblog: resolve deadlocks
Although ever present, an increased regression introduced with
commit b6bee33182 (liblog: logd:
support logd.timestamp = monotonic).

A signal handler can interrupt in locked context, if log is written
in the signal handler, we are in deadlock. Block signals while we
are locked. Separate out timestamp lock from is loggable lock to
reduce contention situations. Provide a best-guess response if
lock would fail in timestamp path.

Bug: 25563384
Change-Id: I6dccd6b99ebace1c473c03a785a35c63ed5c6a8a
2015-11-12 15:39:04 -08:00
Nick Kralevich
c39ba5ae32 Enable hidepid=2 on /proc
Add the following mount options to the /proc filesystem:

  hidepid=2,gid=3009

This change blocks /proc access unless you're in group 3009
(aka AID_READPROC).

Please see
  https://github.com/torvalds/linux/blob/master/Documentation/filesystems/proc.txt
for documentation on the hidepid option.

hidepid=2 is preferred over hidepid=1 since it leaks less information
and doesn't generate SELinux ptrace denials when trying to access
/proc without being in the proper group.

Add AID_READPROC to processes which need to access /proc entries for
other UIDs.

Bug: 23310674
Change-Id: I22bb55ff7b80ff722945e224845215196f09dafa
2015-11-09 09:08:46 -08:00
Mark Salyzyn
8897a3fffd Merge "liblog: logd: support logd.timestamp = monotonic" 2015-11-04 00:31:53 +00:00
Mark Salyzyn
50122695d2 logd: pruning time horizon
Estimate a time horizon of ten fold for worst UID pruning

Bug: 24782000
Change-Id: I7118deb6d42531c68ed2ac2a59c57b0580b942cc
2015-11-03 15:34:15 -08:00
Mark Salyzyn
b6bee33182 liblog: logd: support logd.timestamp = monotonic
if ro.logd.timestamp or persist.logd.timestamp are set to the value
monotonic then liblog writer, liblog printing and logd all switch to
recording/printing monotonic time rather than realtime. If reinit
detects a change for presist.logd.timestamp, correct the older entry
timestamps in place.

ToDo: A corner case condition where new log entries in monotonic time
      occur before logd reinit detects persist.logd.timestamp, there
      will be a few out-of-order entries, but with accurate
      timestamps. This problem does not happen for ro.logd.timestamp
      as it is set before logd starts.

NB: This offers a nano second time accuracy on all log entries
    that may be more suitable for merging with other system
    activities, such as systrace, that also use monotonic time. This
    feature is for debugging.

Bug: 23668800
Change-Id: Iee6dab7140061b1a6627254921411f61b01aa5c2
2015-11-03 15:15:51 -08:00
Mark Salyzyn
14105639a9 Merge "logd: statistics report chatty effective percentage"
am: d2c8cf8265

* commit 'd2c8cf826587de9ef1269ba22fd70df00f874c23':
  logd: statistics report chatty effective percentage
2015-10-30 22:02:15 +00:00
Mark Salyzyn
2c0f84da2e Merge "logd: Add support for *.logd.filter"
am: 95ec339f5e

* commit '95ec339f5ef68696630df470e172bbb306faa1b6':
  logd: Add support for *.logd.filter
2015-10-30 22:02:10 +00:00
Mark Salyzyn
c723df805a logd: statistics report chatty effective percentage
Report the ESTIMATED instantaneous percentage decrease or increase
that an UID has to the logs as a result of the chatty filtration.

Bug: 22855208
Change-Id: If1e77afb81a2739a72b39bc7c57071763c1d64d8
2015-10-30 14:54:17 -07:00
Mark Salyzyn
932f7acc81 logd: Add support for *.logd.filter
- Add device (ro.logd.filter), persistent (persist.logd.filter)
  properties to control the default filters
- Allow logcat -P default to produce expected results
- Allow logcat -P disable to produce expected results

Change-Id: I651cb705373ec1e88a99e4b9086da4f9668a468a
2015-10-30 14:54:17 -07:00
Andreas Gampe
31e78998b0 Merge "Logd: Fix missing include of rc file"
am: 054c099f1b

* commit '054c099f1bbc814e7b20cc20c3ba369161e130e7':
  Logd: Fix missing include of rc file
2015-10-23 04:15:51 +00:00
Andreas Gampe
f5852ce11b Logd: Fix missing include of rc file
This went missing in commit aeca97ba1c.

Change-Id: I9d7e48a2ffb1649f3fa515a9d12d5fbdd2ed58b7
2015-10-22 20:36:53 -07:00
Daniel Cashman
2d78403633 Merge "logd: use libpackageparser"
am: 89065609c9

* commit '89065609c904c8ffdf0fdaec13ede2989ceb110e':
  logd: use libpackageparser
2015-10-22 23:12:10 +00:00
William Roberts
aeca97ba1c logd: use libpackageparser
Switch from the internal packages.list file parser
implementation to a common parser library.

Change-Id: I87a406802f95d8e7bfd8ee85f723f80e9e6b6c0c
Signed-off-by: William Roberts <william.c.roberts@intel.com>
2015-10-14 20:22:44 -07:00
Mark Salyzyn
ff5ef45e29 am cee44732: Merge "logd: correct for number of elements in prune"
* commit 'cee44732e61c3e0f9491aa6cea78670ed6c9c89d':
  logd: correct for number of elements in prune
2015-10-13 22:25:43 +00:00
Mark Salyzyn
58b8be8906 logd: correct for number of elements in prune
Chatty logs would distort the average log size by elevating the
elements, but not the size. Add statistical collection for the
number of elements that report chatty, and subtract that from
the number of elements to improve the pruning estimate. Pick
minElements as 1% rather than 10% of the total with this more
accurate number of elements, to a minumum of 4.

Bug: 24511000
Change-Id: I3f36558138aa0b2a50e4fac6440c3a8505d95276
2015-10-13 13:43:16 -07:00
Mark Salyzyn
904864554e am 9b3a2784: Merge "logd: use coalesce instead of merge (cleanup)"
* commit '9b3a2784b94d88492906384ab3f3c9863f6b5eea':
  logd: use coalesce instead of merge (cleanup)
2015-10-13 16:52:02 +00:00
Mark Salyzyn
9b3a2784b9 Merge "logd: use coalesce instead of merge (cleanup)" 2015-10-13 16:47:15 +00:00
Mark Salyzyn
7c009ac097 am 1c04253a: Merge "logd: object layer format statistics"
* commit '1c04253af2077de0250e4082cadd4f9bfffc4ea2':
  logd: object layer format statistics
2015-10-12 22:14:26 +00:00
Mark Salyzyn
1c04253af2 Merge "logd: object layer format statistics" 2015-10-12 22:10:30 +00:00
Mark Salyzyn
aaad42f47c logd: use coalesce instead of merge (cleanup)
- switch to coalesce instead of merge in naming of functions
  and variables. Confusing since we also to merge-sorts and
  other activities in the logger.
- define maxPrune rather than using a number in the code path.

Bug: 24511000
2015-10-09 15:00:44 -07:00
Bill Yi
27bdd9a71c Merge commit '2b88845e4d7ae862d9735eec3c03d633d4f1d58d' into HEAD 2015-10-08 10:37:51 -07:00
Mark Salyzyn
2b25c66070 logd: clearAll by UID speedup
- If doing a clear, skip accounting
- Ensure for busy checking, behind a region lock for instance, only
  break out if there was something to do. Basically move the filter
  actions first, and defer checking the region lock to the ends of
  the loops.

Bug: 23711431
Change-Id: Icc984f406880633516fb17dda84188a30d092e01
2015-10-07 16:24:57 -07:00
Mark Salyzyn
c5dc970edc logd: clear return and deal with busy if readers locked
- Propagate to caller the clearing errors, busy blocked by reader.
- For clear, perform retries within logd with a one second lul each,
  telling readers to skip, but on final retry to kill all readers if
  problem still persists due to block reader (or high volume logspammer).

Bug: 23711431
Change-Id: Ie4c46bc9480a7f49b96a81fae25a95c603270c33
2015-10-07 16:23:57 -07:00
Mark Salyzyn
de4bb9c1a7 logd: update region lock after entry has passed to reader socket
- The reader region level indicates the location to protect, but once
  it has been passed to the calling reader, then allow us to go
  one beyond so that a clear or prune thread can remove the entry.

Bug: 23711431
Change-Id: I0f2389858dd8c83366c034360f67d7c363625b56
2015-10-07 16:23:57 -07:00
Mark Salyzyn
86eb38f3ca logd: clientHasLogCredentials false negatives
Vote three times in /proc/pid/status to look for AID_LOG group

If not, we may default to the callers UID, and the net result is
to perform the task related to that UID. For adb logcat and
shell logcat, the UID is AID_SHELL which typically has no logs,
leaving no net action taken.

Bug: 23711431
Change-Id: I2b5900a2d37173bd995eb308ee9ecafa20602b62
2015-10-07 16:08:28 -07:00
Mark Salyzyn
1c3009c644 am 2b5f97cc: am 88e01661: Merge "logd: optimize code hotspots"
* commit '2b5f97cc0d3ecddcebee0e1ddc79bd7f994dc2b3':
  logd: optimize code hotspots
2015-10-05 16:31:07 +00:00
Mark Salyzyn
b6a1e85004 am be162cf2: am 72146b93: Merge "logd: Add LogUtils.h"
* commit 'be162cf236fb790973a05874919cc01f25434481':
  logd: Add LogUtils.h
2015-10-05 16:31:05 +00:00
Mark Salyzyn
758058ffd8 logd: object layer format statistics
Simplify table generation by placing the line and header formatting
into each type's (UID, PID, TID) object. Switch to const return
values for the ownership passing functions (*ToName() functions
and methods). Use longer variable names to reduce confusion.

Switch from LINES To NUM for pruned column as that more accurately
reflects what is dropped since one entry can contain several lines.

Bug: 22855208
Change-Id: Ib110dce98a68cf5f844eb30f8a192a1f691eeba2
2015-10-05 09:27:57 -07:00
Mark Salyzyn
2b5f97cc0d am 88e01661: Merge "logd: optimize code hotspots"
* commit '88e0166123eca72cbfc28e92d8abaa95b9b9ce0e':
  logd: optimize code hotspots
2015-10-05 16:25:54 +00:00
Mark Salyzyn
be162cf236 am 72146b93: Merge "logd: Add LogUtils.h"
* commit '72146b931ca61e942df91407b561f22648a70bf2':
  logd: Add LogUtils.h
2015-10-05 16:25:53 +00:00
Mark Salyzyn
ddda212faa logd: optimize code hotspots
Discovered that we had a few libc hotspots. Adjust code to generally
reduce or nullify the number of calls to malloc, free, strlen,
strcmp, strncmp, memcmp & strncasecmp. Total gain looks to be about
3% of logd's processing time. malloc still contributes to 3%, but all
others are now total 0.5%.

Bug: 23685592
Change-Id: Ife721121667969260cdb8b055524ae90f5911278
2015-10-02 16:45:22 -07:00
Mark Salyzyn
5ac5c6b193 logd: Add LogUtils.h
Move prototypes from LogBufferElement.h to LogUtils.h

Change-Id: I55e42e17e6c997e35b2b78b87fd2f84f8f393282
2015-10-02 16:36:41 -07:00
Mark Salyzyn
7765a98a90 am 900362c3: am 1b2fb587: Merge changes I42162365,Ia72f1305,I57d1f86c,I026d074e
* commit '900362c36eb531c88d37d8fa57b66c45ed165122':
  logd: klogd deal with nuls in dmesg
  logd: log_strtok_r deal with nuls
  liblog: logprint: printable nul
  logd: klogd: sniff for time correction on Mediatek
2015-09-30 20:11:41 +00:00
Mark Salyzyn
900362c36e am 1b2fb587: Merge changes I42162365,Ia72f1305,I57d1f86c,I026d074e
* commit '1b2fb587eb7db6f9de1dda8663b33d87a2f5a27e':
  logd: klogd deal with nuls in dmesg
  logd: log_strtok_r deal with nuls
  liblog: logprint: printable nul
  logd: klogd: sniff for time correction on Mediatek
2015-09-30 20:04:53 +00:00
Mark Salyzyn
78e4de5899 am db364d25: am 8ec2d8fe: Merge "logd: klogd and Mediatek part deux"
* commit 'db364d251fd72b9689e7573811f6108d9205a4ff':
  logd: klogd and Mediatek part deux
2015-09-30 00:11:21 +00:00
Mark Salyzyn
db364d251f am 8ec2d8fe: Merge "logd: klogd and Mediatek part deux"
* commit '8ec2d8fec6d099d44151b1e29366002e569d3c2a':
  logd: klogd and Mediatek part deux
2015-09-30 00:05:41 +00:00
Mark Salyzyn
151beac76d logd: klogd deal with nuls in dmesg
Switch to using string and length in all transactions, treating
trailing nuls the same as spaces.

ToDo: change dumpstate (bugreport) to use logcat -b printable _regardless_

Bug: 23517551
Change-Id: I42162365e6bf8ed79d356e7b689a673902116fdb
2015-09-29 15:51:38 -07:00
Mark Salyzyn
ea1a241107 logd: log_strtok_r deal with nuls
Rename to log_strntok_r and change from dealing with strings
to dealing with a string and an associated length.

Bug: 23517551
Change-Id: Ia72f1305a53f55eeef9861ac378fb8205fd2378e
2015-09-29 15:43:34 -07:00
Mark Salyzyn
2d159bf3b5 logd: klogd: sniff for time correction on Mediatek
Need some more flexibility when parsing kernel messages
cluttered with extra fluff. This is the minimal relaxation
we can do to the rules to ensure that we work on all
possible devices and kernels when sniffing for time
correction information.

We want to minimize any future maintenance, keep in mind
klogd is a "userdebug" or "eng" feature and is disabled
in "user" builds. Manage expectations.

Bug: 23517551
Change-Id: I026d074e14fb2550e728683e85a973bd87e78a9c
2015-09-29 15:43:07 -07:00
Mark Salyzyn
47dba71f24 logd: klogd and Mediatek part deux
- switch to an open coded strnrchr
- validity checking on n, taglen and b values.

Bug: 23517551
Change-Id: I568e25c5aa6d8474835454a0e83b19c2921b7986
2015-09-29 15:40:08 -07:00
Mark Salyzyn
936d65f59b am cb2ab20d: am 87122ce1: Merge "logd: increase dgram_max_qlen to 600"
* commit 'cb2ab20d480cdf488e5b5bf37dc55ca7f0ddee67':
  logd: increase dgram_max_qlen to 600
2015-09-28 15:54:05 +00:00
Mark Salyzyn
cb2ab20d48 am 87122ce1: Merge "logd: increase dgram_max_qlen to 600"
* commit '87122ce1d60dd6fbc18e747d976db734d502f2bf':
  logd: increase dgram_max_qlen to 600
2015-09-28 15:50:02 +00:00