platform_system_core/logd
Mark Salyzyn 247d682fe1 logd: sepolicy dynamic rate limiting
Processing overhead for selinux violation messages is costly. We want
to deal with bursts of violations, but we have no intent of allowing
that sustained burst to go unabated as there is a cost of processing
and battery usage.

Tunables in libaudit.h are:

AUDIT_RATE_LIMIT_DEFAULT 20        /* acceptable burst rate      */
AUDIT_RATE_LIMIT_BURST_DURATION 10 /* number of seconds of burst */
AUDIT_RATE_LIMIT_MAX     5         /* acceptable sustained rate  */

Since we can only asymptotically handle DEFAULT rate, we set an upper
threshold of half way between the MAX and DEFAULT rate.

Default kernel audit subsystem message rate is set to 20 a second.
If sepolicy exceeds 125 violation messages over up to ten seconds
(>=~12/s), tell kernel audit subsystem to drop the rate to 5 messages
a second.  If rate drops below 50 messages over the past ten seconds
(<5/s), tell kernel it is ok to increase the burst rate back to 20
messages a second.

Test: gTest logd-unit-tests --gtest_filter=logd.sepolicy_rate_limiter_*
Bug: 27878170
Change-Id: I843f8dcfbb3ecfbbe94a4865ea332c858e3be7f2
2017-01-04 14:46:58 -08:00
..
tests logd: sepolicy dynamic rate limiting 2017-01-04 14:46:58 -08:00
Android.mk logd: sum liblog tag messages 2016-12-19 14:03:38 -08:00
CommandListener.cpp logd: add EXIT command 2016-11-17 14:42:33 -08:00
CommandListener.h logd: add EXIT command 2016-11-17 14:42:33 -08:00
event.logtags logd: Replace logd with chatty log tag 2016-07-15 14:57:58 -07:00
FlushCommand.cpp logd: Allow (some) headers to be individually importable 2016-02-25 12:42:31 -08:00
FlushCommand.h liblog: logd: logcat: Split out log/logger.h into public and private. 2016-10-24 11:12:49 -07:00
libaudit.c logd: sepolicy dynamic rate limiting 2017-01-04 14:46:58 -08:00
libaudit.h logd: sepolicy dynamic rate limiting 2017-01-04 14:46:58 -08:00
LogAudit.cpp logd: sepolicy dynamic rate limiting 2017-01-04 14:46:58 -08:00
LogAudit.h logd: sepolicy dynamic rate limiting 2017-01-04 14:46:58 -08:00
LogBuffer.cpp logd: LogBufferElementKey use uint32_t for uid 2016-12-21 12:16:46 -08:00
LogBuffer.h logd: record multiple duplicate messages as chatty 2016-12-15 16:31:51 -08:00
LogBufferElement.cpp logd: record multiple duplicate messages as chatty 2016-12-15 16:31:51 -08:00
LogBufferElement.h logd: record multiple duplicate messages as chatty 2016-12-15 16:31:51 -08:00
LogCommand.cpp logd: liblog: logcat: Add LOG_ID_SECURITY 2015-12-08 16:46:29 -08:00
LogCommand.h Fix google-explicit-constructor warnings in system/core. 2016-07-26 11:26:01 -07:00
logd.rc logd: start logd service in logd uid 2016-11-03 13:34:27 -07:00
LogKlog.cpp logd: record multiple duplicate messages as chatty 2016-12-15 16:31:51 -08:00
LogKlog.h liblog: logd: logcat: Split out log/logger.h into public and private. 2016-10-24 22:53:11 +00:00
LogListener.cpp system/core: preparation to pull back interfaces from android/log.h 2016-10-20 08:11:39 -07:00
LogListener.h
LogReader.cpp utils: Add FastStrcmp.h 2016-12-09 12:40:17 -08:00
LogReader.h Fix google-explicit-constructor warnings in system/core. 2016-07-26 11:26:01 -07:00
LogStatistics.cpp Merge "Revert "logd: trailing spaces in log statistics"" 2016-12-20 18:16:00 +00:00
LogStatistics.h Merge "logd: trailing spaces in log statistics (part deux)" 2016-12-20 18:16:15 +00:00
LogTimes.cpp logd: clarify release_Locked() for static analyzer 2016-03-02 11:37:21 -08:00
LogTimes.h liblog: logd: logcat: Split out log/logger.h into public and private. 2016-10-24 11:12:49 -07:00
LogUtils.h logd: add android::sizesTotal() function 2016-12-19 08:00:16 -08:00
LogWhiteBlackList.cpp logd: Add worst pid of system filter 2015-12-29 09:32:35 -08:00
LogWhiteBlackList.h logd: Add worst pid of system filter 2015-12-29 09:32:35 -08:00
main.cpp logd: Add support for ro.logd.auditd.[main|events] 2017-01-03 09:44:42 -08:00
README.auditd logd: selinux auditd initial commit 2014-04-07 10:51:00 -07:00
README.property logd: Add support for ro.logd.auditd.[main|events] 2017-01-03 09:44:42 -08:00

The properties that logd and friends react to are:

name                       type default  description
ro.logd.auditd             bool   true   Enable selinux audit daemon
ro.logd.auditd.dmesg       bool   true   selinux audit messages sent to dmesg.
ro.logd.auditd.main        bool   true   selinux audit messages sent to main.
ro.logd.auditd.events      bool   true   selinux audit messages sent to events.
persist.logd.security      bool   false  Enable security buffer.
ro.device_owner            bool   false  Override persist.logd.security to false
ro.logd.kernel             bool+ svelte+ Enable klogd daemon
ro.logd.statistics         bool+ svelte+ Enable logcat -S statistics.
ro.debuggable              number        if not "1", logd.statistics &
                                         ro.logd.kernel default false.
logd.logpersistd.enable    bool   auto   Safe to start logpersist daemon service
logd.logpersistd          string persist Enable logpersist daemon, "logcatd"
                                         turns on logcat -f in logd context.
					 Responds to logcatd, clear and stop.
logd.logpersistd.buffer          persist logpersistd buffers to collect
logd.logpersistd.size            persist logpersistd size in MB
persist.logd.logpersistd   string        Enable logpersist daemon, "logcatd"
                                         turns on logcat -f in logd context.
persist.logd.logpersistd.buffer    all   logpersistd buffers to collect
persist.logd.logpersistd.size      256   logpersistd size in MB
persist.logd.size          number  ro    Global default size of the buffer for
                                         all log ids at initial startup, at
                                         runtime use: logcat -b all -G <value>
ro.logd.size               number svelte default for persist.logd.size. Larger
                                         platform default sizes than 256KB are
                                         known to not scale well under log spam
                                         pressure. Address the spam first,
                                         resist increasing the log buffer.
persist.logd.size.<buffer> number  ro    Size of the buffer for <buffer> log
ro.logd.size.<buffer>      number svelte default for persist.logd.size.<buffer>
ro.config.low_ram          bool   false  if true, logd.statistics, logd.kernel
                                         default false, logd.size 64K instead
                                         of 256K.
persist.logd.filter        string        Pruning filter to optimize content.
                                         At runtime use: logcat -P "<string>"
ro.logd.filter       string "~! ~1000/!" default for persist.logd.filter.
                                         This default means to prune the
                                         oldest entries of chattiest UID, and
                                         the chattiest PID of system
                                         (1000, or AID_SYSTEM).
persist.logd.timestamp     string  ro    The recording timestamp source.
                                         "m[onotonic]" is the only supported
                                         key character, otherwise realtime.
ro.logd.timestamp        string realtime default for persist.logd.timestamp
log.tag                   string persist The global logging level, VERBOSE,
                                         DEBUG, INFO, WARN, ERROR, ASSERT or
                                         SILENT. Only the first character is
                                         the key character.
persist.log.tag            string build  default for log.tag
log.tag.<tag>             string persist The <tag> specific logging level.
persist.log.tag.<tag>      string build  default for log.tag.<tag>

NB:
- auto - managed by /init
- bool+ - "true", "false" and comma separated list of "eng" (forced false if
  ro.debuggable is not "1") or "svelte" (forced false if ro.config.low_ram is
  true).
- svelte - see ro.config.low_ram for details.
- svelte+ - see ro.config.low_ram and ro.debuggable for details.
- ro - <base property> temporary override, ro.<base property> platform default.
- persist - <base property> override, persist.<base property> platform default.
- build - VERBOSE for native, DEBUG for jvm isLoggable, or developer option.
- number - support multipliers (K or M) for convenience. Range is limited
  to between 64K and 256M for log buffer sizes. Individual log buffer ids
  such as main, system, ... override global default.
- Pruning filter is of form of a space-separated list of [~][UID][/PID]
  references, where '~' prefix means to blacklist otherwise whitelist. For
  blacklisting, UID or PID may be a '!' to instead reference the chattiest
  client, with the restriction that the PID must be in the UID group 1000
  (system or AID_SYSTEM).