LogTimes has evolved from being simply a store of the last timestamp
that each reader has read to being a class representing an individual
reader thread, including the thread function, so name it
appropriately.
Test: logging unit tests
Change-Id: I6914824376a6ff1f7509e657fa4dc044ead62954
I added mOldest recently before mentally committing to have new code
follow the Google C++ style guide.
Test: build
Change-Id: I6d5bab5833e14ac3808862598a2a60989d805e18
logd needs a pointer to PruneList, but it should not own it and it
should not have initPrune() or formatPrune() functions.
Test: logging unit tests
Change-Id: Id1668c26d07eb5d1e4cf267f5748c20a79f711ae
LogBuffer needs a pointer to LogTags, but it should not own the
instance. It should not provide accessors into LogTags either.
Also, clean up CommandListener a bit.
Test: logging unit tests
Change-Id: Ic0c86a2bac0c4dd80262278588b9fdc2326dbe5b
This code was duplicated throughout LogBuffer.cpp, so refactor it into
a single location and clean up some comments along the way.
This may fix a subtle bug: if `logcat -c` is used from a
non-privileged UID, the current code may set mLast to the oldest seen
log message *from that UID* and not not the oldest log message for
that log id. That may prevent pruning from the start of that log,
resulting in old log entries that are impossible to delete.
Bug: 144382260
Test: logging works, and above scenario is not seen
Change-Id: I1749293ce6ea1697dd8a9258cfd7eab29dbeac6e
This reverts commit 5a34d6ea43.
There is a long standing bug that logd will leak memory during its
prune process if the time on the device changes significantly forwards
then backwards. This is due to using the timestamp of each log
message to determine what log messages are yet to be processed by a
reader thread.
Various attempts have been made to rectify this, but the only solution
that safely fixes this issue is to go back to using sequence numbers
on the log messages.
Bug: 64675203
Bug: 77971811
Bug: 149340579
Bug: 150923384
Test: logcat output looks sane
Change-Id: Ibce79cf184eb29a4914f3e42a8cb2868d04dc165
This reverts commit 0878a7c167.
This is a partial revert of the above commit. It simply removes
pruneMargin from these calculations since it is going away.
Bug: 149340579
Test: --clear works well
Change-Id: I1763be56fae7052058a800fad3b295c73cdcadf6
logd isn't meant to be modularized. The previous user was using a
small subset of LogListener.cpp, which is now copied into their
project.
Test: liblog, logd unit tests
This reverts commit fafea32468.
Change-Id: I05ec764db2d9395f2d5b69a1a610c9c55240ab3a
We don't want to fake socket credentials if they were not provided by
the kernel. If there is a bug preventing us from reading the
credentials then it must be solved directly.
Test: logd, liblog unit tests
Test: boot and ensure overflow uid doesn't show up
This reverts commit c4e4823b00.
Change-Id: I683129a8a214637635f163ae25c39bb8a47cd50f
While a reader is present, consider it a success, and not busy, if a
buffer is pruned down to pruneMargin plus one second of additional
margin of logspan. If not busy, no need to trigger any mitigations
regarding the readers, or to report any errors.
Side Effects are we no longer mitigate the reader when performing
chatty filtration. This is a positive side effect because we were
getting --wrap wakeups that seemed premature.
Add kickMe() and isBusy() methods to ease maintenance and uniformity
of actions.
Test: gTest liblog-unit-tests, logd-unit-tests & logcat-unit-tests
Test: manual: 'logcat -b all -c' repeat in a loop, at various logging
load levels, simultaneously 'logcat -b' all in another session.
Bug: 38046067
Change-Id: I3d0c8a2d416a25c45504eda3bfe70b6f6e09ab27
- android::pidToUid() additional checking. Make sure if we have to
convert a PID to an UID that the parse of /proc/<pid>/status
requires a trailing space after the number
- android::tidToPid() added, in the same vein as android::pidToUid().
- stats.tidToPid() added
- If no credentials, set PID to 0 and UID to DEFAULT_OVERFLOWUID
- If credentialed PID is 0, use stats.tidToPid()
- If credentialed UID is DEFAULT_OVERFLOWUID, use stats.pidToUid()
Test: remove +passcred from logd.rc for daemon and confirm very few
UID=65534 or PID=0 cases actually show up
Bug: 37985222
Change-Id: I7d20506e70e67beb3043d1537cf9450ab58dc278
Separates logd body into a static library liblogd and
virtualize LogBuffer::log to be in a new interface class
LogBufferInterface. User could have different implementation.
Bug: 37756450
Test: liblog-unit-tests, logd-unit-tests and logcat-unit-tests
with (b/37791296).
Change-Id: I1504ff0e992744001b5a2e9abd45692d1318a152
Switch to a reader writer lock for the Element List lock. Also setup
for a reader writer lock for the Times list, but continue to use a
mutex where rdlock() and wrlock() are the same implementation for now.
This should improve general reader performance and prevent blocking of
other reader operations or exit by a single hung logd.reader.per
thread. For example, a full length logcat of an empty buffer (eg:
crash log buffer) will hold a lock while the iterator scans the entire
list.
Test: gTest liblog-unit-tests, logd-unit-tests, logcat-unit-tests
Bug: 37378309
Bug: 37483775
Change-Id: If5723ff4a978e17d828a75321e8f0ba91d4a09e0
Deal with a regression introduced in commit
5a34d6ea43 (logd: drop mSequence from
LogBufferElement) where log_time was compared against nsec() time
miscalculating the watermark boundary. When dealing with logcat
-t/-T, or any tail reading, add a margin to prune to back off by a
period of 3 seconds (pruneMargin).
Test: gTest liblog-unit-tests logcat-unit-tests and logd-unit-tests
Bug: 37378309
Change-Id: I72ea858e4e7b5fa91741ea84c40d2e7c3c4aa031
Move lastTid array from local in LogBuffer::flushTo to per-reader
context in LogTimes::mLastTid and pass into LogBuffer::flushTo.
Replace NULL with nullptr in touched files.
Simplify LogTimeEntry::cleanSkip_Locked initialization of skipAhead
to memset, to match mLastTid memset initialization.
Test: gTest liblog-unit-tests, logd-unit-tests & logcat-unit-tests
Test: adb logcat -b all | grep chatty | grep -v identical
Bug: 36488201
Change-Id: I0c3887f220a57f80c0490be4b182657b9563aa3f
Use getRealTime() instead and leverage private liblog log_time
comparison and math functions. This saves 8 bytes off each
element in the logging database.
Test: gTest liblog-unit-tests logd-unit-tests logcat-unit-tests
Bug: 35373582
Change-Id: Ia55ef8b95cbb2a841ccb1dae9a24f314735b076a
Switch _all_ file's coding style to match to ease all future changes.
SideEffects: None
Test: compile
Bug: 35373582
Change-Id: I470cb17f64fa48f14aafc02f574e296bffe3a3f3
This is the precursor for "Plan B" recovery when access to
/dev/event-log-tags is blocked to untrusted zones. Also
deals with mitigating issues with long-lived mappings that
do not update /dev/event-log-tags when dynamically changed.
Test: gTest logd-unit-test --gtest_filter=logd.getEventTag_42
Bug: 31456426
Bug: 35326290
Change-Id: I3db2e73763603727a369da3952c5ab4cf709f901
Will register a new event tag by name and format, and return an
event-log-tags format response with the newly allocated tag.
If format is not specified, then nothing will be recorded, but
a pre-existing named entry will be listed. If name and format are
not specified, list all dynamic entries. If name=* list all
event log tag entries.
Stickiness through logd crash will be managed with the tmpfs file
/dev/event-log-tags and through a reboot with add_tag entries in
the pmsg last logcat event log. On debug builds we retain a
/data/misc/logd/event-log-tags file that aids stickiness and that
can be picked up by the bugreport.
If we detect truncation damage to /dev/event-log-tags, or to
/data/misc/logd/event-log-tags, rebuild file with a new first line
signature incorporating the time so mmap'd readers of the file can
detect the possible change in shape and order.
Manual testing:
Make sure nc (netcat) is built for the target platform on the host:
$ m nc
Then the following can be used to issue a request on the platform:
$ echo -n 'getEventTag name=<name> format="<format>"\0EXIT\0' |
> nc -U /dev/socket/logd
Test: gTest logd-unit-test --gtest_filter=getEventTag*
Bug: 31456426
Change-Id: I5dacc5f84a24d52dae09cca5ee1a3a9f9207f06d
If a series of messages arrive from a single source with identical
message content payload, then suppress them and generate a chatty
report. The checking is done on a per log id basis.
This alters the assumption that chatty messages are always at the
oldest entries, they now show up in the middle too. To address this
change in behavior we print the first line, a chatty reference
which internally takes little space, then the last line in the series.
This does not conserve processing time in logd, and certainly has no
impact on the long path of formatting and submitting log messages from
from the source, but it may contribute to memory space and signal to
noise savings under heavy spammy loads.
Test: gTest liblog-unit-tests, logd-unit-tests & logcat-unit-tests
Bug: 33535908
Change-Id: I3160c36d4f4e2f8216f528605a1b3993173f4dec
Should use android/log.h instead of log/log.h as a good example
to all others.
Test: Compile
Bug: 26552300
Bug: 31289077
Change-Id: If4c9711eb57267d4707b03d54a932af9de241b13
Always used in combination with log/logger.h except in log_time.cpp,
and not used externally. As a result liblog has to support stl, a
small price to pay since goal is to convert liblog to C++ internally.
Test: compile
Bug: 31456426
Bug: 26552300
Bug: 31289077
Change-Id: I72828ec807d0a2c8e40bbdebd7a69f147a7ca5a9
- Add drop logistics to TagTable
- replace uid references to a key reference since it
is an UID for most buffers, but a TAG for the
events and security buffer
- template the find worst entry mechanics into LogFindWorst class
Bug: 30118730
Change-Id: Ibea4be2c50d6ff4b39039e371365fed2453f17a2
- limit AID_SYSTEM uid or gid to read security buffer messages
- adjust liblog tests to reflect the reality of this adjustment
To fully test all security buffer paths and modes
$ su 0,0,0 /data/nativetest/liblog-unit-tests/liblog-unit-tests --gtest_filter=liblog.__security*
$ su 1000,1000,1000 /data/nativetest/liblog-unit-tests/liblog-unit-tests --gtest_filter=liblog.__security*
$ su 2000,2000,2000 /data/nativetest/liblog-unit-tests/liblog-unit-tests --gtest_filter=liblog.__security*
ToDo: Integrate the above individually into the gTest Q/A testing
Bug: 26029733
Change-Id: Idcf5492db78fa6934ef6fb43f3ef861052675651
Without this change LogBuffer::prune and LogBuffer::erase
contributes 16.7% and 1.79% respectively. With this change,
they contributes 3.06 and 2.33% respectively. Pruning is
performed roughly 1 in every 255 log entries, a periodic
tamer latency spike.
Bug: 23685592
Change-Id: I6ae1cf9f3559bca4cf448efe8bcb2b96a1914c54
Use 1972 as a right delineation. Otherwise use half way point
between the monotonic and realtime. Treat correction factor as
unsigned, ensure that any wrapping to a negative value is
dropped or set to EPOCH. Acknowledge that we can get a more
accurate time track by acquiring the time rather than relying on
healthd timestamp.
Bug: 26331432
Change-Id: I09075fca58676a30cf7d87baf2d4b0f53795abaa
- Add a new statistic that reports per pid and log_id for AID_SYSTEM
- Add a new pruning filter ~1000/! boolean
- Use this new statistic to prune on worst pid within AID_SYSTEM
Bug: 26029733
Bug: 21615139
Bug: 22855208
Change-Id: Iab5dd28f807dcf03d276372853883f3b6afa8294
Primarily gives access to the Chattiest TIDs and TAGs
associated with a pid.
Has a secondary effect of allowing us to pull out the
command line, comm and in some cases the associated
PACKAGE for a specific pid while the logs are still
present even if the executable is gone.
Bug: 26029733
Bug: 21615139
Change-Id: I1ea63165a680a9318360579b70b1512078ed5682
if ro.logd.timestamp or persist.logd.timestamp are set to the value
monotonic then liblog writer, liblog printing and logd all switch to
recording/printing monotonic time rather than realtime. If reinit
detects a change for presist.logd.timestamp, correct the older entry
timestamps in place.
ToDo: A corner case condition where new log entries in monotonic time
occur before logd reinit detects persist.logd.timestamp, there
will be a few out-of-order entries, but with accurate
timestamps. This problem does not happen for ro.logd.timestamp
as it is set before logd starts.
NB: This offers a nano second time accuracy on all log entries
that may be more suitable for merging with other system
activities, such as systrace, that also use monotonic time. This
feature is for debugging.
Bug: 23668800
Change-Id: Iee6dab7140061b1a6627254921411f61b01aa5c2
Chatty logs would distort the average log size by elevating the
elements, but not the size. Add statistical collection for the
number of elements that report chatty, and subtract that from
the number of elements to improve the pruning estimate. Pick
minElements as 1% rather than 10% of the total with this more
accurate number of elements, to a minumum of 4.
Bug: 24511000
Change-Id: I3f36558138aa0b2a50e4fac6440c3a8505d95276
- switch to coalesce instead of merge in naming of functions
and variables. Confusing since we also to merge-sorts and
other activities in the logger.
- define maxPrune rather than using a number in the code path.
Bug: 24511000
- Propagate to caller the clearing errors, busy blocked by reader.
- For clear, perform retries within logd with a one second lul each,
telling readers to skip, but on final retry to kill all readers if
problem still persists due to block reader (or high volume logspammer).
Bug: 23711431
Change-Id: Ie4c46bc9480a7f49b96a81fae25a95c603270c33
Simplify table generation by placing the line and header formatting
into each type's (UID, PID, TID) object. Switch to const return
values for the ownership passing functions (*ToName() functions
and methods). Use longer variable names to reduce confusion.
Switch from LINES To NUM for pruned column as that more accurately
reflects what is dropped since one entry can contain several lines.
Bug: 22855208
Change-Id: Ib110dce98a68cf5f844eb30f8a192a1f691eeba2
With part deux we caused an apparent regression by not checking for
stale recorded iterators. This checking was on-purpose bypassesed
when leading prune entries were to be deleted without touching the
statistics engine due to an in-place merge.
Part deux had us leaving iterators we were not focussed on untouched
which in turn because they were left behind, had a much higher
likelihood of being deleted without touching the statistics engine.
Perform the check every delete.
Bug: 23789348
Change-Id: Idc6cc23d1f9e3b6cd9a083139a0de59479fbfe08
Hold on to last worst uid watermark and bypass a spike to O(n*n*x)
(n=samples, x=number of spammers) wrt chatty trimming.
Bug: 23327476
Change-Id: I9f21ce95e969b67e576417a760f75c4d86acf364
Quick low-risk to resolve possible hash table corruption.
Resolved an unlikely path memory leak.
ToDo: replace lock with nested lock so no lock
helpers are required.
Bug: 22068332
Change-Id: I303ab06608502c7d61d42f111a9c43366f184d0c
- Report applications UID, TID/PID by name.
- change wording to have an accurate connotation
- drop privilege check since filtered upstream
Bug: 19608965
Bug: 20334069
Bug: 20370119
Change-Id: I2b1c26580b4c2de293874214ff5ae745546f3cca
Add a return value for the ::log() methods, this allows
us to optimize the wakeup for the readers to only occur
when the log message is actually placed.
This is for a future where we may dedupe identical log
messages, filter out log messages, and certainly if we
filter the messages out with an internal logd check of
__android_log_is_loggable().
Change-Id: I763b2a7c29502ab7fa0a5d5022c7b60244fcfde4
- Go back to basic requirements
- Simplify
- use hash tables to minimize memory impact
Bug: 19608965
Change-Id: If7becb34354d6415e5c387ecea7d4109a15259c8
- switch to simpler and faster internal sequence number, drops
a syscall overhead on 32-bit platforms.
- add ability to break-out of flushTo loop with filter return -1
allowing in reduction in reader overhead.
Change-Id: Ic5cb2b9afa4d9470153971fc9197b07279e2b79d
This forward port reverts
commit e457b74ce6
No longer as necessary once we add
liblog: Instrument logging of logd write drops
Although this provided an indication of how close statistically we
were to overloading logd it is simpler to understand fails thus to
hunt and peck a corrected value for /proc/sys/net/unix/max_dgram_qlen
Change-Id: I2b30e0fc30625a48fd11a12c2d2cc6a41f26226f
