Commit graph

139 commits

Author SHA1 Message Date
Treehugger Robot
70025585ff Merge "bootstat: record reboot reason history in persist.sys.boot.reason.history" 2018-07-02 20:28:49 +00:00
Mark Salyzyn
5c58c9d566 bootstat: record reboot reason history in persist.sys.boot.reason.history
Retain the last four reboot reasons from latest to oldest, adding a
detail of UTC epoch time for each.  Beef up boot_reason_test.sh to
deal with multi-line content.

Test: boot_reason_test.sh
Bug: 110925971
Change-Id: Ie2600434b95a885693f005a363ee38d9eba18ee7
2018-06-29 13:54:07 -07:00
Mark Salyzyn
88d1b4a326 bootstat: kpdpwr actually means reboot,longkey
Test: compile
Bug: 71809701
Change-Id: I812777645b988f9627bcf62e7ef70cc2085d9106
2018-06-28 17:40:26 +00:00
Elliott Hughes
50a24eb3bc bootstat: remove the only caller of liblogcat.
We need to make progress both on adding the real interface for battery
level and cleaning up logging. This stands in the way of both.

Bug: http://b/77725702
Test: builds
Change-Id: Ia457e497606c2c7965d6895baebb26eef17857c9
Merged-In: Ia457e497606c2c7965d6895baebb26eef17857c9
2018-06-15 12:36:46 -07:00
Mark Salyzyn
adc433d3ea bootstat: clear persist.sys.boot.reason once read
To ensure a surprise reboot does not take the last boot reason on
face value especially if coming from more than one boot sessions ago.
We shift and clear the value from persist.sys.boot.reason to
sys.boot.reason.last and establish a correct last reboot reason in
the canonical sys.boot.reason property.

This effectively deprecates persist.sys.boot.reason as an API.  They
should have been using sys.boot.reason instead for a correctly
determined reasoning.

Test: boot_reason_test.sh
Bug: 86671991
Merged-In: If85750704445088fd62978679ab3a30744c46abb
Change-Id: If85750704445088fd62978679ab3a30744c46abb
2018-06-14 15:36:56 +00:00
Treehugger Robot
cefd4e56ad Merge "bootstat: Remove the CAP_SYSLOG capability" 2018-06-12 20:59:45 +00:00
Luis Hector Chavez
0becca32bc bootstat: Remove the CAP_SYSLOG capability
This change removes the CAP_SYSLOG file based capability from bootstat,
since the intention is that it should not be accessing the logs in the
long term. In order to avoid bitrot, the fallback code that depends on
CAP_SYSLOG has also been removed.

Bug: 62845925
Test: system/core/bootstat/boot_reason_test.sh
Change-Id: I899be44ef3ac1c4d81072f801d55c928ae09bb15
2018-06-11 12:41:24 -07:00
Mark Salyzyn
78e54fd0ab bootstat: ppdone_timeout kernel panic
Add kernel_panic,_sde_encoder_phys_cmd_handle_ppdone_timeout to help track
occurrences.

Test: compile
Bug: 67728931
Change-Id: I3306ae2936dc4355902e838d9ac8d8b11837348d
2018-06-08 10:37:06 -07:00
Tej Singh
993f65619e Fix performance degradation from BootSequence atom
(partial cherry pick from commit fe3e762b6d)

Adding the boot sequence reported atom in ag/3518079 caused the duration
of bootstat to increase, as seen in b/72864061. I isolated the cause
down to calling BootReasonStrToReason. However, this function also gets
called in ReportBootReason, so I created another function that does the
parsing and sets the system boot reason property, and made
RecordBootReason and statsd logging get that property.

Bug: 72864061
Test: rebooted phone, verified boot events were received in adb shell
logcat -b stats and verified adb shell bootstat -p printed correct
values. Ran timing tests as well on walleye with 20 boots: before this
change, the average was ~150-160ms. After, it was ~80ms.
Change-Id: I92dbc9880328835647be7d9d50c7861b42f27bdb
Merged-In: I92dbc9880328835647be7d9d50c7861b42f27bdb
2018-06-04 13:31:05 -07:00
Mark Salyzyn
186f67625c bootstat: refine boot reasons
Received some clarity as to some of the boot reasons.

List of boot reasons and new translations to Canonical boot reason:

- "power_key" -> "cold,powerkey" (existing)
- "usb" -> "cold,charger" (existing)
- "rtc" -> "cold,rtc" (existing)
- "wdt" -> "reboot" (changed)
- "wdt_by_pass_pwk" -> "warm" (changed)
- "tool_by_pass_pwd" -> "reboot,tool" (changed)
- "2sec_reboot" -> "cold,rtc,2sec" (changed)
- "unknown" -> "reboot,unknown" (existing)
- "kernel_panic" (existing)
- "reboot" (existing)
- "watchdog" (existing)

Add the new string to enums for the new Boot Reason.

Test: boot_reason_test.sh (on affected device)
Bug: 74595769
Bug: 63736262
Change-Id: Iecedc3b1f7c47f26d0c77b1f316f745c6d2c1256
2018-06-04 08:32:23 -07:00
Mark Salyzyn
1519925149 bootstat: shutdown reports reboot
Some devices report the following canonical boot reason for all
shutdown operations:

    reboot,kernel_power_off_charging__reboot_system

because shutdown switches to a charging kernel, and reboots into the
system when the user presses the power button.  Thus last kernel
messages arrives as:

    <0>.(0)[53:pmic_thread]reboot: Restarting system with command \
                       'kernel power off charging  reboot system'
     -> "shutdown" (w/o last boot reason)
     -> "shutdown,<subreason>" (w/last boot reason)

The reboot reason from that charging instance propagates as a
fortified boot reason blocking interpretation of the last boot reason
that manages shutdown canonical boot reason determination.  The fix
is to change reboot,kernel_power_off_charging__reboot_system to
shutdown, so that it is viewed as a blunt reason that can be
overridden by last boot reason.

We added the above boot reason to kBootReasonMap because the Bit
Error Handler can use it to reconstruct if there is any damage to
the last kernel messages content.  The sad thing is that the enum
will never propagate as we are filtering it out and reporting
"shutdown" instead.  Of course, we are now covered for a can not
happen.

Test: boot_reason_test.sh
Bug: 74595769
Bug: 63736262
Change-Id: I28987f0871af7d967cc4bbbffed43bd42349acdd
2018-06-04 08:32:11 -07:00
Mark Salyzyn
8ad6e67332 bootstat: Add kernel_panic,modem + related others
Found a kernel modem driver report:

Kernel panic - not syncing: subsys-restart: Resetting the SoC - modem crashed.

Which translates to the canonical boot reason, a wordy:
    kernel_panic,subsys-restart:_resetting_the_soc_-_modem_crashed.

Shortening and ber matching the string, plus others that are possible,
to be more succinct, so added kernel_panic,{modem|adsp|dsps|wcnss}.

Test: build
Bug: 80553005
Change-Id: I969e1da896cd15b82e2fe11ceb77a5f54dfcfbc8
2018-06-01 09:27:48 -07:00
Mark Salyzyn
cd7e122285 bootstat: test reboot rescueparty
This test fails on most devices, gives us a report of devices that
do not propagate the boot reason via the bootloader.  This should
become a bootloader required test.

Test: boot_reason_test.sh optional_rescueparty
Change-Id: Ibdc7b23b025e5d89d659ff08083b2e071343923c
2018-05-07 10:10:18 -07:00
Mark Salyzyn
75046897b3 bootstat: add reboot,rescueparty
Reserve this for Android system use.

Test: none
Change-Id: Iaf29f0fcd1fe68962cf1109843d6452d0ab42e3c
2018-05-03 13:13:34 -07:00
Mark Salyzyn
afd66f2fd3 llkd: bootstat: propagate detailed livelock canonical boot reason
Report kernel_panic,sysrq,livelock,<state> reboot reason via last
dmesg (pstore console).  Add ro.llk.killtest property, which will
allow reliable ABA platforms to drop kill test and go directly
to kernel panic.  This should also allow some manual unit testing
of the canonical boot reason report.

New canonical boot reasons from llkd are:
- kernel_panic,sysrq,livelock,alarm llkd itself locked up (Hail Mary)
- kernel_panic,sysrq,livelock,driver uninterrruptible D state
- kernel_panic,sysrq,livelock,zombie uninterrruptible Z state

Manual test assumptions:
- llkd is built by the platform and landed on system partition
- unit test is built and landed in /data/nativetest (could
  land in /data/nativetest64, adjust test correspondingly)
- llkd not enabled, ro.llk.enable and ro.llk.killtest
  are not set by platform allowing test to adjust all the
  configuration properties and start llkd.
- or, llkd is enabled, ro.llk.enable is true, and killtest is
  disabled, ro.llk.killtest is false, setup by the platform.
  This breaks the go/apct generic operations of the unit test
  for llk.zombie and llk.driver as kernel panic results
  requiring manual intervention otherwise.  If test moves to
  go/apct, then we will be forced to bypass these tests under
  this condition (but allow them to run if ro.llk.killtest
  is "off" so specific testing above/below can be run).

for i in driver zombie; do
        adb shell su root setprop ro.llk.killtest off
        adb shell /data/nativetest/llkd_unit_test/llkd_unit_test --gtest_filter=llkd.${i}
        adb wait-for-device
        adb shell su root setprop ro.llk.killtest off
        sleep 60
        adb shell getprop sys.boot.reason
        adb shell /data/nativetest/llkd_unit_test/llkd_unit_test --gtest_filter=llkd.${i}
done

Test: llkd_unit_test (see test assumptions)
Bug: 33808187
Bug: 72838192
Change-Id: I2b24875376ddfdbc282ba3da5c5b3567de85dbc0
2018-04-18 14:02:16 -07:00
Luis Hector Chavez
03aae156b8 Make bootstat container-friendly
This change allows bootstat to read the ro.boot.boottime_offset
property, which is set on devices where Android runs in a container.
This is because the CLOCK_BOOTTIME clock is not reset when (from
Android's perspective) the device restarts, since the host OS does not
restart itself.

Bug: 77273909
Test: CtsBootStatsTestCases

Change-Id: Ifb792864394be0b4686cc9d555c0afced856f4b4
Merged-In: Ifb792864394be0b4686cc9d555c0afced856f4b4
2018-04-16 11:17:42 -07:00
Mark Salyzyn
3f48fa9cb2 bootstat: refine "Kernel panic - not syncing"
Provide some easy kernel panic subreasons mined from last kmesg,
generates a canonical boot reason (system boot reason) that
may aid triage.

Notably report kernel_panic,hung if [khungtaskd] triggers on
a livelock condition, forms a signals on the dashboards.

Helper function getSubreason modified to optionally enable checking
for a single quote resulting in a refactoring to ease maintenance of
the termination detection in the face of single bit errors heuristics.

Test: boot_reason_test.sh
Bug: 63736262
Bug: 33808187
Change-Id: I7fdd1e57e7a26095738175074306f0d2d59b1d69
2018-03-22 13:57:31 -07:00
Mark Salyzyn
39cc3e7073 bootstat: allow kernel_panic,sysrq to propagate subreason
Allow for a daemon to write to last kmsg to propagate a detailed
subreason to kernel_panic,sysrq actions.  A minor refactor moves
common code into a helper function getSubreason for retrieval and
bit error correction operations.

A sysrq crash generally produces a kernel-provided message:
    SysRq : Trigger a crash
which is used to generate a canonical boot reason kernel_panic,sysrq.
A user daemon could write to /dev/kmsg just prior to the sysrq with
    SysRq : Trigger a crash : '<subreason>'
to change the canonical boot reason to kernel_panic,sysrq,<subreason>.

Administration added pending kBootReasonMap entries present in TRON.

Test: manual echo into /dev/kmsg and /proc/sysrq-trigger and check
Test: boot_reason_test.sh
Bug: 33808187
Bug: 63736262
Change-Id: Ibf5432737e5a3449ebe40a8c6cf2d3e912ed6bbc
2018-03-20 13:11:44 -07:00
Mark Salyzyn
855d205817 bootstat: test error out only on first bad bootloader reason
Hopefully the quick property test is first, setting the stage for
ignoring future failures.  This solves a problem with multiple
test failures directly attributed to a CTS compliance issue
with the bootloader's boot reason.  One test fails, the remainder
get a pass on this one issue.

Test: boot_reason_test.sh
Bug: 63736262
Change-Id: Id56946b6f2f3a33d65bd1830543838f153290759
2018-03-20 13:10:16 -07:00
Mark Salyzyn
281932803d bootstat: add support for suppressing needle match on output
For aliasReasons allow one to optionally suppress needle for output
member using a <bang> (!) character prefix.

Test: boot_reason_test.sh
Bug: 63736262
Bug: 74595769
Change-Id: If35518c08cf909c6c78a16275e9d8dfd0ff839a9
2018-03-16 15:47:27 -07:00
Mark Salyzyn
8aa36c67f8 bootstat: usb and rtc refinement
For all known cases, if usb is present in the bootloader reason,
then it is actually reporting a cold,charger canonical boot reason.
This signifies that the device was powered down, and was woken
up by the charger being connected.

For all known cases, if rtc is present in the bootloader reason,
then it is actually reporting a cold,rtc canonical boot reason.
This signifies that the device was powered down, and was woken
up by the rtc clock.

Test: boot_reason_test.sh
Bug: 74595769
Bug: 63736262
Change-Id: I04bede0b7ccaa1b859943f7def93521a8f7b25c6
2018-03-16 15:47:27 -07:00
Mark Salyzyn
25900dd772 bootstat: add support for regex in aliasReasons and powerkeys
Add support for regex in aliasReasons for the alias member.  Use this
new feature to check powerkey|power_key|PowerKey for a single entry.

Test: boot_reason_test.sh
Bug: 63736262
Change-Id: Ia6add99b9e33f3197643dbaab88dde20aa726f90
2018-03-16 11:41:17 -07:00
Mark Salyzyn
1e7d1c77fa bootstat: alias underline to space in bit error rate handling
When we are matching existing known boot reasons, we should try with
compliant underlines first, then again with underlines replaced with
spaces.  Replace references to Ber with BitError for maintenance
clarity.  Replace helper functions with C++11 lambdas.

Test: boot_reason_test.sh
Bug: 63736262
Change-Id: I91b865013513526a55a85523080c7127f198968c
2018-03-16 10:37:03 -07:00
Mark Salyzyn
2b820536e6 bootstat: comment out duplicate entries in kBootReasonMap
Two entries can be reused. The third "unknown" entry is not really
a duplicate since the kUnknownBootReason is not checked.  Duplicate
entries reused in the future, should have
analysis/uma/configs/clearcut/TRON/histograms.xml updated first.

Test: boot_reason_test.sh
Change-Id: If2071a18160dc2c93e851fecc6b8c11fc76c9845
2018-03-16 10:36:53 -07:00
Mark Salyzyn
853bb80e58 bootstat: 3.18 kernel does not report "SysRq : Trigger a crash"
Use an alternate means to determine that the sysrq crash was
requested.  Also, to be CTS compliant, the kernel_panic subreason
must be in lower case.

Test: boot_reason_test.sh
Bug: 74595769
Bug: 63736262
Change-Id: Ica06960ce62d220a909006e365951376d672b7e6
2018-03-16 09:39:17 -07:00
Mark Salyzyn
7f5a562177 bootstat: test: regression from "Fix perfomance degredation from BootSequence atom"
Added --set_system_boot_reason to expected filter

Test: boot_reason_test.sh
Bug: 74595769
Bug: 63736262
Bug: 72864061
Change-Id: Ie912d097fb6468dd826fd9422758fb9fb3e1ee29
2018-03-15 16:42:31 -07:00
Logan Chien
4892bbf8d2 Fix PDK build
This commit disables "bootstat" in PDK builds because "bootstat" depends
on "libstatslog" (from "frameworks/base") which is not included in PDK
builds as well.

Test: Build a target (described in http://b/72961456) with
`platform.zip` built from master FSK source tree.

Bug: 72961456
Change-Id: I06b1555694510e17ea82d5c6dfcdeaf99b905e4d
2018-02-14 03:03:28 +00:00
James Hawkins
79a4ee260b bootstat: Add more boot reasons.
Bug: none
Test: none
Change-Id: I483e4e2c6ea4219ac21b64a24bfac58c2919a40b
2018-01-26 14:31:04 -08:00
Treehugger Robot
b91021a2a8 Merge "bootstat: Add more boot reasons." 2018-01-09 00:03:22 +00:00
James Hawkins
b607daec94 bootstat: Add more boot reasons.
These appear to be winding down...

Bug: none
Test: none
Change-Id: I00bc84d8e50b1563f97f812f722aa6bdbd6265b2
2018-01-05 14:42:55 -08:00
Ivan Lozano
9ddf491492 Merge "Fix sanitizer errors in bootstat.cpp." 2017-12-21 06:42:55 +00:00
Ivan Lozano
44d3cacfce Fix sanitizer errors in bootstat.cpp.
Integer overflow sanitized builds are throwing an error on the while
loop decrement in the rfind function. This refactors the loop to prevent
decrementing the value on the final iteration.

Test: Compiled and device boots without runtime error.
Bug: 30969751
Change-Id: Ice4532cce933062b3c14adf2d9749cfdea4ad84c
Merged-In: Ice4532cce933062b3c14adf2d9749cfdea4ad84c
2017-12-20 14:59:19 -08:00
Elliott Hughes
579e682628 Add std::string StartsWith*/EndsWith* overloads.
We should have done this from the beginning. Thanks to Windows, we're not
going to be able to switch libbase over to std::string_view any time soon.

Bug: N/A
Test: ran tests
Change-Id: Iff2f56986e39de53f3ac484415378af17dacf26b
2017-12-20 09:42:22 -08:00
James Hawkins
e2c2724aee bootstat: Add tons of new boot reasons.
Bug: none
Test: none
Change-Id: I57e89a20f23d3b3cff2fa6667e25f1c37cb2870c
2017-12-18 13:40:27 -08:00
Elliott Hughes
3289b9c928 Merge "Add OWNERS." 2017-12-07 23:21:26 +00:00
Elliott Hughes
693d63f9cf Add OWNERS.
Bug: N/A
Test: N/A
Change-Id: Ie785058c0f5eb9b4086c98ccba6e63e3ed411b65
2017-12-07 13:30:03 -08:00
James Hawkins
00433a2305 bootstat: Add more boot reasons.
Bug: none
Test: none
Change-Id: I4cc6e2e39b6c5954387006d48f2119211fa37dd5
2017-12-04 14:20:21 -08:00
James Hawkins
f4444f058c bootstat: Add four more boot reasons.
Bug: none
Test: none
Change-Id: Ifd62e3a8bb6142e89fcde819eab1caba3d7933b7
2017-11-30 15:01:40 -08:00
James Hawkins
5f85f8354b bootstat: Three more boot reasons.
Bug: none
Test: none
Change-Id: I37b6ed6ce73f403aa5f9db2c0234e9dc4c0f7c01
2017-11-29 14:30:06 -08:00
James Hawkins
b623389421 Merge "bootstat: Add a few more bootreasons." 2017-11-29 19:29:54 +00:00
James Hawkins
74b1758cad bootstat: Add a few more bootreasons.
Bug: none
Test: none
Change-Id: I9b482e642a9eec26670118a265046194f3cc2a7a
2017-11-28 15:20:44 -08:00
Mark Salyzyn
547063eab6 bootstat: kernel_panic test w/o bootloader or pstore support
If the platform has no bootloader or pstore support, kernel_panic
test should fail if the results are not correct. Drop skipping of
failed test if pstore support is lacking.

If device demonstrably has pstore content support, the result must
be exacting kernel_panic,sysrq. Otherwise accept the less precise
result.

Test: On hikey960 (which currently lacks reliable pstore, or a
      compliant bootloader reporting bootreason), expect failure of:
      system/core/bootstat/boot_reason_test.sh kernel_panic
Bug: 63736262
Change-Id: I071a2a9c00dc522ec037c8a8997fea524d17e6e4
2017-11-20 12:18:09 -08:00
James Hawkins
8ac79bc820 bootstat: Add another boot reason.
Bug: None
Test: None
Change-Id: I865a12e9c8833c4a345284181aa1812be3eddadd
2017-10-31 10:07:34 -07:00
Mark Salyzyn
e7c95fb34f bootstat: test: add error propagation for fastboot format userdata
Test: system/core/bootstat/boot_reason_test.sh optional_factory_reset
Bug: 63736262
Change-Id: I8893df15ef802bb76c0795288df91d8463bfe3d1
2017-10-27 08:32:55 -07:00
Mark Salyzyn
293cb3b217 bootstat: handle a bad bit error rate issue with pstore
Create a private rfind that allows a fuzzy match based on a bit error
rate (BER) of 1 every 8 bits.  last kmsg is affected by pstore ramoops
backing that suffers from data corruption.  Add some additional
validation based on possible data corruption scenarios, as a noisy
match means higher chance of noisy data.

Noisy data notably can affect the battery level detection, but do not
typically result in false positives.  Battery level, or failure, is
the responsibility of the BatteryStats service, providing a positive
signal and strong device-independent algorithm.  The checking done in
bootstat is likely to be deprecated in favour of an API request to
BatteryStats once their algorithms deal with surprise outages due to
aging.

The kernel logging heuristic and BER fixup handily deals with a
prevalent issue where some bootloaders failure to properly notify us
of panics.  This is where the gains are noticed with this improvement.

Test: system/core/bootstat/boot_reason_test.sh
Bug: 63736262
Change-Id: I93b4210f12fb47c5c036f4d6eb4cafeee4896d35
2017-10-26 14:17:14 -07:00
Mark Salyzyn
747c0e6216 bootstat: better validation of battery level (shutdown,battery)
Replace simple strtoull with loop that ensures no leading zeros.
Restrict size of value buffer being checked as allocation was
going to end of retrieved buffer, which can cause unnecessary
memory pressure during boot.

Test: system/core/bootstat/boot_reason_test.sh
Bug: 63736262
Change-Id: Ifdc1d4fd3a73794c001577024ce7cbfde9c25028
2017-10-26 14:17:14 -07:00
Mark Salyzyn
dafced93a5 bootstat: Do not allow unknown boot reasons to land in first field.
If we sniff an unknown boot reason from last kmsg, make sure it
has a "reboot," prefix.

Test: system/core/bootstat/boot_reason_test.sh
Bug: 63736262
Change-Id: Ia1c401b8899d1f0c56bd4f5d8d2d19b7fc889a30
2017-10-26 14:14:40 -07:00
Mark Salyzyn
88d692c09e bootstat: move boot reason validation transformation policy into subroutine.
Allow for future policy adjustments.

SideEffects: None
Test: system/core/bootstat/boot_reason_test.sh
Bug: 63736262
Change-Id: I571fb7dafc6b80c75d2809a3da3f9b96784cef06
2017-10-26 14:11:04 -07:00
Mark Salyzyn
72a8ea3d3c bootstat: test: fix Its_Just_So_Hard_reboot
Add the test injection to known list, and deal with an error
propagation issue.

Test: system/core/bootstat/boot_reason_test.sh Its_Just_So_Hard_reboot
Bug: 63736262
Change-Id: I4799956978a8884c69c830fcedd7febd143651fd
2017-10-26 14:10:31 -07:00
Mark Salyzyn
c89f9da060 bootstat: add shutdown,thermal,battery
Test: system/core/bootreason/boot_reason_test.sh
Bug: 63736262
Change-Id: I30d6b5308fd8616eae31ad13e925d66c9fc07367
2017-10-25 15:45:49 -07:00