Commit graph

29343 commits

Author SHA1 Message Date
Mark Salyzyn
3614a0c5d4 logd: iterator corruption paranoia
Add checking for impossible(tm) scenarios within LogBuffer::flushTo:

1) When iterating through the log entries, check if the iterator
   returns two identical element references and break out of the loop.
2) Cap the maximum number of log entries we will skip while holding
   the iterator lock at 4194304, break out of the loop.

We print a message to the kernel logs if we hit these cases.

ToDo: Remove this paranoia at some future date.

Test: gTest liblog-unit-tests logcat-unit-tests and logd-unit-tests
Bug: 37378309
Change-Id: I789594649db14093238828b9f6d1daeca8b780c2
2017-04-17 15:10:26 -07:00
Mark Salyzyn
5836379b21 logd: regression in handling watermark boundary.
Deal with a regression introduced in commit
5a34d6ea43 (logd: drop mSequence from
LogBufferElement) where log_time was compared against nsec() time
miscalculating the watermark boundary.  When dealing with logcat
-t/-T, or any tail reading, add a margin to prune to back off by a
period of 3 seconds (pruneMargin).

Test: gTest liblog-unit-tests logcat-unit-tests and logd-unit-tests
Bug: 37378309
Change-Id: I72ea858e4e7b5fa91741ea84c40d2e7c3c4aa031
2017-04-17 15:10:26 -07:00
Tom Cherry
ffa38cfc1e Merge "ueventd: create classes for dev and sys permissions" 2017-04-17 21:34:12 +00:00
Wei Wang
66f44d823f Merge "init.rc: set initial cpuset to all cores" 2017-04-17 20:01:43 +00:00
Tom Cherry
cc054c9da8 ueventd: create classes for dev and sys permissions
Create classes for dev and sys permissions and store these permissions
in std::vector instead of the C list.

Test: boot bullhead
Test: init unit tests

Change-Id: I874039a3db29b4c70149506da8e407123ab7eca2
2017-04-17 19:33:04 +00:00
Chris Fries
b7826a74f2 Merge changes from topic '36810152'
* changes:
  fastboot: Support larger transfers during flash
  fastboot: Cap max size sent to libsparse
  fastboot: add AdbWinUsbApi as a required module
2017-04-17 19:31:38 +00:00
Chris Fries
0ea946c007 fastboot: Support larger transfers during flash
Adding methods to queue and download flashable images by fd instead of
by pointer, so that we can deal with sending large (up to 4GB) files
on windows and linux.  This gets past limitations on linux to read
more than 2GB from a file at a time, as well as memory limitations
on win32, in order to download up to 4GB in a single transfer.

Test: fastboot -w
Test: "flash-all" from nexus factory images site (incl. fastboot -w update)
Test: fastboot flash with large and small image, large and small max-download-size
Test: Sanity check flashing on win32, darwin, linux.
Test: Sanity check 3GB image download (with 3GB max-download-size)
      on win32, darwin, linux.

Bug: 36810152
Change-Id: I528d739d344eb080d59d721dadf3b3b34d4b375e
2017-04-17 09:22:49 -05:00
Treehugger Robot
75d195693b Merge "Add a README.md about our shell and utilities." 2017-04-15 15:33:41 +00:00
Bowgo Tsai
e8311491da Merge "Set libavb version into system property for Treble OTA" 2017-04-15 05:49:25 +00:00
Bowgo Tsai
1a898c25f9 Set libavb version into system property for Treble OTA
Set ro.boot.avb_version to "AVB_VERSION_MAJOR.AVB_VERSION_MINOR".
During Treble OTA match, the major version must be the same as that in
the avb metadata on disk, while the minor version can be equal or
greater to that in the avb metadata on disk.

See how avb versioning work on the following link:

  https://android-review.googlesource.com/#/c/342757/

Also renames AvbHashtreeDisabled() -> hashtree_disabled().

Bug: 35322304
Test: Early mount with AVB, checks [ro.boot.avb_version]: [1.0] exists.
Test: Not enable AVB, checks [ro.boot.avb_version] doesn't exists.

Change-Id: I5aaf476ca53c4fe817779518ba14b68ebcfdc6d6
2017-04-15 09:47:31 +08:00
Elliott Hughes
5a0818a633 Add a README.md about our shell and utilities.
This is definitely a FAQ, and it doesn't make sense for me to maintain
this on Google+ when I have the power to just check it in (and now have
a sensible place to do so).

Bug: N/A
Test: N/A
Change-Id: I649803ac846917a45f4b7b89cffe5b450179c479
2017-04-14 17:59:55 -07:00
Keun-young Park
82bd278dc4 Merge "check ext4 magic before running next steps" 2017-04-15 00:26:32 +00:00
Elliott Hughes
eea60804f6 Merge "Revert "Send property_service AVC messages to the kernel audit system"" 2017-04-15 00:13:47 +00:00
Wei Wang
2e83b86a8a init.rc: set initial cpuset to all cores
Starting zygote early requires cpuset to be initialized to all cores for
foreground cpuset. Change to expolit all cores by default at boot and
let device manufacturers override to proper values in device specific
init script.

Bug: 36576280
Test: marlin boot fast and checked cpuset during early boot
Change-Id: I2c1ce0630e58a7b04d1a453c6740d3f0bce9de9f
2017-04-14 15:03:29 -07:00
Tom Cherry
b87f1c1728 Merge changes I9d0482d1,Ib82833be
* changes:
  ueventd: convert platform_names from C list to std::vector
  ueventd: replace char* with std::string in struct uevent
2017-04-14 21:44:35 +00:00
Mark Salyzyn
030b4d1b08 Merge changes I596b8706,I262c0377,Iaf2bee97
* changes:
  liblog: allow event tags to include some punctuations
  liblog: logprint supports number of seconds time event field
  logcat: test: standardize rest() to let logs land when injecting
2017-04-14 21:33:23 +00:00
Treehugger Robot
42d524f73d Merge "logd: statistics truncate name" 2017-04-14 20:48:43 +00:00
Chris Fries
6a99971096 fastboot: Cap max size sent to libsparse
This is required for large (>INT_MAX) sparse limit reported by
the target.

Also, patch up return chains of "int" that need to deal with sizes
bigger than 2GB as well as return negative error codes.

Test: -S works with large max-download-size
Test: Flash 3GB system.img with max-download-size 2.5GB

Bug: 36810152
Change-Id: I562a50eabd706bd5b97c71a1aef07c1ffd1a2e5c
2017-04-14 15:44:20 -05:00
Chris Fries
4bf1be7ec8 fastboot: add AdbWinUsbApi as a required module
fastboot uses AdbWinUsbApi on Windows, let's keep it required.

Test: rm -rf out/host; mma fastboot and confirm AdbWinUsbApi is in out/
Bug: 36810152
Change-Id: Ica8b27cb1d0bca260f716dc61fdcea2ccc282623
2017-04-14 15:43:47 -05:00
Josh Gao
0301b3217b Merge changes from topics 'adb_thread', 'adb_loopback'
* changes:
  adb: statically link libbase into the tests.
  adb: kill adb_thread_{create, join, detach, exit}.
  adb: don't try to resolve 'localhost'
2017-04-14 20:04:04 +00:00
Mark Salyzyn
01bdb04be6 liblog: allow event tags to include some punctuations
event_log_tag parser complains about a period (.) in the name,
we would consider such an enhancement to the tag names possible.
I expect we would want to be able to support alphanumerics,
underscore (_), period (.), minus (-), at (@) and comma (,) for
starters as they are present in the other text log buffer tags.

We introduce a local endOfTag function that is used during parsing and
during android_lookupEventTagNum for submitting new tags.  This
function caused us to enforce const char more closely.  By filtering
in both places we resolve an issue that could have plagued us if
garbage requests were made.

Test: gTest liblog-unit-tests, logd-unit-tests & logcat-unit-tests
Bug: 31456426
Change-Id: I596b8706e843719ddac07ec40e1cd2875c214bed
2017-04-14 12:56:06 -07:00
Mark Salyzyn
5768d3d976 liblog: logprint supports number of seconds time event field
Add s to report time in seconds.  The time could be a period, duration
or monotonic, expanded to seconds, minutes, hours and days.  gTest has
to acquire a dynamic tag allocation as there are no users of this
feature yet.

Looking to the future, audio media logging has binary content similar
to the binary events structures Android logging uses and they have
a definition of a duration field in their internal binary logging, so
may be of use when we unify the logs.

Test: gTest logcat-unit-tests --gtest_filter=*.descriptive
Bug: 31456426
Change-Id: I262c03775983b3bc7b1b00227ce2bb2b0f357bec
2017-04-14 12:54:25 -07:00
Elliott Hughes
d8f9356bec Revert "Send property_service AVC messages to the kernel audit system"
This reverts commit 8adb4d9d12.

Change-Id: I0db29617f323ef31ca5a270d02b7668092e9c9e1
2017-04-14 12:53:19 -07:00
Mark Salyzyn
46186a7238 logcat: test: standardize rest() to let logs land when injecting
Add an internal rest() function to set the standard for how long we
should wait for a log message to land end to end into the logs.  Add
a retry on -EBUSY for all descriptive tests along with a rest() after
log injection so the tests can try to survive heavy Denial Of Service
(DOS) loads.  Some of the resting was done at a coarse 1 second, when
200ms will do just fine.

Fix a few pesky issues surrounding test reliability and correctness:
- stdint.h include missing.
- missing a logcat_executable, means logcatd.descriptive testing was
  testing logcat instead.
- count of interfering tests is 3 now: logcat, liblogcat and logcatd, if
  they all run in the same PID, we have to not fail.
- Added a missed opportunity to add uniqueness to logcat.descriptive sync
  test.

Test: gTest logcat-unit-tests
Bug: 31456426
Change-Id: Iaf2bee97878957f654613bb2e78e32379bb8d1b7
2017-04-14 12:50:59 -07:00
Tom Cherry
1ab8f55344 ueventd: convert platform_names from C list to std::vector
Also simplify this code a bit.

There's only one consumer that removes the /devices/platform prefix,
so have them handle it instead of storing two copies of the string.

Remove an unneeded search for '/' in get_character_device_symlinks()
as a / will always be the next character after a parent path, by
nature of FindPlatformDevice().

Test: boot bullhead
Test: init unit tests

Change-Id: I9d0482d137b1342ae7509ae993ff99198be814f0
2017-04-14 10:23:52 -07:00
Tom Cherry
e3e48214b7 ueventd: replace char* with std::string in struct uevent
Bug: 36250207

Test: Boot bullhead
Test: Boot sailfish, observe no boot time regression
Test: init unit tests

Change-Id: Ib82833bea56bdafbe1d7a045126aaa91a8725d98
2017-04-14 10:23:52 -07:00
Treehugger Robot
e771b69e72 Merge changes I6a29c678,I76212f65
* changes:
  reboot: fix owner and permissions of last_reboot_reason file
  init.rc: create /data/misc/reboot owned by system
2017-04-14 14:16:20 +00:00
Treehugger Robot
d7381375bb Merge changes from topic 'avb-early-mount'
* changes:
  fs_mgr: support AVB in fs_mgr_update_verity_state()
  init: support early_mount with vboot 2.0 (external/avb/libavb)
2017-04-14 06:20:42 +00:00
Bowgo Tsai
aaf70e77dc fs_mgr: support AVB in fs_mgr_update_verity_state()
fs_mgr_update_verity_state() is invoked by 'verity_update_state' in
init.rc. It will then set property "partition.system.verified" and
"partition.vendor.verified" to verify_mode. We should support this for
AVB as well.

Also change the order of static libs in init to fix the build error
after this change:
  system/extras/ext4_utils/ext4_crypt.cpp:69: error: undefined reference to 'property_get'

Bug: 35416769
Test: Mount /system and /vendor with vboot 2.0 (AVB), check the following properties exist.
      - [partition.system.verified]: [2]
      - [partition.vendor.verified]: [2]
Test: Mount /system and /vendor with vboot 1.0, check the following properties exist.
      - [partition.system.verified]: [0]
      - [partition.vendor.verified]: [0]

Change-Id: I4328d66a8cb93f26e7960e620a0b2292d5f15900
2017-04-14 12:10:49 +08:00
Bowgo Tsai
8bba52fc4b init: support early_mount with vboot 2.0 (external/avb/libavb)
libavb requires verifying AVB metadata on all verified partitions at
once. For example, /vbmeta, /boot, /system and /vendor. We need to
invoke device_init() for those partitions even if we only want to early
mount some of them, like /vendor and /system.

This CL gets all AVB partitions and the early mount partitions from
device tree through "firmware/android/vbmeta" and "firmware/fstab",
respectively. The following is an example to early mount /vendor
partition on bullhead:

firmware {
    android {
        compatible = "android,firmware";
        vbmeta {
            compatible = "android,vbmeta";
            parts = "boot,system,vendor";
            by_name_prefix="/dev/block/platform/soc.0/f9824900.sdhci/by-name"
        };
        fstab {
            compatible = "android,fstab";
            vendor {
                compatible = "android,vendor";
                dev = "/dev/block/platform/soc.0/f9824900.sdhci/by-name/vendor";
                type = "ext4";
                mnt_flags = "ro,barrier=1,inode_readahead_blks=8";
                fsmgr_flags = "wait,avb";
            };
        };
    };
};

Bug: 33254008
Test: early mount /vendor with vboot 2.0 (AVB) on bullhead
Test: early mount /system without dm-verity on bullhead
Test: early mount /vendor with vboot 1.0 on sailfish

Change-Id: I89a1f77c97124f309346b33d9e700544b92ecf05
2017-04-14 12:10:20 +08:00
Treehugger Robot
9e3756f532 Merge "liblog: android_log_event_list class permit -EBUSY retry" 2017-04-14 01:23:02 +00:00
Todd Poynor
fc827be3f9 reboot: fix owner and permissions of last_reboot_reason file
Default signature WriteStringToFile creates world-writeable files.
Set owner and group system and remove read/write for non-owner.

Bug: 37251463
Test: Manual: reboot, inspect
Change-Id: I6a29c678168dcae611b120dc52170f4eee7069a9
2017-04-13 18:03:59 -07:00
Todd Poynor
e092b72bf4 init.rc: create /data/misc/reboot owned by system
Directory will be read and modified by system uid.

Bug: 37251463
Test: Manual: reboot and inspect
Change-Id: I76212f65af991ff9ad0969b9c0b8460b80fb9cd2
2017-04-13 18:01:13 -07:00
Keun-young Park
6000a3f657 check ext4 magic before running next steps
- mount, e2fsck, tune2fs will all fail if magic number does not match.
- mismatch always happen for FDE and is wasting boot-up time to try
  all and fail always.
- skip mount steps if it has invalid magic number and do not record
  fs_stat either.
- For ext4 fs with corrupt superblock, e2fsck refuses to do anything if
  superblock magic is invalid. So simply running e2fsck does not help
  anyway.

bug: 36231950
Test: reboot ane check fs_mgr log from dmesg
Change-Id: I9ad9e0cd30fd074b3bbf8f450bd401b133d5771a
2017-04-13 17:11:11 -07:00
Treehugger Robot
d9b45c5811 Merge "fs_mgr: adds/changes some public APIs for early mount in init" 2017-04-14 00:10:02 +00:00
Mark Salyzyn
f31ae3d666 logd: statistics truncate name
Prefix long truncated names with an ellipse (...). Shift left as
much as possible when doing so, but keep spaces between command
name and other tabular fields.

Test: manual/visual
Bug: 37254265
Change-Id: I185b1e121ba911a9410a8b6624e013d5a531962b
2017-04-13 16:33:20 -07:00
Tom Cherry
659b78ed10 Merge changes Ie5ec609a,I5a2ac369,I690137b5
* changes:
  ueventd: Fix up string handling in handle_*_device_event()
  ueventd: convert mkdir_recursive() to std::string
  ueventd: move subsystem logic from code to ueventd.rc
2017-04-13 21:22:45 +00:00
Mark Salyzyn
7ecfd6ac10 liblog: android_log_event_list class permit -EBUSY retry
write() method was one-shot, a second shot on -EBUSY would always
return -EBUSY even if successful.  Reset internal error if -EBUSY
when retransmitting.  write() now reports a positive count for
transmission success.  Composition errors trump transmission errors.

Test: gTest logcat-unit-tests --gtest_filter=*.descriptive while
      under heavy DOS stress levels of logging.
Bug: 31456426
Change-Id: Ib1920c3f10cf1df8ad8eb6a884724794b577b29d
2017-04-13 13:16:59 -07:00
Treehugger Robot
fc500ddb52 Merge "fs_mgr_avb: refactors how vbmeta is loaded" 2017-04-13 19:58:37 +00:00
Treehugger Robot
b26f48f135 Merge "fs_mgr: adding fs_mgr_get_slot_suffix() public API" 2017-04-13 13:26:42 +00:00
Bowgo Tsai
80d1ad17ed fs_mgr: adds/changes some public APIs for early mount in init
Several changes in this CL:
  - Moves class FsManagerAvbHandle to public API
  - Adds a parameter 'wait_for_verity_dev' for FsManagerAvbHandle::SetUpAvb()
    to allow not to wait for verity device gets created
  - Adds FsManagerAvbHandle::AvbHashtreeDisabled() to query whether AVB is disabled
  - Adds fs_mgr_is_avb() to query whether a fstab_rec has MF_AVB flag

Bug: 33254008
Test: test AVB on bullhead
Change-Id: I89c43ca574ae632db8a700fc2590a1f80212c993
2017-04-13 18:29:10 +08:00
Bowgo Tsai
95c966a859 fs_mgr_avb: refactors how vbmeta is loaded
Adds two classes FsManagerAvbhandle and FsManagerAvbVerifier to replace the
following functions or struct:

    - fs_mgr_load_vbmeta_images() -> FsManagerAvbhandle::Open()
    - fs_mgr_unload_vbmeta_images() -> deleted
    - fs_mgr_setup_avb() -> FsManagerAvbhandle::SetUpAvb()

    - androidboot_vbmeta -> FsManagerAvbVerifier
    - load_vbmeta_prop() -> FsManagerAvbVerifier::Create()
    - verify_vbmeta_images() -> FsManagerAvbVerifier::VerifyVbmetaImages()

And only invokes FsManagerAvbhandle::Open() when there is a fstab entry having
'avb' flag (need HASHTREE descriptor). fs_mgr_is_avb_used() can be
removed as it only checks system property "ro.boot.vbmeta.hash_alg" to
decide whether vbmeta needs to be loaded, which might not be accurate.

For example, there are only HASH descriptors in the verified chain but
no HASHTREE descriptors. In this case, the fs_mgr doesn't have to do
anything because it only takes care of HASHTREE descriptors.

Also adds a new class FsManagerAvbOps to provide the C++ binding
FsManagerAvbOps::AvbSlotVerify() for libavb->avb_slot_verify().

Bug: 33254008
Test: test AVB on bullhead
Change-Id: I8fe15ba01c277152630a2a5c1c5c7f25fbf34030
2017-04-13 18:28:53 +08:00
Bowgo Tsai
87d0836cda fs_mgr: adding fs_mgr_get_slot_suffix() public API
The function returns "_a" or "_b" based on two possible values in
kernel cmdline:

  - androidboot.slot = a or b OR
  - androidboot.slot_suffix = _a or _b

Bug: 33254008
Bug: 36533366
Test: boot sailfish
Change-Id: Ia0a524e4145ebf61af5821f42ecad212c95ed748
2017-04-13 18:28:48 +08:00
Treehugger Robot
0de940c26d Merge "Force set timezone to UTC in logd." 2017-04-13 07:14:02 +00:00
Treehugger Robot
86c725a4c2 Merge "libutils: mark as vendor_available" 2017-04-13 01:40:56 +00:00
Josh Gao
47330e21db adb: statically link libbase into the tests.
Make it easier to run the Windows tests with wine.

Test: adb_test
Test: wine adb_test.exe
Change-Id: I243c3adf4f6519dd7a38666ab42b384858351e21
2017-04-12 17:38:09 -07:00
Dmitri Plotnikov
e0ab2557d4 Merge "Passing additional parameter to powerctl" 2017-04-13 00:30:24 +00:00
Vijay Venkatraman
83f50843cb Merge "Moved include/backtrace to libbacktrace/include" 2017-04-13 00:17:13 +00:00
Josh Gao
e1dacfc1b6 adb: kill adb_thread_{create, join, detach, exit}.
We have std::thread now, so we can delete this cruft.

Test: python test_device.py
Test: adb_test
Test: wine adb_test.exe
Test: /data/nativetest/adbd_test/adbd_test
Change-Id: Ie1c1792547b20dec45e2a62ce6515fcb981c3ef8
2017-04-12 17:12:32 -07:00
Tom Cherry
3fa467338f ueventd: Fix up string handling in handle_*_device_event()
Bug: 36250207

Test: Boot bullhead
Test: Boot sailfish, observe no boot time regression
Test: init unit tests

Change-Id: Ie5ec609a3f74bb03f5920734ada4d7de57508de4
2017-04-12 16:36:53 -07:00