Commit graph

44566 commits

Author SHA1 Message Date
Florian Mayer
30c898f1ca Merge "Fix copy / move behaviour of Maps object." 2019-03-06 16:31:10 +00:00
David Anderson
5a465412b1 Merge "Use distinct GateKeeper userids when running a GSI." 2019-03-06 02:40:03 +00:00
Treehugger Robot
2d85880b49 Merge "Clang-format before updating storage proxy" 2019-03-06 00:53:25 +00:00
David Anderson
925c78b7e8 Merge "Support FiemapWriters that extend across multiple files." 2019-03-05 23:06:19 +00:00
David Anderson
79a43cf135 Merge "Add more filesystem support to libfiemap_writer." 2019-03-05 18:14:53 +00:00
Suren Baghdasaryan
a953ae0546 Merge "Add min_score_adj into LmkKillOccurred event" 2019-03-05 18:06:46 +00:00
Mark Salyzyn
4c46285e2e Merge "fs_mgr: remount: add partition argument list" 2019-03-05 17:49:30 +00:00
Florian Mayer
3d67d347f5 Fix copy / move behaviour of Maps object.
Currently, moving or copying a Maps object leads to double free of MapInfo.

Even moving a Maps object  did not prevent this, as after a move
the object only has to be in an "unspecified but valid state", which can
be the original state for a vector of raw pointers (but not for a vector
of unique_ptrs).

Changing to unique_ptrs is the most failsafe way to make sure we never
accidentally destruct MapInfo.

Test: atest libuwindstack_test
      Failed LocalUnwinderTest#unwind_after_dlopen which also fails at master.

Change-Id: Id1c9739b334da5c1ba532fd55366e115940a66d3
2019-03-05 13:05:36 +00:00
Suren Baghdasaryan
ec5e4c6eb1 Add min_score_adj into LmkKillOccurred event
Log min_score_adj when lmkd kills a process to determine the oom_score
levels that lmkd considers during the kill.

Bug: 123024834
Change-Id: I986ae8f2808199b1654bc8d2a32dd88046c79aa3
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
2019-03-05 09:51:06 +00:00
Treehugger Robot
c676cca4f6 Merge changes from topic "apex_earlymount_base"
* changes:
  Activate system APEXes early
  mount /apex during first_stage init
2019-03-05 06:49:02 +00:00
Jiyong Park
dcbaf9f41b Activate system APEXes early
Summary: Boot sequence around apexd is changed to make it possible for
pre-apexd processes to use libraries from APEXes. They no longer need to
wait for the apexd to finish activating APEXes, which again can be
done only after /data/ is mounted. This improves overall boot
performance.

Detail: This change fixes the problem that processes that are started
before apexd (so called pre-apexd processes) can't access libraries
that are provided only by the APEXes but are not found in the system
partition (e.g. libdexfile_external.so, etc.). Main idea is to activate
system APEXes (/system/apex/*.apex) before /data is mounted and then
activate the updated APEXes (/data/apex/*.apex) after the /data mount.

Detailed boot sequence is as follows.

1) init prepares the bootstrap and default mount namespaces. A tmpfs is
mounted on /apex and the propagation type of the mountpoint is set to
private.

2) before any other process is started, apexd is started in bootstrap
mode. When executed in the mode, apexd only activates APEXes under
/system/apex. Note that APEXes activated in this phase are mounted in
the bootstrap mount namespace only.

3) other pre-apexd processes are started. They are in the bootstrap
mount namespace and thus are provided with the libraries from the system
APEXes.

4) /data is mounted. init switches into the default mount namespace and
starts apexd as a daemon as usual.

5) apexd scans both /data/apex and /system/apex, and activate latest
APEXes from the directories. Note that APEXes activated in this phase
are mounted in the default namespaces only and thus are not visible to
the pre-apexd processes.

Bug: 125549215
Test: m; device boots
Change-Id: I21c60d0ebe188fa4f24d6e6861f85ca204843069
2019-03-05 09:47:49 +09:00
Mark Salyzyn
d458066bc7 Merge "adb: use /system/bin/remount command" 2019-03-04 22:27:19 +00:00
Mark Salyzyn
f4e7422c65 fs_mgr: remount: add partition argument list
Can supply a specific partition to remount.  Partitions can be
specified by name or mount point.  Some extra work to differentiate
an unknown partition, invalid partition, or one that is covered by
overlayfs.

Test: adb-remount-test.sh
Bug: 122602260
Change-Id: Iab6f51c2b5ebe01f1cea3fb235445d5e2f495365
2019-03-04 11:13:05 -08:00
Mark Salyzyn
2f6c180749 adb: use /system/bin/remount command
Replace direct logic in support for 'adb remount' with an exec out to
/system/bin/remount to do the heavy lifting.  Remount success and
failure strings are reported by the adb remount service in response
to the various reported errors, freeing up the remount command itself
from the legacy of script expectations.

Test: adb-remount-test.sh
Bug: 122602260
Change-Id: I686fa465f463b881bbb38f709d780a95e463be80
2019-03-04 11:12:54 -08:00
David Anderson
f344d63222 Support FiemapWriters that extend across multiple files.
This introduces a new SplitFiemap class that will divide an allocation
request across multiple FiemapWriters. This is primarily useful on
filesystems that have onerous restrictions on maximum file sizes. Vfat,
for example, supports a maximum of 4GiB, which is too small to satisfy
larger userdata size requests.

Bug: 126230649
Test: fiemap_writer_test gtest
Change-Id: I3c95d341e4e94e0c44bbf0e8553c34ccfdcd155b
2019-03-04 10:08:55 -08:00
Treehugger Robot
6521bf3c98 Merge "ld.config: Permit legacy /system/vendor paths" 2019-03-04 12:58:11 +00:00
Jiyong Park
4ba548d845 mount /apex during first_stage init
/apex is not mounted via init.rc but directly by the first_stage init
before the mount namespaces are configured.

This allows us to change the propagation type for /apex mount point to
private to isolate APEX activatesions across post- and pre-apexd
processes.

Bug: 125549215
Test: m; device boots to the UI

Change-Id: I10e056cd30d64cb702b6c237acd8dab326162884
2019-03-04 16:22:41 +09:00
Felix
9e40e12cea ld.config: Permit legacy /system/vendor paths
Because /vendor is a symlink to /system/vendor on devices without a
dedicated /vendor partition, /system/vendor/lib/* needs to be added to
the permitted paths whereever /vendor/lib/* is permitted.

Reasoning:
Legacy devices are forced to use ld.config.legacy.txt, which is very
permissive. We can prolong support for them and enable them to use the
VNDK if we extend the search paths to include the resolved symlink dirs.

Change-Id: I6b3bb7b86ed82395345a16bdc857353b1b15c704
2019-03-03 14:31:37 +01:00
Petri Gynther
bc6372103a Merge "toolbox: make "r" available for /vendor" 2019-03-03 07:20:04 +00:00
David Anderson
4d4db8c09e Add more filesystem support to libfiemap_writer.
This adds support for vfat filesystems.

Bug: 126230649
Test: fiemap_writer_test gtest
Change-Id: I028cc7d95c313dd3ed826bc44cc3d0ffdcb58597
2019-03-02 00:31:15 -08:00
Treehugger Robot
924eec9518 Merge "libmeminfo: librank optimization using maps instead of vectors" 2019-03-02 03:58:28 +00:00
David Anderson
42294631a7 Merge "Fix double-accounting bug in liblp." 2019-03-02 03:16:10 +00:00
Petri Gynther
7c71e06f8e toolbox: make "r" available for /vendor
Make "r" package available for /vendor, so that we can easily add:

PRODUCT_PACKAGES_DEBUG += r.vendor

in device/google/<device>/device.mk

Change-Id: I46c23a85c95444083b8e4b68961c541d9af9b216
2019-03-01 17:51:56 -08:00
David Anderson
88e36c1514 Fix double-accounting bug in liblp.
When flashing in fastbootd, we create a new MetadataBuilder using the
given super_empty.img and attempt to import the existing partition
table. This will fail if there is some incompatibility in the partition
layout or partition quotas.

This import code was accidentally double-accounting partitions when
determining if they could fit within the group quota, preventing
"fastboot flashall" once partitions reached a certain size.

Bug: 126930319
Test: liblp_test gtest
Change-Id: I89a69cba110b62719197c9a4885cfc5bcf8f009f
2019-03-01 16:17:27 -08:00
Erick Reyes
5a744684c3 libmeminfo: librank optimization using maps instead of vectors
librank was calling std::vector::find_if in nested loops when iterating
through processes and maps.

Changed the implementation to use std::map::insert, this resulted in
a significant improvement to execution speed.

Test: time librank on crosshatch

Before:  1m22.57s real     1m15.87s user     0m05.79s system
After:   0m03.85s real     0m00.88s user     0m02.92s system

Bug: 124523194

Change-Id: I2b519d0bfd7f7929c6c4c6c2374794cb1a744585
Signed-off-by: Erick Reyes <erickreyes@google.com>
2019-03-01 16:17:20 -08:00
David Anderson
b1f97f9d1b Merge "Remove the block size alignment restriction." 2019-03-01 22:48:25 +00:00
Treehugger Robot
8dcd839158 Merge "Reduce log spam from libprocessgroup" 2019-03-01 22:11:15 +00:00
Suren Baghdasaryan
6ad4eb3519 Merge "libprocessgroup: convert Json::Value to const refs" 2019-03-01 18:02:22 +00:00
Suren Baghdasaryan
3af39f36cc Merge "Add vendor cgroups and task_profiles support" 2019-03-01 18:01:39 +00:00
David Anderson
47f7420366 Remove the block size alignment restriction.
Rather than require block-size alignment, instead bump the requested
file size to the necessary alignment. This ensures that the final block
is usable without placing onerous restrictions on the caller to figure
out the file system's block size.

This will require callers (namely, gsid) to track the actual desired
image size separately from the flie size.

This patch also updates tests to use the actual filesize of the
filesystem, rather than hardcoded 4096.

Bug: 126230649
Test: fiemap_writer_test gtest
Change-Id: I000cca274718c3ceac526d7c3392fe3a23bb42bc
2019-03-01 09:52:52 -08:00
Mark Salyzyn
8da7a1e6fd Merge "bootstat: add salyzyn as owner" 2019-03-01 17:48:56 +00:00
Treehugger Robot
c8964edb7e Merge "fs_mgr: fix fs_mgr avb_keys parsing problem" 2019-03-01 15:31:47 +00:00
Josh Gao
b530537d46 Merge "adbd: fix a case where we can fail to join a thread." 2019-03-01 10:16:46 +00:00
Josh Gao
7669ea5a44 Merge changes I4d351f56,Ib9e91157
* changes:
  adbd: tune USB read/write sizes and queue depths.
  adb: add benchmarks for sink/source.
2019-03-01 10:15:54 +00:00
Dima Zavin
83d9b680f5 Merge changes Ia3bba21c,I0ffc159b
* changes:
  fastboot: reset timeout if INFO is received during long commands
  fastboot: switch the timeout clock source to steady_clock
2019-03-01 10:12:39 +00:00
Suren Baghdasaryan
e681df44eb libprocessgroup: convert Json::Value to const refs
Convert Json::Value to const refs to prevent creation of a null member
if specified element does not exist. Fix variable naming in the parser
while we are at it.

Change-Id: I7dabfd1c73983b465984dd6c411c9fe48743a348
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
2019-03-01 00:29:49 +00:00
Suren Baghdasaryan
05da67c390 Add vendor cgroups and task_profiles support
Vendors should be able to specify additional cgroups and task profiles
without changing system files. Add support for /vendor/etc/cgroups.json
and /vendor/etc/task_profiles.json files which will augment cgroups and
task profiles specified in /etc/cgroups.json and /etc/task_profiles.json
system files.

Bug: 124960615
Change-Id: I548c2e866b65c19856a3617d75b75dcd06f7d5b7
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
2019-03-01 00:29:37 +00:00
Josh Gao
5841a96afc adbd: tune USB read/write sizes and queue depths.
Reduce the operation sizes to sizes that are small enough that the
kernel will run the OOM killer to try to make them succeed. Compensate
for the smaller operation sizes by increasing the queue depth.

Bug: http://b/126582877
Test: benchmark_device.py
Change-Id: I4d351f56c8f83fea882614238cc3fec8ba1e9bf9
2019-02-28 15:53:48 -08:00
Josh Gao
b416f2252e adb: add benchmarks for sink/source.
Test: ./benchmark_device.py
Change-Id: Ib9e911574be3afa94efcfd9bc5be0a5cf7b9e2c0
2019-02-28 15:53:48 -08:00
Dima Zavin
6d46a4975e fastboot: reset timeout if INFO is received during long commands
As part of a refactor, commit db511207ed
added a timeout for receving responses for commands. Unfortunately,
the timeout is optimistic as to how quickly target devices can complete
such operations. Flash and erase commands can be quite slow on devices
with traditional flash chips. The fastboot protocol is already adept
at handling such cases because it allows for sending INFO packets during
these periods. We may receive one or more INFO packets during tehse long
operations. Every time we receive an INFO packet, it proves the remote
end is alive, but busy, which should reset our timeout timer.

Change-Id: Ia3bba21c497b22639b626d89711ecd4eb02504ed
Signed-off-by: Dima Zavin <dmitriyz@waymo.com>
2019-02-28 14:55:49 -08:00
Dima Zavin
a5b85a431e fastboot: switch the timeout clock source to steady_clock
Previously, system_clock was used for remote command timeouts
which can get disturbed by NTP adjustments (jumps).

Change-Id: I0ffc159bf34d12e8d3713044524114d60a6a45ca
Signed-off-by: Dima Zavin <dmitriyz@waymo.com>
2019-02-28 14:55:48 -08:00
Mark Salyzyn
fe85df1c0d bootstat: add salyzyn as owner
Test: none
Change-Id: I1d8874dfb5cc9efaed5ad85702a054f1c2a7766f
2019-02-28 14:10:34 -08:00
Josh Gao
e778b3ad71 adbd: fix a case where we can fail to join a thread.
Bug: http://b/126703621
Change-Id: I3061d24bbdc154ebf1f9e3f5a903f01382fa518f
2019-02-28 13:53:24 -08:00
Treehugger Robot
949a561cfa Merge "libfs_avb: Pass const string by reference" 2019-02-28 18:53:27 +00:00
David Anderson
97400bd865 Use distinct GateKeeper userids when running a GSI.
GateKeeper uses userids to key authentication data. This poses a problem
for GSIs, since both the GSI and the host image will be storing separate
authentication data for the same userids.

To account for this, we add a large offset (1000000) to GSI userids
before handing them off to the GateKeeper HAL. Note that
SyntheticPasswordManager uses a similar trick to store Weaver data in
GateKeeper (when a Weaver HAL is not available).

Bug: 123716647
Test: PIN authentication works after booting into a GSI
Change-Id: I714368919f8e46ff1931f350342f09fa09225a35
2019-02-28 08:56:30 -08:00
Treehugger Robot
51eba0944a Merge "Unify some text format in task_profiles.json" 2019-02-28 15:15:20 +00:00
Greg Kaiser
4d6f4cc4fe libfs_avb: Pass const string by reference
Minor performance improvement for ValidatePublicKeyBlob().

Test: TreeHugger
Change-Id: I3498b529abedb8169d2488a2dc04cbd42cdb6f7d
2019-02-28 06:34:25 -08:00
Josh Gao
50cb647e7d Merge "adbd: limit the sizes of aio writes." 2019-02-28 07:25:21 +00:00
Bowgo Tsai
d214b40685 fs_mgr: fix fs_mgr avb_keys parsing problem
We should check if the fs_mgr option starts with "avb_keys" before
"avb". Otherwise, it will treat "avb_keys" as "avb" fs_mgr option.

Bug: 112103720
Test: atest fs_mgr_unit_test

Change-Id: I88446222fa88e8ecfcd6f96d30ad4336ebe146a8
2019-02-28 10:37:37 +08:00
Treehugger Robot
33f344cb36 Merge "libfs_avb: support key rotation for standalone partitions" 2019-02-28 01:13:20 +00:00