Commit graph

43496 commits

Author SHA1 Message Date
Sandeep Patil
c6497eb1dc libmeminfo: defer maps reading only when required for procmeminfo
This restores the original behavior. The main reason it should be this
way is to make the class generic for all things memory under
/proc/<pid>/. For example, with the current behavior, a program that
only needs to read /proc/<pid>/smaps_rollup will end up wasting time and
memory by parsing /proc/<pid>/maps when the object is being constructed.
Same goes for a program that only wants to reset the working set.

The 'ProcMemInfo' object still retains the property that it can only be
used once to read maps and the object must be destroyed + recreated to
update the stats.

Bug: 114325007
Bug: 111694435
Test: libmeminfo_test 1
Test:
 # adb push /google/data/ro/users/ss/sspatil/test-memutils.sh /data/local/tmp/
 # adb push procmem2 /data/local/tmp && adb push procrank2 /data/local/tmp
 # adb root && adb shell
 $ cd /data/local/tmp/
 $ chmod +x test-memutils.sh
 $ ./test-memutils.sh 2>&1 | tee test.log

Change-Id: I856d3b78a0088cff02cbd010b29ffbe0e35f5ee2
Signed-off-by: Sandeep Patil <sspatil@google.com>
2018-12-05 09:31:10 -08:00
Sandeep Patil
f129199350 libmeminfo: Add support to reset workingset without procmeminfo objects
Bug: 114325007
Bug: 111694435
Test: libmeminfo_test 1

Change-Id: I816809cda13983e47294a956347a27085397cbcf
Signed-off-by: Sandeep Patil <sspatil@google.com>
2018-12-05 09:13:20 -08:00
Sandeep Patil
549feab631 libmeminfo: Add support to fiter accounting based on page flags
Bug: 114325007
Bug: 111694435
Test: libmeminfo_test 1 --gtest_filter=ValidateProcMemInfoFlags.*

Change-Id: Ifa7006de78dd909b5b4db282a8c9931ebf97a68a
Signed-off-by: Sandeep Patil <sspatil@google.com>
2018-12-04 15:59:42 -08:00
Sandeep Patil
70fa72dd01 libmeminfo: Add support to read zram memory consumption
Bug: 114325007
Bug: 111694435
Test: libmeminfo_test 1 --gtest_filter=SysMemInfoParse.TestZramTotal
Benchmark: libmeminfo_benchmark --benchmark_filter=BM_.*ZramTotal
Benchmark Result on Blueline:
-----------------------------------------------------------
Benchmark                    Time           CPU Iterations
-----------------------------------------------------------
BM_OldReadZramTotal       3857 ns       3839 ns     134096
BM_NewReadZramTotal       4461 ns       4440 ns     157341

Change-Id: I5220fa17b101981ef859179960fe78fe68e84852
Signed-off-by: Sandeep Patil <sspatil@google.com>
2018-12-04 15:59:39 -08:00
Sandeep Patil
2259fdf7df libmeminfo: Add support for counting swap pages
Adds procmeminfo API to get the vector of swap_offsets
if SWAP is enabled on the device.

Bug: 114325007
Bug: 111694435
Test: libmeminfo_test 1
Change-Id: If0b52d042749a5bcb2c87aa2cb1595190d4769b1
Signed-off-by: Sandeep Patil <sspatil@google.com>
2018-12-04 14:02:28 -08:00
David Anderson
5cd7969177 Merge "liblp: Allocate "b" extents in the second half of super." 2018-12-04 21:53:48 +00:00
Treehugger Robot
46d19d34d1 Merge "adb: fix authentication when reconnecting." 2018-12-04 20:41:14 +00:00
Tom Cherry
bb72b808f4 Merge "Convert fs_mgr_swapon_all() to use the new Fstab struct" 2018-12-04 18:41:56 +00:00
Treehugger Robot
a30cff7f89 Merge "fs_mgr: Change scratch margin to 3 X logical_block_size." 2018-12-04 16:30:25 +00:00
Neil Fuller
87aa446d48 Merge "Add ANDROID_RUNTIME_ROOT environment variable" 2018-12-04 12:39:16 +00:00
Treehugger Robot
239245fe79 Merge "Enable the APEX support on the device side." 2018-12-04 12:29:06 +00:00
Josh Gao
4414e4c728 adb: fix authentication when reconnecting.
If we have multiple keys available for authentication (ADB_VENDOR_KEYS
+ the one in ~/.android), we will still have keys in our list of
avilable keys after we've successfully connected. A subsequent
reconnection will start authorizing using the list of keys after the
key that actually worked, resulting in that session being unauthorized
until another reconnection happens. Clear the key list before
reconnecting to fix this. (We could do this after successfully
connecting, but we need to do this before reconnecting anyway, because
our connection could have died during authorization.)

Bug: http://b/117267347
Test: `adb connect foo; adb -s foo reconnect device` with ADB_VENDOR_KEYS
Change-Id: Ieb7dcc28e333c89ae0d75f97e89bcd1b571cb299
2018-12-04 01:08:55 -08:00
Treehugger Robot
a7f54b1592 Merge "C++17 is the default now." 2018-12-04 07:53:24 +00:00
Dan Willemsen
f56dad3fff Merge "Prevent dependency cycle due to system_shared_libs expansion" 2018-12-04 06:19:09 +00:00
David Anderson
8acb5dc1c5 liblp: Allocate "b" extents in the second half of super.
When allocating "b" partitions on a non-retrofit A/B device, prioritize
regions occuring in the second half of the super partition. To make this
effective, the region covering the midpoint sector is split into two
additional regions.

This will allow OTAs to avoid unecessary fragmentation, since each
slot's partitions will be grouped together, leaving a large chunk of
contiguous space available when the OTA deletes the target slot. Since
updates are not allowed to consume more than half of the super
partition, this should guarantee one extent per partition. Note that, if
this restriction is not in place (for example, a developer flashes a
massive "system_b"), then an additional extent will be allocated due to
the region that was split.

Bug: 120433288
Test: liblp_test gtest
Change-Id: I1797e59e14c8b0d4d0e6855a1d984e8159b21df2
2018-12-03 16:58:34 -08:00
Tom Cherry
0c7fb42949 Merge "fs_mgr: Create a C++ Fstab struct" 2018-12-04 00:10:33 +00:00
Jaewoong Jung
7174ec8fff Merge "Revert "Revert "Revert "adb: append the libwinpthread COPYING to adb's NOTICE."""" 2018-12-03 23:57:42 +00:00
Dan Willemsen
d3825898da Prevent dependency cycle due to system_shared_libs expansion
It was discovered that we were building some objects inconsistently due
to an optimization in cc_library to only build objects once and use them
for both the static and shared libraries. But static libraries didn't
get system_shared_libs set automatically, and we didn't notice that we
would have built the objects differently.

So static libraries now get the default system_shared_libs, we allow
adjusting that for static vs shared in a cc_library, and we disable the
optimization if the linked libraries are configured differently between
static and shared in a single cc_library.

This triggers dependency cycles for static libraries that libc/libdl
use, so fix those cycles here.

Test: treehugger
Change-Id: I6ec2ea45292b602d5c506560de8b8d6ae725be6f
2018-12-03 15:50:00 -08:00
Josh Gao
946f7d44c2 Merge "base: add more error-checking overloads for unique_fd." 2018-12-03 22:41:22 +00:00
Mark Salyzyn
756eebe819 fs_mgr: Change scratch margin to 3 X logical_block_size.
Found a device with a 3/4MB erase block, give a wider margin so that
flashing a single partition is unlikely to be blocked by scratch.

Add documentation to tell developers to flashall to clear scratch.

Test: adb-remount-test.sh
Bug: 109821005
Change-Id: Ic219283f4c42e457b98991bcd1752253e182eff3
2018-12-03 12:15:25 -08:00
Treehugger Robot
307337fb40 Merge changes I6a4c3d9b,Ia5cdb797,I946eb605,Ieff3f6ea
* changes:
  init: overlay: allow fs_mgr_overlayfs_required_devices to report partitions
  fastboot: call fs_mgr_overlayfs_teardown() in UpdateSuper
  fs_mgr: overlay: disable overrides in recovery or fastbootd.
  fs_mgr: overlay: umount scratch more aggressively
2018-12-03 20:14:02 +00:00
Treehugger Robot
5d595da516 Merge "fs_mgr fix flags/fs_mgr_flags mismatch" 2018-12-03 19:07:45 +00:00
Daniel Mentz
9cb7704ab4 Merge "Increase netlink uevent rcvbuf size to 16M" 2018-12-03 18:39:42 +00:00
Daniel Mentz
41aa2c3703 Increase netlink uevent rcvbuf size to 16M
Increase size of the NETLINK_KOBJECT_UEVENT socket receive buffer to
16M.  Also, use SO_RCVBUFFORCE to override any limits set by
/proc/sys/net/core/rmem_max.

We had a couple of instances, where we lost critical uevent messages due
to receive buffer overflows.

Bug: 119933843
Change-Id: I6aab183aa0194e173f9175b47c6beb0835cf6675
2018-12-03 18:39:24 +00:00
Tom Cherry
3055457a96 Convert fs_mgr_swapon_all() to use the new Fstab struct
Bug: 62292478
Test: build
Change-Id: Ifbde514bf73d3ce2f321326291daa025b6afac46
2018-12-03 09:41:01 -08:00
Tom Cherry
d0be7a5277 fs_mgr: Create a C++ Fstab struct
Create a C++ Fstab struct with the intention to eventually deprecate
the legacy C 'fstab' struct.  This also contains functions for
converting between the two fstab variants to ease in the transition.

Bug: 62292478
Test: boot

Change-Id: I6fb51c6a99e65192708792719df01960cf6b432a
2018-12-03 09:40:55 -08:00
Elliott Hughes
3c59cb8420 C++17 is the default now.
Test: builds
Change-Id: I6814455e9ad2cdcf99ad66de38a3ad0bfb440d80
2018-12-03 09:02:18 -08:00
Mark Salyzyn
acf56c297f init: overlay: allow fs_mgr_overlayfs_required_devices to report partitions
init add support for picking up partitions if specified by
fs_mgr_overlayfs_required_devices() as /dev/block/by-name/<partition>

Test: adb-remount-test.sh
Bug: 119885423
Change-Id: I6a4c3d9b5c2b406178f0acf0a86c52ab17209537
2018-12-03 07:33:38 -08:00
Mark Salyzyn
307a41fe9f fastboot: call fs_mgr_overlayfs_teardown() in UpdateSuper
When the update-super command is issued, we want overlayfs overrides
to disappear without a doubt, which includes non-A/B utilizing
/cache/overlay/ tree.  Call fs_mgr_overlayfs_teardown() on successful
return.

Test: adb-remount-test.sh
Bug: 120034852
Change-Id: Ia5cdb797f7e8350b5591a51fc8ae5f323901aee4
2018-12-03 07:33:38 -08:00
Mark Salyzyn
9040aa5665 fs_mgr: overlay: disable overrides in recovery or fastbootd.
Disable overrides if we are booting up in recovery or fastbootd.
Check for existence of /system/bin/recovery.

Test: adb-remount-test.sh
Bug: 120034852
Change-Id: I946eb605300226d31356ecac209d6367f4e13526
2018-12-03 07:33:38 -08:00
Mark Salyzyn
c0966c9a1b fs_mgr: overlay: umount scratch more aggressively
For specific teardown operations, if we mount scratch with the
intention of it being temporary to strike out content for a specific
partition, umount it.  Otherwise it could interfer with fastboot.

Test: adb-remount-test.sh
Bug: 120034852
Change-Id: Ieff3f6ea0bcda5eb540e9acf6cab8a56d1453b3e
2018-12-03 07:33:38 -08:00
Josh Gao
919b70cde1 Enable the APEX support on the device side.
Test: on device
Merged-In: Ic6e237556fa059e02cf9c55d1c034947b7dfb405
Change-Id: Ic6e237556fa059e02cf9c55d1c034947b7dfb405
2018-12-03 22:29:52 +09:00
Treehugger Robot
67fe6f09d5 Merge "check_fs: skip check_fs if there is no matched EXT4|F2FS_SUPER_MAGIC" 2018-12-03 06:28:34 +00:00
Tom Cherry
7a66be1a54 fs_mgr fix flags/fs_mgr_flags mismatch
Mount flags prefixed with MS_ are intended for the 'flags' entry of
fstab_rec, not fs_mgr_flags.

Coincidentally, this probably has always worked since this maps to the
'wait' flag, which is almost always set for system in the first place.

Test: build
Change-Id: Id4862bc087aa98e1f0e17145dc3ed74b5d867596
2018-11-30 16:17:17 -08:00
Tom Cherry
7f5a2d3c8f Merge "fs_mgr: fix typo from off64_t change." 2018-11-30 23:33:50 +00:00
Jaegeuk Kim
5d14b982fa check_fs: skip check_fs if there is no matched EXT4|F2FS_SUPER_MAGIC
We don't need to call fsck, if magic is not matched at all.

Bug: 111895882
Change-Id: Ie728b02ff2fc7d4a20fc011f66643583ea708f9d
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2018-11-30 21:29:15 +00:00
Mark Salyzyn
4115c2caf6 Merge "fs_mgr: overlay: test assumption wrong if verity is enabled" 2018-11-30 20:36:27 +00:00
Tom Cherry
2c25796aa0 fs_mgr: fix typo from off64_t change.
I accidentally changed some _mode enums from unsigned int to off64_t
along with the rest changes.  They probably shouldn't have been
unsigned anyway though, so let's make them plain 'int' while we fix
the original typo.

Test: boot
Change-Id: I0c6c51a1a8170056b99087d249539eb79026cffe
2018-11-30 11:29:46 -08:00
Tom Cherry
9d24734b5c Merge "fs_mgr: fixup 'size' attributes of fstab." 2018-11-30 16:33:54 +00:00
Mark Salyzyn
03d6f9a176 fs_mgr: overlay: test assumption wrong if verity is enabled
Test script assumes overlayfs takeup after disable-verity, although
that can happen if verity is already disabled, it is not the case
for devices that just had it disabled.  Move takeup and first stage
init tests downwards to where the condition is guaranteed.

Differentiate the test for devices that support or have verity, and
those that do not.

Fortify test to also use /data mount to check init first stage limit.

Test: adb-remount-test.sh
Bug: 109821005
Change-Id: I5241d6c97bf374614e50aa0aa18a1b9d118be093
2018-11-30 07:49:32 -08:00
Idries Hamadi
fafd21159c Merge "Fix fastdeploy issues when testing Boat Attack Unity3D demo" 2018-11-30 08:43:05 +00:00
Treehugger Robot
e43c146eb4 Merge "fs_mgr: move AVB related source into libfs_avb" 2018-11-30 06:59:38 +00:00
Treehugger Robot
79bd67bb2f Merge "Support product partition for non-treblized devices" 2018-11-30 02:12:15 +00:00
Treehugger Robot
9b52891918 Merge "Validate allocations against mappings" 2018-11-30 01:33:27 +00:00
Treehugger Robot
7472cf6c53 Merge "fs_mgr: add README.overlayfs.md" 2018-11-29 23:39:38 +00:00
Tom Cherry
ac3c7740ad Merge "init: always create a symlink from /init to /system/bin/init" 2018-11-29 23:22:58 +00:00
Jaewoong Jung
0e55778f4e Revert "Revert "Revert "adb: append the libwinpthread COPYING to adb's NOTICE."""
This reverts commit 1272e7a592.

Reason for revert: All notice fix changes have been merged and running peacefully for a sustained period. Now this can be reverted for good.

Change-Id: I74e4e291a504fcb9a14ae50e916e494c7359db34
2018-11-29 23:14:02 +00:00
Tom Cherry
fafbb648e0 fs_mgr: fixup 'size' attributes of fstab.
'off64_t' is the correct way to specify a size of a file system, so we
update the various fs_mgr fstab attributes accordingly.

Test: boot
Change-Id: I07ebe687b7c215a8d07ce49d6d32e81b798d1cd3
2018-11-29 13:32:35 -08:00
Mark Salyzyn
a9a3b73163 fs_mgr: add README.overlayfs.md
Add documentation regarding overlayfs logistics.

Test: none
Bug: 109821005
Change-Id: I8092fdd548a28f8c860b5dd6175b657728a48801
2018-11-29 12:22:53 -08:00
David Anderson
d57025dc46 Merge "fastboot: Delete logical "other" partitions on retrofit devices." 2018-11-29 19:52:12 +00:00