Commit graph

51824 commits

Author SHA1 Message Date
Yurii Zubrytskyi
6eca0e6999 [adb] Speed up the streaming install
1. Use bigger buffer for transfers - 64kb is the default size
   for push, so let it be the same in streaming

2. Use abb when it's available for lower overhead

3. Add a posix_fadvise() on the source APK

4. Increase buffer sizes for the socketpair that's transferring
the data from adbd.

Overall this saves about 25% time for streaming installations
and makes it faster than the legacy push (at last!)

Test: manual
Change-Id: Ieb84284da2058944815e062ef6e4389b842565fa
2019-08-08 12:55:51 -07:00
Tom Cherry
4389117af7 Merge "liblog: Remove stdbool.h from .cpp files"
am: ca94d79753

Change-Id: I0f78ec9e4f0d11afe7785a94d59a9babae23c221
2019-08-08 12:54:56 -07:00
Elliott Hughes
498163c28a Merge "Remove more dead code."
am: 0c2dab9adf

Change-Id: I52dbcba34d893573b3452c7b084af8840e5c1ec1
2019-08-08 12:40:55 -07:00
Tom Cherry
ca94d79753 Merge "liblog: Remove stdbool.h from .cpp files" 2019-08-08 19:34:41 +00:00
Elliott Hughes
0c2dab9adf Merge "Remove more dead code." 2019-08-08 19:10:40 +00:00
Mark Salyzyn
31c14e15b9 fs_mgr: overlayfs: dig harder for /dev/root equivalent
Check for A/B /dev/block/by-name/system<slot> and also non-A/B
/dev/block/by-name/system to discover what /dev/root can be when
evaluating candidates for using overlayfs.

This is to handle a misconfigured (or legacy) system-as-root device.
It is recommended that the default fstab specifically mentions the
root mount's device node to prevent going down this path.

Test: adb-remount-test.sh
Bug: 138407617
Change-Id: I3853d203b9376d0f848cb490150ff00cc4ed3d5f
2019-08-08 11:50:36 -07:00
Elliott Hughes
addd852165 logcat: switch to <regex>.
This was the only user of the google3 additions to external/pcre.

Test: adb logcat --grep camera
Change-Id: Iaa9dae38d30f60470f6a65b0a0fdd93e01abfc99
2019-08-08 08:53:59 -07:00
vichang
d9153800b4 Merge "Move ICU .dat"
am: ef0bc63043

Change-Id: I263e6dd27acf9c322756bb0111873f7efe156d5c
2019-08-08 04:24:56 -07:00
vichang
ef0bc63043 Merge "Move ICU .dat" 2019-08-08 11:06:53 +00:00
David Anderson
5312c466b9 libsnapshot: Implement merge flow.
This implements InitiateMerge() and WaitForMerge(). InitiateMerge() is
meant to be called after an update has been marked successful.
WaitForMerge() is designed to be called either: immediately after
InitiateMerge, or during each subsequent boot where merging has not
completed.

InitiateMerge converts each snapshot device to a snapshot-merge device.

WaitForMerge polls each snapshot-merge device until no device reports a
"merging" state. One of the following states can result from this:
 - MergeFailed. This will happen if any device failed to merge, or we
   were unable to poll, or any other system-level failure occurred.
 - MergeNeedsReboot. This will happen if a snapshot-merge device has
   completed merging, but we were unable to clean it up due to something
   holding a resource open.
 - MergeCompleted. This indicates that all snapshots completed merging
   and were cleaned up.

If WaitForMerge() returns MergeCompleted, then all snapshots have been
removed and a new update can begin. GetUpdateState() will return None.

MergeFailed and MergeNeedsReboot, on the other hand, are "sticky". They
indicate a merge is still pending. When called again, WaitForMerge()
will poll again to attempt to make more progress in the merge. For
NeedsReboot, a single reboot will ensure all resources are released and
the next WaitForMerge() will successfully finish cleanup. In the failure
case, it is unlikely the next WaitForMerge will succeed, but we always
retry anyway (there is no harm in doing so, and if we get lucky, the
device can take more OTAs).

Bug: 136678799
Test: libsnapshot_test gtests
Change-Id: I5e93fcbffee1973da5ff76363df12d6317a7a7c7
2019-08-07 18:42:23 -07:00
David Anderson
986095036c libsnapshot: Unmap COW devices when deleting snapshots.
When we stopped unmapping snapshots in DeleteSnapshot, this also stopped
unmapping the COW device. Of course, the caller has no way to unmap this
since it's an implementation detail. Therefore unmap it in
DeleteSnapshot.

This scenario can arise after rewriting or removing a snapshot-merge
device.

Bug: 136678799
Test: manual test
Change-Id: I9d4c8bcfbb95b4454edd1cf0853d51b441e895fa
2019-08-07 18:42:22 -07:00
David Anderson
d986fefa3b libsnapshot: Eliminate per-snapshot flocks.
Per-snapshot locks don't solve any problems and add a great deal of
complexity. Instead, refactor the Read/WriteSnapshotStatus methods so
the caller just needs the snapshot name, and is not responsible for
opening a file.

As part of this change, callers of WriteSnapshotStatus must always take
an exclusive flock on the update state file. This is enforced by adding
a helper method to LockedFile to check the lock mode.

Bug: 136678799
Test: libsnapshot_test gtest
Change-Id: Icd580aaec7dfc916b3eed174d86b26688cd2291b
2019-08-07 18:42:22 -07:00
David Anderson
3cb682e369 libsnapshot: Improve first test-run and test cleanup.
This CL fixes a bug where libsnapshot_test failed on the first run. It
also fixes bugs where it could not run if it died in the middle of a
test.

Previously, libsnapshot_test relied on CancelUpdate() to perform
cleanup, which cannot run in certain states. Instead, manually delete
dm devices and COW image files, and forcefully erase any lingering data.

Bug: 136678799
Test: libsnapshot_test gtest
Change-Id: I7b2399a403b387eb47184626e71dcf8674f6ab89
2019-08-07 18:42:21 -07:00
Josh Gao
5ae4a90400 adb: windows: fix writev on sockets.
Bug: http://b/139078301
Test: treehugger
Change-Id: I000c12ef9dedad530a166175cf089df45378f759
2019-08-07 18:26:47 -07:00
Yifan Hong
629fc64dec Merge "liblp: refactor MetadataBuilder::NewForUpdate"
am: d97c946d37

Change-Id: I6f61a3dcec7b59c8e09fc0e45111b95469bd66d8
2019-08-07 17:10:38 -07:00
Treehugger Robot
d97c946d37 Merge "liblp: refactor MetadataBuilder::NewForUpdate" 2019-08-07 23:54:27 +00:00
Elliott Hughes
1547e4a316 Remove more dead code.
Test: treehugger
Change-Id: I2b7f606241cde2c2743d8021fad00d30f7e0b0de
2019-08-07 14:52:16 -07:00
Tom Cherry
d00b3affda liblog: Remove stdbool.h from .cpp files
We're C++ now, so we don't need stdbool.h

Test: build
Change-Id: I2c65f0d3edddaf77256bd0bc3d60378bcfaef180
2019-08-07 14:30:41 -07:00
David Anderson
b8089c639c Merge "libsnapshot: Don't force-unmap in DeleteSnapshot()."
am: 36acd1d246

Change-Id: I256f80b5bdbb1330438a16f2e28f5551de02a7bd
2019-08-07 13:42:06 -07:00
David Anderson
36acd1d246 Merge "libsnapshot: Don't force-unmap in DeleteSnapshot()." 2019-08-07 20:24:46 +00:00
Yifan Hong
8644f52338 Merge changes from topic "liblp_virtualab_prepare"
am: 68aae7d748

Change-Id: I0af79e752e346460e8af8fd20f5a65b34750f6b9
2019-08-07 13:17:40 -07:00
Yifan Hong
68aae7d748 Merge changes from topic "liblp_virtualab_prepare"
* changes:
  liblp: Expose kDefaultGroup.
  liblp: Add PropertyFetcher.
2019-08-07 19:45:44 +00:00
Yifan Hong
6079cd014a liblp: refactor MetadataBuilder::NewForUpdate
Move the logic for retrofit DAP to its new function. The new flow
in NewForUpdate is:
    metadata = ReadMetadata();
    if (retrofit dap) UpdateMetadataForOtherSuper(metadata)
    return metadata;

Test: liblp_test_static
Change-Id: I6890fff3a7c44ebe2004de96b2ccbe1e8ce37546
2019-08-07 12:44:19 -07:00
Vic Yang
c1ed1ebac7 Merge "fs_mgr: Make kDefaultAndroidDtDir constexpr"
am: ef06b4fc7a

Change-Id: I553ca2e8a81b6b48a156c7a3a4843925b6804ac1
2019-08-07 10:47:39 -07:00
Steve Muckle
c4e5646b35 Merge changes from topic "toolbox-modprobe"
am: b02f9b549c

Change-Id: I2da6f431a80fe82070fdd2e1c87c027c81d9bf53
2019-08-07 10:44:50 -07:00
Steve Muckle
e6bf24912a libmodprobe: add verbose mode
am: ded44c06be

Change-Id: I4696ec3115a46806d5a829142902b94b413e7223
2019-08-07 10:44:32 -07:00
Steve Muckle
46b9a98ea6 libmodprobe: add GetAllDependencies
am: 781aa78ee2

Change-Id: I9e7defc1e21efb1c3561079a24ec5852e5a60c03
2019-08-07 10:44:10 -07:00
Steve Muckle
4a644fa521 libmodprobe: add support to list modules
am: 012cfa19b0

Change-Id: I82ec8054ec1709ff18bd8d43851554a7e1625f40
2019-08-07 10:43:36 -07:00
Steve Muckle
5b3690cb73 libmodprobe: add support for a blacklist
am: e31f840a0a

Change-Id: I04851fa1415b0c60af31f9fe053328ec8a9baaf9
2019-08-07 10:43:02 -07:00
Steve Muckle
0459294c3f libmodprobe: support parameters in LoadWithAliases
am: 13700a69d3

Change-Id: I7de4448ea0120be156347131359f5db903ff750f
2019-08-07 10:42:43 -07:00
Steve Muckle
90947b9c42 libmodprobe: add support to remove modules
am: bb58b01574

Change-Id: Ib09a53985855f840a6d23a3c6c555e84c8f761cc
2019-08-07 10:42:10 -07:00
Steve Muckle
4395f58bd5 libmodprobe: make name canonical in LoadWithAliases
am: 73b2928b94

Change-Id: I849ca922250842ed47854c6d2e7dead57ae9fc9b
2019-08-07 10:41:36 -07:00
Steve Muckle
92a432a1e2 libmodprobe: make available in vendor
am: 4104b8803a

Change-Id: I30d6abcd0bcc53101029de024773d8120e41cc24
2019-08-07 10:40:51 -07:00
Treehugger Robot
ef06b4fc7a Merge "fs_mgr: Make kDefaultAndroidDtDir constexpr" 2019-08-07 17:22:37 +00:00
Treehugger Robot
b02f9b549c Merge changes from topic "toolbox-modprobe"
* changes:
  toolbox: add modprobe
  libmodprobe: add verbose mode
  libmodprobe: add GetAllDependencies
  libmodprobe: add support to list modules
  libmodprobe: add support for a blacklist
  libmodprobe: support parameters in LoadWithAliases
  libmodprobe: add support to remove modules
  libmodprobe: make name canonical in LoadWithAliases
  libmodprobe: make available in vendor
2019-08-07 17:09:09 +00:00
Yifan Hong
5b4b38cee0 liblp: Expose kDefaultGroup.
kDefaultGroup is the 'default' group that is never deleted
and has no size constraints. During a virtual a/b update,
the following series of action happens:
- move partitions out of groups that will be shrunk / deleted
  (into default)
- shrink / delete these groups
- grow / add other groups
- move those partitions to their destination group.

Otherwise, the metadata is inconsistent.

Bug: 138816109
Test: builds

Change-Id: I57cf4b44dda948377c6b3153756f469caa4652bc
2019-08-06 18:46:47 -07:00
David Anderson
aefb3b17fb libsnapshot: Don't force-unmap in DeleteSnapshot().
It is hard to re-use this function in the merge code when it forcefully
unmaps the snapshot, because the snapshot may have been rewritten to be
a dm-linear device. Instead, leave the decision up to the caller.

Bug: 136678799
Test: libsnapshot_test gtest
Change-Id: I03c027c0781696885a5a5654d3049287cc16ecd0
2019-08-06 18:16:04 -07:00
Yifan Hong
04d91871df liblp: Add PropertyFetcher.
Use dependency injection so that GetProperty / GetBoolProperty
can be mocked in tests.

Test: run liblp_test_static
Change-Id: I8efa85fbbd7aebce2541f748f840e512f3729c30
2019-08-06 17:39:37 -07:00
Josh Gao
3346339fcb adb: fix transport acquisition for forward, reverse.
Previously, we were calling acquire_one_transport with all empty
parameters, which would work when only one device is connected, but fail
when there are multiple. We've already acquired a transport and put it
into the socket as part of the forward request, so just use that
directly.

Bug: http://b/136198949
Test: test_device.py with multiple devices connected
Change-Id: I4d6bda45b36b71e418ecd9ead61b7379e68aa19b
2019-08-06 17:01:11 -07:00
Daniel Norman
f45125137c Merge "Adds check_interface_{restart,start,stop} check_builtins."
am: 0d061b258a

Change-Id: If100261f522a25fe85a10396d65bc065dd786774
2019-08-06 16:44:49 -07:00
Josh Gao
da9c767826 adb: don't manually construct device selection prefix in forward.
Use the existing infrastructure instead.

Test: test_adb.py
Test: test_device.py
Change-Id: I72576e1fb922af71852b50ddb3f4cb5c39eb333f
2019-08-06 16:43:34 -07:00
Daniel Norman
0d061b258a Merge "Adds check_interface_{restart,start,stop} check_builtins." 2019-08-06 23:21:29 +00:00
Yifan Hong
a716e1d0ff Merge "libsnapshot: correct DeviceInfo paths"
am: f9d5327287

Change-Id: Ifd9441bd8fc26289fc89293f6ecb66feadffd071
2019-08-06 15:43:45 -07:00
Treehugger Robot
f9d5327287 Merge "libsnapshot: correct DeviceInfo paths" 2019-08-06 22:12:53 +00:00
Vic Yang
5caa3e9e1a fs_mgr: Make kDefaultAndroidDtDir constexpr
Change its type from std::string to constexpr char[] so that it does
not need to be constructed at runtime, which pollutes the bss page
unnecessarily.

Bug: 138856262
Test: Along with another fix for libbootloader_message.so, see that bss
      section for libbootloader_message.so is now clean on cuttlefish
      for several processes.

Change-Id: I1c8ff9cfeb25164a352cef89cfa7be889b3d3387
2019-08-06 14:15:33 -07:00
Steve Muckle
ded44c06be libmodprobe: add verbose mode
Change-Id: I2be18320461cd712a4828400b8f29bb5f07c801f
2019-08-06 13:58:13 -07:00
Steve Muckle
781aa78ee2 libmodprobe: add GetAllDependencies
Add a method to retrieve the dependencies (both hard and soft) of a
module.

Change-Id: Ie44ceb3e36856bb1a3e68c5d3c0d55a38deb0ef9
2019-08-06 13:58:13 -07:00
Steve Muckle
012cfa19b0 libmodprobe: add support to list modules
List the known modules with a name matching a given pattern.

Change-Id: I7f6bd1f09a688c66682f94c5837e61d7dc61c1f7
2019-08-06 13:58:13 -07:00
Steve Muckle
e31f840a0a libmodprobe: add support for a blacklist
If the blacklist is enabled, blacklisted modules are treated as though
they are not present.

Change-Id: Ie8712f24298e78f92d5028b1ca3a8a3e07a9190a
2019-08-06 13:58:13 -07:00
Steve Muckle
13700a69d3 libmodprobe: support parameters in LoadWithAliases
Add support to specify module parameters in LoadWithAliases. These
parameters will be appended to any which are specified for the module in
modules.options.

Change-Id: I9aff1656ea397826f815b658b3b52c1892748601
2019-08-06 13:58:13 -07:00