Commit graph

48342 commits

Author SHA1 Message Date
Josh Gao
8ad965ae5e tombstoned: start immediately after /data is mounted.
Catch as many early-boot crashes as we can by starting tombstoned
immediately after /data is mounted.

Bug: http://b/139864948
Test: adb shell su 0 dmesg | grep "starting service"
Change-Id: I7f8821102191a445e87020f3efa59a2e0620d9db
2019-08-22 15:19:44 -07:00
David Anderson
9612e2767e Merge "libsnapshot: Use a real fake super partition for tests." 2019-08-22 20:39:59 +00:00
David Anderson
7e5bae88b8 libsnapshot: Use a real fake super partition for tests.
We are currently creating test partitions via ImageManager, which
unfortunately leads to a lot of hacks in my local tree to test
first-stage init. ImageManager devices look nothing like real partitions
in super, and it's a lot of work to massage it to act the same.

Instead, let's create an actual super partition. We can do this via
ImageManager, and give it its very own partition table. To make this
work, we need a special IPartitionOpener that will redirect requests for
"super" to our temporary device. A new .cpp file has been added to house
this. A few other tidbits have moved there as well.

This makes setup code a bit more complicated, but now our tests will
behave much closer to a real device.

Bug: 139204329
Test: libsnapshot_test gtest
Change-Id: I2f40109cd3c1d6343d6f52979789e25f9afd533a
2019-08-22 13:28:29 -07:00
David Anderson
c5b3c88f8a Merge "liblp: Support sdcards in PartitionOpener." 2019-08-22 19:12:40 +00:00
Tom Cherry
afe45ab5f2 Merge changes Id9534a59,I3ee81536
* changes:
  init: run property service in a thread
  Revert "init: Handle properties in the background of calling fs_mgr"
2019-08-22 17:24:18 +00:00
Treehugger Robot
6b20b7cff8 Merge "add nopreload option in public.libraries.txt" 2019-08-22 07:14:31 +00:00
David Anderson
b21f3dd761 Merge "libdm: Add a helper function for GetState+DeleteDevice checks." 2019-08-22 00:57:01 +00:00
David Anderson
251ec05f01 liblp: Support sdcards in PartitionOpener.
Before ImageManager was introduced, gsid avoided using PartitionOpener
when writing to external media. PartitionOpener couldn't interact with
non-boot devices, because it prepends /dev/block/by-name. We hacked
around this in both gsid and in first-stage init, which manually detects
the problem and prepends /dev/block instead.

After the ImageManager refactoring, sdcard support broke in gsid,
because it started relying on PartitionOpener. Let's fix this by allowing
/dev/block for mmcblk* names in PartitionOpener.

Bug: 139204329
Test: fiemap_image_test gtest
Change-Id: Ic1cbdbe0a18fc09522ee38cc62b35fd8193ce250
2019-08-21 12:27:56 -07:00
David Anderson
370226e662 libdm: Add a helper function for GetState+DeleteDevice checks.
Bug: N/A
Test: manual test
Change-Id: Ie97b63b53f548af1e82b73893fa491fac42b32ea
2019-08-21 12:27:56 -07:00
Tom Cherry
26f5e7da3a init: run property service in a thread
It's been a long standing issue that init cannot respond to property
set messages when it is running a builtin command.  This is
particularly problematic when the commands involve IPC to vold or
other daemons, as it prevents them from being able to set properties.

This change has init run property service in a thread, which
eliminates the above issue.

This change may also serve as a starting block to running property
service in an entirely different process to better isolate init from
handling property requests.

Test: CF boots, walleye boots, properties are set appropriately
Change-Id: Id9534a5916abb2f7d2a49cda54e33c1b69c50c2f
2019-08-21 08:26:09 -07:00
David Anderson
63fe0e9a36 Merge "fs_mgr: Allow using major:minor device strings in CreateLogicalPartition." 2019-08-20 20:42:24 +00:00
Josh Gao
c2f53906e8 Merge changes Iacf0093a,I40be3504
* changes:
  adb: fdevent: add fdevent_context_epoll.
  adb: extract soon-to-be-common code.
2019-08-20 20:00:34 +00:00
Jiyong Park
5db5d198ff add nopreload option in public.libraries.txt
A lib with 'nopreload' option in public.libraries.txt is not preloaded
during zygote. This is useful for seldom used public libraries; they
don't contribute to the zygote startup time and only affect the apps
that they are used.

Bug: 132911956
Test: libnativeloader_test
Change-Id: I6f97c90e6721aec7f2f96c8fc7b963b34f8edd3e
2019-08-21 00:35:34 +09:00
Yi Kong
ebfd2a468b Merge "Fix ODR violation" 2019-08-19 23:58:50 +00:00
Josh Gao
ef356fa5bd Merge "adb: fix mac build" 2019-08-19 23:18:01 +00:00
Adrian Roos
68c14d10b8 adb: fix mac build
Ensure that off64_t is defined when building adb for mac

Fixes: 139646013
Test: make # On mac
Change-Id: I32b68738d71c8a63a35c22434092933dcabac1ce
(cherry picked from commit 2167b9082d)
2019-08-19 10:51:00 -07:00
Yi Kong
0f6537c437 Fix ODR violation
host_init_verifier statically links libprocessgroup, which has shared
dependency on libjsoncpp. Right now it also has shared dependency on
libjsoncpp, resulting in ODR violation:

 ERROR: AddressSanitizer: odr-violation (0x7fce895f3220):
 [1] size=8 'Json::kNullRef' external/jsoncpp/src/lib_json/json_value.cpp:37:22
 [2] size=8 'Json::kNullRef' external/jsoncpp/src/lib_json/json_value.cpp:37:22
 These globals were registered at these points:
 [1]:
 #0 0x56276926ef0d (/usr/local/google/buildbot/src/android/master/out/host/linux-x86/bin/host_init_verifier+0xd1f0d)
 #1 0x7fce894ac1d9 (/lib64/ld-linux-x86-64.so.2+0x101d9)

 [2]:
 #0 0x56276926ef0d (/usr/local/google/buildbot/src/android/master/out/host/linux-x86/bin/host_init_verifier+0xd1f0d)
 #1 0x7fce894ac1d9 (/lib64/ld-linux-x86-64.so.2+0x101d9)

Move host_init_verifier to libjsoncpp_headers to fix the ODR violation.

Bug: 139546461
Bug: 131328001
Test: Run ASAN host_init_verifier
Change-Id: I54a51138d61fff37f092ffaa0758817600a0af9d
2019-08-18 17:57:35 -07:00
Yifan Hong
70a368119d Merge "liblp: UpdateMetadataForInPlaceSnapshot" 2019-08-17 00:18:54 +00:00
Josh Gao
7195a09fca Merge "Revert "adb: daemon: Assign valid fd to usb_handle ep0 file descriptor"" 2019-08-16 21:02:05 +00:00
Mark Salyzyn
e5aa946b63 Merge "fs_mgr: overlayfs: empty system partitions excluded." 2019-08-16 17:41:32 +00:00
Yifan Hong
1d6d4507a4 Merge "liblp: CreateLogicalPartition with a given mapped name" 2019-08-16 17:41:22 +00:00
David Anderson
c1c6008b78 Merge "libsnapshot: Improve test reliability." 2019-08-16 14:54:22 +00:00
David Anderson
f79af03011 Merge "libsnapshot: Don't compare strings for snapshot state." 2019-08-16 14:53:14 +00:00
David Anderson
3b5d5b5b8a libsnapshot: Improve test reliability.
The test suite is still quite buggy if interrupted. This fixes a number
of issues (such as bad ordering of setup calls), and refactors things to
add more ASSERTs.

Bug: 139204329
Test: libsnapshot_test gtest
Change-Id: I224608715c29f343b34512a9ac1143f0dde932e9
2019-08-15 17:42:51 -07:00
David Anderson
f27c063ba9 libsnapshot: Don't compare strings for snapshot state.
Convert the string field to an enum. We still write a string back to the
state file.

Bug: N/A
Test: libsnapshot_test gtest
Change-Id: I7cc1cb597dacd7d6faaaba05fb01c0a86bd54c8f
2019-08-15 17:42:47 -07:00
Dan Albert
8c2b3a8cde Revert "adb: daemon: Assign valid fd to usb_handle ep0 file descriptor"
This reverts commit b2d79c5e48.

Bug: http://b/135753060
Test: treehugger
Change-Id: I2b1b2b7dfb7bab783e4ed40e295329525ceaa0dd
(cherry-picked from internal commit ba4684c2b2)
2019-08-15 17:27:49 -07:00
Yifan Hong
536dd4292e Merge "Rename libfiemap libraries." 2019-08-15 23:52:23 +00:00
David Anderson
4929df0c44 fs_mgr: Allow using major:minor device strings in CreateLogicalPartition.
IPartitionOpener is useful for writing gtests, however, it can't easily
be used with CreateLogicalPartition. fs_mgr assumes the super partition
name will map to /dev/block/by-name/super whereas in tests we want to
redirect it to a different block device.

This CL makes two changes. First, it adds a new method to IPartitionOpener
to return a "device string" for a device name. The string must either be
an absolute path (for example /dev/block/by-name/super) or a major:minor
sequence, since device-mapper will accept either.

Second, CreateLogicalPartition now accepts an optional IPartitionOpener.
When converting block devices to paths, it uses the opener instead of
automatically prepending /dev/block/by-name.

Bug: 139204329
Test: liblp_test gtest
      libsnapshot_test gtest

Change-Id: Id6b3120cc2ef5c0dd941b29ff96215ad3c8ec848
2019-08-15 16:39:48 -07:00
Tom Cherry
3707d328a1 Revert "init: Handle properties in the background of calling fs_mgr"
This reverts commit 71bdf2820e.
Test: boot
2019-08-15 13:07:24 -07:00
Yifan Hong
f093f2173b Merge changes from topic "liblp_virtualab_prepare"
* changes:
  liblp: Don't halve super partition for virtual a/b devices
  liblp: MetadataBuilder::GetBlockDevicePartitionName
  liblp_test_static: GetMockedInstance->GetMockedPropertyFetcher
  libsnapshot: UnmapSnapshot: don't delete device that doesn't exist.
  liblp: change some functions to use string_view
2019-08-15 19:29:34 +00:00
Mark Salyzyn
18085418ee fs_mgr: overlayfs: empty system partitions excluded.
Beside using the 1% threshold for determining that a filesystem
should be an overlayfs candidate, do so also if the free space
is below 8MB.  This will cover nearly empty filesystems that can
not be right-sized below the smallest ext4 filesystem size.

Adjust the test to cover the new cases.

Should test all system partitions, including new /system_ext and
any others that get added in the future.

Fix instance of dir that were supposed to be die.  Using || die
does not work as expected within a function, so use if ! <cond>.

Test: adb-remount-test.sh
Bug: 139471424
Change-Id: If3e4cac5dce7bee998727052130eaf362e9e2fb5
2019-08-15 11:37:18 -07:00
Yurii Zubrytskyi
5f0e5e0ecc Merge "[base] Make MappedFile work with OS file handles" 2019-08-15 16:50:31 +00:00
Treehugger Robot
31f6263bc1 Merge "libsparse: replace legacy min() with std::min()" 2019-08-15 14:32:36 +00:00
Treehugger Robot
205ae55baf Merge "Initialize pointer member of LibaryNamespaces" 2019-08-15 06:47:44 +00:00
Treehugger Robot
12b0eb363d Merge "neuralnetworks namespace links to system namespace in [vendor] section" 2019-08-15 06:31:45 +00:00
cfig
946da7c528
libsparse: replace legacy min() with std::min()
this makes libsparse compile with MacOS clang++
Test: "libsparse" compiles and works fine under both Linux and MacOS

Change-Id: Ifcf018b856d2fe5f1dac7eeddd72de2810be66c5
2019-08-15 14:07:51 +08:00
Jiyong Park
1d89a7e899 neuralnetworks namespace links to system namespace in [vendor] section
This change fixes a bug that neuralnetworks namespace is linked to the
default namespace in [vendor] section. Default namespace is for
/vendor/lib in [vendor] section. System libraries are available in the
'system' namespace.

Bug: 139113342
Test: N/A
Change-Id: I1b6751e5e33522f9829f3d9ad7371efdce01b0f0
2019-08-15 09:48:07 +09:00
Jiyong Park
bb710ee6e4 Initialize pointer member of LibaryNamespaces
Explicitly initialize the pointer member `app_main_namespace_` of the
class LibraryNamespaces.

Test: atest libnativeloader_tests
Change-Id: I3932e7fb3c4074a127fcdbdedbd9c88652a7e36c
2019-08-15 08:10:55 +09:00
Yifan Hong
293cd027f7 liblp: UpdateMetadataForInPlaceSnapshot
For Virtual A/B devices, rename partitions / groups ending in
source slot to target slot, so that the metadata can later
be written to target metadata slot.

Test: liblp_test_static
Bug: 135752105
Change-Id: I6a7b25e8da2808b1831f317760f4345df6b22689
2019-08-14 14:27:03 -07:00
Yifan Hong
5217c1cf0e Rename libfiemap libraries.
Update Android.bp for libfiemap library renames.

Test: builds

Change-Id: I318c5f64452726579cebb41792b95f6b9e47fbb8
2019-08-14 14:17:22 -07:00
Yifan Hong
fadce1ee1f liblp: Don't halve super partition for virtual a/b devices
For Virtual A/B devices, we snapshot the partitions. Hence
there is no need to prioritize using the second half of super
partition for _b partitions.

Test: liblp_test_static
Bug: 135752105

Change-Id: I8bd669e6ffa0a4d114a1b386133179bd2819099b
2019-08-14 14:17:22 -07:00
Yifan Hong
8367b9f73a liblp: MetadataBuilder::GetBlockDevicePartitionName
Allow to retrieve name of a block device partition at
a certain index. If the index is out of range, return
empty string.

This is needed for an hypothetical edge case when an
intersection of two LinearExtent needs to be computed,
and the two metadata have a different list of block
device names. In reality, Virtual A/B devices are always
launched with dynamic partitions, so it will always return
"super" for index 0 and "" otherwise.

Test: liblp_test_static
Bug: 135752105

Change-Id: I9ea59edefdc41d0e69e3644aa2452676372938b4
2019-08-14 14:17:22 -07:00
Treehugger Robot
19b68a66e5 Merge "Fix more endian.h issues" 2019-08-14 21:06:28 +00:00
Josh Gao
b43ad44f4c adb: fdevent: add fdevent_context_epoll.
Implement an epoll-backed fdevent_context, that reduces overhead when
we're polling many file descriptors. FdeventTest.smoke goes from ~5.2s
to ~3.3s when run on the host (after this patch's modification to change
it from chaining 10 file descriptors together to 512).

Test: adb_test on host
Test: adbd_test on blueline
Test: test_adb.py
Test: test_device.py
Change-Id: Iacf0093aa7bebea31e447c2cb012af72d8c3297e
2019-08-14 13:56:08 -07:00
Josh Gao
e22cb9f306 adb: extract soon-to-be-common code.
As a side effect, delete FDE_ACTIVE, which was always set on every
fdevent, and FDE_PENDING, which was an internal implementation detail.

This patch removes spin detection, which will be reimplemented
separately later.

Test: adb_test on host
Test: adbd_test on blueline
Change-Id: I40be3504ce03c4fae5e071fa018542a051b7511d
2019-08-14 13:56:08 -07:00
Yifan Hong
4644f84208 liblp_test_static: GetMockedInstance->GetMockedPropertyFetcher
Rename it to be a better name.
Test: run it

Change-Id: Idcf271e84f8758737a98a4f6c18ea6744b91f79b
2019-08-14 13:33:59 -07:00
Yifan Hong
94da8944f5 libsnapshot: UnmapSnapshot: don't delete device that doesn't exist.
Otherwise, DeleteDevice will just return false and it will
be stuck here.

Test: apply OTA after first boot, update_engine can unmap system_b
Bug: 135752105

Change-Id: If2b00336f4b948ba5f57a5e129fc070a739848fa
2019-08-14 13:33:59 -07:00
Yifan Hong
b278e013b9 liblp: change some functions to use string_view
Converting all of them to string_view requires some
non-trivial amount of refactoring (because there is not
an implicit conversion from string_view to string), and
hence may be done later if necessary.

Test: builds
Bug: 135752105
Change-Id: I936ae7117500fadc7b34a5b91b279bc863d9919d
2019-08-14 13:33:59 -07:00
Yurii Zubrytskyi
d77c99ebc3 [base] Make MappedFile work with OS file handles
adb uses its own implementation of Windows HANDLE->int mapping,
and it doesn't play well with _get_osfhandle() function used in
MappedFile::FromFd().

This CL adds another function that accepts raw handle, so adb
can pass it directly

+ make constant functions 'const'
+ make the MappedFile movable, as nothing prevents it from being
  one

Test: libbase_test

Change-Id: Ifde4a4094b910e9c7b431126ecf3fef5aa3bb4a6
2019-08-14 10:04:32 -07:00
Treehugger Robot
61bffa5fc0 Merge "[adb] Use STL for ParseUInt() implementation" 2019-08-14 06:32:44 +00:00