Commit graph

58560 commits

Author SHA1 Message Date
Daniel Rosenberg
9cfc622cb2 Merge "Set GC threshold when mounting with checkpointing" into rvc-dev 2020-03-14 00:09:18 +00:00
TreeHugger Robot
bec237bdee Merge "liblog: move LOG_ID_DEFAULT into the log_id enum" into rvc-dev 2020-03-13 23:57:30 +00:00
TreeHugger Robot
8a1aba349c Merge "liblog: fix header guards ordering" into rvc-dev 2020-03-13 23:46:52 +00:00
Tom Cherry
1dc67f3015 liblog: fix header guards ordering
__ANDROID_API__ isn't defined on host, so this check will fail due to
-Wundef.  The || is short circuited through, so we can check for host
first and only have the right side of the || evaluated for device
builds.

Bug: 150898477
Bug: 151255209
Test: build
Merged-In: I2e0bc055ac0c7ba8262875801cdaed9cc30d2cef
Change-Id: I2e0bc055ac0c7ba8262875801cdaed9cc30d2cef
(cherry picked from commit 227ac56454)
2020-03-13 14:12:46 -07:00
Tom Cherry
bc2e58d6c2 liblog: move LOG_ID_DEFAULT into the log_id enum
As requested during the API review.

Bug: 150898477
Test: liblog and libbase unit tests
Merged-In: Id67203bbfbc3e116b6b6efd40428afe48203dcfc
Change-Id: Id67203bbfbc3e116b6b6efd40428afe48203dcfc
(cherry picked from commit 6085cd5a64)
2020-03-13 14:10:08 -07:00
Tom Cherry
a9a6d49ec0 liblog: use int32_t and uint32_t for new NDK APIs
As requested during the API review.

Bug: 150898477
Test: liblog and libbase unit tests
Merged-In: I0c1dd57f7499e432bb881e1da1beb55f1ff7de05
Change-Id: I0c1dd57f7499e432bb881e1da1beb55f1ff7de05
(cherry picked from commit f1a975bece)
2020-03-13 14:09:59 -07:00
Ruchir Rastogi
e4e0566e02 Merge "Support atom-level annotations within AStatsEvent" into rvc-dev 2020-03-13 20:16:51 +00:00
Daniel Rosenberg
7c59f1ae9d Set GC threshold when mounting with checkpointing
This steps up our garbage collection threshold over time when mounting
userdata with checkpointing=disable for f2fs. With this scheme, we step
up the percent of disk we will tolerate being unusable during
checkpointing. At 100%, the filesystem will be able to always mount.
This means we will attempt to mount at most 10 times, for a max time of
about 50 seconds.

Also logs mount time under ro.boottime.init.mount.[target]

If the device does not support checkpoint=disable:[n%], it will mount
with checkpoint=disable, which is equivalent to checkpoint=disable:0%

Test: Boot in checkpointing mode
Bug: 150506496
Change-Id: I047ff98513f4ea832e1216b91e454a57fd2e8bf6
2020-03-13 20:12:02 +00:00
Eric Biggers
fcdaee4861 Merge "init.rc: disable kernel module autoloading" into rvc-dev 2020-03-13 19:10:19 +00:00
Alex Buynytskyy
33ac1b05f9 Implement the new v4 signing scheme in adb
To be submitted along with changes in apksigner tool and the framework.
Merged to AOSP after that.

Test: adb install --incremental <apk>

go/apk-v4-signature-format

Bug: b/151241461
Change-Id: I26e187f8e389e31e2759037057b96fc6c9cb1e94
2020-03-13 11:04:17 -07:00
Jiyong Park
a94047be70 Merge "Make libselinux a stub library" into rvc-dev 2020-03-13 05:16:49 +00:00
Jiyong Park
8ce4a3067c Make libselinux a stub library
libselinux is currently being copied to APEXes. This is risky because
the library is not designed to be portable; part of it is tied to the
specific version of the Android that it was developed for.

This change fixes the problem by declaring that the library supports
a stub with the list of C APIs that are included in the stub. Then there
is only one copy of libselinux in /system/lib and other APEXes use the
copy by dynamically linking to it.

Also, adbd no longer statically links to it, because doing so brings
libselinux in it.

Bug: 151053366
Test: m com.android.adbd. It doesn't include libselinux in it.
Test: m com.android.adbd-deps-info. then inspect
out/soong/com.android.adbd-deps-info.txt. The dependency to libselinux
is shown as '(external)'.

Exempt-From-Owner-Approval: cherry-pick from AOSP

Merged-In: If418cbe3abdeacb759d59052e6dca4c2067678dd
(cherry picked from commit 3ffdad0cb5)
Change-Id: If418cbe3abdeacb759d59052e6dca4c2067678dd
2020-03-13 05:16:28 +00:00
Yifan Hong
5bcf6498c3 Merge "Add OptimizeSourceCopyOperation" into rvc-dev 2020-03-12 23:53:03 +00:00
Ruchir Rastogi
e712eed24c Support atom-level annotations within AStatsEvent
Refactors implementation details to allow for atom-level annotations.
This CL does not change the API surface. addBoolAnnotation and
addInt32Annotation work for both atom-level and proto field annotations.

Test: bit libstatssocket_test:*
Bug: 151158794
Change-Id: I2c340b201aeb3bcddd5cfde171b7c4df1d8d69a4
2020-03-12 15:21:26 -07:00
Eric Biggers
b6545b80a0 init.rc: disable kernel module autoloading
There is a longstanding bug where file-based encryption causes spurious
SELinux denials of module_request because it uses the kernel's crypto
API, and the crypto API tries to autoload kernel modules.

While this sometimes indicate missing kconfig options, it can still
happen even if all needed kconfig options are enabled.  This is because
a crypto algorithm can be a composition like "hmac(sha512)", and the
crypto API will first look for the full composition before it
instantiates it using the components like "hmac" and "sha512".  But
often an implementation of the full composition doesn't exist.

However, as far as I can tell, Android doesn't actually use kernel
module autoloading at all.  First, Android never changes
/proc/sys/kernel/modprobe from the default of "/sbin/modprobe", yet this
isn't where modprobe is located on Android.  Android's SELinux policy
contains a neverallow rule that ensures that only init (not even
vendor_init) can write to this setting, so vendors can't be changing it.

Vendors could potentially be setting CONFIG_STATIC_USERMODEHELPER_PATH,
which overrides the path of all usermode helpers including modprobe.
But this is a relatively new kconfig option, available only in
android-4.14 and later.  Also, for a vendor to actually do this they'd
also need to extend the SELinux policy with a domain_auto_trans rule to
allow their usermode helper to be executed by the kernel.

Android does increasingly use kernel modules, and GKI (Generic Kernel
Image) will require them.  However, the modules are actually inserted by
userspace by 'init', not autoloaded.

It's possible to disable kernel module autoloading completely by setting
/proc/sys/kernel/modprobe to an empty string.  So, let's do that.

This prevents lots of spurious SELinux denials, and allows removing
unnecessary rules to allow or dontaudit the module_request permission.

Note: when the kernel doesn't have CONFIG_ANDROID_BINDERFS enabled, this
change exposes a kernel bug that causes a WARNING in get_fs_type().  To
avoid this WARNING, a kernel fix should be applied too -- currently
under discussion upstream
(https://lkml.kernel.org/r/20200310223731.126894-1-ebiggers@kernel.org).

Bug: 130424539
Bug: 132409186
Bug: 144399145
Bug: 146477240
Bug: 148005188
Bug: 149542343

Test: Tested on cuttlefish and coral:

    - Checked that /proc/sys/kernel/modprobe contains /sbin/modprobe
      before this change, and the empty string after.

    - Checked that if all SELinux rules for module_request are removed,
      there are SELinux denials for module_request before this change
      but none after.

    - Ran lsmod both before and after and verified that the list is the
      same, i.e. checked that this change doesn't break how Android
      actually loads kernel modules.

Change-Id: I4132fe1a491e7b789311afcf693c1f6493fb9dc5
Merged-In: I4132fe1a491e7b789311afcf693c1f6493fb9dc5
2020-03-12 14:20:40 -07:00
Yifan Hong
d8a38e7c23 Add OptimizeSourceCopyOperation
... so that an operation can be skipped partially. For example, if
an operation contains blocks:
    563412 -> 123456
... then optimized operation is:
    5612 -> 1256

Test: update_engine_unittests
Test: apply incremental OTA
Bug: 148623880

In an experiment, this reduces CoW size of an incremental update
package by 200MB (out of 700MB).

Change-Id: I86ca23fd589ddbc84c81318283b5f4e71782a759
Merged-In: I86ca23fd589ddbc84c81318283b5f4e71782a759
2020-03-12 10:00:12 -07:00
Jiyong Park
c2a354be68 Merge "don't include liblog to APEXes" into rvc-dev 2020-03-12 04:10:15 +00:00
TreeHugger Robot
575d8c2c9e Merge "adbd: make libadbd_services cc_library again." into rvc-dev 2020-03-12 02:06:09 +00:00
Tom Cherry
293786f941 Merge "Revert "Stop & Resume property service when switching to bootstrap namespace"" into rvc-dev 2020-03-11 23:25:52 +00:00
Josh Gao
d1ee5085f3 adbd: make libadbd_services cc_library again.
For currently unknown reasons, sideloading is broken with
libadbd_services as a cc_library_static.

Partial revert of commit a9b62d5452.

Bug: http://b/151056300
Test: xunchang@ tested manually
Change-Id: Iaffad9c476ba0adcffc5db512ba4a7ee0fb5cb22
(cherry picked from commit 7f8a37c8c7)
2020-03-11 23:18:41 +00:00
Jaegeuk Kim
26471f7470 Merge "run ext4 tools in first_stage_ramdisk" into rvc-dev 2020-03-11 20:05:22 +00:00
Nikita Ioffe
ccb0608a32 Merge "fs_mgr_get_mounted_entry_for_userdata: handle logical partitions" into rvc-dev 2020-03-11 19:17:30 +00:00
Jaegeuk Kim
253b0729bc run ext4 tools in first_stage_ramdisk
Bug: 149039306
Change-Id: I646960baa3c0d5a0dedda82b1fdc3816321aaa2e
Merged-In: I646960baa3c0d5a0dedda82b1fdc3816321aaa2e
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2020-03-11 16:39:58 +00:00
Nikita Ioffe
f495fe798b fs_mgr_get_mounted_entry_for_userdata: handle logical partitions
In case of GSI on DSU, /data is mounted on the following stack:
userdata -> userdata_gsi -> actual block device, and fstab returned from
ReadDefaultFstab() function will say that /data should be
mounted onto userdata_gsi. This configuration wasn't supported.

To support it, fs_mgr_get_mounted_entry_for_userdata now unwinds the
dm-device stack /data is mounted onto, and for each entry in fstab
resolves corresponding block device (in case of logical partitions by
asking DeviceMapper to resolve name -> path, and for other cases by
reading a symlink), and then checking whenever resolved block device
belongs to the unwinded stack.

Test: on both normal boot & DSU
Test: atest CtsUserspaceRebootHostSideTestCases
Test: atest CtsFsMgrTestCases
Bug: 148612148
Change-Id: I1bb3e64ae92d56de905b4f7f1997b7499b101b0f
2020-03-11 15:10:20 +00:00
Jiyong Park
01aa81cec5 don't include liblog to APEXes
liblog is a platform library that provides stable C API. There is no
need to include the library, especialy by statically linking to it, in
any APEX. It not only wastes the storage/ram, but also is incorrect
because the socket interface to logd which is implemented in liblog is
not guaranteed to be stable.

Fixing this issue by converting static_libs: ["liblog"] into
shared_libs: ["liblog"], in which case the dependency to the library
is satisfied via the stub variant of the library.

As a result, we could restrict the availablity of the library to
the platform and the runtime APEX.

Exempt-From-Owner-Approval: already approved when this was in internal
master (ag/10572699)

Bug: http://b/151051671
Bug: http://b/150827719
Test: m
Merged-In: I5aab863cb12b8767b6979255c247000a59355b0e
(cherry picked from commit 95b6f45b0e)
Change-Id: I5aab863cb12b8767b6979255c247000a59355b0e
2020-03-11 18:30:06 +09:00
Jiyong Park
846d14799b Merge changes from topic "liblog" into rvc-dev
* changes:
  Whole static link libasync_safe to libbacktrace.a
  libbase uses liblog symbols via dlsym when it is built for APEX
2020-03-11 03:25:29 +00:00
Tom Cherry
0c3d4843de Revert "Stop & Resume property service when switching to bootstrap namespace"
Dependent change was reverted.

This reverts commit 352ae2d225.

Bug: 150863651
Test: build
Merged-In: I2e11b782fcd2b7baded133a841fdcf519f702b84
Change-Id: I2e11b782fcd2b7baded133a841fdcf519f702b84
2020-03-10 16:38:44 -07:00
Josh Gao
8c6d98af1c Merge "adb: defer acknowledgement of pushed files until the end." into rvc-dev 2020-03-10 22:33:22 +00:00
TreeHugger Robot
3ae89289bd Merge "liblog: add API level guard to NDK log.h header" into rvc-dev 2020-03-10 21:27:45 +00:00
Yifan Hong
77d51eb888 Merge changes from topic "no_snapshot_callstack" into rvc-dev
* changes:
  Revert "libsnapshot::RemoveAllUpdateState log stack"
  Revert "libsnapshot callstack files readable by dumpstate"
  libsnapshot: delete InitiateMergeAndWait.
2020-03-10 19:18:48 +00:00
Tom Cherry
060f58ad35 liblog: add API level guard to NDK log.h header
Bug: 119867234
Test: build
Merged-In: I6fdd7c070af06f04314b9bf7bbfede2a106e0d7c
Change-Id: I6fdd7c070af06f04314b9bf7bbfede2a106e0d7c
(cherry picked from commit 59e93a55fc)
2020-03-10 11:55:43 -07:00
Tom Cherry
1588e9b0c2 liblog: minimum_log_priority should be atomic
In case multiple threads try to reference this variable while it is
being set, it should be atomic so that all threads always see a valid
value.

Bug: 150898477
Test: liblog, libbase unit tests
Merged-In: If6c9e291f2471b96a752dc6e76e3e63458b71391
Change-Id: If6c9e291f2471b96a752dc6e76e3e63458b71391
(cherry picked from commit bbb16022c7)
2020-03-10 08:46:15 -07:00
Howard Chen
c7e257c1e9 Convert gsid to use the dynamic AIDL service infrastructure
Replace the original on demand start mechanism with the new dynamic
AIDL service infrastructure to resolve a possible race condition.

Bug: 149130673
Test: gsi_tool status

Merged-In: Ia5f32579a8dcf62d700d974c7f4e3c65647f3b8b
Change-Id: Ia5f32579a8dcf62d700d974c7f4e3c65647f3b8b
2020-03-10 17:35:41 +08:00
Josh Gao
6519ad6355 adb: defer acknowledgement of pushed files until the end.
Previously, we were waiting for the other end to respond after every
file sent, which results in massive slowdown when there's any amount of
latency on the transport.

This improves performance on a cuttlefish instance with ~7ms RTT from:

    system/: 2037 files pushed, 0 skipped. 2.8 MB/s (762803979 bytes in 262.964s)

to:

    system/: 2037 files pushed, 0 skipped. 11.9 MB/s (762803979 bytes in 61.278s)

Bug: https://issuetracker.google.com/150827486
Test: ./test_device.py
Change-Id: I3a0c893faa5d455cc6ccbc86915a17e1b5abbfbe
(cherry picked from commit 64ff82ba68)
2020-03-09 18:57:15 -07:00
Jiyong Park
7c8e2b247d Whole static link libasync_safe to libbacktrace.a
When libbacktrace.a is statically lined to somewhere, that library had
to add libasync_safe.a to static_libs because libbacktrace.a has
references to libasync_safe.a. But libbacktace depending on
libasync_safe is an implementation detail of libbacktrace, and therefore
its client shouldn't be affected by it.

Fixing this by doing the whole static link to libasync_safe to
libbacktrace.a so that the former is included in libbacktrace.a

Bug: 149569129
Test: m
Change-Id: If7366a240bc945dda9944fe7c111e10d328165bb
2020-03-10 10:08:26 +09:00
Jiyong Park
5280b5c03e libbase uses liblog symbols via dlsym when it is built for APEX
libbase is a popular library that is used by many APEXes either directly
or transitively. It is being used by several Mainline modules that were
launched with Q, in which case everything in the APEX - including
libbase - shouldn't use new APIs that are added post Q, i.e. R.

libbase however is using a few new R symbols from liblog, and this is
preventing those Q-launching Mainline modules that are built in R source
tree from being installed to Q devices.

Fortunately, the dependencies to the new R symbols are guarded with a
flag; when the existence of the symbols are not guaranteed, it uses
dlsym. This change fixes the aforementioned problem by turning on the
flag also when libbase is built for an APEX.

Bug: 149569129
Test: TARGET_BUILD_APPS=com.android.media
vendor/google/build/build_mainline_modules.sh
adb install --staged out/dist/mainline_modules_arm64/com.android.media.apex
adb reboot
The APEX is installed and mediaextractor process doesn't crash

Change-Id: I44b5ec028850613cb45fc3e792f43cd8e87cfd00
2020-03-10 10:08:26 +09:00
Yifan Hong
21ab3adc73 Revert "libsnapshot::RemoveAllUpdateState log stack"
Revert submission snapshotctl_callstack

Reason for revert: Callstack should not be logged
Reverted Changes:
Ib80c74a9a:Temporarily add libutilscallstack dependency
I2dfb6b7f1:libsnapshot::RemoveAllUpdateState log stack

Test: none
Bug: 148818798

Change-Id: I34683e93f10971629737f6fe648b25c6066c702f
2020-03-09 16:16:33 -07:00
Yifan Hong
611c6f7d79 Revert "libsnapshot callstack files readable by dumpstate"
This reverts commit faa49d1d00.

Reason for revert: Callstack should not be logged.

Test: none
Bug: 148818798

Change-Id: Ie5506ae0c7408e255a464b2f403d0a47d272229e
2020-03-09 16:16:33 -07:00
Yifan Hong
ffd1f52c22 libsnapshot: delete InitiateMergeAndWait.
Now that update_engine is responsible for initiating the
merge, InitiateMergeAndWait function becomes useless.

Bug: 147696014
Test: libsnapshot_test
Change-Id: I5473dc543ca8ac2fd31f597720847b02d0e5e33d
2020-03-09 16:16:33 -07:00
Ruchir Rastogi
7a287d0751 Use dlsym to load libstatssocket API
Also fixes a bug in the tests where the platform version was being
checked by querying version.release instead of version.codename
(currently, version.release == 10, while version.codename == R).

Test: bit libstatspush_compat_test:* (on Q and R)
Test: libsstatssocket.so is not placed in resolv apex (on R)

1|bonito:/ $ ls apex/com.android.resolv/lib64
libclang_rt.ubsan_standalone-aarch64-android.so
libcrypto.so
libnetd_resolv.so
libssl.so

Test: NETWORK_DNS_EVENT_REPORTED atom is logged to statsd (on R)
    - adb shell cmd stats print-stats (atom 116 count > 0)

Bug: 148743333
Change-Id: Ib3eaa32835905bcf6e3b003054bf0f3c4e7ec0a4
Merged-In: Ib3eaa32835905bcf6e3b003054bf0f3c4e7ec0a4
(cherry picked from commit 9848eb4f1c)
2020-03-09 15:16:11 -07:00
TreeHugger Robot
19212e3212 Merge "Use libbase to get the elapsed time instead of libutils" into rvc-dev 2020-03-09 20:53:24 +00:00
David Anderson
114a86ebb4 Merge "libdm: Fix WaitForFile early-returning on failed accesses." into rvc-dev 2020-03-09 19:08:43 +00:00
Luke Huang
7452e46ae0 Use libbase to get the elapsed time instead of libutils
To minimize the external library dependencies

Bug: 149721367
Test: atest
Change-Id: I7b79223ad58c2af94d01a35ad39be6cdd1de7f47
Merged-In: I7b79223ad58c2af94d01a35ad39be6cdd1de7f47
(cherry picked from commit cab9854fa9)
2020-03-06 22:09:44 +00:00
Nikita Ioffe
bbbd1a8e41 Merge "Move GetMountedEntryForUserdata to fs_mgr.h" into rvc-dev 2020-03-06 19:45:54 +00:00
TreeHugger Robot
5f3afb215b Merge "debuggerd: add an opt-out for OEMs that would rather collect core dumps." into rvc-dev 2020-03-06 19:42:41 +00:00
Nikita Ioffe
8f50cfc28d Move GetMountedEntryForUserdata to fs_mgr.h
Logic of unwinding dm-device stack to figure out what entry was used to
mount userdata turned out to be a little bit more involved, and it
shouldn't be part of libfstab

This CL just moves code around and cleans API a little bit, actual fix
will be in the follow-up CL.

Test: atest CtsFsMgrTest
Test: atest CtsUserspaceRebootHostSideTestCases
Bug: 148612148
Change-Id: If0f8a765dba63adb0e6a711eb81fffdcabea3480
2020-03-06 18:31:38 +00:00
Yifan Hong
f9783dc18d Merge changes from topic "snapshot_rollback" into rvc-dev
* changes:
  init: restorecon libsnapshot rollback indicator
  init: Make sure ImageManager status files are labelled.
2020-03-06 18:01:06 +00:00
Elliott Hughes
76eb935bd4 debuggerd: add an opt-out for OEMs that would rather collect core dumps.
On userdebug/eng devices, check a system property to see whether we
should create tombstones or not. OEMs that would rather have core dumps
can set this property and configure /proc/sys/kernel/core_pattern
appropriately.

Bug: https://issuetracker.google.com/149663286
Test: set the property, cause a crash
Change-Id: If894b4582a1820b64bdae819cec593b7710cb6e3
(cherry picked from commit 530ab81e42)
2020-03-06 09:21:45 -08:00
Automerger Merge Worker
88b85abc6a [automerger skipped] Add cpu-set properties to serializer test am: 6ea9b8b580 -s ours
am skip reason: Change-Id I1232fec5f1d0be7e5fb6aa5c46f6c2c5529bfc6c with SHA-1 8e77b789f8 is in history

Change-Id: I0e13c779e06c07a648e4ac8ccbfeec437f4dbb7a
2020-03-06 16:19:26 +00:00
Yifan Hong
dd4e7ef6ce init: restorecon libsnapshot rollback indicator
The rollback indicator is created during first stage init and
is unlabeled. Need to restorecon it properly so that it can
be properly deleted by update_engine later.

Bug: 149956852
Bug: 150907679
Test: apply OTA, reboot, look at serial console
Change-Id: Ie954023b557865691fec2ae378145596bf1973a9
2020-03-05 21:56:03 -08:00