Commit graph

81774 commits

Author SHA1 Message Date
Yi-Yo Chiang
6b57c885d3 Merge "init_first_stage: Disable ThinLTO" into main 2023-07-21 05:08:13 +00:00
David Anderson
89d22239d2 Merge "snapuserd: Fix ubsan when reading a single aligned sector." into main 2023-07-20 16:23:08 +00:00
Yi-Yo Chiang
4d6fa8ccaf init_first_stage: Disable ThinLTO
Static executables + x86 target build + ThinLTO produces bug behavior.
Global variables are not constructor initialized, resulting in faulty
runtime behavior.

Bug: 169004486
Bug: 291033685
Test: Treehugger
Change-Id: I777016cceb4851f2b432a37bc4d29aed56c23804
2023-07-20 18:47:16 +08:00
Jakob Vukalović
998c3b4fbe Merge "ueventd: Fix creation of VFIO dev nodes" into main 2023-07-20 09:04:25 +00:00
David Anderson
e1132a9c2f snapuserd: Fix ubsan when reading a single aligned sector.
When a read request is for a single sector, and the sector is
block-aligned, it falls into the ReadAlignedSector path, which assumes
block-sized reads. Fix this by clamping the expected size.

Bug: 291862304
Test: manual test of ReadWorker::ReadAlignedSector
      full OTA
Change-Id: I00e460c333e8a9a4dc2433443e3633f3d794da1d
2023-07-19 23:05:58 -07:00
Treehugger Robot
fe1fa35823 Merge "storageproxyd: Start binder thread pool" into main 2023-07-19 22:52:21 +00:00
Stephen Crane
fb92cd3c22 storageproxyd: Start binder thread pool
The Trusty storage proxy requires that the suspend service is started to
acquire a wakelock for UFS RPMB operations. Without the binder thread
pool running, starting this service results in at least a 1s polling
delay. This change ensures that we start the thread pool before handling
any RPMB operations, so acquiring the wakelock will complete as soon as
the service is ready without needing to poll once per second.

Test: m storageproxyd
Test: Artificially delay suspend_service to check if we poll
Bug: 281951047
Change-Id: I1a4cdd48d57201b0cf9c24523d22e5bdbcea376a
2023-07-19 18:37:52 +00:00
Jakob Vukalovic
e377432924 ueventd: Fix creation of VFIO dev nodes
VFIO nodes, both the container (`vfio`) node and group (numbered)
nodes, should be located in `/dev/vfio`. This change prevents
ueventd from flattening that structure.

Test: Bind a device to VFIO driver to create a VFIO group
Change-Id: I635e9febe6bb52718df263e735479f361eacad4c
2023-07-19 10:03:10 +01:00
Eric Miao
4a33c22c77 Merge "String8: fix infinite loop and segmentation fault in removeAll()" into main 2023-07-19 02:03:40 +00:00
Daniel Zheng
f161ee2771 Merge "Adding documentation" into main 2023-07-19 00:20:59 +00:00
Eric Miao
c6ce48ef19 String8: fix infinite loop and segmentation fault in removeAll()
Bug: 290835996
Test: libutils_fuzz_string8 for several minutes

String8::removeAll() has 2 serious problems:

1. When `other` is an empty string, `removeAll()` will loop infinitely
   due to below process:

   a) with `other` being empty string `""`, find() will call strstr()
      on an empty string, which always returns `mString`, and thus
      find() always return 0 in this case
   b) with find() returns 0 for empty string, the next while loop in
      String8::removeAll() will keep loop infinitely as `index` will
      always be 0

   This CL fixes this problem by returning true if `other` is an empty
   string (i.e. `strlen(other) == 0`), this follows the logic that an
   empty string will always be found and no actual remove needs to be
   done.

2. When `other` is a NULL string, strstr() has undefined behavior. See
   https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf.

   This undefined behavior on Android unfortunately causes immediate
   segmentation fault as the current `strstr` implementation in bionic
   libc doesn't check `needle` being NULL, and an access to a NULL
   location is performed to check if the `needle` string is an empty
   string, and thus causes segmentation fault.

   This CL gives an error message and aborts instead of having a
   segfault, and to keep some backward compatibility.

   This CL also adds test for String8::removeAll()

Change-Id: Ie2ccee6767efe0fed476db4ec6072717198279e9
2023-07-18 16:02:07 -07:00
Daniel Zheng
658182b8e7 Adding documentation
adding --help documentation to fastboot for --disable-super-optimization
and --disable-fastboot-info

Test: fastboot -h
Change-Id: Ia8993b3894d302a63cc97796d66e0af3fb004eef
2023-07-18 15:36:26 -07:00
Treehugger Robot
435b520bf0 Merge "Fix libutils_fuzz_string8 deadlock." into main 2023-07-18 21:49:06 +00:00
Hao Chen
8dad4b2b8a Merge "threads.h: avoid defining gettid on glibc >= 2.30" into main 2023-07-18 21:38:34 +00:00
Steven Moreland
749becfa68 Fix libutils_fuzz_string8 deadlock.
Bug: 290835996
Test: libutils_fuzz_string8 for several minutes
Change-Id: I9b312dd968c380f4fa2a837d38121d0a7a7ac7b1
2023-07-18 21:04:07 +00:00
Hao Chen
253445ce3a threads.h: avoid defining gettid on glibc >= 2.30
The issue in https://bugs.chromium.org/p/chromium/issues/detail?id=1182060 also
exists on glibc 2.30 and 2.31 since `gettid` was Introduced in glibc 2.30.

See https://man7.org/linux/man-pages/man2/gettid.2.html

Bug: 285204695
Test: Build
Change-Id: I7e534edf8c0a20c415232bcfffabbf2c1d6eec98
2023-07-18 19:52:19 +00:00
David Anderson
72c3ca1604 Merge "snapuserd: Refactor how buffers are managed." into main 2023-07-18 16:42:22 +00:00
Eric Biggers
0306bc1a31 Merge "init: avoid ERROR log due to missing SEPolicy.zip" into main 2023-07-18 15:38:20 +00:00
David Anderson
3930625074 snapuserd: Refactor how buffers are managed.
Currently all I/O is processed through BufferSink. However, the
allocation of buffers is implicit in low-level helper functions, which
have no knowledge of how much data will actually be sent. As a result,
"allocation of buffers" and "tracking of how many bytes were written" is
disjoint. This will make it very difficult to break dependence on
dm-user, without a much more complex API.

This patch refactors how BufferSink is used. First, GetPayloadBuffer has
been deprecated in favor of AcquireBuffer. AcquireBuffer performs both
allocation and write tracking. The number of bytes written is not
necessarily the number of bytes allocated, and the new API allows for
this.

The "Process" helpers now take in an explicit buffer, and their callers
are responsible for allocating a buffer. This allows for
ReadUnalignedSector to have proper buffer offset tracking.

Because write tracking is now accurate, no explicit size needs to be
passed to WriteDmUserPayload. It can simply read the current watermark
in BufferSink.

This patch also removes XorSink, since its dependence on BufferSink made
this change more difficult.

Bug: 288273605
Test: snapuserd_test
Change-Id: Id9ca5044f9c0386f351b250349793b6b276b01b7
2023-07-17 17:10:19 -07:00
Treehugger Robot
1c77579a06 Merge "Look for radio.img and bootloader.img in ANDROID_PRODUCT_OUT" into main 2023-07-17 21:51:01 +00:00
Paul Lawrence
556c46bfb9 Look for radio.img and bootloader.img in ANDROID_PRODUCT_OUT
Test: fastboot flash bootloader and fastboot flash radio both work
Change-Id: I79674469c0df3dd592e5fcf840fef0dcc3f11d33
2023-07-17 14:05:21 -07:00
Eric Biggers
53ed745e3f init: avoid ERROR log due to missing SEPolicy.zip
One of the first ERROR messages in logcat of a normal boot of Cuttlefish
is from failure to open SEPolicy.zip.  This condition is expected.
Therefore don't try to load SEPolicy.zip when it doesn't exist.  This
replaces the following log messages:

0     0 I init    : Error: Apex SEPolicy failed signature check
0     0 I init    : Loading APEX Sepolicy from /system/etc/selinux/apex/SEPolicy.zip
0     0 E init    : Failed to open package /system/etc/selinux/apex/SEPolicy.zip: No such file or directory

... with just:

0     0 I init    : No APEX Sepolicy found

Change-Id: If3a77407c35130165df5782b9ef91912e8374dbf
2023-07-17 20:45:44 +00:00
Eric Biggers
42164ff920 Merge changes from topic "fsverity-init-cleanup" into main
* changes:
  init.rc: stop using fsverity_init --lock
  init: remove unfinished fsverity signature support for APEX sepolicy
2023-07-17 20:10:28 +00:00
David Anderson
83ebc4376d Merge changes I1625d1a6,I2db9cfa2,I59c31318,Ic0ed1a8d,I612374bb into main
* changes:
  snapuserd: Move Process ops out of Worker.
  snapuserd: Move more fields out of Worker.
  snapuserd: Split more methods out of Worker.
  snapuserd: Create a ReadWorker class.
  snapuserd: Create a MergeWorker class.
2023-07-17 18:44:46 +00:00
David Anderson
f489314dec Merge changes from topic "lpdump-snapshots" into main
* changes:
  Remove local_include_dirs from fs_mgr_defaults.
  Allow processes in the "system" group to read /metadata/ota.
2023-07-17 16:55:14 +00:00
Treehugger Robot
de5ba5c43a Merge "libsnapshot: GetReminingTime not required after mapping dm-user" into main 2023-07-17 16:52:51 +00:00
Akilesh Kailash
74c55e94c9 libsnapshot: GetReminingTime not required after mapping dm-user
After MapDmUserCow, there is no further calls in this function. Snapshot
is mapped; Hence, checking remaining time is not required as
it may return false if the time was exceeded. This would mean function returning false even though snapshots were successfully mapped.

Bug: 291288998
Test: OTA

Change-Id: I4fd8fa6ef90a48885392297f217361507268ba51
Signed-off-by: Akilesh Kailash <akailash@google.com>
2023-07-15 05:44:05 +00:00
Eric Biggers
6044e8e444 Merge "init.rc: start update_verifier instead of update_verifier_nonencrypted" into main 2023-07-14 20:32:30 +00:00
Andrew Walbran
dcac88ceac Merge "Add safety comments." into main 2023-07-14 19:34:11 +00:00
David Anderson
07339000d4 Remove local_include_dirs from fs_mgr_defaults.
fs_mgr_defaults gets used in lots of places, propagate
local_include_dirs to where it's needed so it doesn't cause problems on
modules that don't have an "includes" directory.

Bug: 291083311
Test: builds
Change-Id: Ief2f6362f584b99d3e2dc7dc1b46450609ad189d
2023-07-14 12:30:35 -07:00
David Anderson
f572053dab Merge "libsnapshot: Move export_include_dirs to correct location." into main 2023-07-14 17:12:52 +00:00
David Anderson
a71faae86f Allow processes in the "system" group to read /metadata/ota.
lpdumpd runs as "system", not "root". Adjust the DAC permissions of
/metadata/ota so it can call SnapshotManager::Dump.

Bug: 291083311
Test: lpdump
Change-Id: I97fd7eb2055cf6d31fd42f1021e2f99edbdb838a
2023-07-14 09:01:05 -07:00
David Anderson
edeae250c9 libsnapshot: Move export_include_dirs to correct location.
Bug: 291083311
Test: builds
Change-Id: I7a547bb0b2c06f312f83a6b8659a4e30e97c8438
2023-07-14 09:01:04 -07:00
David Anderson
3d0a332ff5 Merge "fs_mgr: Split fs_mgr_overlayfs into two separate files." into main 2023-07-14 15:56:15 +00:00
Kiyoung Kim
e7abd861da Merge "Enable ABI dump for libcutils" into main 2023-07-14 00:57:02 +00:00
David Anderson
bb235ebf48 Merge "snapuserd: Fix bool/int return value mismatches." into main 2023-07-13 21:17:30 +00:00
David Anderson
f8f3b627ae fs_mgr: Split fs_mgr_overlayfs into two separate files.
One of the paint points for fs_mgr_overlayfs is that mounting and
scratch management code are somewhat unrelated but very intertwined in
the same file.

Split it into two files: fs_mgr_overlayfs_mount, which is only for
mount-related code, and fs_mgr_overlayfs_control, which is only for
setup/teardown code.

The code removed from fs_mgr_overlayfs_control.cpp is code that moved
to fs_mgr_overlayfs_mount.cpp.

This converts a bunch of functions to "static", and splits
fs_mgr_priv_overlayfs.h into two new files (fs_mgr_overlayfs_mount.h and
fs_mgr_overlayfs_control.h).

Bug: N/A
Test: remount
Change-Id: I83da43652b4787f344da75a1d30177df1d7f63b2
2023-07-13 13:01:20 -07:00
Eric Miao
e91538a6de Merge "libutils: Improve performance of utf8_to_utf16/utf16_to_utf8" into main 2023-07-13 16:46:44 +00:00
David Anderson
8bc625e9ca snapuserd: Move Process ops out of Worker.
These are so small they can be inlined into MergeWorker. Sharing these
methods will be difficult after decoupling from dm-user, since
acquisition of buffers will change.

Bug: 288273605
Test: snapuserd_test
Change-Id: I1625d1a6e55bcb2041f73453ca15a01f98263e8a
2023-07-13 08:46:41 -07:00
David Anderson
77280caed1 snapuserd: Fix bool/int return value mismatches.
Bug: 288273605
Test: snapuserd_test, ota
Change-Id: I06d641d3255711dd51583ea9bd9fd6a1acdc7bb2
2023-07-13 08:46:41 -07:00
David Anderson
c2d5a19d26 snapuserd: Move more fields out of Worker.
These fields are specific to either ReadWorker or MergeWorker, but not
both.

Bug: 288273605
Test: snapuserd_test
Change-Id: I2db9cfa2a8f034249879517bd90a40babe97bc64
2023-07-13 08:46:41 -07:00
David Anderson
ab57c8a577 snapuserd: Split more methods out of Worker.
This moves ReadWorker-specific methods out of Worker, and moves
remaining Worker methods into a separate worker.cpp file.

Bug: 288273605
Test: snapuserd_test
Change-Id: I59c31318e127db61a5f3a673956865dac97a6e5f
2023-07-13 08:46:41 -07:00
David Anderson
d967591434 snapuserd: Create a ReadWorker class.
This splits the dm-user specific parts of Worker into a derived class.

Bug: 288273605
Test: snapuserd_test
Change-Id: Ic0ed1a8dff30018fa8466e7dc6e92469f1c87579
2023-07-13 08:46:41 -07:00
David Anderson
c28150f56f snapuserd: Create a MergeWorker class.
Merge threads and read threads share some common state but not much.
Splitting into separate classes will help isolate dm-user specific code.

Bug: 288273605
Test: snapuserd_test
Change-Id: I612374bb0072b1eedf32c30270913dbe907cc6ab
2023-07-13 08:46:41 -07:00
Treehugger Robot
ed2618bc98 Merge "Workaround for Global ThinLTO" into main 2023-07-13 09:33:16 +00:00
Jooyung Han
54b86e7b7a Workaround for Global ThinLTO
Global string literals are not initialized correctly with the new
config.

This change is a workaround by changing them into plain C literals until
we have a better solution.

Bug: 291033685
Test: adb-remount-test.sh
Change-Id: I178286133f55ff5dc11030fa132a9e6db0747ae7
2023-07-13 17:36:27 +09:00
Kiyoung Kim
a4648c2be0 Enable ABI dump for libcutils
Enable ABI dump for libcutils, so ABI can be stabilized from any update
after official release.

Bug: 254141417
Test: abidiff intermediates found from libcutils.vendor build
Change-Id: Ic27c82b908b7836c7bc538a24202ed8adba4d048
2023-07-13 07:23:39 +00:00
Treehugger Robot
30718e7431 Merge "libprocessgroup: Remove isolated UID cgroup directories" into main 2023-07-12 23:45:28 +00:00
Eric Miao
cb199b4795 libutils: Improve performance of utf8_to_utf16/utf16_to_utf8
This CL improves the performance of below functions in helping with conversion
between utf8/utf16 with libutils:

  - utf8_to_utf16_length
  - utf8_to_utf16
  - utf16_to_utf8_length
  - utf16_to_utf

The basic idea is to keep the loop as tight as possible for the most
common cases, e.g. in UTF16-->UTF8 case, the most common case is
when the character is < 0x80 (ASCII), next is when it's < 0x0800 (
most Latin), and so on.

This version of implementation reduces the number of instructions
needed for every incoming utf-8 bytes in the original implementation
where:

  1) calculating how many bytes needed given a leading UTF-8 byte
     in utf8_codepoint_len(), it's a very clever way but involves
     multiple instructions to calculate regardless

  2) and an intermediate conversion to utf32, and then to utf16
     utf8_to_utf32_codepoint()

The end result is about ~1.5x throughput improvement.

Benchmark results on redfin (64bit) before the change:

utf8_to_utf16_length: bytes_per_second=307.556M/s
utf8_to_utf16:        bytes_per_second=246.664M/s
utf16_to_utf8_length: bytes_per_second=482.241M/s
utf16_to_utf8:        bytes_per_second=351.376M/s

After the change:

utf8_to_utf16_length: bytes_per_second=544.022M/s
utf8_to_utf16:        bytes_per_second=471.135M/s
utf16_to_utf8_length: bytes_per_second=685.381M/s
utf16_to_utf8:        bytes_per_second=580.004M/s

Ideas for future improvement could include alignment handling and loop
unrolling to increase throughput more.

This CL also fixes issues below:

  1. utf16_to_utf8_length() should return 0 when the source string has
     length of 0, the original code returns -1 as below:

    ssize_t utf16_to_utf8_length(const char16_t *src, size_t src_len)
    {
        if (src == nullptr || src_len == 0) {
            return -1;
        }
	...

  2. utf8_to_utf16() should check whether input string is valid.

Change-Id: I546138a7a8050681a524eabce9864219fc44f48e
2023-07-12 13:23:07 -07:00
T.J. Mercier
9c8c748c11 libprocessgroup: Remove isolated UID cgroup directories
Global UID level cgroup removal was eliminated because of a race
between app launch and app killing using the same directory name. [1]

However isolated app UIDs are assigned sequentially, and are
basically never reused until we wrap around the large range of
isolated UIDs. This leaves thousands of isolated cgroup directories
unused, which consumes kernel memory and increases memory reclaim
overhead. Remove this subset of UID level cgroup directories when
killing process groups.

[1] d0464b0c01

Test: 50 cycle ACT leaves 1000 fewer empty isolated cgroups
Bug: 290953668
Change-Id: If7d2a7b8eec14561a72208049b74ff785ca961bd
2023-07-12 18:29:21 +00:00