Commit graph

88578 commits

Author SHA1 Message Date
Jeffrey Vander Stoep
7f66218e1d Merge "Replace use of deprecated logging functions" into main am: 40b6a8f0c6
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2952670

Change-Id: I215c94fd01b4d814598f9f4f75851e32c9188a6f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-02-06 16:39:21 +00:00
Jeffrey Vander Stoep
40b6a8f0c6 Merge "Replace use of deprecated logging functions" into main 2024-02-06 16:09:30 +00:00
Jeff Vander Stoep
56aedd5551 Replace use of deprecated logging functions
This is needed to upgrade the android_logger crate from 0.12.0
to 0.13.3.

with_max_level provides the same functionality as with_min_level.
The renaming is admittedly confusing, but the new name is accurate
and it makes sense that they deprecated and then removed the
previously poorly named with_min_level.

See crate documentation [1] and code [2].

[1]: https://docs.rs/android_logger/0.12.0/android_logger/struct.Config.html#method.with_min_level
[2]: https://docs.rs/android_logger/0.12.0/src/android_logger/lib.rs.html#227

Bug: 322718401
Test: build and run CF with the change.
Test: m aosp_cf_x86_64_phone
Change-Id: Ib4fbd486267d30e74e886139846950b066848d43
2024-02-06 12:32:36 +01:00
David Dai
f3337f69df Merge "Add AID for Virtual Machines" into main am: 161b80bd02
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2945529

Change-Id: I92a8f7a1c8d8fc5e0fe2c0ae39ac0b018e4cb462
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-02-06 01:59:21 +00:00
David Dai
161b80bd02 Merge "Add AID for Virtual Machines" into main 2024-02-06 01:21:39 +00:00
Treehugger Robot
6901675313 Merge "remount: Auto remount option (-R) also tries to gain root" into main am: eadf6b4bda
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2926792

Change-Id: I346ffe88bc6dfc299140a545164c551410e3082b
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-02-06 00:34:00 +00:00
Treehugger Robot
eadf6b4bda Merge "remount: Auto remount option (-R) also tries to gain root" into main 2024-02-05 23:47:54 +00:00
David Dai
851c9c825f Add AID for Virtual Machines
Add a new AID for Virtual Machines so we can grant
capabilities such as CAP_SYS_NICE.

Bug: 322197421
Test: Build and boots, and verified capabilities

Change-Id: Ie893ba8ed6956a554bccfbd00e4e6fe9212ea77d
Signed-off-by: David Dai <davidai@google.com>
2024-02-05 23:38:44 +00:00
Kenn Frankel
548fffe3bd Merge "Update libstatssocket_lazy to support StringArray" into main am: 7833f27c25
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2945025

Change-Id: I4f80a927a3b500fb22c119f6b6a79c1bb61f2e55
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-02-05 19:59:44 +00:00
Kenn Frankel
7833f27c25 Merge "Update libstatssocket_lazy to support StringArray" into main 2024-02-05 19:16:33 +00:00
Treehugger Robot
1015d263e9 Merge "Restore I/O priority support" into main am: 749e2e8e2d
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2944826

Change-Id: I0ae7e0f1ef9a6ccf484e33ed82f7c1fa8b522c06
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-02-02 01:57:41 +00:00
Treehugger Robot
749e2e8e2d Merge "Restore I/O priority support" into main 2024-02-02 01:13:16 +00:00
Kenn Frankel
374c43e9b7 Update libstatssocket_lazy to support StringArray
Bug: 281162849
Test: LibstatssocketLazyTest

Change-Id: I917589e92db045f96daa9b191fadac50b01704c8
2024-02-01 11:49:51 -08:00
Bart Van Assche
0cf102b703 Restore I/O priority support
CL aosp/2929791 removed I/O priority support to prepare for a clean
revert of the CL that migrates the blkio controller from the v1 to the
v2 cgroup hierarchy. Since there was no other reason to revert the I/O
priority CL, restore I/O priority support.

Bug: 186902601
Change-Id: I1a4053140ab55973878bfeacfb546da3c601a895
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2024-02-01 10:52:13 -08:00
Akilesh Kailash
ae4b84db9a Merge changes from topic "ota-block-size-compression" into main am: 942b213628
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2901263

Change-Id: I003f2970db9ec087df6ab8c8744da6c3429f0952
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-02-01 18:50:34 +00:00
Akilesh Kailash
942b213628 Merge changes from topic "ota-block-size-compression" into main
* changes:
  snapuserd: Add I/O path support for variable block size
  libsnapshot_cow: Support multi-block compression
2024-02-01 18:13:02 +00:00
Akilesh Kailash
60a9e75880 Merge "libsnapshot: set header max_compression" into main am: 13cb6f7b4b
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2938301

Change-Id: I46f55b6a939e88de863f89707dc0a378f23b585a
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-31 21:41:18 +00:00
Akilesh Kailash
3ea911bc06 snapuserd: Add I/O path support for variable block size
The flow of I/O path is as follows:

1: When there is a I/O request for a given sector, we first
check the in-memory COW operation mapping for that sector.

2: If the mapping of sector to COW operation is found, then the
existing I/O path will work seamlessly. Even if the COW operation
encodes multiple blocks, we will discard the remaining data.

3: If the mapping of sector to COW operation is not found:
    a: Find the previous COW operation as the vector has sorted sectors.

    b: If the previous COW operation is a REPLACE op:
        i: Check if the current sector is encoded in the previous COW
	operations compressed block.

	ii: If the sector falls within the range of compressed blocks,
	retrieve the block offset.

	iii: De-compress the COW operation based on the compression
	factor.

	iv: memcpy the data based on the block offset.

	v: cache the COW operation pointer as subsequent I/O requests
	are sequential and can just be a memcpy at the correct offset.
    c: If the previous COW operation is not a REPLACE op or if the
       requested sector does not fall within the compression factor
       of the previous COW operation, then fallback and read the data
       from base device.

Snapshot-merge:

During merge of REPLACE ops, read the entire op in one shot, de-compress
multiple blocks and write all the blocks in one shot.

Performance:

go/variable-block-vabc-perf covers detail performance runs
on Pixel 6 for full and incremental OTA.

Bug: 319309466

Test: snapuserd_test covers all the I/O path with various block sizes.
About 252 cases with all combinations and tunables.

[==========] 252 tests from 4 test suites ran. (702565 ms total)
[  PASSED  ] 252 tests.

On Pixel 6:

=======================================
COW Writer V3:

for i in full, incremental OTA
   for j in 4k, 16k, 32k, 64k, 128, 256k
      for k in lz4, zstd, gz
	 install OTA, reboot, verify merge
=======================================
COW Writer V2:

for i in full, incremental OTA
  for j in 4k
    for k in lz4, zstd, gz
      install OTA, reboot, verity merge

=====================================

Change-Id: I4c3b5c3efa0d09677568b4396cc53db0e74e7c99
Signed-off-by: Akilesh Kailash <akailash@google.com>
2024-01-31 13:28:45 -08:00
Akilesh Kailash
59fa486703 libsnapshot_cow: Support multi-block compression
This patch supports compression for bigger block size.

3 bits [57-59] in the COW Operation "source_info_" field is used to store
the compression factor. Supported compression factors are power of 2
viz: 4k, 8k, 16k, 32k, 64k, 128k, 256k.

Only REPLACE operations will have the bigger block size support for now.
This can be extended to other operations later.

The write path in EmitBlocks() has the core logic wherein consecutive
sequence of REPLACE ops are compressed based on the compression factor
settings. Thus, for a 64k compression factor, there will be just one
COW operation which encodes all the 16 operation and the entire 64k
block is compressed in one shot.

NOTE: There is no read I/O path support in this patch. Subsequent patch
will have the read support.

Performance data (with read I/O path support in subsequent patch):

go/variable-block-vabc-perf covers detail performance runs
on Pixel 6 for full and incremental OTA.

TL;DR:

Performance of a full OTA (All numbers are compared against 4k block
size)
=======================================
Snapshot-size:

~10-11% decrease in snapshot-size (disk-space) for zstd with 256k block
size.

~8% decrease in snapshot-size (disk-space) for lz4

Install time:

~13% decrease in OTA install time for zstd with 256k block size.

Snapshot-merge:

~50% decrease in snapshot-merge time with 256k block size for zstd

Post OTA boot-time:

~10.5 decrease in boot time for 64k block size for zstd

In-memory footprint for COW operations:

~80% decrease in memory footprint for 256k block size. (58MB -> 9.2MB)

============================================

For more improvements, further tuning of zstd/lz4 is
required primariy the compression levels, zstd compression window,
performance of gz with compression levels.

Bug: 319309466

Test: cow_api test covering all the supported block sizes for v3 writer.

On Pixel 6:

=======================================
COW Writer V3:

for OTA in full, incremental OTA
   for block_size in 4k, 16k, 32k, 64k, 128k, 256k
       for compression_algo in lz4, zstd, gz, none
          install OTA, reboot, verify merge
=======================================
COW Writer V2:

for OTA in full, incremental OTA
   for block_size in 4k
      for compression_algo in lz4, zstd, gz, none
          install OTA, reboot, verity merge

=====================================
Change-Id: I96201f1609582aa9d44d8085852e284b0c4a426d
Signed-off-by: Akilesh Kailash <akailash@google.com>
2024-01-31 12:52:31 -08:00
Akilesh Kailash
13cb6f7b4b Merge "libsnapshot: set header max_compression" into main 2024-01-31 20:42:58 +00:00
Daniel Zheng
bcce91603b libsnapshot: set header max_compression
Intermediate CL needed before variable block size can land. Since v3 is
enabled on cuttlefish, the base build needs to write the
compression_factor in order for reader to properly parse. Otherwise
we'll fail OTA test

Test: th
Change-Id: Ia353aae8e668858851073f09308909ae70d7854e
2024-01-31 17:35:00 +00:00
David Drysdale
6c2cb28318 Merge "Secretkeeper: require SECRETKEEPER_ENABLED=true" into main am: 700ff706f1
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2937199

Change-Id: I45d94c65b841765ad81ad34c1ac687441c2c25e6
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-31 08:00:03 +00:00
David Drysdale
700ff706f1 Merge "Secretkeeper: require SECRETKEEPER_ENABLED=true" into main 2024-01-31 07:13:24 +00:00
Jiyong Park
ac6907b466 Merge "Drop CtsInitTestCases from cts" into main am: 0edd5b860a
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2935210

Change-Id: I3c5f869e67c32498c57e99ccdb1784b8036ccde9
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-31 01:31:31 +00:00
Jiyong Park
0edd5b860a Merge "Drop CtsInitTestCases from cts" into main 2024-01-31 00:58:50 +00:00
Xin Li
4090c94e73 Merge "Merge Android 24Q1 Release (ab/11220357)" into aosp-main-future 2024-01-30 22:43:43 +00:00
Akilesh Kailash
c9d7942e5d Merge "libsnapshot: update type" into main am: 14258369ef
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2903390

Change-Id: Ibd57839d562924067c254e160825dcb664f022e5
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-30 19:34:28 +00:00
Daniel Zheng
874d5006a2 Merge "libsnapshot: op_count_max default to max blocks" into main am: 5e4b3263ab
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2898987

Change-Id: I9811519fa263dd64efb9a548446a107db46e00af
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-30 19:33:53 +00:00
Daniel Zheng
2aad9fcba2 Merge "libsnapshot: use compression_factor in ota" into main am: 560c753ee8
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2901503

Change-Id: I6f2f7b9efe981dcdd9c1caa371859ba547b46fcc
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-30 19:33:34 +00:00
Akilesh Kailash
14258369ef Merge "libsnapshot: update type" into main 2024-01-30 19:23:49 +00:00
Daniel Zheng
5e4b3263ab Merge "libsnapshot: op_count_max default to max blocks" into main 2024-01-30 18:49:14 +00:00
Daniel Zheng
560c753ee8 Merge "libsnapshot: use compression_factor in ota" into main 2024-01-30 18:48:58 +00:00
Daniel Zheng
6ca4d66a05 libsnapshot: update type
num_compress_threads should be unsigned integer as it can never be
negative.

Test: th
Change-Id: Ic0456ac717483300fa9cbd55eba5cdd0156207a7
2024-01-30 09:25:23 -08:00
Daniel Zheng
2aed5a5e4c libsnapshot: op_count_max default to max blocks
In the case that op_count_max is read in as zero, we should use the
upper bound of max blocks as the estimation. One case in which this
error can happen is if a v2 cow estimator is used, we should still be
able to run an OTA if we upper bound our ops buffer size estimation.

Test: th
Change-Id: I97ca66368d6631bf43c8911ed66f99c9e8096e2d
2024-01-30 09:25:15 -08:00
Daniel Zheng
2c82c81f12 libsnapshot: use compression_factor in ota
Parse manifest compression_factor and set CowOptions appropriately. This
allows v3 writer to use compression factor in OTA. Updating some
comments about supported compression algorithms

Test: th
Change-Id: I88f254087e536d9e5925064f85317f0acce280ee
2024-01-30 09:25:02 -08:00
David Drysdale
073133811e Secretkeeper: require SECRETKEEPER_ENABLED=true
Make the makefile safer by requiring a specific value for the
environment variable that turns on Secretkeeper

Bug: 306364873
Test: TreeHugger
Change-Id: Ic5bb5e7411a19941f58ec8c973104c1e53f3834f
2024-01-30 16:49:13 +00:00
Xin Li
7f41fd4852 Merge Android 24Q1 Release (ab/11220357)
Bug: 319669529
Merged-In: I856c812d22363cc1d1e8aa88706d4d3b89044f52
Change-Id: I9ef2aa7496bbf81f7f0da6b0ea46dc2157ad91f1
2024-01-29 21:19:57 -08:00
Jiyong Park
75070b37e2 Drop CtsInitTestCases from cts
The test is not eligible for CTS. Reasons:

1. The init behavior does not directly affect app compat. App interact
   with init only for the property service and that part is covered by
the Bionic test already.

2. This test doesn't run against the init binary installed on the
   device. libinit where most of the init functionalities are
implemented is statically linked to this test binary. In other words,
this test is closer to a unit test for init.

3. This test is not compatible with Trunk stable where test and DUT are
   built in different branches. The test depends on several (private)
libraries like libbase and libutils. Since the interfaces of the
libraries may have changed in the main branch, the test binary built
from the old test-dev branch may break.

This change does not remove the test. The test will still run as a unit
test during pre/post submit.

I didn't drop the `Cts` prefix from the name, because that requires
broader changes.

Bug: 320800872
Test: N/A
Change-Id: I1402c08b79b57ad6daa7948fe37f14fbbe36f1d6
2024-01-30 10:24:10 +09:00
Daniel Zheng
f47187745e Merge "libsnapshot: remove op count check" into main am: 81f49ed2e7
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2903467

Change-Id: I78afa5f5f8a039ed50dee9dbb527cc93bc4f3911
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-30 01:09:47 +00:00
Daniel Zheng
0043447ea4 Merge "libsnapshot: v3 structural changes" into main am: a4af286e98
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2933914

Change-Id: I8a362f52f3ae191074117afce8fb99aace62ea3b
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-30 01:09:13 +00:00
Daniel Zheng
81f49ed2e7 Merge "libsnapshot: remove op count check" into main 2024-01-30 00:38:35 +00:00
Daniel Zheng
a4af286e98 Merge "libsnapshot: v3 structural changes" into main 2024-01-30 00:38:27 +00:00
Daniel Zheng
d84857fcfc libsnapshot: remove op count check
With variable block size compression being added, the number of ops
written cannot be calculated directly as easily since one op can cover
the data for multiple ops previously. We can get rid of this check for
XOR and Raw blocks as
within WriteOperation() we already make a check to see if we are
exceeding op_count_max limit.

We still need to keep this check for EmitZeroBlocks and EmitCopyBlocks
since the number of operations is determined ahead of time in those
function calls. Without this check in place, the ops will be added to
cached ops and return true when ops cannot be written.
with this change, v3 cow ota now works on cuttlefish with support for
variable block size compression.

Test: th
Change-Id: Ia55f152f5deb67a9022d0feff112345e72741dd3
2024-01-29 14:04:18 -08:00
Daniel Zheng
903f8e07ff libsnapshot: v3 structural changes
Changes to structure of v3 header + operation needed for variable block
size. Seperating this CL from the variable block size one so we can get
v3 enabled on cuttlefish

the op count type changes are so that op count matches the type of
max_blocks. Max_blocks is used when op buffer size is not set -> we
default to upper bound of one operation per block in the partition.

Test: th
Bug: 307452468
Change-Id: I1a2581763a4fd6be5d5795f7e4781023e9984256
2024-01-29 14:04:17 -08:00
T.J. Mercier
3d3b86548a Merge changes I6e95870e,Ibcef6b73,I302ce3c2 into main am: 96b0065ebf
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2928353

Change-Id: I67f0528a3075f1856fbb5fec5ba10fe65e245e76
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-29 21:56:15 +00:00
T.J. Mercier
96b0065ebf Merge changes I6e95870e,Ibcef6b73,I302ce3c2 into main
* changes:
  libprocessgroup: Use pid_t consistently for TIDs
  libprocessgroup: Use pid_t consistently for PIDs
  libprocessgroup: Use uid_t consistently for UIDs
2024-01-29 21:28:12 +00:00
David Drysdale
f99559a09b Merge "Secretkeeper HAL: support large messages" into main am: b077fcce13
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2921586

Change-Id: I3f852f0e71f4c81f838bb51fb0952adde2147961
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-29 15:04:11 +00:00
David Drysdale
b077fcce13 Merge "Secretkeeper HAL: support large messages" into main 2024-01-29 14:09:42 +00:00
Yi-Yo Chiang
bbb9690935 remount: Auto remount option (-R) also tries to gain root
When doing an auto remount & reboot command and is running as the SHELL
uid, just try to gain root on behalf of the user and retry the remount
command.
If "gain root" failed, then print the message to tell the user to run
"adb root" and retry.

Bug: 322285923
Test: adb unroot && adb remount -vR
Test: adb unroot && adb shell remount -vR
Change-Id: If8e04dc602573c73178c108ef4944f0a985b590e
2024-01-29 15:51:52 +08:00
Treehugger Robot
c4c84371e6 Merge changes from topic "b314861914" into main am: 761db93167
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2929790

Change-Id: I1c86687d1b2bb6fe03cef81c020a73e2019ccde1
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-01-26 16:24:33 +00:00