Commit graph

81979 commits

Author SHA1 Message Date
Daniel Zheng
03acfdbeb6 Optimize zstd compression
reuse the same context for zstd compression. One context should be used
per compression thread for optimal performance. Discussion and
reccomendations can be found at go/android-hw-compression-vendor

Results (level 3 compression)
full ota on raven with optimizations:  1191.304 seconds
without optimizations:  1461.854 seconds

compression ratio remains unchanged and merge time difference are
negligible ~1% delta (probably just noise)

Test: ota_from_target_files, update_device.py
Change-Id: I3feede9f1f119874e369c54b29c594475fbf7376
2023-08-14 16:07:33 -07:00
Daniel Zheng
de6e446efa Adding compressor class
Adding in compressor class to clean up code for cow_compress.cpp.
Since we are making some api changes (to zstd) that are unique to each
compression methods, these should be implementation details should be
hidden to the parent class

Test: m libsnapshot

Change-Id: I9194e2c615aefed078458f525382253228bc1b69
2023-08-14 16:07:26 -07:00
Daniel Zheng
90d4f89e8b Removing extra string creation for log
Removin unneccessary string from being created in logging

Test: m libsnapshot
Change-Id: I123b6a10f349dfb0c275e630b5f02d1cec8c2687
2023-08-14 15:16:36 -07:00
Eric Biggers
87d6c8a1d5 Merge "Avoid two SELinux related error messages at boot time" into main 2023-08-14 17:33:15 +00:00
Jooyung Han
5a2d8514de Merge "Dedup apex-scanning" into main 2023-08-14 08:48:17 +00:00
Tomasz Wasilczyk
6bfa6a0231 Merge "Migrate String8/16 from .string() to c_str()" into main 2023-08-11 22:56:43 +00:00
Tomasz Wasilczyk
067dbcc110 Merge "Add String16::c_str and start using it." into main 2023-08-11 21:39:31 +00:00
Eric Biggers
141c6a886c Avoid two SELinux related error messages at boot time
It is expected that /metadata/ota/rollback-indicator and /metadata/gsi
don't always exist, so don't call selinux_android_restorecon() on them
when they don't exist.  This eliminates the following error messages:

0     0 E selinux : SELinux: Could not get canonical path for /metadata/ota/rollback-indicator restorecon: No such file or directory.
0     0 E selinux : SELinux:  Could not stat /metadata/gsi: No such file or directory.

Test: Booted Cuttlefish and verified the error messages are gone
Change-Id: I94c998556c85adde5f11f134178219ba7880c2be
2023-08-11 21:12:16 +00:00
Tomasz Wasilczyk
18b746188c Migrate String8/16 from .string() to c_str()
This will align usage with std::string.

Bug: 295394788
Test: make checkbuild
Change-Id: Ic5c215b011197950dcbcb0339f44cc68fefc65a1
2023-08-11 19:53:20 +00:00
Tomasz Wasilczyk
90af4157c5 Add String16::c_str and start using it.
This is for parity with String8::c_str and in general the first step
in transition from String8|16.string() - to make it more similar to
std::string.

Bug: 295394788
Test: mma -j
Change-Id: I6c1411bef07c761fb2fb3fb38c27801ac4cffc57
2023-08-11 19:40:34 +00:00
Wei Li
55aed3f908 Merge "Track root directory symlinks created by LOCAL_POST_INSTALL_CMD in ALL_ROOTDIR_SYMLINKS and add them to ALL_DEFAULT_INSTALLED_MODULES, so they can be included in product SBOMs properly." into main 2023-08-11 18:38:12 +00:00
David Drysdale
2f6bd6fa43 Merge "KeyMint HAL: process destroyAttestationIds" into main 2023-08-11 09:26:12 +00:00
Jooyung Han
2982f09d57 Dedup apex-scanning
create_apex_data_dirs() now uses GetApexListFrom(dir) in
apex_init_util.cpp.

This is essentially a refactoring, but there are a few behavioral
changes, which I think make more sense.
- Emits no error when opendir(/apex) fails.
- Emits errors when mkdir fails for each apex.
- Does not abort `perform_apex_config` even though create_apex_data_dirs
  fails.

Bug: 293546778
Test: check /data/misc/apexdata/ after boot
Change-Id: I9d1a9710a6a626eec087c8c0fb1768667ffb036e
2023-08-11 14:14:37 +09:00
Treehugger Robot
236dbc30fc Merge changes from topic "read-bootstrap-apex" into main
* changes:
  Skip bootstrap APEX RC files for the second round
  Read .rc files from bootstrap apexes
2023-08-11 04:48:28 +00:00
Wei Li
d8a270698d Track root directory symlinks created by LOCAL_POST_INSTALL_CMD in
ALL_ROOTDIR_SYMLINKS and add them to ALL_DEFAULT_INSTALLED_MODULES,
so they can be included in product SBOMs properly.

Bug: 272358980
Test: CIs and build/soong/tests/sbom_test.sh
Change-Id: I73dfb34156d681786c013912e59a0d0c0c48ecc7
2023-08-10 18:18:36 -07:00
Jooyung Han
55ef3d6104 Skip bootstrap APEX RC files for the second round
Reading .rc files from bootstrap APEXes causes "double loading".
This works for services because init just ignores duplicates. But it
emits error logs, which can mislead even though there's no actual
errors. Besides, for actions, duplicates can cause a problem when
commands are not idempotent.

So, when loading RC files from APEXes for the second time, we'd better
skip those bootstrap APEXes.

Bug: 290148081
Test: VendorApexHostTestCases
Change-Id: Ia630dbd14046064b5e5c612c01ebacf57091c8d4
2023-08-11 10:02:08 +09:00
Christopher Ferris
9f32fba6d3 Merge "Fix potential miscellaneous debuggerd issues." into main 2023-08-10 19:53:19 +00:00
Treehugger Robot
acdc4848e0 Merge "libsnapshot: Check for valid snapshots based on current slot" into main 2023-08-10 17:29:46 +00:00
Akilesh Kailash
3983f9aa6e libsnapshot: Check for valid snapshots based on current slot
We may have snapshot files in /metadata/ota/snapshot/ which ends with
.tmp such as system_a.tmp - This happens if the device
reboots just before `rename` in `WriteStringToFileAtomic`. This
can lead to spurious merge failures.

Log the error and skip these snapshot files. It is ok to skip
as we will still have original snapshot status files since
we are already in the merge path. Additionally, try to remove
these files when snapshot is deleted.

Bug: 292198189
Test: OTA
Change-Id: I5db3dbd5a919b263ae577185de3e7f79a5e9b89a
Signed-off-by: Akilesh Kailash <akailash@google.com>
2023-08-10 09:36:35 -07:00
Tomasz Wasilczyk
d3550e3314 Remove cutils threads.h header completely.
Bug: 289414897
Test: it builds
Change-Id: Ifcde5849923f19f1c898a6a95811eea55be0d276
2023-08-10 15:12:05 +00:00
Po-yao Chang
6fe9367f7b Run boringssl tests according to ro.zygote
If ro.zygote is zygote64, don't bother running 32-bit test.
Otherwise abilist{32,64} decides what tests to run.

Bug: 291874369
Test: make gsi_arm64-user; Check /system/etc/init/hw
Change-Id: Id10b2242606d6400acc29c3174f713581d6cce2e
2023-08-10 17:12:47 +08:00
Jooyung Han
5c4217cf6e Read .rc files from bootstrap apexes
To start an early_hal service from a bootstrap vendor apex, init now
reads .rc files from bootstrap apexes as well.

In this change, perform_apex_config command is re-purposed to support
bootstrap mode. Now we have some similarity between two apexd calls:

- for bootstrap apexes (in the bootstrap mount namespace):

  exec_start apexd-bootstrap
  perform_apex_config --bootstrap

- for normal apexes (in the default mount namespace):

  restart apexd
  ...
  wait_for_prop apexd.status activated
  perform_apex_config

Note that some tasks in perform_apex_config are not needed in the
bootstrap.  For example, we don't need to create apexdata directories
for bootstrap apexes.

Bug: 290148081
Test: VendorApexHostTestCases
Change-Id: I8f683a4dcd7cd9a2466a4b1b417d84c025c37761
2023-08-10 15:40:06 +09:00
Treehugger Robot
0c0dafae8d Merge "Remove gettid symbol from cutils" into main 2023-08-10 04:13:24 +00:00
Jooyung Han
370ee1bbda Merge "Use /bootstrap-apex for bootstrap APEXes" into main 2023-08-10 02:38:30 +00:00
Christopher Ferris
3a0833c9cd Fix potential miscellaneous debuggerd issues.
Check for the log opening failing.

Add the ability to put error messages in the log and tombstone so
that it's clear if the log reading failed in some way.

Adjust test so that if there is a log or if no log exists, the test
will still pass.

Print an <unknown> if the command line is unreadable instead of nothing.

Test: Ran unit tests.
Test: Induced error and verified error message is save in tombstone.
Change-Id: I2fce8078573b40b9fed3cd453235f3824cadb5e3
2023-08-09 17:31:55 -07:00
Treehugger Robot
cf32fd18d4 Merge "debuggerd: fix passing of fdsan_table to crash_dump" into main 2023-08-09 23:15:06 +00:00
Tomislav Novak
24a7df5d38 debuggerd: fix passing of fdsan_table to crash_dump
Commit aosp/1259140 moved fdsan_table into debugger_process_info, which
is populated conditionally. This introduced a bug where the process that
receives BIONIC_SIGNAL_DEBUGGER (35) does not propagate the fdsan_table
pointer to crash_dump:

  $ adb shell kill -SIG35 <pid>
  $ adb logcat -s DEBUG
  E DEBUG   : failed to read fdsan table entry 0: I/O error

Fdsan in warn-only mode uses BIONIC_SIGNAL_DEBUGGER[1], so the generated
tombstones don't have any fd ownership info.

Fix it by calling get_process_info() irrespective of the signal being
handled, taking care to preserve the previous behavior of not showing
abort messages set by applications in non-fatal dumps.

Test: debuggerd_test
Test: send SIG35 to arbitrary process and inspect the log and tombstone
Test: crasher fdsan_file

[1] 20ad9129e7/libc/bionic/fdsan.cpp (166)

Change-Id: I76931ca4825e846fc99f26fa590c045130abb850
2023-08-09 15:19:51 -07:00
Keith Mok
eb2165079a Merge "Adding force_flash functionality" into main 2023-08-09 20:45:24 +00:00
Daniel Zheng
86f037c245 Merge "decompressor maintenence" into main 2023-08-09 20:04:07 +00:00
Daniel Zheng
dc3c5292c7 Merge "Moving includes" into main 2023-08-09 20:04:00 +00:00
Daniel Zheng
73846d68cf Merge "libsnapshot: take in compression_level" into main 2023-08-09 20:03:44 +00:00
Daniel Zheng
720007ba43 Merge "Adding flag to flash only static partitions" into main 2023-08-09 18:09:00 +00:00
Daniel Zheng
376b70423d Adding force_flash functionality
aosp/2452605 might've accidentally omitted force flash functionality
from fastboot. Some users need to flash dynamic partitions from
bootloader, so we are readding this feature back into fastboot

Test: adb reboot bootloader, fastboot flash system --force

Change-Id: I37a51ebbe38b27ed428c38068d30c0febd7db09c
2023-08-09 10:54:14 -07:00
Treehugger Robot
a981d58972 Merge "Remove sub-includes from libcutils/threads.h" into main 2023-08-09 16:43:32 +00:00
David Drysdale
cedcb4cd92 KeyMint HAL: process destroyAttestationIds
Bugs: 283049310
Test: VtsAidlKeyMintTargetTest using disabled test
Change-Id: I545511a7a701faf8e82dff9c169c3738f2c30e6d
2023-08-09 11:32:52 +01:00
Jooyung Han
566c65239f Use /bootstrap-apex for bootstrap APEXes
This new directory is bind-mounted to /apex in the bootstrap mount
namespace so that apexd-bootstrap mounts bootstrap APEXes there via
/apex.

The directory is shared between two mount namespaces, hence visible
in the default mount namespace.

Bug: 290148078
Test: VendorApexHostTestCases
Change-Id: I841480e41be8def5a4c6a4aa874c4e21465a71d3
2023-08-09 17:27:39 +09:00
Treehugger Robot
790c6315c3 Merge "snapuserd: Remove host_supported from snapuserd_test until it passes." into main 2023-08-09 00:44:39 +00:00
Treehugger Robot
8f5fab42df Merge "crasher: add pac and bti crashes." into main 2023-08-08 23:47:21 +00:00
David Anderson
8665b5f9df snapuserd: Remove host_supported from snapuserd_test until it passes.
Bug: 295074841
Test: snapuserd_test
Change-Id: Ie51263ec6971f9806bcfa2b522e5ed24da5d65cf
2023-08-08 16:34:19 -07:00
Daniel Zheng
5061a1cef8 Merge "Adding struct to hold compresion parameters" into main 2023-08-08 23:08:03 +00:00
Elliott Hughes
1b13b14758 crasher: add pac and bti crashes.
Also add the missing `.size` directives to all the assembler functions
for slightly improved backtraces.

Test: crasher64 pac; crasher64 bti
Change-Id: I8e0c127cbff56c33637e6ca8f1d927b971951807
2023-08-08 16:06:24 -07:00
David Anderson
7bd9162b14 Merge changes Ic18443d4,I9e9af999,I10e099fe,I00cf6ec9,Ia1ea4ba7 into main
* changes:
  snapuserd: Reduce size of COWs in tests.
  snapuserd: Add unit tests for ReadWorker::ReadAlignedSector.
  snapuesrd: Build snapuserd_test on host.
  snapuserd: Add an ITestHarness implementation for hosts.
  snapuserd: Don't specify a path to TemporaryFile.
2023-08-08 22:11:35 +00:00
Daniel Zheng
ad5df32560 decompressor maintenence
cleaning up a bit of code for consistency + adding in zstd to
FromString() function

Test: m libsnapshot
Change-Id: Ic8b7243c5a5ee25326a46e944c13578136f27078
2023-08-08 13:49:27 -07:00
David Anderson
0ff0eeed77 libfs_avb: Allow overriding the slot suffix.
There is no easy way to override fs_mgr_get_slot_suffix for testing or
for simulating slot switches. It relies on global properties, and
since fstab is usually a static library, linked multiple times, we can't
just stick a global variable in fs_mgr_slotselect.

libfs_avb already takes slot suffix arguments in various places. This CL
extends that by adding arguments to AvbHandle and FsManagerAvbOps.

Bug: N/A
Test: snapshotctl
Change-Id: Ide7f052b38249d6bc79a6a2c124baf8ca08bcc3b
2023-08-08 13:01:20 -07:00
David Anderson
9c525ff0ed Merge changes Ib33dc593,I6550682f,Ic0681cbf,Iaaf96a37,I89b15492 into main
* changes:
  snapuserd: Factor a base class out of tests.
  snapuserd: Add error propagation to tests.
  snapuserd: Add diagnostics for debugging races.
  snapuserd: Fix race condition in HandleManager shutdown.
  snapuserd: Remove test dependence on LoopDevice.
2023-08-08 19:46:53 +00:00
Daniel Zheng
fe6c1d55f7 Moving includes
Removing unused includes + moving includes to file that uses them

Test: m libsnapshot
Change-Id: Iacb1d1d0d17cf9059583443e9a17fdc1007bc399
2023-08-08 12:29:14 -07:00
Daniel Zheng
50460fa3ac libsnapshot: take in compression_level
Changing libsnapshot to have configurable compression_level

Test: ota_from_target_files
Change-Id: I4c050a2c83cc0fa9079f94dd2d9009f291358740
2023-08-08 12:29:12 -07:00
David Anderson
1333d87bfe snapuserd: Reduce size of COWs in tests.
This greatly reduces the runtime of snapuserd_test, from 4 minutes to
about 1.5 minutes.

Bug: 269361087
Test: snapuserd_test
Change-Id: Ic18443d4e4318f6ef6adadeabd47ccceeeb19b6f
2023-08-08 10:41:56 -07:00
David Anderson
dbda1300ae snapuserd: Factor a base class out of tests.
This factors a lighter weight class out of SnapuserdTest, so we can
construct tests without needing a HandlerManager/daemon.

Bug: 288273605
Test: snapuserd_test
Change-Id: Ib33dc593a5b3d3df86853e73f245918ef36fd9fa
2023-08-08 10:41:56 -07:00
David Anderson
6aec095d68 snapuserd: Add unit tests for ReadWorker::ReadAlignedSector.
These tests are for real bugs that were previously not testable.

Bug: 288273605
Test: snapuserd_test
Change-Id: I9e9af999e4f5f988f4538750eba109f6b2fe448c
2023-08-08 10:41:56 -07:00