Commit graph

58319 commits

Author SHA1 Message Date
Tianjie
0d05b7a8fa Update libsnapshot to handle partial update
For partial update, the payload will not include all dynamic
partitions on the device, update the logic libsnapshot to better
handle such case.

Bug: 157778739
Test: run a partial OTA, unit tests pass

Change-Id: I4339a81ed31161bab3ba9666c1d05fb8bf57dbf9
2020-06-09 12:53:45 -07:00
Alistair Delva
a3ed46533c Merge changes from topic "default-fstab-swapon_all"
* changes:
  Respect ro.boot.fstab_suffix in swapon_all
  Add documentation for umount_all
2020-06-08 23:41:51 +00:00
Alistair Delva
de28a8651f Respect ro.boot.fstab_suffix in swapon_all
While mount_all and umount_all were updated to use ro.boot.fstab_suffix,
I neglected to update swapon_all. Trivially copied from umount_all.

Bug: 142424832
Change-Id: Icd706fe7a1fe16c687cd2811b0a3158d7d2e224e
Merged-In: Icd706fe7a1fe16c687cd2811b0a3158d7d2e224e
2020-06-08 23:41:40 +00:00
Treehugger Robot
ad3874e025 Merge "Stop using varargs in libkeyutils." 2020-06-08 21:24:18 +00:00
Josh Gao
b99f194d05 Merge "Move libadbd_auth, libadbd_fs to adbd_system_binaries." 2020-06-08 21:00:19 +00:00
Alistair Delva
6c6a36fb31 Add documentation for umount_all
The mount_all and swapon_all commands are documented, but umount_all
is not. Add some documentation.

Bug: 142424832
Change-Id: I7e4dcb4d222b787350a79c9e312062cac9eeb4d8
2020-06-08 10:51:40 -07:00
Treehugger Robot
cbe630dbc6 Merge "Logtags for input flinger" 2020-06-08 17:47:31 +00:00
Paul Lawrence
2709331f90 Merge "Set block size in dm-bow" 2020-06-08 17:07:36 +00:00
Elliott Hughes
44a5f0ed7f Stop using varargs in libkeyutils.
It's error-prone, and our specific usage of it here upsets ubsan.

Bug: https://issuetracker.google.com/158428513
Test: treehugger
Change-Id: I3a6b68865e6b4c37ac005f5f24c3d6e1de7c5bac
2020-06-08 09:27:17 -07:00
Tom Cherry
71de690f60 Merge "logcat: filter based on UID" 2020-06-08 15:58:39 +00:00
Jooyung Han
cba44a3c56 Merge "Make libsync a stubs library" 2020-06-08 07:11:10 +00:00
Treehugger Robot
a552265ec9 Merge "Fix fastboot flash-all on Nexus 7." 2020-06-06 00:26:19 +00:00
Rick Yiu
cb63bed8b5 Merge "Fine tune blkio setting to improve boot time" 2020-06-06 00:17:09 +00:00
Elliott Hughes
6a7ff5811d Fix fastboot flash-all on Nexus 7.
The Nexus 7 bootloader just returns the empty string for unknown
variables, which confused the new snapshot code.

Bug: https://issuetracker.google.com/158232468
Test: no Nexus 7 available during covid-19 :-(
Change-Id: I35ff8889b27944e8b7426eca4f513d9fa562c6d4
2020-06-05 14:09:21 -07:00
Siarhei Vishniakou
9243090395 Logtags for input flinger
Reserve logtags 62000 -> 62199 for inputflinger

Bug: 153092301
Test: build with ag/10998853
Change-Id: If87f8e8e39bd445dfa7c96a542482c9ff3bf3fd3
2020-06-05 12:37:43 -07:00
Tom Cherry
9291f0f566 logcat: filter based on UID
Bug: 153761317
Test: logging works normally without --uid
Test: logging filters based on uid with --uid
Change-Id: I6843fa635831c727c0236ac5534862e11447484b
2020-06-05 10:02:44 -07:00
Tom Cherry
a1f1775172 Merge changes I957c519b,Iaf4e389e,I92f8056b
* changes:
  logd: remove faulty optimization
  logd: don't store the map key in its value
  logd: use libbase logging
2020-06-05 15:21:13 +00:00
Paul Lawrence
2f0c6cb0f7 Set block size in dm-bow
Fix block-level checkpointing to work correctly when used in combination with
512 byte hardware sectors and metadata encryption with dm-default-key v2.

Bug: 153512828
Test: Parameter is passed to dm-bow based on first_api_level
Change-Id: Ic0a071221559271db20b06b2f17459b5b041e02d
2020-06-05 07:47:45 -07:00
Jooyung Han
78fc553d68 Make libsync a stubs library
libsync is an NDK/LLNDK library but it's missing "stubs" key. So, when
it is referenced by an APEX, it is bundled in APEX package.

By adding "stubs" property, we can make it a stubs library and APEXes
use it from the system instead of bundling it.

Note that the symbol(sync_wait) is exposed to APEX because it is used
by libui which is used by media APEXes again.

Bug: 158270824
Test: lunch mini_armv7a_neon   # no VNDK
      m com.andorid.media.swcodec
      // see if libsync is not in the APEX
Change-Id: I39e682328acb5cc363a4242601e5bf1470938dac
2020-06-05 17:13:57 +09:00
Tom Cherry
b0263af5a8 logd: remove faulty optimization
TagNameKey contains a pointer to a std::string and a std::string_view,
such it can both own a string or reference a different string.  This
is meant to be a memory optimization.

This, however, is actually a net pessimization.  Due to these three
below cases and typical usage pattern.

Cases:
1) In the case where TagNameKey owns the string, 3 words are wasted,
one for the pointer and two for the std::string_view.

2) In the case where TagNameKey references a short string, the same 3
words are wasted.  This is because std::string has a feature called
"Short String Optimization" that means std::string does not allocate
for strings of sizes <= 10 on 32bit devices and <= 22 on 64bit
devices.

3) In the case where TagNameKey references a longer string than the
"Short String Optimization" limits, then this saves the string's
length in bytes.

Usage pattern:
After boot on 32 bit cuttlefish, there were 679 entries for the first
case, and only 69 in the third case.  The 679 entries have an overhead
of 679 * 3 * sizeof(void*) = 679 * 12 = 8148 bytes.  The 69 strings in
the third case have a total length and therefore savings of 1352
bytes.  This is a net pessimization of 6796 bytes.

I expect this same ratio to be similar throughout the device's uptime.

This situation is worse on 64 bit devices.  If cuttlefish were 64 bit,
then there would have been only 18 items in the third case due to the
larger "Short String Optimization" capacity, and the cost for the
first case would have doubled.

Given the above and the cost of maintaining extra code, this
optimization is removed and a std::string is used as the hash table
key instead.

Test: logging unit tests
Change-Id: I957c519b19edca4f7fc531d96b7144cf68bf4e16
2020-06-04 10:33:07 -07:00
Rick Yiu
ad74d8dee0 Fine tune blkio setting to improve boot time
Bug: 133200996
Test: boot time test
Change-Id: I5262c28596adb7e849b202b8a163c190818f271a
2020-06-04 14:28:19 +08:00
Treehugger Robot
e10fb5f5c5 Merge "libsparse: Limit block size to 64 MB" 2020-06-04 03:32:55 +00:00
Tom Cherry
4596b78d12 logd: don't store the map key in its value
The keys are already available when iterating through the maps, so
this only serves to waste memory.

Test: unit tests
Change-Id: Iaf4e389eb0f0990e7113cd78be1773e767a356d4
2020-06-03 16:30:58 -07:00
Tom Cherry
f93b4006e0 logd: use libbase logging
We can use libbase logging to output to the kernel log instead of the
'prdebug' function, so use that instead.

Bonus #1: we can now use CHECK().
Bonus #2: logging unit tests automatically output to stderr.
Bonus #3: We see dependent library's logs instead of losing them to
the void.

Test: logging unit tests
Test: logs show appropriately in dmesg / stderr
Test: CHECK() works
Change-Id: I92f8056b4820dc4998996cf46460568085299700
2020-06-03 16:29:48 -07:00
Treehugger Robot
ff502379b7 Merge "adb: treat non-successful inc-server runs as failures" 2020-06-03 21:28:53 +00:00
Treehugger Robot
dc9acdd221 Merge "first_stage_init: support kernel module directories" 2020-06-03 19:08:11 +00:00
Treehugger Robot
a990f0760d Merge "add libmodprobe api to query the number of modules loaded" 2020-06-03 19:06:13 +00:00
Steve Muckle
d6d38c326e first_stage_init: support kernel module directories
Kernel modules may be located within directories in /lib/modules.
Attempt to load kernel modules from each directory that has a name starting with
the major and minor version of the currently running kernel. If a single
kernel module is successfully loaded from a directory, that directory is
treated as the correct kernel module directory for the system. No other
kernel module directories are searched and any kernel module load errors
in that directory are fatal. If the attempt to load the first kernel
module from a directory fails, or if there are no kernel modules in a
directory, then the search proceeds to the next directory.

If no kernel module is successfully loaded from any directory as above,
an attempt is made to load kernel modules from the top level at
/lib/modules/.

Bug: 157645635
Change-Id: I92eadd8617f03a645da460ccb776bc04fa541f00
2020-06-03 10:51:51 -07:00
Steve Muckle
b0c4881ea9 add libmodprobe api to query the number of modules loaded
In the short term this will be used to help implement support for
multiple kernel module directories but it may be useful in other
contexts as well.

Bug: 157645635
Change-Id: I15a252b6e9394292c8f3557ed65112c935830441
2020-06-03 10:51:45 -07:00
Tom Cherry
430dbf32f8 Merge "logd: Drop the LogStatistics dependency on LogBufferElement" am: 671f39fdf2
Original change: https://android-review.googlesource.com/c/platform/system/core/+/1322276

Change-Id: Icdf95edb626f7e2d0425e77894c2a1295cb46476
2020-06-03 15:28:25 +00:00
Tom Cherry
671f39fdf2 Merge "logd: Drop the LogStatistics dependency on LogBufferElement" 2020-06-03 15:15:00 +00:00
Yurii Zubrytskyi
924915bacd adb: treat non-successful inc-server runs as failures
This way if an unknown device returns anything but a clear
success report from an incremental installation adb would retry
non-incrementally

Bug: 158006039
Test: manual install on R and Q
Change-Id: I9950273bd60ba75fe12a3706d2ccbee136987d13
2020-06-02 23:38:51 -07:00
Woody Lin
a386482f1d libsparse: Limit block size to 64 MB
Limit the block size to 64 MB before fastboot executable binary for
windows 64-bit is released.

Bug: 156057250
Change-Id: Ic4edb963a3d99f718d7630aba3f351729a84e994
2020-06-03 11:45:17 +08:00
Josh Gao
0b3202f826 Merge "adb: implement zstd compression for file sync." am: 450e83eb8f
Original change: https://android-review.googlesource.com/c/platform/system/core/+/1319091

Change-Id: Ia34e6e6222efb0c7319e1f6e227c4aa45d92c1a3
2020-06-03 02:07:10 +00:00
Josh Gao
450e83eb8f Merge "adb: implement zstd compression for file sync." 2020-06-03 01:56:31 +00:00
Treehugger Robot
f476677b4e Merge "check_ms_os_desc: fix buffer overflow." am: f7fed04ace
Original change: https://android-review.googlesource.com/c/platform/system/core/+/1322275

Change-Id: I5bb717db05585f9c119b1b60f4a4f013da9e87a6
2020-06-03 01:32:40 +00:00
Treehugger Robot
f7fed04ace Merge "check_ms_os_desc: fix buffer overflow." 2020-06-03 01:09:48 +00:00
Alex Buynytskyy
3fc21d4802 Merge "Signature checks." am: 0aa0581404
Original change: https://android-review.googlesource.com/c/platform/system/core/+/1321890

Change-Id: Ia2249d3db0e7b7dd048b6ef09780bc156d1bcc6b
2020-06-03 00:44:25 +00:00
Alex Buynytskyy
0aa0581404 Merge "Signature checks." 2020-06-03 00:30:34 +00:00
Alex Buynytskyy
7405a92e35 Signature checks.
- signature size validation,
- no-signature files support (e.g. .dm).

Bug: 157077910 150803885
Fixes: 157077910
Test: atest adb_test adbd_test

Change-Id: I03a4f32a87568bd6f447f66c8aab666fb0b27199
2020-06-02 23:12:58 +00:00
Tom Cherry
3dd3ec3514 logd: Drop the LogStatistics dependency on LogBufferElement
Other log buffers may not use LogBufferElement, so we should decouple
the two classes.  This uses an intermediate LogStatisticsElement
structs instead of passing a large number of parameters to each
function.

This additionally moves IsBinary() and the GetTag() functions out into
LogUtils.h since they can be used generically by other users.

Test: logging unit tests
Change-Id: I71f53257342c067bcccd5aa00bae47f714cd7c66
2020-06-02 16:02:56 -07:00
Josh Gao
a2929d5751 check_ms_os_desc: fix buffer overflow.
Bug: http://b/155505587
Test: check_ms_os_desc
Change-Id: Ie99715ab62571d24460c3a1fb859d22951f30cb8
2020-06-02 15:24:47 -07:00
Tom Cherry
a9b4222196 Merge "logd: move leading_dropped logic into FlushTo()" am: a8b8d108e5
Original change: https://android-review.googlesource.com/c/platform/system/core/+/1320576

Change-Id: I8129ea7791ef4291e2a7d8ca8b14698a933fc8c7
2020-06-02 21:56:12 +00:00
Tom Cherry
a8b8d108e5 Merge "logd: move leading_dropped logic into FlushTo()" 2020-06-02 21:51:17 +00:00
Treehugger Robot
1d73552164 Merge "adb: don't use parameterized in test_adb.py." am: 648266ab1c
Original change: https://android-review.googlesource.com/c/platform/system/core/+/1322179

Change-Id: Icca5eb952bc158013efdb6851d8d6f7a1c794cba
2020-06-02 21:49:15 +00:00
Treehugger Robot
648266ab1c Merge "adb: don't use parameterized in test_adb.py." 2020-06-02 21:31:21 +00:00
Tom Cherry
b3e163399a logd: move leading_dropped logic into FlushTo()
This logic isn't generic, so it should not be in the generic
LogReaderThread.

Moreover, it's currently broken in essentially every case except when
filtering by UID, because it runs as in the filter functions before
the actual filtering by pid/etc takes place.  For example, when
filtering by pid, it's possible to get leading chatty messages.  The
newly added test was failing previously but is fixed by this change.

It's fundamentally broken in the tail case.  Take this example:
1: Normal message
2: Chatty message
3: Normal message
4: Normal message

If you read that log buffer with a tail value of 3, there are three
possible outcomes:
1) Messages #2-4, however this would include a leading chatty message,
   which is not allowed.
2) Messages #3-4, however this is only 2, not 3 messages.
3) Messages #1-4, however this is 4, more than the 3 requested
   messages.

This code chooses 2) as the correct solution, in this case, we don't
need to account for leading chatty messages when counting the total
logs in the buffer.  A test is added for this case as well.

Test: new unit test
Change-Id: Id02eb81a8e77390aba4f85aac659c6cab498dbcd
2020-06-02 13:26:48 -07:00
Tom Cherry
6e63019c63 Merge "logd: format LogBufferElement and LogStatistics correctly" am: 4ab0bc414d
Original change: https://android-review.googlesource.com/c/platform/system/core/+/1322875

Change-Id: I871b7dcbc38e911665a5df2565b2a2cb07c2a0e2
2020-06-02 20:23:30 +00:00
Josh Gao
3f2f3bacf9 adb: don't use parameterized in test_adb.py.
The test bots don't have this module installed.

Test: treehugger
Change-Id: I849a075b784eac296608e45bec7b2fce38f56ac4
2020-06-02 13:14:45 -07:00
Tom Cherry
4ab0bc414d Merge "logd: format LogBufferElement and LogStatistics correctly" 2020-06-02 20:07:52 +00:00