tequilaOS/platform_bionic
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
42788 commits 1 branch 0 tags 80 MiB
Commit graph

19 commits

Author SHA1 Message Date
Elliott Hughes
cbcccd53eb Microoptimize vdso lookup. 
Actually reusing the hash table stuff from the linker would be a lot of
work (and the benefit unclear for so few symbols), but we can at least
break out of loops as soon as we have what we need.

Test: treehugger
Change-Id: I9cbe4636a3e3163bd302c3676eb674a4278dbf2e
 2023-12-12 15:13:03 -08:00
Elliott Hughes
fce8a155df __riscv_hwprobe: don't try to set errno. 
When used in an ifunc resolver, errno@plt won't be available. This is
the API the rivos folks contributing to glibc are leaning towards, for
the same reason. Hit by the berberis folks because they don't implement
the syscall so they were trying to set errno to ENOSYS.

Tested by looking at the generated assembler, and also disabling the
vdso (since on actual systems, this will go via the vdso).

Test: treehugger
Change-Id: Ie2779110f141f20efe97cb892fbdefd808b5339b
 2023-08-29 15:41:36 -07:00
Elliott Hughes
9a7d048712 riscv64: use vdso for __riscv_hwprobe(). 
Also de-pessimize time(), where the vdso entrypoint only exists on
x86/x86-64 anyway.

Bug: https://github.com/google/android-riscv64/issues/8
Test: strace
Change-Id: I14cb2a3130b6ff88d06d43ea13d3a825a26de290
 2023-07-31 12:24:31 -07:00
Ryan Prichard
746ad15912 Replace TLS_SLOT_BIONIC_PREINIT w/ shared globals 
Instead of passing the address of a KernelArgumentBlock to libc.so for
initialization, use __loader_shared_globals() to initialize globals.

Most of the work happened in the previous CLs. This CL switches a few
KernelArgumentBlock::getauxval calls to [__bionic_]getauxval and stops
routing the KernelArgumentBlock address through the libc init functions.

Bug: none
Test: bionic unit tests
Change-Id: I96c7b02c21d55c454558b7a5a9243c682782f2dd
 2018-12-03 17:14:44 -08:00
Elliott Hughes
f1515f6408 Clean up the time(2) implementation. 
This is also slightly faster for the no VDSO case (56ns vs 66ns).

Bug: N/A
Test: ran tests, benchmarks
Change-Id: I2b0edd06ee6942eb57c32678279278a53ca5ee9b
 2018-01-12 15:20:28 -08:00
Mark Salyzyn
4473ccd5b0 bionic: add vdso time() 
time() can be a hot call, and it currently uses __vdso_gettimeofday,
which is already pretty fast (~3 times faster than the syscall),
but with a __vdso_time call it is ~3 times even faster, in part
because __vdso_time does not require interlocking with updates,
and the read for just the seconds is atomic.  __vdso_time is
always available, whereas __vdso_gettimeofday is gated on access
to the physical timers.  arm improvement is compelling (x10),
x86 improvement is even more pronounced (x100).

[TL;DR]

w/vdso32 kernel patches, locked cores to MAX, little cores only.

BEFORE:

hikey960 vdso (aarch64):

----------------------------------------------------------------------
Benchmark                               Time           CPU Iterations
----------------------------------------------------------------------
BM_time_clock_gettime                  48 ns         48 ns   15414753
BM_time_clock_gettime_syscall         175 ns        175 ns    4062031
BM_time_clock_gettime_REALTIME         44 ns         44 ns   15897875
BM_time_clock_gettime_BOOTTIME         47 ns         47 ns   14307903
BM_time_clock_gettime_TAI             210 ns        210 ns    3341372
BM_time_clock_gettime_unsupported     100 ns        100 ns    7030649
BM_time_gettimeofday                   47 ns         47 ns   14969643
BM_time_gettimeofday_syscall          163 ns        163 ns    4283542
BM_time_time                           59 ns         59 ns   11815385

hikey960 vdso32 (aarch32):

----------------------------------------------------------------------
Benchmark                               Time           CPU Iterations
----------------------------------------------------------------------
BM_time_clock_gettime                  90 ns         90 ns    7572898
BM_time_clock_gettime_syscall         251 ns        251 ns    2763442
BM_time_clock_gettime_REALTIME         81 ns         80 ns    8699536
BM_time_clock_gettime_BOOTTIME         97 ns         97 ns    7256667
BM_time_clock_gettime_TAI             272 ns        272 ns    2570419
BM_time_clock_gettime_unsupported     160 ns        160 ns    4379819
BM_time_gettimeofday                   73 ns         73 ns    9608922
BM_time_gettimeofday_syscall          200 ns        199 ns    3527957
BM_time_time                          123 ns        123 ns    5651095

x86_64 (glibc):

--------------------------------------------------------------------
Benchmark                             Time           CPU Iterations
--------------------------------------------------------------------
BM_time_clock_gettime                  21 ns         21 ns   28873070
BM_time_clock_gettime_syscall         224 ns        224 ns    3095370
BM_time_clock_gettime_REALTIME         17 ns         17 ns   42083086
BM_time_clock_gettime_BOOTTIME        239 ns        239 ns    2924015
BM_time_clock_gettime_TAI             236 ns        236 ns    2961423
BM_time_clock_gettime_unsupported     221 ns        221 ns    3357696
BM_time_gettimeofday                 22 ns         22 ns   27975154
BM_time_gettimeofday_syscall        238 ns        238 ns    2882032
BM_time_time                          2 ns          2 ns  340354885
BM_time_time_syscall                207 ns        207 ns    3383073

imx7d_pico IOT nyc (w/arm,cpu-registers-not-fw-configured) (armv7a):
(virtual timers)

Benchmark                           Time(ns)    CPU(ns) Iterations
------------------------------------------------------------------
BM_time_clock_gettime                     20        477    1489362
BM_time_clock_gettime_syscall             20        487    1458333
BM_time_clock_gettime_REALTIME            19        464    1400000
BM_time_clock_gettime_BOOTTIME            29        700    1000000
BM_time_clock_gettime_TAI                 29        690    1000000
BM_time_clock_gettime_unsupported          9        227    3043478
BM_time_gettimeofday                      18        444    1555556
BM_time_gettimeofday_syscall              19        456    1555556
BM_time_time                              21        497    1166667

imx7d_pico IOT nyc (wo/arm,cpu-registers-not-fw-configured) (armv7a):
(physical timers)

Benchmark                           Time(ns)    CPU(ns) Iterations
------------------------------------------------------------------
BM_time_clock_gettime                      6        144    4666667
BM_time_clock_gettime_syscall             20        486    1400000
BM_time_clock_gettime_REALTIME             6        136    5000000
BM_time_clock_gettime_BOOTTIME             6        153    4375000
BM_time_clock_gettime_TAI                 31        760    1000000
BM_time_clock_gettime_unsupported         10        233    3043478
BM_time_gettimeofday                       6        140    5000000
BM_time_gettimeofday_syscall              19        450    1555556
BM_time_time                               9        203    3500000

AFTER:

hikey960 vdso (aarch64):

--------------------------------------------------------------------
Benchmark                             Time           CPU Iterations
--------------------------------------------------------------------
BM_time_clock_gettime                48 ns         48 ns   15414753
BM_time_clock_gettime_syscall       175 ns        175 ns    4062031
BM_time_clock_gettime_REALTIME       44 ns         44 ns   15897875
BM_time_clock_gettime_BOOTTIME       47 ns         47 ns   14307903
BM_time_clock_gettime_TAI           210 ns        210 ns    3341372
BM_time_clock_gettime_unsupported   100 ns        100 ns    7030649
BM_time_gettimeofday                 47 ns         47 ns   14975314
BM_time_gettimeofday_syscall        164 ns        164 ns    4278797
BM_time_time                         16 ns         16 ns   42932165

hikey960 vdso32 (aarch32):

--------------------------------------------------------------------
Benchmark                             Time           CPU Iterations
--------------------------------------------------------------------
BM_time_clock_gettime                90 ns         90 ns    7572898
BM_time_clock_gettime_syscall       251 ns        251 ns    2763442
BM_time_clock_gettime_REALTIME       81 ns         80 ns    8699536
BM_time_clock_gettime_BOOTTIME       97 ns         97 ns    7256667
BM_time_clock_gettime_TAI           272 ns        272 ns    2570419
BM_time_clock_gettime_unsupported   160 ns        160 ns    4379819
BM_time_gettimeofday                 73 ns         73 ns    9596230
BM_time_gettimeofday_syscall        199 ns        199 ns    3575428
BM_time_time                         35 ns         35 ns   19798801

imx7d_pico IOT nyc (w/arm,cpu-registers-not-fw-configured) (armv7a):

Benchmark                           Time(ns)    CPU(ns) Iterations
------------------------------------------------------------------
BM_time_clock_gettime                     20        477    1489362
BM_time_clock_gettime_syscall             20        487    1458333
BM_time_clock_gettime_REALTIME            19        464    1400000
BM_time_clock_gettime_BOOTTIME            29        700    1000000
BM_time_clock_gettime_TAI                 29        690    1000000
BM_time_clock_gettime_unsupported          9        227    3043478
BM_time_gettimeofday                      18        444    1555556
BM_time_gettimeofday_syscall              19        456    1555556
BM_time_time                               2         50   11666667

imx7d_pico IOT nyc (wo/arm,cpu-registers-not-fw-configured) (armv7a):

Benchmark                           Time(ns)    CPU(ns) Iterations
------------------------------------------------------------------
BM_time_clock_gettime                      6        144    4666667
BM_time_clock_gettime_syscall             20        486    1400000
BM_time_clock_gettime_REALTIME             6        136    5000000
BM_time_clock_gettime_BOOTTIME             6        153    4375000
BM_time_clock_gettime_TAI                 31        760    1000000
BM_time_clock_gettime_unsupported         10        233    3043478
BM_time_gettimeofday                       6        140    5000000
BM_time_gettimeofday_syscall              19        450    1555556
BM_time_time                               2         50   10000000

Test: bionic-unit-tests --gtest_filter=time.time
      taskset F bionic-benchmarks --bionic_xml=vdso.xml \
          --benchmark_filter='BM_time_(time*|clock_gettime*|gettimeofday*)'
Bug: 63737556
Change-Id: I81b088a12ca41a6c4733d46c5477527777138efa
 2018-01-11 15:46:11 +00:00
Mark Salyzyn
79249b0897 bionic: add vdso clock_getres 
clock_getres() should not be a hot call, nevertheless it is
~6-7 times faster for supported clock ids if it uses
__vdso_clock_getres if available.  There is a 3% performance
penalty for unsupported clock ids via __vdso_clock_getres with
respect to a direct syscall.

[TL;DR]

w/vdso32 kernel patches, locked cores to MAX, little cores only.

BEFORE:

hikey960 vdso (aarch64):

----------------------------------------------------------------------
Benchmark                               Time           CPU Iterations
----------------------------------------------------------------------
BM_time_clock_getres                  126 ns        126 ns    5577874
BM_time_clock_getres_syscall          127 ns        127 ns    5505016
BM_time_clock_getres_REALTIME         126 ns        126 ns    5574682
BM_time_clock_getres_BOOTTIME         126 ns        126 ns    5575237
BM_time_clock_getres_TAI              126 ns        126 ns    5576810
BM_time_clock_getres_unsupported      128 ns        128 ns    5480189

hikey960 vdso32 (aarch32):

----------------------------------------------------------------------
Benchmark                               Time           CPU Iterations
----------------------------------------------------------------------
BM_time_clock_getres                  199 ns        199 ns    3508708
BM_time_clock_getres_syscall          220 ns        220 ns    3184676
BM_time_clock_getres_REALTIME         199 ns        199 ns    3509697
BM_time_clock_getres_BOOTTIME         199 ns        199 ns    3513551
BM_time_clock_getres_TAI              200 ns        199 ns    3512412
BM_time_clock_getres_unsupported      196 ns        196 ns    3575609

x86_64 (glibc):

---------------------------------------------------------------------
Benchmark                              Time           CPU Iterations
---------------------------------------------------------------------
BM_time_clock_getres                 252 ns        252 ns    2370263
BM_time_clock_getres_syscall         215 ns        215 ns    3287497
BM_time_clock_getres_REALTIME        214 ns        214 ns    3294228
BM_time_clock_getres_BOOTTIME        213 ns        213 ns    3277519
BM_time_clock_getres_TAI             213 ns        213 ns    3294991
BM_time_clock_getres_unsupported     206 ns        206 ns    3450654

imx7d_pico IOT nyc (w/arm,cpu-registers-not-fw-configured) (armv7a):
(Virtual Timers)

Benchmark                           Time(ns)    CPU(ns) Iterations
------------------------------------------------------------------
BM_time_clock_getres                      16        345    2000000
BM_time_clock_getres_syscall              16        339    2121212
BM_time_clock_getres_REALTIME             17        350    2058824
BM_time_clock_getres_BOOTTIME             17        345    2000000
BM_time_clock_getres_TAI                  16        350    2000000
BM_time_clock_getres_unsupported          13        284    2500000

AFTER:

hikey960 vdso (aarch64):

---------------------------------------------------------------------
Benchmark                              Time           CPU Iterations
---------------------------------------------------------------------
BM_time_clock_getres                  18 ns         18 ns   37880389
BM_time_clock_getres_syscall         127 ns        127 ns    5520029
BM_time_clock_getres_REALTIME         18 ns         18 ns   37879962
BM_time_clock_getres_BOOTTIME         19 ns         18 ns   37878361
BM_time_clock_getres_TAI             131 ns        131 ns    5368484
BM_time_clock_getres_unsupported      97 ns         97 ns    7182864

hikey960 vdso32 (aarch32):

---------------------------------------------------------------------
Benchmark                              Time           CPU Iterations
---------------------------------------------------------------------
BM_time_clock_getres                  36 ns         36 ns   19205240
BM_time_clock_getres_syscall         212 ns        212 ns    3297100
BM_time_clock_getres_REALTIME         36 ns         36 ns   19219109
BM_time_clock_getres_BOOTTIME         36 ns         36 ns   19222490
BM_time_clock_getres_TAI             206 ns        206 ns    3402868
BM_time_clock_getres_unsupported     159 ns        159 ns    4409492

imx7d_pico IOT nyc (wo/arm,cpu-registers-not-fw-configured) (armv7a):
(Physical Timers)

Benchmark                           Time(ns)    CPU(ns) Iterations
------------------------------------------------------------------
BM_time_clock_getres                       2         48   14000000
BM_time_clock_getres_syscall              14        335    2058824
BM_time_clock_getres_REALTIME              2         49   14583333
BM_time_clock_getres_BOOTTIME              2         48   14000000
BM_time_clock_getres_TAI                  14        350    2058824
BM_time_clock_getres_unsupported           8        203    3500000

Test: taskset F \
        /data/benchmarktest{64}/bionic-benchmarks/bionic-benchmarks \
        --bionic_xml=vdso.xml --benchmark_filter=BM_time_clock_getres*
Bug: 63737556
Change-Id: I80c0a5106625d76720287f715fcf145d2aad1705
 2017-12-07 09:41:48 -08:00
Elliott Hughes
9591df5294 Fix vdso system call fallback failures. 
When a vdso call falls back to making a regular system call, the inline
code for the system call doesn't know about errno and just leaves the
usual kernel result to be translated. Add the missing translation.

Also fix the defaults for non-vdso systems so we actually take the
fallback path (and so avoid unintentionally doing the errno translation
twice in those cases).

Bug: http://b/69626243
Test: ran new tests from http://b/63737556
Change-Id: If379632ea2e059e3d3bc3ff41bf3608dc05fb0a3
 2017-11-28 09:09:01 -08:00
Elliott Hughes
c51a404b1a All architectures have AT_SYSINFO_EHDR now. 
Bug: N/A
Test: builds
Change-Id: Ibc894be98ed0781c8b991ffadff34f616b934aa7
 2017-09-22 13:08:50 -07:00
Goran Ferenc
996f6decfa MIPS: 32/64-bit VDSO support 
File libc/SYSCALLS.TXT is updated to generate bionic's system call wrappers
for clock_gettime() & gettimeofday() that will be called if kernel vdso
implementations fail to execute.
The system call wrappers are generated using a python script gensyscalls.py.

Since all architectures support vdso now, there is no more need for conditional
statements regarding supported architectures in libc/bionic/vdso.cpp &
libc/private/bionic_vdso.h files.

Test: builds
Change-Id: I7213f29c179a7929851499d78a72900638ae861a
Signed-off-by: Goran Ferenc <goran.ferenc@imgtec.com>
 2017-06-01 08:50:57 -07:00
Elliott Hughes
4ce902c30e Support 32-bit ARM vdso. 
Linux 4.1 added this.

Bug: http://b/19198045
Change-Id: I28be802ff403a61dd6733a001411b3ff05fef5a5
 2016-07-12 13:55:51 -07:00
Josh Gao
93c0f5ee00 Move VDSO pointers to a shared globals struct. 
Change-Id: I01cbc9cf0917dc1fac52d9205bda2c68529d12ef
 2015-10-09 15:59:04 -07:00
Robert Jarzmik
10726d52ac libc: arch-x86: implement kernel vdso time functions 
This patch give the possibility of time vdso support on 32bit kernel.
If the 32bit x86 kernel provides gettimeofday() and clock_gettime()
primitives in vdso. In this case make bionic use them. If the kernel
doesn't provide them, fallback to the legacy system call versions.

Change-Id: I87b772a9486fa356903e1f98f486ab9eb0b6f6f7
Signed-off-by: Robert Jarzmik <robert.jarzmik@intel.com>
Signed-off-by: Mingwei Shi <mingwei.shi@intel.com>
 2015-07-22 01:31:38 +00:00
Elliott Hughes
613f814508 Revert "Revert "make vdso function pointers read-only at runtime"" 
This reverts commit 1946856b1f.

This goes back to the original scheme of PROT_NONEing a page within
libc. Allocating a new page didn't fail safe for cases where these
functions are called from the dynamic linker.

Bug: http://b/22568628
Change-Id: I3e7241c8b54c27ea4a898bc952375c1e9ae38c80
 2015-07-20 15:59:33 -07:00
Elliott Hughes
1946856b1f Revert "make vdso function pointers read-only at runtime" 
This reverts commit df1a3c6d21.

This change prevented N9 from booting (http://b/22568628).

Change-Id: I071d6d6a0ae7881d65641839e665acdcf58462b4
 2015-07-20 17:30:33 +00:00
Daniel Micay
df1a3c6d21 make vdso function pointers read-only at runtime 
Global, writable function pointers are low-hanging fruit for hijacking
control flow with an overflow from a global buffer or an arbitrary write
vulnerability. This moves the function pointer table into a dedicated
page and makes it read-only at runtime, similar to RELRO.

This increases the memory usage of the library by just under one page.
This could be avoided by having the linker load the vdso by replacing
weak symbols. It's not significant within the Zygote spawning model
though because it's read-only after early init.

Change-Id: Id7a49c96c1b15c2e1926528304b3c54a81e78caf
 2015-07-17 11:11:42 -07:00
Elliott Hughes
05fc1d7050 Add missing includes. 
Change-Id: Ibf549266a19a67eb9158d341a69dddfb654be669
 2015-01-28 19:23:11 -08:00
Elliott Hughes
8b5df3920f Turn on -Wold-style-cast and fix the errors. 
A couple of dodgy cases where we cast away const, but otherwise pretty boring.

Change-Id: Ibc39ebd525377792b5911464be842121c20f03b9
 2015-01-21 17:09:58 -08:00
Elliott Hughes
625993dfbb Use VDSO for clock_gettime(2) and gettimeofday(2). 
Bug: 15387103
Change-Id: Ifc3608ea65060c1dc38120b10b6e79874f182a36
 2014-07-16 14:27:43 -07:00