The libcorkscrew stack unwinder does not understand cfi directives,
so add .save directives so that it can function properly.
Also add the directives in to strcmp.S and fix a missing set of
directives in cortex-a9/memcpy_base.S.
Bug: 10345269
Change-Id: I043f493e0bb6c45bd3f4906fbe1d9f628815b015
This change pulls the memcpy code out into a new file so that the
__strcpy_chk and __strcat_chk can use it with an include.
The new versions of the two chk functions uses assembly versions
of strlen and memcpy to implement this check. This allows near
parity with the assembly versions of strcpy/strcat. It also means that
as memcpy implementations get faster, so do the chk functions.
Other included changes:
- Change all of the assembly labels to local labels. The other labels
confuse gdb and mess up backtracing.
- Add .cfi_startproc and .cfi_endproc directives so that gdb is not
confused when falling through from one function to another.
- Change all functions to use cfi directives since they are more powerful.
- Move the memcpy_chk fail code outside of the memcpy function definition
so that backtraces work properly.
- Preserve lr before the calls to __fortify_chk_fail so that the backtrace
actually works.
Testing:
- Ran the bionic unit tests. Verified all error messages in logs are set
correctly.
- Ran libc_test, replacing strcpy with __strcpy_chk and replacing
strcat with __strcat_chk.
- Ran the debugger on nexus10, nexus4, and old nexus7. Verified that the
backtrace is correct for all fortify check failures. Also verify that
when falling through from __memcpy_chk to memcpy that the backtrace is
still correct. Also verified the same for __memset_chk and bzero.
Verified the two different paths in the cortex-a9 memset routine that
save variables to the stack still show the backtrace properly.
Bug: 9293744
(cherry-picked from 2be91915dc)
Change-Id: Ia407b74d3287d0b6af0139a90b6eb3bfaebf2155
This change creates assembler versions of __memcpy_chk/__memset_chk
that is implemented in the memcpy/memset assembler code. This change
avoids an extra call to memcpy/memset, instead allowing a simple fall
through to occur from the chk code into the body of the real
implementation.
Testing:
- Ran the libc_test on __memcpy_chk/__memset_chk on all nexus devices.
- Wrote a small test executable that has three calls to __memcpy_chk and
three calls to __memset_chk. First call dest_len is length + 1. Second
call dest_len is length. Third call dest_len is length - 1.
Verified that the first two calls pass, and the third fails. Examined
the logcat output on all nexus devices to verify that the fortify
error message was sent properly.
- I benchmarked the new __memcpy_chk and __memset_chk on all systems. For
__memcpy_chk and large copies, the savings is relatively small (about 1%).
For small copies, the savings is large on cortex-a15/krait devices
(between 5% to 30%).
For cortex-a9 and small copies, the speed up is present, but relatively
small (about 3% to 5%).
For __memset_chk and large copies, the savings is also small (about 1%).
However, all processors show larger speed-ups on small copies (about 30% to
100%).
Bug: 9293744
Merge from internal master.
(cherry-picked from 7c860db074)
Change-Id: I916ad305e4001269460ca6ebd38aaa0be8ac7f52
This change pulls the memcpy code out into a new file so that the
__strcpy_chk and __strcat_chk can use it with an include.
The new versions of the two chk functions uses assembly versions
of strlen and memcpy to implement this check. This allows near
parity with the assembly versions of strcpy/strcat. It also means that
as memcpy implementations get faster, so do the chk functions.
Other included changes:
- Change all of the assembly labels to local labels. The other labels
confuse gdb and mess up backtracing.
- Add .cfi_startproc and .cfi_endproc directives so that gdb is not
confused when falling through from one function to another.
- Change all functions to use cfi directives since they are more powerful.
- Move the memcpy_chk fail code outside of the memcpy function definition
so that backtraces work properly.
- Preserve lr before the calls to __fortify_chk_fail so that the backtrace
actually works.
Testing:
- Ran the bionic unit tests. Verified all error messages in logs are set
correctly.
- Ran libc_test, replacing strcpy with __strcpy_chk and replacing
strcat with __strcat_chk.
- Ran the debugger on nexus10, nexus4, and old nexus7. Verified that the
backtrace is correct for all fortify check failures. Also verify that
when falling through from __memcpy_chk to memcpy that the backtrace is
still correct. Also verified the same for __memset_chk and bzero.
Verified the two different paths in the cortex-a9 memset routine that
save variables to the stack still show the backtrace properly.
Bug: 9293744
Change-Id: Id5aec8c3cb14101d91bd125eaf3770c9c8aa3f57
(cherry picked from commit 2be91915dc)
Create one version of strcat/strcpy/strlen for cortex-a15/krait and another
version for cortex-a9.
Tested with the libc_test strcat/strcpy/strlen tests.
Including new tests that verify that the src for strcat/strcpy do not
overread across page boundaries.
NOTE: The handling of unaligned strcpy (same code in strcat) could probably
be optimized further such that the src is read 64 bits at a time instead of
the partial reads occurring now.
strlen improves slightly since it was recently optimized.
Performance improvements for strcpy and strcat (using an empty dest string):
cortex-a9
- Small copies vary from about 5% to 20% as the size gets above 10 bytes.
- Copies >= 1024, about a 60% improvement.
- Unaligned copies, from about 40% improvement.
cortex-a15
- Most small copies exhibit a 100% improvement, a few copies only
improve by 20%.
- Copies >= 1024, about 150% improvement.
- Unaligned copies, about 100% improvement.
krait
- Most small copies vary widely, but on average 20% improvement, then
the performance gets better, hitting about a 100% improvement when
copies 64 bytes of data.
- Copies >= 1024, about 100% improvement.
- When coping MBs of data, about 50% improvement.
- Unaligned copies, about 90% improvement.
As strcat destination strings get larger in size:
cortex-a9
- about 40% improvement for small dst strings (>= 32).
- about 250% improvement for dst strings >= 1024.
cortex-a15
- about 200% improvement for small dst strings (>=32).
- about 250% improvement for dst strings >= 1024.
krait
- about 25% improvement for small dst strings (>=32).
- about 100% improvement for dst strings >=1024.
Merge from internal master.
(cherry-picked from d119b7b6f4)
Change-Id: I296463b251ef9fab004ee4dded2793feca5b547a
This change creates assembler versions of __memcpy_chk/__memset_chk
that is implemented in the memcpy/memset assembler code. This change
avoids an extra call to memcpy/memset, instead allowing a simple fall
through to occur from the chk code into the body of the real
implementation.
Testing:
- Ran the libc_test on __memcpy_chk/__memset_chk on all nexus devices.
- Wrote a small test executable that has three calls to __memcpy_chk and
three calls to __memset_chk. First call dest_len is length + 1. Second
call dest_len is length. Third call dest_len is length - 1.
Verified that the first two calls pass, and the third fails. Examined
the logcat output on all nexus devices to verify that the fortify
error message was sent properly.
- I benchmarked the new __memcpy_chk and __memset_chk on all systems. For
__memcpy_chk and large copies, the savings is relatively small (about 1%).
For small copies, the savings is large on cortex-a15/krait devices
(between 5% to 30%).
For cortex-a9 and small copies, the speed up is present, but relatively
small (about 3% to 5%).
For __memset_chk and large copies, the savings is also small (about 1%).
However, all processors show larger speed-ups on small copies (about 30% to
100%).
Bug: 9293744
Change-Id: I8926d59fe2673e36e8a27629e02a7b7059ebbc98
Create one version of strcat/strcpy/strlen for cortex-a15/krait and another
version for cortex-a9.
Tested with the libc_test strcat/strcpy/strlen tests.
Including new tests that verify that the src for strcat/strcpy do not
overread across page boundaries.
NOTE: The handling of unaligned strcpy (same code in strcat) could probably
be optimized further such that the src is read 64 bits at a time instead of
the partial reads occurring now.
strlen improves slightly since it was recently optimized.
Performance improvements for strcpy and strcat (using an empty dest string):
cortex-a9
- Small copies vary from about 5% to 20% as the size gets above 10 bytes.
- Copies >= 1024, about a 60% improvement.
- Unaligned copies, from about 40% improvement.
cortex-a15
- Most small copies exhibit a 100% improvement, a few copies only
improve by 20%.
- Copies >= 1024, about 150% improvement.
- Unaligned copies, about 100% improvement.
krait
- Most small copies vary widely, but on average 20% improvement, then
the performance gets better, hitting about a 100% improvement when
copies 64 bytes of data.
- Copies >= 1024, about 100% improvement.
- When coping MBs of data, about 50% improvement.
- Unaligned copies, about 90% improvement.
As strcat destination strings get larger in size:
cortex-a9
- about 40% improvement for small dst strings (>= 32).
- about 250% improvement for dst strings >= 1024.
cortex-a15
- about 200% improvement for small dst strings (>=32).
- about 250% improvement for dst strings >= 1024.
krait
- about 25% improvement for small dst strings (>=32).
- about 100% improvement for dst strings >=1024.
Change-Id: Ifd091ebdbce70fe35a7c5d8f71d5914255f3af35
This is needed when passing -mcpu=cortex-a9 or higher on a modern
toolchain for prebuilt library compatibility
Change-Id: I73eb2393377914ae26216a8c2828ad973d1c1225
Tested using a static version of the strlen libc_test program
on a nexus7 that uses the generic code.
Merge from internal master.
(cherry-picked from d8d10a8994)
Change-Id: I88f7dc01dc5b5c3ac2d5580d92153bc1bc36c564
This optimized version is primarily targeted at cortex-a15.
Tested on all nexus devices using the system/extras/libc_test strlen test.
Tested alignments from 1 to 32 that are powers of 2.
Tested that strlen does not cross page boundaries at all alignments.
Speed improvements listed below:
cortex-a15
- Sizes >= 32 bytes, ~75% improvement.
- Sizes >= 1024 bytes, ~250% improvement.
cortex-a9
- Sizes >= 32 bytes, ~75% improvement.
- Sizes >= 1024 bytes, ~85% improvement.
krait
- Sizes >= 32 bytes, ~95% improvement.
- Sizes >= 1024 bytes, ~160% improvement.
Merge from internal master.
(cherry-picked from 2fc0717977)
Change-Id: I1ceceb4e745fd68e9d946f96d1d42e0cdaff6ccf
We cleaned up the auto-generated ones a while back to not touch
the stack unnecessarily if they have <= 4 arguments. This patch
cleans up some hand-crafted ones.
Also improve comments in clone.S.
Change-Id: I8850bf98f2b26829385315304472a760e6880ed8
Tested using a static version of the strlen libc_test program
on a nexus7 that uses the generic code.
Change-Id: If04d15dcb6c0b18f27f2fefadca5510ed49016c5
This memcpy code uses NEON/VFP to achieve very good performance
on ARMv7-A processors. It is specifically tuned for A15 but should
provide good performance on A9 also. It is equivalent to the code
in cortex-strings rev 116.
This patch is a follow up the existing gerrit change:
I7f6f77995f3ca903ad9c66d14261441667a2a935
This version includes a tweak for performance on misaligned
buffers and splits the header comment into license and
documentation sections.
Change-Id: Ibd2e23c8d8e01357ba0247be1d05192de3ceba69
Signed-off-by: Will Newton <will.newton@linaro.org>
This memcpy code uses NEON/VFP to achieve very good performance
on ARMv7-A processors. It is specifically tuned for A15 but should
provide good performance on A9 also. It is equivalent to the code
in cortex-strings rev 116.
This patch is a follow up the existing gerrit change:
I7f6f77995f3ca903ad9c66d14261441667a2a935
But this version includes a tweak for performance on misaligned
buffers.
Change-Id: I285abac0068f8ae29a1cbf7862ea8590aadaf0a7
Signed-off-by: Will Newton <will.newton@linaro.org>
Streamline the memcpy a bit removing some unnecessary instructions.
The biggest speed improvement comes from changing the size of
the preload. On krait, the sweet spot for the preload in the main
loop is twice the L1 cache line size.
In most cases, these small tweaks yield > 1000MB/s speed ups. As
the size of the memcpy approaches about 1MB, the speed improvement
disappears.
Change-Id: Ief79694d65324e2db41bee4707dae19b8c24be62
This uses the new code original submitted as memcpy.a15.S as
the base. However, the old code handled unaligned src/dst better
so that was spliced in. I optimized the original unaligned code by
removing a few unnecessary instructions. I optimized the a15 code by
rewriting the pre and post code. I also modified the main loop to add
a pld so that larger copies would not stall waiting for memory.
Test cases for the new memcpy:
- Copy all sized values from 0 to 1024 bytes, using whatever alignment
is returned by malloc.
For each alignment case described below, the test copied from 0 to 128
bytes.
- Src and dst pointers are both aligned to the same value, starting
at one going through every power of two up to and including 128.
- Src aligned to double word boundary, dst aligned to word boundary.
- Src aligned to word boundary, dst aligned to double word boundary.
- Src aligned to 16 bit boundary, dst aligned to word boundary.
- Src aligned to word boundary, dst aligned to 16 byte boundary.
- Src aligned to word boundary, dst aligned to 1 byte from a word
boundary.
- Src aligned to word boundary, dst aligned to 2 bytes from a word
boundary.
- Src aligned to word boundary, dst aligned to 3 bytes from a word
boundary.
- Src aligned to 1 byte from a word boundary, dst aligned to a word
boundary.
- Src aligned to 2 bytes from a word boundary, dst aligned to a word
boundary.
- Src aligned to 3 bytes from a word boundary, dst aligned to a word
boundary.
Cases to verify the unaligned source code properly aligns to a 16 bit
boundary.
- Src aligned to 1 byte from a 128 bit boundary, dst aligned to
4 + 128 bit boundary.
- Src aligned to 1 byte from a 128 bit boundary, dst aligned to
8 + 128 bit boundary.
- Src aligned to 1 byte from a 128 bit boundary, dst aligned to
12 + 128 bit boundary.
- Src aligned to 1 byte from a 128 bit boundary, dst aligned to
16 + 128 bit boundary.
In all cases, a two byte fencepost was placed at the end of the
destination to verify that only the requested number of bytes were copied.
Bug: 8005082
Merge from internal master.
(cherry-picked from commit 21ede92d79)
Change-Id: Ief70c9e6dc8c6473ae245b6570b2c266fed9618c
This lets us move all the ARM syscall stubs over to the kernel <asm/unistd.h>.
Our generated <sys/linux-syscalls.h> is now unused, but I'll remove that in a
later change.
Change-Id: Ie5ff2cc4abce1938576af7cbaef615a79c7f310d
For some reason, socketcalls.c was only being compiled for ARM, where
it makes no sense. For x86 we generate stubs for the socket functions
that use __NR_socketcall directly.
Change-Id: I84181e6183fae2314ae3ed862276eba82ad21e8e
<sys/linux-syscalls.h> only contains constants for the syscalls
we're generating stubs for. We want all the syscalls available
on the architecture in question.
Keep using <sys/linux-syscalls.h> on ARM for now because the
__NR_ARM_set_tls and __NR_ARM_cacheflush values aren't in <asm/unistd.h>.
Change-Id: I66683950d87d9b18d6107d0acc0ed238a4496f44
This uses the new strcmp.a15.S code as the basis for new versions
of strcmp.S.
The cortex-a15 code is the performance optimized version of strcmp.a15.S
taken with only the addition of a few pld instructions.
The cortex-a9 code is the same as the cortex-a15 code except that the
unaligned strcmp code was taken from the original strcmp.S.
The krait code is the same as the cortex-a15 code except that one path
in the unaligned strcmp code was taken from the original strcmp.S code
(the 2 byte overlap case).
The generic code is the original unmodified strmp.S from the bionic
subdirectory.
All three new versions underwent these test cases:
Strings the same, all same size:
- Both pointers double word aligned.
- One pointer double word aligned, one pointer word aligned.
- Both pointers word aligned.
- One pointer double word aligned, one pointer 1 off a word alignment.
- One pointer double word aligned, one pointer 2 off a word alignment.
- One pointer double word aligned, one pointer 3 off a word alignment.
- One pointer word aligned, one pointer 1 off a word alignment.
- One pointer word aligned, one pointer 2 off a word alignment.
- One pointer word aligned, one pointer 3 off a word alignment.
For all cases where it made sense, the two pointers were also tested
swapped.
Different strings, all same size:
- Single difference at double word boundary.
- Single difference at word boudary.
- Single difference at 1 off a word alignment.
- Single difference at 2 off a word alignment.
- Single difference at 3 off a word alignment.
Different sized strings, strings the same until the end:
- Shorter string ends on a double word boundary.
- Shorter string ends on word boundary.
- Shorter string ends at 1 off a word boundary.
- Shorter string ends at 2 off a word boundary.
- Shorter string ends at 3 off a word boundary.
For all different cases, run them through the same pointer alignment
cases when the strings are the same size.
For all cases the two pointers were also tested swapped.
Bug: 8005082
Merge from internal master.
(cherry-picked from commit a9a5870d16)
Change-Id: I4c2b98f8a50804fb98ab67f75e9d660f1315a144
We only need one logging API, and I prefer the one that does no
allocation and is thus safe to use in any context.
Also use O_CLOEXEC when opening the /dev/log files.
Move everything logging-related into one header file.
Change-Id: Ic1e3ea8e9b910dc29df351bff6c0aa4db26fbb58
Move arch specific code for arm, mips, x86 into separate
makefiles.
In addition, add different arm cpu versions of memcpy/memset.
Bug: 8005082
Merge from internal master (acdde8c1cf).
Change-Id: I04f3d0715104fab618e1abf7cf8f7eec9bec79df
The attached patch provides a new implementation of strcmp for ARM,
using LDRD instead of LDR whenever possible.
For older architectures that do not support LDRD, this implementation
uses the same algorithm as before.
Testing and benchmarking:
* Validation: successfully passes a test that compares different strings
of length 1-128 and offsets 0-8 from a word boundary. Checked on
qemu/A15/A9, ARM/Thumb mode, Big/Little Endian.
* Integration with gcc: no regression on qemu for arm-none-eabi --with-cpu
a15/a9 --with-mode arm/thumb.
Change-Id: I9e230e1b99dbdc9119b69ee858a89038c516a4ea
Signed-off-by: Vassilis Laganakos <vasileios.laganakos@arm.com>
The strategy for large block sizes is LDRD and STRD with offset addressing,
where the main loop copies 64 bytes in every iteration, (i.e., 8 calls to
LDRD and STRD pairs), interleaving load and stores (i.e., the pairs of LDRD
and STRD of the same data are consecutive instructions), and the writeback
of an updated address is a separate instruction, which allows us to write
back the accumulated update once per iteration.
This strategy is implemented in memcpy.S. In some configurations, a plain
version of memcpy (included from memcpy-stub.c) is used instead of the
optimized one.
Validation:
* Correctness: checked memcpy using a test harness for block sizes
ranging between 1 to 128, and source and destination buffers alignment
ranging in { 0,1,2,3,4,8,12 } bytes each.
* Performance: benchmarking on Cortex-A15 FPGA indicates that this strategy
is better for A15 than the strategy used by glibc and even slightly better
than using NEON. Benchmarking on Cortex-A9 bare metal and Linux shows
that the proposed strategy is reasonable: not as fast as the version of
memcpy from glibc (which is the best open source strategy for A9), but
comparable with csl and bionic.
* Integration with GCC: no regression for arm-none-eabi --with-cpu
cortex-a15 and cortex-a9.
Change-Id: Ied56354d8992c62ae3e02d582a2bd55585d814b9
Signed-off-by: Vassilis Laganakos <vasileios.laganakos@arm.com>
Fix the pthread_setname_np test to take into account that emulator kernels are
so old that they don't support setting the name of other threads.
The CLONE_DETACHED thread is obsolete since 2.5 kernels.
Rename kernel_id to tid.
Fix the signature of __pthread_clone.
Clean up the clone and pthread_setname_np implementations slightly.
Change-Id: I16c2ff8845b67530544bbda9aa6618058603066d
If r0 == 0, we're the child. If r0 > 0, we're the parent.
Otherwise set errno.
The __bionic_clone code I copy & pasted was wrong. This patch
fixes both.
Bug: 3461078
Change-Id: Ibb7d6cc7e54e666841f2f0dc59a141a0b31982e4
MIPS and x86 appear to have been correct already.
(Also fix unit tests that ASSERT_EQ with errno so that the
arguments are in the retarded junit order.)
Bug: 3461078
Change-Id: I2418ea98927b56e15b4ba9cfec97f5e7094c6291
There's now only one place where we deal with this stuff, it only needs to
be parsed once by the dynamic linker (rather than by each recipient), and it's
now easier for us to get hold of auxv data early on.
Change-Id: I6314224257c736547aac2e2a650e66f2ea53bef5
This brings us up to date with FreeBSD HEAD, fixes various bugs, unifies
the set of functions we support on ARM, MIPS, and x86, fixes "long double",
adds ISO C99 support, and adds basic unit tests.
It turns out that our "long double" functions have always been broken
for non-normal numbers. This patch fixes that by not using the upstream
implementations and just forwarding to the regular "double" implementation
instead (since "long double" on Android is just "double" anyway, which is
what BSD doesn't support).
All the tests pass on ARM, MIPS, and x86, plus glibc on x86-64.
Bug: 3169850
Bug: 8012787
Bug: https://code.google.com/p/android/issues/detail?id=6697
Change-Id: If0c343030959c24bfc50d4d21c9530052c581837
Replace a kernel header file dependency with files from NetBSD.
They're more complete, and ELF is ELF, whether you're on Linux or a BSD.
Bug: 7973611
Change-Id: I83ee719e7efdf432ec2ddbe8be271d05b2f558d7
We had two copies of the backtrace code, and two copies of the
libcorkscrew /proc/pid/maps code. This patch gets us down to one.
We also had hacks so we could log in the malloc debugging code.
This patch pulls the non-allocating "printf" code out of the
dynamic linker so everyone can share.
This patch also makes the leak diagnostics easier to read, and
makes it possible to paste them directly into the 'stack' tool (by
using relative PCs).
This patch also fixes the stdio standard stream leak that was
causing a leak warning every time tf_daemon ran.
Bug: 7291287
Change-Id: I66e4083ac2c5606c8d2737cb45c8ac8a32c7cfe8
In the default case, we don't need to use the stack, we can save r7 with
ip register (that what does eglibc).
This allow to fix vfork data corruption
(see 3884bfe966), because vfork now don't
use the stack.
If the platform code is compiled with -mcpu=cortex-a15, then without this
change prebuilt libraries built against -march=armv7 cannot resolve the
dependency on __aeabi_idiv (provided by libgcc.a).
Bug: 7961327
cherry-picked from internal master.
Change-Id: I8fe59a98eb53d641518b882523c1d6a724fb7e55
Add signalfd() call to bionic.
Adding the signalfd call was done in 3 steps:
- add signalfd4 system call (function name and syscall
number) to libc/SYSCALLS.TXT
- generate all necessary headers by calling
libc/tools/gensyscalls.py. This patch is adding
the generated files since the build system
does not call gensyscalls.py.
- create the signalfd wrapper in signalfd.cpp and add
the function prototype to sys/signalfd.h
(cherry-pick of 0c11611c11, modified to
work with older versions of GCC still in use on some branches.)
Change-Id: I4c6c3f12199559af8be63f93a5336851b7e63355
Previously we'd been relying on getting the machine-specific <endian.h>
instead of the top-level <endian.h>, and <sys/endian.h> was basically broken.
Now, with this patch and the previous patch we should have <endian.h>
and <sys/endian.h> behaving the same. This is basically how NetBSD's endian.h
works, and was probably how ours was originally intended to work.
Bug: http://code.google.com/p/android/issues/detail?id=39824
Change-Id: I71de5a507e633de166013a658b5764df9e1aa09c
The near duplicates upset fussier compilers that insist that
typedefs be exactly the same, but the fix isn't to make all
copies identical...
Change-Id: Icfdace41726f36ec33c9ae919dbb5a54d3529cc9
We'd manually hacked _BYTE_ORDER into the arm and mips "_types.h" headers,
but not into the x86 one. Judging by upstream, _BYTE_ORDER should be in
the "endian.h" headers instead, so let's uniformly do that.
I've also ironed out some of the other differences between the different
architectures' header files too.
Bug: http://code.google.com/p/android/issues/detail?id=39824
Change-Id: I19d3af7ffd74e1c02b1b6886aec0f0d11f44ab8d
Adds new code to function memset, optimized for Cortex A9.
Copyright (C) ST-Ericsson SA 2010
Added neon implementation
Author: Henrik Smiding henrik.smiding@stericsson.com for ST-Ericsson.
Change-Id: Id3c87767953439269040e15bd30a27aba709aef6
Signed-off-by: Christian Bejram <christian.bejram@stericsson.com>
Adds new code to memcpy function, optimized for Cortex A9.
Adds new ARM-only loop, for operations where source and
destination are aligned.
Copyright (C) ST-Ericsson SA 2010
Modified neon implementation to fit Cortex A9 cache line size,
for those running 32 bytes L2 cache line size.
Also split the implementation in aligned and unaligned access,
for those that allows unaligned memory access with Neon.
For totally aligned operations, arm-only code is used.
Change-Id: I95ebf6164cd6486b12a7e3e98e369db21e7e18d2
Author: Henrik Smiding henrik.smiding@stericsson.com for ST-Ericsson.
Signed-off-by: Christian Bejram <christian.bejram@stericsson.com>
This patch updates the C library headers to provide ucontext_t
definitions for three architectures.
+ Fix <signal.h> to always define 'struct sigcontext'.
The new declarations are announced with new macros defined in
<sys/cdefs.h> in order to make it easier to adapt client code
that already defines its own, incompatible, versions of the
structures seen here.
http://code.google.com/p/android/issues/detail?id=34784
Change-Id: Ie78c48690a4ce61c50593f6c39639be7fead3596
Add getsid() system call to bionic for
all architectures. This is needed for various tools
(e.g. perf).
Adding the getsid system call was done in 3 steps:
() add getsid system call (function name and syscall
number) to libc/SYSCALLS.TXT
() generate all necessary headers by calling
libc/tools/gensyscalls.py. This patch is adding
the generated files since the build system
does not call gensyscalls.py.
() add the system call signature to libc/include/unistd.h
Change-Id: Id69a257e13ec02e1a44085a6b217a3f19ab025b1
Signed-off-by: Irina Tirdea <irina.tirdea@intel.com>
Legacy ARM shared libraries use this generic version of atexit(),
which queues exit functions for invocation at program exit, at
which time the library may have been dlclose()'d, causing the
program to crash.
Change-Id: I41ae153c23268daa65ede7fb8966fc3e9caec369
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@gmail.com>
To properly support legacy ARM shared libraries, libc.so needs
to export the symbols __dso_handle and atexit, even though
these are now supplied by the crt startup code.
This patch reshuffles the existing CRT_LEGACY_WORKAROUND
conditionally compiled code slightly so it works as the
original author likely intended.
Change-Id: Id6c0e94dc65b7928324a5f0bad7eba6eb2f464b9
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@gmail.com>
The runtime linker parses the ELF section headers to
discover the size of the init_array and fini_array, so
there is no point in putting NULL terminators at the end.
Change-Id: I3246cd585efce9314155600277dd829e9f37d04f
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@gmail.com>
None of the supported ARCHs actually populate these sections,
so there is no point in keeping them in the binaries.
Change-Id: I21a364f510118ac1114e1b49c53ec8c895c6bc6b
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@gmail.com>
Useful if you're trying to defeat ASLR, otherwise not
so much ...
Change-Id: I17ebb50bb490a3967db9c3038f049adafe2b8ea7
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@gmail.com>
This header is used on bionic build and should be propagated into
sysroot on toolchain rebuild. Discussion re. this header is here:
http://gcc.gnu.org/ml/gcc-patches/2012-08/msg00936.html
It is available already in mips NDK platforms:
development/ndk/platforms/android-9/arch-mips/include/link.h
Change-Id: I39ff467cdac9f448e31c11ee3e14a6200e82ab57
Signed-off-by: Pavel Chupin <pavel.v.chupin@intel.com>
Add a GNU_STACK marker to crtend* files. This tells the linker
that these files do not require an executable stack.
When linking, a missing GNU_STACK marker in any .o file can prevent
the compiler from automatically marking the final executable as NX
safe (executable stack not required). In Android, we normally work
around this by adding -Wa,--noexecstack / -Wl,-z,noexecstack.
For files like crtend.S / crtend_so.S, which are included in every
executable / shared library, it's better to add the GNU_STACK note
directly to the assembly file. This allows the compiler to
automatically mark the final executable as NX safe without any
special command line options.
References: http://www.gentoo.org/proj/en/hardened/gnu-stack.xml
Change-Id: I07bd058f9f60ddd8b146e0fb36ba26ff84c0357d
Move the stackpointer so a captured signal does not corrupt
stack variables needed for __thread_entry.
Change-Id: I3e1e7b94a6d7cd3a07081f849043262743aa8064
Rewrite
crtbegin.S -> crtbegin.c
crtbegin_so.S -> crtbegin_so.c
This change allows us to generate PIC code without relying
on text relocations.
As a consequence of this rewrite, also rewrite
__dso_handle.S -> __dso_handle.c
__dso_handle_so.S -> __dso_handle_so.c
atexit.S -> atexit.c
In crtbegin.c _start, place the __PREINIT_ARRAY__, __INIT_ARRAY__,
__FINI_ARRAY__, and __CTOR_LIST__ variables onto the stack, instead of
passing a pointer to the text section of the binary.
This change appears sorta wonky, as I attempted to preserve,
as much as possible, the structure of the original assembly.
As a result, you have C files including other C files, and other
programming uglyness.
Result: This change reduces the number of files with text-relocations
from 315 to 19 on my Android build.
Before:
$ scanelf -aR $OUT/system | grep TEXTREL | wc -l
315
After:
$ scanelf -aR $OUT/system | grep TEXTREL | wc -l
19
Change-Id: Ib9f98107c0eeabcb606e1ddc7ed7fc4eba01c9c4
crtbegin_dynamic and crtbegin_static are essentially identical,
minus a few trivial differences (comments and whitespace).
Eliminate duplicates.
Change-Id: Ic9fae6bc9695004974493b53bfc07cd3bb904480
Adds new code to function memcmp, optimized for Cortex A9.
Copyright (C) ST-Ericsson SA 2010
Added neon optimization
Change-Id: I8864d277042db40778b33232feddd90a02a27fb0
Author: Henrik Smiding henrik.smiding@stericsson.com for ST-Ericsson.
Signed-off-by: Christian Bejram <christian.bejram@stericsson.com>
Modify the dynamic linker so that executables can be loaded
at locations other than 0x00000000.
Modify crtbegin* so that non-PIC compilant "thumb interwork
veneers" are not created by the linker.
Bug: 5323301
Change-Id: Iece0272e2b708c79034f302c20160e1fe9029588
Some SoCs that support NEON nevertheless perform better with a non-NEON than a
NEON memcpy(). This patch adds build variable ARCH_ARM_USE_NON_NEON_MEMCPY,
which can be set in BoardConfig.mk. When ARCH_ARM_USE_NON_NEON_MEMCPY is
defined, we compile in the non-NEON optimized memcpy() even if the SoC supports
NEON.
Change-Id: Ia0e5bee6bad5880ffc5ff8f34a1382d567546cf9
So that we can always get the full stack trace regardless of gcc's handling
of the "noreturn" attribute associated with abort().
(Cherry pick of Id264a5167e7cabbf11515fbc48f5469c527e34d4.)
Bug: 6455193
Conflicts:
libc/Android.mk
Change-Id: I568fc5303fd1d747075ca933355f914122f94dac
So that we can always get the full stack trace regardless of gcc's handling
of the "noreturn" attribute associated with abort().
[cherry-picked from master]
BUG:6455193
Change-Id: I0102355f5bf20e636d3feab9d1424495f38e39e2
ARM Cortex A8 use 64 bytes and ARM Cortex A9 use 32 bytes cache line
size.
The following patch:
Adds code to adjust memcpy cache line size to match A9 cache line
size.
Adds a flag to select between 32 bytes and 64 bytes cache line
size.
Copyright (C) ST-Ericsson SA 2010
Modified neon implementation to fit Cortex A9 cache line size
Author: Henrik Smiding henrik.smiding@stericsson.com for
ST-Ericsson.
Change-Id: I8a55946bfb074e6ec0a14805ed65f73fcd0984a3
Signed-off-by: Christian Bejram <christian.bejram@stericsson.com>
The END macro was put too far down which made the linker complain about
it. Move up to the end of the code.
Change-Id: Ica71a9c6083b437d2213c7cefe34b0083c78f16b
Marking segments read-only was pushing the alignment of __on_dlclose by
2 bytes making it unaligned. This change makes sure the ARM code is
aligned to the 4 byte boundary.
Bug: 6313309
Change-Id: Ic2bf475e120dd61225ec19e5d8a9a8b1d0b7f081
This change fixes a segmentation fault in the libc unwinder when it goes
past __libc_init.
Unwind instructions for __libc_init direct it to grab the return address from
the stack frame. Without this change, the unwinder gets a wild address and
looks up further unwind instructions for the routine at that address. If it's
unlucky enough to hit an existing function, it will try to unwind it. Bad
things happen then.
With this change, the return address always points to the _start function,
which does not have unwind instructions associated with it. This stop the
unwind process.
__libc_init never returns, so this does not affect program execution, other
than adding 4 bytes on the main thread stack.
Change-Id: Id58612172e8825c8729cccd081541a13bff96bd0
Use the same pattern in atexit.S to reference __dso_handle in a way that
doesn't require a TEXTREL flag to be set.
Change-Id: Id69d20863ee203d2b2f7ef0db230f9b548657741
Some platform libraries built for ICS do not work with master
because of some refactoring in frameworks/base.
Make sure that these libgcc symbols are always present in our libc
Change-Id: Ib8d345878be0ba711f051082a778f5cc1f1b3a19
Signed-off-by: Dima Zavin <dima@android.com>
This patch defines a few new macros that can be used to control the
visibility of symbols exported by the C library:
- ENTRY_PRIVATE() can be used in assembly sources to indicate
that an assembler function should have "hidden" visibility, i.e.
will never be exported by the C library's shared library.
This is the equivalent of using __LIBC_HIDDEN__ for a C function,
but ENTRY_PRIVATE() works like ENTRY(), and must be used with
END() to tag the end of the function.
- __LIBC_ABI_PUBLIC__ can be used to tag a C functions as being
part of the C library's public ABI. This is important for a
few functions that must be exposed by the NDK to maintain
binary compatibility.
Once a symbol has been tagged with this macro, it shall
*never* be removed from the library, even if it becomes
directly unused due to implementation changes
(e.g. __is_threaded).
- __LIBC_ABI_PRIVATE__ can be used for C functions that should
always be exported by the C library because they are used by
other libraries in the platform, but should not be exposed
by the NDK. It is possible to remove such symbols from the
implementation if all callers are also modified.
+ Add missing END() assembly macro for x86
Change-Id: Ia96236ea0dbec41d57bea634b39d246b30e5e234
The xattr system calls are required for the SE Android userspace in
order to get and set file security contexts. In particular, libselinux
requires these calls.
Change-Id: I78f5eb3d8f3384aed0a5e7c6a6f001781d982017
Add bionic libc to support readahead system call.
This is needed to enable sreadahead to work.
Change-Id: I3856e1a3833db82e6cf42fd34af7631bd40cc723
Author: Winson Yung <winson.w.yung@intel.com>
Signed-off-by: Bruce Beare <bruce.j.beare@intel.com>
The function must be named __atomic_cmpxchg, not __android_cmpxchg.
This typo broke existing prebuilt binaries (they couldn't be loaded
at runtime anymore).
Change-Id: I25ca7d18329817f0056e616a0409113269ad7b1f
Use tgkill instead of tkill to implement pthread_kill.
This is safer in the event that the thread has already terminated
and its id has been reused by a different process.
Change-Id: Ied715e11d7eadeceead79f33db5e2b5722954ac9
__atomic_cmpxchg and other related atomic operations did not
provide memory barriers, which can be a problem for non-platform
code that links against them when it runs on multi-core devices.
This patch does two things to fix this:
- It modifies the existing implementation of the functions
that are exported by the C library to always provide
full memory barriers. We need to keep them exported by
the C library to prevent breaking existing application
machine code.
- It also modifies <sys/atomics.h> to only export
always-inlined versions of the functions, to ensure that
any application code compiled against the new header will
not rely on the platform version of the functions.
This ensure that said machine code will run properly on
all multi-core devices.
This is based on the GCC built-in sync primitives.
The end result should be only slightly slower than the
previous implementation.
Note that the platform code does not use these functions
at all. A previous patch completely removed their usage in
the pthread and libstdc++ code.
+ rename arch-arm/bionic/atomics_arm.S to futex_arm.S
+ rename arch-x86/bionic/atomics_x86.S to futex_x86.S
+ remove arch-x86/include/sys/atomics.h which already
provided inlined functions to the x86 platform.
Change-Id: I752a594475090cf37fa926bb38209c2175dda539
Modify the dynamic linker so that executables can be loaded
at locations other than 0x00000000.
Modify crtbegin* so that non-PIC compilant "thumb interwork
veneers" are not created by the linker.
Bug: 5323301
Change-Id: Iece0272e2b708c79034f302c20160e1fe9029588
Without this change strcmp size is zero (not set), and it gets
ignored by Valgrind. Changes to memcpy and atexit don't affect the
generated binary in any way.
Change-Id: I05818cb5951f75901dc8c0eef02807a2e83a9231
This patch ensure that __aeabi_f2uiz is embedded in our C library.
This is needed to avoid breaking certain applications when they are
loaded in ICS. It is likely that the issue is due to mis-linked
binaries generated with the stand-alone toolchain (the problem
should not exist if you use ndk-build), but this fix is easier
than asking all app developers to fix their custom build system.
If you want more technical details, read the comments inside
libgcc_compat.c
Change-Id: I59ac1fc781ecb70b90b5573c5a3c67560ca8f270
Unfortunately, legacy .so files for ARM don't have a correct crtbegin file.
Consequently, we have to grandfather the old __dso_handle behaviour.
Add some ifdefs for ARM to allow it to use the old code until we can work
out a transition.
Change-Id: I6a28f368267d792c94e1d985d8344023bc632f6f
Author: H.J. Lu <hongjiu.lu@intel.com>
Signed-off-by: Bruce Beare <bruce.j.beare@intel.com>
Reference results of the experiments on TI OMAP3430 at 600 MHz
$ bench_strcmp -N "strcmp_1k" -s 1k -I 200
[original C code]
prc thr usecs/call samples errors cnt/samp size
strcmp_1k 1 1 10.38000 102 0 15000 1024
[ARM optimized code]
prc thr usecs/call samples errors cnt/samp size
strcmp_1k 1 1 3.08840 88 0 15000 1024
The work was derived from ARM Ltd, contributed to newlib, and reworked
for Android by Linaro.
Change-Id: Ib0d5755e1eb9adb07d80ef0252f57a5c4c57a425
Signed-off-by: Jim Huang <jserv@0xlab.org>
The kernel has supported this syscall for quite some time now,
but bionic did not. Now that there is a need for it, let's
add it to bionic.
Change-Id: Ifcef3e46f1438d79435b600c4e6063857ab16903
Used the new gensyscalls.py script to update the ARM syscalls with the
BEGIN(x) and END(x) macros to give size information for the code of the
syscall. Useful for valgrind.
Change-Id: I8c481c8928401ac110fd19b087f7d67e4db39326
Add a macro to annotate function end and start using both ENTRY and END
for each function. This allows valgrind (and presumably other debugging
tools) to use the debug symbols to trace the functions.
Change-Id: I5f09cef8e22fb356eb6f5cee952b031e567599b6
This patch improves the handling of 64-bit parameters in syscalls on ARM.
The ARM EABI mandates that 64-bit quantities be passed in even/odd register
pairs, which requires special treatment.
This allows us to simplify our implementations of pread() and pwrite()
and remove the C stubs for pread64() and pwrite64().
Also add ftruncate64() to <unistd.h>
Change-Id: I407e2fd223ba0093dd2d0b04c6152fadfc9ce3ef
Bug 3107933
This adds the cpu_set_t type definition and related functions
and macros used to handle CPU thread affinity.
sched_getcpu()
sched_setaffinity()
sched_getaffinity()
Change-Id: If382ecafde8926341a88478062b275553645065b
This elimates the makefile warning at the beginning of each build:
libc_common_intermediates/arch-arm/syscalls/eventfd.o' given more than once in the same rule.
Change-Id: I25cc0c373d55b5036dd17811a595c097fd6ca69a
The change explicitly isolates the assembly-only macros in header
<machine/cpu-features.h> in order to prevent mis-inclusion in C/C++
source files.
Change-Id: I0258e87c5ac3fd24944fb227290ac3b9cac4bfba
Merge commit 'bd8d987b3c3aa6d9d00cede2cb091f00bdb42204' into gingerbread-plus-aosp
* commit 'bd8d987b3c3aa6d9d00cede2cb091f00bdb42204':
libc: remove C++ comments from public headers.
Merge commit '3cf53d1a7814e1520df09d24b009c16f4f27db0d' into gingerbread-plus-aosp
* commit '3cf53d1a7814e1520df09d24b009c16f4f27db0d':
Fixes for the ARM-specific bswap_16, bswap_32, and bswap_64.
1. Make the feature test work by excluding known-deficient processors, so
we don't have to maintain a complete list of all the processors that support
REV and REV16.
2. Don't abuse 'register' to get an effect similar to GCC's +l constraint,
but which was unnecessarily restrictive.
3. Fix __swap64md so _x isn't clobbered, breaking 64-bit swaps.
4. Make <byteswap.h> (which declars bswap_16 and friends) use <endian.h>
rather than <sys/endian.h>, so we get the machine-dependent implementations.
Change-Id: I6a38fad7a9fbe394aff141489617eb3883e1e944
Merge commit '312be567a03aaf851707a268807ee666b12f8c74' into gingerbread-plus-aosp
* commit '312be567a03aaf851707a268807ee666b12f8c74':
Use ARMv6 instruction for handling byte order
Merge commit '3a1bbee36b773862e88c6f1895b607c0cd81b499' into gingerbread-plus-aosp
* commit '3a1bbee36b773862e88c6f1895b607c0cd81b499':
Add eventfd system call support
ARMv6 ISA has several instructions to handle data in different byte order.
For endian conversion (byte swapping) of single data words, it might be a
good idea to use the REV/REV16 instruction simply.
Change-Id: Ic4a5ed6254e082763e54aa70d428f59a0088636e
With this patch _and_ an upcoming build/ patch, the destruction
of static C++ objects contained in shared libraries will happen
properly when dlclose() is called.
Note that this change introduces crtbegin_so.S and crtend_so.S which
are currently ignored by the build system.
+ move definition of __dso_handle to the right place
(before that, all shared libraries used the __dso_handle
global variable from the C library).
Note that we keep a 'weak' __dso_handle in aeabi.c to avoid
breaking the build until the next patch to build/core/combo/
appears. We will be able to remove that later.
+ move bionic/aeabi.c to arch-arm/bionic/ (its proper location)
NOTE: The NDK will need to be modified to enable this feature in
the shared libraries that are generated through it.
Change-Id: I99cd801375bbaef0581175893d1aa0943211b9bc
With this patch, _and_ an upcoming build/ patch, the destruction
of static C++ objects contained in shared libraries will happen
properly when dlclose() is called.
Note that this change introduces crtbegin_so.S and crtend_so.S which
are currently ignored by the build system.
+ move definition of __dso_handle to the right place
(before that, all shared libraries used the __dso_handle
global variable from the C library).
Note that we keep a 'weak' __dso_handle in aeabi.c to avoid
breaking the build until the next patch to build/core/combo/
appears. We will be able to remove that later.
+ move bionic/aeabi.c to arch-arm/bionic/ (its proper location)
Change-Id: Ie771aa204e3acbdf02fd30ebd4150373a1398f39
NOTE: The NDK will need to be modified to enable this feature in
the shared libraries that are generated through it.
Update ARM atomic ops to use LDREX/STREX. Stripped out #if 0 chunk.
Insert explicit memory barriers in pthread and semaphore code.
For bug 2721865.
Change-Id: I0f153b797753a655702d8be41679273d1d5d6ae7
Added an underscore to _ARM_HAVE_LDREX_STREX to make it match the others.
Added __ARM_HAVE_DMB and __ARM_HAVE_LDREXD when appropriate.
Fixed some typos.
Change-Id: I2f55febcff4aeb7de572a514fb2cd2f820dca27c
GDB looks for specific opcode sequences when trying to recognize a stack
frame as a signal trampoline. The sequences it looks for happen to be those
created when SA_RESTORER is set, since glibc always sets a restorer. This
patch does the same here, so that the trampolines can be correctly identified.
Change-Id: I0ac574a68818cb24d939c3527f3aaeb04b853d04
This does not change the implementation of conditional variables
since we're waiting for other system components to properly use
pthread_condattr_init/setpshared before that.
Also remove an obsolete x86 source file.
Change-Id: Ia3e3fbac35b87a534fb04d4381c3c66b975bc8f7
