This gives us:
* <dirent.h>
struct dirent64
readdir64, readdir64_r, alphasort64, scandir64
* <fcntl.h>
creat64, openat64, open64.
* <sys/stat.h>
struct stat64
fstat64, fstatat64, lstat64, stat64.
* <sys/statvfs.h>
struct statvfs64
statvfs64, fstatvfs64.
* <sys/vfs.h>
struct statfs64
statfs64, fstatfs64.
This also removes some of the incorrect #define hacks we've had in the
past (for stat64, for example, which we promised to clean up way back
in bug 8472078).
Bug: 11865851
Bug: 8472078
Change-Id: Ia46443521918519f2dfa64d4621027dfd13ac566
1. Moved arch-specific setup to their own files:
- <arch>/<arch>.mk, arch-specific configs. Variables in those config
end with the arch name.
- removed the extra complexity introduced by function libc-add-cpu-variant-src,
which seems to be not very useful these days.
2. Separated out the crt object files generation rules and set up the
rules for both TARGET_ARCH and TARGET_2ND_ARCH.
3. Build all the libraries for both TARGET_ARCH and TARGET_2ND_ARCH,
with the arch-specific LOCAL_ variables.
Bug: 11654773
Change-Id: I9c2d85db0affa49199d182236d2210060a321421
Remove the linker's reliance on BSD cruft and use the glibc-style
ElfW macro. (Other code too, but the linker contains the majority
of the code that needs to work for Elf32 and Elf64.)
All platforms need dl_iterate_phdr_static, so it doesn't make sense
to have that part of the per-architecture configuration.
Bug: 12476126
Change-Id: I1d7f918f1303a392794a6cd8b3512ff56bd6e487
Most of <machine/_types.h> was either unused, wrong, or identical across
all 32-/64-bit architectures.
I'm not a huge fan of <sys/_types.h> either, but moving the bits we need
up into there is a step forward.
Bug: 12213562
Change-Id: Id13551c78966e324beee2dd90c5575e37d2a71e6
This is required to make the Nexus 10 graphics driver work on a system
compiled with gcc 4.9.
Change-Id: If3f3d488652a736d9ea3e583548d74fae3ffa902
Signed-off-by: Bernhard Rosenkränzer <Bernhard.Rosenkranzer@linaro.org>
The situation here is a bit confusing. On 64-bit, rlimit and rlimit64 are
the same, and so getrlimit/getrlimit64, setrlimit/setrlimit64,
and prlimit/prlimit64 are all the same. On 32-bit, rlimit and rlimit64 are
different. 32-bit architectures other than MIPS go one step further by having
an even more limited getrlimit system call, so arm and x86 need to use
ugetrlimit instead of getrlimit. Worse, the 32-bit architectures don't have
64-bit getrlimit- and setrlimit-equivalent system calls, and you have to use
prlimit64 instead. There's no 32-bit prlimit system call, so there's no
easy implementation of that --- what should we do if the result of prlimit64
won't fit in a struct rlimit? Since 32-bit survived without prlimit/prlimit64
for this long, I'm not going to bother implementing prlimit for 32-bit.
We need the rlimit64 functions to be able to build strace 4.8 out of the box.
Change-Id: I1903d913b23016a2fc3b9f452885ac730d71e001
This patch switches to using the uapi constants. It also adds the missing
setns system call, fixes sched_getcpu's error behavior, and fixes the
gensyscalls script now ARM is uapi-only too.
Change-Id: I8e16b1693d6d32cd9b8499e46b5d8b0a50bc4f1d
Also make the other architectures more similar to one another,
use NULL instead of 0 in calling code, and remove an unused #define.
Change-Id: I52b874afb6a351c802f201a0625e484df6d093bb
This is needed if we use Clang to compile Bionic, which won't include
__popcountsi2 anymore as Clang generates inline instructions. However
prebuilt binary blobs still depend on libc.so to resolve __popcountsi2.
Change-Id: I9001a3884c4be250c0ceebcd79922783fae1a0b7
This is a better solution than the old __warn_references because it's
a compile-time rather than link-time warning, it doesn't rely on something
that doesn't appear to be supported by gold (which is why you only used
to see these warnings on mips builds), and the errors refer to the exact
call site(s) rather than just telling you which object file contains a
reference to the bad function.
This is primarily so we can build bionic for aarch64; building libc.so
caused these warnings to fire (because link time is the wrong time) and
warnings are errors.
Change-Id: I5df9281b2a9d98b164a9b11807ea9472c6faa9e3
Since the ENTRY/END macros now have .cfi_startproc/.cfi_endproc, most of the
custom arm assembly has no unwind information. Adding the proper cfi directives
for these and removing the arm directives.
Update the gensyscalls.py script to add these cfi directives for the generated
assembly. Also fix the references to non-uapi headers to the proper uapi
header.
In addition, remove the kill.S, tkill.S, tgkill.S for arm since they are not
needed at all. The unwinder (libunwind) is able to properly unwind using the
normal abort.
After this change, I can unwind through the system calls again.
Bug: 11559337
Bug: 11825869
Bug: 11321283
Change-Id: I18b48089ef2d000a67913ce6febc6544bbe934a3
The kernel now maintains the pthread_internal_t::tid field for us,
and __clone was only used in one place so let's inline it so we don't
have to leave such a dangerous function lying around. Also rename
files to match their content and remove some useless #includes.
Change-Id: I24299fb4a940e394de75f864ee36fdabbd9438f9
Let the kernel keep pthread_internal_t::tid updated, including
across forks and for the main thread. This then lets us fix
pthread_join to only return after the thread has really exited.
Also fix the thread attributes of the main thread so we don't
unmap the main thread's stack (which is really owned by the
dynamic linker and contains things like environment variables),
which fixes crashes when joining with an exited main thread
and also fixes problems reported publicly with accessing environment
variables after the main thread exits (for which I've added a new
unit test).
In passing I also fixed a bug where if the clone(2) inside
pthread_create(3) fails, we'd unmap the child's stack and TLS (which
contains the mutex) and then try to unlock the mutex. Boom! It wasn't
until after I'd uploaded the fix for this that I came across a new
public bug reporting this exact failure.
Bug: 8206355
Bug: 11693195
Bug: https://code.google.com/p/android/issues/detail?id=57421
Bug: https://code.google.com/p/android/issues/detail?id=62392
Change-Id: I2af9cf6e8ae510a67256ad93cad891794ed0580b
We only need it for MAX_ERRNO, and it's time we had somewhere to put
the little assembler utility macros we've been putting off writing.
Change-Id: I9354d2e0dc47c689296a34b5b229fc9ba75f1a83
In practice, thanks to all the registers the stubs don't actually change,
but it's confusing to have an incorrect declaration.
I suspect that fcntl remains broken for aarch64; it happens to work for
x86_64 because the first vararg argument gets placed in the right register
anyway, but I have no reason to believe that's true for aarch64.
This patch adds a unit test, though, so we'll be able to tell when we get
as far as running the unit tests.
Change-Id: I58dd0054fe99d7d51d04c22781d8965dff1afbf3
I've left the exit_group syscall as _exit because otherwise we'd have to
convince the compiler that our _exit (which just calls __exit_group) is
actually "noreturn", and it seems like that would be less clean than just
cutting out the middleman.
We'll just have to trust ourselves not to add anything to SYSCALLS.TXT
that ought to be private but that only has a single leading underscore.
Hopefully we can manage that.
Change-Id: Iac47faea9f516186e1774381846c54cafabc4354
(aarch64 kernels only have the newer system calls.)
Also expose the new functionality that's exposed by glibc in our header files.
Change-Id: I45d2d168a03f88723d1f7fbf634701006a4843c5
Modern architectures only get the *at(2) system calls. For example,
aarch64 doesn't have open(2), and expects userspace to use openat(2)
instead.
Change-Id: I87b4ed79790cb8a80844f5544ac1a13fda26c7b5
Also clean up <signal.h> and revert the hacks that were necessary
for 64-bit in linker/debugger.cpp until now.
Change-Id: I3b0554ca8a49ee1c97cda086ce2c1954ebc11892
For some reason the new cortex-a15 memcpy code from ARM is really bad
for really large copies. This change forces us to go down the old path
for all copies.
All of my benchmarks show the new version is faster for large copies, but
something is going on that I don't understand.
Bug: 10838353
Change-Id: I01c16d4a2575e76f4c69862c6f78fd9024eb3fb8
I originally modified the krait mainloop prefetch from cacheline * 8 to * 2.
This causes a perf degradation for copies bigger than will fit in the cache.
Fixing this back to the original * 8. I tried other multiples, but * 8 is th
sweet spot on krait.
Bug: 11221806
Change-Id: I1f75fad6440f7417e664795a6e7b5616f6a29c45
Let's have both use rt_sigprocmask, like in glibc. The 64-bit ABIs
can share the same code as the 32-bit ABIs.
Also, let's test the return side of these calls, not just the
setting.
Bug: 11069919
Change-Id: I11da99f85b5b481870943c520d05ec929b15eddb
The x86_64 build was failing because clone.S had a call to __thread_entry which
was being added to a different intermediate .a on the way to making libc.so,
and the linker couldn't guarantee statically that such a relocation would be
possible.
ld: error: out/target/product/generic_x86_64/obj/STATIC_LIBRARIES/libc_common_intermediates/libc_common.a(clone.o): requires dynamic R_X86_64_PC32 reloc against '__thread_entry' which may overflow at runtime; recompile with -fPIC
This patch addresses that by ensuring that the caller and callee end up in the
same intermediate .a. While I'm here, I've tried to clean up some of the mess
that led to this situation too. In particular, this removes libc/private/ from
the default include path (except for the DNS code), and splits out the DNS
code into its own library (since it's a weird special case of upstream NetBSD
code that's diverged so heavily it's unlikely ever to get back in sync).
There's more cleanup of the DNS situation possible, but this is definitely a
step in the right direction, and it's more than enough to get x86_64 building
cleanly.
Change-Id: I00425a7245b7a2573df16cc38798187d0729e7c4
We shouldn't have been passing the bottom 32 bits of the address used
for pthread_join to the kernel.
Change-Id: I487e5002d60c27adba51173719213abbee0f183f
This is basically the other half of I5de76f6c46ac87779f207d568a86bb453e2414de
from Pavel Chupin <pavel.v.chupin@intel.com>, but taking the exact upstream
_types.h instead of the modified version. (I was confused when I suggested
otherwise.)
I've also cleaned up the internal_types.h situation; we weren't gaining
anything from these empty files, and there is no upstream internal_types.h
for x86_64.
Change-Id: I802a9a6a8df1c979e820659212c75a47c2ef392e
memcpy.a15.S/strcmp.a15.S files were submitted by ARM for use as the basis
for the memcpy/strcmp implementations in cortex-a15.
memset.S was moved in to the generic directory.
NOTE: memcpy.a9.S was submitted by Linaro to be the basis for the memcpy
for cortex-a9/cortex-a15 but has not been incorporated yet.
Bug: 10971279
Merge from internal master.
(cherry-picked from 48fc3e8b9f)
Change-Id: I8f9297578990d517f004e4e8840e2b2cbd5a47d8
The check for __ARM_FEATURE_DSP being defined is pointless since it
is always defined.
Bug: 10971279
Merge from internal master.
(cherry-picked from d2642fa70c)
Change-Id: If23ab3271f4da0c38cd531ffdc9a7e5eed6ec5dc
Much of the per-architecture duplication can be removed, so let's do so
before we add the 64-bit architectures.
Change-Id: Ieb796503c8e5353ea38c3bab768bb9a690c9a767
I accidentally did a signed comparison of the size_t values passed in
for three of the _chk functions. Changing them to unsigned compares.
Add three new tests to verify this failure is fixed.
Bug: 10691831
Merge from internal master.
(cherry-picked from 883ef2499c)
Change-Id: Id9a96b549435f5d9b61dc132cf1082e0e30889f5
The backtrace when a fortify check failed was not correct. This change
adds all of the necessary directives to get a correct backtrace.
Fix the strcmp directives and change all labels to local labels.
Testing:
- Verify that the runtime can decode the stack for __memcpy_chk, __memset_chk,
__strcpy_chk, __strcat_chk fortify failures.
- Verify that gdb can decode the stack properly when hitting a fortify check.
- Verify that the runtime can decode the stack for a seg fault for all of the
_chk functions and for memcpy/memset.
- Verify that gdb can decode the stack for a seg fault for all of the _chk
functions and for memcpy/memset.
- Verify that the runtime can decode the stack for a seg fault for strcmp.
- Verify that gdb can decode the stack for a seg fault in strcmp.
Bug: 10342460
Bug: 10345269
Merge from internal master.
(cherry-picked from 05332f2ce7)
Change-Id: Ibc919b117cfe72b9ae97e35bd48185477177c5ca
The libcorkscrew stack unwinder does not understand cfi directives,
so add .save directives so that it can function properly.
Also add the directives in to strcmp.S and fix a missing set of
directives in cortex-a9/memcpy_base.S.
Bug: 10345269
Merge from internal master.
(cherry-picked from 5f7ccea3ff)
Change-Id: If48a216203216a643807f5d61906015984987189
I accidentally did a signed comparison of the size_t values passed in
for three of the _chk functions. Changing them to unsigned compares.
Add three new tests to verify this failure is fixed.
Bug: 10691831
Change-Id: Ia831071f7dffd5972a748d888dd506c7cc7ddba3
The backtrace when a fortify check failed was not correct. This change
adds all of the necessary directives to get a correct backtrace.
Fix the strcmp directives and change all labels to local labels.
Testing:
- Verify that the runtime can decode the stack for __memcpy_chk, __memset_chk,
__strcpy_chk, __strcat_chk fortify failures.
- Verify that gdb can decode the stack properly when hitting a fortify check.
- Verify that the runtime can decode the stack for a seg fault for all of the
_chk functions and for memcpy/memset.
- Verify that gdb can decode the stack for a seg fault for all of the _chk
functions and for memcpy/memset.
- Verify that the runtime can decode the stack for a seg fault for strcmp.
- Verify that gdb can decode the stack for a seg fault in strcmp.
Bug: 10342460
Bug: 10345269
Change-Id: I1dedadfee207dce4a285e17a21e8952bbc63786a
The libcorkscrew stack unwinder does not understand cfi directives,
so add .save directives so that it can function properly.
Also add the directives in to strcmp.S and fix a missing set of
directives in cortex-a9/memcpy_base.S.
Bug: 10345269
Change-Id: I043f493e0bb6c45bd3f4906fbe1d9f628815b015
This change pulls the memcpy code out into a new file so that the
__strcpy_chk and __strcat_chk can use it with an include.
The new versions of the two chk functions uses assembly versions
of strlen and memcpy to implement this check. This allows near
parity with the assembly versions of strcpy/strcat. It also means that
as memcpy implementations get faster, so do the chk functions.
Other included changes:
- Change all of the assembly labels to local labels. The other labels
confuse gdb and mess up backtracing.
- Add .cfi_startproc and .cfi_endproc directives so that gdb is not
confused when falling through from one function to another.
- Change all functions to use cfi directives since they are more powerful.
- Move the memcpy_chk fail code outside of the memcpy function definition
so that backtraces work properly.
- Preserve lr before the calls to __fortify_chk_fail so that the backtrace
actually works.
Testing:
- Ran the bionic unit tests. Verified all error messages in logs are set
correctly.
- Ran libc_test, replacing strcpy with __strcpy_chk and replacing
strcat with __strcat_chk.
- Ran the debugger on nexus10, nexus4, and old nexus7. Verified that the
backtrace is correct for all fortify check failures. Also verify that
when falling through from __memcpy_chk to memcpy that the backtrace is
still correct. Also verified the same for __memset_chk and bzero.
Verified the two different paths in the cortex-a9 memset routine that
save variables to the stack still show the backtrace properly.
Bug: 9293744
(cherry-picked from 2be91915dc)
Change-Id: Ia407b74d3287d0b6af0139a90b6eb3bfaebf2155
This change creates assembler versions of __memcpy_chk/__memset_chk
that is implemented in the memcpy/memset assembler code. This change
avoids an extra call to memcpy/memset, instead allowing a simple fall
through to occur from the chk code into the body of the real
implementation.
Testing:
- Ran the libc_test on __memcpy_chk/__memset_chk on all nexus devices.
- Wrote a small test executable that has three calls to __memcpy_chk and
three calls to __memset_chk. First call dest_len is length + 1. Second
call dest_len is length. Third call dest_len is length - 1.
Verified that the first two calls pass, and the third fails. Examined
the logcat output on all nexus devices to verify that the fortify
error message was sent properly.
- I benchmarked the new __memcpy_chk and __memset_chk on all systems. For
__memcpy_chk and large copies, the savings is relatively small (about 1%).
For small copies, the savings is large on cortex-a15/krait devices
(between 5% to 30%).
For cortex-a9 and small copies, the speed up is present, but relatively
small (about 3% to 5%).
For __memset_chk and large copies, the savings is also small (about 1%).
However, all processors show larger speed-ups on small copies (about 30% to
100%).
Bug: 9293744
Merge from internal master.
(cherry-picked from 7c860db074)
Change-Id: I916ad305e4001269460ca6ebd38aaa0be8ac7f52
This change pulls the memcpy code out into a new file so that the
__strcpy_chk and __strcat_chk can use it with an include.
The new versions of the two chk functions uses assembly versions
of strlen and memcpy to implement this check. This allows near
parity with the assembly versions of strcpy/strcat. It also means that
as memcpy implementations get faster, so do the chk functions.
Other included changes:
- Change all of the assembly labels to local labels. The other labels
confuse gdb and mess up backtracing.
- Add .cfi_startproc and .cfi_endproc directives so that gdb is not
confused when falling through from one function to another.
- Change all functions to use cfi directives since they are more powerful.
- Move the memcpy_chk fail code outside of the memcpy function definition
so that backtraces work properly.
- Preserve lr before the calls to __fortify_chk_fail so that the backtrace
actually works.
Testing:
- Ran the bionic unit tests. Verified all error messages in logs are set
correctly.
- Ran libc_test, replacing strcpy with __strcpy_chk and replacing
strcat with __strcat_chk.
- Ran the debugger on nexus10, nexus4, and old nexus7. Verified that the
backtrace is correct for all fortify check failures. Also verify that
when falling through from __memcpy_chk to memcpy that the backtrace is
still correct. Also verified the same for __memset_chk and bzero.
Verified the two different paths in the cortex-a9 memset routine that
save variables to the stack still show the backtrace properly.
Bug: 9293744
Change-Id: Id5aec8c3cb14101d91bd125eaf3770c9c8aa3f57
(cherry picked from commit 2be91915dc)
Create one version of strcat/strcpy/strlen for cortex-a15/krait and another
version for cortex-a9.
Tested with the libc_test strcat/strcpy/strlen tests.
Including new tests that verify that the src for strcat/strcpy do not
overread across page boundaries.
NOTE: The handling of unaligned strcpy (same code in strcat) could probably
be optimized further such that the src is read 64 bits at a time instead of
the partial reads occurring now.
strlen improves slightly since it was recently optimized.
Performance improvements for strcpy and strcat (using an empty dest string):
cortex-a9
- Small copies vary from about 5% to 20% as the size gets above 10 bytes.
- Copies >= 1024, about a 60% improvement.
- Unaligned copies, from about 40% improvement.
cortex-a15
- Most small copies exhibit a 100% improvement, a few copies only
improve by 20%.
- Copies >= 1024, about 150% improvement.
- Unaligned copies, about 100% improvement.
krait
- Most small copies vary widely, but on average 20% improvement, then
the performance gets better, hitting about a 100% improvement when
copies 64 bytes of data.
- Copies >= 1024, about 100% improvement.
- When coping MBs of data, about 50% improvement.
- Unaligned copies, about 90% improvement.
As strcat destination strings get larger in size:
cortex-a9
- about 40% improvement for small dst strings (>= 32).
- about 250% improvement for dst strings >= 1024.
cortex-a15
- about 200% improvement for small dst strings (>=32).
- about 250% improvement for dst strings >= 1024.
krait
- about 25% improvement for small dst strings (>=32).
- about 100% improvement for dst strings >=1024.
Merge from internal master.
(cherry-picked from d119b7b6f4)
Change-Id: I296463b251ef9fab004ee4dded2793feca5b547a
This change creates assembler versions of __memcpy_chk/__memset_chk
that is implemented in the memcpy/memset assembler code. This change
avoids an extra call to memcpy/memset, instead allowing a simple fall
through to occur from the chk code into the body of the real
implementation.
Testing:
- Ran the libc_test on __memcpy_chk/__memset_chk on all nexus devices.
- Wrote a small test executable that has three calls to __memcpy_chk and
three calls to __memset_chk. First call dest_len is length + 1. Second
call dest_len is length. Third call dest_len is length - 1.
Verified that the first two calls pass, and the third fails. Examined
the logcat output on all nexus devices to verify that the fortify
error message was sent properly.
- I benchmarked the new __memcpy_chk and __memset_chk on all systems. For
__memcpy_chk and large copies, the savings is relatively small (about 1%).
For small copies, the savings is large on cortex-a15/krait devices
(between 5% to 30%).
For cortex-a9 and small copies, the speed up is present, but relatively
small (about 3% to 5%).
For __memset_chk and large copies, the savings is also small (about 1%).
However, all processors show larger speed-ups on small copies (about 30% to
100%).
Bug: 9293744
Change-Id: I8926d59fe2673e36e8a27629e02a7b7059ebbc98
Create one version of strcat/strcpy/strlen for cortex-a15/krait and another
version for cortex-a9.
Tested with the libc_test strcat/strcpy/strlen tests.
Including new tests that verify that the src for strcat/strcpy do not
overread across page boundaries.
NOTE: The handling of unaligned strcpy (same code in strcat) could probably
be optimized further such that the src is read 64 bits at a time instead of
the partial reads occurring now.
strlen improves slightly since it was recently optimized.
Performance improvements for strcpy and strcat (using an empty dest string):
cortex-a9
- Small copies vary from about 5% to 20% as the size gets above 10 bytes.
- Copies >= 1024, about a 60% improvement.
- Unaligned copies, from about 40% improvement.
cortex-a15
- Most small copies exhibit a 100% improvement, a few copies only
improve by 20%.
- Copies >= 1024, about 150% improvement.
- Unaligned copies, about 100% improvement.
krait
- Most small copies vary widely, but on average 20% improvement, then
the performance gets better, hitting about a 100% improvement when
copies 64 bytes of data.
- Copies >= 1024, about 100% improvement.
- When coping MBs of data, about 50% improvement.
- Unaligned copies, about 90% improvement.
As strcat destination strings get larger in size:
cortex-a9
- about 40% improvement for small dst strings (>= 32).
- about 250% improvement for dst strings >= 1024.
cortex-a15
- about 200% improvement for small dst strings (>=32).
- about 250% improvement for dst strings >= 1024.
krait
- about 25% improvement for small dst strings (>=32).
- about 100% improvement for dst strings >=1024.
Change-Id: Ifd091ebdbce70fe35a7c5d8f71d5914255f3af35
This is needed when passing -mcpu=cortex-a9 or higher on a modern
toolchain for prebuilt library compatibility
Change-Id: I73eb2393377914ae26216a8c2828ad973d1c1225
Tested using a static version of the strlen libc_test program
on a nexus7 that uses the generic code.
Merge from internal master.
(cherry-picked from d8d10a8994)
Change-Id: I88f7dc01dc5b5c3ac2d5580d92153bc1bc36c564
This optimized version is primarily targeted at cortex-a15.
Tested on all nexus devices using the system/extras/libc_test strlen test.
Tested alignments from 1 to 32 that are powers of 2.
Tested that strlen does not cross page boundaries at all alignments.
Speed improvements listed below:
cortex-a15
- Sizes >= 32 bytes, ~75% improvement.
- Sizes >= 1024 bytes, ~250% improvement.
cortex-a9
- Sizes >= 32 bytes, ~75% improvement.
- Sizes >= 1024 bytes, ~85% improvement.
krait
- Sizes >= 32 bytes, ~95% improvement.
- Sizes >= 1024 bytes, ~160% improvement.
Merge from internal master.
(cherry-picked from 2fc0717977)
Change-Id: I1ceceb4e745fd68e9d946f96d1d42e0cdaff6ccf
We cleaned up the auto-generated ones a while back to not touch
the stack unnecessarily if they have <= 4 arguments. This patch
cleans up some hand-crafted ones.
Also improve comments in clone.S.
Change-Id: I8850bf98f2b26829385315304472a760e6880ed8
Tested using a static version of the strlen libc_test program
on a nexus7 that uses the generic code.
Change-Id: If04d15dcb6c0b18f27f2fefadca5510ed49016c5
This memcpy code uses NEON/VFP to achieve very good performance
on ARMv7-A processors. It is specifically tuned for A15 but should
provide good performance on A9 also. It is equivalent to the code
in cortex-strings rev 116.
This patch is a follow up the existing gerrit change:
I7f6f77995f3ca903ad9c66d14261441667a2a935
This version includes a tweak for performance on misaligned
buffers and splits the header comment into license and
documentation sections.
Change-Id: Ibd2e23c8d8e01357ba0247be1d05192de3ceba69
Signed-off-by: Will Newton <will.newton@linaro.org>
This memcpy code uses NEON/VFP to achieve very good performance
on ARMv7-A processors. It is specifically tuned for A15 but should
provide good performance on A9 also. It is equivalent to the code
in cortex-strings rev 116.
This patch is a follow up the existing gerrit change:
I7f6f77995f3ca903ad9c66d14261441667a2a935
But this version includes a tweak for performance on misaligned
buffers.
Change-Id: I285abac0068f8ae29a1cbf7862ea8590aadaf0a7
Signed-off-by: Will Newton <will.newton@linaro.org>
Streamline the memcpy a bit removing some unnecessary instructions.
The biggest speed improvement comes from changing the size of
the preload. On krait, the sweet spot for the preload in the main
loop is twice the L1 cache line size.
In most cases, these small tweaks yield > 1000MB/s speed ups. As
the size of the memcpy approaches about 1MB, the speed improvement
disappears.
Change-Id: Ief79694d65324e2db41bee4707dae19b8c24be62
This uses the new code original submitted as memcpy.a15.S as
the base. However, the old code handled unaligned src/dst better
so that was spliced in. I optimized the original unaligned code by
removing a few unnecessary instructions. I optimized the a15 code by
rewriting the pre and post code. I also modified the main loop to add
a pld so that larger copies would not stall waiting for memory.
Test cases for the new memcpy:
- Copy all sized values from 0 to 1024 bytes, using whatever alignment
is returned by malloc.
For each alignment case described below, the test copied from 0 to 128
bytes.
- Src and dst pointers are both aligned to the same value, starting
at one going through every power of two up to and including 128.
- Src aligned to double word boundary, dst aligned to word boundary.
- Src aligned to word boundary, dst aligned to double word boundary.
- Src aligned to 16 bit boundary, dst aligned to word boundary.
- Src aligned to word boundary, dst aligned to 16 byte boundary.
- Src aligned to word boundary, dst aligned to 1 byte from a word
boundary.
- Src aligned to word boundary, dst aligned to 2 bytes from a word
boundary.
- Src aligned to word boundary, dst aligned to 3 bytes from a word
boundary.
- Src aligned to 1 byte from a word boundary, dst aligned to a word
boundary.
- Src aligned to 2 bytes from a word boundary, dst aligned to a word
boundary.
- Src aligned to 3 bytes from a word boundary, dst aligned to a word
boundary.
Cases to verify the unaligned source code properly aligns to a 16 bit
boundary.
- Src aligned to 1 byte from a 128 bit boundary, dst aligned to
4 + 128 bit boundary.
- Src aligned to 1 byte from a 128 bit boundary, dst aligned to
8 + 128 bit boundary.
- Src aligned to 1 byte from a 128 bit boundary, dst aligned to
12 + 128 bit boundary.
- Src aligned to 1 byte from a 128 bit boundary, dst aligned to
16 + 128 bit boundary.
In all cases, a two byte fencepost was placed at the end of the
destination to verify that only the requested number of bytes were copied.
Bug: 8005082
Merge from internal master.
(cherry-picked from commit 21ede92d79)
Change-Id: Ief70c9e6dc8c6473ae245b6570b2c266fed9618c
This lets us move all the ARM syscall stubs over to the kernel <asm/unistd.h>.
Our generated <sys/linux-syscalls.h> is now unused, but I'll remove that in a
later change.
Change-Id: Ie5ff2cc4abce1938576af7cbaef615a79c7f310d
For some reason, socketcalls.c was only being compiled for ARM, where
it makes no sense. For x86 we generate stubs for the socket functions
that use __NR_socketcall directly.
Change-Id: I84181e6183fae2314ae3ed862276eba82ad21e8e
<sys/linux-syscalls.h> only contains constants for the syscalls
we're generating stubs for. We want all the syscalls available
on the architecture in question.
Keep using <sys/linux-syscalls.h> on ARM for now because the
__NR_ARM_set_tls and __NR_ARM_cacheflush values aren't in <asm/unistd.h>.
Change-Id: I66683950d87d9b18d6107d0acc0ed238a4496f44
This uses the new strcmp.a15.S code as the basis for new versions
of strcmp.S.
The cortex-a15 code is the performance optimized version of strcmp.a15.S
taken with only the addition of a few pld instructions.
The cortex-a9 code is the same as the cortex-a15 code except that the
unaligned strcmp code was taken from the original strcmp.S.
The krait code is the same as the cortex-a15 code except that one path
in the unaligned strcmp code was taken from the original strcmp.S code
(the 2 byte overlap case).
The generic code is the original unmodified strmp.S from the bionic
subdirectory.
All three new versions underwent these test cases:
Strings the same, all same size:
- Both pointers double word aligned.
- One pointer double word aligned, one pointer word aligned.
- Both pointers word aligned.
- One pointer double word aligned, one pointer 1 off a word alignment.
- One pointer double word aligned, one pointer 2 off a word alignment.
- One pointer double word aligned, one pointer 3 off a word alignment.
- One pointer word aligned, one pointer 1 off a word alignment.
- One pointer word aligned, one pointer 2 off a word alignment.
- One pointer word aligned, one pointer 3 off a word alignment.
For all cases where it made sense, the two pointers were also tested
swapped.
Different strings, all same size:
- Single difference at double word boundary.
- Single difference at word boudary.
- Single difference at 1 off a word alignment.
- Single difference at 2 off a word alignment.
- Single difference at 3 off a word alignment.
Different sized strings, strings the same until the end:
- Shorter string ends on a double word boundary.
- Shorter string ends on word boundary.
- Shorter string ends at 1 off a word boundary.
- Shorter string ends at 2 off a word boundary.
- Shorter string ends at 3 off a word boundary.
For all different cases, run them through the same pointer alignment
cases when the strings are the same size.
For all cases the two pointers were also tested swapped.
Bug: 8005082
Merge from internal master.
(cherry-picked from commit a9a5870d16)
Change-Id: I4c2b98f8a50804fb98ab67f75e9d660f1315a144
We only need one logging API, and I prefer the one that does no
allocation and is thus safe to use in any context.
Also use O_CLOEXEC when opening the /dev/log files.
Move everything logging-related into one header file.
Change-Id: Ic1e3ea8e9b910dc29df351bff6c0aa4db26fbb58
Move arch specific code for arm, mips, x86 into separate
makefiles.
In addition, add different arm cpu versions of memcpy/memset.
Bug: 8005082
Merge from internal master (acdde8c1cf).
Change-Id: I04f3d0715104fab618e1abf7cf8f7eec9bec79df
The attached patch provides a new implementation of strcmp for ARM,
using LDRD instead of LDR whenever possible.
For older architectures that do not support LDRD, this implementation
uses the same algorithm as before.
Testing and benchmarking:
* Validation: successfully passes a test that compares different strings
of length 1-128 and offsets 0-8 from a word boundary. Checked on
qemu/A15/A9, ARM/Thumb mode, Big/Little Endian.
* Integration with gcc: no regression on qemu for arm-none-eabi --with-cpu
a15/a9 --with-mode arm/thumb.
Change-Id: I9e230e1b99dbdc9119b69ee858a89038c516a4ea
Signed-off-by: Vassilis Laganakos <vasileios.laganakos@arm.com>
