* Bionic benchmarks results at the bottom
* This is a squash of the following commits:
libc: ARM64: optimize memset.
This is an optimized memset for AArch64. Memset is split into 4 main
cases: small sets of up to 16 bytes, medium of 16..96 bytes which are
fully unrolled. Large memsets of more than 96 bytes align the
destination and use an unrolled loop processing 64 bytes per
iteration. Memsets of zero of more than 256 use the dc zva
instruction, and there are faster versions for the common ZVA sizes 64
or 128. STP of Q registers is used to reduce codesize without loss of
performance.
Change-Id: I0c5b5ec5ab8a1fd0f23eee8fbacada0be08e841f
libc: ARM64: improve performance in strlen
Change-Id: Ic20f93a0052a49bd76cd6795f51e8606ccfbf11c
libc: ARM64: Optimize memcpy.
This is an optimized memcpy for AArch64. Copies are split into 3 main
cases: small copies of up to 16 bytes, medium copies of 17..96 bytes
which are fully unrolled. Large copies of more than 96 bytes align
the destination and use an unrolled loop processing 64 bytes per
iteration. In order to share code with memmove, small and medium
copies read all data before writing, allowing any kind of overlap. On
a random copy test memcpy is 40.8% faster on A57 and 28.4% on A53.
Change-Id: Ibb9483e45bbc0e8ca3d5ce98a31c55dfd8a5ac28
libc: AArch64: Tune memcpy
* Further tuning for performance.
Change-Id: Id08eaab885f9743fa7575077924a947c1b88e4ff
libc: ARM64: optimize memmove for Cortex-A53
* Sadly does not work on Denver or Kryo, so can't go to generic
This is an optimized memmove for AArch64. All copies of up to 96
bytes and all backward copies are done by the new memcpy. The only
remaining case is large forward copies which are done in the same way
as the memcpy loop, but copying from the end rather than the start.
Tested on the Nextbit Robin with MSM8992 (Snapdragon 808):
Before
BM_string_memcmp/8 1000k 27 0.286 GiB/s
BM_string_memcmp/64 50M 20 3.053 GiB/s
BM_string_memcmp/512 20M 126 4.060 GiB/s
BM_string_memcmp/1024 10M 234 4.372 GiB/s
BM_string_memcmp/8Ki 1000k 1726 4.745 GiB/s
BM_string_memcmp/16Ki 500k 3711 4.415 GiB/s
BM_string_memcmp/32Ki 200k 8276 3.959 GiB/s
BM_string_memcmp/64Ki 100k 16351 4.008 GiB/s
BM_string_memcpy/8 1000k 13 0.612 GiB/s
BM_string_memcpy/64 1000k 8 7.187 GiB/s
BM_string_memcpy/512 50M 38 13.311 GiB/s
BM_string_memcpy/1024 20M 86 11.858 GiB/s
BM_string_memcpy/8Ki 5M 620 13.203 GiB/s
BM_string_memcpy/16Ki 1000k 1265 12.950 GiB/s
BM_string_memcpy/32Ki 500k 2977 11.004 GiB/s
BM_string_memcpy/64Ki 500k 8003 8.188 GiB/s
BM_string_memmove/8 1000k 11 0.684 GiB/s
BM_string_memmove/64 1000k 16 3.855 GiB/s
BM_string_memmove/512 50M 57 8.915 GiB/s
BM_string_memmove/1024 20M 117 8.720 GiB/s
BM_string_memmove/8Ki 2M 853 9.594 GiB/s
BM_string_memmove/16Ki 1000k 1731 9.462 GiB/s
BM_string_memmove/32Ki 500k 3566 9.189 GiB/s
BM_string_memmove/64Ki 500k 7708 8.501 GiB/s
BM_string_memset/8 1000k 16 0.487 GiB/s
BM_string_memset/64 1000k 16 3.995 GiB/s
BM_string_memset/512 50M 37 13.489 GiB/s
BM_string_memset/1024 50M 58 17.405 GiB/s
BM_string_memset/8Ki 5M 451 18.160 GiB/s
BM_string_memset/16Ki 2M 883 18.554 GiB/s
BM_string_memset/32Ki 1000k 2181 15.022 GiB/s
BM_string_memset/64Ki 500k 4563 14.362 GiB/s
BM_string_strlen/8 1000k 8 0.965 GiB/s
BM_string_strlen/64 1000k 16 3.855 GiB/s
BM_string_strlen/512 20M 92 5.540 GiB/s
BM_string_strlen/1024 10M 167 6.111 GiB/s
BM_string_strlen/8Ki 1000k 1237 6.620 GiB/s
BM_string_strlen/16Ki 1000k 2765 5.923 GiB/s
BM_string_strlen/32Ki 500k 6135 5.341 GiB/s
BM_string_strlen/64Ki 200k 13168 4.977 GiB/s
After
BM_string_memcmp/8 1000k 21 0.369 GiB/s
BM_string_memcmp/64 1000k 28 2.272 GiB/s
BM_string_memcmp/512 20M 128 3.983 GiB/s
BM_string_memcmp/1024 10M 234 4.375 GiB/s
BM_string_memcmp/8Ki 1000k 1732 4.728 GiB/s
BM_string_memcmp/16Ki 500k 3485 4.701 GiB/s
BM_string_memcmp/32Ki 500k 7031 4.660 GiB/s
BM_string_memcmp/64Ki 200k 14296 4.584 GiB/s
BM_string_memcpy/8 1000k 5 1.458 GiB/s
BM_string_memcpy/64 1000k 7 8.952 GiB/s
BM_string_memcpy/512 50M 36 13.907 GiB/s
BM_string_memcpy/1024 20M 80 12.750 GiB/s
BM_string_memcpy/8Ki 5M 572 14.307 GiB/s
BM_string_memcpy/16Ki 1000k 1165 14.053 GiB/s
BM_string_memcpy/32Ki 500k 3141 10.430 GiB/s
BM_string_memcpy/64Ki 500k 7008 9.351 GiB/s
BM_string_memmove/8 50M 7 1.074 GiB/s
BM_string_memmove/64 1000k 9 6.593 GiB/s
BM_string_memmove/512 50M 37 13.502 GiB/s
BM_string_memmove/1024 20M 80 12.656 GiB/s
BM_string_memmove/8Ki 5M 573 14.281 GiB/s
BM_string_memmove/16Ki 1000k 1168 14.018 GiB/s
BM_string_memmove/32Ki 1000k 2825 11.599 GiB/s
BM_string_memmove/64Ki 500k 6548 10.008 GiB/s
BM_string_memset/8 1000k 7 1.038 GiB/s
BM_string_memset/64 1000k 8 7.151 GiB/s
BM_string_memset/512 1000k 29 17.272 GiB/s
BM_string_memset/1024 50M 53 18.969 GiB/s
BM_string_memset/8Ki 5M 424 19.300 GiB/s
BM_string_memset/16Ki 2M 846 19.350 GiB/s
BM_string_memset/32Ki 1000k 2028 16.156 GiB/s
BM_string_memset/64Ki 500k 4514 14.517 GiB/s
BM_string_strlen/8 1000k 7 1.120 GiB/s
BM_string_strlen/64 1000k 16 3.918 GiB/s
BM_string_strlen/512 50M 64 7.894 GiB/s
BM_string_strlen/1024 20M 104 9.815 GiB/s
BM_string_strlen/8Ki 5M 664 12.337 GiB/s
BM_string_strlen/16Ki 1000k 1291 12.682 GiB/s
BM_string_strlen/32Ki 1000k 2940 11.143 GiB/s
BM_string_strlen/64Ki 500k 6440 10.175 GiB/s
Change-Id: I635bd2798a755256f748b2af19b1a56fb85a40c6
These directories all have Android.bp files that are always used now,
delete the Android.mk files.
Change-Id: Ib0ba2d28bff88483b505426ba61606da314e03ab
It turns out that at least the Nexus 9 kernel is built without CONFIG_QUOTA.
If we decide we're going to mandate quota functionality, I'm happy for us to
be a part of CTS that ensures that happens, but I don't want to be first, so
there's not much to test here other than "will it compile?". The strace
output looks right though.
Bug: http://b/27948821
Bug: http://b/27952303
Change-Id: If667195eee849ed17c8fa9110f6b02907fc8fc04
{get,set}domainname aren't in POSIX but are widely-implemented
extensions.
The Linux kernel provides a setdomainname syscall but not a symmetric
getdomainname syscall, since it expects userspace to get the domain name
from uname(2).
Change-Id: I96726c242f4bb646c130b361688328b0b97269a0
Signed-off-by: Greg Hackmann <ghackmann@google.com>
This involves actually implementing assembler __memset_chk for arm64,
but that's easily done.
Obviously I'd like this for all architectures (and all the string functions),
but this is low-hanging fruit...
Change-Id: I70ec48c91aafd1f0feb974a2555c51611de9ef82
Our FORTIFY _chk functions' implementations were very repetitive and verbose
but not very helpful. We'd also screwed up and put the SSIZE_MAX checks where
they would never fire unless you actually had a buffer as large as half your
address space, which probably doesn't happen very often.
Factor out the duplication and take the opportunity to actually show details
like how big the overrun buffer was, or by how much it was overrun.
Also remove the obsolete FORTIFY event logging.
Also remove the unused __libc_fatal_no_abort.
This change doesn't improve the diagnostics from the optimized assembler
implementations.
Change-Id: I176a90701395404d50975b547a00bd2c654e1252
Exactly which functions get a stack protector is up to the compiler, so
let's separate the code that sets up the environment stack protection
requires and explicitly build it with -fno-stack-protector.
Bug: http://b/26276517
Change-Id: I8719e23ead1f1e81715c32c1335da868f68369b5
The mremap definition was incorrect (unsigned long instead of int) and
it was missing the optional new_address parameter.
Change-Id: Ib9d0675aaa098c21617cedc9b2b8cf267be3aec4
This moves the generic arm/arm64/x86 settings into the main makefiles
and makes the rest of them derivatives. This better aligns with how
soong handles arch/cpu variants.
Also updates the Android.bp to make it consistent with the make
versions.
Change-Id: I5a0275d992bc657459eb6fe1697ad2336731d122
A continuation of commit 2825f10b7f.
Add O_PATH compatibility support for flistxattr(). This allows
a process to list out all the extended attributes associated with
O_PATH file descriptors.
Change-Id: Ie2285ac7ad2e4eac427ddba6c2d182d41b130f75
Support O_PATH file descriptors when handling fgetxattr and fsetxattr.
This avoids requiring file read access to pull extended attributes.
This is needed to support O_PATH file descriptors when calling
SELinux's fgetfilecon() call. In particular, this allows the querying
and setting of SELinux file context by using something like the following
code:
int dirfd = open("/path/to/dir", O_DIRECTORY);
int fd = openat(dirfd, "file", O_PATH | O_NOFOLLOW);
char *context;
fgetfilecon(fd, &context);
This change was motivated by a comment in
https://android-review.googlesource.com/#/c/152680/1/toys/posix/ls.c
Change-Id: Ic0cdf9f9dd0e35a63b44a4c4a08400020041eddf
All arch-arm and arch-arm64 .S files were compiled
by gcc with and without this patch. The output object files
were identical. When compiled with llvm and this patch,
the output files were also identical to gcc's output.
BUG: 18061004
Change-Id: I458914d512ddf5496e4eb3d288bf032cd526d32b
The visibility control in pthread_atfork.h is incorrect.
It breaks 64bit libc.so by hiding pthread_atfork.
This reverts commit 6df122f852.
Change-Id: I21e4b344d500c6f6de0ccb7420b916c4e233dd34
This doesn't affect code like Chrome that correctly ignores EINTR on
close, makes code that tries TEMP_FAILURE_RETRY work (where before it might
have closed a different fd and appeared to succeed, or had a bogus EBADF),
and makes "goto fail" code work (instead of mistakenly assuming that EINTR
means that the close failed).
Who loses? Anyone actively trying to detect that they caught a signal while
in close(2). I don't think those people exist, and I think they have better
alternatives available.
Bug: https://code.google.com/p/chromium/issues/detail?id=269623
Bug: http://b/20501816
Change-Id: I11e2f66532fe5d1b0082b2433212e24bdda8219b
Apparently clang really doesn't want you to take the address of a builtin.
Since this is only a temporary hack, let's just shrug and accept that
clang-built volantis images won't work until we have new NVIDIA blobs.
Bug: http://b/20065774
Change-Id: I4c8e893b15a1af8f9c54d3f89bfef112b63d09b4
NVIDIA binary blobs are assuming that __cache_clear, _Unwind_Backtrace,
and _Unwind_GetIP are all in some library that they link, but now we've
cleaned up this leakage, they're no longer getting it. Deliberately leak
the symbols from libc.so until we get new blobs.
Bug: http://b/20065774
Change-Id: I92ef07b2bce8d1ad719bf40dab41d745cd6904d4
The kernel system call faccessat() does not have any flags arguments,
so passing flags to the kernel is currently ignored.
Fix the kernel system call so that no flags argument is passed in.
Ensure that we don't support AT_SYMLINK_NOFOLLOW. This non-POSIX
(http://pubs.opengroup.org/onlinepubs/9699919799/functions/access.html)
flag is a glibc extension, and has non-intuitive, error prone behavior.
For example, consider the following code:
symlink("foo.is.dangling", "foo");
if (faccessat(AT_FDCWD, "foo", R_OK, AT_SYMLINK_NOFOLLOW) == 0) {
int fd = openat(AT_FDCWD, "foo", O_RDONLY | O_NOFOLLOW);
}
The faccessat() call in glibc will return true, but an attempt to
open the dangling symlink will end up failing. GLIBC documents this
as returning the access mode of the symlink itself, which will
always return true for any symlink on Linux.
Some further discussions of this are at:
* http://lists.landley.net/pipermail/toybox-landley.net/2014-September/003617.html
* http://permalink.gmane.org/gmane.linux.lib.musl.general/6952
AT_SYMLINK_NOFOLLOW seems broken by design. I suspect this is why this
function was never added to POSIX. (note that "access" is pretty much
broken by design too, since it introduces a race condition between
check and action). We shouldn't support this until it's clearly
documented by POSIX or we can have it produce intuitive results.
Don't support AT_EACCESS for now. Implementing it is complicated, and
pretty much useless on Android, since we don't have setuid binaries.
See http://git.musl-libc.org/cgit/musl/commit/?id=0a05eace163cee9b08571d2ff9d90f5e82d9c228
for how an implementation might look.
Bug: 18867827
Change-Id: I25b86c5020f3152ffa3ac3047f6c4152908d0e04
The overflow's actually in the generic C implementation of memchr.
While I'm here, let's switch our generic memrchr to the OpenBSD version too.
Bug: https://code.google.com/p/android/issues/detail?id=147048
Change-Id: I296ae06a1ee196d2c77c95a22f11ee4d658962da
* changes:
Use LOCAL_LDFLAGS_64 instead of enumerating 64-bit architectures
Fix typo in cpu variant makefile depenendency for arm64
Remove libc_static_common_src_files
Share LP32 makefile settings between arches
In https://android-review.googlesource.com/#/c/127908/5/libc/SYSCALLS.TXT@116
Elliott said:
for LP64 these will be hidden. for LP32 we were cowards and left
them all public for compatibility (though i don't think we ever
dremeled to see whether it was needed). we don't have an easy
way to recognize additions, though, so we can't prevent adding
new turds.
Add a mechanism to prevent the adding of new turds, and use that
mechanism on the fchmod/fchmodat system calls.
Bug: 19233951
Change-Id: I98f98345970b631a379f348df57858f9fc3d57c0
Many libc functions have an option to not follow symbolic
links. This is useful to avoid security sensitive code
from inadvertantly following attacker supplied symlinks
and taking inappropriate action on files it shouldn't.
For example, open() has O_NOFOLLOW, chown() has
lchown(), stat() has lstat(), etc.
There is no such equivalent function for chmod(), such as lchmod().
To address this, POSIX introduced fchmodat(AT_SYMLINK_NOFOLLOW),
which is intended to provide a way to perform a chmod operation
which doesn't follow symlinks.
Currently, the Linux kernel doesn't implement AT_SYMLINK_NOFOLLOW.
In GLIBC, attempting to use the AT_SYMLINK_NOFOLLOW flag causes
fchmodat to return ENOTSUP. Details are in "man fchmodat".
Bionic currently differs from GLIBC in that AT_SYMLINK_NOFOLLOW
is silently ignored and treated as if the flag wasn't present.
This patch provides a userspace implementation of
AT_SYMLINK_NOFOLLOW for bionic. Using open(O_PATH | O_NOFOLLOW),
we can provide a way to atomically change the permissions on
files without worrying about race conditions.
As part of this change, we add support for fchmod on O_PATH
file descriptors, because it's relatively straight forward
and could be useful in the future.
The basic idea behind this implementation comes from
https://sourceware.org/bugzilla/show_bug.cgi?id=14578 , specifically
comment #10.
Change-Id: I1eba0cdb2c509d9193ceecf28f13118188a3cfa7
