Mainly just factoring out the code, but there are two functional
changes here too:
1. The inline assembler was missing `volatile`, making the hwcap
check ineffective (because the compiler would sometimes move the
MIDR_EL1 read above the hwcap check).
2. The previous code accepted variants 0x0 to 0x5 while the comment
said 0x1 to 0x5. The comment was correct.
I resisted the temptation to actually have a table to search on the assumption that it'll be a while before we need such a thing.
Bug: https://issuetracker.google.com/330105715
Change-Id: I9fdc1e70e49b26ef32794b55ca5e5fd37f1163f9
Submitted on behalf of a third-party: Linaro Limited
License rights, if any, to the submission are granted solely by the
copyright owner of such submission under its applicable intellectual
property.
Copyright (c) 2012, Linaro Limited
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the Linaro nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Origin Project URL: https://android.googlesource.com/platform/bionic/
Commit ID: 7e4fa56099
Third Party code includes additions/modifications from Qualcomm Innovation Center, Inc.
Test: All
Change-Id: I479a572a325e27262d27aa37c516618e4322e9bb
Submitted on behalf of a third-party: Arm Limited
License rights, if any, to the submission are granted solely by the
copyright owner of such submission under its applicable intellectual
property.
Copyright (c) 2012-2022, Arm Limited.
SPDX-License-Identifier: MIT OR Apache-2.0 WITH LLVM-exception
Origin Project URL: https://github.com/ARM-software/optimized-routines
Tag: v24.01
Third Party code includes additions/modifications from Qualcomm Innovation Center, Inc.
Test: All
Change-Id: I0c97398a435e3f8ddf8ad38bc6bd71cc0d78aea5
Not useful right now, but Qualcomm has an Oryon memset they'd like to
use, and there's no reason to treat memrchr as a weird special case.
Bug: https://issuetracker.google.com/330105715
Test: treehugger
Change-Id: Id879479bf4f45433debcb3fe08cfa96bb1eb3b93
We cannot use a WriteProtected because we are accessing it in a
multithreaded context.
Test: atest memtag_stack_dlopen_test w/ MTE
Test: atest bionic-unit-tests w/ MTE
Test: atest bionic-unit-tests on _fullmte
Bug: 328256432
Change-Id: I39faa75f97fd5b3fb755a46e88346c17c0e9a8e2
We would get the SP inside of memtag_handle_longjmp, which could prevent
us from detecting the case where a longjmp is going into a function that
had already returned. This changes makes the behaviour more predictable.
Change-Id: I75bf931c8f4129a2f38001156b7bbe0b54a726ee
clang complains if you define a symbol and _then_ make it weak, rather
than the other way round:
/tmp/setjmp-c3c977.s:90:1: warning: sigsetjmp changed binding to STB_WEAK
.weak sigsetjmp;
^
Test: treehugger
Change-Id: Iee6b0ea456bb2e92aea810ce45f171caabaa89d2
No actual effect on the code, but misleading and wrong. (The previous
change only fixed the argument types; I didn't notice that some of the
return types were wrong too.)
Test: treehugger
Change-Id: I1ee5c48e2652fd8cbf8178d5659e57f79e61898e
This patch doesn't *enable* the SVE optimized routines, but it does let
us see if switching them to ifuncs will cause any app compat issues, so
that we can more easily use the optimized routines in future.
Test: treehugger
Change-Id: Ic5fe570bd21687da397b48127bf688f7ec68dd0c
The MTE-compatible routines are now faster than the incompatible ones,
so they merged them upstream.
I've left the ifunc boilerplate on the assumption that I'll be back
later to enable the new SVE variants.
Test: treehugger
Change-Id: Ic894bfb350b9aa70e307bca1c4978624b3e5f4fd
This one's a bit simpler, because there is only one upstream memset()
implementation.
Test: treehugger
Change-Id: I2536d0eb72adaacfa6a0e40d2bd29fc833988c16
Outsource this to them, and choose the best of the two options available
based on the hardware we're running on.
Test: treehugger
Change-Id: I2fa7555c971b64a6decca132210e901ffa248efa
These two will stay behind when we move memcpy()/memmove()/memset() over
to arm-optimized-routines (which leaves fortify to us).
Test: treehugger
Change-Id: Ie683f71a5a141263ce3f4e8811df9eaf667584f4
Just to make it clear that there's nothing interesting going on here ---
there's just one user, and the only symbol here is __memcpy().
Test: treehugger
Change-Id: I62d72c43c4c6d30442f05c1e08a0cb1a1ec42a8a
The compiler turns our C wmemmove() into one shift instruction and a
branch, which is plenty for a function no-one uses anyway.
Why don't I just leave this alone, since we already have it? Because I'm
looking at finishing the project of "switch to arm-optimized-routines"
and getting rid of our assembler here, and Arm agrees that this isn't
worth having optimized assembler for in their optimized assembler
project, judging by its absence.
Test: treehugger
Change-Id: I985801241a8cbd7dbda51a447946affb1402effb
With memtag_stack, each function is responsible for cleaning up
allocation tags for its stack frame. Allocation tags for anything below
SP must match the address tag in SP.
Both vfork and longjmp implement non-local control transfer which
abandons part of the stack without proper cleanup. Update allocation
tags:
* For longjmp, we know both source and destination values of SP.
* For vfork, save the value of SP before exit() or exec*() - the only
valid ways of ending the child process according to POSIX - and reset
tags from there to SP-in-parent.
This is not 100% solid and can be confused by a number of hopefully
uncommon conditions:
* Segmented stacks.
* Longjmp from sigaltstack into the main stack.
* Some kind of userspace thread implementation using longjmp (that's UB,
longjmp can only return to the caller on the current stack).
* and other strange things.
This change adds a sanity limit on the size of the tag cleanup. Also,
this logic is only activated in the binaries that carry the
NT_MEMTAG_STACK note (set by -fsanitize=memtag-stack) which is meant as
a debugging configuration, is not compatible with pre-armv9 CPUs, and
should not be set on production code.
Bug: b/174878242
Test: fvp_mini with ToT LLVM (more test in a separate change)
Change-Id: Ibef8b2fc5a6ce85c8e562dead1019964d9f6b80b
Normally, platform-specific note types in the toolchain are prefixed
with the platform name. Because we're exposing the NT_TYPE_MEMTAG and
synthesizing the note in the toolchain in an upcoming patch
(https://reviews.llvm.org/D118948), it's been requested that we change
the name to include the platform prefix.
While NT_TYPE_IDENT and NT_TYPE_KUSER aren't known about or synthesized
by the toolchain, update those references as well for consistency.
Bug: N/A
Test: Build Android
Change-Id: I7742e4917ae275d59d7984991664ea48028053a1
arm64 was already being careful, but x86/x86-64 and 32-bit ARM could be
caught by a signal in a state where the stack pointer was mangled.
For 32-bit ARM I've taken care with the link register too, to avoid
potential issues with unwinding.
Bug: http://b/152210274
Test: treehugger
Change-Id: I1ce285b017a633c732dbe04743368f4cae27af85
The toolchain is new enough that should be able to use the actual
instructions now...
Test: treehugger
Change-Id: I30aafcdc5386268344c40dc6cc9a22caf591915a
Now that the feature guarded by this flag has landed in Linux 5.10
we no longer need the flag, so we can remove it.
Bug: 135772972
Change-Id: I02fa50848cbd0486c23c8a229bb8f1ab5dd5a56f
Use a note in executables to specify
(none|sync|async) heap tagging level. To be extended with (heap x stack x
globals) in the future. A missing note disables all tagging.
Bug: b/135772972
Test: bionic-unit-tests (in a future change)
Change-Id: Iab145a922c7abe24cdce17323f9e0c1063cc1321
The most notable change is in sigsetjmp/siglongjmp. The former
stores LR signed with the current SP into jmp_buf. Calling siglongjmp
reads a signed LR and the corresponding SP from jmp_buf. This way not
only the checksum provides some means of integrity protection but
Pointer Authentication too.
Test: Tested on FVP with BTI enabled.
Change-Id: I9d720239775f8d2829a677901f546c4b14b5cbe5
ARM has released the remaining MTE string routines, so let's start
using them. The strnlen implementation is now compatible with MTE,
so it no longer needs to be an ifunc.
Bug: 135772972
Change-Id: I9de7fb44447aa1b878f4ad3f62cb0129857b43ad
Our various fd debugging facilities get extremely confused by a vforked
process closing file descriptors in preparation to exec: fdsan can
abort, and fdtrack will delete backtraces for any file descriptors that
get closed. Keep track of whether we're in a vforked child in order to
be able to detect this.
Bug: http://b/153926671
Test: 32/64-bit bionic-unit-tests on blueline, x86_64 emulator
Change-Id: I8a082fd06bfdfef0e2a88dbce350b6f667f7df9f
This includes optimized strrchr and strchrnul routines, and an MTE-compatible
strlen routine.
Bug: 135772972
Change-Id: I48499f757cdc6d3e77e5649123d45b17dfa3c6b0
As it turns out, our "generic" arm64 implementations of certain string.h
functions are not actually generic, since they will eagerly read memory
possibly outside of the bounds of an MTE granule, which may lead to a segfault
on MTE-enabled hardware. Therefore, move the implementations into a "default"
directory and use ifuncs to select between them and a new set of "mte"
implementations, conditional on whether the hardware and kernel support MTE.
The MTE implementations are currently naive implementations written in C
but will later be replaced with a set of optimized assembly implementations.
Bug: 135772972
Change-Id: Ife37c4e0e6fd60ff20a34594cc09c541af4d1dd7
There are places in frameworks and art code that directly included
private bionic header files. Move these files to the new platform
include files.
This change also moves the __get_tls.h header file to tls.h and includes
the tls defines header so that there is a single header that platform
code can use to get __get_tls and the defines.
Also, simplify the visibility rules for platform includes.
Bug: 141560639
Test: Builds and bionic unit tests pass.
Change-Id: I9e5e9c33fe8a85260f69823468bc9d340ab7a1f9
Merged-In: I9e5e9c33fe8a85260f69823468bc9d340ab7a1f9
(cherry picked from commit 44631c919a)
...all the better to switch to a genrule rather than checking in
generated source.
This also removes all the code in the script to deal with git,
rather than fix it. We won't need that where we're going.
Test: boots
Change-Id: I468ce019d4232a7ef27e5cb5cfd89f4c2fe4ecbd
Call a hwasan hook in the parent return path for vfork() to let hwasan
update its shadow. See https://github.com/google/sanitizers/issues/925
for more details.
Bug: 112438058
Test: bionic-unit-tests
Change-Id: I9a06800962913e822bd66e072012d0a2c5be453d
This lets us do two things:
1) Make setjmp and longjmp compatible with shadow call stack.
To avoid leaking the shadow call stack address into memory, only the
lower log2(SCS_SIZE) bits of x18 are stored to jmp_buf. This requires
allocating an additional guard page so that we're guaranteed to be
able to allocate a sufficiently aligned SCS.
2) SCS overflow detection. Overflows now result in a SIGSEGV instead
of corrupting the allocation that comes after it.
Change-Id: I04d6634f96162bf625684672a87fba8b402b7fd1
Test: bionic-unit-tests
HWASan needs to re-tag the newly unallocated stack space to match SP.
Bug: 112438058
Test: SANITIZE_TARGET=hwaddress
Change-Id: I4dddef542d802d63bdea59e32a03425a2c4f870b
Conditional branch has limited range (1MB) and can not be extended by
the linker. The current distance (in walleye build) is 500KB, about
half of the maximum. HWASan pushes it over the limit.
Replace conditional branch with regular branch, which has longer
range (26 vs 19 bits offset) and can be extended in the linker if
needed.
Bug: 112437884
Bug: 12231437
Test: SANITIZE_TARGET=hwaddress
Change-Id: Idc083fb557ab3a859541beb009809992406a6703
