f5f04b19fe
The first time should_trace() returns true, bionic_trace_begin() calls open() on trace_marker. The problem is that open() can call bionic_trace_begin(). We've observed this happening, for example when: * fdtrack is enabled. dlopen("libfdtrack.so") can be used to enable fdtrack on a process. * ThreadA is busy unwinding inside fdtrack and is holding an fdtrack internal mutex. * ThreadB calls bionic_trace_begin() for the first time since the property "debug.atrace.tags.enableflags" contains ATRACE_TAG_BIONIC. * ThreadB calls open("/sys/kernel/tracing/trace_marker"). Since fdtrack is enabled, ThreadB tries to do unwinding as well. * ThreadB, inside fdtrack's unwinding tries to grab the same mutex that ThreadA is holding. * Mutex contention is reported using bionic_systrace, therefore bionic_trace_begin() is called again on ThreadB. * ThreadB tries to grab g_lock in bionin_systrace.cpp, but that's already held by ThreadB itself, earlier on the stack. Therefore ThreadB is stuck. I managed to reproduce the above scenario by manually pausing ThreadA inside unwinding with a debugger and letting ThreadB hitting bionic_trace_begin() for the first time. We could avoid using g_lock while calling open() (either by releasing g_lock and reacquiring it later, or by using atomics), but bionic_trace_begin() would try to call open() again. In my tests, open() does not call bionic_trace_begin() a third time, because fdtrack has reentrancy protection, but there might be another code path inside open that calls bionic_trace_begin again (it could be racy or only happen in certain configurations). This commit fixes the problem by implementing reentrancy protection in bionic_systrace. Sample callstack from ThreadA deadlocked before the fix: ``` * frame #0: 0x0000007436db077c libc.so`syscall at syscall.S:41 frame #1: 0x0000007436db0ba0 libc.so`bionic_trace_begin(char const*) [inlined] __futex(ftx=0x000000743737a548, op=<unavailable>, value=2, timeout=0x0000000000000000, bitset=-1) at bionic_futex.h:45:16 frame #2: 0x0000007436db0b8c libc.so`bionic_trace_begin(char const*) [inlined] __futex_wait_ex(ftx=0x000000743737a548, value=2) at bionic_futex.h:66:10 frame #3: 0x0000007436db0b78 libc.so`bionic_trace_begin(char const*) [inlined] Lock::lock(this=0x000000743737a548) at bionic_lock.h:67:7 frame #4: 0x0000007436db0b74 libc.so`bionic_trace_begin(char const*) [inlined] should_trace() at bionic_systrace.cpp:38:10 frame #5: 0x0000007436db0b74 libc.so`bionic_trace_begin(message="Contending for pthread mutex") at bionic_systrace.cpp:59:8 frame #6: 0x0000007436e193e4 libc.so`NonPI::MutexLockWithTimeout(pthread_mutex_internal_t*, bool, timespec const*) [inlined] NonPI::NormalMutexLock(mutex=0x0000007296cae9f0, shared=0, use_realtime_clock=false, abs_timeout_or_null=0x0000000000000000) at pthread_mutex.cpp:592:17 frame #7: 0x0000007436e193c8 libc.so`NonPI::MutexLockWithTimeout(mutex=0x0000007296cae9f0, use_realtime_clock=false, abs_timeout_or_null=0x0000000000000000) at pthread_mutex.cpp:719:16 frame #8: 0x0000007436e1912c libc.so`::pthread_mutex_lock(mutex_interface=<unavailable>) at pthread_mutex.cpp:839:12 [artificial] frame #9: 0x00000071a4e5b290 libfdtrack.so`std::__1::mutex::lock() [inlined] std::__1::__libcpp_mutex_lock(__m=<unavailable>) at __threading_support:256:10 frame #10: 0x00000071a4e5b28c libfdtrack.so`std::__1::mutex::lock(this=<unavailable>) at mutex.cpp:31:14 frame #11: 0x00000071a4e32634 libfdtrack.so`unwindstack::Elf::Step(unsigned long, unwindstack::Regs*, unwindstack::Memory*, bool*, bool*) [inlined] std::__1::lock_guard<std::__1::mutex>::lock_guard(__m=0x0000007296cae9f0) at __mutex_base:104:27 frame #12: 0x00000071a4e32618 libfdtrack.so`unwindstack::Elf::Step(this=0x0000007296cae9c0, rel_pc=66116, regs=0x0000007266ca0470, process_memory=0x0000007246caa130, finished=0x0000007ff910efb4, is_signal_frame=0x0000007ff910efb0) at Elf.cpp:206:31 frame #13: 0x00000071a4e2b3b0 libfdtrack.so`unwindstack::LocalUnwinder::Unwind(this=0x00000071a4ea1528, frame_info=<unavailable>, max_frames=34) at LocalUnwinder.cpp:102:22 frame #14: 0x00000071a4e2a3ec libfdtrack.so`fd_hook(event=<unavailable>) at fdtrack.cpp:119:18 frame #15: 0x0000007436dbf684 libc.so`::__open_2(pathname=<unavailable>, flags=<unavailable>) at open.cpp:72:10 frame #16: 0x0000007436db0a04 libc.so`bionic_trace_begin(char const*) [inlined] open(pathname=<unavailable>, flags=524289) at fcntl.h:63:12 frame #17: 0x0000007436db09f0 libc.so`bionic_trace_begin(char const*) [inlined] get_trace_marker_fd() at bionic_systrace.cpp:49:25 frame #18: 0x0000007436db09c0 libc.so`bionic_trace_begin(message="pthread_create") at bionic_systrace.cpp:63:25 ``` Bug: 213642769 Change-Id: I10d331859045cb4a8609b007f5c6cf2577ff44df
145 lines
5 KiB
C++
145 lines
5 KiB
C++
/*
|
|
* Copyright (C) 2008 The Android Open Source Project
|
|
* All rights reserved.
|
|
*
|
|
* Redistribution and use in source and binary forms, with or without
|
|
* modification, are permitted provided that the following conditions
|
|
* are met:
|
|
* * Redistributions of source code must retain the above copyright
|
|
* notice, this list of conditions and the following disclaimer.
|
|
* * Redistributions in binary form must reproduce the above copyright
|
|
* notice, this list of conditions and the following disclaimer in
|
|
* the documentation and/or other materials provided with the
|
|
* distribution.
|
|
*
|
|
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
|
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
|
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
|
|
* FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
|
|
* COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
|
|
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
|
|
* BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
|
|
* OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
|
|
* AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
|
* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
|
|
* OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
* SUCH DAMAGE.
|
|
*/
|
|
|
|
#pragma once
|
|
|
|
#include <locale.h>
|
|
#include <mntent.h>
|
|
#include <stdio.h>
|
|
#include <sys/cdefs.h>
|
|
#include <sys/param.h>
|
|
|
|
#include <platform/bionic/tls.h>
|
|
|
|
#include "platform/bionic/macros.h"
|
|
#include "grp_pwd.h"
|
|
|
|
/** WARNING WARNING WARNING
|
|
**
|
|
** This header file is *NOT* part of the public Bionic ABI/API and should not
|
|
** be used/included by user-serviceable parts of the system (e.g.
|
|
** applications).
|
|
**/
|
|
|
|
class pthread_internal_t;
|
|
|
|
// This struct is small, so the linker can allocate a temporary copy on its
|
|
// stack. It can't be combined with pthread_internal_t because:
|
|
// - native bridge requires pthread_internal_t to have the same layout across
|
|
// architectures, and
|
|
// - On x86, this struct would have to be placed at the front of
|
|
// pthread_internal_t, moving fields like `tid`.
|
|
// - We'd like to avoid having a temporary pthread_internal_t object that
|
|
// needs to be transferred once the final size of static TLS is known.
|
|
struct bionic_tcb {
|
|
void* raw_slots_storage[BIONIC_TLS_SLOTS];
|
|
|
|
// Return a reference to a slot given its TP-relative TLS_SLOT_xxx index.
|
|
// The thread pointer (i.e. __get_tls()) points at &tls_slot(0).
|
|
void*& tls_slot(size_t tpindex) {
|
|
return raw_slots_storage[tpindex - MIN_TLS_SLOT];
|
|
}
|
|
|
|
// Initialize the main thread's final object using its bootstrap object.
|
|
void copy_from_bootstrap(const bionic_tcb* boot) {
|
|
// Copy everything. Problematic slots will be reinitialized.
|
|
*this = *boot;
|
|
}
|
|
|
|
pthread_internal_t* thread() {
|
|
return static_cast<pthread_internal_t*>(tls_slot(TLS_SLOT_THREAD_ID));
|
|
}
|
|
};
|
|
|
|
/*
|
|
* Bionic uses some pthread keys internally. All pthread keys used internally
|
|
* should be created in constructors, except for keys that may be used in or
|
|
* before constructors.
|
|
*
|
|
* We need to manually maintain the count of pthread keys used internally, but
|
|
* pthread_test should fail if we forget.
|
|
*
|
|
* These are the pthread keys currently used internally by libc:
|
|
* _res_key libc (constructor in BSD code)
|
|
*/
|
|
|
|
#define LIBC_PTHREAD_KEY_RESERVED_COUNT 1
|
|
|
|
/* Internally, jemalloc uses a single key for per thread data. */
|
|
#define JEMALLOC_PTHREAD_KEY_RESERVED_COUNT 1
|
|
#define BIONIC_PTHREAD_KEY_RESERVED_COUNT (LIBC_PTHREAD_KEY_RESERVED_COUNT + JEMALLOC_PTHREAD_KEY_RESERVED_COUNT)
|
|
|
|
/*
|
|
* Maximum number of pthread keys allocated.
|
|
* This includes pthread keys used internally and externally.
|
|
*/
|
|
#define BIONIC_PTHREAD_KEY_COUNT (BIONIC_PTHREAD_KEY_RESERVED_COUNT + PTHREAD_KEYS_MAX)
|
|
|
|
class pthread_key_data_t {
|
|
public:
|
|
uintptr_t seq; // Use uintptr_t just for alignment, as we use pointer below.
|
|
void* data;
|
|
};
|
|
|
|
// ~3 pages. This struct is allocated as static TLS memory (i.e. at a fixed
|
|
// offset from the thread pointer).
|
|
struct bionic_tls {
|
|
pthread_key_data_t key_data[BIONIC_PTHREAD_KEY_COUNT];
|
|
|
|
locale_t locale;
|
|
|
|
char basename_buf[MAXPATHLEN];
|
|
char dirname_buf[MAXPATHLEN];
|
|
|
|
mntent mntent_buf;
|
|
char mntent_strings[BUFSIZ];
|
|
|
|
char ptsname_buf[32];
|
|
char ttyname_buf[64];
|
|
|
|
char strerror_buf[NL_TEXTMAX];
|
|
char strsignal_buf[NL_TEXTMAX];
|
|
|
|
group_state_t group;
|
|
passwd_state_t passwd;
|
|
|
|
char fdtrack_disabled;
|
|
char bionic_systrace_disabled;
|
|
char padding[2];
|
|
|
|
// Initialize the main thread's final object using its bootstrap object.
|
|
void copy_from_bootstrap(const bionic_tls* boot __attribute__((unused))) {
|
|
// Nothing in bionic_tls needs to be preserved in the transition to the
|
|
// final TLS objects, so don't copy anything.
|
|
}
|
|
};
|
|
|
|
class KernelArgumentBlock;
|
|
extern "C" void __libc_init_main_thread_early(const KernelArgumentBlock& args, bionic_tcb* temp_tcb);
|
|
extern "C" void __libc_init_main_thread_late();
|
|
extern "C" void __libc_init_main_thread_final();
|