Kernel headers coming from:
Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.6
Add a new method for removing structures. This is to deal with the kernel
headers changing some definitions of timeval to __kernel_old_timeval
and itimerval to __kernel_old_itimerval. Remove the __kernel_old_XX
strutures and change the other structures to the previous definitions.
This only works so long as these structures stay the same, if they
diverge, then a different strategy will need to be implemented.
Test: Booted cuttlefish/walleye.
Test: Ran bionic-unit-tests on cuttlefish/walleye.
Change-Id: I0a61f4fa6e4155c602e0414d9b38c2e1637829af
These just cause confusion because they often have different
values/layouts, but they're never actually used.
Test: treehugger
Change-Id: I424034088e017c919f62fcefa7d6d3f903f31cfb
On Android, fcntl is always implemented by fcntl64(2). This means that
an LP32 binary can `fcntl(F_SETLK, struct flock)` (because fcntl64(2)
passes through to the 32-bit fcntl(2) to handle F_SETLK), and it can
also `fcntl(F_SETLK64, struct flock64)`. What it can't do before this
patch is set _FILE_OFFSET_BITS=64 and then `fcntl(F_SETLK, struct
flock)` where that `struct flock` is actually implicitly `struct
flock64`.
Move the kernel uapi structs out of the way, define them ourselves based
on __LP64__ and _FILE_OFFSET_BITS, and fix up the relevant F_ constants.
(Also add a .clang-format to turn off clang-format in libc/include/.)
Bug: N/A
Test: treehugger (and strace!)
Change-Id: Iccd6c83d9133e1efcf93a7b49a6ae0f1bbd3d58b
Update generate_uapi_headers.sh to checkout the android mainline
kernel. Also, add a small modification to look for the kernel directory
in common not linux-stable.
Remove deprecated android headers from android/uapi/linux. Also,
remove f_accessory.h since it's in the android mainline kernel.
Test: Builds and runs on walleye.
Change-Id: Ia371305e19f56e6bcc2db6d5b4d299819f07ffc6
Also, fix a bug in update_all.py when the syscalls file does not change.
Test: Builds and boots on a taimen.
Change-Id: If85b00daef2c176f804a0861894f5bbca9c6d5df
Enable the use case where we run clean_header.py from outside of
$ANDROID_BUILD_TOP. Previously, this script required the current working
directory to be under $ANDROID_BUILD_TOP. Running it from a different
directory resulted in the following error message:
clean_header.py: error: Not in android tree pointed at by ANDROID_BUILD_TOP (....)
Change-Id: I48210ea1a0033228a9aaa4124d28247b07cee6d4
The function cleanupFile should always return a single value (as opposed
to a tuple or list). In addition, if it encounters an error, it is
expected to return a value that evaluates to False. As it stands,
however, it returns (None, None) in certain error cases. Change this
function to return None, in those cases.
We previously saw the following error message, when we tried to run
clean_header.py on a non-existent file.
Traceback (most recent call last):
File "clean_header.py", line 208, in <module>
b.updateGitFiles()
File "utils.py", line 164, in updateGitFiles
self._writeFile(dst)
File "utils.py", line 136, in _writeFile
f.write(self.new_data[dst])
TypeError: expected a string or other character buffer object
Change-Id: I5f717dd1a4388f598f0fd4bfd5e6129017de9095
Don't print an extraneous newline character at the end of the include
file if clean_header.py is used in the non-update mode. This is
necessary to achieve consistent results in the update and non-update
modes.
Running clean_header in the non-update mode and redirecting the output
to a file should have the same result as using the update mode and have
the script write to the file directly.
Change-Id: I6b176c5365840d66e4499bacd205f1fa77302a2b
Write to correct dst_file when in update mode. This enables use cases
like the following:
../../../bionic/libc/kernel/tools/clean_header.py -k original-kernel-headers/ -d kernel-headers/ -u linux/ion.h
Previously, we had to cd into kernel-headers/ and then run
../../../../bionic/libc/kernel/tools/clean_header.py -k ../original-kernel-headers/ -d . -u linux/ion.h
because the tool didn't allow the relative path to be different from the
destination path.
Change-Id: I8c5e284ce7a6737d77a2f5ead3e7e5db01317425
The current version has these bugs:
- Adding a semicolon after a function results in the removal of structures
following the function.
- Function like macros get removed on accident rather than on purpose.
- It removes extern "C" { completely, which might not be a bug, but doesn't
seem right.
I couldn't easily fix any of these problems because the code depends heavily
on the header being correct.
New unit tests added for the function to cover all of these cases.
A follow-on CL will include the updated headers.
Bug: 112290385
Test: Passes all new unit tests.
Test: When run on the current kernel headers, the generated headers are
Test: nearly the same, missing data is being added.
Change-Id: Ib22a5f2e78873544e8a9d54e385af1156b2a72bb
When a construct like:
if defined(something)
blocks1
elif 1
blocks2
else
blocks3
endif
The parser would put the first clause but then simply omit the elif and
put all of blocks2 without a terminating #endif. The code also did
something similar when the #else was an #endif.
Also convert all of the unit tests to real unit tests and only run
them if you run cpp.py by itself. Added new unit tests to cover the
new cases.
Test: Ran cpp.py unit tests.
Test: Reran update_all.py and verified nothing changed, and that running
Test: it on the new kernel headers that exposed this problem.
Change-Id: Ie168511303c4e15afdb60c37baef75a966ca29a8
The current code checks out the android kernel tree. Long ago, we moved
to using the linux kernel source code. Modify the script to actually
do this.
Bug: 6653610
Test: Ran the script using the --download_kernel option and without.
Change-Id: I769e769edea272f4f8c9c8d2e76bd336059c66e3
This adds a new mechanism to say "replace struct S with #include <bits/S.h>".
Also switch epoll_event over to the new mechanism.
Also use the kernel's struct sockaddr_storage directly rather than behind
an unnecessary #define.
This patch also removes some dead code in the header scrubber. This code
still needs rewriting completely. I learned that a "block" isn't necessarily
a single struct definition, say; it might be a run of them. It seems like
a block is a run of preprocessor directives or a run of regular code.
Bug: https://issuetracker.google.com/36987220
Test: new test
Change-Id: Ic6a5c09559766a4babe3cd4c3ea538b885e07308
Remove the hiding of the kernel structure binder_fd_array_object. This
structure now matches the structure used in the binder code.
Load the libclang_android.so shared library directly for parsing.
This file changed name in a recent update to the prebuilts.
Test: Compiles arm/arm64/x86/x86_64.
Test: Boots on hikey and boots on a sailfish.
Test: Ran bionic unit tests on hikey and sailfish.
Change-Id: I141a4b93ac3511cd58f4d12bb3c0d4efaa4c2742
/usr/bin/python may be python3. We should respect PATH to find the python
executable so it can be locally overridden to be python2.
Test: Build libc, repo upload
Change-Id: Iaddd7cd4a1c2177c32786e4fa0fc664ab0ad36de
Callers are supposed to #include <scsi/sg.h> but if we tell soong to add
bionic/libc/kernel/android/ to the include path, the uapi headers in there
would be (unintentionally) accessible as either <linux/name.h> or
<uapi/linux/name.h>.
Bug: N/A (hit while upgrading strace to 4.17)
Test: builds
Change-Id: I8d47dd51da688c38f747a255d401dfb2c209c805
Having
WARNING: DO NOT EDIT, AUTO-GENERATED CODE - SEE TOP FOR INSTRUCTIONS
every four lines made the headers harder to read, made the diffs much worse
each time we upgraded, and wasn't really providing any benefit. Before the
next uapi update, let's just stop doing this.
Bug: N/A
Test: builds, manually inspected files look right
Change-Id: Id7088cf750894c9d24950f3d53587fe3156c4f7d
This is a bit bogus because it's been removed from glibc (though not
thoroughly) and is never useful on Android (because the system calls
in question are compiled out of Android kernels, and SELinux would
disallow them even if you weren't running an Android kernel). This
also means that on glibc you need to include <linux/sem.h> for this
and on bionic you need <sys/sem.h> (and for either if you #include
the other file, you won't get this union).
Bug: https://github.com/android-ndk/ndk/issues/400
Test: added new test
Change-Id: I47f721da77515531f616d6ad8479bfbc9b60ee47
Fix the list of static functions coming from swab.h to match the
ones exported by glibc.
Force the definition of __HAVE_BUILTIN_BSWAP{16,32,64}__ so the headers
use the builtin.
Add a unit test to guarantee that kernel header updates do not break this.
Test: Built and booted angler.
Test: Built the bionic unit tests for arm, arm64, mips, x86, x86_64.
Test: Ran the new test on angler and glibc.
Change-Id: I4ce229e3f198c204186d72bf22dd97b5cdf239e4
The previous versions of the scripts did a lot of redundant changes
and were hard to follow.
I rewrote most of update_all.py so that it's clear about what's going on.
I updated clean_header.py to change the cleanupFile function so that
there is no magic about where the destination file is going to wind up.
Now the caller specifies the final location.
I updated utils.py so that if you are trying to do an update in one
location, but your lunch target is from another location, it causes
an error.
Bug: 35726570
Change-Id: Ic5a44d90c2774a627eecde34c0c403bc925a497c
Test: Ran the updater and verified it works properly.
Test: Verified that doing an update in one tree to another tree
Test: fails.
Modify the kernel header update script to rename C++ keyword
struct members to something else. This fixes problems when including
these kernel files in C++ code.
Also, make a manual change to linux/fib_rules.h and comment out the
value FRA_PAD since it conflicts with a special android kernel value
FRA_UID_START that has not been upstreamed yet.
Test: Built aosp_angler, aosp_x86_64, aosp_mips (bionic only)
Test: Booted on angler and ran bionic unit tests
Test: Built these changes in internal master
Change-Id: Ia22b7e3ca409404696dba76311f49157f4a9ceed
Also fix <sys/ipc.h>.
Not useful except to systems/bringup folks for testing. Trivial tests
added, and double-checked under strace to see that things look right.
x86 -- which works differently to everything else -- tested on the host.
Bug: http://b/27952303
Change-Id: I328534e994ae9e90755f545478fba03038c0bb94
Modify the generate script to add a new option and copy the types.h
arm header file since it's not currently being created properly. Also
manually generate the arm types.h uapi header since it's not being
properly generated right now.
Modify both generate scripts to delete the target directories before
adding the files to handle moved/deleted header files.
Move the common/scsi headers into android and delete the common
directory. Change the scripts to reflect this change.
Update the scsi headers since they've been modified in upstream kernels.
Bug: 30072483
Change-Id: Ia43d4b238b6a041350d60cc30184ecbd4829d7d5
The comment about "other stuff" referred to pre-uapi headers. Everything
in the current <linux/udp.h> should be exposed to userspace. The only
problem is that BSD and Linux use different names for the members of
struct udphdr. We can move the Linux udphdr out of the way and use an
anonymous union to get the best of both worlds. (Though unfortunately
this means that code that includes <linux/udp.h> directly instead of
using <netinet/udp.h> now won't have any definition of struct udphdr.
We've taken the stance in the past that you shouldn't include a linux/
header if there's a standard equivalent --- you should rely on us
transitively including it for you.)
Change-Id: Ie625892441b0edd8df3b76d3fcf2cbe299077bc4
This changes the scripts so that if some kernel files exists
in external/kernel-headers/modified, that they will be preferred
over the same files found in original. This is to support the case
where the kernel headers cannot be taken without some small modifications.
Included with this change, is a general cleanup of the python scripts.
This also modifies the generate uapi headers script to indicate if the
source of the modified headers has changed.
Change-Id: Id13523b244ced52a2ecd9f1399c43996dd8296fa
Prebuilt shared libraries (libclang.so, libLLVM.so and etc) have been
moved to prebuilts/sdk/tools/linux/lib64. Update the search path in
cpp.py to match the change.
Bug: 20485471
Change-Id: Ib7784db4d5529d16a1e2bfc07cb0237929bc5a64
now, some script to update kernel-headers is not working.
because of wrong variable value and test codes.
so this issue is fixed.
Change-Id: Iffae9607858cc3c1c58fa24244be217b5a1ab06e
Other changes:
- Modify update_all.py to skip ion header files when importing into aosp.
- Fix generate_uapi_headers.sh to handle imports from a linux-stable kernel.
Change-Id: I1ad81b9ccb063c21740f9875f2cc1238052cd4b3
Define CONFIG_32BIT as _ABIO32 rather than as 1;
it needs to be undefined for calls from mips64.
We need defined(CONFIG_32BIT) == !defined(CONFIG_64BIT),
and get that via defined(_ABIO32) == !defined(__LP64__).
Define CONFIG_CPU_LITTLE_ENDIAN to get correctly-placed
struct msqid64_ds fillers on ipc calls from mips32.
Without this, big-endian fillers were erroneously used.
This bug exists in prior mips32 Android releases.
Define __SANE_USERSPACE_TYPES__ to use int_ll64.h
rather than int_ll64.h on mips64.
This depends on external/kernel-headers patch
https://android-review.googlesource.com/#/c/98915/http://patchwork.linux-mips.org/patch/6890/
Change-Id: If65fa80746533723cdf46eebe95733bea7ce24ba
There's no point having always-false tests. It just makes the headers
harder to read.
Bug: 11560081
Change-Id: I6187755e1514ca9ff5642b3c1b0489f22edddaf5
If you rewrite the tokens of a #if you need to rewrite the expression to match
because either might be used later. This was showing up as SIGRTMAX being
rewritten in a #define but not in the #ifndef that guarded it, for which case
I've added a unit test.
Change-Id: I6929675461a1afe272edd667594529fd84a3dc4d
__SIGRTMIN will continue to tell the truth. This matches glibc's
behavior (as evidenced by the fact that we don't need a special case
in the strsignal test now).
Change-Id: I1abe1681d516577afa8cd39c837ef12467f68dd2
Our sigset_t definition hasn't been tied to our NSIG definition since we
switched to uapi headers, so we can now fix it without breaking the LP32 ABI.
The kernel uapi headers define and use _NSIG, so we need to have our scripts
rename the kernel's definitions out of the way, then we can define _NSIG
and NSIG in terms of the kernel's off-by-one value.
Bug: 12938442
Change-Id: Ic7c86fd5be5ad1d822f7b2b1d88c8a0d70a1ac0f