Commit graph

509 commits

Author SHA1 Message Date
Christopher Ferris
bb9fcb4636 Update to v5.6 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.6

Add a new method for removing structures. This is to deal with the kernel
headers changing some definitions of timeval to __kernel_old_timeval
and itimerval to __kernel_old_itimerval. Remove the __kernel_old_XX
strutures and change the other structures to the previous definitions.

This only works so long as these structures stay the same, if they
diverge, then a different strategy will need to be implemented.

Test: Booted cuttlefish/walleye.
Test: Ran bionic-unit-tests on cuttlefish/walleye.
Change-Id: I0a61f4fa6e4155c602e0414d9b38c2e1637829af
2020-04-10 10:53:48 -07:00
Christopher Ferris
d32ca14ea3 Update to v5.5 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.5

Test: Boots on walleye.
Test: Ran bionic-unit-tests on walleye.
Test: Boots on cuttlefish.
Test: Ran bionic-unit-tests on cuttlefish.
Change-Id: I57387d3c31e0ba5ad125ffe291cecf365c7b374e
Merged-In: I57387d3c31e0ba5ad125ffe291cecf365c7b374e
2020-02-07 20:14:42 +00:00
Elliott Hughes
c2faf235c0 Stop generating unused headers.
These just cause confusion because they often have different
values/layouts, but they're never actually used.

Test: treehugger
Change-Id: I424034088e017c919f62fcefa7d6d3f903f31cfb
2020-02-03 17:56:06 -08:00
Christopher Ferris
ce11673cad Merge "Fully disable clang format where needed." 2020-01-30 21:47:39 +00:00
Christopher Ferris
1cc755c8cf Fully disable clang format where needed.
Even with formatting off, clang still tries to rearrange the include
files or the using statements, so disable that too.

Test: Verified that the include directories are not rearranged.
Change-Id: I991a1b2bfa94a8202c5a486664658d654f1c7811
2020-01-30 08:10:17 -08:00
Elliott Hughes
09e77f35ab _FILE_OFFSET_BITS=64 support for fcntl.
On Android, fcntl is always implemented by fcntl64(2). This means that
an LP32 binary can `fcntl(F_SETLK, struct flock)` (because fcntl64(2)
passes through to the 32-bit fcntl(2) to handle F_SETLK), and it can
also `fcntl(F_SETLK64, struct flock64)`. What it can't do before this
patch is set _FILE_OFFSET_BITS=64 and then `fcntl(F_SETLK, struct
flock)` where that `struct flock` is actually implicitly `struct
flock64`.

Move the kernel uapi structs out of the way, define them ourselves based
on __LP64__ and _FILE_OFFSET_BITS, and fix up the relevant F_ constants.

(Also add a .clang-format to turn off clang-format in libc/include/.)

Bug: N/A
Test: treehugger (and strace!)
Change-Id: Iccd6c83d9133e1efcf93a7b49a6ae0f1bbd3d58b
2020-01-29 19:36:14 -08:00
Christopher Ferris
fdaf82f96b Link .clang-format file to system/core one.
Use the .clang-format-2 found in system/core instead of this which is
not actually being used.

Also, enable clang-format running by default.

All upstream directories are marked as ignoring formatting so that
their source files are not modified.

Test: NA
Change-Id: Icee6030f373fa5f072df162f97e6f34320e3d89a
2020-01-29 15:14:20 -08:00
Christopher Ferris
c58208bd3a Merge "Update documentation around kernel header imports." 2019-12-16 17:48:52 +00:00
Josh Gao
afe7e53003 Clone with --depth 1 in generate_uapi_headers.sh --download-kernel.
Test: manual
Change-Id: Id60b354fb1914470315976dd103d2665a6fefab8
2019-12-13 17:24:44 -08:00
Christopher Ferris
86a8f3f32d Update documentation around kernel header imports.
Test: NA
Change-Id: I8866ef30efbc565e8996c574a9f895c574cd8e55
2019-12-13 17:20:20 -08:00
Christopher Ferris
9584fa46e9 Update to android-mainline kernel headers v5.4.
Update generate_uapi_headers.sh to checkout the android mainline
kernel. Also, add a small modification to look for the kernel directory
in common not linux-stable.

Remove deprecated android headers from android/uapi/linux. Also,
remove f_accessory.h since it's in the android mainline kernel.

Test: Builds and runs on walleye.
Change-Id: Ia371305e19f56e6bcc2db6d5b4d299819f07ffc6
2019-12-12 14:59:25 -08:00
Christopher Ferris
b8a95e2186 Update to kernel headers v5.3.2.
Test: Builds and run unit tests on taimen/cuttlefish.
Change-Id: I6ebd8f179d159ac974555e8edca588083e8081b3
2019-10-03 10:59:32 -07:00
Christopher Ferris
e892fd6b17 Update to kernel headers v5.2.11.
Test: Build and boots on taimen.
Test: Bionic unit tests all pass on taimen.
Change-Id: I80abb6f9abbff9ca20ce6c7c912a259b5ca86fa2
2019-09-03 15:56:49 -07:00
Christopher Ferris
fa59a10bcd Update to kernel headers v5.2.6.
Also, fix a bug in update_all.py when the syscalls file does not change.

Test: Builds and boots on a taimen.
Change-Id: If85b00daef2c176f804a0861894f5bbca9c6d5df
2019-08-05 12:59:58 -07:00
Christopher Ferris
aeddbcfb2c Update to kernel headers v5.2.
Test: Booted a taimen, ran unit bionic unit tests.
Change-Id: I3522c59793bbcef98ea515996a481d92f14b8816
2019-07-08 13:54:11 -07:00
Christopher Ferris
24f97eb27b Update to kernel headers to v5.1.3.
Test: Builds and boots taimen.
Test: Ran bionic unit tests.
Change-Id: Ieaca8709d568f075df8cac60c9da2c0ed1916963
2019-05-20 12:58:13 -07:00
Elliott Hughes
782c485880 Generate assembler system call stubs via genrule.
There's no need to check in generated code.

Test: builds & boots
Change-Id: Ife368bca4349d4adeb0666db590356196b4fbd63
2019-04-16 12:31:00 -07:00
Daniel Mentz
d12d6f67bc clean_header: Run outside of $ANDROID_BUILD_TOP
Enable the use case where we run clean_header.py from outside of
$ANDROID_BUILD_TOP. Previously, this script required the current working
directory to be under $ANDROID_BUILD_TOP. Running it from a different
directory resulted in the following error message:

 clean_header.py: error: Not in android tree pointed at by ANDROID_BUILD_TOP (....)

Change-Id: I48210ea1a0033228a9aaa4124d28247b07cee6d4
2019-04-15 15:19:31 -07:00
Daniel Mentz
6d6b4cedd1 clean_header: Fix error handling for no-such-file case
The function cleanupFile should always return a single value (as opposed
to a tuple or list). In addition, if it encounters an error, it is
expected to return a value that evaluates to False. As it stands,
however, it returns (None, None) in certain error cases. Change this
function to return None, in those cases.

We previously saw the following error message, when we tried to run
clean_header.py on a non-existent file.

Traceback (most recent call last):
  File "clean_header.py", line 208, in <module>
    b.updateGitFiles()
  File "utils.py", line 164, in updateGitFiles
    self._writeFile(dst)
  File "utils.py", line 136, in _writeFile
    f.write(self.new_data[dst])
TypeError: expected a string or other character buffer object

Change-Id: I5f717dd1a4388f598f0fd4bfd5e6129017de9095
2019-04-15 15:03:39 -07:00
Treehugger Robot
9586c0f4f4 Merge "Move kernel README files to Markdown." 2019-04-12 03:11:30 +00:00
Elliott Hughes
c4c2e24d5f <bits/glibc-syscalls.h>: only regenerate when we have new uapi headers.
Test: update_all.py
Change-Id: Iaa92dce263197f5a0e7d2dce5e00a31372dcb3e9
2019-04-11 14:19:17 -07:00
Elliott Hughes
adcb515bfc Move kernel README files to Markdown.
Test: N/A
Change-Id: If987d8a24baf87adba20914896d29c94fc01cbfd
2019-04-11 13:38:25 -07:00
Daniel Mentz
bb4cf7b8a7 clean_header: Don't print extraneous newline char
Don't print an extraneous newline character at the end of the include
file if clean_header.py is used in the non-update mode. This is
necessary to achieve consistent results in the update and non-update
modes.

Running clean_header in the non-update mode and redirecting the output
to a file should have the same result as using the update mode and have
the script write to the file directly.

Change-Id: I6b176c5365840d66e4499bacd205f1fa77302a2b
2019-03-29 15:17:48 -07:00
Daniel Mentz
316f4a499c clean_header: Write to correct dst_file
Write to correct dst_file when in update mode. This enables use cases
like the following:

../../../bionic/libc/kernel/tools/clean_header.py -k original-kernel-headers/ -d kernel-headers/ -u linux/ion.h

Previously, we had to cd into kernel-headers/ and then run

../../../../bionic/libc/kernel/tools/clean_header.py -k ../original-kernel-headers/ -d . -u linux/ion.h

because the tool didn't allow the relative path to be different from the
destination path.

Change-Id: I8c5e284ce7a6737d77a2f5ead3e7e5db01317425
2019-03-29 21:41:15 +00:00
Christopher Ferris
e6be532ed6 Update kernel headers to v5.0.3.
Test: Build and boot taimen.
Change-Id: I6ed881b5dc6dc7d0a6cc56edcd09b42ddd185b62
Merged-In: I6ed881b5dc6dc7d0a6cc56edcd09b42ddd185b62
2019-03-21 15:31:06 +00:00
Christopher Ferris
d842e43e1d Update to v5.0 kernel headers.
Test: Builds and boots on taimen.
Change-Id: I13843bf1ab30ea89a50852adc88f2cba8401bded
2019-03-07 11:19:22 -08:00
Christopher Ferris
86a4837a8e Update to v4.20.1 kernel headers.
This includes one manual change:

In the file bionic/libc/kernel/uapi/linux/in.h, the macro IN_BADCLASS
was not definied correctly. Change the macro from:

  #define IN_BADCLASS(a) ((((long int) (a)) == 0xffffffff)

to:

  #define IN_BADCLASS(a) (((long int) (a)) == (long int)0xffffffff)

This change is being pushed to the upstream kernels.

Test: Builds and boots.
Change-Id: Ia304773a9dc6789b34d9769d73742384d6afb571
Merged-In: Ia304773a9dc6789b34d9769d73742384d6afb571
(cherry picked from commit 967fb01cce)
2019-01-15 07:33:14 -08:00
Christopher Ferris
48fe0aef16 Update kernel headers with new parser.
Test: Builds.
Change-Id: I3f0714d53ac893ccc3e66f7a92d0ea7a6737f1c3
2019-01-10 17:33:56 -08:00
Christopher Ferris
658b16fd90 Rewrite removeVarsAndFuncs.
The current version has these bugs:
- Adding a semicolon after a function results in the removal of structures
  following the function.
- Function like macros get removed on accident rather than on purpose.
- It removes extern "C" { completely, which might not be a bug, but doesn't
  seem right.

I couldn't easily fix any of these problems because the code depends heavily
on the header being correct.

New unit tests added for the function to cover all of these cases.

A follow-on CL will include the updated headers.

Bug: 112290385

Test: Passes all new unit tests.
Test: When run on the current kernel headers, the generated headers are
Test: nearly the same, missing data is being added.
Change-Id: Ib22a5f2e78873544e8a9d54e385af1156b2a72bb
2019-01-10 15:15:15 -08:00
Christopher Ferris
9ce28844db Update to v4.19 kernel headers.
Test: Builds and boots.
Change-Id: I99a9ed79666e143b47f02ca4e59eed94f69b7e4a
(cherry picked from commit a981e2e52e)
2018-10-30 09:21:16 -07:00
Christopher Ferris
f2484aaed7 Fix handling of #elif.
When a construct like:

if defined(something)
  blocks1
elif 1
  blocks2
else
  blocks3
endif

The parser would put the first clause but then simply omit the elif and
put all of blocks2 without a terminating #endif. The code also did
something similar when the #else was an #endif.

Also convert all of the unit tests to real unit tests and only run
them if you run cpp.py by itself. Added new unit tests to cover the
new cases.

Test: Ran cpp.py unit tests.
Test: Reran update_all.py and verified nothing changed, and that running
Test: it on the new kernel headers that exposed this problem.
Change-Id: Ie168511303c4e15afdb60c37baef75a966ca29a8
2018-10-26 22:42:26 +00:00
Christopher Ferris
76a1d45599 Update to v4.17.3 kernel headers.
Test: Builds, boots on a walleye.
Change-Id: I389d8b61ec00ea309e38d1b1a2e0dace48c21edb
2018-06-27 14:12:29 -07:00
Christopher Ferris
764049dce4 Modify the script to use the linux source tree.
The current code checks out the android kernel tree. Long ago, we moved
to using the linux kernel source code. Modify the script to actually
do this.

Bug: 6653610

Test: Ran the script using the --download_kernel option and without.
Change-Id: I769e769edea272f4f8c9c8d2e76bd336059c66e3
2018-05-24 15:10:33 -07:00
Christopher Ferris
5ed15ba733 Remove ion.h from bionic uapi headers.
Bug: 77976082

Test: Ran the generate script and verified ion.h is deleted.
Change-Id: I33b3aeef66f08f35dd496a073931b2fe2c80a1d6
2018-04-24 13:33:30 -07:00
Christopher Ferris
038ef23f62 Use the ion.h from the kernel headers.
All code that needs to use the frozen version of ion.h header should
have a copy, so move to the version as parsed from the real headers.

Test: Compiled and boots.
Change-Id: If9feae9c163ca64c4862045e86611fd3cf1fc18e
2018-02-22 20:22:14 +00:00
Christopher Ferris
934ec94955 Update to v4.15 kernel headers.
Test: Compiles, boots bullhead/hikey960.
Change-Id: I118beb8b6cac0881b1270f9bf6981959297a41a8
2018-02-01 10:51:01 -08:00
Elliott Hughes
64f355ffbd Use in_addr in ip_mreq_source and ip_msfilter.
This adds a new mechanism to say "replace struct S with #include <bits/S.h>".

Also switch epoll_event over to the new mechanism.

Also use the kernel's struct sockaddr_storage directly rather than behind
an unnecessary #define.

This patch also removes some dead code in the header scrubber. This code
still needs rewriting completely. I learned that a "block" isn't necessarily
a single struct definition, say; it might be a run of them. It seems like
a block is a run of preprocessor directives or a run of regular code.

Bug: https://issuetracker.google.com/36987220
Test: new test
Change-Id: Ic6a5c09559766a4babe3cd4c3ea538b885e07308
2017-12-14 10:07:27 -08:00
Elliott Hughes
8c75675ec6 Merge "Change ARG_MAX/_SC_ARG_MAX back to a constant." 2017-11-16 21:46:49 +00:00
Elliott Hughes
baed51ee3a Change ARG_MAX/_SC_ARG_MAX back to a constant.
As per the lkml thread https://lkml.org/lkml/2017/11/1/946.

Bug: http://b/65818597
Test: ran tests
Change-Id: I7a0610e6903e6761f2b31416e2f5017bd7a60659
2017-11-15 15:14:35 -08:00
Christopher Ferris
1308ad3ab3 Update to kernel headers v4.14.
Remove the hiding of the kernel structure binder_fd_array_object. This
structure now matches the structure used in the binder code.

Load the libclang_android.so shared library directly for parsing.
This file changed name in a recent update to the prebuilts.

Test: Compiles arm/arm64/x86/x86_64.
Test: Boots on hikey and boots on a sailfish.
Test: Ran bionic unit tests on hikey and sailfish.
Change-Id: I141a4b93ac3511cd58f4d12bb3c0d4efaa4c2742
2017-11-15 12:23:56 -08:00
Stephen Crane
77bb564dc7 Use env to invoke python
/usr/bin/python may be python3. We should respect PATH to find the python
executable so it can be locally overridden to be python2.

Test: Build libc, repo upload
Change-Id: Iaddd7cd4a1c2177c32786e4fa0fc664ab0ad36de
2017-08-31 15:11:50 -07:00
Christopher Ferris
02ce96835b Revert "Revert "Move back to the previous version of ion.h.""
This reverts commit 89d6c321f8.

Reason for revert: Still a bunch of branches/builds depending on this incorrectly.

Change-Id: I8de678848b0e5445837c02c0942f7388d27dbfd7
2017-08-03 18:55:45 +00:00
Christopher Ferris
89d6c321f8 Revert "Move back to the previous version of ion.h."
This reverts commit 1a0344d65d.

Reason for revert: Fixed the original problem.

Change-Id: Ibf4ba05160048a3216e0cc49cc88519f73d00fa4
2017-08-01 19:32:14 +00:00
Christopher Ferris
1a0344d65d Move back to the previous version of ion.h.
The new kernel headers changed this and lots of parts of the tree
want the old definition, so restore the previous version.

Test: Builds.
Change-Id: I725e56bb7c51c9fcd23ebc4b67eac33d66e196e7
2017-07-29 13:45:07 -07:00
Christopher Ferris
525ce914ed Update to kernel headers v4.12.3.
Test: Built angler.
Change-Id: Icbcf4fac2334de8409b049ed7a3b4c24b4e98ce9
2017-07-27 16:01:39 -07:00
Elliott Hughes
5059939c87 Enable exporting the non-uapi scsi headers.
Callers are supposed to #include <scsi/sg.h> but if we tell soong to add
bionic/libc/kernel/android/ to the include path, the uapi headers in there
would be (unintentionally) accessible as either <linux/name.h> or
<uapi/linux/name.h>.

Bug: N/A (hit while upgrading strace to 4.17)
Test: builds
Change-Id: I8d47dd51da688c38f747a255d401dfb2c209c805
2017-05-25 19:17:55 -07:00
Elliott Hughes
96c1db7b9d Remove the repetitive warnings from the uapi headers.
Having

  WARNING: DO NOT EDIT, AUTO-GENERATED CODE - SEE TOP FOR INSTRUCTIONS

every four lines made the headers harder to read, made the diffs much worse
each time we upgraded, and wasn't really providing any benefit. Before the
next uapi update, let's just stop doing this.

Bug: N/A
Test: builds, manually inspected files look right
Change-Id: Id7088cf750894c9d24950f3d53587fe3156c4f7d
2017-05-25 13:48:01 -07:00
Elliott Hughes
497ad30d7d Make union semun usable.
This is a bit bogus because it's been removed from glibc (though not
thoroughly) and is never useful on Android (because the system calls
in question are compiled out of Android kernels, and SELinux would
disallow them even if you weren't running an Android kernel). This
also means that on glibc you need to include <linux/sem.h> for this
and on bionic you need <sys/sem.h> (and for either if you #include
the other file, you won't get this union).

Bug: https://github.com/android-ndk/ndk/issues/400
Test: added new test
Change-Id: I47f721da77515531f616d6ad8479bfbc9b60ee47
2017-05-18 15:05:26 -07:00
Christopher Ferris
ee1e0a34ae Update static functions/macro definitions.
Fix the list of static functions coming from swab.h to match the
ones exported by glibc.

Force the definition of __HAVE_BUILTIN_BSWAP{16,32,64}__ so the headers
use the builtin.

Add a unit test to guarantee that kernel header updates do not break this.

Test: Built and booted angler.
Test: Built the bionic unit tests for arm, arm64, mips, x86, x86_64.
Test: Ran the new test on angler and glibc.

Change-Id: I4ce229e3f198c204186d72bf22dd97b5cdf239e4
2017-04-21 20:59:53 -07:00
Christopher Ferris
b7e8fe5412 Merge "Refactor the kernel update scripts." 2017-03-03 23:40:12 +00:00