Commit graph

528 commits

Author SHA1 Message Date
Colin Cross
d27506716a Define sockaddr_storage in bionic
Bionic currently renames the kernel's __kernel_sockaddr_storage
to sockaddr_storage.  We would like to reuse the bionic kernel
headers for musl, but musl defines its own sockaddr_storage,
causing redefinition errors when the musl and kernel headers
are both included.

Give bionic its own definition of sockaddr_storage so the kernel
one can be left as __kernel_sockaddr_storage, avoiding the
conflict in musl.

Some of the structs in netinet/in.h contain sockaddr_storage members.
There are definitions in linux/in.h, but they now use the
__kernel_sockaddr_storage naming.  To avoid having to copy the structs
from linux/in.h into netinet/in.h, add some #define hackery to
rename __kernel_sockaddr_storage to sockaddr_storage when including
linux/in.h from netinet/in.h.

Bug: 190084016
Test: m checkbuild
Test: bionic-unit-tests
Change-Id: I8f654511722b3a4a8e0bb146e4418ebcd370305c
2021-07-14 14:20:36 -07:00
Treehugger Robot
df901df77a Merge "Support replacing nested kernel structs" 2021-07-12 23:13:59 +00:00
Christopher Ferris
fcc3b4f45d Update to v5.13 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.13

Test: Builds, bionic unit tests.
Change-Id: Ia4360ca4a2808206f64187dcb234d73c5752624b
2021-07-01 01:30:21 +00:00
Colin Cross
68ec53e141 Support replacing nested kernel structs
When replacing structs with bits/ includes, count the number of
opening and closing braces to find the end of the struct.

Test: tools/update_all.py
Change-Id: Ic9712a385da517710ceffccbdb223c89ae0b5f65
2021-06-29 14:22:30 -07:00
Christopher Ferris
a9750eda0d Update to v5.12 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.12

Test: Boots flame device and runs bionic unit tests.
Test: Boots acloud device and runs bionic unit tests.

Change-Id: I8ac107ce9d4978be3ef9517b90ad6ecafd06785a
2021-05-03 15:01:39 -07:00
Christopher Ferris
e6b7969dd1 Merge "Fix last python3 issues." 2021-04-21 20:02:10 +00:00
Christopher Ferris
a2142d2835 Fix last python3 issues.
Includes pointing to the python3 version of the clang bindings.

Also, remove stale .gitignore line.

Test: Ran bionic/libc/kernel/tools/update_all.py and verified
Test: the files generated the same exact way.
Change-Id: I4eb9dd7382bca013f70d921b6ef48c7e7478615a
2021-04-20 17:36:07 -07:00
Dan Albert
8607c084c2 Fix overly-zealous strip.
The argument to this is the characters to strip, so `line.strip(line)`
just returns the empty string.

Test: None?
Bug: None
Change-Id: I4f62bffcd00936e4eef837a28b78023fcad54bb0
2021-04-20 16:05:02 -07:00
Christopher Ferris
ac7ec11473 Update for python3.
This fixes all of the problems with our kernel scripts, but not
the clang python script problems.

I also removed the updateGitFiles function since that code was
just silently failing any way. I replaced all calls with updateFiles.

Test: Ran script using python2 to verify it still works.
Test: Run script in python3 verifying that it starts to run.

Change-Id: I223a31a8324c59e6bc4067f48a6110361b3e26e8
2021-04-19 17:02:38 -07:00
Elliott Hughes
6b586e7709 Force everything to use python3 for consistency.
Rather than "whatever people have installed as 'python' on their machine".

I've removed check-symbols.py because that's been broken for years and
we never even noticed, and I'm not sure it's worth fixing.

Test: treehugger, manual
Change-Id: Ieb996bbdf790a18d4b1fb46a409cc240ba2a2a49
2021-04-15 13:39:08 -07:00
Christopher Ferris
7e2f6ffeaf Update kernel update documentation.
Parts of the document have gone out of date, so update it.

Test: NA
Change-Id: Ia4bc534ca1cc9a538d5995126c541c2cd7029c57
2021-03-18 00:38:34 -07:00
Christopher Ferris
05667cd66a Update to v5.11 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.11

Test: Built cuttlefish and flame images. Ran bionic unit tests on both.
Change-Id: Ie60337aafad4bda55af99b6c8fe9f56bf2fa787f
2021-02-16 16:01:34 -08:00
Christopher Ferris
32ff3f831c Update to v5.10 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.10

Test: Built cuttlefish and flame images. Ran bionic unit tests on both.
Change-Id: I37ffc850970adcce1febbe2269c202632fce763a
2020-12-14 22:37:59 -08:00
Christopher Ferris
ccf17f0ec7 Clean up header files from kernel directory.
Add a make distclean for each supported arch before generating headers
and at the end so these files are not left hanging around in the
kernel directory.

Bug: 174795055

Test: Verified all generated files are deleted after running.
Test: Verified this generates the exact same files as before this change.
Change-Id: I0f66f580af2ea50c190eb532032017e782d7cae6
2020-12-03 18:18:52 -08:00
Christopher Ferris
25c18d45cf Update to v5.9 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.9

Test: Boots cuttlefish 64bit, passes 32 bit and 64 bit bionic unit tests.
Change-Id: Ib5503355b238ea75595538e63eb000c867d06ef7
2020-10-19 15:48:58 -07:00
Christopher Ferris
8177cdf178 Update to v5.8 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.8

Test: NA
Change-Id: I2231c877589820fc09800a200cf4ac62ba74b04c
2020-08-04 13:50:49 -07:00
Elliott Hughes
cf346532fc More cleanup for #inclusivefixit.
Found manually with grep, since the script seems to miss stuff.

Test: treehugger
Change-Id: I5933cbade9792801d4a0bec1ccb077efa6ad8fbc
2020-07-31 10:35:03 -07:00
Christopher Ferris
430527b4cd Update the generate script.
The arm kvm.h file has been deprecated, so nobody should be including it.
Therefore, remove the hack to copy the file out of the tools directory.

Also, update to remove the arm kvm.h file.

Test: Ran generate and update script and verified only the arm kvm.h
Test: was deleted.
Test: Builds and boots on a walleye.
Change-Id: I95dcc8877bdb2da2858b0b9d0aa95d1d2072cf9a
2020-06-04 12:32:09 -07:00
Christopher Ferris
af09c709d5 Update to v5.7 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.7

Update the generate_uapi_headers.sh where the types.h file is now in
the right place, but kvm.h is not.

Test: Booted cuttlefish/bonito.
Test: Ran bionic-unit-tests on cuttlefish/bonito.
Change-Id: Ice9ce370a658e320b80f564b34a4431927fcf100
2020-06-02 10:58:28 -07:00
Christopher Ferris
bb9fcb4636 Update to v5.6 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.6

Add a new method for removing structures. This is to deal with the kernel
headers changing some definitions of timeval to __kernel_old_timeval
and itimerval to __kernel_old_itimerval. Remove the __kernel_old_XX
strutures and change the other structures to the previous definitions.

This only works so long as these structures stay the same, if they
diverge, then a different strategy will need to be implemented.

Test: Booted cuttlefish/walleye.
Test: Ran bionic-unit-tests on cuttlefish/walleye.
Change-Id: I0a61f4fa6e4155c602e0414d9b38c2e1637829af
2020-04-10 10:53:48 -07:00
Christopher Ferris
d32ca14ea3 Update to v5.5 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.5

Test: Boots on walleye.
Test: Ran bionic-unit-tests on walleye.
Test: Boots on cuttlefish.
Test: Ran bionic-unit-tests on cuttlefish.
Change-Id: I57387d3c31e0ba5ad125ffe291cecf365c7b374e
Merged-In: I57387d3c31e0ba5ad125ffe291cecf365c7b374e
2020-02-07 20:14:42 +00:00
Elliott Hughes
c2faf235c0 Stop generating unused headers.
These just cause confusion because they often have different
values/layouts, but they're never actually used.

Test: treehugger
Change-Id: I424034088e017c919f62fcefa7d6d3f903f31cfb
2020-02-03 17:56:06 -08:00
Christopher Ferris
ce11673cad Merge "Fully disable clang format where needed." 2020-01-30 21:47:39 +00:00
Christopher Ferris
1cc755c8cf Fully disable clang format where needed.
Even with formatting off, clang still tries to rearrange the include
files or the using statements, so disable that too.

Test: Verified that the include directories are not rearranged.
Change-Id: I991a1b2bfa94a8202c5a486664658d654f1c7811
2020-01-30 08:10:17 -08:00
Elliott Hughes
09e77f35ab _FILE_OFFSET_BITS=64 support for fcntl.
On Android, fcntl is always implemented by fcntl64(2). This means that
an LP32 binary can `fcntl(F_SETLK, struct flock)` (because fcntl64(2)
passes through to the 32-bit fcntl(2) to handle F_SETLK), and it can
also `fcntl(F_SETLK64, struct flock64)`. What it can't do before this
patch is set _FILE_OFFSET_BITS=64 and then `fcntl(F_SETLK, struct
flock)` where that `struct flock` is actually implicitly `struct
flock64`.

Move the kernel uapi structs out of the way, define them ourselves based
on __LP64__ and _FILE_OFFSET_BITS, and fix up the relevant F_ constants.

(Also add a .clang-format to turn off clang-format in libc/include/.)

Bug: N/A
Test: treehugger (and strace!)
Change-Id: Iccd6c83d9133e1efcf93a7b49a6ae0f1bbd3d58b
2020-01-29 19:36:14 -08:00
Christopher Ferris
fdaf82f96b Link .clang-format file to system/core one.
Use the .clang-format-2 found in system/core instead of this which is
not actually being used.

Also, enable clang-format running by default.

All upstream directories are marked as ignoring formatting so that
their source files are not modified.

Test: NA
Change-Id: Icee6030f373fa5f072df162f97e6f34320e3d89a
2020-01-29 15:14:20 -08:00
Christopher Ferris
c58208bd3a Merge "Update documentation around kernel header imports." 2019-12-16 17:48:52 +00:00
Josh Gao
afe7e53003 Clone with --depth 1 in generate_uapi_headers.sh --download-kernel.
Test: manual
Change-Id: Id60b354fb1914470315976dd103d2665a6fefab8
2019-12-13 17:24:44 -08:00
Christopher Ferris
86a8f3f32d Update documentation around kernel header imports.
Test: NA
Change-Id: I8866ef30efbc565e8996c574a9f895c574cd8e55
2019-12-13 17:20:20 -08:00
Christopher Ferris
9584fa46e9 Update to android-mainline kernel headers v5.4.
Update generate_uapi_headers.sh to checkout the android mainline
kernel. Also, add a small modification to look for the kernel directory
in common not linux-stable.

Remove deprecated android headers from android/uapi/linux. Also,
remove f_accessory.h since it's in the android mainline kernel.

Test: Builds and runs on walleye.
Change-Id: Ia371305e19f56e6bcc2db6d5b4d299819f07ffc6
2019-12-12 14:59:25 -08:00
Christopher Ferris
b8a95e2186 Update to kernel headers v5.3.2.
Test: Builds and run unit tests on taimen/cuttlefish.
Change-Id: I6ebd8f179d159ac974555e8edca588083e8081b3
2019-10-03 10:59:32 -07:00
Christopher Ferris
e892fd6b17 Update to kernel headers v5.2.11.
Test: Build and boots on taimen.
Test: Bionic unit tests all pass on taimen.
Change-Id: I80abb6f9abbff9ca20ce6c7c912a259b5ca86fa2
2019-09-03 15:56:49 -07:00
Christopher Ferris
fa59a10bcd Update to kernel headers v5.2.6.
Also, fix a bug in update_all.py when the syscalls file does not change.

Test: Builds and boots on a taimen.
Change-Id: If85b00daef2c176f804a0861894f5bbca9c6d5df
2019-08-05 12:59:58 -07:00
Christopher Ferris
aeddbcfb2c Update to kernel headers v5.2.
Test: Booted a taimen, ran unit bionic unit tests.
Change-Id: I3522c59793bbcef98ea515996a481d92f14b8816
2019-07-08 13:54:11 -07:00
Christopher Ferris
24f97eb27b Update to kernel headers to v5.1.3.
Test: Builds and boots taimen.
Test: Ran bionic unit tests.
Change-Id: Ieaca8709d568f075df8cac60c9da2c0ed1916963
2019-05-20 12:58:13 -07:00
Elliott Hughes
782c485880 Generate assembler system call stubs via genrule.
There's no need to check in generated code.

Test: builds & boots
Change-Id: Ife368bca4349d4adeb0666db590356196b4fbd63
2019-04-16 12:31:00 -07:00
Daniel Mentz
d12d6f67bc clean_header: Run outside of $ANDROID_BUILD_TOP
Enable the use case where we run clean_header.py from outside of
$ANDROID_BUILD_TOP. Previously, this script required the current working
directory to be under $ANDROID_BUILD_TOP. Running it from a different
directory resulted in the following error message:

 clean_header.py: error: Not in android tree pointed at by ANDROID_BUILD_TOP (....)

Change-Id: I48210ea1a0033228a9aaa4124d28247b07cee6d4
2019-04-15 15:19:31 -07:00
Daniel Mentz
6d6b4cedd1 clean_header: Fix error handling for no-such-file case
The function cleanupFile should always return a single value (as opposed
to a tuple or list). In addition, if it encounters an error, it is
expected to return a value that evaluates to False. As it stands,
however, it returns (None, None) in certain error cases. Change this
function to return None, in those cases.

We previously saw the following error message, when we tried to run
clean_header.py on a non-existent file.

Traceback (most recent call last):
  File "clean_header.py", line 208, in <module>
    b.updateGitFiles()
  File "utils.py", line 164, in updateGitFiles
    self._writeFile(dst)
  File "utils.py", line 136, in _writeFile
    f.write(self.new_data[dst])
TypeError: expected a string or other character buffer object

Change-Id: I5f717dd1a4388f598f0fd4bfd5e6129017de9095
2019-04-15 15:03:39 -07:00
Treehugger Robot
9586c0f4f4 Merge "Move kernel README files to Markdown." 2019-04-12 03:11:30 +00:00
Elliott Hughes
c4c2e24d5f <bits/glibc-syscalls.h>: only regenerate when we have new uapi headers.
Test: update_all.py
Change-Id: Iaa92dce263197f5a0e7d2dce5e00a31372dcb3e9
2019-04-11 14:19:17 -07:00
Elliott Hughes
adcb515bfc Move kernel README files to Markdown.
Test: N/A
Change-Id: If987d8a24baf87adba20914896d29c94fc01cbfd
2019-04-11 13:38:25 -07:00
Daniel Mentz
bb4cf7b8a7 clean_header: Don't print extraneous newline char
Don't print an extraneous newline character at the end of the include
file if clean_header.py is used in the non-update mode. This is
necessary to achieve consistent results in the update and non-update
modes.

Running clean_header in the non-update mode and redirecting the output
to a file should have the same result as using the update mode and have
the script write to the file directly.

Change-Id: I6b176c5365840d66e4499bacd205f1fa77302a2b
2019-03-29 15:17:48 -07:00
Daniel Mentz
316f4a499c clean_header: Write to correct dst_file
Write to correct dst_file when in update mode. This enables use cases
like the following:

../../../bionic/libc/kernel/tools/clean_header.py -k original-kernel-headers/ -d kernel-headers/ -u linux/ion.h

Previously, we had to cd into kernel-headers/ and then run

../../../../bionic/libc/kernel/tools/clean_header.py -k ../original-kernel-headers/ -d . -u linux/ion.h

because the tool didn't allow the relative path to be different from the
destination path.

Change-Id: I8c5e284ce7a6737d77a2f5ead3e7e5db01317425
2019-03-29 21:41:15 +00:00
Christopher Ferris
e6be532ed6 Update kernel headers to v5.0.3.
Test: Build and boot taimen.
Change-Id: I6ed881b5dc6dc7d0a6cc56edcd09b42ddd185b62
Merged-In: I6ed881b5dc6dc7d0a6cc56edcd09b42ddd185b62
2019-03-21 15:31:06 +00:00
Christopher Ferris
d842e43e1d Update to v5.0 kernel headers.
Test: Builds and boots on taimen.
Change-Id: I13843bf1ab30ea89a50852adc88f2cba8401bded
2019-03-07 11:19:22 -08:00
Christopher Ferris
86a4837a8e Update to v4.20.1 kernel headers.
This includes one manual change:

In the file bionic/libc/kernel/uapi/linux/in.h, the macro IN_BADCLASS
was not definied correctly. Change the macro from:

  #define IN_BADCLASS(a) ((((long int) (a)) == 0xffffffff)

to:

  #define IN_BADCLASS(a) (((long int) (a)) == (long int)0xffffffff)

This change is being pushed to the upstream kernels.

Test: Builds and boots.
Change-Id: Ia304773a9dc6789b34d9769d73742384d6afb571
Merged-In: Ia304773a9dc6789b34d9769d73742384d6afb571
(cherry picked from commit 967fb01cce)
2019-01-15 07:33:14 -08:00
Christopher Ferris
48fe0aef16 Update kernel headers with new parser.
Test: Builds.
Change-Id: I3f0714d53ac893ccc3e66f7a92d0ea7a6737f1c3
2019-01-10 17:33:56 -08:00
Christopher Ferris
658b16fd90 Rewrite removeVarsAndFuncs.
The current version has these bugs:
- Adding a semicolon after a function results in the removal of structures
  following the function.
- Function like macros get removed on accident rather than on purpose.
- It removes extern "C" { completely, which might not be a bug, but doesn't
  seem right.

I couldn't easily fix any of these problems because the code depends heavily
on the header being correct.

New unit tests added for the function to cover all of these cases.

A follow-on CL will include the updated headers.

Bug: 112290385

Test: Passes all new unit tests.
Test: When run on the current kernel headers, the generated headers are
Test: nearly the same, missing data is being added.
Change-Id: Ib22a5f2e78873544e8a9d54e385af1156b2a72bb
2019-01-10 15:15:15 -08:00
Christopher Ferris
9ce28844db Update to v4.19 kernel headers.
Test: Builds and boots.
Change-Id: I99a9ed79666e143b47f02ca4e59eed94f69b7e4a
(cherry picked from commit a981e2e52e)
2018-10-30 09:21:16 -07:00
Christopher Ferris
f2484aaed7 Fix handling of #elif.
When a construct like:

if defined(something)
  blocks1
elif 1
  blocks2
else
  blocks3
endif

The parser would put the first clause but then simply omit the elif and
put all of blocks2 without a terminating #endif. The code also did
something similar when the #else was an #endif.

Also convert all of the unit tests to real unit tests and only run
them if you run cpp.py by itself. Added new unit tests to cover the
new cases.

Test: Ran cpp.py unit tests.
Test: Reran update_all.py and verified nothing changed, and that running
Test: it on the new kernel headers that exposed this problem.
Change-Id: Ie168511303c4e15afdb60c37baef75a966ca29a8
2018-10-26 22:42:26 +00:00