Commit graph

549 commits

Author SHA1 Message Date
Pat Tjin
00fde2aff6 Modify path for python clang support package
Due to commit 594844244237091347b62e8fbbc8a2851258db50 (update
stable to r468909b) the path for python3-clang support was moved from
   prebuilts/clang/host/linux-x86/clang-stable/lib64/python3/site-packages
to:
   prebuilts/clang/host/linux-x86/clang-stable/lib/python3/site-packages

Change the imports to point to the correct place.

Without the change:
  Traceback (most recent call last):
    File "bionic/libc/kernel/tools/./clean_header.py", line 72, in <module>
      import sys, cpp, kernel, glob, os, re, getopt
    File "bionic/libc/kernel/tools/cpp.py", line 19, in <module>
      import clang.cindex
  ModuleNotFoundError: No module named 'clang'

Bug: 254401594
Test: execute clean_headers.py
Signed-off-by: Pat Tjin <pattjin@google.com>
Change-Id: Ibf1423fbe78ee009366e26905c4a09e68ac8ff9b
2022-10-21 18:27:24 +00:00
Elliott Hughes
473dcc59f3 Remove Commodore 64 support.
https://en.wikipedia.org/wiki/KERNAL

Test: treehugger
Change-Id: I42fc21daa051eca29e6bda7f29c81dd8f353c14e
2022-10-19 16:13:09 +00:00
Elliott Hughes
bba6ddff12 Kernel header scrubber: remove unused map entries.
I was going to add riscv64 and define __riscv_xlen to 64 here, but our
"preprocessor" in python can't simpify `MACRO_NAME == 123` expressions
anyway. Rather than add another empty map, remove the other unused maps
and explain why this functionality is still useful as long as the legacy
32-bit ABIs are still with us.

Test: treehugger
Change-Id: I553059942920e7d323a0ca8a6fd5f39bc2d35063
2022-10-06 20:40:44 +00:00
Elliott Hughes
de1df210bb Remove a mips-specific hack.
Rather than add risc-v to this map, let's just remove the map!

Test: treehugger
Change-Id: I118c74c61af95a96f8e31ae6e422619ed86c6a3b
2022-10-05 00:05:05 +00:00
Elliott Hughes
dc810a3c13 Remove unused variable.
Looks like we're not using this any more, so rather than add risc-v,
let's just remove this list.

Test: treehugger
Change-Id: I24bb22235c55d48c7dcbae94514ce8dcd03521c3
2022-10-04 20:21:20 +00:00
Treehugger Robot
4d974e29d5 Merge "Clarify the docs for updating kernel headers." 2022-10-03 18:00:52 +00:00
Elliott Hughes
93fb6b9459 Clarify the docs for updating kernel headers.
When I followed these instructions for risc-v, I mistakenly thought
that the auto-download option was a convenience that would get the
currently-approved headers, not the absolute latest. Make it clearer in
the documentation that this is for experts only, not for production.

Test: N/A
Change-Id: Iedce8ca0403b83e13bbe339cf343f85a37e99334
2022-10-01 15:29:37 +00:00
Elliott Hughes
5e7f8f116d Initial import of the risc-v uapi headers.
From external/kernel-headers.

Signed-off-by: Mao Han <han_mao@linux.alibaba.com>
Signed-off-by: Xia Lifang <lifang_xia@linux.alibaba.com>
Signed-off-by: Chen Guoyin <chenguoyin.cgy@linux.alibaba.com>
Signed-off-by: Wang Chen <wangchen20@iscas.ac.cn>
Signed-off-by: Lu Xufan <luxufan@iscas.ac.cn>
Test: treehugger
Change-Id: Id73a86c9df77825853004137f8b801b24bba93f9
2022-10-01 15:10:58 +00:00
Elliott Hughes
05ef5d46c9 Add riscv64 to the list of uapi architectures.
Signed-off-by: Mao Han <han_mao@linux.alibaba.com>
Signed-off-by: Xia Lifang <lifang_xia@linux.alibaba.com>
Signed-off-by: Chen Guoyin <chenguoyin.cgy@linux.alibaba.com>
Signed-off-by: Wang Chen <wangchen20@iscas.ac.cn>
Signed-off-by: Lu Xufan <luxufan@iscas.ac.cn>
Test: bionic/libc/kernel/tools/generate_uapi_headers.sh
Change-Id: I926e1ceaab78b1bb9ae00cf39eb14005aae38e48
2022-09-30 21:37:57 +00:00
Christopher Ferris
80ae69d11e Update to v5.19 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.19

Test: Builds on coral.
Test: All bionic unit tests pass on a coral device.
Test: Able to log in to wembley system.
Change-Id: I6fa6f6edb59f2cef990a7ba96877d9f2d3974c62
2022-08-10 12:58:45 -07:00
Christopher Ferris
3fc4e11607 Add support for removing structs that cross blocks.
The latest kernel has a struct that needs to be removed,
but the struct crosses blocks. Add support for removing a
struct that does cross blocks. This support is very primitive,
and will not parse any really complicated struct that uses
defines in complicated ways.

Combine the kernel_structs_to_remove and kernel_struct_replacements
structure into a single map. This allows marking a structure
to be removed as replaced with an #include <bits/STRUCT.h>.

The new support for the remove of structures is all in the
removeStructs function.

Raise an exception if the struct parsing does not work properly.

Add new unit tests for all of the new code.

In addition, fix the algorithm for deleting the uapi directory
before it gets updated. A new file BUILD was checking in that
directory, so delete everything in the directory except that
BUILD file.

Test: Unit tests pass.
Test: Running update_all.py results in the no unexpected changes.
Change-Id: I9a8cef0321beaf71d03b5b874327747a7edb6119
2022-08-09 17:49:31 -07:00
Christopher Ferris
e0cb4e1094 Fix old python2 test issue.
The map function returns a list in python2, but not in python3.
Force the test in cpp.py get_blocks() function to return a list.

Test: All cpp.py unit tests pass.
Change-Id: I1a63b92ac6cdb785830846fbd7e726eca7520e96
2022-08-05 12:59:05 -07:00
Spandan Das
3523e8e393 Declare libc's contributions to API surface(s) in Multi-tree
As part of go/multitree-design, build dependencies between API domains
must be via stable API surfaces. The contributions to the API surfaces
will be declared in BUILD files so that we can skip the
migration from Soong->Bazel entirely for this (go/multi-tree-api-export)

libc is one such component of the system API domain that contributes to
the public API surface. This CL declares libc's contributions (its
headers/.map.txt/other_metadata). These BUILD files were generated
manually by looking at existing Android.bp files.

Test: STANDALONE_BAZEL=1 tools/bazel cquery
//bionic/libc:libc_contributions --output=starlark
--starlark:expr="providers(target).get(\"//build/bazel/rules/apis:cc_api_contribution.bzl%CcApiContributionInfo\")"
2>/dev/null | less

Change-Id: If2a429b782651872fd6d8bf8c291439fdf838d85
2022-06-29 20:40:11 +00:00
Christopher Ferris
10a76e674d Update to v5.18 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.18

The file bionic/libc/kernel/uapi/linux/android/binder.h had a bug
in the original 5.18 kernel headers. This was fixed upstream, so
add a comment to the structure but leave the incorrect field the
same as the 5.17 kernel headers.

Bug: 234125620

Test: Builds.
Test: All bionic unit tests pass on a coral device.
Test: Able to log in to wembley system.
Change-Id: Ia72cafbe71fd894c599e15aa5334d7bbcbe74778
2022-06-08 13:31:52 -07:00
Greg Kaiser
55b5639cca Revert "Update to v5.18 kernel headers."
Revert "Update to v5.18 kernel headers."

Revert submission 2106847-kernel_update

Reason for revert: b/234125620
Reverted Changes:
I5844ca1fc:Update to v5.18 kernel headers.
I5700813ee:Update to v5.18 kernel headers.
Bug: 234125620

Change-Id: I07a39aa9c01939455b5a2a9da5fc04be497ca399
2022-05-27 20:41:14 +00:00
Christopher Ferris
5d3679612f Update to v5.18 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.18

Test: Builds.
Test: All bionic unit tests pass on a coral device.
Change-Id: I5700813eec0fb0db55bee4d75b129400b7a239bd
2022-05-25 14:30:25 -07:00
Christopher Ferris
9dfa2e55ed Set VIDEO_MAX_FRAME value back to 64.
This value was changed from 64 to 32, but it really should not
have gone down. So fixing back to 64 manually, and then the
next kernel header update should remove the comment but leave
the value unchanged.

Bug: 228783882

Test: Compiles.
Change-Id: I2d81d7bc76fcaa905dce0abb094d61d02c481a5e
2022-04-21 10:42:09 -07:00
Christopher Ferris
1ed5534657 Update to v5.17 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.17

Test: Boots on a flame and all bionic unit tests pass.
Change-Id: I7057d7308241f3acfa600597d287994c39ababbc
2022-03-22 16:48:40 -07:00
Christopher Ferris
a479261feb Update to v5.16 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.16

Test: Builds and bionic unit tests pass.
Change-Id: I2522c4f2a0efb9f8a193e1f2d65868e478217502
2022-01-10 13:51:15 -08:00
Christopher Ferris
2abfa9e4f8 Update to v5.15 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.15

Test: Builds, and ran bionic unit tests on flame.
Change-Id: I86901ea6d89fb0f7384b90a47bef35d194083c24
2021-11-02 15:53:58 -07:00
Christopher Ferris
3a39c0bc43 Update to v5.14 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.14

Test: Builds, bionic unit tests.
Change-Id: Iff3424da9fbf7ae89ebeb6daabb34c4aa650901f
2021-09-02 00:03:38 +00:00
Colin Cross
d27506716a Define sockaddr_storage in bionic
Bionic currently renames the kernel's __kernel_sockaddr_storage
to sockaddr_storage.  We would like to reuse the bionic kernel
headers for musl, but musl defines its own sockaddr_storage,
causing redefinition errors when the musl and kernel headers
are both included.

Give bionic its own definition of sockaddr_storage so the kernel
one can be left as __kernel_sockaddr_storage, avoiding the
conflict in musl.

Some of the structs in netinet/in.h contain sockaddr_storage members.
There are definitions in linux/in.h, but they now use the
__kernel_sockaddr_storage naming.  To avoid having to copy the structs
from linux/in.h into netinet/in.h, add some #define hackery to
rename __kernel_sockaddr_storage to sockaddr_storage when including
linux/in.h from netinet/in.h.

Bug: 190084016
Test: m checkbuild
Test: bionic-unit-tests
Change-Id: I8f654511722b3a4a8e0bb146e4418ebcd370305c
2021-07-14 14:20:36 -07:00
Treehugger Robot
df901df77a Merge "Support replacing nested kernel structs" 2021-07-12 23:13:59 +00:00
Christopher Ferris
fcc3b4f45d Update to v5.13 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.13

Test: Builds, bionic unit tests.
Change-Id: Ia4360ca4a2808206f64187dcb234d73c5752624b
2021-07-01 01:30:21 +00:00
Colin Cross
68ec53e141 Support replacing nested kernel structs
When replacing structs with bits/ includes, count the number of
opening and closing braces to find the end of the struct.

Test: tools/update_all.py
Change-Id: Ic9712a385da517710ceffccbdb223c89ae0b5f65
2021-06-29 14:22:30 -07:00
Christopher Ferris
a9750eda0d Update to v5.12 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.12

Test: Boots flame device and runs bionic unit tests.
Test: Boots acloud device and runs bionic unit tests.

Change-Id: I8ac107ce9d4978be3ef9517b90ad6ecafd06785a
2021-05-03 15:01:39 -07:00
Christopher Ferris
e6b7969dd1 Merge "Fix last python3 issues." 2021-04-21 20:02:10 +00:00
Christopher Ferris
a2142d2835 Fix last python3 issues.
Includes pointing to the python3 version of the clang bindings.

Also, remove stale .gitignore line.

Test: Ran bionic/libc/kernel/tools/update_all.py and verified
Test: the files generated the same exact way.
Change-Id: I4eb9dd7382bca013f70d921b6ef48c7e7478615a
2021-04-20 17:36:07 -07:00
Dan Albert
8607c084c2 Fix overly-zealous strip.
The argument to this is the characters to strip, so `line.strip(line)`
just returns the empty string.

Test: None?
Bug: None
Change-Id: I4f62bffcd00936e4eef837a28b78023fcad54bb0
2021-04-20 16:05:02 -07:00
Christopher Ferris
ac7ec11473 Update for python3.
This fixes all of the problems with our kernel scripts, but not
the clang python script problems.

I also removed the updateGitFiles function since that code was
just silently failing any way. I replaced all calls with updateFiles.

Test: Ran script using python2 to verify it still works.
Test: Run script in python3 verifying that it starts to run.

Change-Id: I223a31a8324c59e6bc4067f48a6110361b3e26e8
2021-04-19 17:02:38 -07:00
Elliott Hughes
6b586e7709 Force everything to use python3 for consistency.
Rather than "whatever people have installed as 'python' on their machine".

I've removed check-symbols.py because that's been broken for years and
we never even noticed, and I'm not sure it's worth fixing.

Test: treehugger, manual
Change-Id: Ieb996bbdf790a18d4b1fb46a409cc240ba2a2a49
2021-04-15 13:39:08 -07:00
Christopher Ferris
7e2f6ffeaf Update kernel update documentation.
Parts of the document have gone out of date, so update it.

Test: NA
Change-Id: Ia4bc534ca1cc9a538d5995126c541c2cd7029c57
2021-03-18 00:38:34 -07:00
Christopher Ferris
05667cd66a Update to v5.11 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.11

Test: Built cuttlefish and flame images. Ran bionic unit tests on both.
Change-Id: Ie60337aafad4bda55af99b6c8fe9f56bf2fa787f
2021-02-16 16:01:34 -08:00
Christopher Ferris
32ff3f831c Update to v5.10 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.10

Test: Built cuttlefish and flame images. Ran bionic unit tests on both.
Change-Id: I37ffc850970adcce1febbe2269c202632fce763a
2020-12-14 22:37:59 -08:00
Christopher Ferris
ccf17f0ec7 Clean up header files from kernel directory.
Add a make distclean for each supported arch before generating headers
and at the end so these files are not left hanging around in the
kernel directory.

Bug: 174795055

Test: Verified all generated files are deleted after running.
Test: Verified this generates the exact same files as before this change.
Change-Id: I0f66f580af2ea50c190eb532032017e782d7cae6
2020-12-03 18:18:52 -08:00
Christopher Ferris
25c18d45cf Update to v5.9 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.9

Test: Boots cuttlefish 64bit, passes 32 bit and 64 bit bionic unit tests.
Change-Id: Ib5503355b238ea75595538e63eb000c867d06ef7
2020-10-19 15:48:58 -07:00
Christopher Ferris
8177cdf178 Update to v5.8 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.8

Test: NA
Change-Id: I2231c877589820fc09800a200cf4ac62ba74b04c
2020-08-04 13:50:49 -07:00
Elliott Hughes
cf346532fc More cleanup for #inclusivefixit.
Found manually with grep, since the script seems to miss stuff.

Test: treehugger
Change-Id: I5933cbade9792801d4a0bec1ccb077efa6ad8fbc
2020-07-31 10:35:03 -07:00
Christopher Ferris
430527b4cd Update the generate script.
The arm kvm.h file has been deprecated, so nobody should be including it.
Therefore, remove the hack to copy the file out of the tools directory.

Also, update to remove the arm kvm.h file.

Test: Ran generate and update script and verified only the arm kvm.h
Test: was deleted.
Test: Builds and boots on a walleye.
Change-Id: I95dcc8877bdb2da2858b0b9d0aa95d1d2072cf9a
2020-06-04 12:32:09 -07:00
Christopher Ferris
af09c709d5 Update to v5.7 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.7

Update the generate_uapi_headers.sh where the types.h file is now in
the right place, but kvm.h is not.

Test: Booted cuttlefish/bonito.
Test: Ran bionic-unit-tests on cuttlefish/bonito.
Change-Id: Ice9ce370a658e320b80f564b34a4431927fcf100
2020-06-02 10:58:28 -07:00
Christopher Ferris
bb9fcb4636 Update to v5.6 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.6

Add a new method for removing structures. This is to deal with the kernel
headers changing some definitions of timeval to __kernel_old_timeval
and itimerval to __kernel_old_itimerval. Remove the __kernel_old_XX
strutures and change the other structures to the previous definitions.

This only works so long as these structures stay the same, if they
diverge, then a different strategy will need to be implemented.

Test: Booted cuttlefish/walleye.
Test: Ran bionic-unit-tests on cuttlefish/walleye.
Change-Id: I0a61f4fa6e4155c602e0414d9b38c2e1637829af
2020-04-10 10:53:48 -07:00
Christopher Ferris
d32ca14ea3 Update to v5.5 kernel headers.
Kernel headers coming from:

Git: https://android.googlesource.com/kernel/common/
Branch: android-mainline
Tag: android-mainline-5.5

Test: Boots on walleye.
Test: Ran bionic-unit-tests on walleye.
Test: Boots on cuttlefish.
Test: Ran bionic-unit-tests on cuttlefish.
Change-Id: I57387d3c31e0ba5ad125ffe291cecf365c7b374e
Merged-In: I57387d3c31e0ba5ad125ffe291cecf365c7b374e
2020-02-07 20:14:42 +00:00
Elliott Hughes
c2faf235c0 Stop generating unused headers.
These just cause confusion because they often have different
values/layouts, but they're never actually used.

Test: treehugger
Change-Id: I424034088e017c919f62fcefa7d6d3f903f31cfb
2020-02-03 17:56:06 -08:00
Christopher Ferris
ce11673cad Merge "Fully disable clang format where needed." 2020-01-30 21:47:39 +00:00
Christopher Ferris
1cc755c8cf Fully disable clang format where needed.
Even with formatting off, clang still tries to rearrange the include
files or the using statements, so disable that too.

Test: Verified that the include directories are not rearranged.
Change-Id: I991a1b2bfa94a8202c5a486664658d654f1c7811
2020-01-30 08:10:17 -08:00
Elliott Hughes
09e77f35ab _FILE_OFFSET_BITS=64 support for fcntl.
On Android, fcntl is always implemented by fcntl64(2). This means that
an LP32 binary can `fcntl(F_SETLK, struct flock)` (because fcntl64(2)
passes through to the 32-bit fcntl(2) to handle F_SETLK), and it can
also `fcntl(F_SETLK64, struct flock64)`. What it can't do before this
patch is set _FILE_OFFSET_BITS=64 and then `fcntl(F_SETLK, struct
flock)` where that `struct flock` is actually implicitly `struct
flock64`.

Move the kernel uapi structs out of the way, define them ourselves based
on __LP64__ and _FILE_OFFSET_BITS, and fix up the relevant F_ constants.

(Also add a .clang-format to turn off clang-format in libc/include/.)

Bug: N/A
Test: treehugger (and strace!)
Change-Id: Iccd6c83d9133e1efcf93a7b49a6ae0f1bbd3d58b
2020-01-29 19:36:14 -08:00
Christopher Ferris
fdaf82f96b Link .clang-format file to system/core one.
Use the .clang-format-2 found in system/core instead of this which is
not actually being used.

Also, enable clang-format running by default.

All upstream directories are marked as ignoring formatting so that
their source files are not modified.

Test: NA
Change-Id: Icee6030f373fa5f072df162f97e6f34320e3d89a
2020-01-29 15:14:20 -08:00
Christopher Ferris
c58208bd3a Merge "Update documentation around kernel header imports." 2019-12-16 17:48:52 +00:00
Josh Gao
afe7e53003 Clone with --depth 1 in generate_uapi_headers.sh --download-kernel.
Test: manual
Change-Id: Id60b354fb1914470315976dd103d2665a6fefab8
2019-12-13 17:24:44 -08:00
Christopher Ferris
86a8f3f32d Update documentation around kernel header imports.
Test: NA
Change-Id: I8866ef30efbc565e8996c574a9f895c574cd8e55
2019-12-13 17:20:20 -08:00