It should make it easier to reproduce bugs found by OSS-Fuzz locally
without docker. The fuzz target can be built and run with the corpus
OSS-Fuzz has accumulated so far by running the following commands:
```
./scripts/oss-fuzz.sh
wget https://storage.googleapis.com/selinux-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/selinux_secilc-fuzzer/public.zip
unzip -d CORPUS public.zip
./out/secilc-fuzzer CORPUS/
```
It was tested in https://github.com/google/oss-fuzz/pull/6026
by pointing OSS-Fuzz to the branch containing the patch and
running all the tests with all the sanitizers and fuzzing engines
there: https://github.com/google/oss-fuzz/actions/runs/1024673143
[v2]
[1] oss-fuzz: make shellcheck happy
[2] oss-fuzz: build libsepol only
The fuzz target covers libsepol so it's unnecessary to build everything
else. Apart from that, the "LDFLAGS" kludge was removed since libsepol
is compatible with the sanitizers flags passed via CFLAGS only. It
should be brought back one way or another eventually though to fix
build failures like
```
clang -L/home/vagrant/selinux/selinux/DESTDIR/usr/lib -L/home/vagrant/selinux/selinux/DESTDIR/usr/lib -L../src sefcontext_compile.o ../src/regex.o -lselinux -lpcre ../src/libselinux.a -lsepol -o sefcontext_compile
/usr/bin/ld: sefcontext_compile.o: in function `usage':
/home/vagrant/selinux/selinux/libselinux/utils/sefcontext_compile.c:271: undefined reference to `__asan_report_load8'
/usr/bin/ld: /home/vagrant/selinux/selinux/libselinux/utils/sefcontext_compile.c:292: undefined reference to `__asan_handle_no_return'
/usr/bin/ld: sefcontext_compile.o: in function `asan.module_ctor':
```
[3] oss-fuzz: make it possible to run the script more than once
by removing various build artifacts
[4] oss-fuzz: make it possible to run the script from any directory
[5] oss-fuzz: be a little bit more specific about what the script does
[6] oss-fuzz: stop overwriting all the Makefiles
Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
Acked-by: Nicolas Iooss <nicolas.iooss@m4x.org>
- use multiple jobs
- define _FORTIFY_SOURCE=2 to enable checks on standard string handling
functions due to macro/intrinsic overloads or function attributes
- allow to override clang and scan-build binaries, i.e. for using
versioned ones
- set PYTHON_SETUP_ARGS accordingly on Debian
- enable common warning -Wextra
- print build result
Signed-off-by: Christian Göttsche <cgzones@googlemail.com>
Now that F34 has been released, it's time to update the CI Vagrantfile
to use the new Fedora version. This also fixes the failure in the
recently added vsock_socket test that depends on a bugfix, which made it
to the F34 image's kernel, but is not in the F33 image's.
Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
Acked-by: Petr Lautrbach <plautrba@redhat.com>
Following Petr Lautrbach's suggestion, release a snapshot of the source
repository next to the individual archives which constitute a release.
While at it, make scripts/release more robust:
- Fix many warnings reported by shellcheck, by quoting strings.
- Use bash arrays for DIRS and DIRS_NEED_PREFIX
- Merge DIRS and DIRS_NEED_PREFIX into a single array, in order to
produce SHA256 digests that are directly in alphabetical order, for
https://github.com/SELinuxProject/selinux/wiki/Releases
- Use "set -e" in order to fail as soon as a command fails
- Change to the top-level directory at the start of the script, in order
to be able to run it from anywhere.
- Use `cat $DIR/VERSION` and `git -C $DIR` instead of `cd $i ; cat VERSION`
in order to prevent unexpected issues from directory change.
Finally, if version tags already exists, re-use them. This enables using
this script to re-generate the release archive (and check that they
really match the git repository). Currently, running scripts/release
will produce the same archives as the ones published in the 3.2-rc1
release (with the same SHA256 digests as the ones on the release page,
https://github.com/SELinuxProject/selinux/wiki/Releases). This helps to
ensure that the behaviour of the script is still fine.
Suggested-by: Petr Lautrbach <plautrba@redhat.com>
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
Using Vagrant makes reproducing and debugging CI issues easier: after
"vagrant up", a test virtual machine is up and running, and ready to run
"fedora-test-runner.sh". In order to make using this VM even easier, a
helper script, "run-selinux-test.sh" is created inside and instructions
on how to use it are documented at the beginning of Vagrantfile.
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
Since the format of git archive is inferred from the output file,
it's safe to remove the pipe for gzip.
Signed-off-by: Hu Keping <hukeping@huawei.com>
Now that the standard testing is run on GitHub Actions, we can remove it
from .travis.yml, leaving only the selinux-testsuite sanity check, which
can't be migrated. Hopefully this will save some precious minutesfrom
the limited "plan" Travis gave us...
Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
The testsuite now runs filesystem tests also on other filesystems than
just the default ext4. [1] That means a few more userspace utilities are
needed to format these filesystems. [2]
[1] 071ec9c5e5
[2] d4e507f78a
Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
The testsuite will soon be switching to testing multiple filesystems,
which exposes a bug in F32 image's kernel. Since Fedora 33 has been
released recently and the testsuite runs just fine on it, just bump the
image version to avoid the bug.
This commit also fixes the script to read out the Fedora image version
from environment variables instead of using hard-coded values.
Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
git log -1 may use a pager to output long messages, and when the pager
is invoked, leads to stalls on the ci system waiting for user input.
Use --oneline to print the short part of the commit message and
the digest. This information is for debug/informational purposes only,
so truncating the output is sufficient.
Reported-by: Stephen Smalley <sds@tycho.nsa.gov>
Signed-off-by: William Roberts <william.c.roberts@intel.com>
License the ci scripts with a permissive, OSI approved license, such as
MIT.
Signed-off-by: William Roberts <william.c.roberts@intel.com>
[omosnace: removed the dummy copyright header from LICENSE]
Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
Not everyone's github project is "selinux" so use the projects
name, as derived from TRAVIS_BUILD_DIR. TRAVIS_BUILD_DIR is
the absolute path to the project checkout on disk, so the
basename should be sufficient. The script that runs in the KVM
environment also needs to be updated where it can find the
selinux project code, so we pass it in through an env variable
in the ssh command.
Tested on Travis CI here:
- https://travis-ci.org/github/williamcroberts/selinux/jobs/697307824
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Acked-by: Petr Lautrbach <plautrba@redhat.com>
The current Travis CI runs the userspace tooling and libraries against
policy files, but cannot test against an SELinux enabled kernel. Thus,
some tests are not being done in the CI. Travis, unfortunately only
provides Ubuntu images, so in order to run against a modern distro with
SELinux in enforcing mode, we need to launch a KVM with something like
Fedora.
This patch enables this support by launching a Fedora32 Cloud Image with
the SELinux userspace library passed on from the Travis clone, it then
builds and replaces the current SELinux bits on the Fedora32 image and
runs the SELinux testsuite.
The cloud image run can be controlled with the TRAVIS env variable:
TRAVIS_CLOUD_IMAGE_VERSION. That variable takes the major and minor
version numbers in a colon delimited string, eg: "32:1.6".
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Acked-by: Stephen Smalley <stephen.smalley.work@gmail.com>
When a branch has '.py' suffix git creates a file with the same suffix and this
file is found by the `find . -name '*.py'` command. Such files from './git' need
to be filtered out.
Fixes:
$ PATH="$VIRTUAL_ENV/bin:$PATH" ./scripts/run-flake8
Analyzing 189 Python scripts
./.git/logs/refs/heads/semanage-test.py:1:42: E999 SyntaxError: invalid syntax
./.git/refs/heads/semanage-test.py:1:4: E999 SyntaxError: invalid syntax
The command "PATH="$VIRTUAL_ENV/bin:$PATH" ./scripts/run-flake8" exited with 1.
Signed-off-by: Petr Lautrbach <plautrba@redhat.com>
Acked-by: Nicolas Iooss <nicolas.iooss@m4x.org>
On systems using non-default `PREFIX`, `LIBDIR`, `SHLIBDIR`, `BINDIR`
or `SBINDIR`, running
`DESTDIR=/path/to/destdir ./scripts/env_use_destdir make test`
does not perform the intended behavior, because the testing programs and
libraries are installed into locations that are not added to
`LD_LIBRARY_PATH` nor `PATH`.
More precisely, with `LIBDIR=/usr/lib64 SHLIBDIR=/lib64`, `env_use_destdir`
does not work. Fix this by adding the installation directories relative
to `DESTDIR` in `LD_LIBRARY_PATH` and `PATH`.
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
Acked-by: Petr Lautrbach <plautrba@redhat.com>
libselinux and libsemanage use:
PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig
import *; print(get_python_lib(plat_specific=1,
prefix='$(PREFIX)'))")
while python/semanage and python/sepolgen/src/sepolgen use:
PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig
import *; print(get_python_lib(prefix='$(PREFIX)'))")
This is right: libselinux and libsemanage's Python bindings use native
code (thus "plat_specific=1") while the others only install Python
files.
Nevertheless `scripts/env_use_destdir` only runs the second command
when computing `$PYTHONPATH`. When using this script to run `make test`
in a minimal Fedora 31 environment, this leads to an error such as:
make[2]: Entering directory '/code/python/sepolicy'
Traceback (most recent call last):
File "test_sepolicy.py", line 117, in <module>
import selinux
ModuleNotFoundError: No module named 'selinux'
Fix this by also adding `get_python_lib(plat_specific=1)` to the
computed `$PYTHONPATH`.
While at it, preserve `$PYTHONPATH` instead of resetting it. This makes
it easier to work with Python virtual environments.
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
Several static analyzers (clang's one, Facebook Infer, etc.) warn about
NULL pointer dereferences after a call to CU_ASSERT_PTR_NOT_NULL_FATAL()
in the test code written using CUnit framework. This is because this
CUnit macro is too complex for them to understand that the pointer
cannot be NULL: it is translated to a call to CU_assertImplementation()
with an argument as TRUE in order to mean that the call is fatal if the
asserted condition failed (cf.
http://cunit.sourceforge.net/doxdocs/group__Framework.html).
A possible solution could consist in replacing the
CU_ASSERT_..._FATAL() calls by assert() ones, as most static analyzers
know about assert(). Nevertheless this seems to go against CUnit's API.
An alternative solution consists in overriding CU_ASSERT_..._FATAL()
macros in order to expand to assert() after a call to the matching
CU_ASSERT_...() non-fatal macro. This appears to work fine and to remove
many false-positive warnings from various static analyzers.
As this substitution should only occur when using static analyzer, put
it under #ifdef __CHECKER__, which is the macro used by sparse when
analyzing the Linux kernel.
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
- Python 2.7 is planned to be the last of the 2.x releases
- It's generally advised to use Python 3
- Majority of python/ scripts are already switched python3
- Users with python 2 only can still use:
$ make PYTHON=/usr/bin/python ....
Signed-off-by: Petr Lautrbach <plautrba@redhat.com>
- new release files are created in release/$RELEASE_TAG
- download links refers to new release assets
Signed-off-by: Petr Lautrbach <plautrba@redhat.com>
Set-up environment variables in order to use DESTDIR in Python, Ruby,
etc. This makes testing Python scripts easier.
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
When running flake8 on a directory, it does not analyze files without an
extension, like semanage_migrate_store, mlscolor-test, etc. Use grep to
find files with a Python shebang and build a list which is then given to
flake8.
This commit is possible now that some clean-up patches have been
applied, such as commit 69c56bd2f6 ("python/chcat: improve the code
readability") and b7227aaec1 ("mcstrans: fix Python linter warnings on
test scripts") and 3cb974d2d2 ("semanage_migrate_store: fix many
Python linter warnings").
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
PLY (Python Lex-Yacc) 3.11 has been released in February 2018:
- http://www.dabeaz.com/ply/index.html
- https://github.com/dabeaz/ply/releases/tag/3.11
Copy lex.py and yacc.py from this new release.
This fixes the following warning from "make test":
python run-tests.py
../src/./sepolgen/lex.py:634: DeprecationWarning: Using or importing
the ABCs from 'collections' instead of from 'collections.abc' is
deprecated, and in 3.8 it will stop working
if isinstance(t, collections.Callable):
(Python 3.3 moved collections.Callable to collections.abc.Callable)
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
Python does not need to end a statement with a semicolon. Doing this
gets reported by linters such as flake8 ("E703 statement ends with a
semicolon").
Remove such semicolons in the code and enable this warning in
scripts/run-flake8.
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
flake8 is a Python linter which is able to detect issues in Python code
(syntax errors, undefined variables, etc.). It has been used to find
bugs in the project. In order to prevent the introduction of new bugs
which can be detected by it, add a script which runs it and use it in
Travis-CI.
flake8 can be used to detect code which is not written according to PEP8
style guide (which forbids whitespaces in some places, enforces the use
of space-indenting, specifies how many blank lines are used between
functions, etc.). As SELinux code does not follow this style guide,
scripts/run-flake8 disables many warnings related to this when running
the linter.
In order to silence flake8 warnings, the Python code can also be
modified. However fixing every "do not use bare 'except'" in the project
needs to be done carefully and takes much time.
This is why the warnings which are disabled have been ordered in three
lists:
* The warnings which can be activated in a not-so-distant future after
the code has been modified.
* The warnings related to PEP8 which cannot be activated without a major
cleaning work of the codebase (for example to modify white spaces)
* The warnings which are introduced by code generated by SWIG 3.0.12,
which would require patches in SWIG in order to be activated (there
is right now only one such warning).
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
Using clang's static analyzer is as simple as running "scan-build make",
but in order to obtain clean and reproducible results, the build
environment has to be cleaned beforehand ("make clean distclean").
Moreover the project requires running "make install" before "make test"
in order to install the dependencies needed for the tests, and running
these tests with the newly-built libraries requires a specific
LD_LIBRARY_PATH. This new script takes care of setting up everything
which is needed.
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
make-update script is for generating an update for an existing release,
e.g. for a bug fix update.
release script is for generating a new release.
Per-package tagging may fail on the release script if the version
has not changed; this is harmless.
Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
Tag the release before generating it, and automatically generate
a template for use in updating the wiki page. Also, make the
output directory unique to each release, delete any prior contents,
and tell the user where to find it.
Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
