platform_bionic/benchmarks/README.md

Bionic Benchmarks

Bionic benchmarks is a command line tool for measuring the runtimes of libc functions. It is built on top of Google benchmarks with some additions to organize tests into suites.

Running the benchmarks

Device benchmarks

$ mma
$ adb remount
$ adb sync
$ adb shell /data/nativetest/bionic-benchmarks/bionic-benchmarks
$ adb shell /data/nativetest64/bionic-benchmarks/bionic-benchmarks

When operated without specifying an xml file, the default is to use the xml file called full.xml found in the directory suites/ bound in the same directory as the bionic-benchmarks executable.

To use a different xml file, use the --bionic_xml=FILE.XML option. By default, this option searches for the xml file in the suites/ directory. If it doesn't exist in that directory then the file will be found as relative to the current directory. If the option specifies the full path to an xml file such as /data/nativetest/suites/example.xml, it will be used as is.

You can use --benchmark_filter=getpid to just run benchmarks with "getpid" in their name.

Host benchmarks

See the benchmarks/run-on-host.sh script. The default for host tests is to use the host.xml file in the suites directory instead of full.xml. The host benchmarks can be run with 32- or 64-bit bionic, or the host glibc.

Suites

Suites are stored in the suites/ directory and can be chosen with the command line flag '--bionic_xml'. When operated without specifying an xml file, the default is to use the file called full.xml found in the directory suites/ bound in the same directory as the bionic-benchmarks executable.

To use a different xml file, use the --bionic_xml=FILE.XML option. By default, this option searches for the xml file in the suites/ directory. If it doesn't exist in that directory then the file will be found as relative to the current directory. If the option specifies the full path to an xml file such as /data/nativetest/suites/example.xml, it will be used as is.

If no xml file is specified through the command-line option, the default is to use suites/full.xml. However, for the host bionic benchmarks (bionic-benchmarks-glibc), the default is to use suites/host.xml.

Format

The format for a benchmark is:

<fn>
    <name>BM_sample_benchmark</name>
    <cpu><optional_cpu_to_lock></cpu>
    <iterations><optional_iterations_to_run></iterations>
    <args><space separated list of function args|shorthand></args>
</fn>

xml-specified values for iterations and cpu take precedence over those specified via command line (via '--bionic_iterations' and '--bionic_cpu', respectively.)

To make small changes in runs, you can also schedule benchmarks by passing in their name and a space-separated list of arguments via the 'bionic_extra' command line flag, e.g. '--bionic_extra="BM_string_memcpy AT_COMMON_SIZES"' or '--bionic_extra="BM_string_memcmp 32 8 8"'

Note that benchmarks will run normally if extra arguments are passed in, and it will fail with a segfault if too few are passed in.

Shorthand

For the sake of brevity, multiple runs can be scheduled in one xml element by putting one of the following in the args field:

NUM_PROPS
MATH_COMMON
AT_ALIGNED_<ONE|TWO>BUF
AT_<any power of two between 2 and 16384>_ALIGNED_<ONE|TWO>BUF
AT_COMMON_SIZES

Definitions for these can be found in bionic_benchmarks.cpp, and example usages can be found in the suites directory.

Unit Tests

Bionic benchmarks also has its own set of unit tests, which can be run from the binary in /data/nativetest[64]/bionic-benchmarks-tests