benchmark 1.9.4
|
If you see this error:
you might want to disable the CPU frequency scaling while running the benchmark, as well as consider other ways to stabilize the performance of your system while benchmarking.
Exactly how to do this depends on the Linux distribution, desktop environment, and installed programs. Specific details are a moving target, so we will not attempt to exhaustively document them here.
One simple option is to use the cpupower
program to change the performance governor to "performance". This tool is maintained along with the Linux kernel and provided by your distribution.
It must be run as root, like this:
After this you can verify that all CPUs are using the performance governor by running this command:
The benchmarks you subsequently run will have less variance.
If you see this error:
you might want to disable the ASLR security hardening feature while running the benchmark.
The simplest way is to add
as the first line of your main()
function. It will try to disable ASLR for the current processor, and, if successful, re-execute the binary. Note that personality(2)
may be forbidden by e.g. seccomp (which happens by default if you are running in a Docker container).
Note that if you link to benchmark_main
already does that for you.
To globally disable ASLR on Linux, run
To run a single benchmark with ASLR disabled on Linux, do:
Note that for the information on how to disable ASLR on other operating systems, please refer to their documentation.
The Linux CPU frequency governor discussed above is not the only source of noise in benchmarks. Some, but not all, of the sources of variance include:
These can cause variance in benchmarks results within a single run (--benchmark_repetitions=N
) or across multiple runs of the benchmark program.
Reducing sources of variance is OS and architecture dependent, which is one reason some companies maintain machines dedicated to performance testing.
Some of the easier and effective ways of reducing variance on a typical Linux workstation are:
/sys
file system (see the LLVM project's Benchmarking tips).Further resources on this topic: