All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Benchmarker Class Reference

#include <benchmark.h>

Public Types

enum  Unit : int {
  Unit::autounit, Unit::ns, Unit::us, Unit::ms,

Public Member Functions

 Benchmarker ()
template<typename FUNC , typename... ARGS>
double operator() (string_view name, FUNC func, ARGS &&...args)
double avg () const
double stddev () const
double range () const
double median () const
Benchmarkeriterations (size_t val)
size_t iterations () const
Benchmarkertrials (size_t val)
size_t trials () const
Benchmarkerwork (size_t val)
size_t work () const
Benchmarkerexclude_outliers (int e)
int exclude_outliers () const
Benchmarkerverbose (int v)
int verbose () const
Benchmarkerindent (int spaces)
int indent () const
Benchmarkerunits (Unit s)
Unit units () const
const std::stringname () const


OIIO_API std::ostream & operator<< (std::ostream &out, const Benchmarker &bench)

Detailed Description

Benchmarker is a class to assist with "micro-benchmarking". The goal is to discern how long it takes to run a snippet of code (function, lambda, etc). The code will be run in some number of trials, each consisting of many iterations, yielding statistics about the run time of the code.

Tne number of trials is user-selectable, with a reasonable default of 10 trials. The number of iterations per trial may be set explicitly, but the default is to automatically compute a reasonable number of iterations based on their timing. For most use cases, it's fire and forget.

Generally, the most and least expesive trials will be discarded (all sorts of things can happen to give you a few spurious results) and then the remainder of trials will be used to compute the average, standard deviation, range, and median value, in ns per iteration as well as millions of executions per second. The default behavior it just to echo the relevant statistics to the console.

The basic use illustrated by this example in which we try to assess the difference in speed between acos() and fast_acos():

Benchmarker bench;
float val = 0.5f;
clobber (val);  // Scrub compiler's knowledge of the value
bench ("acos", [&](){ DoNotOptimize(std::acos(val)); });
bench ("fast_acos", [&](){  // alternate indentation style

Which produces output like this: acos : 4.3 ns, 230.5 M/s (10x2097152, sdev=0.4ns rng=31.2%, med=4.6) fast_acos : 3.4 ns, 291.2 M/s (10x2097152, sdev=0.4ns rng=33.0%, med=3.4)

Some important details:

After declaring the Benchmarker, a number of options can be set: number of trials to run, iterations per trial (0 means automatic detection), verbosity, whether (or how many) outliers to exclude. You can chain them together if you want: bench.iterations(10000).trials(10);

It can be VERY hard to get valid benchmarks without the compiler messing up your results. Some tips:

  • Code that is too fast will not be reliable. Anything that appears to take less than 1 ns actually prints "unreliable" instead of full stats, figuring that it is likely that it has been inadvertently optimized away.
  • Use the DoNotOptimize() call on any final results computed by your benchmarked code, or else the compiler is likely to remove the code that leads to any values it thinks will never be used.
  • Beware of the compiler constant folding operations in your code – do not pass constants unless you want to benchmark its performance on known constants, and it is probably smart to ensure that all variables acccessed by your code should be passed to clobber() before running the benchmark, to confuse the compiler into not assuming its value.

Definition at line 142 of file benchmark.h.

Member Enumeration Documentation

enum Benchmarker::Unit : int

Definition at line 232 of file benchmark.h.

Constructor & Destructor Documentation

Benchmarker::Benchmarker ( )

Definition at line 144 of file benchmark.h.

Member Function Documentation

double Benchmarker::avg ( ) const

Definition at line 162 of file benchmark.h.

Benchmarker& Benchmarker::exclude_outliers ( int  e)

Definition at line 205 of file benchmark.h.

int Benchmarker::exclude_outliers ( ) const

Definition at line 210 of file benchmark.h.

Benchmarker& Benchmarker::indent ( int  spaces)

Definition at line 224 of file benchmark.h.

int Benchmarker::indent ( ) const

Definition at line 229 of file benchmark.h.

Benchmarker& Benchmarker::iterations ( size_t  val)

Definition at line 170 of file benchmark.h.

size_t Benchmarker::iterations ( ) const

Definition at line 175 of file benchmark.h.

double Benchmarker::median ( ) const

Definition at line 165 of file benchmark.h.

const std::string& Benchmarker::name ( ) const

Definition at line 245 of file benchmark.h.

template<typename FUNC , typename... ARGS>
double Benchmarker::operator() ( string_view  name,
FUNC  func,
ARGS &&...  args 

Definition at line 151 of file benchmark.h.

double Benchmarker::range ( ) const

Definition at line 164 of file benchmark.h.

double Benchmarker::stddev ( ) const

Definition at line 163 of file benchmark.h.

Benchmarker& Benchmarker::trials ( size_t  val)

Definition at line 178 of file benchmark.h.

size_t Benchmarker::trials ( ) const

Definition at line 183 of file benchmark.h.

Benchmarker& Benchmarker::units ( Unit  s)

Definition at line 238 of file benchmark.h.

Unit Benchmarker::units ( ) const

Definition at line 243 of file benchmark.h.

Benchmarker& Benchmarker::verbose ( int  v)

Definition at line 215 of file benchmark.h.

int Benchmarker::verbose ( ) const

Definition at line 220 of file benchmark.h.

Benchmarker& Benchmarker::work ( size_t  val)

Definition at line 193 of file benchmark.h.

size_t Benchmarker::work ( ) const

Definition at line 198 of file benchmark.h.

Friends And Related Function Documentation

OIIO_API std::ostream& operator<< ( std::ostream &  out,
const Benchmarker bench 

The documentation for this class was generated from the following file: