Gauge: a clone of criterion
This is a clone of criterion with a code / dependencies on a diet. It works the same way as criterion for outputing to terminal benchmark data.
features compared to criterion
- JSON export
- Glob benchmark matching
- Small condensed output (
- Raw measurements dumping (CSV)
Future Feature Plan
- Remove further dependencies
- storing benchmarks data in CSV and JSON
- Make the library more useful as a standalone library to gather benchmark numbers related to functions in a programatic way
It’s hard to compare many benchmarks with criterion, so gauge has a
identity mean 41.65 ns ( +- 2.246 ns ) slow mean 163.9 ns ( +- 9.683 ns )
Direct dependencies removed compared to criterion
Number of total dependencies (direct & indirect):
- gauge: 12 dependencies
- criterion: 63 dependencies
- Glob 0.8.0
- abstract-deque 0.3
- abstract-par 0.3.3
- aeson 220.127.116.11
- ansi-terminal 0.6.3.1
- ansi-wl-pprint 0.6.7.3
- array 0.5.1.1
- attoparsec 0.13.1.0
- base-compat 0.9.3
- base-orphans 0.6
- binary 0.8.3.0
- blaze-builder 0.4.0.2
- bytestring 0.10.8.1
- cassava 0.4.5.1
- cereal 0.5.4.0
- code-page 0.1.3
- containers 0.5.7.1
- directory 18.104.22.168
- dlist 0.8.0.3
- erf 22.214.171.124
- exceptions 0.8.3
- filepath 126.96.36.199
- ghc-boot-th 8.0.2
- hashable 188.8.131.52
- integer-gmp 184.108.40.206
- integer-logarithms 1.0.2
- js-flot 0.8.3
- js-jquery 3.2.1
- math-functions 0.2.1.0
- microstache 220.127.116.11
- monad-par 0.3.4.8
- monad-par-extras 0.3.3
- mtl 2.2.1
- mwc-random 0.13.6.0
- optparse-applicative 0.13.2.0
- parallel 18.104.22.168
- parsec 3.1.11
- pretty 22.214.171.124
- process 126.96.36.199
- random 1.1
- scientific 0.3.5.2
- statistics 0.14.0.2
- stm 188.8.131.52
- tagged 0.8.5
- template-haskell 184.108.40.206
- text 220.127.116.11
- time 18.104.22.168
- time-locale-compat 0.1.1.3
- transformers-compat 0.5.1.4
- unix 22.214.171.124
- unordered-containers 0.2.8.0
- uuid-types 1.0.3
- vector-algorithms 0.7.0.1
- vector-binary-instances 0.2.3.5
- vector-th-unbox 0.2.1.6
Criterion graph of dependencies:
Gauge graph of dependencies:
whnfAppIOfunctions, which take a function and its argument separately like
whnf, but whose function returns
whnfIO. This is useful for benchmarking functions in which the bulk of the work is not bound by IO, but by pure computations that might otherwise be optimized away if the argument is known statically.
Bug Fix: Pass
-m exactoption to the child processes used to run benchmarks in an isolated manner. This avoids running a wrong benchmark due to the default prefix match.
- Add a new benchmark matching option “-m exact” to match the benchmark name exactly.
- Write data to CSV file in quick mode too.
- Fix the CSV file header to match with the data rows for the
- Fix issue with GC metrics in 32 bits that would silently wrap and failure in optional machinery.
- Simplify dependencies in tests using foudation checks.
- Inline math-functions & mwc-random:
- Remove most functions, instances and types, that are unnecessary for gauge
- Remove unsafe seeding with partial seed (unused in gauge anyway)
- Remove vector-th-unbox dependency (transitively template-haskell, pretty, ghc-boot-th)
- Remove time dependency
- Re-add Gauge.Benchmark to Gauge.Main to keep the transition between criterion and gauge easy
- Fix cycles reporting on linux, osx and windows
- Add some extra callstack for reporting on partial function
- Fix compilation with Semigroup => Monoid (compilation on 8.4). still unsupported
- Add some color on terminal output
Usability: Simplify and organize the documentation and user APIs.
- Add measurement and reporting of more performance counters on Unices (collected via getrusage) for example page faults, user time, system time and rss are now available in verbose mode.
- Re-enable CSV analysis with the same output format as criterion (
- Add CSV measurement dumping with
Control: Provide better control over measurement process with
--quickflag that provides results much faster (10x) without using statistical analysis.
- Fix a bug in GC stats collection and reporting with GHC 8.2 that caused incorrect reporting of some GC stats.
- Fix a bug in statistical regression that caused incorrect reporting of mean and other stats.
- Improve reliability by isolating benchmarks from one another using the
--measure-withflag. The results of one benchmark are no longer affected by other benchmarks because each benchmark runs in a separate process.
- Introduce an optional value type
Optionalwith an efficient runtime representation to replace the ad-hoc fromXXX functions and the untyped approach.
--measure-onlyflag that allows just measurement and no analysis or reporting.
- Provide modular build, measurement code is cleanly separated from
statistical analysis code. As a result a leaner version can now be built
without analysis code (controlled by the
- Clean, refactor & rewrite source code
- Remove code-page dependency
- Simplify monad handling, remove foundation as dependency
- condensed display with
- remove optparse-applicative
- remove bunch of dependencies
- initial import of criterion-126.96.36.199