tasty-silver

A fancy test runner, including support for golden tests.

https://github.com/phile314/tasty-silver

Version on this page:3.2.3
LTS Haskell 20.23:3.3.1.1
Stackage Nightly 2023-06-04:3.3.1.1
Latest on Hackage:3.3.1.1

See all snapshots tasty-silver appears in

MIT licensed by Philipp Hausmann, Roman Cheplyaka and others
Maintained by Philipp Hausmann, Andreas Abel
This version can be pinned in stack with:tasty-silver-3.2.3@sha256:1f373c27f65ffbc1e9db9a07023d83c84e493e66858f7ee610495539ad57f23d,2846

Module documentation for 3.2.3

Hackage version tasty-silver on Stackage Nightly Stackage LTS version Haskell-CI macOS windows Hackage CI

Tasty Silver

This package provides support for «golden testing».

A golden test is an IO action that writes its result to a file. To pass the test, this output file should be identical to the corresponding «golden» file, which contains the correct result for the test.

Interactive Mode

If the test runner is called with the -i option, the diff of any failing golden test is shown to the user. Based upon this diff, the user can choose to update the golden standard or to fix the test case as necessary. Interactive mode requires that at least git diff and less is available, or preferrably wdiff and colordiff for character-based diffs.

Examples

For a non-trivial example see the Agda tests, which is used for testing the Agda compiler.

Maintainers

Philipp Hausmann is the primary maintainer. Andreas Abel is co-maintainer.

Changes

Changes

Version 3.2.3 (13 Sep 2021)

  • Tested with GHC 7.4 - 9.0 (fixed compilation with GHC 7.4 - 7.8)
  • CI via GitHub Actions on platforms ubuntu, macOS, windows.

Version 3.2.2

  • Fix cabal warning (#27, thanks to felixonmars)

Version 3.2.1

  • Fix option parser (#25)

Version 3.2

  • Compatibility with tasty 1.4 (breaks compatibility with older versions of tasty)

Version 3.1.15

  • Fix missing space in git diff calls introduced in v3.1.14 (#22, thanks to croyzor)

Version 3.1.14

  • Fix wrong interpretation of git diff exit codes (#21, thanks to croyzor)

Version 3.1.13

  • Add option to disable ansi tricks (#18, thanks to L-TChen)

Version 3.1.12

  • Fix compilation with GHC 8.4 (thanks to asr)

Version 3.1.11

  • Fix compilation with GHC 8.4

Version 3.1.10

  • Better error handling for calls to external tools (git diff)

Version 3.1.9

  • Fix compilation with optparse-applicative 0.13.*.
  • Provide character-level diff if wdiff and colordiff are available.

Version 3.1.8.1

  • Fix compilation with GHC 8.

Version 3.1.8

  • Make update function optional for test cases.

Version 3.1.7

  • Add feature to disable certain tests, still showing them in the UI but not running them.
  • Fix a concurrency issue in the interactive test runner.

Version 3.1.6

  • Expose regex filter modules.
  • Fix issue with regex filters when used together with withResource nodes.

Version 3.1.5

  • Add experimental –regex-include option to select tests using a regex. This option is highly experimental and may change in later versions!
  • The –regex-include/–regex-exclude option may be given multiple times now. The exclusion regexes are applied first, after that all inclusion regexes.

Version 3.1.4

  • Add experimental –regex-exclude option to filter out tests using a regex. This option is highly experimental and may change in later versions!

Version 3.1.3

  • Use package temporary instead of temporary-rc.
  • Re-add command line options for test runner which were accidentally removed.

Version 3.1.2

  • Add non-interactive mode to test runner, printing diffs/actual values directly to stdout. Useful for (travis) CI.

Version 3.1.1

  • Report success instead of failure if new result is accepted in interactive mode.

Version 3.1

  • Fixed & tested support for GHC 7.4.2 - 7.10.1
  • Added missing lower bound for bytestring
  • Removed upper bounds for most dependencies
  • Enable travis CI builds

Version 3.0 - 3.0.2.2

  • Refactored API
  • Add interactive mode

Version 2.2.2.4

  • Warn when some tests threw exceptions during --accept
  • Properly handle exceptions; don’t swallow Ctrl-C

Version 2.2.2.3

Restore compatibility with older compilers

Version 2.2.2.1

Relax Cabal dependency

Version 2.2.2

Add findByExtension

Version 2.2.1.2

Catch exceptions when accepting golden tests

Version 2.2.1.1

Switch to temporary-rc

Version 2.2.1

  • Fix a bug where the result of the comparison function would reference yet unread data from a semiclosed file and the file gets closed, leading to a runtime exception
  • Export writeBinaryFile
  • Improve the docs
  • Update to work with tasty-0.8

Version 2.2.0.2

Update to work with tasty-0.7

Version 2.2.0.1

Update to work with tasty-0.5

Version 2.2

Migrate to ingredients

Version 2.1

Add goldenVsStringDiff

Version 2.0.1

Update to work with tasty-0.2

Version 2.0

Initial release of tasty-golden (derived from test-framework-golden-1.1.x)

A service provided by FP Complete