A fancy test runner, including support for golden tests.
|LTS Haskell 20.16:||220.127.116.11|
|Stackage Nightly 2023-03-27:||18.104.22.168|
|Latest on Hackage:||22.214.171.124|
Module documentation for 126.96.36.199
This package provides support for «golden testing».
A golden test is an IO action that writes its result to a file. To pass the test, this output file should be identical to the corresponding «golden» file, which contains the correct result for the test.
If the test runner is called with the
-i option, the diff of any failing golden test is shown
to the user. Based upon this diff, the user can choose to update the golden standard or to
fix the test case as necessary. Interactive mode requires that at least
git diff and
available, or preferrably
colordiff for character-based diffs.
tasty-silver aims to work under Linux, macOS, and Windows. In
particular, it should work in the GitHub CI virtual
macOS, GHC ≥ 7.10 is required, as GHC ≤ 7.8 produces code that is not compatible with the System Integrity Protection mechanism of Mac OS X. In particular, you could see errors like:
/usr/bin/less: getPermissions:fileAccess: permission denied (Operation not permitted)
On Windows, the colored diff may not be available as it depends on the availability of
For a non-trivial example see the Agda tests, which is used for testing the Agda compiler.
Philipp Hausmann is the primary maintainer. Andreas Abel is co-maintainer.
Version 188.8.131.52 (8 May 2022)
- Fix compilation with
- Tested with GHC 7.4 - 9.2.2
Version 3.3.1 (13 Oct 2021)
- No longer treat
git diffwarnings as failure (#33)
Version 3.3 (20 Sep 2021)
- Windows portability (#16):
- Calls to
git diffare no longer indirected via
- When indirection via
sh -cis used, backslashes in filenames are converted to slashes.
- Calls to
- Tested with GHC 7.4 - 9.2.1-RC1
Version 3.2.3 (13 Sep 2021)
- Tested with GHC 7.4 - 9.0 (fixed compilation with GHC 7.4 - 7.8)
- CI via GitHub Actions on platforms
Version 3.2.2 (22 Jun 2021)
- Fix cabal warning (#27, thanks to felixonmars)
Version 3.2.1 (22 Dec 2020)
- Fix option parser (#25)
Version 3.2 (21 Dec 2020)
- Compatibility with tasty 1.4 (breaks compatibility with older versions of tasty)
Version 3.1.15 (8 Jun 2020)
- Fix missing space in git diff calls introduced in v3.1.14 (#22, thanks to croyzor)
Version 3.1.14 (8 Jun 2020)
- Fix wrong interpretation of git diff exit codes (#21, thanks to croyzor)
Version 3.1.13 (12 Jul 2019)
- Add option to disable ansi tricks (#18, thanks to L-TChen)
Version 3.1.12 (24 Sep 2018)
- Fix compilation with GHC 8.4 (thanks to asr)
Version 3.1.11 (29 Dec 2017)
- Fix compilation with GHC 8.4
Version 3.1.10 (1 Apr 2017)
- Better error handling for calls to external tools (
Version 3.1.9 (29 Aug 2016)
- Fix compilation with optparse-applicative 0.13.*.
- Provide character-level diff if wdiff and colordiff are available.
Version 184.108.40.206 (19 Jan 2016)
- Fix compilation with GHC 8.
Version 3.1.8 (10 Nov 2015)
- Make update function optional for test cases.
Version 3.1.7 (14 May 2015)
- Add feature to disable certain tests, still showing them in the UI but not running them.
- Fix a concurrency issue in the interactive test runner.
Version 3.1.6 (14 May 2015)
- Expose regex filter modules.
- Fix issue with regex filters when used together with withResource nodes.
Version 3.1.5 (12 Apr 2015)
- Add experimental –regex-include option to select tests using a regex. This option is highly experimental and may change in later versions!
- The –regex-include/–regex-exclude option may be given multiple times now. The exclusion regexes are applied first, after that all inclusion regexes.
Version 3.1.4 (12 Apr 2015)
- Add experimental –regex-exclude option to filter out tests using a regex. This option is highly experimental and may change in later versions!
Version 3.1.3 (6 Apr 2015)
- Use package temporary instead of temporary-rc.
- Re-add command line options for test runner which were accidentally removed.
Version 3.1.2 (6 Apr 2015)
- Add non-interactive mode to test runner, printing diffs/actual values directly to stdout. Useful for (travis) CI.
Version 3.1.1 (4 Apr 2015)
- Report success instead of failure if new result is accepted in interactive mode.
Version 3.1 (4 Apr 2015)
- Fixed & tested support for GHC 7.4.2 - 7.10.1
- Added missing lower bound for bytestring
- Removed upper bounds for most dependencies
- Enable travis CI builds
Version 3.0 - 220.127.116.11
- Refactored API
- Add interactive mode
- Warn when some tests threw exceptions during
- Properly handle exceptions; don’t swallow Ctrl-C
Restore compatibility with older compilers
Catch exceptions when accepting golden tests
- Fix a bug where the result of the comparison function would reference yet unread data from a semiclosed file and the file gets closed, leading to a runtime exception
- Improve the docs
- Update to work with
Update to work with
Update to work with
Migrate to ingredients
Update to work with
Initial release of
tasty-golden (derived from