Tasty is a modern testing framework for Haskell.

It lets you combine your unit tests, golden tests, QuickCheck/SmallCheck properties, and any other types of tests into a single test suite.


  • Run tests in parallel but report results in a deterministic order
  • Filter the tests to be run using patterns specified on the command line
  • Hierarchical, colored display of test results
  • Reporting of test statistics
  • Acquire and release resources (sockets, temporary files etc.) that can be shared among several tests
  • Extensibility: add your own test providers and ingredients (runners) above and beyond those provided

To find out what's new, read the change log.


Here's how your test.hs might look like:

import Test.Tasty
import Test.Tasty.SmallCheck as SC
import Test.Tasty.QuickCheck as QC
import Test.Tasty.HUnit

import Data.List
import Data.Ord

main = defaultMain tests

tests :: TestTree
tests = testGroup "Tests" [properties, unitTests]

properties :: TestTree
properties = testGroup "Properties" [scProps, qcProps]

scProps = testGroup "(checked by SmallCheck)"
  [ SC.testProperty "sort == sort . reverse" $
      \list -> sort (list :: [Int]) == sort (reverse list)
  , SC.testProperty "Fermat's little theorem" $
      \x -> ((x :: Integer)^7 - x) `mod` 7 == 0
  -- the following property does not hold
  , SC.testProperty "Fermat's last theorem" $
      \x y z n ->
        (n :: Integer) >= 3 SC.==> x^n + y^n /= (z^n :: Integer)

qcProps = testGroup "(checked by QuickCheck)"
  [ QC.testProperty "sort == sort . reverse" $
      \list -> sort (list :: [Int]) == sort (reverse list)
  , QC.testProperty "Fermat's little theorem" $
      \x -> ((x :: Integer)^7 - x) `mod` 7 == 0
  -- the following property does not hold
  , QC.testProperty "Fermat's last theorem" $
      \x y z n ->
        (n :: Integer) >= 3 QC.==> x^n + y^n /= (z^n :: Integer)

unitTests = testGroup "Unit tests"
  [ testCase "List comparison (different length)" $
      [1, 2, 3] `compare` [1,2] @?= GT

  -- the following test does not hold
  , testCase "List comparison (same length)" $
      [1, 2, 3] `compare` [1,2,2] @?= LT

And here is the output of the above program:

(Note that whether QuickCheck finds a counterexample to the third property is determined by chance.)


tasty is the core package. It contains basic definitions and APIs and a console runner.

In order to create a test suite, you also need to install one or more «providers» (see below).


The following providers exist:

It's easy to create custom providers using the API from Test.Tasty.Providers.


Ingredients represent different actions that you can perform on your test suite. One obvious ingredient that you want to include is one that runs tests and reports the progress and results.

Another standard ingredient is one that simply prints the names of all tests.

It is possible to write custom ingredients using the API from Test.Tasty.Runners.

Some ingredients that can enhance your test suite are:

  • tasty-ant-xml adds a possibility to write the test results in a machine-readable XML format, which is understood by various CI systems and IDEs
  • tasty-rerun adds support for minimal test reruns by recording previous test runs and using this information to filter the test tree. For example, you can use this ingredient to only run failed tests, or only run tests that threw an exception.
  • tasty-html adds the possibility to write the test results as a HTML file
  • tasty-stats adds the possibility to collect statistics of the test suite in a CSV file.

Other packages

  • tasty-th automatically discovers tests based on the function names and generate the boilerplate code for you
  • tasty-hunit-adapter converts existing HUnit test suites into tasty test suites
  • tasty-discover automatically discovers your tests.
  • tasty-expected-failure provides test markers for when you expect failures or wish to ignore tests.


Options allow one to customize the run-time behavior of the test suite, such as:

  • mode of operation (run tests, list tests, run tests quietly etc.)
  • which tests are run (see «Patterns» below)
  • parameters of individual providers (like depth of search for SmallCheck)

Setting options

There are two main ways to set options:


When using the standard console runner, the options can be passed on the command line or via environment variables. To see the available options, run your test suite with the --help flag. The output will look something like this (depending on which ingredients and providers the test suite uses):

% ./test --help
Mmm... tasty test suite

Usage: test [-p|--pattern PATTERN] [-t|--timeout DURATION] [-l|--list-tests]
            [-j|--num-threads NUMBER] [-q|--quiet] [--hide-successes]
            [--color never|always|auto] [--quickcheck-tests NUMBER]
            [--quickcheck-replay SEED] [--quickcheck-show-replay]
            [--quickcheck-max-size NUMBER] [--quickcheck-max-ratio NUMBER]
            [--quickcheck-verbose] [--smallcheck-depth NUMBER]

Available options:
  -h,--help                Show this help text
  -p,--pattern PATTERN     Select only tests which satisfy a pattern or awk
  -t,--timeout DURATION    Timeout for individual tests (suffixes: ms,s,m,h;
                           default: s)
  -l,--list-tests          Do not run the tests; just print their names
  -j,--num-threads NUMBER  Number of threads to use for tests execution
  -q,--quiet               Do not produce any output; indicate success only by
                           the exit code
  --hide-successes         Do not print tests that passed successfully
  --color never|always|auto
                           When to use colored output (default: 'auto')
  --quickcheck-tests NUMBER
                           Number of test cases for QuickCheck to generate
  --quickcheck-replay SEED Random seed to use for replaying a previous test run
                           (use same --quickcheck-max-size)
  --quickcheck-show-replay Show a replay token for replaying tests
  --quickcheck-max-size NUMBER
                           Size of the biggest test cases quickcheck generates
  --quickcheck-max-ratio NUMBER
                           Maximum number of discared tests per successful test
                           before giving up
  --quickcheck-verbose     Show the generated test cases
  --smallcheck-depth NUMBER
                           Depth to use for smallcheck tests

Every option can be passed via environment. To obtain the environment variable name from the option name, replace hyphens - with underscores _, capitalize all letters, and prepend TASTY_. For example, the environment equivalent of --smallcheck-depth is TASTY_SMALLCHECK_DEPTH.

Note on boolean options: by convention, boolean ("on/off") options are specified using a switch on the command line, for example --quickcheck-show-replay instead of --quickcheck-show-replay=true. However, when passed via the environment, the option value needs to be True or False (case-insensitive), e.g. TASTY_QUICKCHECK_SHOW_REPLAY=true.

If you're using a non-console runner, please refer to its documentation to find out how to configure options during the run time.


You can also specify options in the test suite itself, using localOption. It can be applied not only to the whole test tree, but also to individual tests or subgroups, so that different tests can be run with different options.

It is possible to combine run-time and compile-time options, too, by using adjustOption. For example, make the overall testing depth configurable during the run time, but increase or decrease it slightly for individual tests.

This method currently doesn't work for ingredient options, such as --quiet or --num-threads. You can set them by setting the corresponding environment variable before calling defaultMain:

import Test.Tasty
import System.Environment

main = do
  setEnv "TASTY_NUM_THREADS" "1"
  defaultMain _


It is possible to restrict the set of executed tests using the -p/--pattern option.

Tasty patterns are very powerful, but if you just want to quickly run tests containing foo somewhere in their name or in the name of an enclosing test group, you can just pass -p foo. If you need more power, or if that didn't work as expected, read on.

A pattern is an awk expression. When the expression is evaluated, the field $1 is set to the outermost test group name, $2 is set to the next test group name, and so on up to $NF, which is set to the test's own name. The field $0 is set to all other fields concatenated using / as a separator.

As an example, consider a test inside two test groups:

  testGroup "One" [ testGroup "Two" [ testCase "Three" _ ] ]

When a pattern is evaluated for the above test case, the available fields and variables are:

$0 = "One/Two/Three"
$1 = "One"
$2 = "Two"
$3 = "Three"
NF = 3

Here are some examples of awk expressions accepted as patterns:

  • $2 == "Two" — select the subgroup Two
  • $2 == "Two" && $3 == "Three" — select the test or subgroup named Three in the subgroup named Two
  • $2 == "Two" || $2 == "Twenty-two" — select two subgroups
  • $0 !~ /skip/ or ! /skip/ — select tests whose full names (including group names) do not contain the word skip
  • $NF !~ /skip/ — select tests whose own names (but not group names) do not contain the word skip
  • $(NF-1) ~ /QuickCheck/ — select tests whose immediate parent group name contains QuickCheck

As an extension to the awk expression language, if a pattern pat contains only letters, digits, and characters from the set [_/ ], it is treated like /pat/ (and therefore matched against $0). This is so that we can use -p foo as a shortcut for -p /foo/.

The only deviation from awk that you will likely notice is that Tasty does not implement regular expression matching. Instead, $1 ~ /foo/ means that the string foo occurs somewhere in $1, case-sensitively. We want to avoid a heavy dependency of regex-tdfa or similar libraries; however, if there is demand, regular expression support could be added under a cabal flag.

The following operators are supported (in the order of decreasing precedence):



Type of Result




Type of expr



Field reference





Logical not

Unary minus





expr + expr

expr - expr







expr expr

String concatenation



expr < expr

expr <= expr

expr != expr

expr == expr

expr > expr

expr >= expr

Less than

Less than or equal to

Not equal to

Equal to

Greater than

Greater than or equal to













expr ~ pat

expr !~ pat

(pat must be a literal, not an expression, e.g. /foo/)

Substring match

No substring match





expr && expr

Logical AND



expr || expr

Logical OR



expr1 ? expr2 : expr3

Conditional expression

Type of selected
expr2 or expr3


The following built-in functions are supported:

substr(s, m[, n])

Return the at most n-character substring of s that begins at position m, numbering from 1. If n is omitted, or if n specifies more characters than are left in the string, the length of the substring will be limited by the length of the string s.


Convert the string s to lower case.


Convert the string s to upper case.

match(s, pat)

Return the position, in characters, numbering from 1, in string s where the pattern pat occurs, or zero if it does not occur at all. pat must be a literal, not an expression, e.g. /foo/.


Return the length, in characters, of its argument taken as a string, or of the whole record, $0, if there is no argument.

Running tests in parallel

In order to run tests in parallel, you have to do the following:

  • Compile (or, more precisely, link) your test program with the -threaded flag;
  • Launch the program with +RTS -N -RTS.


To apply timeout to individual tests, use the --timeout (or -t) command-line option, or set the option in your test suite using the mkTimeout function.

Timeouts can be fractional, and can be optionally followed by a suffix ms (milliseconds), s (seconds), m (minutes), or h (hours). When there's no suffix, seconds are assumed.


./test --timeout=0.5m

sets a 30 seconds timeout for each individual test.

Options controlling console output

The following options control behavior of the standard console interface:

Run the tests but don't output anything. The result is indicated only by the exit code, which is 1 if at least one test has failed, and 0 if all tests have passed. Execution stops when the first failure is detected, so not all tests are necessarily run. This may be useful for various batch systems, such as commit hooks.
Report only the tests that has failed. Especially useful when the number of tests is large.
Don't run the tests; only list their names, in the format accepted by --pattern.
Whether to produce colorful output. Accepted values: never, always, auto. auto means that colors will only be enabled when output goes to a terminal and is the default value.

Custom options

It is possible to add custom options, too.

To do that,

  1. Define a datatype to represent the option, and make it an instance of IsOption
  2. Register the options with the includingOptions ingredient
  3. To query the option value, use askOption.

See the Custom options in Tasty article for some examples.

Project organization and integration with Cabal

There may be several ways to organize your project. What follows is not Tasty's requirements but my recommendations.

Tests for a library

Place your test suite sources in a dedicated subdirectory (called tests here) instead of putting them among the main library sources.

The directory structure will be as follows:


test.hs is where your main function is defined. The tests may be contained in test.hs or spread across multiple modules (Mod1.hs, Mod2.hs, ...) which are then imported by test.hs.

Add the following section to the cabal file (my-project.cabal):

test-suite test
      base >= 4 && < 5
    , tasty >= 0.7 -- insert the current version here
    , my-project   -- depend on the library we're testing
    , ...

Tests for a program

All the above applies, except you can't depend on the library if there's no library. You have two options:

  • Re-organize the project into a library and a program, so that both the program and the test suite depend on this new library. The library can be declared in the same cabal file.
  • Add your program sources directory to the Hs-source-dirs. Note that this will lead to double compilation (once for the program and once for the test suite).


  1. How do I make some tests execute after others?

    Currently, your only option is to make all tests execute sequentially by setting the number of tasty threads to 1 (example). See #48 for the discussion.

  2. When my tests write to stdout/stderr, the output is garbled. Why is that and what do I do?

    It is not recommended that you print anything to the console when using the console test reporter (which is the default one). See #103 for the discussion.

    Some ideas on how to work around this:

    • Use testCaseSteps (for tasty-hunit only).
    • Use a test reporter that does not print to the console (like tasty-ant-xml).
    • Write your output to files instead.


Blog posts and other publications related to tasty. If you wrote or just found something not mentioned here, send a pull request!


Roman Cheplyaka is the primary maintainer.

Oliver Charles is the backup maintainer. Please get in touch with him if the primary maintainer cannot be reached.




Fix a bug where a test suite that uses resources would hang if interrupted

Version 1.0.1

  • Add a safeReadBool function, for case-insensitive parsing of boolean options
  • Convert all tasty's own options to case-insensitive


Adjust lower bounds for the dependencies (mtl and optparse-applicative)

Version 1.0

  • New pattern language (see the README and/or the blog post)
  • Make the clock dependency optional


Fix compatibility with GHC 8.4

Version 0.12

Backward compat breaking revision of Test.Tasty.Ingredients.ConsoleReporter that exposes the name of tests/groups.

Version 0.11.3

Expose and document several of the internals of Test.Tasty.Ingredients.ConsoleReporter.


Fix compatibility with GHC 7.4


  1. Make the --quiet mode more efficient on a large number of tests
  2. Fix a bug where a cursor would disappear if the test suite was terminated by a signal other than SIGINT.


Make filtering tests (-p) work faster


Fix a critical bug in the quiet mode (-q/--quiet): the exit status could be wrong or the test suite could hang.


Fix compatibility with the latest unbounded-delays

Version 0.11.2

Add composeReporters, a function to run multiple reporter ingredients

Version 0.11.1

Introduce mkOptionCLParser and mkFlagCLParser


Fix compatibility with optparse-applicative-0.13


Switch from regex-tdfa-rc to regex-tdfa, which got a new maintainer.


Clarify IsTest’s specification with regard to exceptions


Use monotonic clock when measuring durations.

Version 0.11

New field resultShortDescription of Result


  • Improve the docs
  • Fix compatibility with GHC HEAD


  • Prevent parsing non-positive number of threads via program options (#104)
  • Buffer output to avoid slowdowns when printing test results (#101)
  • Default to using the maximum number of available cores for test execution

Version 0.10.1

Export Test.Tasty.Runners.formatMessage


Don't output ANSI codes for the Emacs terminal emulator


Better handle the situation when there are no ingredients to run


Split the changelog into per-project changelogs


Update to optparse-applicative 0.11

Version 0.10

  • Add the --color option
  • Timings
    • Introduce the Time type synonym
    • Change the types of launchTestTree and TestReporter to accept the total run time
    • consoleTestReporter now displays the timings


Upgrade to optparse-applicative-0.10.


Be careful not to export the Show (a -> b) instance, see https://github.com/feuerbach/tasty/issues/71


Hide cursor when running tests


Fix for GHC 7.9


Remove the old 'colors' flag description from the cabal file


Make ansi-terminal an unconditional dependency

Version 0.8

  • Test.Tasty.Ingredients is now exposed
  • Test.Tasty.Ingredients.Basic is added, which exports the ingredients defined in the tasty package. These exports should now be used instead of ones exported from Test.Tasty.Runners
  • The Result type is now structured a bit differently. Providers now should use testPassed and testFailed functions instead of constructing Results directly.
  • Add «quiet mode» (see README)
  • Add «hide successes» mode (see README)
  • Add short command-line options: -j for --num-threads, -p for --pattern
  • Add timeout support
  • AppMonoid is renamed to Traversal for consistency with the 'reducers' package. Another similar wrapper, Ap, is introduced.
  • Fix a resources bug (resources were not released if the test suite was interrupted)
  • The type of launchTestTree is changed. It now takes a continuation as an argument. This is necessary to fix the bug mentioned above.
  • Add flagCLParser to be used as the optionCLParser implementation for boolean options.
  • Add the ability to pass options via environment

Version 0.7

  • Use regex-tdfa instead of regex-posix (which is a native implementation, and as such is more portable)
  • foldTestTree now takes the algebra in the form of a record rather than multiple arguments, to minimize breakage when new nodes are added or existing ones change
  • withResource now passes the IO action to get the resource to the inner test tree

Version 0.6

  • Better handling of exceptions that arise during resource creation or disposal
  • Expose the AppMonoid wrapper
  • Add askOption and inludingOptions


Depend on ansi-terminal >= 0.6.1. This fixes some issues with colors on Windows.

Version 0.5.2

  • Export Result and Progress from Test.Tasty.Runners
  • Make it clear that only GHC 7.4+ is supported

Version 0.5.1

Export ResourceSpec from Test.Tasty.Runners

Version 0.5

Add a capability to acquire and release resources. See the «Resources» section in the Test.Tasty docs.

For the end users, the API is backwards-compatible.

Test runners may have to be adjusted — there is a new constructor of TestTree and a new argument of foldTestTree.

Version 0.4.2

Add defaultIngredients


Print the failure description in red


Fix a bug (#25)

Version 0.4

The big change in this release is introduction of ingredients, which is a replacement for runners. But unless you have a custom runner, this is unlikely to affect you much.

The Ingredient data type has replaced the Runner type.

The following functions have been renamed and possibly changed their types:

  • defaultMainWithRunnerdefaultMainWithIngredients
  • treeOptionParsersuiteOptionParser
  • getTreeOptionstreeOptions
  • runUIconsoleTestReporter

Added in this release:

  • suiteOptions
  • optionParser
  • functions operating on ingredients
  • testsNames
  • the listingTests ingredient and its option, ListTests

NumThreads is no longer a core option, but is automatically included in the test reporting ingredients (see its haddock).

Version 0.3.1

  • Proper reporting of (some) non-terminating tests (#15)
  • Upgrade to optparse-applicative 0.6

Version 0.3

  • Restrict dependency versions
  • Fix a bug where non-terminating test would lead to a deadlock (#15)

Version 0.2

  • Add an execRunner function
  • Make Runner return IO Bool

Version 0.1.1

Set lower bound on optparse-applicative dependency version

comments powered byDisqus