This Ingredient for tasty testing framework
allows to filter a test tree depending
on an outcome of the previous run.
This may be useful in many scenarios,
especially when a test suite grows large.
For example, tasty-rerun allows:
Rerun only tests, which failed during the last run (--rerun).
Combined with live reloading (e. g., using ghcid or stack test --file-watch),
it gives an ultimate power to focus on broken parts
and put them back in shape, enjoying a tight feedback loop.
Rerun only tests, which have beed added since the last saved test run.
This comes handy when writing a new module, which does not affect other
parts of the system, or adding new test cases.
Rerun only tests, which passed during the last saved test run.
Sometimes a part of the test suite is consistently failing
(e. g., an external service is temporarily down), but you want be sure
that you are not breaking anything else in course of your work.
To add it to your test suite just replace Test.Tasty.defaultMain
with Test.Tasty.Ingredients.Rerun.defaultMainWithRerun:
import Test.Tasty
import Test.Tasty.Ingredients.Rerun
main :: IO ()
main = defaultMainWithRerun tests
tests :: TestTree
tests = undefined
Use --help to list command-line options:
--rerun
Rerun only tests, which failed during the last run.
If the last run was successful, execute a full test
suite afresh. A shortcut for --rerun-update --rerun-filter failures,exceptions --rerun-all-on-success.
--rerun-update
Update the log file to reflect latest test outcomes.
--rerun-filter CATEGORIES
Read the log file and rerun only tests from a given
comma-separated list of categories: failures,
exceptions, new, successful. If this option is
omitted or the log file is missing, rerun everything.
--rerun-all-on-success
If according to the log file and --rerun-filter there
is nothing left to rerun, run all tests. This comes
especially handy in stack test --file-watch or
ghcid scenarios.
--rerun-log-file FILE
Location of the log file (default: .tasty-rerun-log).
Changes
1.1.18
Support tasty 1.4.
1.1.17
Add defaultMainWithRerun,
a drop-in replacement for defaultMain.
1.1.16
New command-line option --rerun-all-on-success.
New command-line shortcut --rerun.
1.1.15
Bump upper bound of base.
Restore missing -j command-line option.
1.1.14
Support tasty 1.2.
1.1.13
Bump upper bound of base.
1.1.12
Bump upper bound of tasty.
1.1.11
Bump upper bound of base.
1.1.10
Bump upper bound of tasty.
1.1.9
Bump upper bound of tasty.
1.1.8
Bump upper bound of tasty.
1.1.7
Allow base < 4.11.
1.1.6
Allow base 4.9 for building with GHC 8.0
1.1.5
Supports tasty < 0.12.
1.1.4
Supports base <= 4.9, tagged <= 0.9
1.1.3
Supports tasty =< 0.11
1.1.2
Allow base 4.7 for building with GHC 7.8
1.1.1
Update to work with tasty >= 0.8
1.1.0
The TestTree is filtered using a custom traversal now, rather than a
TreeFold. This gives better guarantees that the TestTree is only
reduced and that nodes (such as WithResources) continue to work. The
resulting filtered TestTree now has the same shape as the original
tree, but filtered tests are transformed into TestGroups with no tests.
This is a fairly major change to how the filtering is performed, so this
is a new major release, and previous versions are now considered
deprecated.
1.0.1
Now supports filtering TestTrees that use resources.
1.0.0
Initial release. Supports the --rerun-update, --rerun-log-file and
--rerun-filter options. Supported filters are new, failures,
exceptions and successful.