system-test

Runs system tests of applications. https://github.com/ExcaliburZero/system-test-haskell

Latest on Hackage:0.1.2

This package is not currently in any snapshots. If you're interested in using it, we recommend adding it to Stackage Nightly. Doing so will make builds more reliable, and allow stackage.org to host generated Haddocks.

MIT licensed by Christopher Wells
Maintained by cwellsny@nycap.rr.com

System Test Travis CI Status

System Test is a Haskell application which allows you to specify and run system tests of applications.

Recording of System Test

System tests can be defined in json files with the following format, see examples/ for more examples:

[
  {
    "name": "Hello World Test",
    "command": "echo 'Hello, World!'",
    "expectedOutput": "Hello, World!"
  },
  {
    "name": "GoodBye World Test",
    "command": "echo 'GoodBye, World!'",
    "expectedOutput": "Hello, World!"
  }
]

Note: System Test does not run the test commands in sandboxes, so be careful of running system tests that may cause harm to your system.

Usage

The System Test executable file should be run with all of the files containing system tests being passed in as arguments.

system-test test1.txt test2.txt
system-test tests/*.txt

Test File Structure

The system test files are formatted using JSON. The file should contain a list of Tests, each of which should have Strings for the test name, command, and expected output.

[
  {
    "name": "Hello World Test",
    "command": "echo 'Hello, World!'",
    "expectedOutput": "Hello, World!"
  },
  {
    "name": "GoodBye World Test",
    "command": "echo 'GoodBye, World!'",
    "expectedOutput": "Hello, World!"
  },
  {
    "name": "MultiLine Test",
    "command": "echo '1' && echo '2'",
    "expectedOutput": "1\n2"
  }
]

License

System Test is available under the MIT License, see LICENSE for more information.

Changes

Changelog

0.1.2

  • Fix handling of when no test files are given.
  • Add changelog to package.
  • Add an animated screenshot of the program to the readme file.
  • Restructure the program into a library and an executable to allow for easier testing.
  • Change the build tool from Cabal to Stack.
  • Add system and unit tests.
  • Add coverage reports through coveralls.

0.1.1

  • Add a notice to the readme that the tests are not run in a sandbox.
  • Improve the Cabal build configuration file.
  • Add a check for package validity.
  • Set Cabal to compile with checks for warnings.
  • Fix several GHC warnings.
  • Add a changelog.
  • Add information on how to make a release of the program.

0.1.0

  • Enable running of system tests.
  • Print out system test results in color, based on the results of the test.
  • Gives a failure status when any of the system tests fail.
  • Allow system test definitions in JSON files.
comments powered byDisqus