hpc-coveralls converts and sends Haskell projects hpc code coverage to coverall.io.
hpc-coveralls is still under development and any contributions are welcome!
Below is the simplest example of configuration for your project
language: haskell ghc: 7.8 script: - cabal configure --enable-tests --enable-library-coverage && cabal build && cabal test after_script: - cabal install hpc-coveralls - hpc-coveralls [options] [test-suite-names]
When building with Cabal 1.22 or a newer version, use the
--enable-coverage flag instead of
If the build fails during the test phase with an error message starting by "hpc:", just replace the
cabal test command by
run-cabal-test, as in the following example:
before_install: - cabal install hpc-coveralls script: - cabal configure --enable-tests --enable-library-coverage && cabal build - run-cabal-test [options] [cabal-test-options] after_script: - hpc-coveralls [options] [test-suite-names]
This will prevent the build to fail because of hpc related reasons, which are usually not fatal and should not affect the coverage data. Details are available in the next section.
You may also experience some issues related to your project dependencies, which can be solved by using the
Another way to solve problems related dependencies is to install hpc-coveralls in a sandbox, as in the example below:
after_script: - cabal sandbox init && cabal install hpc-coveralls - .cabal-sandbox/bin/hpc-coveralls [options] [test-suite-names]
For a real world example usage, please refer to this-project
.travis.yml file (result on coveralls).
Other real world examples can be found on this wiki page which contains a list of GitHub repositories using hpc-coveralls.
The run-cabal-test command
Under certain conditions related to the project structure and the version of hpc,
cabal test may output an error message and exit with the error code
1, which would result in a build failure.
To prevent this from happening, hpc-coveralls provides the
run-cabal-test command which runs
cabal test and returns with
0 if the following regular expression never matches any line of the output:
/^Test suite .*: FAIL$/
Below are some of the conditions under which you will likely need to use
- when using GHC 7.6 (hpc 0.6 known issue)
- when using GHC 7.8 with multiple test suites covering the same module(s) (issue #18) (fixed in GHC 7.10)
--cabal-name option can be used to specify a custom executable name instead of the default
cabal when calling
Below is an example which can be useful for projects with a Travis configuration based on multi-ghc-travis:
The hpc-coveralls command
This command parses the hpc generated output, converts its to Coveralls json format and finally sends it to coveralls.io over http.
Multiple test suites can be specified, in which case the coverage report will be made of the merged coverage data generated by the specified test suites.
For example, if your test suite are named
test2, use the command as follows:
hpc-coveralls test1 test2
--exclude-dir option allows to exclude source files located under a given directory from the coverage report.
You can exclude source files located under the
test/ directory by using this option as in the following example:
hpc-coveralls --exclude-dir=test [test-suite-names]
You can specify multiple excluded folders by using the following example syntax:
hpc-coveralls --exclude-dir=test1 --exclude-dir=test2 [test-suite-names]
As Coveralls doesn't support partial-line coverage yet, hpc-coveralls currently converts hpc coverage data into line based coverage data, which is the only format supported at the moment.
--coverage-mode option allows to configure how the coverage data is converted into Coveralls format, based on your needs.
Below are the two modes currently available, with an explanation of what the hit count values mean.
0 : the line is never hit,
1 : the line is partially covered,
2 : the line is fully covered.
AllowPartialLines conversion mode follows the same convention as the one used by cloverage coveralls output for Clojure projects code coverage.
0 : the line is never hit or only partially covered,
1 : the line is fully covered.
Please also note that there is an open issue on coveralls issue tracker in order to improve this (add support for partial line coverage).
This option allows to specify your repo token when sending the report to coveralls.io.
This boolean option prints the raw json coverage report to be sent to coveralls.io.
This boolean option prevents hpc-coveralls from sending the coverage report to coveralls.io.
This option can be used together with
--display-report for testing purpose.
For example, you can try various combinations of the other options and confirm the difference in the resulting report outputs.
This boolean option enables curl verbose mode and prints the raw json response received after posting the coverage report to coveralls.io.
Use this option to specify the cabal file of the coverage report target package. This might be required in some cases, especially when building with cabal >= 1.22 and ghc >= 7.10, although hpc-coveralls assumes the package cabal file to be the unique file of extension ".cabal" in the current directory if it exists. For further details check this issue.
This option allows you to override the
service_name value from the report sent to coveralls.io.
You will have to specify it for example when using Travis-pro as in the example below as there is currently no way to programmatically determine:
Because of the way hpc works, coverage data is only generated for modules that are referenced directly or indirectly by the test suites. As a result, the total package coverage computed by coveralls may be higher than what it really is. An option will be added soon in order to allow specifying source folders to include in the total coverage computation.
hpc-coveralls is still under development and any contributions are welcome!
Please share your comments and suggestions on hpc-coveralls Gitter channel!
- HPC publication: http://ittc.ku.edu/~andygill/papers/Hpc07.pdf
Allow directory 1.3 (issue #66)
Allow aeson 1.1 (issue #65)
- List Pathshpccoveralls in other-modules (issue #59)
- Upload sdist output to transfer.sh as part of Travis build in order to simplify releases
Allow aeson 1.0 (issue #60)
Fix dependencies to allow aeson 0.11 (issue #58)
Add upper bounds to fix issue related to retry-0.7 (issue #55)
Fix output of --numeric-version command line option (issue #53)
Relax transformers dependency lower bound (issue #50)
- Add support for cabal 1.22 / ghc 7.10 (issue #44)
- Add --service-name option to support Travis-pro (issue #49)
- Fix instructions and target hpc data directory for Cabal 1.22 / GHC 7.10 (issue #38)
- Print hpc coverage data directory tree when failing to read tix file (issue #39)
- Improve handling of coveralls response reading/parsing (issue #41)
- Send "source digest" instead of "source" (issue #43)
- Add --curl-verbose flag to enable curl verbose mode and replace --print-response (issue #42)
Send git repository info when using other CI services than Travis (issue #37)
Add option to send repo token (issue #36)
Include additional test modules in package generated by sdist (issue #34)
- Introduce retry policy to http requests (issue #31)
- Fix coverage result reading (issue #32)
Fix coverage conversion rule for otherwise (issue #20)
- Safer implementation of the result coverage value retrieval from coveralls.io (issue #25)
- Set the delay before retrieving the result to 10 seconds (issue #26)
- Add flag to print the raw json responses from coveralls.io (issue #24)
- Retrieve and display total coverage result on success response from coveralls.io (issue #21)
otherwiseas fully covered (issue #3)
- Add option to configure the coverage data conversion (issue #15)
- Add option to prevent from sending the coverage report (issue #17)
- Support setting multiple test suites (issue #14)
- Process --exclude-dir value as a string prefix instead of a regex (issue #16)
- Prevent double compilation (issue #11)
- Concurrently process cabal test stdout and stderr channels (issue #12)
- Test with GHC 7.8.2
- Additional CI services support (issue #1)
- Fixed an issue in which mix files could not be found, @maoe contribution (issue #5)
- Introduced a command line argument to exclude files located under a given folder from the coverage report (issue #6)
- Return with non-zero exit code when the tix file is not found (issue #7)
- Introduced a command line argument to specify a custom cabal executable name (issue #8)
- Parse and display response from coveralls.io (issue #9)
- Initial release