cabal-doctest

A Setup.hs helper for running doctests

https://github.com/ulidtko/cabal-doctest

Version on this page:1.0.10
LTS Haskell 22.43:1.0.11
Stackage Nightly 2024-12-03:1.0.11
Latest on Hackage:1.0.11

See all snapshots cabal-doctest appears in

BSD-3-Clause licensed by Oleg Grenrus
Maintained by Max Ulidtko
This version can be pinned in stack with:cabal-doctest-1.0.10@sha256:ebe93d13ffa657e8fa512ecc462ccafd59c497456be4355496914a11db03cb1f,2047

Module documentation for 1.0.10

cabal-doctest

Hackage Haskell-CI

A Setup.hs helper for running doctests.

Simple example

Follow simple example for the common case of a single-library .cabal package with doctests.

To recap the example’s code:

  1. specify build-type: Custom in your .cabal file;

  2. declare dependencies of Setup.hs:

    custom-setup
     setup-depends:
       base >= 4 && <5,
       cabal-doctest >= 1 && <1.1
    

    See Notes below for a caveat with cabal-install < 2.4.

  3. Populate Setup.hs like so:

    module Main where
    
    import Distribution.Extra.Doctest (defaultMainWithDoctests)
    
    main :: IO ()
    main = defaultMainWithDoctests "doctests"
    

    Assuming your test-suite is called doctests, this Setup will generate a Build_doctests module during package build. If your test-suite goes by name foo, defaultMainWithDoctests "foo" creates a Build_foo module.

  4. Use the generated module in a testsuite, simply like so:

    module Main where
    
    import Build_doctests (flags, pkgs, module_sources)
    import Data.Foldable (traverse_)
    import System.Environment (unsetEnv)
    import Test.DocTest (doctest)
    
    main :: IO ()
    main = do
        traverse_ putStrLn args -- optionally print arguments
        unsetEnv "GHC_ENVIRONMENT" -- see 'Notes'; you may not need this
        doctest args
      where
        args = flags ++ pkgs ++ module_sources
    

Ultimately, cabal test or stack test should run the doctests of your package.

Example with multiple cabal components

cabal-doctest also supports more exotic use cases where a .cabal file contains more components with doctests than just the main library, including:

  • doctests in executables,
  • doctests in internal libraries (if using Cabal-2.0 or later).

Unlike the simple example shown above, these examples involve named components. You don’t need to change the Setup.hs script to support this use case. However, in this scenario Build_doctests will generate extra copies of the flags, pkgs, and module_sources values for each additional named component.

The simplest approach is to use x-doctest-components field in .cabal:

x-doctest-components: lib lib:internal exe:example

In that case, the test driver is generally:

module Main where

import Build_doctests (Component (..), components)
import Data.Foldable (for_)
import System.Environment (unsetEnv)
import Test.DocTest (doctest)

main :: IO ()
main = for_ components $ \(Component name flags pkgs sources) -> do
    print name
    putStrLn "----------------------------------------"
    let args = flags ++ pkgs ++ sources
    for_ args putStrLn
    unsetEnv "GHC_ENVIRONMENT"
    doctest args

There is also a more explicit approach: if you have an executable named foo, then Build_doctest will contain flags_exe_foo, pkgs_exe_foo, and module_sources_exe_foo. If the name has hyphens in it (e.g., my-exe), cabal-doctest will convert them to underscores (e.g., you’d get flags_my_exe, pkgs_my_exe, module_sources_my_exe). Internal library bar values will have a _lib_bar suffix.

An example testsuite driver for this use case might look like this:

module Main where

import Build_doctests
       (flags,            pkgs,            module_sources,
        flags_exe_my_exe, pkgs_exe_my_exe, module_sources_exe_my_exe)
import Data.Foldable (traverse_)
import System.Environment (unsetEnv)
import Test.DocTest

main :: IO ()
main = do
    unsetEnv "GHC_ENVRIONMENT"
    -- doctests for library
    traverse_ putStrLn libArgs
    doctest libArgs

    -- doctests for executable
    traverse_ putStrLn exeArgs
    doctest exeArgs
  where
    libArgs = flags            ++ pkgs            ++ module_sources
    exeArgs = flags_exe_my_exe ++ pkgs_exe_my_exe ++ module_sources_exe_my_exe

See the multiple-components-example.

Additional configuration

The cabal-doctest based Setup.hs supports a few extensions fields in pkg.cabal files to customise the doctest runner behaviour, without customising the default doctest.hs.

test-suite doctests:
  if impl(ghc >= 8.0)
    x-doctest-options: -fdiagnostics-color=never
  x-doctest-source-dirs: test
  x-doctest-modules: Servant.Utils.LinksSpec
  • x-doctest-options Additional arguments passed into doctest command.
  • x-doctest-modules Additional modules to doctest. May be useful if you have doctest in test or executables (i.e not default library component).
  • x-doctest-src-dirs Additional source directories to look for the modules.

Notes

  • If support for cabal-install < 2.4 is required, you’ll have to add Cabal to setup-depends; see issue haskell/cabal#4288.

  • Some versions of Cabal (for instance, 2.0) can choose to build a package’s doctest test suite before the library. However, in order for cabal-doctest to work correctly, the library must be built first, as doctest relies on the presence of generated files that are only created when the library is built. See #19.

    A hacky workaround for this problem is to depend on the library itself in a doctests test suite. See simple-example.cabal for a demonstration. (This assumes that the test suite has the ability to read build artifacts from the library, a separate build component. In practice, this assumption holds, which is why this library works at all.)

  • custom-setup section is supported starting from cabal-install-1.24. For older cabal-install's you have to install custom setup dependencies manually.

  • stack respects custom-setup starting from version 1.3.3. Before that you have to use explicit-setup-deps setting in your stack.yaml; stack#2094.

  • With base < 4.7 (GHC < 7.8, pre-2014), System.Environment.unsetEnv function will need to be imported from base-compat library. It is already in transitive dependencies of doctest. Simply declare the dependency upon base-compat, and then import System.Environment.Compat (unsetEnv) if you need that old GHC.

  • You can use x-doctest-options field in test-suite doctests to pass additional flags to the doctest.

  • For build-type: Configure packages, you can use defaultMainAutoconfWithDoctests function to make custom Setup.hs script.

  • If you use the default . in hs-source-dirs, then running doctests might fail with weird errors (ambiguous module errors). Workaround is to move sources under src/ or some non-top-level directory.

  • The extensions: field isn’t supported. Upgrade your .cabal file to use at least cabal-version: >= 1.10 and use default-extensions or other-extensions.

  • If you use QuickCheck properties (prop>) in your doctests, the test-suite doctest should depend on QuickCheck and template-haskell. This is a little HACK: These dependencies aren’t needed to build the doctests test-suite executable. However, as we let Cabal resolve dependencies, we can pass the resolved (and installed!) package identifiers to to the doctest command. This way, QuickCheck and template-haskell are available to doctest, otherwise you’ll get errors like:

      Variable not in scope:
        mkName
          :: [Char]
             -> template-haskell-2.11.1.0:Language.Haskell.TH.Syntax.Name
    

    or

      Variable not in scope:
        polyQuickCheck
          :: Language.Haskell.TH.Syntax.Name -> Language.Haskell.TH.Lib.ExpQ
    
  • From version 2, Stack sets the GHC_ENVIRONMENT variable, and GHC (as invoked by doctest) will pick that up. This is undesirable: cabal-doctest passes all the necessary information on the command line already, and can lead to ambiguous module errors as GHC will load the environment in addition to what cabal-doctest instructs it to.

    Hence, cabal-doctest tells GHC to ignore package environments altogether on the command line. However, this is only possible since GHC 8.2. If you are using cabal-doctest with Stack 2 and GHC 8.0 or earlier and seeing ambiguous module errors or other mysterious failures, try manually unsetting GHC_ENVIRONMENT before invoking doctest.

  • If you are on Nix. doctest will not pick up your version of GHC if you don’t point it towards it, and therefore will result in “cannot satisfy -package-id” errors. You will need to set NIX_GHC and NIX_GHC_LIBDIR within your environment in order for doctest to pick up your GHC. Put the following in shell.nix and run nix-shell.

    # shell.nix
    { pkgs ? import <nixpkgs> {} }:
    let
      myHaskell = (pkgs.haskellPackages.ghcWithHoogle (p: with p; [
        # Put your dependencies here
        containers
        hslogger
      ]));
    in
    pkgs.mkShell {
      name = "myPackage";
    
      # These environment variables are important. Without these,
      # doctest doesn't pick up nix's version of ghc, and will fail
      # claiming it can't find your dependencies
      shellHook = ''
        export NIX_GHC=${myHaskell}/bin/ghc
        export NIX_GHC_LIBDIR=${myHaskell}/lib/ghc-8.10.7
      '';
      buildInputs = with pkgs; [
        myHaskell
      ];
    }
    

Copyright

Copyright 2017 Oleg Grenrus.

With contributions from:

  • Ryan Scott
  • Andreas Abel
  • Max Ulidtko

Available under the BSD 3-clause license.

Changes

1.0.10 – 2024-06-26

  • Maintainership hand-over. See cabal-doctest#79.
  • Support GHC 9.4, 9.6, 9.8, 9.10.
  • Drop support & CI for GHC < 8.0.

1.0.9 – 2021-11-07

  • Support GHC-9.2, base-4.16, and Cabal-3.6 (thanks Alistair Burrowes).

1.0.8 – 2019-10-02

  • Pass -package-env=- when compiler supports it.
  • Amend examples to unsetEnv "GHC_ENVIRONMENT".

1.0.7 – 2019-08-26

  • Make Distribution.Extra.Doctest -Wall-clean.
  • Support GHC-8.8, base-4.13, and Cabal-3.0.

1.0.6 – 2018-01-28

  • Hook haddock build too. Fixes issue when haddock fails, as Build_doctests isn’t generated.

1.0.5 – 2018-01-26

  • Add a hack so Build_doctests module is automatically added to to other-modules and autogen-modules when compiled with Cabal-2.0. Thanks to that, we don’t get warnings because of -Wmissing-home-modules.

1.0.4 – 2017-12-05

  • Add support for doctests in executables and (with Cabal-2.0 or later) internal libraries. Refer to the README for more details.

1.0.3 – 2017-11-02

  • Add an explicit Prelude import to Build_doctests.

1.0.2 – 2017-05-16

  • Add defaultMainAutoconfWithDoctests and addDoctestsUserHook.
  • Add support for .hsc and other preprocessed files (#8).
  • Add support for x-doctest-source-dirs and x-doctest-modules.

1.0.1 – 2017-05-05

  • Add support for x-doctest-options cabal-file field.
  • Proper support for GHC-8.2.1 and Cabal-2.0.0.0.
  • Add support to default-extensions in library.

1 – 2017-01-31

  • First version. Released on an unsuspecting world.