generalize counter-examples of test properties
|Version on this page:||0.3.3|
|LTS Haskell 20.23:||0.4.6|
|Stackage Nightly 2023-05-29:||0.4.6|
|Latest on Hackage:||0.4.6|
Extrapolate is a property-based testing library for Haskell capable of reporting generalized counter-examples.
To install the latest version of Extrapolate from Hackage using cabal, just:
$ cabal update $ cabal install extrapolate
To test if it installed correctly, follow through the next section.
To use Extrapolate, you first import the
then pass any properties you with to test to the function
$ ghci > import Test.Extrapolate > check $ \x y -> x + y == y + (x :: Int) +++ OK, passed 360 tests. > import Data.List (nub) > check $ \xs -> nub xs == (xs :: [Int]) *** Failed! Falsifiable (after 3 tests): [0,0] Generalization: x:x:_
+ is commutative. The function
nub is not an identity.
Configuring the number of tests
To increase the number of tests, use the
$ ghci > import Test.Extrapolate > check `for` 1000 $ \x y -> x + y == y + (x :: Int) +++ OK, passed 1000 tests.
Customizing the background functions (allowed in side-conditions)
To customize the background functions, use the
$ ghci > import Test.Extrapolate > import Data.List (nub) > let hasDups xs = nub xs /= (xs :: [Int]) > check `withBackground` [constant "hasDups" hasDups] $ \xs -> nub xs == (xs :: [Int]) *** Failed! Falsifiable (after 3 tests): [0,0] Generalization: x:x:_ Conditional Generalization: xs when hasDups xs
Perhaps the example above is silly (
hasDups is the negation of the property
itself!), but it illustrates the use of
withBackground can be used in conjunction:
> check `for` 100 `withBackground` [...] $ property
Don’t forget the dollar sign
Consider the following (faulty) sort function and a property about it:
sort :: Ord a => [a] -> [a] sort  =  sort (x:xs) = sort (filter (< x) xs) ++ [x] ++ sort (filter (> x) xs) prop_sortCount :: Ord a => a -> [a] -> Bool prop_sortCount x xs = count x (sort xs) == count x xs where count x = length . filter (== x)
After testing the property, Extrapolate returns a fully defined counter-example along with a generalization:
> import Test.Extrapolate > check (prop_sortCount :: Int -> [Int] -> Bool) *** Failed! Falsifiable (after 4 tests): 0 [0,0] Generalization: x (x:x:_)
This hopefully makes it easier to find the source of the bug. In this case, the faulty sort function discards repeated elements.
For more examples, see the eg folder. For type signatures, other options and uses, see Extrapolate’s API documentation.
There are two other tools for Haskell capable of producing generalized counter-examples: SmartCheck and Lazy SmallCheck 2012.
Extrapolate was accepted for presentation at IFL 2017, see the pre-symposium proceedings paper about Extrapolate. Extrapolate is also subject to a chapter in a PhD Thesis (2017).