Roman Cheplyaka

Custom options in Tasty

Published on

Tasty 0.6 is released, making it possible to create custom options just for your test suite!

Add your own option in three easy steps:

  1. Define a datatype to represent the option, and make it an instance of IsOption
  2. Register the options with the includingOptions ingredient
  3. To query the option value, use askOption.

Examples follow.

Ignoring a test

My use case is a test suite that has a number of tests that fail on a certain build bot. I can’t fix the build bot configuration ATM, so I’d like to be able to mark these tests as known-fail in the build script for this particular build bot. — 23Skidoo

To some extent this is just a way around Tasty’s limited pattern language (which will improve, too!), but I still find it pretty nice.

With the following code, you can disable the second test by passing a --buildbot command-line option.

{-# LANGUAGE DeriveDataTypeable #-}

import Test.Tasty
import Test.Tasty.Options
import Test.Tasty.HUnit
import Data.Typeable (Typeable)
import Data.Tagged
import Data.Proxy
import Options.Applicative

newtype BuildBot = BuildBot Bool
  deriving (Eq, Ord, Typeable)

instance IsOption BuildBot where
  defaultValue = BuildBot False
  parseValue = fmap BuildBot . safeRead
  optionName = return "buildbot"
  optionHelp = return "Running under a build bot"
  optionCLParser =
    fmap BuildBot $
    switch
      (  long (untag (optionName :: Tagged BuildBot String))
      <> help (untag (optionHelp :: Tagged BuildBot String))
      )

main = defaultMainWithIngredients ings $
  askOption $ \(BuildBot bb) ->
  testGroup "Tests" $
  [ testCase "Successful test" $ return () ] ++
  if bb
    then []
    else [ testCase "Failing test" $ assertFailure "build bot" ]
  where
    ings =
      includingOptions [Option (Proxy :: Proxy BuildBot)] :
      defaultIngredients

Controlling the depth

When running the tests is there any general solution to set the Depth parameter for the (individual) tests, or better yet, a fine grained solution to set the Depth parameter for individual fields? — jules

Not that I recommend doing this — see this answer.

But here’s how you can do it if you’re sure you want it.

(This was also possible to hack with earlier versions of Tasty — see this gist).

{-# LANGUAGE DeriveDataTypeable #-}

import Test.Tasty
import Test.Tasty.Options
import Test.Tasty.SmallCheck
import Test.SmallCheck.Series
import Control.Applicative
import Data.Proxy
import Data.Typeable

data T1 = T1 { p1 :: Int,
               p2 :: Char,
               p3 :: Int
             } deriving (Eq, Show)

newtype P1Depth = P1Depth { getP1Depth :: Int }
  deriving Typeable

instance IsOption P1Depth where
  defaultValue = P1Depth 5
  parseValue = fmap P1Depth . safeRead
  optionName = return "smallcheck-depth-p1"
  optionHelp = return "Depth to use for p1"

t1Series
  :: Monad m
  => Int -- depth of p1
  -> Series m T1
t1Series d = decDepth $
  T1 <$> localDepth (const d) series <~> series <~> series

main :: IO ()
main = defaultMainWithIngredients (optsIng : defaultIngredients) $
  askOption $ \(P1Depth p1d) ->
    testProperty "Test1" $
      over (t1Series p1d) $
        \x -> x == x
  where
    optsIng = includingOptions [Option (Proxy :: Proxy P1Depth)]

To increase the depth of p1 to 20, pass --smallcheck-depth-p1 20 on the command line.