StateT vs. IORef: a benchmark

Sometimes I’m writing an IO loop in Haskell, and I need some sort of a counter or accumulator. The two main options are to use a mutable reference (IORef) or to put a StateT transformer on top the IO monad.

I was curious, though, if there was a difference in efficiency between these two approaches. Intuitively, IORefs are dedicated heap objects, while a StateT transformer’s state becomes “just” a local variable, so StateT might optimize better. But how much of a difference does it make?

So I benchmarked the four functions, all of which calculate the sum of numbers between 1 and n = 1000.

base_sum simply calls sum from the base package; state_sum and stateT_sum maintain the accumulator using the State Int and StateT Int IO monads, respectively, and ioref_sum uses an IORef within the IO monad. And here are the results, as reported by criterion.

I’m not sure how stateT_sum manages to be faster than state_sum and base_sum (this doesn’t appear to be a statistical fluke), but what’s clear is that ioref_sum is significantly slower of them all.

So if 3ns per state access matter to you, go for StateT even when you are in IO.

(Update: also check out the comments on reddit, especially the ones by u/VincentPepper.)

Here’s the full benchmark code. It was compiled with -O2 by GHC 8.8.4 and run on AMD Ryzen 7 3700X.

import Criterion
import Criterion.Main

import Control.Monad.State
import Data.IORef

base_sum :: Int -> Int
base_sum n = sum [1 .. n]

state_sum :: Int -> Int
state_sum n = flip execState 0 $
  forM_ [1..n] $ \i ->
    modify' (+i)

stateT_sum :: Int -> IO Int
stateT_sum n = flip execStateT 0 $
  forM_ [1..n] $ \i ->
    modify' (+i)

ioref_sum :: Int -> IO Int
ioref_sum n = do
  ref <- newIORef 0
  forM_ [1..n] $ \i ->
    modifyIORef' ref (+i)
  readIORef ref

main = do
  let n = 1000
  defaultMain
    [ bench "base_sum" $ whnf base_sum n
    , bench "state_sum" $ whnf state_sum n
    , bench "stateT_sum" $ whnfAppIO stateT_sum n
    , bench "ioref_sum" $ whnfAppIO ioref_sum n
    ]