polysemy

Build Status Hackage Hackage Zulip chat

Overview

polysemy is a library for writing high-power, low-boilerplate domain specific languages. It allows you to separate your business logic from your implementation details. And in doing so, polysemy lets you turn your implementation code into reusable library code.

It’s like mtl but composes better, requires less boilerplate, and avoids the O(n^2) instances problem.

It’s like freer-simple but more powerful.

It’s like fused-effects but with an order of magnitude less boilerplate.

Additionally, unlike mtl, polysemy has no functional dependencies, so you can use multiple copies of the same effect. This alleviates the need for ~~ugly hacks~~ band-aids like classy lenses, the ReaderT pattern and nicely solves the trouble with typed errors.

Concerned about type inference? polysemy comes with its companion polysemy-plugin, which helps it perform just as well as mtl‘s! Add polysemy-plugin to your package.yaml or .cabal file’s dependencies section to use. Then turn it on with a pragma in your source files:

{-# OPTIONS_GHC -fplugin=Polysemy.Plugin #-}

Or by adding -fplugin=Polysemy.Plugin to your package.yaml/.cabal file ghc-options section.

Features

  • Effects are higher-order, meaning it’s trivial to write bracket and local as first-class effects.
  • Effects are low-boilerplate, meaning you can create new effects in a single-digit number of lines. New interpreters are nothing but functions and pattern matching.

Tutorials and Resources

Examples

Make sure you read the Necessary Language Extensions before trying these yourself!

Teletype effect:

{-# LANGUAGE TemplateHaskell, LambdaCase, BlockArguments, GADTs
           , FlexibleContexts, TypeOperators, DataKinds, PolyKinds, ScopedTypeVariables #-}

import Polysemy
import Polysemy.Input
import Polysemy.Output

data Teletype m a where
  ReadTTY  :: Teletype m String
  WriteTTY :: String -> Teletype m ()

makeSem ''Teletype

teletypeToIO :: Member (Embed IO) r => Sem (Teletype ': r) a -> Sem r a
teletypeToIO = interpret \case
  ReadTTY      -> embed getLine
  WriteTTY msg -> embed $ putStrLn msg

runTeletypePure :: [String] -> Sem (Teletype ': r) a -> Sem r ([String], a)
runTeletypePure i
  -- For each WriteTTY in our program, consume an output by appending it to the
  -- list in a ([String], a)
  = runOutputMonoid pure
  -- Treat each element of our list of strings as a line of input
  . runInputList i
  -- Reinterpret our effect in terms of Input and Output
  . reinterpret2 \case
      ReadTTY -> maybe "" id <$> input
      WriteTTY msg -> output msg


echo :: Member Teletype r => Sem r ()
echo = do
  i <- readTTY
  case i of
    "" -> pure ()
    _  -> writeTTY i >> echo


-- Let's pretend
echoPure :: [String] -> Sem '[] ([String], ())
echoPure = flip runTeletypePure echo

pureOutput :: [String] -> [String]
pureOutput = fst . run . echoPure

-- echo forever
main :: IO ()
main = runM . teletypeToIO $ echo

Resource effect:

{-# LANGUAGE TemplateHaskell, LambdaCase, BlockArguments, GADTs
           , FlexibleContexts, TypeOperators, DataKinds, PolyKinds
           , TypeApplications #-}

import Polysemy
import Polysemy.Input
import Polysemy.Output
import Polysemy.Error
import Polysemy.Resource

-- Using Teletype effect from above

data CustomException = ThisException | ThatException deriving Show

program :: Members '[Resource, Teletype, Error CustomException] r => Sem r ()
program = catch @CustomException work \e -> writeTTY $ "Caught " ++ show e
 where
  work = bracket (readTTY) (const $ writeTTY "exiting bracket") \input -> do
    writeTTY "entering bracket"
    case input of
      "explode"     -> throw ThisException
      "weird stuff" -> writeTTY input *> throw ThatException
      _             -> writeTTY input *> writeTTY "no exceptions"

main :: IO (Either CustomException ())
main
  = runFinal
  . embedToFinal @IO
  . resourceToIOFinal
  . errorToIOFinal @CustomException
  . teletypeToIO
  $ program

Easy.

Friendly Error Messages

Free monad libraries aren’t well known for their ease-of-use. But following in the shoes of freer-simple, polysemy takes a serious stance on providing helpful error messages.

For example, the library exposes both the interpret and interpretH combinators. If you use the wrong one, the library’s got your back:

runResource
    :: forall r a
     . Sem (Resource ': r) a
    -> Sem r a
runResource = interpret $ \case
  ...

makes the helpful suggestion:

• 'Resource' is higher-order, but 'interpret' can help only
  with first-order effects.
  Fix:
    use 'interpretH' instead.
• In the expression:
    interpret
      $ \case

Necessary Language Extensions

You’re going to want to stick all of this into your package.yaml file.

  ghc-options: -O2 -flate-specialise -fspecialise-aggressively
  default-extensions:
    - DataKinds
    - FlexibleContexts
    - GADTs
    - LambdaCase
    - PolyKinds
    - RankNTypes
    - ScopedTypeVariables
    - TypeApplications
    - TypeOperators
    - TypeFamilies

Building with Nix

The project provides a basic nix config for building in development. It is defined as a flake with backwards compatibility stubs in default.nix and shell.nix.

To build the main library or plugin:

nix-build -A polysemy
nix-build -A polysemy-plugin

Flake version:

nix build
nix build '.#polysemy-plugin'

To inspect a dependency:

nix repl

> p = import ./.
> p.unagi-chan

To run a shell command with all dependencies in the environment:

nix-shell --pure
nix-shell --pure --run 'cabal v2-haddock polysemy'
nix-shell --pure --run ghcid

Flake version:

nix develop -i # just enter a shell
nix develop -i -c cabal v2-haddock polysemy
nix develop -i -c haskell-language-server-wrapper # start HLS for your IDE

What about performance? (TL;DR)

Previous versions of this README mentioned the library being zero-cost, as in having no visible effect on performance. While this was the original motivation and main factor in implementation of this library, it turned out that optimizations we depend on, while showing amazing results in small benchmarks, don’t work in bigger, multi-module programs, what greatly limits their usefulness.

What’s more interesting though is that this isn’t a polysemy-specific problem - basically all popular effects libraries ended up being bitten by variation of this problem in one way or another, resulting in visible drop in performance compared to equivalent code without use of effect systems.

Why did nobody notice this?

One factor may be that while GHC’s optimizer is very, very good in general in optimizing all sorts of abstraction, it’s relatively complex and hard to predict - authors of libraries may have not deemed location of code relevant, even though it had big effect at the end. The other is that maybe it doesn’t matter as much as we like to tell ourselves. Many of these effects libraries are used in production and they’re doing just fine, because maximum performance usually matters in small, controlled areas of code, that often don’t use features of effect systems at all.

What can we do about this?

Luckily, the same person that uncovered this problems proposed a solution - set of primops that will allow interpretation of effects at runtime, with minimal overhead. It’s not zero-cost as we hoped for with polysemy at first, but it should have negligible effect on performance in real life and compared to current solutions, it should be much more predictable and even resolve some problems with behaviour of specific effects. You can try out experimental library that uses proposed features here.

When it comes to polysemy, once GHC proposal lands, we will consider the option of switching to an implementation based on it. This will probably require some breaking changes, but should resolve performance issues and maybe even make implementation of higher-order effects easier.

If you’re interested in more details, see Alexis King’s talk about the problem, Sandy Maguire’s followup about how it relates to polysemy and GHC proposal that adds features needed for new type of implementation.

TL;DR

Basically all current effects libraries (including polysemy and even mtl) got performance wrong - but, there’s ongoing work on extending GHC with features that will allow for creation of effects implementation with stable and strong performance. It’s what polysemy may choose at some point, but it will probably require few breaking changes.

Acknowledgements, citations, and related work

The following is a non-exhaustive list of people and works that have had a significant impact, directly or indirectly, on polysemy’s design and implementation:

Changes

Changelog for polysemy

Unreleased

1.9.1.0 (2023-04-09)

Other Changes

  • Support GHC 9.6

Breaking Changes

Other Changes

1.9.0.0 (2022-12-28)

Breaking Changes

  • Slightly modified the signatures of the various Scoped interpreters.

Other Changes

  • Added runScopedNew, a simple but powerful Scoped interpreter. runScopedNew can be considered a sneak-peek of the future of Scoped, which will eventually receive a major API rework to make it both simpler and more expressive.
  • Fixed a bug in various Scoped interpreters where a scoped usage of an effect always relied on the nearest enclosing use of scoped from the same Scoped effect, rather than the scoped which handles the effect.
  • Added Polysemy.Opaque, a module for the Opaque effect newtype, meant as a tool to wrap polymorphic effect variables so they don’t jam up resolution of Member constraints.
  • Expose the type alias Scoped_ for a scoped effect without callsite parameters.

1.8.0.0 (2022-12-22)

Breaking Changes

  • Removed Polysemy.View
  • Removed Polysemy.Law
  • Removed (@) and (@@) from Polysemy
  • Removed withLowerToIO from Polysemy. Use withWeavingToFinal instead.
  • Removed asyncToIO and lowerAsync from Polysemy.Async. Use asyncToIOFinal instead.
  • Removed lowerEmbedded from Polysemy.IO. Use embedToMonadIO instead.
  • Removed lowerError from Polysemy.Error. Use errorToIOFinal instead.
  • Removed resourceToIO and lowerResource from Polysemy.Resource. Use resourceToIOFinal instead.
  • Removed runFixpoint and runFixpointM from Polysemy.Fixpoint. Use fixpointToFinal instead.
  • Changed semantics of errorToIOFinal so that it no longer catches errors from other handlers of the same type.
  • The semantics of runScoped has been changed so that the provided interpreter is now used only once per use of scoped, instead of each individual action.

Other Changes

  • Exposed send from Polysemy.
  • Dramatically improved build performance of projects when compiling with -O2.
  • Removed the debug dump-core flag.
  • Introduced the new meta-effect Scoped, which allows running an interpreter locally whose implementation is deferred to a later stage.
  • Fixed a bug in various Scoped interpreters where any explicit recursive interpretation of higher-order computations that the handler may perform are ignored by the interpreter, and the original handler was reused instead.

1.7.1.0 (2021-11-23)

Other Changes

  • Support GHC 9.2.1

1.7.0.0 (2021-11-16)

Breaking Changes

  • Added interpreters for AtomicState that run in terms of State.
  • Removed MemberWithError
  • Removed DefiningModule

Other Changes

  • The internal ElemOf proof is now implemented as an unsafe integer, significantly cutting down on generated core.
  • Polysemy no longer emits custom type errors for ambiguous effect actions. These have long been rendered moot by polysemy-plugin, and the cases that they still exist are usually overeager (and wrong.)
  • As a result, the core produced by polysemy is significantly smaller. Programs should see a reduction of ~20% in terms and types, and ~60% in coercions.

1.6.0.0 (2021-07-12)

Breaking Changes

  • Deprecate traceToIO and replace it with traceToStdout and traceToStderr
  • Support GHC 9.0.1

Other Changes

  • Added the combinator insertAt, which allows adding effects at a specified index into the effect stack
  • Added Semigroup and Monoid for Sem

1.5.0.0 (2021-03-30)

Breaking Changes

  • Dropped support for GHC 8.4

Other Changes

  • Added InterpretersFor as a shorthand for interpreters consuming multiple effects
  • Added runTSimple and bindTSimple, which are simplified variants of runT and bindT

1.4.0.0 (2020-10-31)

Breaking Changes

  • Added Polysemy.Async.cancel to allow cancelling Async action (possible name collision) (#321, thanks to @aidangilmore)

Other Changes

  • Added Polysemy.Input.inputs to mirror Polysemy.Reader.asks (#327, thanks to @expipiplus1)
  • Added Polysemy.Resource.bracket_ (#335, thanks to @expipiplus1)
  • Support GHC 8.10.x (#337, #382)
  • Restrict the existentially quantified monad in a Weaving to Sem r (#333, thanks to @A1kmm)
  • Added raise2Under and raise3Under (#369)
  • Added Raise and Subsume (#370)
  • Fixed memory leaks in Applicative (Sem r) methods (#372, thanks to @goertzenator)
  • Smaller suggestions and fixes (thanks to @jeremyschlatter, @galagora and @felixonmars)

1.3.0.0 (2020-02-14)

Breaking Changes

  • The semantics for runNonDet when <|> is used inside a higher-order action of another effect has been reverted to that of 1.1.0.0 and earlier. (See issue #246)
  • Type parameters for outputToTrace have been rearranged (thanks to @juanpaucar)

Other Changes

  • Added Bundle effect, for bundling multiple effects into a single one.
  • Added Tagged effect, for annotating and disambiguating identical effects.
  • Added View effect, an Input-like effect for caching an expensive computation.
  • Added fromException/Via and fromExceptionSem/Via
  • Added note
  • Added catchJust, try and tryJust (thanks to @bolt12)
  • Using listen with runWriterTVar or writerToIO will no longer delay writing until the listen completes.
  • Added runStateSTRef and stateToST (thanks to @incertia)
  • Added execState and execLazyState (thanks to @tjweir)
  • Added Polysemy.Law, which offers machinery for creating laws for effects.
  • Added Polysemy.Membership for retrieving and making use of effect membership proofs.

1.2.3.0 (2019-10-29)

  • Polysemy now works on GHC 8.8.1 (thanks to @googleson78 and @sevanspowell)
  • Exported MemberWithError from Polysemy
  • Added rewrite and transform interpretation combinators

1.2.2.0 (2019-10-22)

  • Fixed a bug in resourceToIO and resourceToIOFinal that prevented the finalizers from being called in BracketOnError when the computation failed due to a Sem failure
  • Added atomicGets (thanks to @googleson78)
  • Added sequenceConcurrently to Polysemy.Async (thanks to @spacekitteh)

1.2.1.0 (2019-09-15)

  • Added InterpreterFor (thanks to @bolt12)
  • Bumped bounds for first-class-families

1.2.0.0 (2019-09-04)

Breaking Changes

  • All lower- interpreters have been deprecated, in favor of corresponding -Final interpreters.
  • runFixpoint and runFixpointM have been deprecated in favor of fixpointToFinal.
  • The semantics for runNonDet when <|> is used inside a higher-order action of another effect has been changed.
  • Type variables for certain internal functions, failToEmbed, and atomicState' have been rearranged.

Other changes

  • Added Final effect, an effect for embedding higher-order actions in the final monad of the effect stack. Any interpreter should use this instead of requiring to be provided an explicit lowering function to the final monad.
  • Added Strategy environment for use together with Final
  • Added asyncToIOFinal, a better alternative of lowerAsync
  • Added errorToIOFinal, a better alternative of lowerError
  • Added fixpointToFinal, a better alternative of runFixpoint and runFixpointM
  • Added resourceToIOFinal, a better alternative of lowerResource
  • Added outputToIOMonoid and outputToIOMonoidAssocR
  • Added stateToIO
  • Added atomicStateToIO
  • Added runWriterTVar, writerToIOFinal, and writerToIOAssocRFinal
  • Added writerToEndoWriter
  • Added subsume operation
  • Exposed raiseUnder/2/3 in Polysemy

1.1.0.0 (2019-08-15)

Breaking Changes

  • MonadFail is now implemented in terms of Fail, instead of NonDet (thanks to @KingoftheHomeless)
  • LastMember has been removed. withLowerToIO and all interpreters that make use of it now only requires Member (Embed IO) r (thanks to @KingoftheHomeless)
  • State and Writer now have better strictness semantics

Other Changes

  • Added AtomicState effect (thanks to @KingoftheHomeless)
  • Added Fail effect (thanks to @KingoftheHomeless)
  • Added runOutputSem (thanks to @cnr)
  • Added modify', a strict variant of modify (thanks to @KingoftheHomeless)
  • Added right-associative variants of runOutputMonoid and runWriter (thanks to @KingoftheHomeless)
  • Added runOutputMonoidIORef and runOutputMonoidTVar (thanks to @KingoftheHomeless)
  • Improved Fixpoint so it won’t always diverge (thanks to @KingoftheHomeless)
  • makeSem will now complain if DataKinds isn’t enabled (thanks to @pepegar)

1.0.0.0 (2019-07-24)

Breaking Changes

  • Renamed Lift to Embed (thanks to @googleson78)
  • Renamed runAsyncInIO to lowerAsync
  • Renamed runAsync to asyncToIO
  • Renamed runBatchOutput to runOutputBatched
  • Renamed runConstInput to runInputConst
  • Renamed runEmbed to runEmbedded (thanks to @googleson78)
  • Renamed runEmbedded to lowerEmbedded
  • Renamed runErrorAsAnother to mapError
  • Renamed runErrorInIO to lowerError
  • Renamed runFoldMapOutput to runOutputMonoid
  • Renamed runIO to embedToMonadIO
  • Renamed runIgnoringOutput to ignoreOutput
  • Renamed runIgnoringTrace to ignoreTrace
  • Renamed runInputAsReader to inputToReader
  • Renamed runListInput to runInputList
  • Renamed runMonadicInput to runInputSem
  • Renamed runOutputAsList to runOutputList
  • Renamed runOutputAsTrace to outputToTrace
  • Renamed runOutputAsWriter to outputToWriter
  • Renamed runResourceBase to resourceToIO
  • Renamed runResourceInIO to lowerResource
  • Renamed runStateInIORef to runStateIORef
  • Renamed runTraceAsList to runTraceList
  • Renamed runTraceAsOutput to traceToOutput
  • Renamed runTraceIO to traceToIO
  • Renamed sendM to embed (thanks to @googleson78)
  • The NonDet effect is now higher-order (thanks to @KingoftheHomeless)

Other Changes

  • Added evalState and evalLazyState
  • Added runNonDetMaybe (thanks to @KingoftheHomeless)
  • Added nonDetToError (thanks to @KingoftheHomeless)
  • Haddock documentation for smart constructors generated via makeSem will no longer have weird variable names (thanks to @TheMatten)

0.7.0.0 (2019-07-08)

Breaking Changes

  • Added a Pass constructor to Writer (thanks to @KingoftheHomeless)
  • Fixed a bug in runWriter where the MTL semantics wouldn’t be respected (thanks to @KingoftheHomeless)
  • Removed the Censor constructor of Writer (thanks to @KingoftheHomeless)
  • Renamed Yo to Weaving
  • Changed the visible type applications for asks, gets, runEmbedded, fromEitherM and runErrorAsAnother

Other Changes

  • Fixed haddock generation

0.6.0.0 (2019-07-04)

Breaking Changes

  • Changed the type of runBatchOutput to be more useful (thanks to @Infinisil)

Other Changes

  • THE ERROR MESSAGES ARE SO MUCH BETTER :party: :party: :party:
  • Added runEmbedded to Polysemy.IO
  • Added runOutputAsList to Polysemy.Output (thanks to @googleson78)
  • Asymptotically improved the performance of runTraceAsList (thanks to @googleson78)

0.5.1.0 (2019-06-28)

  • New combinators for Polysemy.Error: fromEither and fromEitherM

0.5.0.1 (2019-06-27)

  • Fixed a bug where intercept and interceptH wouldn’t correctly handle higher-order effects

0.5.0.0 (2019-06-26)

Breaking Changes

  • Removed the internal Effect machinery

New Effects and Interpretations

  • New effect; Async, for describing asynchronous computations
  • New interpretation for Resource: runResourceBase, which can lower Resource effects without giving a lowering natural transformation
  • New interpretation for Trace: runTraceAsList
  • New combinator: withLowerToIO, which is capable of transforming IO-invariant functions as effects.

Other Changes

  • Lots of hard work on the package and CI infrastructure to make it green on GHC 8.4.4 (thanks to @jkachmar)
  • Changed the order of the types for runMonadicInput to be more helpful (thanks to @tempname11)
  • Improved the error machinery to be more selective about when it runs
  • Factored out the TH into a common library for third-party consumers

0.4.0.0 (2019-06-12)

Breaking Changes

  • Renamed runResource to runResourceInIO

Other Changes

  • Added runResource, which runs a Resource purely
  • Added onException, finally and bracketOnError to Resource
  • Added a new function, runResource which performs bracketing for pure code

0.3.0.1 (2019-06-09)

  • Fixed a type error in the benchmark caused by deprecation of Semantic

0.3.0.0 (2019-06-01)

Breaking Changes

  • Removed all deprecated names
  • Moved Random effect to polysemy-zoo

Other Changes

  • makeSem can now be used to create term-level operators (thanks to @TheMatten)

0.2.2.0 (2019-05-30)

  • Added getInspectorT to the Tactical functions, which allows polysemy code to be run in external callbacks
  • A complete rewrite of Polysemy.Internal.TH.Effect (thanks to @TheMatten)
  • Fixed a bug in the TH generation of effects where the splices could contain usages of effects that were ambiguous

0.2.1.0 (2019-05-27)

  • Fixed a bug in the Alternative instance for Sem, where it would choose the last success instead of the first
  • Added MonadPlus and MonadFail instances for Sem

0.2.0.0 (2019-05-23)

Breaking Changes

  • Lower precedence of .@ and .@@ to 8, from 9

Other Changes

  • Fixed a serious bug in interpretH and friends, where higher-order effects would always be run with the current interpreter.
  • Users need no longer require inlineRecursiveCalls — the polysemy-plugin-0.2.0.0 will do it automatically when compiling with -O
  • Deprecated inlineRecursiveCalls; slated for removal in the next version

0.1.2.1 (2019-05-18)

  • Give explicit package bounds for dependencies
  • Haddock improvements
  • Remove Typeable machinery from Polysemy.Internal.Union (thanks to @googleson78)

0.1.2.0 (2019-04-26)

  • runInputAsReader, runTraceAsOutput and runOutputAsWriter have more generalized types
  • Added runStateInIO
  • Added runOutputAsTrace
  • Added Members (thanks to @TheMatten)

0.1.1.0 (2019-04-14)

  • Added runIO interpretation (thanks to @adamConnerSax)
  • Minor documentation fixes

0.1.0.0 (2019-04-11)

  • Initial release