Streaming data processing library.

Version on this page:1.2.5
LTS Haskell 8.12:1.2.10
Stackage Nightly 2017-04-24:1.2.10
Latest on Hackage:1.2.10
MIT licensed by Michael Snoyman


conduit is a solution to the streaming data problem, allowing for production, transformation, and consumption of streams of data in constant memory. It is an alternative to lazy I\/O which guarantees deterministic resource handling.

For more information about conduit in general, and how this package in particular fits into the ecosystem, see the conduit homepage.



  • Add PrimMonad instances for ConduitM and Pipe #306

  • Ensure downstream and inner sink receive same inputs in passthroughSink #304




  • Expose yieldM for ConduitM #270

  • Fix test suite compilation on older GHCs

  • In zipConduitApp, left bias not respected mixing monadic and non-monadic conduits #263

  • Fix benchmark by adding a type signature

  • Doc updates

  • resourcet cannot be built with GHC 8 #242
  • Remove upper bound on transformers #253


  • sourceToList
  • Canonicalise Monad instances #237


  • mapAccum and mapAccumM should be strict in their state #218

  • Some documentation improvements


1.2.3 Expose connect and fuse as synonyms for $$ and =$=, respectively.

1.2.2 Lots more stream fusion.

1.2 Two performance optimizations added. (1) A stream fusion framework. This is a non-breaking change. (2) Codensity transform applied to the ConduitM datatype. This only affects users importing the .Internal module. Both changes are thoroughly described in the following to blog posts: Speeding up conduit, and conduit stream fusion.

1.1 Refactoring into conduit and conduit-extra packages. Core functionality is now in conduit, whereas most common helper modules (including Text, Binary, Zlib, etc) are in conduit-extra. To upgrade to this version, there should only be import list and conduit file changes necessary.

1.0 Simplified the user-facing interface back to the Source, Sink, and Conduit types, with Producer and Consumer for generic code. Error messages have been simplified, and optional leftovers and upstream terminators have been removed from the external API. Some long-deprecated functions were finally removed.

0.5 The internals of the package are now separated to the .Internal module, leaving only the higher-level interface in the advertised API. Internally, switched to a Leftover constructor and slightly tweaked the finalization semantics.

0.4 Inspired by the design of the pipes package: we now have a single unified type underlying Source, Sink, and Conduit. This type is named Pipe. There are type synonyms provided for the other three types. Additionally, BufferedSource is no longer provided. Instead, the connect-and-resume operator, $$+, can be used for the same purpose.

0.3 ResourceT has been greatly simplified, specialized for IO, and moved into a separate package. Instead of hard-coding ResourceT into the conduit datatypes, they can now live around any monad. The Conduit datatype has been enhanced to better allow generation of streaming output. The SourceResult, SinkResult, and ConduitResult datatypes have been removed entirely.

0.2 Instead of storing state in mutable variables, we now use CPS. A Source returns the next Source, and likewise for Sinks and Conduits. Not only does this take better advantage of GHC\'s optimizations (about a 20% speedup), but it allows some operations to have a reduction in algorithmic complexity from exponential to linear. This also allowed us to remove the Prepared set of types. Also, the State functions (e.g., sinkState) use better constructors for return types, avoiding the need for a dummy state on completion.

0.1 BufferedSource is now an abstract type, and has a much more efficient internal representation. The result was a 41% speedup on microbenchmarks (note: do not expect speedups anywhere near that in real usage). In general, we are moving towards BufferedSource being a specific tool used internally as needed, but using Source for all external APIs.

0.0 Initial release.

Used by 357 packages:
BCMtools, BiobaseFasta, BiobaseTurner, BlastHTTP, Buster, CurryDB, DnaProteinAlignment, EntrezHTTP, JunkDB, JunkDB-driver-gdbm, JunkDB-driver-hashtables, MFlow, MusicBrainz, OnRmt, Yablog, ac-machine-conduit, alpino-tools, amazon-products, amazonka, amazonka-core, amazonka-s3-streaming, amazonka-test, amqp-conduit, approx-rand-test, ascii85-conduit, atndapi, atom-conduit, attoparsec-conduit, authenticate, autonix-deps, autonix-deps-kf5, aws, aws-dynamodb-conduit, aws-dynamodb-streams, aws-elastic-transcoder, aws-kinesis, aws-kinesis-client, aws-kinesis-reshard, aws-sdk, aws-sdk-xml-unordered, aws-simple, aws-sns, azure-acs, azure-servicebus, azurify, b9, base64-conduit, bbi, beam, bert, binary-conduit, biostockholm, bitcoin-api-extra, bits-conduit, bittorrent, blaze-builder-conduit, buildwrapper, bzlib-conduit, cab, cabal-src, cassava-conduit, cassy, cereal-conduit, classy-prelude-conduit, coinbase-exchange, conduit-audio, conduit-audio-lame, conduit-audio-samplerate, conduit-audio-sndfile, conduit-combinators, conduit-connection, conduit-extra, conduit-find, conduit-iconv, conduit-merge, conduit-network-stream, conduit-parse, conduit-resumablesink, conduit-tokenize-attoparsec, couchdb-conduit, cpio-conduit, cqrs-sqlite3, cqrs-test, craft, credentials, credentials-cli, crypto-conduit, cryptohash-conduit, cryptonite-conduit, csv-conduit, cypher, d-bus, data-object-yaml, decoder-conduit, dice-entropy-conduit, dingo-core, dns, docker, dockercook, dropbox-sdk, dtd, dynamodb-simple, editor-open, engine-io-yesod, esqueleto, eurofxref, fb, feed-crawl, feed-gipeda, filesystem-conduit, find-conduit, fluent-logger-conduit, fold-debounce-conduit, fsnotify-conduit, ftp-conduit, gc-monitoring-wai, gist, git-annex, gitlib, gitlib-cmdline, gitlib-libgit2, gitlib-s3, gitlib-test, gitlib-utils, gnss-converters, gogol, gogol-core, google-drive, gpx-conduit, gridfs, hOpenPGP, hackage-mirror, hackage-proxy, haddocset, hails, hascar, haskdeep, haskell-aliyun, haskell-ftp, haskell-spacegoo, haskoin-core, haskoin-node, haskoin-wallet, hats, haxl-amazonka, haxl-facebook, hdbi-conduit, hell, hemokit, hissmetrics, hoodle-extra, hoogle, hopenpgp-tools, hops, hs-pkpass, hslogstash, hstradeking, html-conduit, http-client-auth, http-conduit, http-conduit-browser, http-conduit-downloader, http-proxy, http-reverse-proxy, hunt-server-cli, hw-conduit, hw-json, hw-json-lens, hw-kafka-conduit, hw-succinct, hw-xml, hzulip, iap-verifier, ig, imagesize-conduit, imm, irc-client, irc-conduit, jammittools, json-rpc, jsonrpc-conduit, kcd, keiretsu, keter, lambdatwit, legion, legion-discovery, legion-extra, leksah, leksah-server, libarchive-conduit, libjenkins, libssh2-conduit, lio-simple, llvm-tools, lord, lz4-conduit, lzma-conduit, macbeth-lib, magicbane, mailgun, mangopay, markdown, markdown-kate, marvin, masakazu-bot, mediabus, mediabus-fdk-aac, mediabus-rtp, memcache-conduit, mighttpd2, mime-mail-ses, minio-hs, mirror-tweet, monad-logger, mongoDB, msgpack-rpc, murmur, ndjson-conduit, network-conduit, network-conduit-tls, network-msgpack-rpc, notmuch-web, ntrip-client, nvim-hs, omnicodec, open-symbology, opml-conduit, osdkeys, osm-conduit, osm-download, pager, pagerduty, patterns, paypal-api, pcap-conduit, persistent, persistent-mongoDB, persistent-mysql, persistent-mysql-haskell, persistent-odbc, persistent-postgresql, persistent-relational-record, persistent-sqlite, persistent-zookeeper, pgdl, pgstream, phoityne, phoityne-vscode, phone-push, pipes-conduit, pontarius-xmpp, powerqueue-distributed, process-conduit, project-template, publicsuffixlistcreate, pubnub, push-notify, req-conduit, rss-conduit, rtcm, samtools-conduit, sandi, sbp, sbp2udp, scion-browser, scrz, second-transfer, selenium-server, seqloc-datafiles, serf, shell-conduit, skylark-client, sloane, smtps-gmail, snappy-conduit, soap, socketio, sproxy2, sssp, stack, stack-run, stackage-cli, stackage-curator, stackage-metadata, stm-conduit, stm-firehose, stomp-conduit, stomp-queue, store, stutter, tablestorage, tagstream-conduit, tamarin-prover, tcache-AWS, textocat-api, thumbnail-plus, time-warp, tkyprof, twhs, twitter-conduit, typed-process, unix-process-conduit, uri-conduit, vector-conduit, wai-app-file-cgi, wai-conduit, wai-lite, wai-middleware-cache, wai-middleware-cache-redis, wai-middleware-consul, wai-static-cache, wai-static-pages, wai-util, warp-tls-uid, welshy, withdependencies, wolf, wsdl, wybor, xinput-conduit, xls, xlsior, xlsx, xlsx-templater, xml-catalog, xml-conduit, xml-conduit-parse, xml-html-qq, xml2json, xournal-parser, yackage, yahoo-finance-conduit, yaml, yesod, yesod-auth, yesod-auth-fb, yesod-bin, yesod-content-pdf, yesod-core, yesod-eventsource, yesod-fb, yesod-mangopay, yesod-persistent, yesod-platform, yesod-s3, yesod-sitemap, yesod-static, yesod-test-json, yesod-websockets, yxdb-utils, zendesk-api, zeromq3-conduit, zim-parser, zip, zip-conduit, zlib-conduit
comments powered byDisqus