streamly-core

Streaming, parsers, arrays, serialization and more

https://streamly.composewell.com

LTS Haskell 22.18:0.2.2
Stackage Nightly 2024-04-27:0.2.2
Latest on Hackage:0.2.2

See all snapshots streamly-core appears in

BSD-3-Clause licensed by Composewell Technologies
Maintained by [email protected]
This version can be pinned in stack with:streamly-core-0.2.2@sha256:e44a0f6e4870df7a206dbaad531deac7907a73a883fbdb5a460d63a8e086efe7,19728

Module documentation for 0.2.2

Used by 2 packages in nightly-2024-03-29(full list with versions):
streamly, streamly-process

For upgrading to streamly-0.9.0+ please read the Streamly-0.9.0 upgrade guide.

Streamly is a standard library for Haskell that focuses on C-like performance, modular combinators, and streaming data flow model. Streamly consists of two packages: "streamly-core" and "streamly". streamly-core provides basic features, and depends only on GHC boot libraries (see note below), while streamly provides higher-level features like concurrency, time, lifted exceptions, and networking. For documentation, visit the Streamly website.

The streamly-core package provides the following functionality:

  • Streams as composable producers of a sequence of values.

  • Streams provide all the functionality provided by Haskell lists but in an effectful, streaming fashion with better performance.

  • Streams provide ListT and logic programming functionality as well.

  • Folds as composable stream consumers that reduce the streams to a single value or reduce segments of streams to transform the stream.

  • Parsers as more powerful, composable stream consumers supporting standard parser combinators with backtracking but in a streaming fashion.

  • Arrays with streaming interfaces providing high performance, modularity, and concise interface as all streaming operations can be performed on arrays.

  • Arrays can be immutable or mutable, unboxed or boxed, pinned or unpinned.

  • Arrays generalize the functionality provided by bytestring and text packages.

  • Interoperability with bytestring and text is provided via separate packages.

  • Arrays and folds provide natural builder functionality so there are no separate builder modules.

  • High performance binary serialization with configurable JSON like features.

  • Streaming combinators for unicode text processing, providing functionality equivalent to the text package.

  • String interpolation for convenient construction of strings.

  • Streaming console IO (stdin/stdout) operations.

  • Streaming file and directory IO operations.

This package covers some or all of the functionality covered by streaming, pipes, conduit, list-t, logic-t, foldl, attoparsec, array, primitive, vector, vector-algorithms, binary, cereal, store, bytestring, text, stringsearch, interpolate. Streamly provides a consistent, concise, modular and performant interface for all this functionality.

Note: The dependencies "heaps" and "monad-control" are included in the package solely for backward compatibility, and will be removed in future versions.

Changes

Changelog

0.2.2 (Jan 2024)

  • Add fixities infixr 5 for cons and consM functions.
  • Fix a bug in Array Eq instance when the type is a sum type with differently sized constructors.
  • lpackArraysChunksOf, compact, writeChunksWith, putChunksWith now take the buffer size in number of array elements instead of bytes.

0.2.1 (Dec 2023)

  • Make the serialization of the unit constructor deterministic.
  • Expose pinnedSerialize & deserialize via Streamly.Data.Array.

0.2.0 (Nov 2023)

See 0.1.0-0.2.0 API Changelog for a full list of API changes in this release. Only a few significant changes are mentioned here.

Breaking Changes

  • ParserK in Streamly.Data.ParserK is not implicitly specialized to arrays anymore. To adapt to the new code, change ParserK a m b to ParserK (Array a) m b where the Array type comes from Streamly.Data.Array. This change also affected the signatures of parseChunks and parseBreakChunks.
  • Changed the signature of ‘Streamly.Data.Stream.handle’ to make the exception handler monadic.
  • Behavior change: Exceptions are now rethrown promptly in bracketIO.

Enhancements

  • Serialization: Added a Streamly.Data.MutByteArray module with a Serialize type class for fast binary serialization. The Data.Array module supplies the serialize and deserialize operations for arrays.
  • Unpinned Arrays: Unboxed arrays are now created unpinned by default, they were created pinned earlier. During IO operations, unpinned arrays are automatically copied to pinned memory. When arrays are directly passed to IO operations programmers can choose to create them pinned to avoid a copy. To create pinned arrays, use the internal APIs with the pinned* prefix.
  • StreamK now supports native exception handling routines (handle, bracketIO). Earlier we had to convert it to the Stream type for exception handling.

Deprecations

See 0.1.0-0.2.0 API Changelog for a full list of deprecations.

Internal API Changes

  • Fold constructor has changed, added a final field to support finalization and cleanup of a chain of folds. The extract field is now used only for mapping the fold internal state to fold result for scanning purposes. If your fold does not require cleanup you can just use your existing extract function as final as well to adapt to this change.
  • Many low level internal modules have been removed, they are entirely exported from higher level internal modules. If you were importing any of the missing low level modules then import the higher level modules instead.
  • Internal module changes:
    • Streamly.Internal.Serialize.FromBytes -> Streamly.Internal.Data.Binary.Parser
    • Streamly.Internal.Serialize.ToBytes -> Streamly.Internal.Data.Binary.Stream
    • Streamly.Internal.Data.Unbox is now exported via Streamly.Internal.Data.Serialize
    • Streamly.Internal.Data.IORef.Unboxed is now exported via Streamly.Internal.Data.Serialize

0.1.0 (March 2023)

Also see streamly-core-0.1.0 API Changelog or https://hackage.haskell.org/package/streamly-core-0.1.0/docs/docs/ApiChangelogs/0.1.0.txt

streamly package is split into two packages, (1) streamly-core that has only GHC boot library depdendecies, and (2) streamly that contains higher level operations (including concurrent ones) with additional dependencies.

  • Moved the following modules from streamly package to the streamly-core package:
    • Streamly.Console.Stdio
    • Streamly.Data.Fold
    • Streamly.Data.Unfold
    • Streamly.FileSystem.Handle
    • Streamly.Unicode.Stream
  • Added the following new modules:
    • Streamly.Data.Array
    • Streamly.Data.Array.Generic
    • Streamly.Data.MutArray
    • Streamly.Data.MutArray.Generic
    • Streamly.Data.Parser
    • Streamly.Data.ParserK
    • Streamly.Data.Stream
    • Streamly.Data.StreamK
    • Streamly.FileSystem.Dir
    • Streamly.FileSystem.File
    • Streamly.Unicode.Parser
    • Streamly.Unicode.String