pooled-io

The motivation for this package was to run computations on multiple cores that need to write intermediate results to disk. The functions restrict the number of simultaneously running jobs to a user given number or to the number of capabilities the Haskell program was started with, i.e. the number after the RTS option -N.

There are some flavors of this functionality:

• Control.Concurrent.PooledIO.Independent: run independent actions without results in parallel

• Control.Concurrent.PooledIO.Final: run independent actions with a final result in parallel

• Control.Concurrent.PooledIO.InOrder: run jobs in parallel with data dependencies like make -j n

Additionally there is the module Control.Concurrent.PooledIO.Sequence that helps to serialize I/O actions from multiple threads. It is certainly most useful in connection with Control.Concurrent.PooledIO.Independent.

Related packages:

• lazyio: interleave IO actions in a single thread

• async: start threads and wait for their results, forward exceptions, but do not throttle concurrency with respect to number of available cores

• parallel-tasks:

• parallel-io:

• threadPool:

• threads-pool:

• cio:

• Control-Engine:

• taskpool, async-pool: