Run jobs on a limited number of threads and support data dependencies http://hub.darcs.net/thielema/pooled-io/
|Latest on Hackage:||0.0.2.1|
This package is not currently in any snapshots. If you're interested in using it, we recommend adding it to Stackage Nightly. Doing so will make builds more reliable, and allow stackage.org to host generated Haddocks.
The motivation for this package was to run computations on multiple cores
that need to write intermediate results to disk.
The functions restrict the number of simultaneously running jobs
to a user given number or to the number of capabilities
the Haskell program was started with,
i.e. the number after the RTS option
There are some flavors of this functionality:
Control.Concurrent.PooledIO.Independent: run independent actions without results in parallel
Control.Concurrent.PooledIO.Final: run independent actions with a final result in parallel
Control.Concurrent.PooledIO.InOrder: run jobs in parallel with data dependencies like
make -j n
Additionally there is the module
that helps to serialize I/O actions from multiple threads.
It is certainly most useful in connection with
lazyio: interleave IO actions in a single thread
async: start threads and wait for their results, forward exceptions, but do not throttle concurrency with respect to number of available cores