MIT licensed by Freckle Engineering
Maintained by [email protected]
This version can be pinned in stack with:faktory-1.1.3.0@sha256:29ac65bf4fa7426b60d93408660806ca467b697d7bec29e8ccd3a274c3051e54,9973
# faktory\_worker\_haskell

[![Hackage](https://img.shields.io/hackage/v/faktory.svg?style=flat)](https://hackage.haskell.org/package/faktory)
[![Stackage Nightly](http://stackage.org/package/faktory/badge/nightly)](http://stackage.org/nightly/package/faktory)
[![Stackage LTS](http://stackage.org/package/faktory/badge/lts)](http://stackage.org/lts/package/faktory)
[![CI](https://github.com/freckle/faktory_worker_haskell/actions/workflows/ci.yml/badge.svg)](https://github.com/freckle/faktory_worker_haskell/actions/workflows/ci.yml)

Haskell client and worker process for the Faktory background job server.

Architecture overview from [Ruby client README](https://github.com/contribsys/faktory_worker_ruby#readme):

```
+--------------------+
| |
| Faktory |
| Server |
+---------->>>>| +>>>>--------+
| | | |
| | | |
| +--------------------+ |
+-----------------+ +-------------------+
| | | |
| Client | | Worker |
| pushes | | pulls |
| jobs | | jobs |
| | | |
| | | |
+-----------------+ +-------------------+
```

- Client - an API any process can use to push jobs to the Faktory server.
- Worker - a process that pulls jobs from Faktory and executes them.
- Server - the Faktory daemon which stores background jobs in queues to be
processed by Workers.

This package contains only the client and worker parts. The server part is
[here](https://github.com/contribsys/faktory/)

## Installation

- Hackage: http://hackage.haskell.org/package/faktory
- Stackage: *Coming soon*

## Faktory Documentation

See the [wiki](//github.com/contribsys/faktory/wiki) for more
details.

## Usage

<!--
```haskell
import Data.Aeson
import Prelude
import Faktory.Producer
import Faktory.Job
import Faktory.Worker
import GHC.Generics
import Text.Markdown.Unlit ()

{- Don't actually run anything -}
main :: IO ()
main = if True then pure () else (workerMain >> producerMain)
workerMain :: IO ()
producerMain :: IO ()
```
-->

### Job

Any value can be a "Job" that is pushed and pulled to and from Faktory via its
`ToJSON` and `FromJSON` instances:

```haskell
newtype MyJob = MyJob
{ myJobMessage :: String
}
deriving stock Generic
deriving anyclass (ToJSON, FromJSON)
```

### Worker

```haskell
workerMain = runWorkerEnv $ \job -> do
-- Process your Job here
putStrLn $ jobJid job
putStrLn $ myJobMessage $ jobArg job

-- If any exception is thrown, the job will be marked as Failed in Faktory
-- and retried. Note: you will not otherwise hear about any such exceptions,
-- unless you catch-and-rethrow them yourself.
```

### Producer

`Producer` wraps `Client` for push-only usage.

```haskell
producerMain = do
producer <- newProducerEnv

jobId <- perform mempty producer $ MyJob "Hello world"

print jobId

closeProducer producer
```

### Configuration

When using `envSettings`, the following variables will be used:

- `FAKTORY_PROVIDER`: the name of another environment variable where the
connection string can be found. Defaults to `FAKTORY_URL`.
- `FAKTORY_URL` (or whatever you named in `FAKTORY_PROVIDER`): connection string
to the Faktory server. Format is
`tcp(+tls)://(:password@)host:port(/namespace)`. Defaults to
`tcp://localhost:4719`. `namespace` is prependend to queue names on job
submission and worker consumption.

When using `envWorkerSettings`, the following variables are also used:

- `FAKTORY_QUEUE`: the name of the queue to consume from. Default is "default".
- `FAKTORY_WORKER_ID`: the Id to use for this Worker. Default is to assign a
random one.

## Examples

See the [examples](./examples). To run them:

1. Run a local Faktory server

```console
docker run --rm \
--publish 7419:7419 \
--publish 7420:7420 \
contribsys/faktory
```

1. Run the consumer example

```console
% stack exec faktory-example-consumer
Starting consumer loop
```

(Assumes you've built the project.)

1. Submit a Job through the producer example

```console
% stack exec faktory-example-producer hello world
Pushed job: "ljcjlbexbgun"
```

*NOTE*: if you submit "BOOM" as a Job, the processing loop will raise an
exception, so you can see how a Failed Job looks in Faktory.

1. See that your Job was processed back in the consumer

```console
% stack exec faktory-example-consumer
Starting consumer loop
hello world
```

## Development & Tests

```console
stack build --dependencies-only --test --no-run-tests
stack build --pedantic --test --no-run-tests
stack build --pedantic --test
```

- `FactorySpec` requires a local Faktory server is running, and it will flush
all Jobs from this server as part of running the tests.
- The tests for `BATCH` require testing against an Enterprise Faktory image

---

[CHANGELOG](./CHANGELOG.md) | [LICENSE](./LICENSE)

Changes

Unreleased

v1.1.3.0

  • Add Faktory.Pool

    This incurs the new dependencies, unliftio, resource-pool, and microlens.

v1.1.2.7

  • Fix handling decoding errors of the Job payload. Previously, it would log an error and neither ACK nor FAIL the Job. Now it FAILs the Job.

v1.1.2.6

  • Support ghc-9.8 and aeson-2.2

v1.1.2.5

  • Migrate to crypton-connection
  • Remove CI for GHC 8.6

v1.1.2.4

  • Fix jobBatchId to work for all job types. Faktory seems to use both bid and _bid in a jobs custom object when enqueing jobs. This allows the parser to use both

v1.1.2.3

  • Set KeepAlive in connections to Faktory (@jagonalez)

v1.1.2.2

  • Support GHCs 9.0 and 9.2

v1.1.2.1

  • Support aeson 2.x

v1.1.2.0

  • Add reserveFor and jobReserveForMicroseconds for setting ACK window for individual jobs.
  • Timeout jobs that have exceeded their reserve_for setting. Jobs without an explicit reserve_for will default to Faktory’s 1800 second timeout.
  • Allow configuration of default job options via settingsDefaultJobOptions.

v1.1.1.0

  • Add jobRemainingRetries

v1.1.0.0

  • Pass value of type Job arg (not arg) to run-worker loops

    This will give consumer loops access to details like jobJid and jobOptions, so they can (for example) call TRACK SET.

    Call jobArg to get back what you were getting before this change.

  • Support BATCH STATUS

  • Add tracked Job Option

  • Deprecate trackPerform (use perform (options <> tracked) instead)

v1.0.3.1

  • Export lower-level BATCH functions

v1.0.3.0

  • Support for TRACK (Enterprise only)

v1.0.2.3

  • Remove dependencies upper bounds

v1.0.2.2

  • Relax dependencies upper bounds

v1.0.2.1

  • Fix bug in at parsing of consumed Job payloads

v1.0.2.0

  • Partial BATCH support (Enterprise only)
  • Support for custom field in Job payloads
  • Lower-level buildJob and commandByteString functions

v1.0.1.6

  • Relax dependencies upper bounds

v1.0.1.5

  • Maintain version bounds

v1.0.1.4

  • Various CI and dependency bounds changes

v1.0.1.3

  • Add support for queue namespacing

v1.0.1.2

  • Fix internal handling of invalid Server Replies

v1.0.1.1

  • Include non-OK reply in commandOK error
  • Build with GHC-8.8

v1.0.1.0

  • Upgrade to megaparsec-7

v1.0.0.0

Initial release.