statsd-rupp

statsd-rupp is a StatsD client for Haskell.

It works in two ways, which you can use separately or together:

• As a client: it sends each sample to the StatsD server as it happens, with optional sampling.
• As an aggregator: it keeps samples in memory and every flush interval sends statistics, such as counts, rates, means, medians and percentiles, like a StatsD server does. This lets you feed backends that speak the StatsD line protocol but do not aggregate, such as some Graphite and InfluxDB relays.

Metrics go over UDP and send errors are ignored, so a missing StatsD server never breaks your program.

• API documentation
• Changelog
• Report a bug

Contents

• Why statsd-rupp
• Design principles
• Features
• Installation
• Quick start
• Tutorial: instrumenting a web service
• Concepts
• Usage guide
• Configuration
• Metric names
• Module overview
• Development
• Related projects
• License

Why statsd-rupp

Metrics are how you find out what a program in production is doing: how many requests it handles, how slow they are, how close it is to falling over. StatsD is one of the simplest ways to get them out. Each sample is one line of text sent over UDP to a collector, so instrumenting code costs almost nothing and needs no connection management.

The classic setup has a StatsD daemon next to the program that aggregates samples into counts, rates and percentiles before they reach the time series database. Many modern collectors accept the StatsD protocol but skip that aggregation and store every line they receive. A busy service then floods them with one line per request, and percentiles of response times are nowhere to be found.

statsd-rupp covers both setups. It can send samples like any StatsD client, and it can do the daemon’s job inside the process: keep every reading in memory, and once per interval send a fixed set of statistics. The traffic stays constant however busy the program gets. It was written to monitor Haskoin Store.

Design principles

• Metrics never hurt the program. Sending is fire-and-forget UDP and errors are ignored. A collector that is down costs lost samples, never failed requests.
• Mistakes fail early. Bad keys, duplicate keys and bad configuration are refused when you create the metric or the client, so a typo shows up at start-up, not as a missing graph weeks later.
• Register once, update cheaply. Creating a metric returns a handle; updating it is one short STM transaction and at most one UDP packet.
• Sampling saves traffic, not accuracy. Sampling thins out the samples you send. Aggregated statistics always count every reading.
• Familiar names. Aggregated metrics follow the naming of the StatsD daemon (stats.counters.x.count, stats.timers.x.mean_90), so existing dashboards and alerts keep working.

Features

Area	What you get
Metric types	Counters, gauges, timings and sets
Samples	Each reading sent as it happens, with per-metric sampling rates
Aggregation	Counts and rates; gauge values; distinct set sizes; timing count, rate, mean, median, standard deviation, minimum, maximum, sum, sum of squares and percentiles
Naming	Configurable namespace and prefixes; key validation
Transport	UDP to any host and port, with an optional newline after each line
Monads	Works in any MonadIO; withStats needs MonadUnliftIO

What is not included

• No tags (the DogStatsD extension) and no histogram or distribution types.
• No TCP transport and no batching of several lines in one packet.
• No server: this library only sends.

Installation

statsd-rupp is on Hackage and in Stackage snapshots. It has no C dependencies. It needs GHC 9.2 or later, because it uses OverloadedRecordDot and NoFieldSelectors.

Step by step

1. Install Stack or Cabal.

2. Add statsd-rupp to the build-depends of your package (see below).

3. Start something that listens for StatsD on UDP port 8125. To see the raw lines while developing, netcat is enough:

   nc -u -l 8125          # some versions need: nc -u -l -p 8125
   

4. Run the quick start program with statsdServer = "127.0.0.1". You should see lines like myapp.requests:1|c straight away, and a burst of myapp.stats. lines every second.

With Cabal

build-depends:
    base
  , statsd-rupp

With Stack

# package.yaml
dependencies:
  - statsd-rupp

If your resolver does not include it, add it to stack.yaml:

extra-deps:
  - statsd-rupp-0.5.0.1

Quick start

import System.Metrics.StatsD

main :: IO ()
main = do
  let cfg = defStatConfig {namespace = "myapp", statsdServer = "statsd.local"}
  withStats cfg $ \stats -> do
    requests <- newStatCounter stats "requests" 1 -- send every sample
    latency <- newStatTiming stats "latency" 10 -- send 1 sample in 10
    users <- newStatGauge stats "users" 0 -- start at 0
    visitors <- newStatSet stats "visitors"

    incrementCounter requests 1
    addTiming latency 42
    incrementGauge users 1
    newSetElement visitors "alice"

With the default configuration this sends myapp.requests:1|c, myapp.users:+1|g and myapp.visitors:alice|s right away. Every second it sends myapp.stats.counters.requests.count, myapp.stats.timers.latency.mean and the other statistics listed under Metric names.

Tutorial: instrumenting a web service

This shows how to decide what to measure and how, for a typical request handler.

1. Decide what you need to know

For a service, start with the four questions that most incidents come down to: how much traffic, how many errors, how slow, and how saturated. Each maps to a metric type:

Question	Metric	Type
How much?	requests	counter
How many fail?	errors	counter
How slow?	latency	timing
How full?	queue.length	gauge

2. Create the metrics at start-up

Create them once, next to the rest of your configuration, and pass the handles to the code that needs them:

{-# LANGUAGE OverloadedRecordDot #-}
{-# LANGUAGE ScopedTypeVariables #-}

import GHC.Clock (getMonotonicTimeNSec)
import System.Metrics.StatsD
import UnliftIO (SomeException, throwIO, try)

data Metrics = Metrics
  { requests :: StatCounter,
    errors :: StatCounter,
    latency :: StatTiming,
    queue :: StatGauge
  }

newMetrics :: Stats -> IO Metrics
newMetrics s =
  Metrics
    <$> newStatCounter s "requests" 10 -- busy: send 1 sample in 10
    <*> newStatCounter s "errors" 1 -- rare: send them all
    <*> newStatTiming s "latency" 10
    <*> newStatGauge s "queue.length" 0

The sampling rates follow the traffic: frequent events can be sampled, rare and important ones should not be. The aggregated statistics count everything either way.

3. Record around the work

handle :: Metrics -> Request -> IO Response
handle m req = do
  incrementCounter m.requests 1
  start <- getMonotonicTimeNSec
  r <- try (serve req)
  end <- getMonotonicTimeNSec
  addTiming m.latency (fromIntegral ((end - start) `div` 1000000))
  case r of
    Left (e :: SomeException) -> incrementCounter m.errors 1 >> throwIO e
    Right resp -> return resp

Use a monotonic clock (GHC.Clock.getMonotonicTimeNSec) for durations, so that clock adjustments do not produce negative or huge timings.

4. Choose the mode

• If a real StatsD daemon aggregates for you, turn aggregation off with reportStats = False and let it do its job.
• If your collector stores raw lines, turn samples off with reportSamples = False and let the library aggregate.
• Leave both on while you find out which one your setup needs.

5. Read the results

With aggregation, stats.timers.latency.mean_95 is the average of the fastest 95% of requests in each interval, upper_95 is the 95th percentile, and count_ps is requests per second. Alert on upper_95 and on the ratio of stats.counters.errors.count to stats.counters.requests.count.

Concepts

The client handle

withStats resolves the server address, opens a UDP socket, and passes a Stats handle to your function. If aggregation is on, it also runs a thread that flushes statistics. Everything stops when the function returns. A Stats can be shared between threads.

Registering metrics

Each metric is registered once with a newStat* function, which returns a handle to update it with. Keys may contain only ASCII letters, digits, ., _ and -. Registering a key twice, or an invalid key, throws an IOError.

Sampling

Counters and timings take a sampling rate n. With samples turned on, one reading in every n is sent, tagged with the rate 1/n (|@0.100000 for n = 10) so that the server can scale it back up. Use 1 to send every reading and 0 to send none. Aggregated statistics always include every reading.

Flushing

Every flushInterval milliseconds, the aggregated data of every metric is sent and reset: counters go back to zero, timings and sets are emptied, gauges keep their value.

Usage guide

Counters

c <- newStatCounter stats "jobs.done" 1
incrementCounter c 1
incrementCounter c 5

Gauges

g <- newStatGauge stats "queue.length" 0
setGauge g 10
incrementGauge g 3
decrementGauge g 1

The aggregated value never drops below zero and saturates at maxBound. In the StatsD protocol a negative value means “decrease”, so setGauge g (-1) is seen by a server as a decrement.

Timings

t <- newStatTiming stats "db.query" 1
start <- getCurrentTime
runQuery
end <- getCurrentTime
addTiming t (round (1000 * diffUTCTime end start))

Timings are whole numbers, normally milliseconds.

Sets

s <- newStatSet stats "active_users"
newSetElement s userId

The aggregated statistic is the number of distinct elements seen in the flush interval.

Samples only, or statistics only

-- a plain StatsD client
defStatConfig {reportStats = False}

-- an aggregator that sends nothing until it flushes
defStatConfig {reportSamples = False}

Configuration

Start from defStatConfig and override the fields you need.

Field	Default	Meaning
reportStats	True	Aggregate and send statistics every flush interval
reportSamples	True	Send each (sampled) reading as it happens
namespace	""	Prefix for all metric names
prefixStats	"stats"	Prefix for aggregated statistics, after the namespace
prefixCounter	"counters"	Prefix for counter statistics
prefixTimer	"timers"	Prefix for timing statistics
prefixGauge	"gauges"	Prefix for gauge statistics
prefixSet	"sets"	Prefix for set statistics
statsdServer	"127.0.0.1"	StatsD host name or address
statsdPort	8125	StatsD UDP port
flushInterval	1000	Milliseconds between flushes; must be positive
timingPercentiles	[90, 95]	Percentiles to compute for timings, each from 1 to 99
appendNewline	False	End each line with a newline, as some servers need

withStats fails with error if a prefix or the namespace is not a valid key, the flush interval is not positive, a percentile is out of range, or the server name cannot be resolved.

Metric names

With namespace = "myapp" and the default prefixes:

What	Names
Samples	myapp.<key>
Counter statistics	myapp.stats.counters.<key>.count, .rate (per second)
Gauge statistics	myapp.stats.gauges.<key>
Set statistics	myapp.stats.sets.<key>.count
Timing statistics	myapp.stats.timers.<key>.<stat>

Timing statistics are count, count_ps (per second), std, mean, upper, lower, sum, sum_squares and median. For each percentile N, the same statistics except count_ps and std are also sent with an _N suffix, computed over the lowest N% of the timings: for example mean_90 and upper_95.

Without a namespace, names start at stats. and samples at the key.

Module overview

Module	Contents
System.Metrics.StatsD	The public API: configuration, withStats, metric types
System.Metrics.StatsD.Internal	Aggregation, the line format and its parser; may change between minor versions

Development

git clone https://github.com/jprupp/statsd-rupp.git
cd statsd-rupp

stack build                        # build the library
stack test                         # run the tests
stack haddock --no-haddock-deps    # build the API documentation

cabal build and cabal test work too.

package.yaml is the source of the package description. Change it rather than statsd-rupp.cabal, which hpack generates.

The tests are in test/Spec.hs and use hspec and QuickCheck. They check the line format against its parser, the statistics against simple reference implementations, and what each metric type sends, through a local socket pair.

Contributing

Pull requests are welcome.

• Format the code with Ormolu.
• Add tests for the behaviour you change.
• Describe the change in CHANGELOG.md, which follows Keep a Changelog. Versions follow the Haskell Package Versioning Policy.

Related projects

• haskoin-store reports its metrics with this library.

License

statsd-rupp is released under the BSD 3-Clause License.