memcache
A memcached client library.
https://github.com/dterei/memcache-hs#readme
| LTS Haskell 24.67: | 0.3.0.2 |
| Stackage Nightly 2026-11-19: | 0.5.0.0 |
| Latest on Hackage: | 0.5.0.0 |
memcache-0.5.0.0@sha256:4bde7c34e8af1ed3e5e4dd71d81edcf25496e5459691241de293ea5c04398ee4,5465Module documentation for 0.5.0.0
memcache: Haskell Memcached Client
A client library for a memcached cluster.
It supports the Meta Text memcached protocol and username/password authentication. Keys containing whitespace or control bytes are sent using its base64 binary-key mode. The encoded key must still fit within 250 bytes. It supports connecting to a single, or a cluster of memcached servers. When connecting to a cluster, keys are routed by ketama consistent hashing, compatible with libmemcached’s weighted ketama distribution, so a key goes to the same server as from libmemcached-based clients given the same server list. Version 0.5 changed the routing, so upgrading moves keys to different servers, once; see Upgrading from 0.4.
This library requires memcached 1.6.10 or newer. Older servers reject the
meta set command this library stores values with, without reading the value
that follows it, and then run the value’s bytes as commands. So each new
connection asks the server for its version before it is used: until that check
passes, the only requests sent are the authentication exchange, when
credentials are set, and version. Unless the server reports 1.6.10 or later,
the connection is closed and the operation fails with
ProtocolError (UnknownOp message), where the message quotes the server’s
reply, or, if the server answered with an error such as
CLIENT_ERROR unauthenticated, with the error that reply gives any request. A
refused version isn’t retried, and doesn’t mark the server as failed. The
check sees only the server the client connects to: through a proxy, such as
memcached’s own -o proxy_config mode, it sees the proxy’s version, so every
memcached behind a proxy must be 1.6.10 or newer too.
Authentication uses a memcached auth file with -Y authfile, an experimental
ASCII-protocol auth mode added in memcached 1.5.15; the old binary protocol
-S SASL mode is not supported. Enabling -Y also disables binary and UDP
protocols on the server. getMany uses quiet misses, so a server connection
failure can be indistinguishable from a cache miss.
The client operations, including getMany, use direct Meta Text encoders and
the Response type; the former request and response wrapper types are no longer
provided.
Usage
Add memcache to your package’s build-depends. Everything most programs need
is exported from Database.Memcache.Client:
{-# LANGUAGE OverloadedStrings #-}
module Main (main) where
import qualified Database.Memcache.Client as M
main :: IO ()
main = do
-- Connect to memcached on 127.0.0.1:11211 with the default options.
mc <- M.newClient [M.def] M.def
-- Store a value with flags 0 that never expires, then read it back along
-- with its flags and CAS version.
_ <- M.set mc "greeting" "hello" 0 0
hit <- M.get mc "greeting"
case hit of
Nothing -> putStrLn "miss"
Just (value, _flags, version) -> do
print value
-- Only overwrite the value if it hasn't changed since we read it.
updated <- M.cas mc "greeting" "hello again" 0 0 version
putStrLn (maybe "changed by someone else" (const "updated") updated)
-- Create the counter as 1 if it is missing, otherwise add 1 to it.
visits <- M.increment mc "visits" 1 1 0 0
print (fst <$> visits)
M.quit mc
Keys and values are strict ByteStrings. Misses are returned as Nothing,
while unexpected failures are thrown as a MemcacheError or a network
IOException. Operations that take a CAS Version skip the check when it is
0. increment and decrement create a missing key with the given initial
value, unless the expiration is maxBound. The
Database.Memcache.Client
documentation covers every operation, along with the timeouts, retries and
failover configured through Options.
Create one client when your program starts and share it: it is safe to use
from many threads, and it keeps its connections to each server open between
requests. A client created for each request opens a new connection to every
server it uses (and authenticates, if credentials are set) every time.
M.withClient closes a client’s connections when an action ends, so use it
around your whole program, a worker or a test, not around single requests.
Each server’s pool holds one connection by default (with resource-pool 0.3,
one for each capability), so concurrent requests to the same server wait for
each other; see Architecture Notes to allow more. The
wait counts towards each attempt’s optsServerTimeout. An attempt that runs
out of time while still waiting is retried like any failed attempt, but if it
was the last attempt, the operation fails with Timeout and the server isn’t
marked dead. An attempt that runs out of time afterwards, while opening,
authenticating or using its connection, counts as a server failure, however
long it waited first.
A failed operation is retried, except that a request that mustn’t be sent
twice (increment, decrement, append, prepend, add, delete, and
cas or replace with a version) isn’t sent again once it may have reached
the server: the failure is thrown, and the server is marked dead. Other
operations are retried after any failure, so the server may carry out their
request twice. Set optsServerRetries to 0 for no retries.
A SERVER_ERROR reply is a failure too, as a timeout is, unless its text
names a status, such as running out of memory. One that names a status, and
the other error replies, such as CLIENT_ERROR, are thrown at once and don’t
count against the server. memcached’s proxy sends a SERVER_ERROR when a
backend fails, so behind it, a request for a key on a failed backend marks the
whole proxy dead: at once if the request mustn’t be sent twice, and otherwise
once its retries get the same reply.
Servers and authentication
newClient takes one ServerSpec per server, and routes each key to one of
them. Credentials are set per server:
mc <- M.newClient
[ M.ServerSpec "cache1.internal" "11211" (M.Auth "user" "secret")
, M.ServerSpec "cache2.internal" "11211" (M.Auth "user" "secret")
]
M.def
parseServerSpec, from Database.Memcache.Types, reads a server from a
memcached://[user:password@]host[:port] URI instead. The host defaults to
127.0.0.1 and the port to 11211:
import Database.Memcache.Types (parseServerSpec)
spec <- either fail pure (parseServerSpec "memcached://user:[email protected]:11211")
mc <- M.newClient [spec] M.def
The server must be started with -Y <authfile>, where the auth file holds one
username:password line per user. This library connects without TLS, so
credentials and all data are sent unencrypted; only rely on them within a
trusted network.
Unix sockets
A host that begins with / is the path of a Unix socket instead, for a
memcached on the same machine. Start memcached with -s <path> to listen on
the socket, which stops it listening on the network, and with -a <mode> to
set the socket’s permissions: the default, 0700, lets only memcached’s own
user connect.
mc <- M.newClient [M.ServerSpec "/run/memcached/memcached.sock" "11211" M.NoAuth] M.def
As a URI, the path follows an empty host, and there is no port:
memcached:///run/memcached/memcached.sock, or
memcached://user:secret@/run/memcached/memcached.sock with credentials. An
@ in the path is written %40; a raw one is an error, so that a URI with one
slash too many, such as memcached:///user:[email protected], is never
taken for a socket path holding the password. The error says that credentials
go right after memcached://: such a URI needs a slash fewer, not its @
written %40, which would make it that socket path after all.
The client doesn’t use a socket’s port to connect, but the port still names
the server, along with the path, when keys are routed, so changing it moves
keys to other servers. "11211", which a URI gives, names the socket by its
path alone, as libmemcached’s memcached_server_add does, and "0" names it
path:0, as libmemcached’s memcached_server_add_unix_socket and pylibmc do.
The operating system limits the path’s length: 108 bytes on Linux and 104 on macOS, including the terminating NUL.
Amazon ElastiCache
Database.Memcache.ElastiCacheClient creates a client from an ElastiCache
configuration endpoint (the address containing .cfg). It discovers the
cluster’s nodes, then refreshes the node list in a background thread on the
given interval, in microseconds as for threadDelay:
import qualified Database.Memcache.ElastiCacheClient as EC
endpoint <- either fail pure $
EC.parseConfigurationEndpoint "memcached://mycluster.abc123.cfg.use1.cache.amazonaws.com:11211"
mc <- EC.newClient M.def (60 * 1000000) endpoint
Discovery connects to the configuration endpoint like any other server, so it fails for a cluster running a memcached older than 1.6.10.
Upgrading from 0.4
Version 0.5 requires memcached 1.6.10 or newer and refuses an older server, so upgrade any older memcached first, including any behind a proxy.
Version 0.5 routes keys with ketama consistent hashing, where 0.4 sent keys to no more than two servers at a time. With n servers, upgrading moves all but about one key in n to a different server, once: about half the keys with 2 servers, 70% with 3 and 88% with 8. Expect a burst of cache misses.
While old and new versions run side by side, as in a rolling deploy, they read and write a moved key on different servers. One can then read a stale copy of a key that the other has changed or deleted, until the copy expires. To avoid that, either switch every instance at once, at a quiet time, optionally flushing the cache afterwards, or switch in stages, deploying each step to every instance before starting the next:
- Upgrade with
optsGetServerForKey = legacyServerForKey, 0.4’s router, now deprecated, fromDatabase.Memcache.Cluster. Keys stay where they are, as long as the build uses the samehashableversion and platform as before. With-Werror, also pass-Wno-deprecations. - Add a second client for the same servers that uses
ketamaServerForKey. Whenever a key is changed or deleted, also delete it through that client, unless both clients send it to the same server. - If a server was marked failed less than your longest expiry time ago, flush the cache first. Then swap the two clients’ routers.
- Once your longest expiry time has passed since step 3 was on every instance, remove the second client. If some keys never expire, flush the cache instead of waiting, and then remove the second client.
The
Database.Memcache.Cluster
documentation has the details.
optsServerRetries changed too. It now counts the retries after an
operation’s first attempt, where 0.4 counted the attempts, so a value of 1 or
more that you set yourself makes one more attempt than in 0.4. Set it one
lower to keep the old number of attempts. The default went from 2 to 1, which
still means two attempts.
When a server’s host name resolves to several addresses, the client now tries
each in turn, where 0.4 tried only the first. If the addresses are separate
memcached instances, give each its own ServerSpec: while the first refuses
connections, new connections go to the next one that accepts, and so to another
cache.
Licensing
This library is BSD-licensed.
Tools
This library also includes a few tools for manipulating and experimenting with memcached servers.
OpGen– A load generator for memcached. Doesn’t collect timing statistics, other tools like mutilate already do that very well. This tool is useful in conjunction with mutilate.Loader– A tool to load random data of a certain size into a memcached server. Useful for priming a server for testing.
Architecture Notes
We’re relying on Data.Pool for thread safety right now. Each server has its
own pool, and grabbing a connection from it (Server.withSocket) blocks other
requests using that connection until the operation completes. getMany,
setMany and deleteMany are the exception: they pipeline Meta Text requests
to each server in batches, each batch over one pooled connection.
By default each server’s pool holds a single connection (soNumResources = 1;
with resource-pool 0.3, one for each capability), so concurrent operations
against the same server wait for each other. Raising soNumResources in the
ServerOptions set through optsServerOptions allows concurrent operations
without requiring a separate pool implementation.
Either way, a pool is fine for now.
Other clients
Get involved!
We are happy to receive bug reports, fixes, documentation enhancements, and other improvements.
Please report bugs via the github issue tracker, and security issues as described in the security policy.
Master git repository:
git clone https://github.com/dterei/memcache-hs.git
Development
The package builds with Stack or cabal, on GHC 9.2 or newer:
stack build
stack test
cabal build
cabal test all
stack.yaml uses Stackage LTS 22.14 (GHC 9.6). The oldest and newest snapshots
CI tests against have their own files, for example
stack --stack-yaml stack-lts-20.26.yaml test for GHC 9.2.
There are two test suites:
specholds the unit and property tests, which run against an in-process mock server and need nothing installed.integrationstarts its ownmemcachedservers on free local ports and tests against them, so it needs memcached 1.6.13 or newer on yourPATH. Without it, the suite reports a single pending test instead of failing. With 1.6.10 to 1.6.12, which the library accepts, two of its tests fail, as those servers answer anappendorprependwith the version the request carried, or 0, rather than the new one. memcached refuses to run as root without-u, so run the suite as an ordinary user.
CI lints with HLint 3.10, so please
check that hlint . prints No hints before opening a pull request. For any
user-visible change, propose a line for CHANGELOG.md in the
pull request’s description; don’t edit the file, as the next release’s pull
request adds the lines.
CI runs both build tools on every pull request. The CI workflow builds and
tests with Stack against each stack*.yaml snapshot, and runs HLint and
weeder. The Cabal workflow builds and tests with cabal on GHC 9.2 to 9.14,
builds the library against the oldest dependency versions its bounds allow,
and builds and tests the source distribution; it also runs every Monday, so a
new dependency release that breaks the build shows up without a push. Both
install memcached, so the integration suite runs in full.
Authors
This library is written and maintained by David Terei ([email protected]).
Contributions have been made by the following great people:
- Alfredo Di Napoli ([email protected])
- Amit Levy
- Chris Martin
- drlkf
- Joel McCracken
- Steven Leiva
Changes
0.5.0.0 - November 18th, 2026
This release changes how a cluster routes keys to its servers, so upgrading
moves keys to other servers, once; see “Upgrading from 0.4” in the README. It
also changes what optsServerRetries counts, the types of two lower-level
functions, and some return values and exceptions, so read the breaking
changes first. Two security fixes stop memcached from running the bytes of a
value being stored as commands, and memcached 1.6.10 or later is now
required; see Security.
Security
Both fixes are also in 0.4.2.1.
set,add,replace,cas,append,prependandsetManyreject a value larger than 1GiB, memcached’s largest possible item size, withOpError ErrValueTooLargebefore sending anything. A value of about 2GiB or more (longer than 2147483645 bytes) previously made memcached run the value’s bytes as commands, so storing untrusted data could run commands on the server, and a value over 1GiB could time out, be sent again and mark the server dead. Such a value no longer reaches the server, so it no longer makes memcached delete the key’s existing value, as memcached does whenever it rejects a store as too large; a value within 1GiB but over the server’s own limit (-I, 1MiB by default) still does. Every operation that sends a value now evaluates it first, before its timeout starts: an exception it throws is no longer retried and no longer marks the server dead, and time spent computing it no longer counts towardsoptsServerTimeout.setManyevaluates every value before it sends anything, so a list of lazily computed values is all in memory at once, and one value over the limit stores none of the pairs. The request encoders inDatabase.Memcache.Socketstill encode a value of any length; don’t send one longer than 1GiB.- memcached 1.6.10 or later is now required (previously documented as
1.6.0). memcached before 1.6.10 rejects the meta set command this library
stores values with, without reading the value that follows it, and then
runs the value’s bytes as commands, so storing untrusted data could run
commands on the server. Each new connection now asks the server for its
version before it is used: until that check passes, the only requests sent
are the authentication exchange, when credentials are set, and
version. A server older than 1.6.10, or any other reply that is neither an error nor a readable version, is refused withProtocolError (UnknownOp ...), whose message quotes the reply; the connection is closed, and the refusal is not retried and doesn’t mark the server dead. An error reply, such asCLIENT_ERROR unauthenticated, throws what it throws for any request, and so does a reply that can’t be parsed, such as an empty line or an overlong one, which counts against the server. This covers connections over TCP and Unix sockets, and ElastiCache discovery, which now fails for a cluster running a memcached older than 1.6.10. Through a proxy, the check sees only the proxy’s version, so every memcached behind it must be 1.6.10 or later too. Each new connection takes one more round trip, which counts towardsoptsServerTimeout. Connections made directly withDatabase.Memcache.Socket.newConnectionare not checked; sendstoreRequest,setManyRequestandmodifyRequestonly to 1.6.10 or later.
Breaking changes
- memcached 1.6.10 or later is now required (see Security): a new
connection to an older server is refused, so every operation that needs
one fails with
ProtocolError (UnknownOp ...). - Clusters route keys with ketama consistent hashing, compatible with
libmemcached (see New):
ketamaServerForKeyis the new defaultoptsGetServerForKey. 0.4’s router sent keys to no more than two servers at a time, and the server it chose for a key could change with thehashableversion. With n servers, upgrading moves all but about one key in n to a different server, once: about half the keys with 2 servers, 70% with 3 and 88% with 8. A single server is unaffected. While old and new versions run side by side, as in a rolling deploy, they read and write a moved key on different servers; “Upgrading from 0.4”, in the README and inDatabase.Memcache.Cluster, gives two ways to switch. One of them useslegacyServerForKey, 0.4’s router, whichDatabase.Memcache.Clusternow exports, deprecated, for one release; it will be removed in the next major version. A key whose server is marked failed now goes to the next live server on the ring, and other keys stay where they are. - With keys spread over every server,
getManysends each server its keys in batches of up to 256, one round trip per batch and one server after another, so it can take longer than before with three or more servers. It still returns its results grouped by server, so their order changes too. - Servers keep the order they are configured or discovered in, instead of
being sorted by
sid:getServersandstatslist them in that order,flushgoes through them in it, andversionasks the first live one. optsServerRetriesnow counts retries, as its documentation said, rather than attempts: an operation is tried up tooptsServerRetries + 1times, and once for 0 or a negative value. So a value of 1 or more that you set yourself makes one more attempt than in 0.4; set it one lower to keep the old number of attempts.minBound, which used to retry practically without end, now gives one attempt. The default drops from 2 to 1, so by default an operation is still tried twice.increment,decrement,append,prepend,add,delete, andcasandreplacewith a version other than 0 are no longer sent again once their request may have reached the server, so they can’t take effect twice or report failure because their first request succeeded. Once the client has begun to send the request, a failure such as a timeout, a network error, a dropped connection, a reply that can’t be read or aSERVER_ERRORwhose text names no status is thrown as it is, whatever retries are left, and marks the server dead. A failure before sending, such as a refused connection or a timeout while waiting for a pooled connection, is still retried. Before sending one of these requests on a pooled connection, the client checks, without waiting, that nothing is left to read on it and that the server hasn’t closed or reset it, as memcached closes idle connections when it restarts or after-o idle_timeout. If the check fails, the client retries the request on a new connection when retries are left; on Windows, the check only sees what has already been read, so it can’t see a closed connection. Other operations are still retried after any failure that counts against the server, so the server may carry out their request twice.- When a server’s host name resolves to several addresses, the client now
tries each in turn, not just the first (see Fixes). So when the first
address refuses and a later one doesn’t answer, an attempt throws
ClientError Timeoutonce itsoptsServerTimeoutruns out, instead of the first address’sIOExceptionat once. If the addresses are separate memcached instances, give each its ownServerSpec: while the first refuses connections, new connections go to the next one that accepts, and so to another cache. appendandprependreturnNothingwhen a non-zero version doesn’t match, ascasandreplacedo, instead of throwingOpError ErrKeyExists.flushstops at the first server whose reply it rejects (see Fixes), leaving the servers after it unflushed, where 0.4 checked such a reply only after sendingflush_allto every server.- memcached’s capitalised out-of-memory errors are thrown as
OpError ErrOutOfMemory, like the lowercase ones, instead ofProtocolError ServerError(see Fixes), so the request is no longer sent again, and they don’t count towards marking the server dead. Code that matchedServerErrorfor them should matchErrOutOfMemoryinstead. - Numbers in replies (value lengths, CAS versions, flags and counter values)
must be plain decimal digits that fit their type, as memcached writes them
(see Fixes), where 0.4 also read forms such as
(5)and0x10, and wrapped numbers around to fit. So from a server or proxy that writes them otherwise, a value length now throwsBadLength, and a CAS version, flags or counter value reads as 0. AVAreply without a value length, which used to read as a hit with an empty value, throwsBadLengthtoo. displayExceptionfor aMemcacheErrorgives a one-line message, such asmemcached operation failed: key not found, where it used to give the same text asshow, such asOpError ErrKeyNotFound; GHC 9.10 and later also print it for an uncaught exception. Any detail, such as text from the server, is cut to its first 200 characters, with backslashes and characters outside printable ASCII escaped.showis unchanged, so code that parsesdisplayException’s text should match on the constructors, or useshow, instead.ElastiCacheClient.newClientthrowsClientError Timeoutunless its first discovery, connecting included, finishes withinoptsServerTimeout. It had no time limit, so with the default of 750ms, a slow DNS lookup or connection can now make it fail. AnERROR,CLIENT_ERRORorSERVER_ERRORreply to the request for the cluster configuration, such as theERRORthat plain memcached gives, throws aStringExceptionfromUnliftIO.Exceptionquoting the line, escaped and cut to 200 characters, wherenewClientused to block forever, or throwProtocolError UnexpectedEOFif the endpoint then closed the connection. Discovery now callsoptsServerSpecsToServerswith one node at a time.statsthrowsBadLengthfor a server’s reply of more than 262,144 entries, or of more than 16MiB counting each line as it was sent, such as the reply tostats connson a server with more than about 65,500 open connections.statsretries it, as it does other server failures, and it counts towards marking the server dead.recvRawUntilthrowsBadLengthonce it has read more than 1MiB without its delimiter, so ElastiCache discovery does too for a version line or configuration reply that long. No option raises these limits.parseServerSpec, and soparseConfigurationEndpoint, percent-decodes the username and password of amemcached://URI to the bytes they encode, somemcached://user:p%[email protected]has the passwordp@ss, and returnsLeftif the decoded credentials failvalidateCredentials.validateCredentials, which authentication also uses, now allows:in passwords, as memcached’s auth file splits each line at its first:, and rejects NUL in usernames and passwords, so connecting with a NUL in either now throwsClientError InvalidAuthenticationbefore anything is sent. An IPv6 address in brackets, as inmemcached://[::1]:11211, now gives the host::1, without the brackets, and the%25that introduces a zone ID is decoded, so[fe80::1%25eth0]givesfe80::1%eth0. A path after an empty host, other than slashes alone, as inmemcached:///run/memcached.sock, is now a Unix socket (see New), where 0.4 ignored it and gave127.0.0.1; with a port, or with%00or a raw@in the path, such a URI now returnsLeft.showfor anAuthentication, and so for aServerSpec, prints the password as<redacted>, and the error thatparseServerSpecandparseConfigurationEndpointgive for an invalid URI no longer quotes it, so that passwords stay out of logs.getManyRequestinDatabase.Memcache.Socket, and the request builders thatkeyedBatchOpinDatabase.Memcache.Clustertakes, have a newIntargument: an opaque, which the request must send unchanged, in decimal, as itsOflag, so that its reply can be matched to it (see Fixes).getManyRequestsends it in place of thekflag, so its replies no longer echo the key.Database.Memcache.Clientexports more names (see New), such asKey,Version,ServeranddefaultOptions, andDatabase.Memcache.Cluster,Database.Memcache.Server,Database.Memcache.SocketandDatabase.Memcache.Typesexport new names too. A new name can clash with another in a module that imports one of these modules unqualified without listing the names it imports, even one that hides some withhiding.
New
- Unix sockets, contributed by drlkf: a
ServerSpecwhose host begins with/connects to the Unix socket at that path, for a memcached on the same machine.parseServerSpecreads one from a URI with a path, other than slashes alone, after an empty host, such asmemcached:///run/memcached.sock, ormemcached://user:secret@/run/memcached.sockwith credentials. The path is percent-decoded to bytes. A port or%00is an error, and so is an@not written%40, so that a URI with one slash too many, such asmemcached:///user:[email protected], isn’t taken for a socket path holding the password. The client doesn’t use a socket’s port to connect, but the port still names the server when keys are routed:"11211", which a URI gives, names the socket by its path alone, as libmemcached’smemcached_server_adddoes, and"0"names itpath:0, as libmemcached’smemcached_server_add_unix_socketand pylibmc do. ElastiCache discovery fails, rather than connect to a socket on this machine, if the endpoint lists a node whose host is a path. - Keys go to the same server as with other clients that use libmemcached’s
weighted ketama distribution, given the same host strings and ports in the
same order and equal weights: libmemcached with
MEMCACHED_BEHAVIOR_KETAMA_WEIGHTED, php-memcached withOPT_LIBKETAMA_COMPATIBLE, pylibmc with theketama_weightedbehavior, and twemproxy withhash: md5anddistribution: ketamaand its servers listed without node names. In the rare case that two servers have a point with the same value, libmemcached before 1.1, which php-memcached and pylibmc may be built with, and twemproxy may choose differently for the keys just below it. The new moduleDatabase.Memcache.Ketama(Ring,ring,serverName,keyHash,owner,ownersandringPoints) works out which server a key belongs to, and builds rings with weights, which clusters don’t use.Database.Memcache.Clusterexports the default router asketamaServerForKey, andisAlivefor a custom router that skips servers marked failed, andDatabase.Memcache.ServerexportsserverHostandserverPort. withClientcreates a client, runs an action with it, and closes the client’s connections withquitwhen the action returns or throws.setManyanddeleteManystore and delete many keys, pipelining the requests to each server asgetManydoes.setManyreturns the new version of each key, grouped by server;deleteManyreturns once each key has been deleted or found missing, without saying which. Their request encoders,setManyRequestanddeleteManyRequest, are inDatabase.Memcache.Socket.defaultOptions,defaultServerOptionsanddefaultServerSpecname the values thatdefgives. They are exported fromDatabase.Memcache.Cluster,Database.Memcache.ServerandDatabase.Memcache.Typesrespectively, and all three fromDatabase.Memcache.Client.Database.Memcache.Clientalso exportsparseServerSpec,Retries,ServerOptions(..), the abstractServertype, andKey,Value,Initial,Delta,Expiration,FlagsandVersion, the types its own functions use.- For lower-level use:
closeAllinDatabase.Memcache.Servercloses every connection to a server, asquitdoes for each server of a client;keyedOpExpecting,anyOpExpectingandallOpExpectinginDatabase.Memcache.Clusterreject a reply whose code is none of those given, closing its connection, andkeyedOpExpectingOncealso doesn’t send its request again once it may have reached the server; andcheckIdleinDatabase.Memcache.Socketchecks, without waiting, that nothing is left to read on a connection and that the server hasn’t closed it (on Windows, it only checks what has already been read).
Fixes
- Fix
getManydropping keys, or returning one key’s value as another’s, when it couldn’t match the key that memcached echoed in a reply: a key containing byte 0xa0, such as UTF-8"voilà"; a key with a space or control byte once an increment has created it or made its value longer, or an append or prepend has rebuilt it; and a plain key that another client stored with thebflag. When such a key held a line break, the echo also ended the reply line inside the key, sogetManycould return part of the key as another key’s value, fail withBadLengthand mark a healthy server dead, or end the batch early and leave the rest of its replies on the pooled connection, to be read as the replies to later requests.getMany‘s requests no longer ask memcached to echo the key, and batch replies are now matched to their requests by memcached’s opaque (O) flag instead. A reply that matches no request in its batch, which used to be dropped, or that comes out of order, now throwsUnexpectedResponseand closes the connection; like any server failure, the batch is then sent again on a new connection if retries are left, and the failure counts towards marking the server dead. - Reply lines are split into tokens on spaces alone, as memcached writes
them, so a token that
recvResponsereturns keeps any other whitespace byte, such as a tab or 0xa0, and so dostatsvalues. - In single-key operations,
version,flushand authentication, a reply with an unknown code, or one the request can’t get, as when an earlier request’s reply is still on the connection, now closes the connection instead of returning it to the pool out of step, where later requests would read each other’s replies. It throwsUnknownStatusorUnexpectedResponse, where some such replies used to be taken as an answer: as success by a store,increment,decrement,delete,flushor authentication, and as a miss byget,gatandtouchfor anNF, and bycas,replace,append,prepend,increment,decrementanddeletefor anEN.flushnow stops at the first server whose reply it rejects, leaving the servers after it unflushed, where it used to check such a reply only after sendingflush_allto every server (see Breaking changes). IngetMany,setManyanddeleteMany, a reply with an unknown code throwsUnknownStatusand closes the connection; the batch is then sent again on a new connection if retries are left, and the failure counts towards marking the server dead.getManyused to skip such a reply, or throw for it without sending the batch again or counting the failure. - memcached’s capitalised out-of-memory errors,
Out of memory allocating new item(fromincrementanddecrement) andOut of memory during read, are reported asOpError ErrOutOfMemory, like the lowercase ones (see Breaking changes). They were thrown asProtocolError ServerError, so the request was sent again and a full but healthy server was marked dead. - An attempt that runs out of
optsServerTimeoutwhile still waiting for a free pooled connection is no longer blamed on the server: it is retried as before, but if it was the last attempt, the operation throwsClientError Timeoutwithout marking the server dead. Running out of time later, while connecting, authenticating or using the connection, still counts against the server. A connection is now opened, and authenticated, once its request has a place in the pool, so a failed connect no longer leaves requests queued for the pool blocked until their own timeout, as it did with resource-pool 0.3.0.0 to 0.5.0.0. AnallOp'operation must make its requests through theServerit is given for its timeouts to count against the server. - A failed attempt that counts against the server now closes the server’s
idle pooled connections before the operation is retried or the server is
marked dead, as a timeout already did. So with
soNumResourcesof 2 or more, a memcached restart or memcached’sidle_timeoutno longer sends each retry to another closed connection, failing the operation and marking a healthy server dead. With resource-pool 0.2, an asynchronous exception, such as acancel, that arrived while those connections were being closed could be lost, so that the request carried on as if it hadn’t arrived, or cut the closing short, leaving some of them open; with later versions, it could still cut the closing short between two of the pool’s stripes.closeinDatabase.Memcache.Server, which closes them, now holds such an exception back until every idle connection is closed, ascloseAlldoes, so they are all closed first, and the exception then ends the request. - Retries no longer run with asynchronous exceptions masked. Like the first
attempt, each runs in the caller’s masking state, so
optsServerTimeoutand the caller’s own asynchronous exceptions, such as acancel, can interrupt a retry that keeps working without blocking. - When a server’s host name resolves to several addresses, the client tries
each in turn, within the attempt’s
optsServerTimeout, and keeps the first connection that succeeds until it is closed; it used to try only the first address. If every address fails, the error is still the first address’s. An address that doesn’t answer uses up the attempt’s time, so when the first address refuses and a later one doesn’t answer, an attempt now throwsClientError Timeoutwhere it used to throw the first address’s error at once (see Breaking changes). A failed authentication, or a server version the client refuses, is not retried against another address. If the addresses are separate memcached instances, then while the first refuses connections, new connections go to the next one that accepts, and so to another cache, and stay with it until they are closed; give each instance its ownServerSpec. quitno longer sends memcached’squitcommand, so it never connects to a server, and it never throws, even when every server is marked dead. It closes the client’s connections to every server, including those marked dead: idle ones at once, and each one in use as soon as its request finishes. An asynchronous exception, such as acancel, that arrives while it closes them is held back until every server’s idle connections are closed. A request that starts later, or is still waiting for a connection, or opening one, whenquitruns, opens a new one, which stays open as usual.ElastiCacheClient.newClientno longer stops refreshing the node list after the first refresh that fails. A refresh that throws, gets an error reply or runs out of time keeps the current servers, and discovery tries again at the next interval. Its error is dropped, where the first refresh that threw used to end discovery and print its exception on stderr. Each refresh has the first discovery’s time limit,optsServerTimeout, and connects to the endpoint afresh, looking its name up again, as a connection kept from an earlier refresh could lead to a node that ElastiCache has since replaced. A refresh also keeps theServer, and so the pooled connections and failure state, of each node still listed, where it used to replace every server and leave the old connections open. It closes the connections of a node no longer listed, or listed with another IP address, as when ElastiCache replaces it: idle ones at once, and one in use as soon as its request finishes, so that requests connect again, looking the node’s name up afresh.- ElastiCache discovery checks the endpoint’s replies more strictly. A
configuration that lists more than 1,000 nodes, the same host and port
twice, or a node that isn’t a plain host and port now fails discovery, as
a reply that can’t be parsed does:
ElastiCacheClient.newClientthrows aStringException, and a refresh keeps the current servers. A node isn’t a plain host and port if its host or port is empty, if its host is an IPv6 address in brackets, or if its host or port has a character, such as/,?or@, that would change the host or port or add credentials, ash/xon port 22122 used to givehon port 11211; an empty host used to give127.0.0.1, and an empty port failed only at the first request. A version of more than 64 bytes in the endpoint’s version reply fails discovery too, where parsing it took time that grew faster than the square of its length. Discovery’s errors escape the text they quote from the endpoint and cut it to 200 characters, asdisplayExceptiondoes for aMemcacheError. The refresh thread runs with asynchronous exceptions unmasked, even whennewClientis called with them masked, as in the acquire action of abracket, so that each refresh’s time limit can interrupt it. - In
getMany,setManyanddeleteMany, a bareEN(a miss reply without an opaque) now counts as the reply to the next request. One after the reply to the batch’s last request throwsUnexpectedResponse, and so does a reply to a request that a bareENwas counted for, or to an earlier one, so a server that sends bareENs without end can no longer keep a batch reading. statsnow closes each connection it reads a reply from, instead of returning it to the pool, so a later request to each server opens a new one in its place. A key holding a line break, whichstats cachedumpcan list as it is stored, could end the reply early asstatsread it, and the rest of the reply was then read as the reply to a later request on the same connection. The bytes after such a line break are still read as lines of their own, so they can add entries or end the reply early, butstatsno longer sends its request again or marks the server dead when they make an empty line, or aSERVER_ERRORline naming no status: after a reply’s first line, it throws the error of any empty or error line once every server has answered.- Numbers in replies (value lengths, CAS versions, flags and counter values)
are read only as plain decimal digits that fit their type, as memcached
writes them (see Breaking changes), where they used to be read leniently,
as with
(5)or0x10, or wrapped around to fit. A value length in any other form, or aVAreply without one, throwsBadLength; such aVAreply used to read as a hit with an empty value. A CAS version, flags or counter value in any other form reads as 0, as a missing one does, so such a CAS skips the check when passed back to an operation that takes aVersion.
Documentation and development
- The README and
Database.Memcache.Clientexplain why to create one client and share it, and say more about timeouts, retries and failover, including which requests are never sent twice.Database.Memcache.Clusterdocuments routing, failover and upgrading from 0.4, which the README sums up, and the README has a section on Unix sockets. - The README and the docs say which error replies count against a server: a
SERVER_ERRORwhose text names no status, such as memcached’s proxy sends when a backend fails, counts against the server as a timeout does, whileERROR,CLIENT_ERRORand aSERVER_ERRORthat names a status are thrown at once without counting against it. So behind memcached’s proxy, a request for a key on a failed backend marks the whole proxy dead: at once if it mustn’t be sent twice, and otherwise once its retries get the same reply. The docs ofTimeoutandNoServersReadynow say when each is thrown. - The docs say what each resource-pool version does with
soNumResourcesandsoNumStripes. With 0.3, which the bounds allow, a pool has a stripe for each capability, whateversoNumStripesis, so a pool with the default options holds one connection for each capability, and a thread takes connections only from its own capability’s stripe. With 0.4 or later,soNumResourcesis split between the stripes, and creating a client throws anErrorCallif it is smaller thansoNumStripes. - New dependency:
cryptohash-md5, for the MD5 hashes of the ketama ring. Thespectest suite now also depends ondirectory,hashable,timeandunix(except on Windows), andintegrationonunliftio. - Expand the test suite. Routing is checked against the servers that
libmemcached-awesome 1.1.4 chooses for the same keys, including its own
test fixture, and
legacyServerForKeyagainst a copy of 0.4’s router. Mock tests cover the new operations, replies out of step, retries, failover and pool timeouts, closing connections, connecting to each resolved address, Unix sockets, and ElastiCache discovery and refreshes. The integration tests also run against memcached restarted, paused withSIGSTOP, listening on a Unix socket and out of memory, and test ElastiCache discovery against plain memcached. - The
integrationtest suite needs memcached 1.6.13 or later. With 1.6.10 to 1.6.12, which the library accepts, two of its tests fail, as those servers answer anappendorprependwith the version the request carried, or 0, rather than the new one. The documentation ofappendandprependsays so, as passing that 0 back as a version skips the check. - The
parserbenchmark also times hashing a key, building a ring, finding a key’s server on it and routing a key through a cluster. - CI runs on the
ubuntu-26.04runner image, named in the workflows instead ofubuntu-latest, installs the gold linker that older GHCs need there, keeps its caches per OS release, and usesfreckle/stack-action5.7.33.
0.4.2.1 - November 18th, 2026
No API changes from 0.4.2.0.
set,add,replace,cas,appendandprependreject a value larger than 1GiB, memcached’s largest possible item size, withOpError ErrValueTooLargebefore sending anything. A value of about 2GiB or more (longer than 2147483645 bytes) previously made memcached run the value’s bytes as commands, so storing untrusted data could run commands on the server, and a value over 1GiB could time out, be sent again and mark the server dead. Such a value no longer reaches the server, so it no longer makes memcached delete the key’s existing value, as memcached does whenever it rejects a store as too large; a value within 1GiB but over the server’s own limit (-I, 1MiB by default) still does. Every operation that sends a value now evaluates it first, before its timeout starts: an exception it throws is no longer retried and no longer marks the server dead, and time spent computing it no longer counts towardsoptsServerTimeout. The request encoders inDatabase.Memcache.Socketstill encode a value of any length; don’t send one longer than 1GiB.- memcached 1.6.10 or later is now required (previously documented as
1.6.0). memcached before 1.6.10 rejects the meta set command this library
stores values with, without reading the value that follows it, and then
runs the value’s bytes as commands, so storing untrusted data could run
commands on the server. Each new connection now asks the server for its
version before it is used: until that check passes, the only requests sent
are the authentication exchange, when credentials are set, and
version. A server older than 1.6.10, or any other reply that is neither an error nor a readable version, is refused withProtocolError (UnknownOp ...), whose message quotes the reply; the connection is closed, and the refusal is not retried and doesn’t mark the server dead. An error reply, such asCLIENT_ERROR unauthenticated, throws what it throws for any request, and so does a reply that can’t be parsed, such as an empty line or an overlong one, which counts against the server. This covers ElastiCache discovery, which now fails for a cluster running a memcached older than 1.6.10. Through a proxy, the check sees only the proxy’s version, so every memcached behind it must be 1.6.10 or later too. Each new connection that an operation opens takes one more round trip, which counts towardsoptsServerTimeout. Connections made directly withDatabase.Memcache.Socket.newConnectionare not checked; sendstoreRequestandmodifyRequestonly to 1.6.10 or later.
0.4.2.0 - October 20th, 2026
No API changes from 0.4.0.0. This is the first Hackage release since 0.4.0.0, so it also includes the 0.4.1.0 changes below.
- Fix
getManyreturning one key’s value under another key’s name when a plain key equals the other’s base64 form, such as"IA=="and" ". - Requests that carry a value or a number are now encoded into a single
buffer instead of a chain of appends that copied the value once per piece.
Encoding a 64KiB
settakes about 4µs instead of about 57µs. - Document that a
Versionof0skips the CAS check, whatcas,replace,append,prependanddeletedo for a missing key or a version mismatch, and whenincrementanddecrementcreate a missing key. Add the missingOverloadedStringsto the usage examples inDatabase.Memcache.Clientand the package description, add usage and development instructions to the README, and replace the templateSECURITY.mdwith a full security policy.
0.4.1.0 - October 19th, 2026
Not uploaded to Hackage; these changes first ship there in 0.4.2.0. No API or behaviour changes from 0.4.0.0.
- Reading a large value no longer allocates memory quadratic in its size. Each receive is now capped at 64KiB, so fetching a 32MiB value allocates about 67MB rather than about 1.7GB.
- Drop the dependency on
errors(and, through it,safeandtransformers-compat), which was only used for a one-line helper. - Replace the hpack
package.yamlwith a hand-writtenmemcache.cabal(cabal-version: 3.0) that lists the exposed modules explicitly and gives each component only the dependencies it uses. RemoveSetup.hs, which only did whatbuild-type: Simpledoes anyway. The license field is now the SPDX identifierBSD-3-Clause; the license itself is unchanged. - Declare support for GHC 9.2 to 9.14 in
tested-with. - Fix the
parserbenchmark, which no longer compiled, and extend it to cover key encoding, more requests, and reading CAS, flags and numbers out of responses. - Expand the package description. Hackage already shows it, as a revision of 0.4.0.0.
- Document the
InvalidAuthenticationandKeyTooLongconstructors ofClientError, and fix typos in the Haddock documentation. - Greatly expand the test suite. Unit tests against mock servers now cover
every request encoder and client operation, retries and failover,
authentication,
parseServerSpecand ElastiCache discovery, and the integration tests also run against memcached in-Yauth-file mode and against a three-server cluster. - CI builds and tests with cabal on GHC 9.2 to 9.14 and from the source distribution, builds the library against the oldest dependency versions the bounds allow, and runs weekly, as well as building with Stack and running HLint and weeder.
0.4.0.0 - October 19th, 2026
- Replace the deprecated binary protocol with buffered Meta Text requests.
- Add base64 binary-key support,
getMany, hspec tests, andAuth. - Remove binary protocol and builder dependencies.
- Replace request/response wrapper types with direct Meta Text encoders and
Response. - Authentication now uses the server’s experimental
-Y authfilemode (introduced in memcached 1.5.15) instead of binary-SSASL. encodeKeynow returns a validatedWireKey; request builders and the low-levelkeyedOpAPIs accept encoded keys to avoid partial validation and repeated encoding.- Requires memcached 1.6.0 or newer for the meta commands.
statsreturns[(Server, StatResults)]rather than[(Server, Maybe StatResults)]; theNothingcase was unreachable. Stat values containing spaces are no longer truncated, and stats arguments may now contain spaces (stats cachedump 1 100).StatuslosesErrInvalidArgsandErrUnknownCommand, which have no Meta Text equivalent and now surface asProtocolError.ErrValueTooLarge,ErrOutOfMemoryandErrValueNonNumericare recovered from the server’s error text so they keep being reported asOpError.- A command the server rejects (
ERROR,CLIENT_ERROR) or answers with an operation status is no longer retried, and no longer counts towards marking a server dead. getManywrites each batch in a single bounded send, so a large key list cannot deadlock against the server’s replies.- Response lines are capped at 8KiB, bounding memory use on a desynchronised connection.
- Authentication first checks that the server is in auth-file mode and
refuses to send credentials otherwise, so they are never stored as a plain
authitem. Credentials with a forbidden delimiter, or a server that isn’t in auth-file mode, raiseClientError InvalidAuthentication, and credentials the server rejects raiseOpError TextAuthFail. Neither is retried or marks the server dead. - Empty keys, and keys longer than 250 bytes, are rejected with
ClientError KeyTooLongbefore anything is sent. appendandprependon a missing key returnNothinginstead of throwingOpError ErrItemNotStored.statsaccepts replies made of lines other thanSTAT, such as those fromstats resetandstats cachedump.Database.Memcache.Socketsends and receives on a bufferedConnectionwithsendRawandrecvResponse, replacingsendandrecv.- ElastiCache auto discovery no longer holds the server list lock while waiting for the next refresh.
- Exception helpers now come from
unliftio, which is a new dependency. - Library dependencies declare lower bounds, set to the oldest versions CI builds against (GHC 9.2, Stackage LTS 20.26).
- Depend on
data-defaultinstead of the deprecateddata-default-class, which is now a compatibility shim overdata-default0.8. - Add
Database.Memcache.ElastiCacheClient, which creates a client from an Amazon ElastiCache configuration endpoint and refreshes its node list on an interval. - Add
parseServerSpec, which reads aServerSpecfrom amemcached://URI. OptionsgainsoptsServerOptions, whoseServerOptionsset the size, stripes and keep-alive of each server’s connection pool, andoptsServerSpecsToServers, which controls how servers are created.Database.Memcache.ServerexportsnewServerDefaultin place ofnewServer.
0.3.0.2 - March 27th, 2024
- Make the key hashing algorithm configurable by clients – that is, let clients provide a function mapping a key to a server.
0.3.0.1 - January 17th, 2021
- No changes, needed a new release to fix a Hackage upload issue.
0.3.0.0 - January 17th, 2021
- Bump package dependencies for newer GHC/back/network.
- Update code to work with newer dependencies.
0.2.0.1 - November 2nd, 2016
- Fix compatibility with latest
data-default-class - Add new ReqRaw type for external clients to implement custom requests. Quite a hack right now, so behind a WARNING pragma.
0.2.0.0 - May 27th, 2016
- Big design change to reduce code duplication (
Protocolmodule gone). - Remove
Optionstype - just fixed configuration for now. - Design change also allows proper retry handling on operation failure - we retry an operation against the same server, but after N consecutive failures, we mark the server as dead and don’t try using it again until M seconds has passed.
- Simplify exception hierarchy - just one type
MemcacheErrornow for exceptions. - Remove
defaultOptionsanddefaultServerSpec, will revisit usefulness. - Remove many
Typeableinstances. - Support better testing with a mock Memcached server.
- Fix bug in socket handling - detected EOF properly.
- Greatly improve documentation.
- Use
data-default-classfor defaults of servers and options.
0.1.0.1 - February 26th, 2016
- Consistent usage of ‘memcached’ instead of ‘memcache’.
- Document
Database.Memcache.Client. - Add inline pragmas in appropriate places.
- Fix bug handling fragmented IP packets (Alfredo Di Napoli).
0.1.0.0 - May 18th, 2015
- First proper release (although still lots of rough edges!).
- Filled out
Data.Memcache.Clientto a complete API. - Integrated cluster and authentication handling.
- Better error and exception handling.
- Fix compilation under GHC 7.10.
0.0.1 - May 5th, 2015
- Initial (incomplete) support for a cluster of memcached servers.
- Fixed compilation under GHC 7.4.
0.0.0 - August 23rd, 2013
- Initial release. Support for a single server.