Redis development for caching, queues and real-time features

Redis suits workloads that are fast, small and can tolerate its trade-offs:

• You want a cache in front of a slower store. Cached entries are copies of data kept elsewhere, so evicting them is usually safe.
• You need queues or event processing. Lists suit simple queues, and streams suit event processing with at-least-once delivery.
• A small set of keys receives most of the reads. Redis eviction policies are designed for that pattern.
• You have short-lived state with a natural lifetime, such as counters, rate-limit windows or sessions, and want expiry times and atomic operations.

Redis is not the answer to every data need:

• Your data set must fit in memory. Redis keeps its data in RAM, and you cap it with a memory limit.
• You cannot lose any data. Snapshots can lose the most recent minutes, the append-only log can lose about a second with the usual setting, and acknowledged writes can be lost in a failover. Keep a durable database as the system of record.
• Untrusted clients would connect directly. Redis is designed to be reached by trusted clients inside trusted environments, so mobile apps and browsers should go through your application.
• You need rollback. Redis transactions run commands together but do not roll back.

Redis stores strings, hashes, lists, sets, sorted sets and streams, and its documentation also lists JSON, geospatial, bitmap and probabilistic types. Keys can expire on their own. Publish and subscribe sends messages to channels with at-most-once delivery, while streams are an append-only log that persists messages and supports at-least-once delivery. Transactions and Lua scripts run commands in isolation. Client-side caching lets an application keep copies of keys that the server invalidates when they change.

For durability, Redis offers snapshots, an append-only file, both together, or neither, which is sometimes chosen for a pure cache. Replication is asynchronous by default, and Redis Cluster shards data across nodes. Access control lists limit each user to chosen commands, keys and channels, and connections can use TLS.

• Memory and eviction: with the no-eviction policy, writes return errors at the memory limit, and policies that evict only keys with expiry times behave the same when no keys have one. We choose the policy to match how the application reads, and leave memory free for replication and persistence buffers.
• Durability: the default append-only setting writes to disk every second. A primary with persistence turned off that restarts empty can wipe its replicas, so we plan restarts as well as backups.
• Key names: Redis has no namespaces, so we use a clear object-type:id pattern and keep keys short.
• Security: the default user has no password, so the server is never exposed directly. We set access control lists, use TLS, and firewall the port.
• Slow commands: Redis mostly runs commands on one thread, so a slow command or a KEYS call stalls every client. We use SCAN and watch the slow log.
• Connections: long-lived connections with pooling. Multiplexing cannot support blocking pops.

1. Decide what it is for: a cache, a queue, short-lived state or a data store, because each needs different settings
2. Set the limits first: memory limit, eviction policy and persistence, written down with the reason
3. Lock it down: access control lists, TLS and a firewall before any data goes in
4. Hand over properly: the configuration in your repository, and notes on what happens when the server restarts or fills up

Technology facts on this page are taken from the official documentation:

• Redis documentation: data store
• Redis documentation: data types
• Redis documentation: key eviction
• Redis documentation: persistence
• Redis documentation: replication
• Redis documentation: security
• Redis documentation: publish and subscribe
• Redis documentation: transactions
• Redis documentation: latency