One of the hottest topics today in the Redis eco-system is Redis Streams. Redis Streams revolutionizes the way message queues are handled in the same manner everything else is implemented in Redis: distributed, high performance, simple and with low latency.
As a result of its popularity amongst our open source community and with an intent to always be in tandem with the open source version, we have created a preview version that will allow Redis Enterprise customers to experience Redis Streams and get the feel on its performance and usability. This release is aimed at gathering feedback and while certainly stable, we do not recommend using this release on production systems.
This release is based on Redis Enterprise 5.2, and packs in many advantages compared to legacy pub-sub message queues. Redis Streams is a new data type in Redis 5.0, that provides a super fast in-memory abstraction of an append only log. The main advantages of Redis Streams are the highly efficient consumer groups, allowing groups of consumers to uniquely consume from different parts of the same stream of messages, and the blocking operations that allow a consumer to wait until a new data is added to the stream.
Like all data types, a Stream is stored in Redis, in-memory and under a single key (the equivalent of a Kafka topic). Producers can append messages to a stream, and consumers can subscribe to them (push) or read them periodically (pull). Consumer groups are also available, allowing for simplicity in scaling consumer process.
Messages contain a millisecond-resolution creation timestamp, an auto-generated sequence number (for ordering within the given millisecond), and the data itself. The data can be stored in one or more named fields, each being a Redis String.
Because all data in Redis is stored in RAM, Streams are partially compressed to save space and can be truncated based on their size or a timestamp (capped streams). In addition, Redis Streams offer a unique capability for removing single items from the middle of a stream to accommodate for privacy and regulation concerns. They will soon support Streams intersection capability as well.
Redis Streams provides both at-most and at-least-once delivery semantics. Both producers and consumers connect using the established protocol and clients.
The following table outlines a brief comparison of Redis Streams compared with more traditional Pub-Sub messaging queues:
Redis is well-known for being fast and simple, and Streams is no different in that sense. The performance of Streams operations is on par with other Redis operations. With Streams being a 1st-class citizen of the Redis server, it benefits from all Redis’ existing capabilities, including persistency for durability, replication, and clustering.
Check it out today!