Fast data ingest – collecting, storing and processing streaming data in large volume and velocity – is a common requirement for most Big Data use cases. Production use, resilient to data loss, requires high performance to prevent massive data backlog.
This whitepaper outlines the typical challenges encountered while designing a fast data ingest solution, and how Redis, with its ability to support millions of writes per second with sub-millisecond latency, Pub/Sub capabilities, and data structures such as Lists and Sorted Sets enables you to address these with very little effort.
The whitepaper explains three ways of achieving fast data ingest along with code samples:
- Fast data ingest with Redis Pub/Sub
- Fast data ingest with the List data structure
- Fast data ingest with Sorted Sets as a Time-series Database