Redis Enterprise Flash
Redis Enterprise Flash (RF) offers users of Redis Enterprise Pack and Redis Enterprise Cloud Private the unique ability to have very large Redis database but at significant cost savings. Where standard Redis databases must all be in RAM, Redis Enterprise Flash enables your Redis databases to span both RAM and dedicated flash memory (SSD). Whilst keys are always stored in RAM, RF intelligently manages the location of their values (RAM vs Flash) in the database via a LRU-based (least-recently-used) mechanism. Hot values will be in RAM, but infrequently used, or warm values, will be ejected to flash memory. This enables you to have much larger datasets with RAM-like latency and performance, but at dramatically lower cost than an all-RAM database.
By using Redis Enterprise Flash to distribute the data between RAM and flash memory, which is much cheaper than RAM, you can lower your TCO and better utilize hardware, hypervisor, and cloud resources. In many cases, Redis Enterprise Flash can cut resource costs by over 70% when compared to an all-RAM Redis Enterprise Pack deployment.
Unlike standard Redis Enterprise Pack installations, implementing Redis Enterprise Flash requires pre-planning around memory and overall sizing. There are a few critical recommendations
- The flash memory should be local to the server/VM/instance/container as opposed to network attached.
- The flash memory should be dedicated to RF and not shared with other parts of the database, (e.g. durability, binaries, etc.).
- The flash memory should be NVMe based for best performance.
For more information on Ephemeral and Persistent Storage in Redis Enterprise Pack, please go here.
When running on a cloud environment, the flash memory for Redis Enterprise Flash should be on the ephemeral SSDs of the cloud instance and persistent database storage should be network attached, e.g AWS EBS. For AWS, we specifically recommend “Storage Optimized I3 – High I/O Instances” because of the performance of NVMe for flash memory.
When running RF on-premise, it is best to use local internal flash memory in each server (preferably NVMe SSDs for their exceptional performance). The Redis Enterprise Pack database persistent and ephemeral storage can be on different disks, either local or attached.
When you begin planning the deployment of Redis Enterprise Flash in Production, we recommend working closely with the Redis Labs technical team for sizing and performance tuning.
Tunable RAM to Flash Ratio
You can easily configure or tune the ratio of RAM-to-Flash for each database independently, optimizing performance for your specific use case. This is an online operation requiring no downtime for your database. Think of this like a gas pedal in a car, the database speeds up as you give it more gas (RAM). We recommend you keep at least 20% of all values in RAM.
Working Set Management
Of your dataset, perhaps there is a subset of highly active objects considered the application’s “working set.” Redis Enterprise Flash will intelligently manage the location of the working set (RAM) and the infrequently accessed keys (flash memory), based on LRU (least-recently-used) on a per-object basis.
Redis Client Support
Just like all-RAM databases, RF is compatible with existing Redis applications. Databases that employ RF are identical to all-RAM Redis Enterprise Pack databases in characteristics and features.
Redis Enterprise Flash vs Disk Based Databases
Flash memory can be SATA or NVMe based storage devices, but NVMe is where you will see the performance benefits. There are many databases in the industry that are disk based. Disk-based databases use RAM to cache part of the data for fast access. However, this approach is different than extending RAM in a number of ways.
- Hot Value Handling: In many applications, a large portion of operations utilize only a subset of keys in the database. For example, the same keys may receive multiple writes repeatedly in a short amount of time. Under these conditions, disk-based databases have to repeatedly do I/O to save the updates to disk. In contrast, with RF, the hot values stay in RAM and repeated writes to the same key do not generate IO to the flash memory.
- Write Performance: With pure disk-based databases, the writes to the disk have to be durable. To ensure durable writes, disk-based databases use techniques like WAL (write ahead log) or Redo-Logs. In contrast, when RF ejects a value from RAM to Flash, the write operation does not incur the expensive WAL or Redo log techniques. In other words, write amplification with durable writes is much slower than writes RF performs to extend RAM. That said, you can still do durable writes with RF, but there are some considerations.
- Future Proof: In recent years, with the emergence of persistent memory technologies, memory has been moving to converge with storage. Persisted memory technologies like 3d XPoint are good examples of these technologies. These technologies assume that your application is aware of which parts of the dataset should be kept in RAM and which part are OK to store in persistent memory. If your application is not specifically designed for this technology, persistent-memory performance is going to be very slow and perhaps unpredictable.Redis Enterprise Flash, in contrast, is application agnostic as it performs this function on the server side and your application has no need to understand where the data resides. Your application issues the same commands it always has with Redis and Redise Pack. You get the benefits of RF now and into the future regardless of how flash memory evolves.
To create Redis Enterprise Flash databases you must meet the following prerequisites:
- Have ephemeral, persistent and flash memory mounted on Redise Pack nodes with the proper disk size, per the Redis Enterprise Flash memory guidelines.
- Have all software and hardware requirements in place.
- Perform the initial setup and creation of a new cluster.
Once these requirements are met, both Redis Enterprise Flash databases and all-RAM databases can be created and managed in the same cluster. For additional details, refer to Creating a new database.
When Redis Enterprise Flash is enabled, additional settings and metrics are available in the system.