Chapter 9: Reducing memory use

In this chapter, we’ll cover three important methods to help reduce your memory
use in Redis. By reducing the amount of memory you use in Redis, you can reduce
the time it takes to create or load a snapshot, rewrite or load an append-only file,
reduce slave synchronization time,¹ and store more data in Redis without additional
hardware.

We’ll begin this chapter by discussing how the use of short data structures in
Redis can result in a more efficient representation of the data. We’ll then discuss
how to apply a concept called sharding to help make some larger structures small.²
Finally, we’ll talk about packing fixed-length data into STRINGs for even greater
memory savings.

When used together, these methods helped me to reduce memory use from more
than 70 gigabytes, split across three machines, down to under 3 gigabytes on a single
machine. As we work through these methods, remember that some of our earlier
problems would lend themselves well to these optimizations, which I’ll point out when
applicable. Let’s get started with one of the first and easiest methods to reduce memory
use: short structures.

¹ Snapshots, append-only file rewriting, and slave synchronization are all discussed in chapter 4.

² Our use of sharding here is primarily driven to reduce memory use on a single server. In chapter 10, we’ll apply similar techniques to allow for increased read throughput, write throughput, and memory partitioning across multiple Redis servers.