e-Book - Redis in Action

This book covers the use of Redis, an in-memory database/data structure server.
  • Foreword
  • Preface
  • Acknowledgments
  • About this Book
  • About the Cover Illustration
  • Part 1: Getting Started
  • Part 2: Core concepts
  • Part 3: Next steps
  • Appendix A
  • Appendix B
  • Buy the paperback

    Chapter 9: Reducing memory use

    This chapter covers

    • Short structures
    • Sharded structures
    • Packing bits and bytes

    In this chapter, we’ll cover three important methods to help reduce your memory
    use in Redis. By reducing the amount of memory you use in Redis, you can reduce
    the time it takes to create or load a snapshot, rewrite or load an append-only file,
    reduce slave synchronization time,1 and store more data in Redis without additional

    We’ll begin this chapter by discussing how the use of short data structures in
    Redis can result in a more efficient representation of the data. We’ll then discuss
    how to apply a concept called sharding to help make some larger structures small.2
    Finally, we’ll talk about packing fixed-length data into STRINGs for even greater
    memory savings.

    When used together, these methods helped me to reduce memory use from more
    than 70 gigabytes, split across three machines, down to under 3 gigabytes on a single
    machine. As we work through these methods, remember that some of our earlier
    problems would lend themselves well to these optimizations, which I’ll point out when
    applicable. Let’s get started with one of the first and easiest methods to reduce memory
    use: short structures.

    1 Snapshots, append-only file rewriting, and slave synchronization are all discussed in chapter 4.

    2 Our use of sharding here is primarily driven to reduce memory use on a single server. In chapter 10, we’ll apply similar techniques to allow for increased read throughput, write throughput, and memory partitioning across multiple Redis servers.