e-Book - Redis in Action

This book covers the use of Redis, an in-memory database/data structure server.
  • Foreword
  • Preface
  • Acknowledgments
  • About this Book
  • About the Cover Illustration
  • Part 1: Getting Started
  • Part 2: Core concepts
  • Part 3: Next steps
  • Appendix A
  • Appendix B
  • Buy the paperback

    4.2.3 Master/slave chains

    Some developers have found that when they need to replicate to more than a handful
    of slaves, some networks are unable to keep up—especially when replication is being
    performed over the internet or between data centers. Because there’s nothing particularly
    special about being a master or a slave in Redis, slaves can have their own slaves,
    resulting in master/slave chaining.

    Operationally, the only difference in the replication process that occurs is that if a
    slave X has its own slave Y, when slave X hits step 4 from table 4.2, slave X will disconnect
    slave Y, causing Y to reconnect and resync.

    When read load significantly outweighs write load, and when the number of reads
    pushes well beyond what a single Redis server can handle, it’s common to keep adding
    slaves to help deal with the load. As load continues to increase, we can run into situations
    where the single master can’t write to all of its slaves fast enough, or is overloaded
    with slaves reconnecting and resyncing. To alleviate such issues, we may want
    to set up a layer of intermediate Redis master/slave nodes that can help with replication
    duties similar to figure 4.1.

    Figure 4.1An example Redis master/slave replica tree with nine lowest-level slaves and three
    intermediate replication helper servers

    Though the example shown in figure 4.1 may not necessarily need to be in a tree
    structure, remembering and understanding that this is both possible and reasonable
    for Redis replication can help you later.

    Back in section 4.1.2, we talked about using append-only files with syncing to limit the opportunities for us to lose data. We could prevent data loss almost entirely (except for system or hard drive crashes) by syncing every write to disk, but then we
    end up limiting performance severely. If we tell Redis to sync every second, we’re able
    to get the performance we need, but we could lose up to a second of writes if bad
    things happen. But by combining replication and append-only files, we can ensure
    that data gets to disk on multiple machines.

    In order to ensure that data gets to disk on multiple machines, we must obviously
    set up a master with slaves. By configuring our slaves (and optionally our master) with
    appendonly yes and appendfsync everysec, we now have a group of machines that
    will sync to disk every second. But that’s only the first part: we must wait for the write
    to reach the slave(s) and check to make sure that the data reached disk before we
    can continue.