This book covers the use of Redis, an in-memory database/data structure server.

open all | close all

4.2.3 Master/slave chains

Some developers have found that when they need to replicate to more than a handful
of slaves, some networks are unable to keep up—especially when replication is being
performed over the internet or between data centers. Because there’s nothing particularly
special about being a master or a slave in Redis, slaves can have their own slaves,
resulting in master/slave chaining.

Operationally, the only difference in the replication process that occurs is that if a
slave X has its own slave Y, when slave X hits step 4 from table 4.2, slave X will disconnect
slave Y, causing Y to reconnect and resync.

When read load significantly outweighs write load, and when the number of reads
pushes well beyond what a single Redis server can handle, it’s common to keep adding
slaves to help deal with the load. As load continues to increase, we can run into situations
where the single master can’t write to all of its slaves fast enough, or is overloaded
with slaves reconnecting and resyncing. To alleviate such issues, we may want
to set up a layer of intermediate Redis master/slave nodes that can help with replication
duties similar to figure 4.1.

Figure 4.1An example Redis master/slave replica tree with nine lowest-level slaves and three
intermediate replication helper servers

Though the example shown in figure 4.1 may not necessarily need to be in a tree
structure, remembering and understanding that this is both possible and reasonable
for Redis replication can help you later.

Back in section 4.1.2, we talked about using append-only files with syncing to limit the opportunities for us to lose data. We could prevent data loss almost entirely (except for system or hard drive crashes) by syncing every write to disk, but then we
end up limiting performance severely. If we tell Redis to sync every second, we’re able
to get the performance we need, but we could lose up to a second of writes if bad
things happen. But by combining replication and append-only files, we can ensure
that data gets to disk on multiple machines.

In order to ensure that data gets to disk on multiple machines, we must obviously
set up a master with slaves. By configuring our slaves (and optionally our master) with
appendonly yes and appendfsync everysec, we now have a group of machines that
will sync to disk every second. But that’s only the first part: we must wait for the write
to reach the slave(s) and check to make sure that the data reached disk before we
can continue.