EBOOK – REDIS IN ACTION

This book covers the use of Redis, an in-memory database/data structure server.

open all | close all

4.2.4 Verifying disk writes

Verifying that the data we wrote to the master made it to the slave is easy: we merely
need to write a unique dummy value after our important data, and then check for it
on the slave. But verifying that the data made it to disk is more difficult. If we wait at
least one second, we know that our data made it to disk. But if we’re careful, we may
be able to wait less time by checking the output of INFO for the value of
aof_pending_bio_fsync, which will be 0 if all data that the server knows about has
been written to disk. To automate this check, we can use the function provided in the
next listing, which we’d call after writing our data to the master by passing both the
master and slave connections.

Listing 4.3Listing 4.3 The wait_for_sync() function
def wait_for_sync(mconn, sconn):
   identifier = str(uuid.uuid4())

   mconn.zadd('sync:wait', identifier, time.time())

Add the token to the master.

   while not sconn.INFO()['master_link_status'] != 'up':

Wait for the slave to sync (if necessary).

      time.sleep(.001)

   while not sconn.zscore('sync:wait', identifier):

Wait for the slave to receive the data change.

      time.sleep(.001)

   deadline = time.time() + 1.01
   while time.time() < deadline:

Wait up to one second.

      if sconn.INFO()['aof_pending_bio_fsync'] == 0:
         break

Check to see if the data is known to be on disk.

      time.sleep(.001)

   mconn.zrem('sync:wait', identifier)
   mconn.zremrangebyscore('sync:wait', 0, time.time()-900)

Clean up our status and clean out older entries that may have been left there.

OTHER INFORMATION FROM THE INFO COMMANDThe INFO command can offer a wide range of information about the current status of a Redis server—memory
used, the number of connected clients, the number of keys in each database,
the number of commands executed since the last snapshot, and more. Generally
speaking, INFO is a good source of information about the general state of
our Redis servers, and many resources online can explain more.

To ensure correct operation, this function will first verify that the slave is connected to
the master. It’ll then poll the slave, looking for the value that it had added to the sync
wait ZSET. After it has found that the value has made it to the slave, it’ll then check on
the status of the Redis write buffer, waiting for it to either say that there are no pending
syncs to disk (signaling that the change had made it to disk), or wait for up to one second.
We wait for one second under the assumption that after one second, the data had
been synced to disk, but there’s so much writing to Redis that we didn’t catch when the
data had been synced. After verifying the write to disk, we then clean up after ourselves.

By combining replication and append-only files, we can configure Redis to be resilient
against system failures.