e-Book - Redis in Action

This book covers the use of Redis, an in-memory database/data structure server.
  • Foreword
  • Preface
  • Acknowledgments
  • About this Book
  • About the Cover Illustration
  • Part 1: Getting Started
  • Part 2: Core concepts
  • Part 3: Next steps
  • Appendix A
  • Appendix B
  • Buy the paperback

    4.2.4 Verifying disk writes

    Verifying that the data we wrote to the master made it to the slave is easy: we merely
    need to write a unique dummy value after our important data, and then check for it
    on the slave. But verifying that the data made it to disk is more difficult. If we wait at
    least one second, we know that our data made it to disk. But if we’re careful, we may
    be able to wait less time by checking the output of INFO for the value of
    aof_pending_bio_fsync, which will be 0 if all data that the server knows about has
    been written to disk. To automate this check, we can use the function provided in the
    next listing, which we’d call after writing our data to the master by passing both the
    master and slave connections.

    Listing 4.3Listing 4.3 The wait_for_sync() function
    def wait_for_sync(mconn, sconn):
       identifier = str(uuid.uuid4())

       mconn.zadd('sync:wait', identifier, time.time())

    Add the token to the master.

       while not sconn.INFO()['master_link_status'] != 'up':

    Wait for the slave to sync (if necessary).


       while not sconn.zscore('sync:wait', identifier):

    Wait for the slave to receive the data change.


       deadline = time.time() + 1.01
       while time.time() < deadline:

    Wait up to one second.

          if sconn.INFO()['aof_pending_bio_fsync'] == 0:

    Check to see if the data is known to be on disk.


       mconn.zrem('sync:wait', identifier)
       mconn.zremrangebyscore('sync:wait', 0, time.time()-900)

    Clean up our status and clean out older entries that may have been left there.

    OTHER INFORMATION FROM THE INFO COMMANDThe INFO command can offer a wide range of information about the current status of a Redis server—memory
    used, the number of connected clients, the number of keys in each database,
    the number of commands executed since the last snapshot, and more. Generally
    speaking, INFO is a good source of information about the general state of
    our Redis servers, and many resources online can explain more.

    To ensure correct operation, this function will first verify that the slave is connected to
    the master. It’ll then poll the slave, looking for the value that it had added to the sync
    wait ZSET. After it has found that the value has made it to the slave, it’ll then check on
    the status of the Redis write buffer, waiting for it to either say that there are no pending
    syncs to disk (signaling that the change had made it to disk), or wait for up to one second.
    We wait for one second under the assumption that after one second, the data had
    been synced to disk, but there’s so much writing to Redis that we didn’t catch when the
    data had been synced. After verifying the write to disk, we then clean up after ourselves.

    By combining replication and append-only files, we can configure Redis to be resilient
    against system failures.