e-Book - Redis in Action

This book covers the use of Redis, an in-memory database/data structure server.
  • Foreword
  • Preface
  • Acknowledgments
  • About this Book
  • About the Cover Illustration
  • Part 1: Getting Started
  • Part 2: Core concepts
  • Part 3: Next steps
  • Appendix A
  • Appendix B
  • Buy the paperback

    6.2.5 Locks with timeouts

    As mentioned before, our lock doesn’t handle cases where a lock holder crashes without
    releasing the lock, or when a lock holder fails and holds the lock forever. To handle
    the crash/failure cases, we add a timeout to the lock.

    In order to give our lock a timeout, we’ll use EXPIRE to have Redis time it out automatically.
    The natural place to put the EXPIRE is immediately after the lock is
    acquired, and we’ll do that. But if our client happens to crash (and the worst place for
    it to crash for us is between SETNX and EXPIRE), we still want the lock to eventually
    time out. To handle that situation, any time a client fails to get the lock, the client will
    check the expiration on the lock, and if it’s not set, set it. Because clients are going to
    be checking and setting timeouts if they fail to get a lock, the lock will always have a
    timeout, and will eventually expire, letting other clients get a timed-out lock.

    What if multiple clients set expiration times simultaneously? They’ll run at essentially
    the same time, so expiration will be set for the same time.

    Adding expiration to our earlier acquire_lock() function gets us the updated
    acquire_lock_with_timeout() function shown here.

    Listing 6.11The acquire_lock_with_timeout() function
    def acquire_lock_with_timeout(
       conn, lockname, acquire_timeout=10, lock_timeout=10):
       identifier = str(uuid.uuid4())

    A 128-bit random identifier.

       lock_timeout = int(math.ceil(lock_timeout))

    Only pass integers to our EXPIRE calls.

       end = time.time() + acquire_timeout
       while time.time() < end:
          if conn.setnx(lockname, identifier):
             conn.expire(lockname, lock_timeout)

    Get the lock and set the expiration.

             return identifier
          elif not conn.ttl(lockname):
             conn.expire(lockname, lock_timeout)

    Check and update the expiration time as necessary.

       return False

    This new acquire_lock_with_timeout() handling timeouts. It ensures that locks
    expire as necessary, and that they won’t be stolen from clients that rightfully have them.
    Even better, we were smart with our release lock function earlier, which still works.

    NOTEAs of Redis 2.6.12, the SET command added options to support a combination
    of SETNX and SETEX functionality, which makes our lock acquire
    function trivial. We still need the complicated release lock to be correct.

    In section 6.1.2 when we built the address book autocomplete using a ZSET, we went
    through a bit of trouble to create start and end entries to add to the ZSET in order to
    fetch a range. We also postprocessed our data to remove entries with curly braces
    ({}), because other autocomplete operations could be going on at the same time.
    And because other operations could be going on at the same time, we used WATCH so that we could retry. Each of those pieces added complexity to our functions, which
    could’ve been simplified if we’d used a lock instead.

    In other databases, locking is a basic operation that’s supported and performed
    automatically. As I mentioned earlier, using WATCH, MULTI, and EXEC is a way of having
    an optimistic lock—we aren’t actually locking data, but we’re notified and our
    changes are canceled if someone else modifies it before we do. By adding explicit
    locking on the client, we get a few benefits (better performance, a more familiar programming
    concept, easier-to-use API, and so on), but we need to remember that Redis
    itself doesn’t respect our locks. It’s up to us to consistently use our locks in addition to
    or instead of WATCH, MULTI, and EXEC to keep our data consistent and correct.

    Now that we’ve built a lock with timeouts, let’s look at another kind of lock called a
    counting semaphore. It isn’t used in as many places as a regular lock, but when we need
    to give multiple clients access to the same information at the same time, it’s the perfect
    tool for the job.