EBOOK – REDIS IN ACTION

This book covers the use of Redis, an in-memory database/data structure server.

open all | close all

6.2.5 Locks with timeouts

As mentioned before, our lock doesn’t handle cases where a lock holder crashes without
releasing the lock, or when a lock holder fails and holds the lock forever. To handle
the crash/failure cases, we add a timeout to the lock.

In order to give our lock a timeout, we’ll use EXPIRE to have Redis time it out automatically.
The natural place to put the EXPIRE is immediately after the lock is
acquired, and we’ll do that. But if our client happens to crash (and the worst place for
it to crash for us is between SETNX and EXPIRE), we still want the lock to eventually
time out. To handle that situation, any time a client fails to get the lock, the client will
check the expiration on the lock, and if it’s not set, set it. Because clients are going to
be checking and setting timeouts if they fail to get a lock, the lock will always have a
timeout, and will eventually expire, letting other clients get a timed-out lock.

What if multiple clients set expiration times simultaneously? They’ll run at essentially
the same time, so expiration will be set for the same time.

Adding expiration to our earlier acquire_lock() function gets us the updated
acquire_lock_with_timeout() function shown here.

Listing 6.11The acquire_lock_with_timeout() function
def acquire_lock_with_timeout(
   conn, lockname, acquire_timeout=10, lock_timeout=10):
   identifier = str(uuid.uuid4())

A 128-bit random identifier.

   lock_timeout = int(math.ceil(lock_timeout))

Only pass integers to our EXPIRE calls.

   end = time.time() + acquire_timeout
   while time.time() < end:
      if conn.setnx(lockname, identifier):
         conn.expire(lockname, lock_timeout)

Get the lock and set the expiration.

         return identifier
      elif not conn.ttl(lockname):
         conn.expire(lockname, lock_timeout)

Check and update the expiration time as necessary.

      time.sleep(.001)

   return False

This new acquire_lock_with_timeout() handling timeouts. It ensures that locks
expire as necessary, and that they won’t be stolen from clients that rightfully have them.
Even better, we were smart with our release lock function earlier, which still works.

NOTEAs of Redis 2.6.12, the SET command added options to support a combination
of SETNX and SETEX functionality, which makes our lock acquire
function trivial. We still need the complicated release lock to be correct.

In section 6.1.2 when we built the address book autocomplete using a ZSET, we went
through a bit of trouble to create start and end entries to add to the ZSET in order to
fetch a range. We also postprocessed our data to remove entries with curly braces
({}), because other autocomplete operations could be going on at the same time.
And because other operations could be going on at the same time, we used WATCH so that we could retry. Each of those pieces added complexity to our functions, which
could’ve been simplified if we’d used a lock instead.

In other databases, locking is a basic operation that’s supported and performed
automatically. As I mentioned earlier, using WATCH, MULTI, and EXEC is a way of having
an optimistic lock—we aren’t actually locking data, but we’re notified and our
changes are canceled if someone else modifies it before we do. By adding explicit
locking on the client, we get a few benefits (better performance, a more familiar programming
concept, easier-to-use API, and so on), but we need to remember that Redis
itself doesn’t respect our locks. It’s up to us to consistently use our locks in addition to
or instead of WATCH, MULTI, and EXEC to keep our data consistent and correct.

Now that we’ve built a lock with timeouts, let’s look at another kind of lock called a
counting semaphore. It isn’t used in as many places as a regular lock, but when we need
to give multiple clients access to the same information at the same time, it’s the perfect
tool for the job.