EBOOK – REDIS IN ACTION

This book covers the use of Redis, an in-memory database/data structure server.

open all | close all

11.2.2 Rewriting our lock

As you may remember from section 6.2, locking involved generating an ID, conditionally
setting a key with SETNX, and upon success setting the expiration time of the key.
Though conceptually simple, we had to deal with failures and retries, which resulted
in the original code shown in the next listing.

Listing 11.4Our final acquire_lock_with_timeout() function from section 6.2.5
def acquire_lock_with_timeout(
    conn, lockname, acquire_timeout=10, lock_timeout=10):
    identifier = str(uuid.uuid4())

A 128-bit random identifier.

    lockname = 'lock:' + lockname
    lock_timeout = int(math.ceil(lock_timeout))

Only pass integers to our EXPIRE calls.

    end = time.time() + acquire_timeout
    while time.time() < end:
        if conn.setnx(lockname, identifier):
            conn.expire(lockname, lock_timeout)

Get the lock and set the expiration.

            return identifier
        elif not conn.ttl(lockname):
            conn.expire(lockname, lock_timeout)

Check and update the expiration time as necessary.

        time.sleep(.001)

    return False

There’s nothing too surprising here if you remember how we built up to this lock in
section 6.2. Let’s go ahead and offer the same functionality, but move the core locking
into Lua.

Listing 11.5A rewritten acquire_lock_with_timeout() that uses Lua
def acquire_lock_with_timeout(
    conn, lockname, acquire_timeout=10, lock_timeout=10):
    identifier = str(uuid.uuid4())
    lockname = 'lock:' + lockname
    lock_timeout = int(math.ceil(lock_timeout))
    acquired = False
    end = time.time() + acquire_timeout
    while time.time() < end and not acquired:
        acquired = acquire_lock_with_timeout_lua(
            conn, [lockname], [lock_timeout, identifier]) == 'OK'

Actually acquire the lock, checking to verify that the Lua call completed successfully.

        time.sleep(.001 * (not acquired))

    return acquired and identifier

acquire_lock_with_timeout_lua = script_load('''
if redis.call('exists', KEYS[1]) == 0 then

If the lock doesn’t already exist, again remembering that tables use 1-based indexing.

    return redis.call('setex', KEYS[1], unpack(ARGV))

Set the key with the provided expiration and identifier.

end
''')

There aren’t any significant changes in the code, except that we change the commands
we use so that if a lock is acquired, it always has a timeout. Let’s also go ahead
and rewrite the release lock code to use Lua.

Previously, we watched the lock key, and then verified that the lock still had the
same value. If it had the same value, we removed the lock; otherwise we’d say that the
lock was lost. Our Lua version of release_lock() is shown next.

Listing 11.6A rewritten release_lock() that uses Lua
def release_lock(conn, lockname, identifier):
    lockname = 'lock:' + lockname
    return release_lock_lua(conn, [lockname], [identifier])'

Call the Lua function that releases the lock.

release_lock_lua = script_load(''
if redis.call('get', KEYS[1]) == ARGV[1] then

Make sure that the lock matches.

    return redis.call('del', KEYS[1]) or true

Delete the lock and ensure that we return true.

end
''')

Unlike acquiring the lock, releasing the lock became shorter as we no longer needed
to perform all of the typical WATCH/MULTI/EXEC steps.

Reducing the code is great, but we haven’t gotten far if we haven’t actually improved
the performance of the lock itself. We’ve added some instrumentation to the locking
code along with some benchmarking code that executes 1, 2, 5, and 10 parallel processes
to acquire and release locks repeatedly. We count the number of attempts to acquire the
lock and how many times the lock was acquired over 10 seconds, with both our original
and Lua-based acquire and release lock functions. Table 11.2 shows the number of calls
that were performed and succeeded.

Table 11.2Performance of our original lock against a Lua-based lock over 10 seconds

Benchmark configuration

Tries in 10 seconds

Acquires in 10 seconds

Original lock, 1 client

31,359

31,359

Original lock, 2 clients

30,085

22,507

Original lock, 5 clients

47,694

19,695

Original lock, 10 clients

71,917

14,361

Lua lock, 1 client

44,494

44,494

Lua lock, 2 clients

50,404

42,199

Lua lock, 5 clients

70,807

40,826

Lua lock, 10 clients

96,871

33,990

Looking at the data from our benchmark (pay attention to the right column), one
thing to note is that Lua-based locks succeed in acquiring and releasing the lock in
cycles significantly more often than our previous lock—by more than 40% with a single
client, 87% with 2 clients, and over 100% with 5 or 10 clients attempting to acquire
and release the same locks. Comparing the middle and right columns, we can also see
how much faster attempts at locking are made with Lua, primarily due to the reduced
number of round trips.

But even better than performance improvements, our code to acquire and release
the locks is significantly easier to understand and verify as correct.

Another example where we built a synchronization primitive is with semaphores;
let’s take a look at building them next.