EBOOK – REDIS IN ACTION

This book covers the use of Redis, an in-memory database/data structure server.

  • Foreword
  • Preface
  • Acknowledgments
  • About this Book
  • About the Cover Illustration
  • Part 1: Getting Started
  • Part 2: Core concepts
  • Part 3: Next steps
  • Appendix A
  • Appendix B
  • Buy the paperback

    11.2.2 Rewriting our lock

    As you may remember from section 6.2, locking involved generating an ID, conditionally
    setting a key with SETNX, and upon success setting the expiration time of the key.
    Though conceptually simple, we had to deal with failures and retries, which resulted
    in the original code shown in the next listing.

    Listing 11.4Our final acquire_lock_with_timeout() function from section 6.2.5
    def acquire_lock_with_timeout(
        conn, lockname, acquire_timeout=10, lock_timeout=10):
    
        identifier = str(uuid.uuid4())
    

    A 128-bit random identifier.

        lockname = 'lock:' + lockname
    
        lock_timeout = int(math.ceil(lock_timeout))
    

    Only pass integers to our EXPIRE calls.

        end = time.time() + acquire_timeout
        while time.time() < end:
    
            if conn.setnx(lockname, identifier):
                conn.expire(lockname, lock_timeout)
    

    Get the lock and set the expiration.

                return identifier
    
            elif not conn.ttl(lockname):
                conn.expire(lockname, lock_timeout)
    

    Check and update the expiration time as necessary.

            time.sleep(.001)
    
        return False
    

    There’s nothing too surprising here if you remember how we built up to this lock in
    section 6.2. Let’s go ahead and offer the same functionality, but move the core locking
    into Lua.

    Listing 11.5A rewritten acquire_lock_with_timeout() that uses Lua
    def acquire_lock_with_timeout(
        conn, lockname, acquire_timeout=10, lock_timeout=10):
        identifier = str(uuid.uuid4())
        lockname = 'lock:' + lockname
        lock_timeout = int(math.ceil(lock_timeout))
        acquired = False
        end = time.time() + acquire_timeout
        while time.time() < end and not acquired:
    
            acquired = acquire_lock_with_timeout_lua(
                conn, [lockname], [lock_timeout, identifier]) == 'OK'
    

    Actually acquire the lock, checking to verify that the Lua call completed successfully.

            time.sleep(.001 * (not acquired))
    
        return acquired and identifier
    
    acquire_lock_with_timeout_lua = script_load('''
    
    if redis.call('exists', KEYS[1]) == 0 then
    

    If the lock doesn’t already exist, again remembering that tables use 1-based indexing.

        return redis.call('setex', KEYS[1], unpack(ARGV))
    

    Set the key with the provided expiration and identifier.

    end
    ''')
    

    There aren’t any significant changes in the code, except that we change the commands
    we use so that if a lock is acquired, it always has a timeout. Let’s also go ahead
    and rewrite the release lock code to use Lua.

    Previously, we watched the lock key, and then verified that the lock still had the
    same value. If it had the same value, we removed the lock; otherwise we’d say that the
    lock was lost. Our Lua version of release_lock() is shown next.

    Listing 11.6A rewritten release_lock() that uses Lua
    def release_lock(conn, lockname, identifier):
        lockname = 'lock:' + lockname
    
        return release_lock_lua(conn, [lockname], [identifier])'
    

    Call the Lua function that releases the lock.

    release_lock_lua = script_load(''
    
    if redis.call('get', KEYS[1]) == ARGV[1] then
    

    Make sure that the lock matches.

        return redis.call('del', KEYS[1]) or true
    

    Delete the lock and ensure that we return true.

    end
    ''')
    

    Unlike acquiring the lock, releasing the lock became shorter as we no longer needed
    to perform all of the typical WATCH/MULTI/EXEC steps.

    Reducing the code is great, but we haven’t gotten far if we haven’t actually improved
    the performance of the lock itself. We’ve added some instrumentation to the locking
    code along with some benchmarking code that executes 1, 2, 5, and 10 parallel processes
    to acquire and release locks repeatedly. We count the number of attempts to acquire the
    lock and how many times the lock was acquired over 10 seconds, with both our original
    and Lua-based acquire and release lock functions. Table 11.2 shows the number of calls
    that were performed and succeeded.

    Table 11.2Performance of our original lock against a Lua-based lock over 10 seconds

    Benchmark configuration

    Tries in 10 seconds

    Acquires in 10 seconds

    Original lock, 1 client

    31,359

    31,359

    Original lock, 2 clients

    30,085

    22,507

    Original lock, 5 clients

    47,694

    19,695

    Original lock, 10 clients

    71,917

    14,361

    Lua lock, 1 client

    44,494

    44,494

    Lua lock, 2 clients

    50,404

    42,199

    Lua lock, 5 clients

    70,807

    40,826

    Lua lock, 10 clients

    96,871

    33,990

    Looking at the data from our benchmark (pay attention to the right column), one
    thing to note is that Lua-based locks succeed in acquiring and releasing the lock in
    cycles significantly more often than our previous lock—by more than 40% with a single
    client, 87% with 2 clients, and over 100% with 5 or 10 clients attempting to acquire
    and release the same locks. Comparing the middle and right columns, we can also see
    how much faster attempts at locking are made with Lua, primarily due to the reduced
    number of round trips.

    But even better than performance improvements, our code to acquire and release
    the locks is significantly easier to understand and verify as correct.

    Another example where we built a synchronization primitive is with semaphores;
    let’s take a look at building them next.