EBOOK – REDIS IN ACTION

This book covers the use of Redis, an in-memory database/data structure server.

  • Foreword
  • Preface
  • Acknowledgments
  • About this Book
  • About the Cover Illustration
  • Part 1: Getting Started
  • Part 2: Core concepts
  • Part 3: Next steps
  • Appendix A
  • Appendix B
  • Buy the paperback

    5.4.3 Automatic Redis connection management

    Manually creating and passing connections to Redis can be tough. Not only do we need
    to repeatedly refer to configuration information, but if we’re using our configuration
    management functions from the last section, we still need to fetch the configuration,
    connect to Redis, and somehow deal with the connection when we’re done. To simplify
    the management of all of these connections, we’ll write a decorator that will take care
    of connecting to all of our Redis servers (except for the configuration server).

    DECORATORSWithin Python there’s a syntax for passing a function X into
    another function Y. This function Y is called a decorator. Decorators are given
    an opportunity to alter the behavior of function X. Some decorators validate
    arguments, other decorators register callbacks, and even others manage connections
    like we intend to.

    Our decorator will take a named configuration as an argument, which will generate a
    wrapper that, when called on the actual function, will wrap the function such that
    later calls will automatically connect to the appropriate Redis server, and that connection
    will be passed to the wrapped function with all of the other arguments that were
    later provided. The next listing has the source for our redis_connection() function.

    Listing 5.16The redis_connection() function/decorator
    REDIS_CONNECTIONS = {}
    
    

    def redis_connection(component, wait=1):
    

    We pass the name of the application component to the decorator.

       key = 'config:redis:' + component
    

    We cache the configuration key because we’ll fetch it every time the function is called.

       def wrapper(function):
    

    Our wrapper takes a function that it wraps with another function.

          @functools.wraps(function)
    

    Copy some useful metadata from the original function to the configuration handler.

          def call(*args, **kwargs):
    

    Create the actual function that will be managing connection information.

             old_config = CONFIGS.get(key, object())       
    

    Fetch the old configuration, if any.

             _config = get_config(
                config_connection, 'redis', component, wait)
    
    

    Get the new configuration, if any.

             config = {}
    

             for k, v in _config.iteritems():
                config[k.encode('utf-8')] = v
    
    

    Make the configuration usable for creating a Redis connection.

             if config != old_config:
                REDIS_CONNECTIONS[key] = redis.Redis(**config)
    
    

    If the new and old configurations don’t match, create a new connection.

             return function(
                REDIS_CONNECTIONS.get(key), *args, **kwargs)
    

    Call and return the result of our wrapped function, remembering to pass the connection and the other matched arguments.

       return call
    

    Return the fully wrapped function.

    return wrapper
    

    Return a function that can wrap our Redis function.

    COMBINING *args AND **kwargsWay back in chapter 1, we first looked at
    default arguments in Python. But here, we’re combining two different forms
    of argument passing. If you’re having difficulty understanding what’s going
    on (which is essentially capturing all positional and named arguments in the
    args and kwargs variables in the function definition, and passing all positional
    and named parameters to the called function), then you should spend
    some time with the Python language tutorial via this shortened URL: http://
    mng.bz/KM5x
    .

    I know that this group of nested functions can be confusing at first, but it really isn’t
    that bad. We have a function, redis_connection(), that takes the named application
    component and returns a wrapper function. That wrapper function is then called with
    the function we want to pass a connection to (the wrapped function), which then
    returns the function caller. This caller handles all of the work of getting configuration
    information, connecting to Redis, and calling our wrapped function. Though it’s a
    mouthful to describe, actually using it is convenient, as you can see by applying it in
    the next listing to our log_recent() function from section 5.1.1.

    Listing 5.17The decorated log_recent() function
    @redis_connection('logs')
    

    The redis_connection() decorator is very easy to use.

    def log_recent(conn, app, message):
    

    The function definition doesn’t change.

       'the old log_recent() code'
    
    

    log_recent('main', 'User 235 logged in')
    

    We no longer need to worry about passing the log server connection when calling log_recent().

    DECORATORSIn addition to the strange argument passing with *args and
    **kwargs from listing 5.16, we’re also using syntax to “decorate” the log function.
    That is to say, we pass a function to a decorator, which performs some
    manipulation on the function before returning the original function, or
    something else. You can read up on the details of what’s going on and why at
    http://www.python.org/dev/peps/pep-0318/.

    Now that you’ve seen how to use the redis_connection() decorator on log_recent(),
    it doesn’t seem so bad, does it? With this better method of handling connections and
    configuration, we’ve just removed a handful of lines from almost every function that
    we’ll be calling. As an exercise, try to add this decorator to the access_time() context
    manager from section 5.2.3 so that we don’t need to pass a connection. Feel free to reuse
    this decorator with all of the other examples in the book.