EBOOK – REDIS IN ACTION

This book covers the use of Redis, an in-memory database/data structure server.

open all | close all

5.4.3 Automatic Redis connection management

Manually creating and passing connections to Redis can be tough. Not only do we need
to repeatedly refer to configuration information, but if we’re using our configuration
management functions from the last section, we still need to fetch the configuration,
connect to Redis, and somehow deal with the connection when we’re done. To simplify
the management of all of these connections, we’ll write a decorator that will take care
of connecting to all of our Redis servers (except for the configuration server).

DECORATORSWithin Python there’s a syntax for passing a function X into
another function Y. This function Y is called a decorator. Decorators are given
an opportunity to alter the behavior of function X. Some decorators validate
arguments, other decorators register callbacks, and even others manage connections
like we intend to.

Our decorator will take a named configuration as an argument, which will generate a
wrapper that, when called on the actual function, will wrap the function such that
later calls will automatically connect to the appropriate Redis server, and that connection
will be passed to the wrapped function with all of the other arguments that were
later provided. The next listing has the source for our redis_connection() function.

Listing 5.16The redis_connection() function/decorator
REDIS_CONNECTIONS = {}

def redis_connection(component, wait=1):

We pass the name of the application component to the decorator.

   key = 'config:redis:' + component

We cache the configuration key because we’ll fetch it every time the function is called.

   def wrapper(function):

Our wrapper takes a function that it wraps with another function.

      @functools.wraps(function)

Copy some useful metadata from the original function to the configuration handler.

      def call(*args, **kwargs):

Create the actual function that will be managing connection information.

         old_config = CONFIGS.get(key, object())       

Fetch the old configuration, if any.

         _config = get_config(
            config_connection, 'redis', component, wait)

Get the new configuration, if any.

         config = {}

         for k, v in _config.iteritems():
            config[k.encode('utf-8')] = v

Make the configuration usable for creating a Redis connection.

         if config != old_config:
            REDIS_CONNECTIONS[key] = redis.Redis(**config)

If the new and old configurations don’t match, create a new connection.

         return function(
            REDIS_CONNECTIONS.get(key), *args, **kwargs)

Call and return the result of our wrapped function, remembering to pass the connection and the other matched arguments.

   return call

Return the fully wrapped function.

return wrapper

Return a function that can wrap our Redis function.

COMBINING *args AND **kwargsWay back in chapter 1, we first looked at
default arguments in Python. But here, we’re combining two different forms
of argument passing. If you’re having difficulty understanding what’s going
on (which is essentially capturing all positional and named arguments in the
args and kwargs variables in the function definition, and passing all positional
and named parameters to the called function), then you should spend
some time with the Python language tutorial via this shortened URL: http://
mng.bz/KM5x
.

I know that this group of nested functions can be confusing at first, but it really isn’t
that bad. We have a function, redis_connection(), that takes the named application
component and returns a wrapper function. That wrapper function is then called with
the function we want to pass a connection to (the wrapped function), which then
returns the function caller. This caller handles all of the work of getting configuration
information, connecting to Redis, and calling our wrapped function. Though it’s a
mouthful to describe, actually using it is convenient, as you can see by applying it in
the next listing to our log_recent() function from section 5.1.1.

Listing 5.17The decorated log_recent() function
@redis_connection('logs')

The redis_connection() decorator is very easy to use.

def log_recent(conn, app, message):

The function definition doesn’t change.

   'the old log_recent() code'

log_recent('main', 'User 235 logged in')

We no longer need to worry about passing the log server connection when calling log_recent().

DECORATORSIn addition to the strange argument passing with *args and
**kwargs from listing 5.16, we’re also using syntax to “decorate” the log function.
That is to say, we pass a function to a decorator, which performs some
manipulation on the function before returning the original function, or
something else. You can read up on the details of what’s going on and why at
http://www.python.org/dev/peps/pep-0318/.

Now that you’ve seen how to use the redis_connection() decorator on log_recent(),
it doesn’t seem so bad, does it? With this better method of handling connections and
configuration, we’ve just removed a handful of lines from almost every function that
we’ll be calling. As an exercise, try to add this decorator to the access_time() context
manager from section 5.2.3 so that we don’t need to pass a connection. Feel free to reuse
this decorator with all of the other examples in the book.