10.2.1 Handling shard configuration
As you may remember from chapter 5, we wrote a method to create and use named
Redis configurations automatically. This method used a Python decorator to fetch
configuration information, compare it with preexisting configuration information,
and create or reuse an existing connection. We’ll extend this idea to add support for
sharded connections. With these changes, we can use much of our code developed in
chapter 9 with minor changes.
To get started, first let’s make a simple function that uses the same configuration
layout that we used in our connection decorator from chapter 5. If you remember, we
use JSON-encoded dictionaries to store connection information for Redis in keys of
the format config:redis:<component>. Pulling out the connection management
part of the decorator, we end up with a simple function to create or reuse a Redis connection,
based on a named configuration, shown here.
This simple function fetches the previously known as well as the current configuration.
If they’re different, it updates the known configuration, creates a new connection,
and then stores and returns that new connection. If the configuration hasn’t
changed, it returns the previous connection.
When we have the ability to fetch connections easily, we should also add support
for the creation of sharded Redis connections, so even if our later decorators aren’t
useful in every situation, we can still easily create and use sharded connections. To
connect to a new sharded connection, we’ll use the same configuration methods,
though sharded configuration will be a little different. For example, shard 7 of component
logs will be stored at a key named config:redis:logs:7. This naming scheme will let us reuse the existing connection and configuration code we already
have. Our function to get a sharded connection is in the following listing.
Now that we have a simple method of fetching a connection to a Redis server that’s
sharded, we can create a decorator like we saw in chapter 5 that creates a sharded connection