This book covers the use of Redis, an in-memory database/data structure server.

open all | close all

4.6 Performance considerations

When coming from a relational database background, most users will be so happy with
improving performance by a factor of 100 times or more by adding Redis, they won’t
realize that they can make Redis perform even better. In the previous section, we introduced
non-transactional pipelines as a way to minimize the number of round trips
between our application and Redis. But what if we’ve already built an application, and
we know that it could perform better? How do we find ways to improve performance?

Improving performance in Redis requires having an understanding of what to
expect in terms of performance for the types of commands that we’re sending to
Redis. To get a better idea of what to expect from Redis, we’ll quickly run a benchmark
that’s included with Redis, redis-benchmark, as can be seen in listing 4.10. Feel
free to explore redis-benchmark on your own to discover the performance characteristics
of your server and of Redis.

Listing 4.10Running redis-benchmark on an Intel Core-2 Duo 2.4 GHz desktop
$ redis-benchmark -c 1 -q

We run with the ‘-q’ option to get simple output and ‘-c 1’ to use a single client.

PING (inline): 34246.57 requests per second
PING: 34843.21 requests per second
MSET (10 keys): 24213.08 requests per second
SET: 32467.53 requests per second
GET: 33112.59 requests per second
INCR: 32679.74 requests per second
LPUSH: 33333.33 requests per second
LPOP: 33670.04 requests per second
SADD: 33222.59 requests per second
SPOP: 34482.76 requests per second
LPUSH (again, in order to bench LRANGE): 33222.59 requests per second
LRANGE (first 100 elements): 22988.51 requests per second
LRANGE (first 300 elements): 13888.89 requests per second
LRANGE (first 450 elements): 11061.95 requests per second
LRANGE (first 600 elements): 9041.59 requests per second

The output of redis-benchmark shows a group of commands that are typically used in
Redis, as well as the number of commands of that type that can be run in a single second.
A standard run of this benchmark without any options will try to push Redis to its
limit using 50 clients, but it’s a lot easier to compare performance of a single benchmark
client against one copy of our own client, rather than many.

When looking at the output of redis-benchmark, we must be careful not to try to
directly compare its output with how quickly our application performs. This is
because redis-benchmark doesn’t actually process the result of the commands that it
performs, which means that the results of some responses that require substantial
parsing overhead aren’t taken into account. Generally, compared to redis-benchmark
running with a single client, we can expect the Python Redis client to perform at
roughly 50–60% of what redis-benchmark will tell us for a single client and for nonpipelined
commands, depending on the complexity of the command to call.

If you find that your commands are running at about half of what you’d expect
given redis-benchmark (about 25–30% of what redis-benchmark reports), or if you
get errors reporting “Cannot assign requested address,” you may be accidentally creating
a new connection for every command.

I’ve listed some performance numbers relative to a single redis-benchmark client
using the Python client, and have described some of the most likely causes of slowdowns
and/or errors in table 4.5.

This list of possible performance issues and solutions is short, but these issues
amount to easily 95% of the performance-related problems that users report on a regular
basis (aside from using Redis data structures incorrectly). If we’re experiencing
slowdowns that we’re having difficulty in diagnosing, and we know it isn’t one of the
problems listed in table 4.5, we should request help by one of the ways described in
section 1.4.

Table 4.5A table of general performance comparisons against a single redis-benchmark client and
what may be causing potential slowdowns

Performance or error

Likely cause


50–60% of redis-benchmark for a single client

Expected performance without pipelining


25–30% of redis-benchmark for a single client

Connecting for every command/group of commands

Reuse your Redis connections

Client error: “Cannot assign requested address”

Connecting for every command/group of commands

Reuse your Redis connections

Most client libraries that access Redis offer some level of connection pooling built in.
For Python, we only need to create a single redis.Redis() for every unique Redis
server we need to connect to (we need to create a new connection for each numbered
database we’re using). The redis.Redis() object itself will handle creating connections
as necessary, reusing existing connections, and discarding timed-out connections.
As written, the Python client connection pooling is both thread safe and fork() safe.