EBOOK – REDIS IN ACTION

This book covers the use of Redis, an in-memory database/data structure server.

  • Foreword
  • Preface
  • Acknowledgments
  • About this Book
  • About the Cover Illustration
  • Part 1: Getting Started
  • Part 2: Core concepts
  • Part 3: Next steps
  • Appendix A
  • Appendix B
  • Buy the paperback

    4.6 Performance considerations

    When coming from a relational database background, most users will be so happy with
    improving performance by a factor of 100 times or more by adding Redis, they won’t
    realize that they can make Redis perform even better. In the previous section, we introduced
    non-transactional pipelines as a way to minimize the number of round trips
    between our application and Redis. But what if we’ve already built an application, and
    we know that it could perform better? How do we find ways to improve performance?

    Improving performance in Redis requires having an understanding of what to
    expect in terms of performance for the types of commands that we’re sending to
    Redis. To get a better idea of what to expect from Redis, we’ll quickly run a benchmark
    that’s included with Redis, redis-benchmark, as can be seen in listing 4.10. Feel
    free to explore redis-benchmark on your own to discover the performance characteristics
    of your server and of Redis.

    Listing 4.10Running redis-benchmark on an Intel Core-2 Duo 2.4 GHz desktop
    $ redis-benchmark -c 1 -q

    We run with the ‘-q’ option to get simple output and ‘-c 1’ to use a single client.

    PING (inline): 34246.57 requests per second
    PING: 34843.21 requests per second
    MSET (10 keys): 24213.08 requests per second
    SET: 32467.53 requests per second
    GET: 33112.59 requests per second
    INCR: 32679.74 requests per second
    LPUSH: 33333.33 requests per second
    LPOP: 33670.04 requests per second
    SADD: 33222.59 requests per second
    SPOP: 34482.76 requests per second
    LPUSH (again, in order to bench LRANGE): 33222.59 requests per second
    LRANGE (first 100 elements): 22988.51 requests per second
    LRANGE (first 300 elements): 13888.89 requests per second
    LRANGE (first 450 elements): 11061.95 requests per second
    LRANGE (first 600 elements): 9041.59 requests per second

    The output of redis-benchmark shows a group of commands that are typically used in
    Redis, as well as the number of commands of that type that can be run in a single second.
    A standard run of this benchmark without any options will try to push Redis to its
    limit using 50 clients, but it’s a lot easier to compare performance of a single benchmark
    client against one copy of our own client, rather than many.

    When looking at the output of redis-benchmark, we must be careful not to try to
    directly compare its output with how quickly our application performs. This is
    because redis-benchmark doesn’t actually process the result of the commands that it
    performs, which means that the results of some responses that require substantial
    parsing overhead aren’t taken into account. Generally, compared to redis-benchmark
    running with a single client, we can expect the Python Redis client to perform at
    roughly 50–60% of what redis-benchmark will tell us for a single client and for nonpipelined
    commands, depending on the complexity of the command to call.

    If you find that your commands are running at about half of what you’d expect
    given redis-benchmark (about 25–30% of what redis-benchmark reports), or if you
    get errors reporting “Cannot assign requested address,” you may be accidentally creating
    a new connection for every command.

    I’ve listed some performance numbers relative to a single redis-benchmark client
    using the Python client, and have described some of the most likely causes of slowdowns
    and/or errors in table 4.5.

    This list of possible performance issues and solutions is short, but these issues
    amount to easily 95% of the performance-related problems that users report on a regular
    basis (aside from using Redis data structures incorrectly). If we’re experiencing
    slowdowns that we’re having difficulty in diagnosing, and we know it isn’t one of the
    problems listed in table 4.5, we should request help by one of the ways described in
    section 1.4.

    Table 4.5A table of general performance comparisons against a single redis-benchmark client and
    what may be causing potential slowdowns

    Performance or error

    Likely cause

    Remedy

    50–60% of redis-benchmark for a single client

    Expected performance without pipelining

    N/A

    25–30% of redis-benchmark for a single client

    Connecting for every command/group of commands

    Reuse your Redis connections

    Client error: “Cannot assign requested address”

    Connecting for every command/group of commands

    Reuse your Redis connections

    Most client libraries that access Redis offer some level of connection pooling built in.
    For Python, we only need to create a single redis.Redis() for every unique Redis
    server we need to connect to (we need to create a new connection for each numbered
    database we’re using). The redis.Redis() object itself will handle creating connections
    as necessary, reusing existing connections, and discarding timed-out connections.
    As written, the Python client connection pooling is both thread safe and fork() safe.