e-Book - Redis in Action

This book covers the use of Redis, an in-memory database/data structure server.
  • Foreword
  • Preface
  • Acknowledgments
  • About this Book
  • About the Cover Illustration
  • Part 1: Getting Started
  • Part 2: Core concepts
  • Part 3: Next steps
  • Appendix A
  • Appendix B
  • Buy the paperback

    5.1.1 Recent logs

    When building a system, knowing what’s important to record can be difficult. Do you
    record every time someone logs in? What about when they log out? Do you log every
    time someone changes their account information? Or do you only log errors and
    exceptions? I can’t answer those questions for you directly, but I can offer a method of keeping a recent list of log messages in Redis, which will let you get a snapshot view of your logs at any time.

    To keep a recent list of logs, we’ll LPUSH log messages to a LIST and then trim that
    LIST to a fixed size. Later, if we want to read the log messages, we can perform a simple
    LRANGE to fetch the messages. We’ll take a few extra steps to support different
    named log message queues and to support the typical log severity levels, but you can
    remove either of those in your own code if you need to. The code for writing recent
    logs to Redis is shown in the next listing.

    Listing 5.1The log_recent() function
    SEVERITY = {
       logging.DEBUG: 'debug',
       logging.INFO: 'info',
       logging.WARNING: 'warning',
       logging.ERROR: 'error',
    logging.CRITICAL: 'critical',
    SEVERITY.update((name, name) for name in SEVERITY.values())

    Set up a mapping that should help turn most logging severity levels into something consistent.

    def log_recent(conn, name, message, severity=logging.INFO, pipe=None):

       severity = str(SEVERITY.get(severity, severity)).lower()

    Actually try to turn a logging level into a simple string.

       destination = 'recent:%s:%s'%(name, severity)

    Create the key that messages will be written to.

       message = time.asctime() + ' ' + message

    Add the current time so that we know when the message was sent.

       pipe = pipe or conn.pipeline()

       pipe.lpush(destination, message)

    Add the message to the beginning of the log list.

       pipe.ltrim(destination, 0, 99)

    Trim the log list to only include the most recent 100 messages.


    Execute the two commands.

    Aside from the part that handles turning the different log levels into useful strings like
    info and debug, the log_recent() function is simple—a quick LPUSH followed by an
    LTRIM. Now that you have a better idea of what’s going on right now, can we discover
    the most common (and maybe the most important) messages?