This book covers the use of Redis, an in-memory database/data structure server.

open all | close all

5.1.1 Recent logs

When building a system, knowing what’s important to record can be difficult. Do you
record every time someone logs in? What about when they log out? Do you log every
time someone changes their account information? Or do you only log errors and
exceptions? I can’t answer those questions for you directly, but I can offer a method of keeping a recent list of log messages in Redis, which will let you get a snapshot view of your logs at any time.

To keep a recent list of logs, we’ll LPUSH log messages to a LIST and then trim that
LIST to a fixed size. Later, if we want to read the log messages, we can perform a simple
LRANGE to fetch the messages. We’ll take a few extra steps to support different
named log message queues and to support the typical log severity levels, but you can
remove either of those in your own code if you need to. The code for writing recent
logs to Redis is shown in the next listing.

Listing 5.1The log_recent() function
   logging.DEBUG: 'debug',
   logging.INFO: 'info',
   logging.WARNING: 'warning',
   logging.ERROR: 'error',
logging.CRITICAL: 'critical',
SEVERITY.update((name, name) for name in SEVERITY.values())

Set up a mapping that should help turn most logging severity levels into something consistent.

def log_recent(conn, name, message, severity=logging.INFO, pipe=None):

   severity = str(SEVERITY.get(severity, severity)).lower()

Actually try to turn a logging level into a simple string.

   destination = 'recent:%s:%s'%(name, severity)

Create the key that messages will be written to.

   message = time.asctime() + ' ' + message

Add the current time so that we know when the message was sent.

   pipe = pipe or conn.pipeline()

   pipe.lpush(destination, message)

Add the message to the beginning of the log list.

   pipe.ltrim(destination, 0, 99)

Trim the log list to only include the most recent 100 messages.


Execute the two commands.

Aside from the part that handles turning the different log levels into useful strings like
info and debug, the log_recent() function is simple—a quick LPUSH followed by an
LTRIM. Now that you have a better idea of what’s going on right now, can we discover
the most common (and maybe the most important) messages?