e-Book - Redis in Action

This book covers the use of Redis, an in-memory database/data structure server.
  • Foreword
  • Preface
  • Acknowledgments
  • About this Book
  • About the Cover Illustration
  • Part 1: Getting Started
  • Part 2: Core concepts
  • Part 3: Next steps
  • Appendix A
  • Appendix B
  • Buy the paperback

    5.1 Logging to Redis

    As we build applications and services, being able to discover information about the
    running system becomes increasingly important. Being able to dig into that information
    to diagnose problems, discover problems before they become severe, or even just
    to discover information about users—these all necessitate logging.

    In the world of Linux and Unix, there are two common logging methods. The first
    is logging to a file, where over time we write individual log lines to a file, and every
    once in a while, we write to a new file. Many thousands of pieces of software have been
    written do this (including Redis itself). But this method can run into issues because we
    have many different services writing to a variety of log files, each with a different way
    of rolling them over, and no common way of easily taking all of the log files and doing
    something useful with them.

    Running on TCP and UDP port 514 of almost every Unix and Linux server available
    is a service called syslog, the second common logging method. Syslog accepts log messages
    from any program that sends it a message and routes those messages to various
    on-disk log files, handling rotation and deletion of old logs. With configuration, it can
    even forward messages to other servers for further processing. As a service, it’s far
    more convenient than logging to files directly, because all of the special log file rotation
    and deletion is already handled for us.

    REPLACING SYSLOGWhether you end up using the logging methods
    described here, you owe it to yourself to consider replacing your current syslog
    daemon (which is likely Rsyslogd) with syslog-ng. Having used and configured
    both systems, I find that the configuration language that syslog-ng
    provides for directing log messages is easier to use. And though I don’t have
    the space or time to build it in this book, building a service that consumes syslog
    messages and puts them into Redis is a great way to offer a layer of indirection
    between what needs to happen now for processing a request, and what
    can happen later (like logging or updating counters).

    Having logs available in files on a single server (thanks to syslog forwarding) is a great
    long-term plan with logging (remember to back them up). In this section, we’ll talk
    about using Redis as a way of keeping more time-sensitive logs, to function as a
    replacement for syslog messages being stored in the short term. Our first view into
    changing logs is a continuously updated stream of recent log messages.