EBOOK – REDIS IN ACTION

This book covers the use of Redis, an in-memory database/data structure server.

open all | close all

8.2 Home timeline

When people visit Twitter after logging in, the first view that they see is what’s referred
to as their home timeline. This is a list of status messages that have been posted by the
user and all of the people they’re following. As the primary entry point to what users
see, this data should be as easy to retrieve as possible.

In this section, we’ll talk about the data to be stored in the home timeline and how
to fetch information to display the home timeline quickly. We’ll also talk about other
important status message timelines.

Figure 8.3When someone visits their home timeline on a site like Twitter, they see the most recently posted messages that people they follow have written. This information is stored as a ZSET of status ID/timestamp pairs. Timestamp information provides the sort order, and the status ID shows us what information to pull in a second step.

As mentioned earlier in this chapter, we want to be able to fetch all of the data required for a given view as quickly as possible. For the home timeline, which will store the list of status messages that have been posted by the people that the current user is following, we’ll use a ZSET to store status IDs as ZSET members, with the timestamp of when the message was posted being used as the score. Figure 8.3 shows an example
home timeline.

Because the home timeline is just referencing status messages—it doesn’t contain the status messages themselves—our function to fetch the most recently posted status messages must also fetch the status message data. The next listing shows the code to fetch a page of messages from the home timeline.

Listing 8.3A function to fetch a page of recent status messages from a timeline
def get_status_messages(conn, uid, timeline='home:', page=1, count=30):

We’ll take an optional “timeline” argument, as well as page size and status message counts.

    statuses = conn.zrevrange(
        '%s%s'%(timeline, uid), (page-1)*count, page*count-1)

Fetch the most recent status IDs in the timeline.

    pipeline = conn.pipeline(True)
    for id in statuses:
        pipeline.hgetall('status:%s'%id)

Actually fetch the status messages themselves.

    return filter(None, pipeline.execute())

Filter will remove any “missing” status messages that had been previously deleted.

That function will fetch status messages in reverse chronological order from the provided
timeline, which defaults to the home timeline.

A second important timeline is the timeline of posts that a user has posted. Where
the home timeline includes posts from other people, the user’s timeline will include
only those posts from the user. These timelines can be seen when visiting a user’s profile,
and are the primary entry point for finding someone interesting. To fetch a page
of statuses from a given user, we can call the same get_messages() function, but we’ll
pass profile: as the timeline argument in our call.

Now that a user can fetch the home timeline, we should discuss how to manage the
list of users that someone is following, and the list of users that are following them.

Figure 8.4To know who’s following a user, we store user ID/timestamp pairs in a ZSET. The user IDs are the people who’re following that user, and the timestamp is when they started following the user. Similarly, the users that a user is following are stored as a ZSET of user ID/timestamp pairs of the user ID of the person being followed, and the timestamp of when the user followed them.