Documentation - Redise Pack

A guide to Redise Pack installation, operation and administration

open all | close all

8.3 Followers/following lists

One of the primary services of a platform like Twitter is for users to share their
thoughts, ideas, and dreams with others. Following someone means that you’re interested
in reading about what they’re saying, with the hope that others will want to follow
you.

In this section, we’ll discuss how to manage the lists of users that each user follows,
and the users that follow them. We’ll also discuss what happens to a user’s home timeline
when they start or stop following someone.

When we looked at the home and profile timelines in the last section, we stored status
IDs and timestamps in a ZSET. To keep a list of followers and a list of those people
that a user is following, we’ll also store user IDs and timestamps in ZSETs as well, with
members being user IDs, and scores being the timestamp of when the user was followed.
Figure 8.4 shows an example of the followers and those that a user is following.

As we start or stop following a user, there are following and followers ZSETs that
need to be updated, as well as counts in the two user profile HASHes. After those ZSETs
and HASHes have been updated, we then need to copy the newly followed user’s status
message IDs from their profile timeline into our home timeline. This is to ensure that
after we’ve followed someone, we get to see their status messages immediately. The
next listing shows the code for following someone.

Listing 8.4Update the following user’s home timeline
HOME_TIMELINE_SIZE = 1000
def follow_user(conn, uid, other_uid):
    fkey1 = 'following:%s'%uid
    fkey2 = 'followers:%s'%other_uid

Cache the following and followers key names.

    if conn.zscore(fkey1, other_uid):
        return None

If the other_uid is already being followed, return.

    now = time.time()
    pipeline = conn.pipeline(True)
    pipeline.zadd(fkey1, other_uid, now)
    pipeline.zadd(fkey2, uid, now)

Add the uids to the proper following and followers ZSETs.

        pipeline.zcard(fkey1)
        pipeline.zcard(fkey2)

Find the size of the following and followers ZSETs.

        pipeline.zrevrange('profile:%s'%other_uid,
            0, HOME_TIMELINE_SIZE-1, withscores=True)

Fetch the most recent HOME_TIMELINE_SIZE status messages from the newly followed user’s profile timeline.

        following, followers, status_and_score = pipeline.execute()[-3:]
        pipeline.hset('user:%s'%uid, 'following', following)
        pipeline.hset('user:%s'%other_uid, 'followers', followers)

Update the known size of the following and followers ZSETs in each user’s HASH.

        if status_and_score:
            pipeline.zadd('home:%s'%uid, **dict(status_and_score))
        pipeline.zremrangebyrank('home:%s'%uid, 0, -HOME_TIMELINE_SIZE-1)

Update the home timeline of the following user, keeping only the most recent 1000 status messages.

        pipeline.execute()
        return True

Return that the user was correctly followed.

CONVERTING A LIST OF TUPLES INTO A DICTIONARYAs part of our follow_user() function, we fetched a list of status message IDs along with their timestamp scores. Because this is a sequence of pairs, we can pass them directly to the dict() type, which will create a dictionary of keys and values, as passed.

This function proceeds in the way we described earlier: we add the appropriate user
IDs to the following and followers ZSETs, get the size of the following and followers
ZSETs, and fetch the recent status message IDs from the followed user’s profile timeline.
After we’ve fetched all of the data, we then update counts inside the user profile
HASHes, and update the following user’s home timeline.

After following someone and reading their status messages for a while, we may get
to a point where we decide we don’t want to follow them anymore. To stop following
someone, we perform essentially the reverse operations of what we’ve discussed:
removing UIDs from followers and following lists, removing status messages, and again
updating the followers/following counts. The code to stop following someone is
shown in the following listing.

Listing 8.5A function to stop following a user
def unfollow_user(conn, uid, other_uid):*
    fkey1 = 'following:%s'%uid
    fkey2 = 'followers:%s'%other_uid

Cache the following and followers key names.

    if not conn.zscore(fkey1, other_uid):
        return None

If the other_uid isn’t being followed, return.

    pipeline = conn.pipeline(True)
    pipeline.zrem(fkey1, other_uid)
    pipeline.zrem(fkey2, uid)

Remove the uids from the proper following and followers ZSETs.

    pipeline.zcard(fkey1)
    pipeline.zcard(fkey2)

Find the size of the following and followers ZSETs.

    pipeline.zrevrange('profile:%s'%other_uid,
        0, HOME_TIMELINE_SIZE-1)

Fetch the most recent HOME_TIMELINE_SIZE status messages from the user that we stopped following.

    following, followers, statuses = pipeline.execute()[-3:]
    pipeline.hset('user:%s'%uid, 'following', following)
    pipeline.hset('user:%s'%other_uid, 'followers', followers)

Update the known size of the following and followers ZSETs in each user’s HASH.

    if statuses:
        pipeline.zrem('home:%s'%uid, *statuses)

Update the home timeline, removing any status messages from the previously followed user.

    pipeline.execute()
    return True

Return that the unfollow executed successfully.

In that function, we updated the following and followers lists, updated the followers
and following counts, and updated the home timeline to remove status messages that
should no longer be there. As of now, that completes all of the steps necessary to start
and stop following a user.

Exercise: Refilling timelines

When someone stops following another user, some number of status messages will
be removed from the former follower’s home timeline. When this happens, we can
either say that it’s okay that fewer than the desired number of status messages are
in the timeline, or we can make an effort to add status messages from the other people
that the user is still following. Can you write a function that will add status messages
to the user’s timeline to keep it full? Hint: You may want to use tasks like we
defined in section 6.4 to reduce the time it takes to return from an unfollow call.

Exercise: Lists of users

In addition to the list of users that someone follows, Twitter also supports the ability
to create additional named lists of users that include the timeline of posts for
just those users. Can you update follow_user() and unfollow_user() to take
an optional “list ID” for storing this new information, create functions to create a
custom list, and fetch the custom list? Hint: Think of it like a different type of follower.
Bonus points: can you also update your function from the “Refilling timelines”
exercise?

Now that we can start or stop following a user while keeping the home timeline
updated, it’s time to see what happens when someone posts a new status update.