e-Book - Redis in Action

This book covers the use of Redis, an in-memory database/data structure server.
  • Foreword
  • Preface
  • Acknowledgments
  • About this Book
  • About the Cover Illustration
  • Part 1: Getting Started
  • Part 2: Core concepts
  • Part 3: Next steps
  • Appendix A
  • Appendix B
  • Buy the paperback

    8.3 Followers/following lists

    One of the primary services of a platform like Twitter is for users to share their
    thoughts, ideas, and dreams with others. Following someone means that you’re interested
    in reading about what they’re saying, with the hope that others will want to follow
    you.

    In this section, we’ll discuss how to manage the lists of users that each user follows,
    and the users that follow them. We’ll also discuss what happens to a user’s home timeline
    when they start or stop following someone.

    When we looked at the home and profile timelines in the last section, we stored status
    IDs and timestamps in a ZSET. To keep a list of followers and a list of those people
    that a user is following, we’ll also store user IDs and timestamps in ZSETs as well, with
    members being user IDs, and scores being the timestamp of when the user was followed.
    Figure 8.4 shows an example of the followers and those that a user is following.

    As we start or stop following a user, there are following and followers ZSETs that
    need to be updated, as well as counts in the two user profile HASHes. After those ZSETs
    and HASHes have been updated, we then need to copy the newly followed user’s status
    message IDs from their profile timeline into our home timeline. This is to ensure that
    after we’ve followed someone, we get to see their status messages immediately. The
    next listing shows the code for following someone.

    Listing 8.4Update the following user’s home timeline
    HOME_TIMELINE_SIZE = 1000
    def follow_user(conn, uid, other_uid):
    
        fkey1 = 'following:%s'%uid
        fkey2 = 'followers:%s'%other_uid

    Cache the following and followers key names.

        if conn.zscore(fkey1, other_uid):
            return None

    If the other_uid is already being followed, return.

        now = time.time()
        pipeline = conn.pipeline(True)
    
        pipeline.zadd(fkey1, other_uid, now)
        pipeline.zadd(fkey2, uid, now)
    

    Add the uids to the proper following and followers ZSETs.

            pipeline.zcard(fkey1)
            pipeline.zcard(fkey2)
    

    Find the size of the following and followers ZSETs.

            pipeline.zrevrange('profile:%s'%other_uid,
                0, HOME_TIMELINE_SIZE-1, withscores=True)
    

    Fetch the most recent HOME_TIMELINE_SIZE status messages from the newly followed user’s profile timeline.

            following, followers, status_and_score = pipeline.execute()[-3:]
            pipeline.hset('user:%s'%uid, 'following', following)
            pipeline.hset('user:%s'%other_uid, 'followers', followers)
    

    Update the known size of the following and followers ZSETs in each user’s HASH.

            if status_and_score:
    
                pipeline.zadd('home:%s'%uid, **dict(status_and_score))
            pipeline.zremrangebyrank('home:%s'%uid, 0, -HOME_TIMELINE_SIZE-1)

    Update the home timeline of the following user, keeping only the most recent 1000 status messages.

            pipeline.execute()
    
            return True
    

    Return that the user was correctly followed.

    CONVERTING A LIST OF TUPLES INTO A DICTIONARYAs part of our follow_user() function, we fetched a list of status message IDs along with their timestamp scores. Because this is a sequence of pairs, we can pass them directly to the dict() type, which will create a dictionary of keys and values, as passed.

    This function proceeds in the way we described earlier: we add the appropriate user
    IDs to the following and followers ZSETs, get the size of the following and followers
    ZSETs, and fetch the recent status message IDs from the followed user’s profile timeline.
    After we’ve fetched all of the data, we then update counts inside the user profile
    HASHes, and update the following user’s home timeline.

    After following someone and reading their status messages for a while, we may get
    to a point where we decide we don’t want to follow them anymore. To stop following
    someone, we perform essentially the reverse operations of what we’ve discussed:
    removing UIDs from followers and following lists, removing status messages, and again
    updating the followers/following counts. The code to stop following someone is
    shown in the following listing.

    Listing 8.5A function to stop following a user
    def unfollow_user(conn, uid, other_uid):*
    
        fkey1 = 'following:%s'%uid
        fkey2 = 'followers:%s'%other_uid

    Cache the following and followers key names.

        if not conn.zscore(fkey1, other_uid):
            return None

    If the other_uid isn’t being followed, return.

        pipeline = conn.pipeline(True)
    
        pipeline.zrem(fkey1, other_uid)
        pipeline.zrem(fkey2, uid)
    

    Remove the uids from the proper following and followers ZSETs.

        pipeline.zcard(fkey1)
        pipeline.zcard(fkey2)
    

    Find the size of the following and followers ZSETs.

        pipeline.zrevrange('profile:%s'%other_uid,
            0, HOME_TIMELINE_SIZE-1)
    

    Fetch the most recent HOME_TIMELINE_SIZE status messages from the user that we stopped following.

        following, followers, statuses = pipeline.execute()[-3:]
    
        pipeline.hset('user:%s'%uid, 'following', following)
        pipeline.hset('user:%s'%other_uid, 'followers', followers)
    

    Update the known size of the following and followers ZSETs in each user’s HASH.

        if statuses:
    
            pipeline.zrem('home:%s'%uid, *statuses)

    Update the home timeline, removing any status messages from the previously followed user.

        pipeline.execute()
    
        return True
    

    Return that the unfollow executed successfully.

    In that function, we updated the following and followers lists, updated the followers
    and following counts, and updated the home timeline to remove status messages that
    should no longer be there. As of now, that completes all of the steps necessary to start
    and stop following a user.

    Exercise: Refilling timelines

    When someone stops following another user, some number of status messages will
    be removed from the former follower’s home timeline. When this happens, we can
    either say that it’s okay that fewer than the desired number of status messages are
    in the timeline, or we can make an effort to add status messages from the other people
    that the user is still following. Can you write a function that will add status messages
    to the user’s timeline to keep it full? Hint: You may want to use tasks like we
    defined in section 6.4 to reduce the time it takes to return from an unfollow call.

    Exercise: Lists of users

    In addition to the list of users that someone follows, Twitter also supports the ability
    to create additional named lists of users that include the timeline of posts for
    just those users. Can you update follow_user() and unfollow_user() to take
    an optional “list ID” for storing this new information, create functions to create a
    custom list, and fetch the custom list? Hint: Think of it like a different type of follower.
    Bonus points: can you also update your function from the “Refilling timelines”
    exercise?

    Now that we can start or stop following a user while keeping the home timeline
    updated, it’s time to see what happens when someone posts a new status update.