EBOOK – REDIS IN ACTION

This book covers the use of Redis, an in-memory database/data structure server.

open all | close all

4.4.3 Purchasing items

To process the purchase of an item, we first WATCH the market and the user who’s buying
the item. We then fetch the buyer’s total funds and the price of the item, and verify
that the buyer has enough money. If they don’t have enough money, we cancel the
transaction. If they do have enough money, we perform the transfer of money
between the accounts, move the item into the buyer’s inventory, and remove the item
from the market. On WATCH error, we retry for up to 10 seconds in total. We can see
the function which handles the purchase of an item in the following listing.

Listing 4.6The purchase_item() function
def purchase_item(conn, buyerid, itemid, sellerid, lprice):
   buyer = "users:%s"%buyerid
   seller = "users:%s"%sellerid
   item = "%s.%s"%(itemid, sellerid)
   inventory = "inventory:%s"%buyerid
   end = time.time() + 10
   pipe = conn.pipeline()

   while time.time() < end:
      try:

         pipe.watch("market:", buyer)

Watch for changes to the market and to the buyer’s account information.

         price = pipe.zscore("market:", item)
         funds = int(pipe.hget(buyer, "funds"))
         if price != lprice or price > funds:
            pipe.unwatch()

Check for a sold/repriced item or insufficient funds.

            return None

         pipe.multi()
         pipe.hincrby(seller, "funds", int(price))
         pipe.hincrby(buyer, "funds", int(-price))
         pipe.sadd(inventory, itemid)
         pipe.zrem("market:", item)
         pipe.execute()

Transfer funds from the buyer to the seller, and transfer the item to the buyer.

         return True

      except redis.exceptions.WatchError:
         pass

Retry if the buyer’s account or the market changed.

   return False

To purchase an item, we need to spend more time preparing the data, and we need to
watch both the market and the buyer’s information. We watch the market to ensure
that the item can still be bought (or that we can notice that it has already been
bought), and we watch the buyer’s information to verify that they have enough money. When we’ve verified that the item is still there, and that the buyer has enough money,
we go about actually moving the item into their inventory, as well as moving money
from the buyer to the seller.

After seeing the available items in the market, Bill (user 27) decides that he wants
to buy ItemM from Frank through the marketplace. Let’s follow along to see how our
data changes through figures 4.5 and 4.6.

If either the market ZSET or Bill’s account information changes between our WATCH
and our EXEC, the purchase_item() function will either retry or abort, based on how
long it has been trying to purchase the item, as shown in listing 4.6.

WHY DOESN’T REDIS IMPLEMENT TYPICAL LOCKING?When accessing data for
writing (SELECT FOR UPDATE in SQL), relational databases will place a lock on
rows that are accessed until a transaction is completed with COMMIT or ROLLBACK.
If any other client attempts to access data for writing on any of the same
rows, that client will be blocked until the first transaction is completed. This
form of locking works well in practice (essentially all relational databases
implement it), though it can result in long wait times for clients waiting to
acquire locks on a number of rows if the lock holder is slow.

Because there’s potential for long wait times, and because the design of Redis
minimizes wait time for clients (except in the case of blocking LIST pops),
Redis doesn’t lock data during WATCH. Instead, Redis will notify clients if someone
else modified the data first, which is called optimistic locking (the actual
locking that relational databases perform could be viewed as pessimistic). Optimistic
locking also works well in practice because clients are never waiting on
the first holder of the lock; instead they retry if some other client was faster.

Figure 4.5Before the item can be purchased, we must watch the market and the buyer’s information to verify that the item is still available, and that the buyer has enough money.
Figure 4.6In order to complete the item purchase, we must actually transfer money from the buyer to the seller, and we must remove the item from the market while adding it to the buyer’s inventory.

In this section, we’ve discussed combining WATCH, MULTI, and EXEC to handle the
manipulation of multiple types of data so that we can implement a marketplace. Given
this functionality as a basis, it wouldn’t be out of the question to make our marketplace
into an auction, add alternate sorting options, time out old items in the market,
or even add higher-level searching and filtering based on techniques discussed in
chapter 7.

As long as we consistently use transactions in Redis, we can keep our data from
being corrupted while being operated on by multiple clients. Let’s look at how we can
make our operations even faster when we don’t need to worry about other clients
altering our data.