11.3.2 Improving the marketplace, again
In section 6.2, we revisited the marketplace example we introduced in section 4.4, replacing our use of WATCH, MULTI, and EXEC with locks, and showed how using coarse-grained and fine-grained locks in Redis can help reduce contention and improve performance.
In this section, we’ll again work on the marketplace, further improving performance by removing the locks completely and moving our code into a Lua script.
Let’s first look at our marketplace code with a lock. As a review of what goes on, first the lock is acquired, and then we watch the buyer’s user information HASH and let the buyer purchase the item if they have enough money. The original function from section 6.2 appears next.
Despite using a lock, we still needed to watch the buyer’s user information HASH to ensure that enough money was available at the time of purchase. And because of that, we have the worst of both worlds: chunks of code to handle locking, and other chunks of code to handle the potential WATCH errors. Of all of the solutions we’ve seen so far, this one is a prime candidate for rewriting in Lua.
When rewriting this function in Lua, we can do away with the locking, the WATCH/MULTI/EXEC transactions, and even timeouts. Our code becomes straightforward and simple: make sure the item is available, make sure the buyer has enough money, transfer the item to the buyer, and transfer the buyer’s money to the seller. The following listing shows the rewritten item-purchasing function.
Just comparing the two code listings, the Lua-based item-purchase code is far easier to understand. And without the multiple round trips to complete a single purchase (locking, watching, fetching price and available money, then the purchase, and then the unlocking), it’s obvious that purchasing an item will be even faster than the finegrained locking that we used in chapter 6. But how much faster?
Exercise: Rewrite item listing in Lua
We rewrote the purchase-item function in Lua for our benchmarks, but can you rewrite the original item-listing function from section 4.4.2 into Lua? Hint: The source code for this chapter includes the answer to this exercise, just as the source code for each of the other chapters includes the solutions to almost all of the exercises.
We now have an item-purchase function rewritten in Lua, and if you perform the exercise, you’ll also have an item-listing function written in Lua. If you read the hint to our exercise, you’ll notice that we also rewrote the item-listing function in Lua. You may remember that at the end of section 6.2.4, we ran some benchmarks to compare the performance of WATCH/MULTI/EXEC transactions against coarse-grained and finegrained locks. We reran the benchmark for five listing and five buying processes using our newly rewritten Lua versions, to produce the last row in table 11.4.
|Listed items||Bought items||Purchase retries||Average wait per purchase|
|5 listers, 5 buyers, no lock||206,000||<600||161,000||498ms|
|5 listers, 5 buyers, with lock||21,000||20,500||0||14ms|
|5 listers, 5 buyers, with fine-grained lock||116,000||111,000||0||<3ms|
|5 listers, 5 buyers, using Lua||505,000||480,000||0||<1ms|
As in other cases where we’ve moved functionality into Lua, we see a substantial performance increase. Numerically, we see an improvement in listing and buying performance of more than 4.25 times compared with fine-grained locking, and see latencies of under 1 millisecond to execute a purchase (actual latencies were consistently around .61 milliseconds). From this table, we can see the performance advantages of coarse-grained locks over WATCH/MULTI/EXEC, fine-grained locks over coarse-grained locks, and Lua over fine-grained locks. That said, try to remember that while Lua can offer extraordinary performance advantages (and substantial code simplification in some cases), Lua scripting in Redis is limited to data we can access from within Lua and Redis, whereas there are no such limits when using locks or WATCH/MULTI/EXEC transactions.
Now that you’ve seen some of the amazing performance advantages that are available with Lua, let’s look at an example where we can save memory with Lua.