Documentation - Redise Pack

A guide to Redise Pack installation, operation and administration

open all | close all

11.1.1 Loading Lua scripts into Redis

Some older (and still used) Python Redis libraries for Redis 2.6 don’t yet offer the capability to load or execute Lua scripts directly, so we’ll spend a few moments to create a loader for the scripts. To load scripts into Redis, there’s a two-part command called SCRIPT LOAD that, when provided with a string that’s a Lua script, will store the script for later execution and return the SHA1 hash of the script. Later, when we want to execute that script, we run the Redis command EVALSHA with the hash that was returned by Redis, along with any arguments that the script needs.

Our code for doing these operations will be inspired by the current Python Redis code. (We use our method primarily because it allows for using any connection we want without having to explicitly create new scripting objects, which can be useful when dealing with server sharding.) When we pass a string to our script_load() function, it’ll create a function that can later be called to execute the script in Redis. When calling the object to execute the script, we must provide a Redis connection, which will then call SCRIPT LOAD on its first call, followed by EVALSHA for all future calls. The script_load() function is shown in the following listing.

Listing 11.1A function that loads scripts to be called later
def script_load(script):
    sha = [None]

Store the cached SHA1 hash of the result of SCRIPT LOAD in a list so we can change it later from within the call() function.

    def call(conn, keys=[], args=[], force_eval=False):

When calling the loaded script, we must provide a connection, a set of keys that the script will manipulate, and any other arguments to the function.

            if not force_eval:
                if not sha[0]:

We’ll only try loading the script if the SHA1 hash isn’t cached.

                        sha[0] = conn.execute_command(
                            "SCRIPT", "LOAD", script, parse="LOAD")

                try:
                            return conn.execute_command(
                               "EVALSHA", sha[0], len(keys), *(keys+args))

Execute the command from the cached SHA1.

                        except redis.exceptions.ResponseError as msg:
                            if not msg.args[0].startswith("NOSCRIPT"):
                                raise

If the error was unrelated to a missing script, raise the exception again. If we received a script-related error, or if we need to force-execute the script, directly execute the script, which will automatically cache the script on the server (with the same SHA1 that we’ve already cached) when done.

                return conn.execute_command(
                    "EVAL", script, len(keys), *(keys+args))
            return call

Return the function that automatically loads and executes scripts when called.

You’ll notice that in addition to our SCRIPT LOAD and EVALSHA calls, we captured an exception that can happen if we’ve cached a script’s SHA1 hash locally, but the server doesn’t know about it. This can happen if the server were to be restarted, if someone had executed the SCRIPT FLUSH command to clean out the script cache, or if we provided connections to two different Redis servers to our function at different times. If we discover that the script is missing, we execute the script directly with EVAL, which caches the script in addition to executing it. Also, we allow clients to directly execute the script, which can be useful when executing a Lua script as part of a transaction or other pipelined sequence.

KEYS AND ARGUMENTS TO LUA SCRIPTSBuried inside our script loader, you may have noticed that calling a script in Lua takes three arguments. The first is a Redis connection, which should be standard by now. The second argument is a list of keys. The third is a list of arguments to the function.

The difference between keys and arguments is that you’re supposed to pass all of the keys that the script will be reading or writing as part of the keys argument. This is to potentially let other layers verify that all of the keys are on the same shard if you were using multiserver sharding techniques like those described in chapter 10.

When Redis cluster is released, which will offer automatic multiserver sharding, keys will be checked before a script is run, and will return an error if any keys that aren’t on the same server are accessed.

The second list of arguments has no such limitation and is meant to hold data
to be used inside the Lua call.

Let’s try it out in the console for a simple example to get started.

>>> ret_1 = script_load("return 1")

Most uses will load the script and store a reference to the returned function.

>>> ret_1(conn)

We can then call the function by passing the connection object and any desired arguments.

1L

Results will be returned and converted into the appropriate Python types, when possible.

As you can see in this example, we created a simple script whose only purpose is to return a value of 1. When we call it with the connection object, the script is loaded and then executed, resulting in the value 1 being returned.

RETURNING NON-STRING AND NON-INTEGER VALUES FROM LUA

Due to limitations in how Lua allows data to be passed in and out of it, some data types that are available in Lua aren’t allowed to be passed out, or are altered before being returned. Table 11.1 shows how this data is altered upon return.

Table 11.1 Values returned from Lua and what they’re translated into
Lua value What happens during conversion to Python
true Turns into 1
false Turns into None
nil Doesn’t turn into anything, and stops remaining values in a table from being returned
1.5 (or any other float) Fractional part is discarded, turning it into an integer
1e30 (or any other large float) Is turned into the minimum integer for your version of Python
"strings" Unchanged
1 (or any other integer +/-253-1) Integer is returned unchanged

Because of the ambiguity that results when returning a variety of data types, you should do your best to explicitly return strings whenever possible, and perform any parsing manually. We’ll only be returning Booleans, strings, integers, and Lua tables (which are turned into Python lists) for our examples.

Now that we can load and execute scripts, let’s get started with a simple example from chapter 8, creating a status message.