download & runRedis Labs Enterprise Cluster

Learn more

free download

Connect & useRedis Cloud





By creating your account, you agree to our Terms of Use

Forgot your password?

Not Registered?

Forgot your password?

Enter your email address below to have your password reset. We'll send you an email with a link to change your password.

Your IP address will be logged and monitored for this request.

Cancel

Your password has reset!

We have sent you an email with a link to change your password.

OK

Blog

Posted on by Dvir Volk

Writing Redis Modules

I've been using Redis for about six years now, and over the years I’ve exploited its few data types to do some pretty versatile things such as geographical queries, text search, machine learning and more.

But over the years, I can't even count the number of times I’ve found myself (as did many other users) wishing that Redis would have this or other data structure, command or capability it is lacking. Not having them forced user to resort to less elegant or less efficient methods to achieve our goals with Redis. And while Lua was a step in the right direction, it has its limitations.

But all that is changing, since Redis now has a (not yet stable) module system that allows developers to write C libraries, which add new capabilities and data structures to Redis - in speeds similar to normal Redis commands.

I personally think this is the most exciting new feature to hit Redis in a long time, and it will go beyond an API - dramatically shaping the dynamics and the usage of Redis in the years to come.

Over the past couple of months, while Salvatore has been working on the pretty big Modules API, we here at Redis Labs have been busy experimenting with it, creating modules that add things like authentication, probabilistic data structures and full-text search — all using new data structures and adding new commands to Redis.

So I wanted to share the lessons I've learned as a sort of basic guide to writing modules. The documentation of the API is very good, but also very long. So take this guide as a TL;DR guide to module writing.

1. Not Your Grandfather's LUA Script

Let's start with the basics: Modules are basically C shared libraries (.so files) that Redis can load at runtime or on startup. They can register new commands that Redis doesn't support, and can access Redis' data to do their thing. But there are a few key differences that make modules a lot more powerful:

2. What's in a Module

What modules basically contain are command handlers. These are C functions with the following signature:

int MyCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)

The idea is pretty simple, as can be seen from the signature: The function returns an integer, either OK or ERR. Usually returning OK, even if returning an error to the user, is fine.

The command handler accepts a RedisModuleCtx* object. This object is opaque to the module developer, but internally it contains the calling client's state, and even internal memory management, which we'll get to later.

Next it receives argv and argc which are basically the argument the user has passed to the command being called. The first argument is the name of the call itself, and the rest are simply parsed arguments from the Redis protocol.

Notice that they are received as RedisModuleString objects, which again, are opaque. They can be converted to normal C strings with zero copy if manipulation is needed.

To activate the module's commands, the standard entry point for a module is a function called `int RedisModule_OnLoad(RedisModuleCtx *ctx)`. This function tells Redis what commands are in the module and maps them to their handler.

Let’s Write a Module

OK, let’s write a Redis Module. We’ll focus on a very simple example of a module that implements a new Redis command - HGETSET <key> <element> <new value>. It’s basically a combination of HGET and HSET, getting the current value in a HASH object, and setting a new value instead of it atomically. This is pretty basic, and can be done with a simple transaction or a LUA script, but it has the advantage of being really simple.

1. Let’s start with a bare command handler:

int HGetSetCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
    return REDISMODULE_OK;
}

Again, this currently does nothing, it just returns the OK code. So let’s give it some substance.

2. Validate the arguments

Remember, our command is HGETSET <key> <element> <new value>, meaning it will always have four arguments in argv. So let’s make sure this is indeed what happens:

/**
HGETSET <key> <element> <new value>
*/

int HGetSetCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {

    if (argc != 4) {
        return RedisModule_WrongArity(ctx);
    }

    Return REDISMODULE_OK;
}

RedisModule_WrongArity will return a standard error to the client in the form of:

(error) ERR wrong number of arguments for 'get' command

3. Activate AutoMemory

One of the great features of the Redis Modules API is automatic resource and memory management. While the module author can allocate and free memory independently, creating Redis resources is managed, and Redis strings, keys and responses allocated during the handler’s lifecycle are freed automatically, if we call RedisModule_AutoMemory.

RedisModule_AutoMemory(ctx);

4. Performing a Redis call

Now we’ll run the first of two Redis calls: HGET. We pass argv[1] and argv[2] which are the key and element as the arguments. We use the generic RedisModule_Call command, which simply allows the module developer to call any existing Redis commands, much like a LUA script.

RedisModuleCallReply *rep = RedisModule_Call(ctx, "HGET", "ss", argv[1], argv[2]);

// And let's make sure it's not an error
if (RedisModule_CallReplyType(rep) == REDISMODULE_REPLY_ERROR) {
    return RedisModule_ReplyWithCallReply(ctx, srep);
}

Notice that RedisModule_Call’s third argument, “ss”, denotes how Redis should treat the passed variadic arguments to the function. “ss” means “two RedisModuleString objects.” Other specifiers are “c” for a c-string, “d” for double, “l” for long, and “b” for a c-buffer (a string followed by its length).

Now let’s perform the second Redis call, HSET:

RedisModuleCallReply *srep = RedisModule_Call(ctx, "HSET", "sss", argv[1], argv[2], argv[3]);

if (RedisModule_CallReplyType(srep) == REDISMODULE_REPLY_ERROR) {
    return RedisModule_ReplyWithCallReply(ctx, srep);
}

Which is done much like the HGET command, except we pass three arguments to it.

5. Return the results

In this simple case, we just need to return the result of HGET, or the value before we changed it. This is done using a simple function - RedisModule_ReplyWithCallReply, which forwards the reply object to the client:

RedisModule_ReplyWithCallReply(ctx, rep);

    return REDISMODULE_OK;

}

And that’s it! Our command handler is ready; we just need to register our module and command handler properly.

6. Initialize the module

The entry point for all Redis Modules is a function called RedisModule_OnLoad, which the developer has to implement. It registers and initializes the module, and registers its commands with Redis so that they can be called.

Initializing our module works like so:

int RedisModule_OnLoad(RedisModuleCtx *ctx) {

    // Register the module itself - it's called example and has an API version of 1
    if (RedisModule_Init(ctx, "example", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) {
        return REDISMODULE_ERR;
    }

    // register our command - it is a write command, with one key at argv[1]
    if (RedisModule_CreateCommand(ctx, "HGETSET", HGetSetCommand, "write", 1, 1, 1) == REDISMODULE_ERR) {
        return REDISMODULE_ERR;
    }

    return REDISMODULE_OK;
}

And that’s about it! Our module is done.

7. A Word on module building

All that’s left is to compile our module. I won’t go into the specifics of creating a makefile for it, but what you need to know is that Redis Modules require no special linking. Once you’ve included the "redismodule.h" file in your module’s files, and implemented the entry point function, that’s all Redis needs to load your module; any other linking is up to you.

Provided here are the commands needed to compile our basic module with gcc:

On Linux:

$ gcc -fPIC -std=gnu99 -c -o module.o module.c
$ ld -o module.so module.o -shared -Bsymbolic -lc

On OSX:

$ gcc -dynamic -fno-common -std=gnu99 -c -o module.o module.c
$ ld -o module.so module.o -bundle -undefined dynamic_lookup -lc

8. Loading our module

Once you’ve built your module, you need to load it. Assuming you’ve downloaded Redis from its latest unstable build (which supports Modules), you just run it with the --loadmodule command-line argument:

redis-server --loadmodule /path/to/module.so

And that’s it! Redis is now running and has loaded our module. We can simply connect with redis-cli and run our commands!

Getting the source

The full source code detailed here can be found as part of RedisModuleSDK, which includes a Module project template, makefile, and a utility library with functions automating some of the more boring stuff around writing modules, that are not included in the original API. You do not have to use it, but feel free to.

Remarkably Simple. Epic Performance.

60

Subscribe

Get the latest blog posts by email

8 32

Redis Watch

Get the hottest Redis news with our
periodic newsletter!

27

Interact with Redis

Enter your Redis commands,
see how fast it responds!

Redis Cloud > |
37 59

Gartner Acknowledges Redis Labs as a Leader

In the 2015 Magic Quadrant for Operational Database Management Systems (ODBMS)

Leaders
47

User Satisfaction and
Market Presence -
Redis Ranked as
#1 NoSQL

g2crowd
64

#2 Database in
User Satisfaction

user satisfaction

g2crowd.com

63

Top 10 Data Stores

Redis in containers
1st
MySQL
2nd
Redis
3rd
MongoDB
4th
PostgreSQL

stackshare.io

45
62

Top 10 Technologies
on Docker

Percent of companies running this technology

Redis in containers

datadoghq.com

28

See Our Open Source Contributions

48

NoSQL Databases
in Containers

Redis in containers

DevOps.com & ClusterHQ.com

11

About Us

Redis Labs is the open source home and commercial provider of Redis, a database benchmarked as the world’s fastest. Gartner has named the company as a Leader in it's 2015 ODBMS Magic Quadrant. Redis Labs' software and service solutions power cutting edge applications with blazing fast enterprise-class Redis and are trusted by thousands of customers for high performance, seamless scalability, true high availability and best-in-class expertise. Redis is ranked the #1 NoSQL (and #2 database) in User Satisfaction and Market Presence by G2 Crowd, the top database technology on Docker by Datadog, the most popular NoSQL database in containers by DevOps.com and ClusterHQ, the #1 NoSQL among Top 10 Data Stores by Stackshare and both the fastest growing database since January 2013 and one of the top three NoSQL databases by DB-engines.

66

Why Redis Labs?

Watch the video
61 53 Stance

"How We Perform
in Peak Times at Sub-millisecond Latencies with Redis Labs"

21

Latest News

19 18 44 7 55
60

Subscribe

Get the latest blog posts by email

8 32

Redis Watch

Get the hottest Redis news with our
periodic newsletter!

27

Interact with Redis

Enter your Redis commands,
see how fast it responds!

Redis Cloud > |
37 59

Gartner Acknowledges Redis Labs as a Leader

In the 2015 Magic Quadrant for Operational Database Management Systems (ODBMS)

Leaders