EBOOK – REDIS IN ACTION

This book covers the use of Redis, an in-memory database/data structure server.

open all | close all

3.1 Strings

You’ll remember from chapters 1 and 2 that STRINGs hold sequences of bytes, not significantly different from strings in many programming languages, or even C/C++– style char arrays. In Redis, STRINGs are used to store three types of values:

  • Byte string values
  • Integer values
  • Floating-point values

Integers and floats can be incremented or decremented by an arbitrary numeric value (integers turning into floats as necessary). Integers have ranges that are equivalent to the platform’s long integer range (signed 32-bit integers on 32-bit platforms, and signed 64-bit integers on 64-bit platforms), and floats have ranges and values limited to IEEE 754 floating-point doubles. This three-way ability to look at the simplest of Redis values can be an advantage; it offers more flexibility in data representation than if only byte string values were allowed.

In this section, we’ll talk about the simplest structure available to Redis, the STRING. We’ll cover the basic numeric increment and decrement operations, followed later by the bit and substring manipulation calls, and you’ll come to understand that even the simplest of structures has a few surprises that can make it useful in a variety of powerful ways.

In table 3.1, you can see the available integer and float increment/decrement operations available on Redis STRINGs.

Table 3.1 Increment and decrement commands in Redis
Command Example use and description
INCR INCR key-name — Increments the value stored at the key by 1
DECR DECR key-name — Decrements the value stored at the key by 1
INCRBY INCRBY key-name amount — Increments the value stored at the key by the provided integer value
DECRBY DECRBY key-name amount — Decrements the value stored at the key by the provided integer value
INCRBYFLOAT INCRBYFLOAT key-name amount — Increments the value stored at the key by the provided float value (available in Redis 2.6 and later)

When setting a STRING value in Redis, if that value could be interpreted as a base-10 integer or a floating-point value, Redis will detect this and allow you to manipulate the value using the various INCR* and DECR* operations. If you try to increment or decrement a key that doesn’t exist or is an empty string, Redis will operate as though that key’s value were zero. If you try to increment or decrement a key that has a value that can’t be interpreted as an integer or float, you’ll receive an error. In the next listing, you can see some interactions with these commands.

Listing 3.1 A sample interaction showing INCR and DECR operations in Redis
>>> conn = redis.Redis()
>>> conn.get('key')

When we fetch a key that doesn’t exist, we get the None value, which isn’t displayed in the interactive console.

>>> conn.incr('key')
1
>>> conn.incr('key', 15)
16

We can increment keys that don’t exist, and we can pass an optional value to increment by more than 1.

>>> conn.decr('key', 5)
11

Like incrementing, decrementing takes an optional argument for the amount to decrement by.

>>> conn.get('key')
'11'

When we fetch the key, it acts like a string.

>>> conn.set('key', '13')
True
>>> conn.incr('key')
14

When we set the key, we can set it as a string, but still manipulate it like an integer.

After reading other chapters, you may notice that we really only call incr(). Internally, the Python Redis libraries call INCRBY with either the optional second value passed, or 1 if the value is omitted. As of this writing, the Python Redis client library supports the full command set of Redis 2.6, and offers INCRBYFLOAT support via an incrbyfloat() method that works the same as incr().

Redis additionally offers methods for reading and writing parts of byte string values (integer and float values can also be accessed as though they’re byte strings, though that use is somewhat uncommon). This can be useful if we were to use Redis STRING values to pack structured data in an efficient fashion, which we’ll talk about in chapter 9. Table 3.2 shows some methods that can be used to manipulate substrings and individual bits of STRINGs in Redis.

Table 3.2 Substring manipulation commands available to Redis
Command Example use and description
APPEND APPEND key-name value — Concatenates the provided value to the string already stored at the given key
GETRANGE GETRANGE key-name start end — Fetches the substring, including all characters from the start offset to the end offset, inclusive
SETRANGE SETRANGE key-name offset value — Sets the substring starting at the provided offset to the given value
GETBIT GETBIT key-name offset — Treats the byte string as a bit string, and returns the value of the bit in the string at the provided bit offset
SETBIT SETBIT key-name offset value — Treats the byte string as a bit string, and sets the value of the bit in the string at the provided bit offset
BITCOUNT BITCOUNT key-name [start end] — Counts the number of 1 bits in the string, optionally starting and finishing at the provided byte offsets
BITOP BITOP operation dest-key key-name [key-name …] — Performs one of the bitwise operations, AND, OR, XOR, or NOT, on the strings provided, storing the result in the destination key

GETRANGE AND SUBSTR In the past, GETRANGE was named SUBSTR, and the Python client continues to use the substr() method name to fetch ranges from the string. When using a version of Redis later than 2.6, you should use the getrange() method, and use substr() for Redis versions before 2.6.

When writing to strings using SETRANGE and SETBIT, if the STRING wasn’t previously long enough, Redis will automatically extend the STRING with nulls before updating and writing the new data. When reading STRINGs with GETRANGE, any request for data beyond the end of the STRING won’t be returned, but when reading bits with GETBIT, any bit beyond the end of the STRING is considered zero. In the following listing, you can see some uses of these STRING manipulation commands.

Listing 3.2 A sample interaction showing substring and bit operations in Redis
>>> conn.append('new-string-key', 'hello ')

Let’s append the string ‘hello ’ to the previously nonexistent key ‘new-string-key’.

6L

When appending a value, Redis returns the length of the string so far.

>>> conn.append('new-string-key', 'world!')
12L

When appending a value, Redis returns the length of the string so far.

>>> conn.substr('new-string-key', 3, 7)

Redis uses 0-indexing, and when accessing ranges, is inclusive of the endpoints by default.

'lo wo'

The string ‘lo wo’ is from the middle of ‘hello world!’

>>> conn.setrange('new-string-key', 0, 'H')

Let’s set a couple string ranges.

12

When setting a range inside a string, Redis also returns the total length of the string.

>>> conn.setrange('new-string-key', 6, 'W')
12
>>> conn.get('new-string-key')

Let’s see what we have now!

'Hello World!'

Yep, we capitalized our H and W.

>>> conn.setrange('new-string-key', 11, ', how are you?')

With setrange, we can replace anywhere inside the string, and we can make the string longer.

25
>>> conn.get('new-string-key')
'Hello World, how are you?'

We replace the exclamation point and add more to the end of the string.

>>> conn.setbit('another-key', 2, 1)

If we write to a bit beyond the size of the string, it’s filled with nulls.

0

Setting bits also returns the value of the bit before it was set.

>>> conn.setbit('another-key', 7, 1)
0
>>> conn.get('another-key')

If you want to interpret the bits stored in Redis, remember that offsets into bits are from the highest-order to the lowest-order.

'!'

We set bits 2 and 7 to 1, which gave us ‘!’, or character 33.

In many other key-value databases, data is stored as a plain string with no opportunities for manipulation. Some other key-value databases do allow you to prepend or append bytes, but Redis is unique in its ability to read and write substrings. In many ways, even if Redis only offered STRINGs and these methods to manipulate strings, Redis would be more powerful than many other systems; enterprising users could use the substring and bit manipulation calls along with WATCH/MULTI/EXEC (which we’ll briefly introduce in section 3.7.2, and talk about extensively in chapter 4) to build arbitrary data structures. In chapter 9, we’ll talk about using STRINGs to store a type of simple mappings that can greatly reduce memory use in some situations.

With a little work, we can store some types of sequences, but we’re limited in the kinds of manipulations we can perform. But if we use LISTs, we have a wider range of commands and ways to manipulate LIST items.