June 9th, 2016 – Edition #65

A Newsletter About Everything Redis

Edition #65
June 9th, 2016

#include "i.h"

I’m flabbergasted at issue #3185: Replace “master” and “slave” terms in Redis.

Cheers,
Itamar

Redis Trivia: Redis EXPIRE Illustrated (NSFW) – http://exeypan.deviantart.com/art/Redis-EXPIRE-539267658

Be social, tweet about Redis Watch: I’m reading Redis Watch #65!

Redis Modules

During the last months Redis has been getting a very special treatment that will change the way it is used for ever. I’m referring, of course, to the new Redis Modules API that was unveiled last month by Salvatore Sanfilippo @antirez at RedisConf (sorry for the secrecy, but everything’s worth trying at least once, right?)

TL;DR Redis Modules are dynamic libraries written in C (but in theory in any language with C bindings will do). They are binary-compatible with future Redis versions and allow anyone to add her/his own commands and data structures. Modules access data in Redis either via a high-level interface that’s not unlike the redis.call function in Redis’ Lua scripts, or via a low-level API that performs nearly as fast as native Redis commands. The API also provides support for things like memory management, arguments, replies and responses.

If you’re interested in more details, here’s where you need to go:

  1. Redis Loadable Modules System (12 minutes to read) – Salvatore Sanfilippo @antirez inaugural post about Redis Modules covers everything from the beginning of time (i.e. v1.0) to the present day. /r/redis HN
  2. The unstable branch of Redis’ repository has everything you need to develop a module under the src/modules directory. Specifically:
    1. INTRO.md is an introduction to the Redis Modules API – start here to get a feel for it.
    2. Follow the examples in helloworld.c to see how the different API calls are used in a module.
    3. The TYPES.md document along with hellotype.c dive into the API’s usage when defining new data types.
  3. Creating a redis Module in 15 lines of code! by Dvir Volk @dvirsky is precisely what it says it is – could it be any simpler? HN
  4. Check out some of open source modules developed by Redis Labs @RedisLabs that are available from the Module Hub, including: full text search, image manipulation, password encryption, probabilistic data structure and extended Redis commands. HN.
  5. At this point you’re probably firing up your code editor to write you first module, so congrats and another tip: we’ve put together a devkit with stuff we found useful for module ninjas – clone the RedisModulesSDK repository to start using it.

The Modules’ launch was also picked by bloggers and the media, including:

  1. Redis launches modules to add extensibility to the open source database by Jordan ‘Jaws’ Novet @jordannovet in VentureBeat @VentureBeat
  2. Redis Modules extend NoSQL database’s capabilities, increase developer involvement by Matt Asay @mjasay in TechRepublic @TechRepublic
  3. Redis plants the seeds for an open source ecosystem by syegulalp @syegulalp in InfoWorld @infoworld
  4. Redis aims for an infinite variety of data structures by Adrían Bridgwater @ABridgwater in ITKnowledgeExchange @ITKE
  5. Redis Announces Redis Modules That Facilitate Development Of Additional Use Cases For Redis Database Functionality by Arnal Dayaratna @cloud4computing
  6. (((Hebrew only))) “303 Redis news” podcast with Dvir Volk @dvirsky with orilahav @orilahav and Ran Tavory @rantav at @reversim

Lastly, but inarguably most importantly, there are signs of modules being developed by the community. The first two open source community-developed Redis modules are:

  1. redis-locks by James Edwards is a module that provides a locking mutex – auther is willing to add semaphore too if there’s demand, so show him your love.
  2. redis_reference from Felipe Cruz @felipejcruz is a specialized, non-production module that is very use-case-specific… but still it’s the second ever module so there’s that 🙂

Virtual Moka Pot

Since time immemorial within the Redis community, a Moka Pot is considered a token of appreciation to a member for her/his contributions. I’d like to present this one to Andy McCurdy @andymccurdy for everything he’s done for us with redis-py, the de-facto standard Redis client for #Python. With a first public commit dating back to November 6th 2009, redis-py is one of the first clients ever to be written for Redis and the fact that it had prevailed the tests of time is a testament of Andy’s dedication and skill.

Because OSS never stops, maintaining a popular project can become an extremely demanding task. Recently I OH Andy tweet “Looking for some folks to help maintain redis-py. It’s been neglected for too long. Please email or DM me.” and I know of at least one very capable maintainer (hint: Dvir Volk @dvirsky) who’ll probably be join Andy in maintaining the excellence.

Andy – for giving us the best Redis client ever (and one of the best Python Redis clients out there ;)), may your coffee always taste as good as a cup of Italian Moka, thank you.

int main(int argc, char **argv) {

Redis 3.2.0 is out! (6.379 minutes to read)

After a long time in the making (and partly because of a detour in favor of the modules API) v3.2 is finally out! With Geo, BITFIELD, #Lua replication & debugging and too many more improvements to copy-paste. Also, redis-cli got a revamp and has useful command hints and its own documentation page 🙂

OH Happy b111 Redis! @itamarhaber > When pronounced “K-LEE” it means tool in Hebrew, and in slang it means an extremely capable person

Redis IRC logs

After the experience of doing the first development iteration of the modules API away from the public’s, Lead Dev and his team have decided not to repeat that. As a result, we’ve moved our discussions to the #redis channel on Freenode and the Redis website now logs it.

STREAM data structure for Redis: let’s design it together! #CallForAction

A suggestion by Timothy Downs @timothy_downs at IRC triggered what is snowballing towards being the 6th core Redis data structure – your opinion matters so jump into to the discussion now. HN

Pattern: weighted random selection of an element #RedisPatterns

A once-you-see-it-you-it-is-so-self-explanatory-that-you-are-ashamed-you-did-not-think-of-this-yourself addition to the documentation that I’m sure your already knew of.

Redis as a Time Series DB (1 hour and 35 minutes to listen)

In case you missed it, here’s a talk that Josiah Carlson @dr_josiah gave a little while back. The slides are at http://www.slideshare.net/josiahcarlson/march-29-2016-dr-josiah-carlson-talks-about-using-redis-as-a-time-series-db.

How to Remove Duplicates in a Large Dataset Reducing Memory Requirements by 99% (probably a few minutes to read)

Probabilistic data structures, such as Bloom Filters and HyperLogLog, are extraordinary contraptions. This is just one of the many uses that they have and for more on these see immediately below.

Scalable Streaming Data Pipelines with Redis (53 slides)

A deck by Avram Lyon @ajlyon from Scopely @scopely on munching streams with Ariel – a system that relies on Redis’ HyperLogLog for aggregation.

Counting Distinct Values With HyperLogLog (8 minutes +/-0.81%)

In case you need an intro/refresher to HLL, Parker Selbert @sorentwo recently wrote one over at @codeship

Leveraging Probabilistic Data Structures for Real Time Analytics with Redis Modules (exactly 39 slides)

By pure chance, here’s presentation I gave on that this very same topic (albeit heavily modules-biased, naturally) earlier this week. And, coincidentally, another improbability was witnessed. #DataStructuresOfThe3rdKind

Josiah Carlson @dr_josiah explains how he indexes.

eleme/corvus #C #foss

A fast and lightweight Redis Cluster Proxy for Redis 3.0 – very interesting.

A long long poll ago in an AJAX call far far away (7 min to read) #NodeJS

What edition of the Redis Watch is complete w/o at least one article from Kyle @stockholmux?

Distributed Locking with Redis and Ruby #Ruby

Mike Perham @mperham, author of #Sidekiq, tests the latter’s concurrent limiter.

anvaka/redis-load-scripts #NodeJS #foss

Andrei Kashcha @anvaka worked around what have been vexing many users by redirecting require to a table that his library composes. A very cool hack.

evanx/redis-scan-bash #bash #foss

The problem: “We know we must avoid redis-cli keys ‘*'” -> the solution: “bashrc script for redis-scan util to iterate Redis keys” by Evan Summers @evanxsummers

Simple Graph Traversals with Redis UDFs #Lua #Howto

Dunno how I misses this beauty from Sai Teja Pratap @yesTeaPea

perrin4869/redis-mdel: mdel lua command for redis clients #NodeJS #foss

… or simply use PDEL from the rxkeys Redis module 😉

RedisLabs/spark-redis v0.3 #Spark #foss

The newest version brings in support for Spark SQL, which IMO is pretty awesome. Special thanks go to Sun He @sunheehnus for making everything possible.

}

The Past

Redis Day (((TLV))) 2016 (12 sessions!)

In case that your geographical skillset had prevented you from attending this once-in-a-lifetime-day-long-all-about-Redis event and celebrating 7 Redis years & 5 birthdays with us, we’ve got you covered – all the good stuff is in the above link for your viewing pleasure including:

Slides from RedisConf 2016

While the videos are being processed (updates to follow via @redisconf & the conf’s website) you can get a sense of what we had at this once-a-year-two-days-long-all-about-Redis event from the presentations. With over 40 speakers, 30 breakout sessions and about 800 attendees (I counted the first 799, but this one guy kept moving so I couldn’t add him) it was a huge success and a memorable happening.

RedisConf 2016

Tommy vTexan Trogden @vTexan provides a fresh perspective on what had transpired.

The Future

Hello, I’m Redis by Mark Paluch @mp911de

How: in English
When: 7pm, Thursday June 9th, 2016
Where: Uni Mannheim A5, 6 im Raum C013 statt (Anfahrtsbeschreibung)
Contact: majug @majug

Redis Labs

Redis Cloud Integrates With Databricks Spark

Together with Databricks @databricks, we’ve integrated our Redis-as-a-Service and their platform just in time for the new release of spark-redis connector (and Spark Summit ;)) – check out this mind-blowing demo that Reynold Xin prepared for RedisConf. PR

Redis Labs Brings In-memory Enterprise-grade Redis Databases to Pivotal Cloud Foundry

We’ve released a new Redis tile for Pivotal @pivotal‘s Cloud Foundry @cloudfoundry platform.

Whitepaper: Redis for Geospatial Data

Learn how Shakespeare would have used Redis to bring Romeo and Juliette together using Redis’ new Geospatial indices.