The Only Newsletter About Redis

Edition #69
May 4th, 2017

# include "i.h"

Hello,

This edition begs to begin with a laudable numbers joke, but I'm refrained from doing that. Instead, let us note how May Fourth is just a play on June Third and snicker at this ripoff of what is truly a glorious day. Snicker snicker.

May the force be with you,
Itamar

int main(int argc, char **argv) {

Redis 4.0 RC3 is out! (3.99998 minutes to read)

Salvatore Sanfilippo @antirez releases the third candidate (technically 4th?) for v4 is out with a boot-load of goodies ranging from features through fixes to improvements across the board. Barring stop-the-world issues and forces of nature, it looks a formal release isn't too far away. Two and a half notable observations about the RC's impressive payload:

  1. A substantial count of major changes

  2. The force community is strong – happy May Fourth!

    1. More parties involved in fixing issues and contributing features
    2. Two of the biggest cloud providers' employees assist in squashing a family of OSS DB bugs (should that become a trend, Matt Asay @mjasay will have to redo Cloud convenience is killing the open source database via InfoWorld @infoworld)

OH Salvatore Sanfilippo @antirez

Redis would not be sustainable without the help received from the community and from the Redis Labs folks (and I would be mad…)

LMDB — First version of Redis written in Tcl #Nerdgasm <- Gelato all'interno

Look what Salvatore Sanfilippo @antirez had found in his email archives – the first version of what would eventually become it and several related trivia facts.

How We Built r/Place (~6:38 minutes to read) #Patterns

tl;dr from BITFIELD's motivations by Salvatore Sanfilippo @antirez:

Fun fact: Reddit's 2017 April fools' project r/place was built using the Redis BITFIELD command in order to take an in-memory representation of the collaborative canvas.

Discussion in /r/programming: https://www.reddit.com/r/programming/comments/656z8t/how_we_built_rplace/

Code repository: reddit-plugin-place-opensource/reddit_place

All thanks for the best ever April Fools' project go to Brian Simpson, Matt Lee @madlymattlee, Dan Ellis @I_am_Dan_Ellis and the entire Reddit @reddit team.

OH Dan Ellis @I_am_Dan_Ellis

It's very cool and was the perfect data structure for our use case. Thanks for making it!

OH Kevin O'Connor @gooeyblob

Wouldn't have been possible without redis! Thank you!

The new LFU mode #RTFM #TIFU

A new section has been added to the documentation about v4's new eviction policy – by Salvatore Sanfilippo @antirez.

Note to self: verify that LFU is a politically correct.

Microservices At The Heart of BBC iPlayer (<1 minute to scroll)

Santoro Simon had attended QCon @QCon London 2017 and took notes of what looks like an educational user story . Of particular (personal) interest was this session from Cem Staveley, but both video and presenter are untraceable to this watcher's eye.

An Alternative Approach to Rate Limiting (9 min read) #Patterns

Nikrad Mahdi @nmahdi reviews the design of the homegrown rate limiting system that's employed by Figma @figmadesign to protect against spam attacks. He provides a wonderfully graphic explanation of three approaches – token bucket, fixed window counters and sliding window log – in native Redis terms.

Connect to Redis Elasticache through an EC2 SSH Tunnel #howto #jailbreak

Due to security reasons, AWS' Redis-as-a-Service is not accessible to public interfaces. While this lockdown prevents some attacks, it also makes it impossible to just connect to from, lets say, your laptop. Jamieson Becker @JamiesonBecker of Userify @userify shows how to set up a tunnel to quickly and effectively work around that issue.

Redis Quickie: Scramble and unscramble your commands in Node.js (3 min read) #NodeJS #BonusValue

A security-oriented piece in the ever-growing series by Kyle @stockholmux, that shows how to use the rename-command configuration directive and call the renamed command from code.

Bonus: whilst researching the this, Kyle also uncovered an issue (#3978) in the cli.

OH Arthur Belair @ArthurBelair

@stockholmux your Node/Redis series is amazing! Thanks for the great content

Redis powered Microservices using Hydra (3 min read) #ELI5 #NodeJS

Carlos Justiniano @cjus provides few-but-well-worded paragraphs about Hydra Microservice @HydraNPM – clear, effective and compelling.

TODO: 1) learn Node.js 2) write a few microservices (how many are enough?) 3) profit!

How is the result of KEYS * sorted? #PHP #SOReadyToHelp

This.

The reply is absolute perfection – enjoy and upvote h0x91B's answer }:-)

OH TJ Miller @SIXLIV3

The more I use Redis, the more I find uses for Redis

Monitoring Redis #howto

Mike Perham @mperham had a productive month preparing for #RailsConf. Between describing how to migrate Sidekiq from using namespaces to numbered databases to releasing Sidekiq v5.0 ('grats!), he also put together this handy reference about the basics of Redis monitoring.

Yourdream/rsbeat #foss #ElasticStack #golang

Complete at least one of the extra credit tasks in the item above – this is a Beat that collects the Redis slowlog and a dashboard to match.

ElasticStack rsbeat

Red hot Redis powers Netcore’s billion-plus connects for SMS campaigns (4 minutes to read)

Anil Shukla @shuklaanil tells how Netcore Solutions @NetcoreSolution had moved from a single Redis instance to a sharded topology. While a little skimpy on details regarding how sharding was actually done, this still makes a short nice read.

OH Joshua Cohen @heyjoshua

Early on at Flickr, I once ran a flushall against a prod redis instance, not realizing a) it was prod and b) it dropped all dbs.

Using Redis as a session store in Go & Counting Stuff with Redis #howto #golang #twofor

It may be too early to tell, but by the looks of it appears that there's a new series author becoming. Paweł Słomka @pawel_slomka is a gopher, and provides two tutorials that I sniffed .

Note: both are extremely satisfying, olfactory-wise, in my opinion

Maintaining a Database of Reputation Data With Splunk #howto #foss #python

If you're already using #Splunk, here's how you can add a reputation database (reputed as a necessary enrichment ingredient) with a little of open source software goodness from RST Cloud @rst_cloud.

GitHub repository: https://github.com/rstcloud/threatdb/

Redis on Autopilot #AutopilotPattern

This introduces the Autopilot Pattern for Redis (no cluster support… yet). By Jason Pincin @jasonpincin of Joyent @joyent, along with the contributions of Dave Dunkin @ddunkin and Faithlife @Faithlife.

Write Fast Apps Using Async Python 3.6 and Redis #howto #foss #Python3

With the Redis-backed (in-memory) db subconscious by Michael Flaxman @mflaxman via Paxos @PaxosGlobal.

Note: smells like ORM

OH Jono @iamjono

I have to say, Redis is the politest and most gentlemanly of all databases: “Redis is now ready to exit, bye bye…”

leonchen83/redis-replicator #foss #Java

While still new-ish, this looks promising in terms of functionality and robustness – a replication tool that supports slave synchronization, persistence files and the latest formats and versions. By leon.chen @chenbaoyi1.

Discussion in /r/redis: https://www.reddit.com/r/redis/comments/64qnt1/i_wrote_a_tool_which_implemented_redis/

Redis Pipeline Batching #howto #CSharp

Three well-executed simple-n-clear diagrams by Taswar Bhatti @taswarbhatti, that make it hard to miss the points of using pipelining and batching (transactions). You can safely treat the code snippets as written in some pseudo language 😉

Note to self: steal these for next training session!

Redis pipeline

72squared/redpipe #foss #Python

My reaction to the project's promise – "Making Redis pipelines easier to use in Python" – was: pipelines work just fine, why change anything? But after reading the (great!) docs about "a paradigm shift" to using "reusable blocks " and "duck-typing"… well, it makes perfect sense. Pythonistas should really check this one out.

Bonus: both stand-alone and cluster modes are supported.

Hangfire.Pro.Redis 2.1.0 #oss #DotNET

In it's latest update, this open-source-but-yet-commercial .NET client reels in support for the cluster and better connection resilience. Via Sergey Odinokov @odinserj of Hangfire @hangfireio.

gabfl/redis-expire-least-used #foss #Lua

A proof of concept by Gabriel Bordeaux @gabrielbordeaux that essentially adds usage-based expiry on top of Redis' TTL.

That's a very cool pattern that can be quite useful, as the graph "proves" – someone could develop a Module (s! Modules! Modules!) for that to take it to the next level…

OH Phillip LeBlanc @leblancphill

New redis slogan: Cache me outside, howbow dah?

Modules! Modules! Modules!

Need for speed (41:54 minutes to watch) #bws17 #Lua

Elena Kolevska @elena_kolevska gave a great talk on getting more out of Redis. Boyan Yordanov @specter_bg summarizes:

Then Elena Kolevska came on stage and showed us how you can improve the performance of Redis, just by reducing the number of requests, using pipelines or Lua scripts. And if you are a really hardcore developer you can go to “warp ten” by writing an extension in C. An Easter egg in her talk was that it was actually a continuation of a talk she gave last year at the Bulgaria PHP conference, which I also had the pleasure to see.

background_keys.c <- the day that Hell froze #API

On the quest of pushing the modules API further and farther, Salvatore Sanfilippo @antirez has made blocking commands even more powerful with a thread-safe context that allows interaction with the core. Put differently, this gist shows how to implement KEYS using a non-blocking, client-blocking module command.

StackExchange.Redis and Redis 4.0 Modules (5 minutes to read) #foss #CSharp

Everything in life comes with a price, and you need to work hard to get anything done in this world. But not always, and never when it comes to my ever-favorite always-there so-ready-to-help superhero Marc Gravell @marcgravell of the Stack Exchange @StackExchange.

Come on DotNETters – make a change and shape how modules will be supported in The #foss client for #DotNET!

OH Marc Gravell @marcgravell

Only way to be sure the SE.Redis "modules" extension API works is to write some… I may or may not have just ported JRediSearch to .NET 🙂

RedisLabsModules/RediSearch 0.15 #foss #C

The one-man-wonder Dvir Volk @dvirsky, with whom I proudly share a room and many opinions, keeps pouring absolute awesomeness into his uber-module. Here's a recap of what you've missed since last time:

  • Queries turned to level 11:

    • Field modifiers for creating complex filtering on multiple fields
    • Allow selecting multiple fields
    • Optional terms and clauses
    • Match on prefix, (e.g. Redis Watc*)
    • Support for NOT queries
  • Results are sortable by property

  • Queries can have payloads (to be used by a custom scoring function)

  • Support for dynamically loading extensions… let me repeat that: Support for dynamically loading extensions! Let that one sink in for a minute. Yes, that means the monster can haz its own modules.

OH Dvir Volk @dvirsky

Yes, this is the first #redis module to have a module system of its own #inception

Inception - spinning top

The Future

Redis BoF at Austin OSCON

When 1: Monday, May 8th, 2017, 7:00 to 8:00 PM
When 2: Wednesday, May 10th, 2017, 7:00 to 8:00 PM
Contact: tague @tague

Introduction to Redis and Python (The Austin Python Meetup)

When: Thursday, May 10, 2017, 7:00 PM
Where: Atlassian @Atlassian, 303 Colorado Street, Suite 1600, Austin, TX

Scaling with Redis for Caching and Background Jobs (Seattle Redis Meetup)

When: Thursday, May 18, 2017, 5:30 PM to 7:30 PM
Where: Napster @napster, 701 5th Avenue, Suite 3100, Seattle, WA
What 1: Scaling with Redis for Caching and Background Jobs
What 2: TBD <- feel free to approach the organizers with suggestions

Redis Conf 2017

Redis Conf 2017

Redis Labs

Docker and Redis Enterprise Pack – Developing Redis Applications on Windows, MacOS or Linux with Containers

We are excited to announce the preview release of the new docker image for Redis Enterprise Pack.

Using Redis on Windows

Microsoft sponsored a project to port Redis onto Windows, but the project was recently discontinued according to Microsoft. What are your options now? Let Redis Labs VP of Product Management Cihan Biyikoglu @cihangirb tell you via DZone @DZone.

The Forrester Wave™: Database-as-a-Service, Q2 2017

Redis Labs is ranked as top #4 database-as-a-service providers – fourth only to AWS, Microsoft and Oracle! – for its score in the current offering category, recognized for its ease of use, scale and high availability functionality.

Questions? Feedback? Anything you want to share? Email or tweet me – I’m highly available 🙂

This newsletter was produced and distributed by Redis Labs, Inc. 700E El Camino Real, Suite 250, Mountain View, CA 94040
Repository | Archive | Subscribe | Follow on Twitter