11.4 Sharding LISTs with Lua
Back in sections 9.2 and 9.3, we sharded HASHes, SETs, and even STRINGs as a way of
reducing memory. In section 10.3, we sharded ZSETs to allow for search indexes to
grow beyond one machine’s memory and to improve performance.
As promised in section 9.2, in this section we’ll create a sharded LIST in order to
reduce memory use for long LISTs. We’ll support pushing to both ends of the LIST,
and we’ll support blocking and nonblocking pops from both ends of the list.
Before we get started on actually implementing these features, we’ll look at how to
structure the data itself.