Configuring Linux Swap for Redise Pack
Swap space is used by the Linux OS to help manage memory (pages) by copying pages from RAM to disk and the OS is configured by default to be fairly aggressive. For Redise Pack (RP) and the way it utilizes and manages memory, it is best to reduce the likelihood of the OS swapping. If you would like to understand why, read more on how RP manages memory for best functionality and performance in the section below.
To change the swappiness setting of the OS, you have to have sudo access or be root:
- Change the setting immediately
sudo sysctl vm.swappiness=0
- Make it so the value will persist after reboot
# Set the value for the running system sudo sh -c 'echo 0 > /proc/sys/vm/swappiness' # Backup the existing sysctl.conf sudo cp -p /etc/sysctl.conf /etc/sysctl.conf.`date +%Y%m%d-%H:%M` # Set the value in /etc/sysctl.conf so it stays after reboot. sudo sh -c 'echo "" >> /etc/sysctl.conf' sudo sh -c 'echo "# Set swappiness to 0 so the OS avoids swapping" >> /etc/sysctl.conf' sudo sh -c 'echo "vm.swappiness = 0" >> /etc/sysctl.conf'
Note: If you have a management platform that builds your servers/images/containers/etc. Or something that makes golden masters, you should modify them with this fix as well.
How does Redise Pack manage memory?
By default, RP manages a node’s memory so data is entirely in RAM for performance reasons. (The exception to this is RP’s Redise Flash feature where Flash storage can be used to store data too.) If there’s not enough RAM available on a node, RP does not allow inserting of more data into databases. This limitation is intensional and ultimately beneficial. It is detrimental to RP for the OS to take control of allocating memory and spill some data into the swap, thus managing memory on RP’s behalf. RP knows better than the OS what it needs and when it needs it. RP protects the existing data and prevents the database from being able to store data into the shards, move the data to another node, or loose it (depends on eviction policy set by the user).
With this of course comes the responsibility to monitor the nodes, clusters, databases, etc., but you are already doing that, right?
What happens when Redise Pack is low on RAM?
If free RAM is low, RP will automatically attempt to migrate shards to other nodes, (if there are any) to free RAM on this node. If that is not possible, RP instructs shards to release memory (which can cause data loss if eviction policy allows it, or OOM replies). If shards cannot free memory, then RP depends on the OS’s OOM killer to kill slaves (but tries to avoid killing masters).
All that said, as always it is best practice to have a proper monitoring platform that will alert you proactively well before a system gets to this point. Maintaining a properly sized cluster is critical to a healthy RP installation and a day to day responsibility like most databases.