I needed to get a copy of the Redis database in one of my devices to use on our CI/CD pipeline and found it more difficult than expected, I thought I might need to do this again in the future, and am documenting my process here.
My Setup:
I have a multi-container set up with our main service container and a Redis image container. This has a shared Service Volume called “redis-data” and is defined by the following docker-compose.yml file.
Balena documentation here.
One of these devices is running on my personal network for development purposes, and this is where I am extracting the Redis dump.rdb file from.
Additionally I have a Jenkins instance running as our CI/CD pipeline on a cloud server, which has Redis running as a service (not a docker container).
Finding the dump.rdb File:
As the redis container is from docker and is using the volume redis-data:/data the dump.rdb file isn’t where I thought it would be, /var/lib/redis/. The file was located in /mnt/data/docker/volumes/<number>_redis-data/_data/dump.rdb where number is a 7 digit number I don’t recognise.
Extracting the dump.rdb File:
As I’m on a shared ssh network with the device it was easy to use scp to get the file onto my computer through this command scp -P 22222 root@192.168.0.20:/mnt/data/docker/volumes/<number>_redis-data/_data/dump.rdb ~. -P means it’s using a different port than usual, specifically port 22222 which is chosen by Balena as documented here.
Getting the dump.rdb File to Server:
As I do the devops on the server this was easy using the scp command again scp dump.rdb root@<ip_address>:~. The scp command is well documented online such as here.
Turning Off the Test Redis Database:
To replace the dump.rdb file I need to turn off the redis service on my cloud server, otherwise when I restart the service it will wipe the data.
As the cloud server has Redis running as a service, not a container or running by itself, this means the conventional commands redis-cli shutdown and redis-server stop don’t work. Shutdown does stop the server, but it restarts immediately, whilst stop doesn’t work at all and gives the error Fatal error, can't open config file 'stop'. The needed command uses systemd, which is obvious to anyone who knows devops well, but took me a little while to realise. The command was sudo service redis stop. At this point we just need to replace the current dump.rdb file, and restart the redis service.
Replace Dump File and Restart
You might wish to make a copy of your current dump.rdb file, which should be fine with:
mv /var/lib/redis/dump.rdb /var/lib/redis/dump.backup.rdb
At this point you just need to us mv or whatever command you want to move the dump.rdb file from ~ to your redis database location, which should be /var/lib/redis/, thus mv -f dump.rdb /var/lib/redis/ which because of the -f flag won’t prompt you for permission to replace the pre-existing file.
Then restart the server with:
sudo service redis start
Wrap Up:
If you aren’t using redis snapshotting for data persistence then you will need to figure out a different way to get the copy using the append-only method, but I didn’t need to do that. If you find this useful, please let me know, and if you think there should be more information here anywhere I’m happy to update it.