Storing images and docker data in an external drive

Product support balenaOS
1

Hi all,

Our Nvidia Jetson has very limited storage about 16GB. We would like to store our docker data in an external drive. However, after adding to fstab I get the following error in dmesg and supervisor fails to start:

[   90.705211] overlayfs: filesystem on '/mnt/external/docker/check-overlayfs-support549496362/upper' not supported as upperdir

I think it’s not happy because the mount point is in a overlayfs. I had turned off readonly for / this is my fstab along with the entry to mount my external drive:

# stock fstab - you probably want to override this with a machine specific one

/dev/root            /                    auto       defaults              1  0
proc                 /proc                proc       defaults              0  0
devpts               /dev/pts             devpts     mode=0620,ptmxmode=0666,gid=5      0  0
tmpfs                /run                 tmpfs      mode=0755,nodev,nosuid,strictatime 0  0
tmpfs                /var/volatile        tmpfs      defaults              0  0

# uncomment this if your device has a SD/MMC/Transflash slot
/dev/nvme0n1p1       /mnt/external          ext4       defaults,sync,noauto  0  0

It appears that mounting only works if I make / in fstab defaults instead of ro (readonly).

The way I achieved the ability for docker to store files elsewhere is by editing /etc/docker/daemon.json so that it is:

{ 
   "data-root": "/mnt/external/docker"
}

Note that I had also migrated all files in /var/lib/docker

Could you please help me figure out why I get the overlayfs error in dmesg and if that is the root cause preventing docker from starting? I’m also happy to take suggestions about how to go about this particular use case.

2

I’m not sure this is possible at all there seems to be many references in balena supervisor and elsewhere referring to /var/lib/docker the default docker data location. So I’m happy to park this one but will look at Communicate outside the container - Balena Documentation as it shows how to mount storage to a container. Perhaps 16GB for our apps is fine for now. But we need the data storage to be bigger than that so mounting to the container should solve our problem.