Shared Memory for a container in docker-compose file

Hello!

We have a multi-service application defined in a docker-compose file. We receive a warning from one of our services about needing to increase the --shm-size for the /dev/shm device to 30% of the total memory of the device to avoid performance degradation. We have done so on our development machines not running balenaOS and not in a fleet, and we have seen a performance improvement by our metrics for our application.

We would like to do the same for our balenaOS fleet devices. There is the shm_size field that can be added to a service in a docker-compose file. Since all of our development machines have different amounts of memory (laptops, desktops, server, etc.), we placed an environment variable in a .env file used it in the docker-compose file. An example:

# .env file
SHM_SIZE=3gb

and

...
api:
  shm_size: ${SHM_SIZE}
...

We thought we could define a device variable for each device in our fleets to account for the different memory sizes of the devices (Jetson Nano, Xavier NX, AGX, and some x86 machines that we installed generic amd64 balenaOS on). However, we are aware that Balena does not currently support variable replacement in docker-compose files. So, this solution will not work.

It does look like the build.shm_size and shm_size fields are supported.

We would like to avoid having separate fleets for each device with the same memory size and creating separate docker-compose files for each device in our application to hard-code the shm_size field values.

Is there another solution/workaround we could use to set the shared memory for a service?

Any information and/or recommendation would be greatly appreciated.

Hey @ts-cfield , if your service is running as privileged you could use an entrypoint script to automatically remount shm based on an env var or by reading the total memory and applying 30% to shm.

Here’s an example of resizing shm in an entrypoint script:

2 Likes

@klutchell, Nice! We are already using an entrypoint script and our container is running as privileged, so I will try adding the resizing code to the entrypoint script and report back. Thank you.

Thank you, @klutchell, for the example. It was very helpful!

Following your recommendations, we updated our entry script:

if [ -z "${SHM_SIZE}" ]; then
    echo "Calculating the SHM_SIZE value as 30% of Total Memory"
    SHM_SIZE =$(free -m | awk '/^Mem/ {printf "%.0fG",0.3*$2/1024}')
fi

echo "Remounting shm with size ${SHM_SIZE} ..."
mount -o remount,size=${SHM_SIZE} /dev/shm
echo "Remounting shm with size ${SHM_SIZE} ...DONE"

Glad it worked out!

1 Like