Can Docker-Compose variables be informed by service/device variables? What about default variables?

Product support balenaOS
1

Hey. Let’s assume I have a docker-compose.yml in my service, and I want to have the user be able to specify the init creation of a database with an ENVVAR.

influxdb:
    restart: always
    image: influxdb:latest
    environment:
      - INFLUXDB_DB=mydb
    volumes:
      - 'sense-data:/data'
    ports:
      - "8086:8086"

In this case INFXLUDB_DB=mydb is what I want to change, but I want the user to be able to change this via a service or device variable.

I want the service or device variable #INFLUX_DB=[db_name] to inform the docker-compose script.

And I also want this variable to =mydb if none are specified by the user.

2

Hi there, the tricky thing with adding the variable in your docker-compose file is that this will be baked into the images you build for that release and won’t be changeable at run time. From what I can gather you want the db name to be changeable by a user after you have pushed your application, is that correct?

4

That’s true. My assumption is that the Docker-Compose would be ran, then that variable would be in the image, not baked in.

So that if a user added a service or device variable such as $DATABASE_NAME=test then it would override what was in the docker-compose. This way I would be able to have a default $DATABASE_NAME of ‘test’, and the user could change it to whatever they want in the balena dashboard. Is there any way I can have this happen?

5

Hi Matthew,

Given a multi-container application, you can set “Device Service Variables” which is basically equivalent to setting an environment variable on a specific service, just that through the Balena Cloud dashboard rather than statically through the docker-compose.yml file.

I believe this should accomplish what you need, and we are at the moment extending our Docker Compose implementation to be more on par with all the features that Docker Compose supports, including better handling of environment variables.

7

No this wouldn’t do what I want. I want a default variable, I can’t do that?

If I did what you said, then $DATABASE_NAME would not have a default value until the user set it. What I want is a default of ‘default_name’ to be applied to the environment, for example,

9

I don’t think that’s possible at the moment given our Docker Compose implementation (which is rapidly progressing). What about having the default value as an environment variable that is different from the one that the user can specify, and choosing between then from the application’s point of view? That will hopefully do until we implement proper environment variable support in our Docker Compose implementation

12

If I’m trying to develop something, what is the point in bothering with Balenaisms that are resultant from failing to keep up with the Docker upstream? Can this not be resolved by just using a Pi with Raspbian? :confused:

13

As a Docker user I understand your concerns. Keep in mind that Docker Compose by itself its not suited for production usage, and that Balena doesn’t simply run docker-compose up. We provide a Docker Compose interface to our more robust, IoT specific, container management solution, and its a relatively new feature.

This means that we have to completely implement the Docker Compose interface, rather than re-using a readily made implementation.

Also keep in mind that all this work is open source, and we really welcome contributions to make our Docker Compose interface on par with the official implementation!

14

Docker-compose isn’t suited for production usage? How so? Also can I actually see some documentation or proof of your “solution” being more valid, robust or IoT specific? That’s a bit vague. Where can I learn about this?

15

Docker Compose has been historically build for development and testing workflows. They have slowly added more features to make it more production ready, but running docker-compose on Raspbian will not give you the same guarantees as balenaOS with balenaEngine, such as mechanism to cope with SD Card corruption, optimised disk, memory, and network Docker usage, failure resistant pull, etc. Many of these features are provided with the help of the balena supervisor, and the way it controls the containers running on the device.

16

Right, I think I understand what you mean. I have once or twice noticed that things don’t work properly on a harsh reboot with docker-compose on armbian/raspbian.

I’ll last it out then, since obviously Balena as a concept is something I enjoy otherwise I wouldn’t be here. Hopefully this is a feature sooner rather than later. Thanks for the explanation.

17

No worries. I definitely understand your concern. We’ve been recently porting various multi container apps to Balena, and we’ve hit many of this issues ourselves already, so I promise we are trying very hard to sort out all these limitations!

19

I’m using Balena in a multicontainer to run Jupyter, all in all this isn’t too ridiculous. For now this will do.

#!/usr/bin/bash

MING_JUPYTER_DEFAULT_PASS=mingstack

# If MING_JUPYTER_PASS is set then hash this password via Jupyter and store it in hashed_password
# otherwise use MING_JUPYTER_DEFAULT_PASS as none has been set by the user

if [[ -n "$MING_JUPYTER_PASS" ]]; then
  hashed_password=$(python3 -c "from notebook.auth import passwd; print(passwd('$MING_JUPYTER_PASS'))")
else
  hashed_password=$(python3 -c "from notebook.auth import passwd; print(passwd('$MING_JUPYTER_DEFAULT_PASS'))")
fi

jupyter lab --ip=0.0.0.0 --NotebookApp.password="$hashed_password"
20

@matthewcroughan thanks for sharing your workaround, I’ve typically employed similar methods myself. For others reading another example of using variables in a similar manner is here: https://github.com/balena-io-projects/balena-sense/blob/master/telegraf/entry.sh

I’m using the presence of the variables to enable/disable configuration blocks which include said variables.