Environment variables are not working at build time

Product support
1

Hello,

I’m currently trying to get my multi container app running on balena OS.
The build fails because the environment variables are not beeing substituted (as documented in docker-compose: environment variables)

Am I right when I assume, that the builder does not have access to the variables I’ve set in my dashboard and it does not support env variables at all?

I need to pull some python packages from a private pypi and therefore I need an auth token during the build.
This token should not appear in my code base, so environment vars are the way to go.

The critical part of the docker-compose file looks like this:

    services:
      web:
        restart: always
        build:
          args:
            auth_token: ${AUTH_TOKEN:?err}
          context: ./web
          dockerfile: Dockerfile

Inside the Dockerfile I set the environment variable like this:

ARG auth_token
ENV AUTH_TOKEN $auth_token

When I run echo $AUTH_TOKEN I just see the literal string I used in the docker-compose.yml, e.g.:
${AUTH_TOKEN:?err}

I also tried the following syntax versions in the compose file:

$AUTH_TOKEN
${AUTH_TOKEN}
'${AUTH_TOKEN}'

So my question is:
How can I use environment variables during the build?
If not, what could be a work around?

Best,
eb

4

Hello there,
Substitute environment variables syntax isn’t yet supported in our compose file. We re actively working on build time secrets, and will make sure to post an update here once they re out. We’ll also be adding build-time env var interpolation soon.

2 Likes
5

Can you give a status update on compose file substitute environment variable syntax support? I use a .env file (https://docs.docker.com/compose/environment-variables/#the-env-file) to populate the image tags to use for my multicontainer docker-compose.yml file, but it seems the .env file is not being used as part of a local balena push (--env seems to be intended for inside the containers, not the docker-compose.yml file). I can obviously do some preprocessing with docker-compose config, but it’d be nice not to have to.

11

Hey @mrjogo

This is definitely on our radar, but I can’t give you a definite timeline right now.

Could you explain your use case a little please? It will enable us to tune the features to what users would actually find useful.

Thanks

13

I develop each container’s Docker image in a different git repo and then autobuild them on Docker Hub. The compose file references those images with an environment variable for each of the tags. This makes it easy to try a new combination of versions, branches, etc simply by changing one of the environment variable. Here’s an example:

docker-compose.yml

version: '2.1'
services:
  service1:
    image: service1:${SERVICE1_TAG}
  service2:
    image: service2:${SERVICE2_TAG}
  service3:
    image: service3:${SERVICE3_TAG}

.env

SERVICE1_TAG=latest
SERVICE2_TAG=development
SERVICE3_TAG=v1.0.0
15

Thanks for that @mrjogo – considering the multiple environments users can build projects on balenaCloud (the cloud builder via git push, local builds via the cli, etc.) it isn’t entirely straightforward to support build-time environment variables (ie. where do values come from?) and last we discussed it we didn’t settle on a nice uniform way to support this, though like Cameron mentioned above it’s a feature we definitely want. In the meantime, the workaround is what you do with docker-compose config unfortunately.

17

Yeah, I understand it’s a little tricky. I actually assumed there would be an entry for the environment variables in the balena.yml file (seemed logical to put it next to build-variables), and that only ones explicitly put there would be replaced in the docker-compose.yml file.

18

That makes sense indeed. I noted this in our feature description.
We’ll update this thread when we have progress on this feature implementation.

20

It took me a while to find out that substitute environment variables isn’t supported.

Especially since the docs explicitly mention that docker-compose v2.1 feature set is supported (which supports env var interpolation).

The docs should be updated.

21

@ftes welcome to the forums. The docs correctly state that a ‘a subset of the Compose v2.1 feature set’ is supported but we could definitely add this information to the ‘known unsupported’ list to save users getting caught out by this.

You are free to create issues in the docs repo or even PRs to implement the addition at any time but for now I’ve created an issue here.

23

Just a quick note to say that thanks to this PR from @ftes this is now mentioned in the docs.

1 Like