I’m then using the ssh key secret in my Dockerfile like this:
FROM wlisac/raspberrypi3-swift:5.0 as builder
# Add credentials and known hosts to builder
RUN mkdir /root/.ssh/ \
&& cp /run/secrets/id_rsa /root/.ssh/id_rsa \
&& chmod 400 /root/.ssh/id_rsa \
&& touch /root/.ssh/known_hosts \
&& ssh-keyscan github.com >> /root/.ssh/known_hosts
# Do stuff that uses a private github.com repo
FROM wlisac/raspberrypi3-swift:5.0
# Copy build artifacts from builder
CMD ["./start.sh"]
This is working with balena push, but it would be nice to find a solution that also works with balena build. A best case scenario would be a “fully portable” Dockerfile that would even work with raw docker build command, too.
Any suggestions on a “portable” approach that works with balena push, balena build, and maybe even docker build?
I’m considering using ARG instead of a mounted secret to allow the Dockerfile to be more portable. This should allow me to pass the ssh key as a build arg to balena build or docker build commands. Although balena push looks like it still requires build args to be defined in the balena.yml file?
The documentation says that secrets are more secure than build args, but I’m not sure it makes a difference in this case since I need(?) to copy the ssh key to the builder image either way (I can’t chmod 400 the ssh key on the secrets mount since it’s a read-only file system).
Any suggestions or best practices folks could share would be very helpful.
The balena secrets looks like a great new feature – hoping I can figure out how to use it correctly
I’m considering using ARG instead of a mounted secret […] The documentation says that secrets are more secure than build args, but I’m not sure it makes a difference in this case since I need(?) to copy the ssh key to the builder image either way
Warning: It is not recommended to use build-time variables for passing secrets like github keys, user credentials etc. Build-time variable values are visible to any user of the image with the docker history command.
But if you want / intend / need to save your secrets on the final image anyway (at /root/.ssh/id_rsa), then I think the secrets feature doesn’t make a difference. You might as well just use plain COPY or ADD in the Dockerfile:
COPY id_rsa /root/.ssh/id_rsa
… which is portable across all Balena and Docker commands. I think the main benefit of the secrets feature is to avoid saving secrets to the image, while allowing them to be used during the build.
But if you want / intend / need to save your secrets on the final image anyway (at /root/.ssh/id_rsa ), then I think the secrets feature doesn’t make a difference. You might as well just use plain COPY or ADD in the Dockerfile:
The ssh key won’t be saved in the final image, only in builder image. I think this helps mitigate risk, but I assume balena still caches the builder image.
Aside from the portability challenges, I’d love to use balena secrets for the git ssh key. However, the permission of the file is too permissive for use with ssh.
Here’s an example Dockerfile that uses the ssh key in the secrets directly without copying to the builder image:
FROM wlisac/raspberrypi3-swift:5.0 as builder
# Add credentials and known hosts to builder
RUN mkdir /root/.ssh/ \
&& echo "Host github.com\n User git\n IdentityFile /run/secrets/id_rsa" > /root/.ssh/config \
&& touch /root/.ssh/known_hosts \
&& ssh-keyscan github.com >> /root/.ssh/known_hosts
# Do stuff that uses a private github.com repo
FROM wlisac/raspberrypi3-swift:5.0
# Copy build artifacts from builder
CMD ["./start.sh"]
Here’s the corresponding error from the build output:
[main] @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[main] @ WARNING: UNPROTECTED PRIVATE KEY FILE! @
[main] @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[main] Permissions 0644 for '/run/secrets/id_rsa' are too open.
[main] It is required that your private key files are NOT accessible by others.
[main] This private key will be ignored.
[main] Load key "/run/secrets/id_rsa": bad permissions
[main] git@github.com: Permission denied (publickey).
[main] fatal: Could not read from remote repository.
[main] Please make sure you have the correct access rights
[main] and the repository exists.
I suspect I’m just missing something. My use case is closely aligned with the original feature description on Trello
Allow users to specify secret files and variables that can be used during build time to authenticate with external services like github, npm, etc. This allows user to build code from private repositories, etc without having these secret keys ending up on the end devices or commited into their git version control
Hi @wlisac - I think you can sort that part of the problem by copying the key to some temporary location, using it, and deleting it within the same RUN step. That way it won’t be stored in any layer of the build image and you can change the permissions of the copied file.
It only solves half of the problem though, cause it’s still not a “portable” solution and it’d only work with balena push. We’ve been discussing making balena secrets compatible with docker’s new build secrets, though they still seem to be rather experimental, and it’d take some time for us to actually start working on this.
Thanks – all of that makes sense. I’ll experiment more with this and report back.
Glad to hear that secret support for balena build is being considered.
In the spirit of unifying the use of balena build and balena push with secrets and build args, it might be nice for balena push to allow build args (and secrets) to be specified as command line options and not require the use of balena.yml. Is that something that has been considered, too?
One benefit of allowing build args or secrets as command line options is we wouldn’t have to duplicate the secret in the .balena/secrets folder.
For example, you could cat the ssh key into a build arg or secret option. I think the build arg approach is already possible today with balena build:
Indeed, being able to specify the secrets in the command line would be nice. There’s some ongoing discussion about this in two GitHub issues on our CLI repo:
This involves some challenges, hopefully there’ll be progress on these with time. And I hope for now you can work it out with the cp, chmod and rm within the same RUN command. But let us know if anything else comes up!
It’s marked as closed, but it’s unclear to me if build time secrets work with balena push <deviceIP>.
It looks like the merged PR is for registry secrets only? I’m asking because I haven’t been able to get balena push <deviceIP> to work with build time secrets.
Apologies for the confusion, it looks like I accidentally referenced the wrong issue in my commit. Secrets are still on the roadmap for local mode, I unfortunately don’t have a timeframe right now though.
@pdcastro@CameronDiver I found a bit of a surprising workaround for using an ssh key stored in the ./balena/secrets folder with balena push <deviceIP>.
It turns out you can COPY the .balena/secrets folder into a /run/secrets folder when using balena push <deviceIP> even though the .balena/secrets folder is ignored with .gitignore.
When using the cloud builders with balena push <MyApp>, the .balena folder is not available to COPY, so we need to use a little workaround to conditionally copy the secrets folder.
The end result is a /run/secrets folder when using balena push <deviceIP> and access to the actual balena secrets mount when using the cloud builders.
FROM wlisac/raspberrypi3-swift:5.0 as builder
# Temporary workaround for local push secrets https://github.com/balena-io/balena-cli/issues/1164
# Note that the .balena/secrets are only copied during a local push; not during a cloud push
COPY README.md .balen[a] /run/
# Add credentials, fetch dependencies, remove credentials
RUN mkdir /root/.ssh/ \
&& cp /run/secrets/id_rsa /root/.ssh/id_rsa \
&& chmod 400 /root/.ssh/id_rsa \
&& touch /root/.ssh/known_hosts \
&& ssh-keyscan github.com >> /root/.ssh/known_hosts \
&& clone_private_repo_here \
&& rm /root/.ssh/id_rsa
FROM wlisac/raspberrypi3-swift:5.0
# Copy build artifacts from builder
CMD ["./start.sh"]
Just to keep you informed on our progress, I’ve implemented the work that is required in one of our satellite modules here: https://github.com/balena-io-modules/resin-multibuild/pull/53. This now just needs implemented into the cli, which one of my colleagues is going to tackle in the coming week or so.
I think the main benefit of the secrets feature is to avoid saving secrets to the image, while allowing them to be used during the build.
