Hi,
TL;DR
Is it possible to have a multi-container multi-device setup with a single code base?
Background
I’ve got a distributed application that for each installation runs on three different devices (2 x RPi3, 1 x Intel NUC). Per now, we push to two different Balena Apps (one for each device type). We use two Dockerfiles (differentiated by dev suffix), and for the RPi’s we use device service variables to differentiate between the two RPi3’s. For now, this setup is sufficient; we can maintain a single code base for all devices, and deploying new code is done by balena push RPi3App && balena push NUCApp.
In the not so distant future, we’d like to split the NUC Dockerfile into three services, and also add a new service to one of the RPi’s. But we still strongly wish to maintain only one code base.
Possible solutions
The first solution I’ve tried is using separate directories – one for each device – for the Docker config, and then using balena push -s <dir> <appname> for deployment. However, this does not work, since the build context lies in the parent directory of the docker config directory.
The next solution I can think of is to ditch the Balena builder. That is, using balena build to create the images, and then balena push to deploy them. That should work, but I haven’t tried it yet.
The nicest solution would, IMO, be if Balena supported multiple Docker Compose files. Then I could create one docker-compose.raspberrypi3, one docker-compose.intel-nuc, and then balena push would determine which Docker Compose file to use for which device type.
Hi, it’s absolutely possible to have a multi-container multi-device setup with a single codebase. However, currently, you can only mix different devices on the same architecture but we are planning to do mixed architecture apps in the future. So in your instance of x86 and ARMv7 it won’t be possible and you’ll have to have two applications.
For the balena build and balena deploy workflow I suggest you review our CLI masterclass here https://www.balena.io/docs/learn/more/masterclasses/cli-masterclass/#5-building-and-deploying-an-application-without-the-builder.
Thanks for your reply, @garethtdavies.
TL;DR – temporary solution:
- Keep the common code base in
<repo>/src/
- For each device type, save
docker-compose.yml and Dockerfiles to <repo>/docker/<devtype>/.
- For each device type, build using
cd docker/<devtype> && docker-compose build
- For each device type, deploy using
cd docker/<devtype> && balena deploy <app>
(If there’s a simpler way to do this, please let me know 
)
Details
Using balena build was not possible, since it is unable to use a context directory that is a parent to the given source directory. Docker Compose, however, does not have such restrictions, so I’m left with building locally. That is no big deal, but being able to use the balenaCloud builder in the future would improve build times.
In order for this solution to work, I had to explicitly set context: ../.. for each service in each docker-compose.yml.
BTW, looking forward to mixed architecture apps! But, AFAICS, unless balenaCloud starts supporting multiple Docker Compose files in a single app, it won’t be of any use for us.
Hi erlendaasland as you found out balena build limits the context, this was implemented as a safety measure so the programs can’t escape the directory you run it in. It makes sense with the use case you describe to either remove this limitation or disable it behind a flag. I’ve created an issue here: https://github.com/balena-io/balena-cli/issues/1704 so the CLI team can take a look at it.
1 Like
Great, thank you @StefKors!
@StefKors: Another option might be to make it possible to specify a Docker Compose file from the CLI. For example:
$ balena deploy <myapp> --build --docker-compose my-docker-compose.yml
Thanks Erland for the additional information and suggestion. I have noted that in GitHub as well.
1 Like