Specify docker compose while pushing

Hi all,

I have two fleets, one for Jetson Nano and one for Pi. One device from the nano fleet, is communicating with one from the pi fleet. Each fleet, has its own docker compose, which their directory tree looks as below:

├───nano
│   ├───Pi
│   │   └───folder1
│   └───docker-compose.yml
│   ├───common
│   └───a.py
└───────docker-compose.yml

In the nano directory, there is /PI and /common directories. The nano folder has a docker-compose.yml inside of it, which builds all the docker images that are related to the nano fleet. The /PI directory, like the nano one, has a docker-compose.yaml that builds the pi docker images. The /common folder has utilities that are used on both the nano and pi docker images.

When I had to make changes to the common folder, they were copied into my docker container with no problem, but the problem is with the pi’s docker compose. I had to build a docker image for each of the docker images that are for the pi and move the common folder inside the pi directory in order to copy the common folder while I’m building the image.
This solution wasn’t the best, as I would need to duplicate the common folder. Docker compose won’t allow to copy a file that isn’t in the running path when I run my balena push command. If I did it from the parent directory ( the /nano one) it would push the nano’s release instead.

One idea was to create a master folder with only three directories in it /pi /nano and /common and have the docker compose for both of the nano and the pi in the master folder, then specify which docker compose to use when I’m pushing a new release or pushing to a local device. I saw that was a requested feature here Balena deploy specific docker-compose file - Product support - balenaForums and also was made on your repo Add ability to select a docker-compose file · Issue #1142 · balena-io/balena-cli (github.com) but no updates on that for a while.

I did try to bind /common as a volume to a folder inside my docker container, but I got Bind mounts are not allowed.

My question is. How can I get /common to be discoverable inside my PI’s docker container so any changes inside of the common folder, would be visible in both my nano and pi docker images?

I’m trying to fit my use case with the least changes in the directory tree. Would be there a way to overcome this problem?

Interesting predicament! Maybe we can go back a bit further and understand what are the differences between the two fleets - I am assuming they are fundamentally different, or you would have both devices together in the same fleet? Forgive me for asking, but sometimes it’s not obvious that you can mix device types in the same fleet!

Next, it’s not clear from your post but how many services are you deploying per fleet - I am assuming more than one?

I’m also wondering if you found the Dockerfile.<device-type> mechanism? See more here: Deploy to your fleet - Balena Documentation

With this you may be able to construct a project organization that only builds the services for the device type of the fleet that you’re currently deploying to.

If your project is open source feel free to share the GitHub repo here and we might be able to make some more suggestions.

1 Like

Hi @chrisys,

Yes, both of the nano and pi, serve different functionalities and they communicate with each other’s.

The Pi is running a docker container with 4 or 5 docker services inside. The Nano is running a docker container with 10 or 11 service inside. Some of the Nano’s services rely on the GPU for inference.

Thanks for sharing the Dockerfile.<device-type> mechanism. I don’t think it fits my use case as I’m using docker compose for the services of each fleet separately. Please correct me if I’m wrong

The project is open sources. you can see it on Github. The docker compose for the Nano is in root directory while for the Pi is in /raspi folder.

I did experiment with <device-type> docker files and you can download my testing code here (pass: Balena)

I renamed the docker file for Pi and made it specific to the same Default device type of the fleet of them. I tried to upload push to my local jetson nano and it did push the Pi’s image also. I think this action shouldn’t happen?

These are the debug logs

[Debug]   Sending request to http://192.168.1.100:48484/v2/local/device-info
[Debug]   Found build tasks:
[Debug]       ftpd: build [.]
[Debug]       servos: build [.]
[Debug]   Resolving services with [jetson-nano|aarch64]
[Debug]   Found project types:
[Debug]       ftpd: Architecture-specific Dockerfile
[Debug]       servos: Standard Dockerfile
[Debug]   Probing remote daemon for cache images
[Debug]   Using 723 on-device images for cache...
[Debug]   Starting builds...
[Debug]   ftpd: Using platform option for build: linux/arm64/v8
[Build]   [ftpd] Step 1/12 : FROM gists/pure-ftpd:1.0.49
[Build]   [servos] Step 1/6 : FROM centipede2donald/raspbian-stretch:pigpio-zmq-byodr-0.25.0
[