I have two fleets, one for Jetson Nano and one for Pi. One device from the nano fleet, is communicating with one from the pi fleet. Each fleet, has its own docker compose, which their directory tree looks as below:
In the nano directory, there is /PI and /common directories. The nano folder has a docker-compose.yml inside of it, which builds all the docker images that are related to the nano fleet. The /PI directory, like the nano one, has a docker-compose.yaml that builds the pi docker images. The /common folder has utilities that are used on both the nano and pi docker images.
When I had to make changes to the common folder, they were copied into my docker container with no problem, but the problem is with the pi’s docker compose. I had to build a docker image for each of the docker images that are for the pi and move the common folder inside the pi directory in order to copy the common folder while I’m building the image.
This solution wasn’t the best, as I would need to duplicate the common folder. Docker compose won’t allow to copy a file that isn’t in the running path when I run my balena push command. If I did it from the parent directory ( the /nano one) it would push the nano’s release instead.
I did try to bind /common as a volume to a folder inside my docker container, but I got Bind mounts are not allowed.
My question is. How can I get /common to be discoverable inside my PI’s docker container so any changes inside of the common folder, would be visible in both my nano and pi docker images?
I’m trying to fit my use case with the least changes in the directory tree. Would be there a way to overcome this problem?
Interesting predicament! Maybe we can go back a bit further and understand what are the differences between the two fleets - I am assuming they are fundamentally different, or you would have both devices together in the same fleet? Forgive me for asking, but sometimes it’s not obvious that you can mix device types in the same fleet!
Next, it’s not clear from your post but how many services are you deploying per fleet - I am assuming more than one?
With this you may be able to construct a project organization that only builds the services for the device type of the fleet that you’re currently deploying to.
If your project is open source feel free to share the GitHub repo here and we might be able to make some more suggestions.
Yes, both of the nano and pi, serve different functionalities and they communicate with each other’s.
The Pi is running a docker container with 4 or 5 docker services inside. The Nano is running a docker container with 10 or 11 service inside. Some of the Nano’s services rely on the GPU for inference.
Thanks for sharing the Dockerfile.<device-type> mechanism. I don’t think it fits my use case as I’m using docker compose for the services of each fleet separately. Please correct me if I’m wrong
The project is open sources. you can see it on Github. The docker compose for the Nano is in root directory while for the Pi is in /raspi folder.
I did experiment with <device-type> docker files and you can download my testing code here (pass: Balena)
I renamed the docker file for Pi and made it specific to the same Default device type of the fleet of them. I tried to upload push to my local jetson nano and it did push the Pi’s image also. I think this action shouldn’t happen?
I did some testing at my end, and I think the reason why this isn’t working as expected is because you’re specifying the Dockerfile explicitly. That is to say, if you specify Dockerfile.raspberrypi4-64 as the Dockerfile, it doesn’t allow for the conditional logic to apply. For it to work you need to specify a directory that contains multiple dockerfiles, then the builder can choose the one that matches the architecture.
Another mechanism that may help here is container contracts, but I believe it’d still require some reorganisation of the project. To summarise, you place a contract.yml within each build context (hence I think it would require separation of all the containers).
FROM ubuntu:latest
CMD echo "Hello Raspberry Pi 5"; sleep 5
This setup allows the use of a single docker-compose.yml file, but will result in only the services running on the devices that match the device type specified in the contract file.
As below, the same code has been pushed to two different fleets with different device types.
Note the ‘optional’ label in the compose file - this is what allows the release to start running on the device even though all the contracts don’t match.
I think though, going back to the original question, this doesn’t directly solve the issue of mounting a common directory to separate containers, but I’m hoping that giving examples of different strategies may at least inspire some alternative solutions to the same problem.
Perhaps (yet) another idea, would be to pull the common elements out into a separate repository, then pull them in at the build stage.
Anywho, hopefully this has been some use and at least food for thought! Cheers
Thanks for the prompted answer. I have been trying to replicate the solution on my side here, but no good. I did three different services with 2 for the nano and one for the pi. I noticed that all three services are uploaded and would run on my Jetson nano.
Is there a problem in my contract.yaml for the pi?
@MWLC I’m guessing, as you mentioned, the problem is because of the build context. I believe the contract file has to be in the root of the build context, so of course if the context is shared between all services that’s not going to help. I would at this point try to rearrange things such that each service has it’s own build context, and at least prove the idea works in the simplest form before gradually re-introducing complexity.
