Hi,
I have a Multicontainer application and I’m trying to figure out if it’s possible to push different services, from different repos, to the same BalenaCloud application.
I have services that are written in different programming languages and are very distinct in their nature, putting everything on the same repo would not be ideal at all.
Thank you.
Best,
Nelson
Hi,
sounds reasonable 
You can separate out the services into their own repositories and build/push them as independent images as long as they are on a public registry.
As your balenaCloud application you can then use a bare repository with just the docker-compose.yml referencing the services.
Hi,
thanks for the quick reply.
That seems like a workaround, am I right? It will make the deployment pipeline a bit more complex to maintain.
Let’s say I build the different images, from different repos, I version and push them to a registry. Up to here, it’s all good. Then I need to maintain a repo, that only contains my docker-compose file. Also ok.
But then I need to manually update the versions of the new images, on the docker-compose file and push to the Balena git repo to trigger an update of my services. This is the part that needs improvements IMO. If we could use environment variables on the docker-compose file, then we would only need to issue docker-compose up -d --build <service> equivalent command to redeploy the update services.
On the public registry point, is it really impossible to use private registries, e.g. GCP container registry? That may be a showstopper for our use case. I wouldn’t mind keeping my images on Balena.io private registries, but on public registries may not be possible.
Thank you.
So just to see if I got that right: You have it working with a monorepo and the command you mentioned but you would like to keep the service code separate?
Would git submodules work for you?
And regarding the private registries, we are actually in the process of merging stuff that will make this possible in the near future.
Hi @robertgzr,
great to know that support for private container registries is being added. Awesome job guys!
Regarding the deployment of multicontainer applications:
Let’s assume that what I have now is something like this on my docker-compose.yml:
version: '2' services: service-1: image: xyz:3.1 service-2: image: querty:0.1
After pushing v0.2 of service-2, I need to go into my docker-compose repo, change the image reference from image: querty:0.1 to image: querty:0.2, push to master, maintain this repo properly versioned, etc…
Instead, I think it would be nice to have something like this:
version: '2' services: service-1: image: xyz:${SERVICE1_TAG} service-2: image: querty:${SERVICE2_TAG}
and then being able to update those environment variables on the application over the API, so that the CD pipeline could be streamlined in a way that doesn’t seem hacky.
On top of that, being able to issue requests over the API to restart/rebuild single services on the application (the equivalent of the docker-compose up -d --build <service> command) would also be great. So that we don’t need to re-deploy all the services everytime we need to update a single service.
Thank you.
Hey @nelson
Some interesting ideas there certainly, and I will make sure that someone in the builder team sees it for feedback. In the meantime, it sounds like you could solve some of this by making our builder create your images. This would mean having your compose file a bit like this:
version: '2'
services:
service-1:
build: ./service-1-code
service-2:
build: ./service-2-code
The images produced are cached so if you only changed service-2 then it wouldn’t need to pull down service-1.
Hi @richbayliss,
thank you for your suggestions. For that use case, I would use git submodules, which works, but it’s still a workaround 
For the time being, I will use the images with their specific versions on the docker-compose file (as soon as support for private container registries is added), and I will keep an eye on updates regarding the multicontainer deployment flows.
I also just tested your approach regarding deploying only the services that are modified, but for me, it downloads and restarts all the services, even though, I only changed the source of 1 service (and with enabled delta updates).
Redeploying only the services that are updated is also a key point on the deployment flow, IMO.
Thank you!
Hi @nelson,
I’ve just quickly tried rebuilding one service in a multicontainer project, and it does work as it should.
For me, the dashboard shows that all services are downloading (this is, I believe, because it’s refreshing the docker-compose), but only the service that’s changed is restarted. The logs should mimic this:
20.12.18 15:21:50 (+0000) Downloaded image 'registry2.balena-cloud.com/v2/427351438e0bc3a46f40b427a41be342@sha256:20890eee6df2fb9d44c804c1b25a570d54fd333eb30929ac2e7f560259b7945f'
20.12.18 15:21:51 (+0000) Killing service 'data sha256:ad4ad63409fef7ed8ab3d0e54e5ff4d9a8a6187b786adb6f082c1b10a3fca7c6'
20.12.18 15:21:55 (+0000) Service exited 'data sha256:ad4ad63409fef7ed8ab3d0e54e5ff4d9a8a6187b786adb6f082c1b10a3fca7c6'
20.12.18 15:21:55 (+0000) Killed service 'data sha256:ad4ad63409fef7ed8ab3d0e54e5ff4d9a8a6187b786adb6f082c1b10a3fca7c6'
20.12.18 15:21:57 (+0000) Deleting image 'registry2.balena-cloud.com/v2/b9fb1a480a9f7967f964eaa8eeaa88cf@sha256:ad37e4075c0ee3a86a0df2ee7169fb81d26c2cd3bf4fb4c52986600107231201'
20.12.18 15:21:57 (+0000) Deleted image 'registry2.balena-cloud.com/v2/b9fb1a480a9f7967f964eaa8eeaa88cf@sha256:ad37e4075c0ee3a86a0df2ee7169fb81d26c2cd3bf4fb4c52986600107231201'
20.12.18 15:21:57 (+0000) Installing service 'data sha256:78830555a552884b6f5b97420f99bcede60a99c25969a2fb6972172efcdca6cc'
20.12.18 15:21:58 (+0000) Installed service 'data sha256:78830555a552884b6f5b97420f99bcede60a99c25969a2fb6972172efcdca6cc'
20.12.18 15:21:58 (+0000) Starting service 'data sha256:78830555a552884b6f5b97420f99bcede60a99c25969a2fb6972172efcdca6cc'
20.12.18 15:22:01 (+0000) Started service 'data sha256:78830555a552884b6f5b97420f99bcede60a99c25969a2fb6972172efcdca6cc'
Could you possibly share the logs for your device when you change a single service, so we can determine what might be happening?
Best regards,
Hi again,
I’d just like to check something with you, as it may explain what you’re seeing.
Were you git pushing, balena pushing or balena deploying your application?
The first two should correctly keep the non-modified services altered, but in fact a balena deploy will redownload and restart all services. Is this what you were seeing?
Does this mean that Balena finally initializes Git submodules on push? I’ve been waiting awhile on that.
@legowerewolf , yes if you use the balena push from the CLI you can use git submodules no problem, balena push doesn’t interact with git or version control at all.
Hi @hedss,
(sorry for the late reply, holiday season!)
I’m git pushing. I will provide you with the logs and more details of my update process as soon as I get back to the office.
Thank you for helping out.
cheers,
nelson
Hi @hedss,
I have been running some tests this morning and I found out what was the problem. Because some of my services are Go applications, I need to add the Gopkg.lock file to my Dockerfiles otherwise the dependencies are rebuilt and it looks like there are changes on those services.
So git push balena works as expected.
thank you!
Thanks for letting us know
Is there an update to this? I can’t put the images in a public docker registry, and I don’t think I want to use sub-modules to bring the packages under one repo.
@iotmaker using the latest CLI this should be possible now: https://www.balena.io/docs/learn/deploy/deployment/#private-base-images