We have two architectures to build and want to use different docker-compose.yaml files for balena deploy.
It seems balena deploy does not work nicely with a setup using the --source attribute, which according to documentation allows us to have different docker-compose.yaml files.
We have two folders for each docker-compose.yaml files: solution-raspi3/docker-compose.yaml solution-amd64/docker-compose.yaml
No matter what we set in the docker-compose.yaml file, the build fails as Docker cannot find the Dockerfiles in the build: of each docker-compose.yaml.
Each docker-compose.yaml has a set a services, each one being something like:
...
myService:
build: ./services/myService
When we use a single docker-compose.yaml, it works fine.
The source target is supposed to be a build directory.
From balena deploy --help:
-s, --source <source> specify an alternate source directory; default is the working directory
Does it work when you move your specific yaml files to their own directories?
The rest of the files will be resolved relative to the specified source directory.
Edit: sorry, misread your post, missed the part where they were already in separate folders.
Edit 2: just tried out a simple test which seems to work:
Files:
Thanks @TJvV but it seems you are using an image in your docker-compose.yaml files.
We are using build: to build our own images and that’s when we see the problem.
[Build] service1 Preparing...
[Build] service2 Preparing...
[Info] Building for aarch64/raspberrypi3-64
[Info] Docker Desktop detected (daemon architecture: "x86_64")
[Info] Docker itself will determine and enable architecture emulation if required,
[Info] without balena-cli intervention and regardless of the --emulated option.
[Debug] Found build tasks:
[Debug] service1: build [./services/service1]
[Debug] service2: build [./services/service2]
[Debug] Resolving services with [raspberrypi3-64|aarch64]
[Build] Built 2 services in 0 seconds
[Error] Deploy failed
Could not find a Dockerfile for this service
Error: Could not find a Dockerfile for this service
at /usr/local/lib/balena-cli/node_modules/resin-bundle-resolve/build/index.js:95:32
at Generator.next (<anonymous>)
at /usr/local/lib/balena-cli/node_modules/resin-bundle-resolve/build/index.js:8:71
at new Promise (<anonymous>)
at __awaiter (/usr/local/lib/balena-cli/node_modules/resin-bundle-resolve/build/index.js:4:12)
at resolveTarStreamOnFinish (/usr/local/lib/balena-cli/node_modules/resin-bundle-resolve/build/index.js:89:12)
at Object.<anonymous> (/usr/local/lib/balena-cli/node_modules/resin-bundle-resolve/build/index.js:49:19)
at Generator.next (<anonymous>)
at /usr/local/lib/balena-cli/node_modules/resin-bundle-resolve/build/index.js:8:71
at new Promise (<anonymous>)
at __awaiter (/usr/local/lib/balena-cli/node_modules/resin-bundle-resolve/build/index.js:4:12)
at Extract.<anonymous> (/usr/local/lib/balena-cli/node_modules/resin-bundle-resolve/build/index.js:47:34)
at Object.onceWrapper (events.js:420:28)
at Extract.emit (events.js:326:22)
at Extract.EventEmitter.emit (domain.js:483:12)
at finishMaybe (/usr/local/lib/balena-cli/node_modules/tar-stream/node_modules/readable-stream/lib/_stream_writable.js:624:14)
From previous event:
at cloneTarStream (/usr/local/lib/balena-cli/node_modules/tar-utils/build/index.js:72:12)
at resolveTasks (/usr/local/lib/balena-cli/build/utils/compose_ts.js:575:43)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async performResolution (/usr/local/lib/balena-cli/build/utils/compose_ts.js:564:5)
at async makeBuildTasks (/usr/local/lib/balena-cli/build/utils/compose_ts.js:541:5)
at async $buildProject (/usr/local/lib/balena-cli/build/utils/compose_ts.js:141:19)
at async awaitInterruptibleTask (/usr/local/lib/balena-cli/build/utils/helpers.js:277:16)
at async buildProject (/usr/local/lib/balena-cli/build/utils/compose_ts.js:127:47)
at async DeployCmd.deployProject (/usr/local/lib/balena-cli/build/commands/deploy.js:96:37)
at async DeployCmd.run (/usr/local/lib/balena-cli/build/commands/deploy.js:50:25)
at async DeployCmd._run (/usr/local/lib/balena-cli/node_modules/@oclif/command/lib/command.js:43:20)
at async Config.runCommand (/usr/local/lib/balena-cli/node_modules/@oclif/config/lib/config.js:175:24)
at async CustomMain.run (/usr/local/lib/balena-cli/node_modules/@oclif/command/lib/main.js:27:9)
at async CustomMain._run (/usr/local/lib/balena-cli/node_modules/@oclif/command/lib/command.js:43:20)
at async /usr/local/lib/balena-cli/build/app.js:76:13
at async Promise.all (index 2)
at async oclifRun (/usr/local/lib/balena-cli/build/app.js:94:5)
at async Object.run (/usr/local/lib/balena-cli/build/app.js:107:9)
at async run (/usr/local/lib/balena-cli/bin/run:20:2)```
From the looks of it, your main problem here is what I mentioned earlier.
The directory being searched for a Dockerfile is relative to your source directory.
By specifying --source deploy-aarch64 and build: ./services/service1, you are telling balena to find your Dockerfile in ./deploy-aarch64/services/service1.
If you move your services into the deploy directories, it will work.
From my testing here, this seems to work as long as the services are actually subdirectories of where your docker-compose.yaml is located.
So, an entry build: ./services/b1
does work, but build: ../services/b1
does not.
For my tests, I have used a device-specific Dockerfile:
thijs@WSL2:/tmp/balena_src$ ls b1/services/b1/
Dockerfile.raspberry-pi
Okay, but that removes the whole usefulness of the --source attribute. The idea is to have different docker-compose.yaml files for each architecture (e.g. some architectures have different services).
If we have to move all the services into EACH “deploy” folder, we would have to replicate the services inside each “deploy” folder or use symlinks.
Much better off would be to add a parameter --file in balena deploy to point to different docker-compose.yaml instead of the default.
I think the main use of the --source flag is meant to be able to run the command wherever you want, instead of having to be in the actual directory.
I am currently not aware of a better way to share services between compose files with balena other than building the images for services separately and using those images in your compose files.
For reference, with docker-compose it is possible by doing something like docker-compose --project-directory /tmp/balena_src --file /tmp/balena_src/b1/docker-compose.yml build.
thijs@WSL2:/tmp/balena_src$ cat services/s1/Dockerfile
FROM balenalib/raspberrypi3
CMD ["balena-idle"]
Building gives something like this:
thijs@MSI-WSL2:/tmp$ docker-compose --project-directory /tmp/balena_src --file /tmp/balena_src/b1/docker-compose.yml build
Building s1
Step 1/2 : FROM balenalib/raspberrypi3
---> 4c43a961009f
Step 2/2 : CMD ["balena-idle"]
---> [Warning] The requested image's platform (linux/arm/v7) does not match the detected host platform (linux/amd64) and no specific platform was requested
---> Running in 82af725d89f8
Removing intermediate container 82af725d89f8
---> 67c74ffc3a37
Successfully built 67c74ffc3a37
Successfully tagged balena_src_s1:latest
Building s2
Step 1/2 : FROM balenalib/raspberrypi3
---> 4c43a961009f
Step 2/2 : CMD ["balena-idle"]
---> [Warning] The requested image's platform (linux/arm/v7) does not match the detected host platform (linux/amd64) and no specific platform was requested
---> Running in db53da566c76
Removing intermediate container db53da566c76
---> 00b3917e53ad
Successfully built 00b3917e53ad
Successfully tagged balena_src_s2:latest
Do note I specified an absolute path for the docker-compose file here.
I think wrapping this functionality of docker-compose or similar could make for a pretty nice feature request.
