I have a number of questions regarding the building / deploying a release.
When I use balena deploy myApp --logs --source --build ., it says it’s up to date. However, I’ve changed things in my Node.js project. So it would be weird if it’s already built. (balena cli v12.5.0, multi-container app, raspberrypi4 application). Even using the --build option, doesn’t trigger a new build.
When I’m building my app using balena build --application myApp --logs --source --build . on my desktop, the building process takes a really long time. It has to compile some things for Node.js (like the sqlite3 module), but will performance increase when I use a Raspberry Pi 4 with Raspbian and Docker? So create a dedicated builder for the Raspberry Pi 4 app?
When I use balena push <local-ip>, it takes quite some time to start building, like a few minutes. That’s before it starts building (so the yellow warning shows up, but then halts). After those few minutes, it starts to build, which is quite fast. My Node.js app is in Typescript, so I have to build before I push. So I can’t use livepush. I can be wrong, but it never took so long to start building in my experience. Is something changed there? (balena cli 12.5.0, balenaOS 2.48.0+rev1)
When using balena push <local-ip>, where does the build take place? On the device or on my PC? And if it’s on my PC, why is that faster than using balena build?
Thank you for contacting us! I will try to address all your points in order:
Make sure that the --source flag points to the right directory, in which you have changed the files.
The building process will take a lot of time because of emulation. In essence, as your computer’s processor is of different architecture than that of Raspberry pi, it will emulate using QEMU. To speed up building you can use balena push which leverages the native ARM cloud builders that we offer.
The build will take place on the device (rpi4), not the dev machine. It is advised to see the relevant documentation for local mode. A nice little hack is the firstly build your project using balena push and then switch to local mode. That way, the containers are already cached inside the device and then the subsequent builds will be done much faster, since the device will not have to rebuild the whole container, but only the layers that you changed (e.g if you changed app.js, only the step that adds app.js into the container, like ADD ‘app.js’.
The --source flag is set at the right directory. It’s a multi-container build, so the docker-compose.yml is at the root, and the containers to be built are in directories (e.g. system, socket) etc, which are used as image in the docker-compose.yml.
I’m using openBalena, which is why I posted this in the openBalena sub-forum :). So I can’t use the native ARM cloud builders.
I’m using local mode to build, but can’t I use the builded image via local mode to deploy it to the openBalena registry? This saves me lots of time (build time on RPI is about 10 minutes, and on my iMac about 40 minutes…)
You can try the deploy command using the --nocache flag to force balenaCLI to rebuild the application. I will communicate it internally in case it’s a bug. Regarding the cloud builders, you are absolutely right, my bad. The essence remains, it’s slow due to emulation. Regarding the last issue, let me check with my colleagues.
My fault, the Typescript build folder was missing. Had to build it again. Logs showed that it had something to do with another step.
About using the builds on a local-mode device, that’d be awesome. Because this saves me lots of time emulating builds and I know for sure that the image works, because I’m testing it at the same time
Specifically about this point, I’ve noticed that the command line is not correctly formed, and sadly the balena-cli does not complain about the bits that it does not recognize in the command line:
We are currently working on migrating the CLI framework from Capitano to oclif, and this work will improve command-line parsing including pointing out unrecognized options that are currently silently ignored.
Thanks for explaining the whole command to me. I’ll try the --nocache option. I probably used --no-cache because Docker uses that (or used?). Or I just misread the Balena CLI documentation. Sorry about that.
But another (maybe interesting) question is: Can I setup a Balena Builder using balenaOS? I’m using Raspbian now to try it out, but it’d be much simpler for me to use balenaOS to do this. This is because setting up and flashing a RPI with BalenaOS and configuring it is much simpler with Balena than using Raspbian and going through steps to set it up.
It’d be very nice to create a Balena builder using Balena
Just a quick addition on this thread. You can run a device in localmode and use that device as a local “builder” of sorts. In essence, when a device is in local-mode, it exposed the docker socket, thus you can point balena-cli to that docker-socket, and the application will be built on that docker instance.
See the following args:
--dockerHost, -h <dockerHost> Docker daemon hostname or IP address (dev machine or balena device)
--dockerPort, -p <dockerPort> Docker daemon TCP port number (hint: 2375 for balena devices)
That’s an awesome addition. Just what I was looking for!
Is it also possible to expose the docker socket when in development mode? So that I can run a container on it by default that’s listening for releases (for other build-related stuff)? It’s no problem to do a simple setup once per device. But this way, I can create kind-of automated builders via balenaOS, which is exactly what I need
Found something interesting
I’ve found the io.balena.features.balena-socket label, maybe this can help me somehow? Create a container with this label, and then expose port 2375 or proxy it?
And maybe I’m wrong, but it still seems like building takes quite some time to start. And I thought this wasn’t the case before. Both on balena-cli 12.5.0 and 12.9.1. SD card of the Pi is a very fast SD card, so that shouldn’t be the issue. And networking is local (via ethernet). Debug logs show nothing between the start of the build and the actual start. Any thoughts?
So you could technically have devices running balenaOS development variants (build engines) in local mode and have some other container manage the interface between your build client and build engines, but this would be a system you would have to implement.
Thanks for your answer. I meant the development image in local mode instead of “Development Mode”. I’ve been working with Balena for quite some years now, but just getting started with openBalena, and thus coming across these problems.
But, when adding io.balena.features.balena-socket as label, I can probably expose the Balena socket through that container, right?
I’ve tried building and deploying via the -h and -p options via a RPI running balenaOS in local mode, and that works perfectly!
However, what doesn’t work perfectly, is that the deploy function doesn’t detect changes in my app. When I’m building my Node.js app from Typescript to JavaScript, a new build directory is created inside one of my Docker directories. The Dockerfile copies this build folder. But, for some reason, it doesn’t know that it’s changed. When using balena push, it knows the changes, because it builds that part again. But using balena deploy, it says no changes are detected and a new release is created (which is strange, because why create a new release without changes?). This is on balena cli v12.5.0.
So the label io.balena.features.balena-socketis described here as bind mounting the balena container engine socket into the container. That means that the container(s) can communicate with the Docker socket running on the hostOS, but the socket isn’t exposed to the external network(s).
I think you need to pass the --build flag to the balena deploy pipeline, as described here. Otherwise it just deploys the existing image, which hasn’t changed, since it hasn’t been rebuilt.
I know it isn’t exposed, so that’s why I want to expose it myself. And I succeeded.
I have created a container which does just that with nginx. I’ve made it open source so you can check it out!
It proxies on port 3000 by default, but can be changed using SOCK_PORT. Also, the server name it binds to is by default _ (0.0.0.0 for nginx). But can be changed using SERVER_NAME.
Now that I’ve made this, I’ll probably create an app for in-house use that registers itself to our servers as a builder, and we can use them as builders for openBalena. And it simple to deploy, because we’re using balenaOS, and don’t have to cross-compile .
Update
I’ve created a release and tested in on a development image and a production image. And both seem to work fine!
Only thing is, docker can’t seem to use it via my proxy, only directly via the development image on port 2375. I get the following errors:
ERRO[0000] failed to dial gRPC: unable to upgrade to h2c, received 400
error during connect: Post http://<my-ip>:3000/v1.39/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=.docker%2FDockerfile.aarch64&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&session=usrz8fiivmyhyujktapofmqwk&shmsize=0&target=&ulimits=null&version=1: context canceled
