Developing on a Balena device

#1

Hi all,

I’ve created this topic to discuss with the Balena team and community users on how everyone is developing on their devices, and if there are any issues or frustrations.

This topic is focussing on HOW you are developing, not issues in your code, but how you’re pushing the containers to your device or working locally on your PC and then creating containers etc

I’ve created this topic because in another topic of mine, we were talking about the balena push command. My opinion is that it’s very slow compared to the docker-compose command (talking about multi-container applications here). I have no idea if it’s fast when using just a single-container application, because I haven’t tried it. But when pushing code to an UP Squared with docker-compose, it works instantly. When using balena push <IP_ADDRESS>, it takes a long time to start building and a long time to start the container. And the UP Squared is a relatively fast board. Without the balena push, I’m missing the Balena ENV vars and Balena labels, so communicating to the supervisor isn’t possible, which is a bummer. So I was wondering, why is it much slower than docker-compose? And is anyone else facing this issue?

For everyone’s information, @CameronDiver has explained this in the other topic:

Hey, I wrote the balena push code so I’ll go through the reasons for this.

And regarding the balena push <IP> , it takes long before the push starts and after building, it takes some time before it really shows some logs. However, when I use docker-compose , it’s all instant.

This is more noticeable with larger code repositories, but essentially it boils down to the way that balena push tars the input to stream to the docker daemon. Because of the handling of things like Dockerfile.templates etc, the pipeline has to read the input at least once, transforming it as it moves through. Add in that the node module we use for this has some inherent problems, the performance is definitely not as good as raw docker-compose. Two things I’ve been planning for this, one is to be able to skip the Dockerfile.template et al. resolution, which will improve things, and another is to start working on a tar module (or changing the current) to be more performance aware (for example we don’t need to add files one by one as we do now, if we have the size beforehand, which we do). Unfortunately these just haven’t taken as large a priority as other more fun features (--live like @shaunmulligan mentioned :slight_smile: )

Another reason for this is that the CLI must talk to the supervisor to negotiate the push, and update the target state. This is necessary, but also can be optimized for fewer calls. Factor in that those calls are often handled by an rpi or similarly low-powered device, the responses can take some time.

Basically, we have this on our radar, it’s just not a huge priority right now.

Also, stuff like docker-compose down or docker-compose kill aren’t supported (afaik) with balena push <IP> .

Again, these are going to be upcoming, we are making a lot of changes to the CLI right now, and this is definitely earmarked as one of them.

Also, this feedback is fantastic :slight_smile:

I, and probably many more developers, understand that on a RPI3, it’s just slow. Regardless if you’re using balena push or docker-compose. The RPI3 is just a “slow” device for container building and starting etc.

So, TL;DR how are you developing on Balena? And are there any issues, frustrations or features you’d like?

Thanks!

X11 hotplug USB