openBalena Production ready?

openBalena
1

Hi all,

As some of you know, we’re keen to use openBalena for our projects. Like some others, we’re creating IoT projects with different requirements and specifications. Some of our projects use cloudBalena, but we’re on the starting line of a new project and there are projects to come. Some of these projects have the ability (and funds) to use cloudBalena. But other projects just don’t have the funds to use cloudBalena and pay per-device. So for these projects, we’d like to use openBalena. But there are a couple of questions from our side, and we probably share these with other (open)Balena users.

First of all, many thanks for creating openBalena! We’ve searched the web for all kinds of IoT fleet management software, and there are a bunch, such as Ubuntu Core. But they just lack some of the features. Balena has a ecosystem that covers all aspects that you’d like when creating an IoT product, such as VPN connection, updates, pre-built images for many boards and an relatively active community. And the setup of balenaOS is great, with the balenaEngine and such!

But we’re wondering, is openBalena production ready? Obviously, I follow all threads regarding openBalena and the (missing) features and bugs, which is completely normal that there are some in a relatively new open source project. But we’d like to have an overview of what is still missing, what is on the roadmap and, the most important, what features are ready in openBalena. And the conclusion: is openBalena, at this stage, production ready? Because openBalena isn’t really documented, which I think is normal because documenting, imho, is a pain in the ass.

Our questions:

  1. Is openBalena steady/robust enough to run on a server with multiple (hundreds) of devices?

  2. Is it possible to SSH to any device that is connected to openBalena?
    I followed the threads, and I haven’t tested it yet, but it is possible afaik via balena tunnel. The only requirements are that the device is connected and has an SSH key installed in the config.json, correct?

  3. Environment variables are supported, but also Service Variables?

  4. This is one of our big questions, HostOS upgrades aren’t supported as of now, but they are on the roadmap. Is a HostOS upgrade necessary when openBalena supports it? Because our devices are going to be placed around the globe, and it isn’t possible to go to all of our devices when they’re placed.

  5. What are the key differences between openBalena and cloudBalena? (Backend only, dashboard isn’t necessary)
    As far as I know, some key differences are the delta updates, diagnostics, multi-user support (Can be implemented in a custom dashboard for example), location tracking, SSH without the pre-defined SSH key

  6. What are the hardware requirements of running openBalena on a server?

  7. I’ve read some documentation about the balena-vpn, and the max of clients is 4096, correct? And if so, is it possible, and how, to add VPN instances, if necessary, to connect more clients?

  8. What features are missing before going to production in your opinion?

I’ll update all questions when they’re answered and add questions from other people to get a nice overview

As said before, we’d like to use openBalena for one of our projects, but 1 big thing holding us back is question 4, the HostOS upgrades, and the (missing) knowledge that openBalena is 100% production ready as of now and is scalable.

We’d like to begin development on our new project as soon as possible, but our goal is to finish early next year. We know this is an open source project, and we’d love to help as much as possible by being a beta-tester with a (increasingly) large fleet or even testing features. But without the reassuring that openBalena is (becoming) production ready, you probably understand it’s a big risk and even a deal-breaker for us…

Also, we’d love to get in contact with the Balena-team to explain our situation and discuss a solution! Because, as said earlier, Balena has the whole package for managing and maintaining an IoT fleet. So feel free to contact me via email or other communication channels!

With kinds regards,

Bart

P.S. I’m sorry for all the text :slight_smile:

2

Hi Bart,

Thank you for your great questions, in order to answer them all, we might need to collect answers from many people and it will take a while. We will try to get back to you as soon as possible.

Thank you

5

Hi @nghiant2710,

No problem at all! Feel free to contact me or have anyone else contact me if you need anything from me or like to discuss some of the questions / projects!

10

Hi,

These are just my views on the topic and shouldn’t be taken as gospel.

Is openBalena steady/robust enough to run on a server with multiple (hundreds) of devices?

This is a tricky one; it is the same base-code that we use in balenaCloud so by that definition it could be considered robust, however at this time we still consider it to be a Beta product when run standalone.

Is it possible to SSH to any device that is connected to openBalena?

Yes, as you have described. Also there is a configuration environment variable `` which can be populated to represent the public SSH key to be pushed into all devices when they join your openBalena instance. It should be in Base64 format, and it would mitigate the use of the value in config.json.

Environment variables are supported, but also Service Variables?

Yes, why do you believe otherwise? If you’re having trouble with it via the CLI then I would recommend using the API directly inline with this documentation: Resources - Balena Documentation

Is a HostOS upgrade necessary when openBalena supports it?

At this point, without being able to do a HostOS upgrade, I wouldn’t personally deploy device where I couldn’t get that back to reflash in the future. This is part of the reason we consider openBalena to be in Beta.

What are the hardware requirements of running openBalena on a server?

Depends entirely on the load you expect it to run with; I have testing VPS instances with 2GB RAM and 2vCPU, and I do builds/development on VirtualBox VMs with about the same. If you have a lot of devices then I would expect your VPN memory usage to rise and require more. If you tunnel traffic over that VPN then it would require again, even more, but I cannot put a limit on this. My recommendation would be to provision a server that you can easily, and cost effectively, upgrade in the future.

I’ve read some documentation about the balena-vpn, and the max of clients is 4096, correct? And if so, is it possible, and how, to add VPN instances, if necessary, to connect more clients?

Could you link to that documentation, as it would help to understand the context it was written under? The way the VPN works is to accept connections from devices, authenticate them against the API, and then register their details with the API so that it can be used. Adding more instances, at this time, isn’t something we have detailed due to the complex way in which you would need to configure the load-balancing infrastructure infront of the instances.

What features are missing before going to production in your opinion?

The big one is HostOS upgrades. As a core team member for the open-source effort, this is certainly a topic which get’s a lot of discussion and is one which I think is really key - the purpose of openBalena is managing fleets of devices; we can push new software to them, but managing the base OS is just as important IMHO.

1 Like
12

Hi @richbayliss,

First, I would like to thank you for the detailed answer! We very much appreciate it!

Second, I’d like to comment on your answers.

Is openBalena steady/robust enough to run on a server with multiple (hundreds) of devices?

This is a tricky one; it is the same base-code that we use in balenaCloud so by that definition it could be considered robust, however at this time we still consider it to be a Beta product when run standalone.

We understand it’s still in Beta. But the fact that it’s the base-code for balenaCloud, gives us confidence that openBalena isn’t going to be abandoned by you guys. openBalena Beta still works for us, with the side note that some (HostOS updates) are necessary to function properly.

Is it possible to SSH to any device that is connected to openBalena?

Yes, as you have described. Also there is a configuration environment variable `` which can be populated to represent the public SSH key to be pushed into all devices when they join your openBalena instance. It should be in Base64 format, and it would mitigate the use of the value in config.json.

I’d like to create an SSH key per-device. This enhances the security a little, because when the SSH key is compromised, only 1 device is compromised. I’ve asked that feature before, so I will test that of course. But this reaction seems like it is possible to push an SSH key to a device with version 2.44. I’ll comment in that thread to ask what this feature actually does.

Environment variables are supported, but also Service Variables?

Yes, why do you believe otherwise? If you’re having trouble with it via the CLI then I would recommend using the API directly inline with this documentation: https://www.balena.io/docs/reference/api/resources/device_service_environment_variable/

That’s my mistake. I thought I read it somewhere. But awesome!

Is a HostOS upgrade necessary when openBalena supports it?

At this point, without being able to do a HostOS upgrade, I wouldn’t personally deploy device where I couldn’t get that back to reflash in the future. This is part of the reason we consider openBalena to be in Beta.

That answer is clear. That’s the same thought we had. We’d like to start production around April 2020 for which this feature is necessary. I understand you can’t give an ETA, but we hope for the best!

What are the hardware requirements of running openBalena on a server?

Depends entirely on the load you expect it to run with; I have testing VPS instances with 2GB RAM and 2vCPU, and I do builds/development on VirtualBox VMs with about the same. If you have a lot of devices then I would expect your VPN memory usage to rise and require more. If you tunnel traffic over that VPN then it would require again, even more, but I cannot put a limit on this. My recommendation would be to provision a server that you can easily, and cost effectively, upgrade in the future.

We had a setup in mind with 4GB RAM and 2vCPU and 150GB SSD. This seems to us like a good start. We’d like to do the builds on other (local) servers / devices and the same for development. The server is very easily upgradeable, so that wouldn’t be an issue.

Only thing is, we expect to grow with 500/750 devices per year. Those aren’t all connected to internet, but let’s say 75% of them are. I know it’s not possible to translate number of devices into required hardware, but are there any numbers available on the average RAM/CPU usage per device?

What features are missing before going to production in your opinion?

The big one is HostOS upgrades. As a core team member for the open-source effort, this is certainly a topic which get’s a lot of discussion and is one which I think is really key - the purpose of openBalena is managing fleets of devices; we can push new software to them, but managing the base OS is just as important IMHO.

Totally agree.

It’s awesome to hear you’re a core team member of the open-source effort. Balena really helps start-ups like ourself to focus on developing devices instead of focussing on a fleet management system. It’s just not always possible for us to use cloudBalena for all of our projects, so openBalena is an awesome development! I hope you guys keep up the good work, and let me know if there is anything we could do to help!

14

I have made sure your comments are seen by our open-souce focused team; they are very appreciated.

I noticed reading back my replies that I forget to mention the ENV var that the API should have to push down a master SSH authorized key: DEVICE_CONFIG_SSH_AUTHORIZED_KEYS

1 Like
22

Hi,
With regards to the amount of extra resources that are required for each extra device running on open-balena, I’m afraid we don’t have any estimate numbers to share.
Please let us know if you have more questions.

Kind regards,
Thodoris