When flashing balenaOS to a device, it’s possible to add an SSH key to be able to SSH or tunnel into production devices, which is awesome. But I have some doubts about the security of this and maybe you guys can help me out.
We’re looking at the ability to add an SSH key after flashing the device. When a device is provisioned with cloudBalena/openBalena and the software is downloaded, the first thing it does is register itself with our server. So the device gets registered to our server, gets a serialnumber and sets the name of that device to that serialnumber on cloudBalena/openBalena. It also recieves a public and private key for communicating to our server. So far so good.
Now we want to generate an SSH key for that device and add that to the OS. Using 1 SSH key for all of our devices isn’t something that I see as very secure. Once that SSH key is compromised, someone can get into all of our devices using SSH. That’s the reason we want to generate an unique SSH key per device. For us, it’s more work to SSH into a device, because we have to locate the SSH key and use that for communication, but I think that’s better than just using 1 SSH key. But I’m here to learn, so if anyone can convince me that it’s not more secure, I’m all ears!
And the reason why we’re not generating the SSH key beforehand and also provisioning our device beforehand (like changing the config.json for the flash file)? That’s because we want to flash our devices with just 1 image. Creating an image per device is time consuming and not very efficient for our production process. And because we’d like to use the EtcherPro when it’s released!
Hi,
Sure a key per device does make sense, but it is much more important how these keys are stored and used.
For example if you use one key for your whole fleet and this key is stored in a HSM, it would be much more secure then when you store unencrypted per device keys in a database.
It should be ensured that nobody has access to the plain keys itself, but systems can only use the keys.
Just using multiple keys does not mean that this is more secure than using single key for the whole fleet. It adds a nice property that if one key is leaked only this device is compromised, but more important is how the keys are stored, used and how the keys can be rotated.
Cheers,
Thanks for your response!
The way the keys are stored is very important, I agree. The way we’d like to store them is on a (local) server that can’t be accessed easily by others. On the device itself, only the public key is stored like in every SSH transaction of course. But if we can add an extra layer of security, by using a SSH key per device, it’s an added bonus. If you have any tips regarding storing all SSH keys, it’s certainly worth knowing about!
But before we get our hopes up and investigate further in storing SSH keys, is it even possible in Balena to add an SSH keys after flashing the device?
Because of the posts in this topic, I’m not opening a new one. We’re discussing internally what’s the best practice for SSH keys. One SSH key or one per device. One SSH key is obviously easier for connecting to a device and just storing one SSH key. But per device has the benefit that when one SSH key is compromised, not all devices are compromised (although, when one is compromised, probably all of them are compromised, because they’re stored on the same storage). But as far as I know, it’s not possible to add SSH keys during runtime, apart from changing the /mnt/boot/config.json manually. Not automatically via a custom script, right?
And if we’re using just one SSH key, and for a reason it’s compromised, it’s also not possible to remove that SSH key from all of our devices and adding a new one to it. The most important part is storing the SSH private key securely, but in case of disaster, we’d like to know what our options are.
FYI, we’re using openBalena for most of our devices.
Hi, we do have a tool that helps automating config.json changes for a fleet of devices. https://github.com/balena-io/configizer. The script can be customized to be used with openBalena.
Thanks! It looks promising! But we’re looking for a solution that can be implemented in our software, so from inside a container, via env vars or via the supervisor API. But that’s not possible?
Hi, not at the moment. The configizer utility is meant to fill the gap for those configuration variables that are not yet able to be configured by the supervisor API.
sorry to ask another question am i correct in thinking i can access any balena device by adding my private ssh key to the config.json directly on the sd card in the pi then can ssh in
Hi @oly99, as long as your ssh key is referenced in the config.json of the device you want to SSH into, you should be able to SSH there. Hope that helps.
thanks i was on the correct track dropped it now come back to this lol im now in , do you happen to know if you can add several aplication ids and user id and mixpanelToken in one config json i have a device i want to run a a docker image on poss thought the balena fleet settings but the manufacturer uses their own ones of each can i add my ones also giving me access using the cloud too