Yeah, the reinstall would likely not be the best way.
the device persistent data cannot really be saved/restored on another device easily. I believe this is on the roadmap as well, but further down
the device ID (and thus the device) you can preserve, by generating a new config.json with the same UUID such as balena config generate --device UUID --generate-device-api-key (see balena config generate --help for more info) and adding that configuration to your image you are using to reinstall the device (e.g. with a preloaded image, so it runs your updated application). Note that the wifi settings of that option are legacy (for balenaOS 2.x need to have separate NetworkManager config files added, tose options by the cli are not effective, we are trying to make this clearer)
probably manually / with some scripting, could get the persistent data out of the device
if I recall, there was some work being done preloading data out of the device, will have to confirm whether thatās working and if does, how exactly?
With these pieces at hand, with an on the spot preloaded image generation might work as you describe. Will be checking it with our team and get back to you with more details.
Persistent data. Not so worried about this. Yes, I believe I can backup/restore this manually through some scripting.
Device ID: generating a new config does not really sound like a scalable solution. Then I would have to provide a machine specific image to be able to upgrade offline. I would much rather have that the installation could just inherit existing device ID (and preferably inherit the persistent data as well).
But I am looking forward to hear whether your team has other input.
Hi,
Depending on what you want to achieve you could use the CLI to put the config.json in the right place for you.
If you have the os image on the filesystem use: balena os configure <image> or if you have the image already flashed to disk you can use balena config inject <file>,
Kind regards,
Theodor
In many industry projects, remote access to edge devices is not possible as they are deployed in isolated production networks, without internet access. Because of this, instant updates or immediate help for customers is not generally available. Using balena, the speed at which new features and updates are provided to Bosch customers helps to maximise product usability and reduce time frames between product upgrades and fixes.
Itās possible that the article writer/editor confused a number of features about the networks and the devices. But I was really hoping that this meant that devices on isolated networks could be updated across the network. We have devices sealed inside of IP67 enclosures in production dairy farm milking parlors. Opening up those enclosures to swap SD cards or connect USB keys is not an easy option. Yet weekly/monthly visits to the farms is performed for various maintenance tasks where they could connect to the network and provide updates to balenaOS devices.
Not much to add to the conversation, but just wanted to share our use case in case Balena devs want more details.
Our product will be sold to research companies that may have sensitive information or company policies that prevent a device from accessing the internet. (We know this for a fact from our first product)
I would like a way for the customer to be able to update the application software on their machine. Our product will have a HDMI and USB output. Interested in updating our containers, updating the OS is a nice to have.
My preferred approach:
We instruct the user to download a file from our website in an internet enabled computer.
Customer copies file to a USB and inserts USB into the product.
Customer installs / extracts the file, and the containers are updated on next reboot.
Alternative approach
We send a fully updated flashed SD Card to the customer.
User opens up the unit and swaps out the SD card.
Shipping a SD Card across the world is cheap. This solution is doable, but I dislike it as we would lose persistent information and the user will have to wait days / weeks to do this.
Also dislike it as the user now has to physically open the unit, which has warranty concerns. Implementation details have to be ironed outā¦i.e - How do we associate the new SD card to the old one? Do they get a new serial number?
Clunky Approach
User opens up the unit and removes the SD Card
User inserts the SD Card into an off the shelf Pi
Connects Pi to the internet and performs an update.
User inserts SD card back into the product
I dislike this approach as it shatters the user experience. (āOh, theyāre just using a Piā) and now they need physical access to a Pi.
Also, the off the shelf Pi now needs to connect to the internet, which againā¦ may not be possible.
If we can solve for the worst case scenario (their device never connects to the internet or a local LAN) then we have a solution for everything I think.
Nice description of the not so nice options. We have exactly the same needs. And I still see the option to ābackup device data + reinstall + restore dataā as the least poor solution.
@imrehg: Did you check if there is some work to benefit from in regards to what you wrote:
if I recall, there was some work being done preloading data out of the device, will have to confirm whether thatās working and if does, how exactly?
Hi @jason10@chuyzoz and @krix . Offline updates is something we have been looking at a bit in the recent months. In quoted passage from @jason10, I believe the article was talking rather about balena Machine which is our on premise product that is capable of running completely disconnected from the internet and can deliver updates to lan connected devices all over a private network. This is one solution for the problem described above and allows a user to visit the network periodically and roll out new releases, however this is an expensive option as it requires us managing a onPrem instance for you.
We have been looking very seriously at how to design USB updates, the major factor at the moment is security (from what I understand) as you no longer have the protection of the HTTPS in the transport, so doing a simple implementation of this would open up the devices to be easily taken over. We want to provide a great and secure experience to do this but for now have no ETA as to when this would be available, but from my side I would like us to build it out this year.
Thanks @shaunmulligan. It would be sufficient for our needs to be able to update using a laptop on the local network rather than a USB key. An asymmetric key pair would probably be sufficient.
Thank you for that suggestion. We will try to keep that in mind when we make a move at this. Unfortunately there is not yet any active development on this.
Does anyone understand if this describes a solution to this general offline-upgrade problem? I donāt understand the workflow and limitations for this approach.
Hi, those are system level operations that are not supported for production devices. For context, they are used internally by balenaCloud when performing hostOS updates. They could be called locally to perform an unsupported update, but at the moment they still require a network connection. There are a couple of open PRs to support offline hostOS updates (https://github.com/balena-os/meta-balena/pull/1826 and https://github.com/balena-os/meta-balena/pull/1827), but as I say the only currently supported update model is using the dashboard.
Hi, just wanted to know if you are working actively on offline device updates?
Or are there any news about tricks for us to make a reinstallation of a device, but preserve the persistant data incl device ID?
Iād also like to voice my support for this issue. chuyzozā preferred approach seems good to me as well.
Some thoughts on security: A public key on the device that confirms the signature of the update is probably enough.
The offline update functionality could be disabled in the supervisor be default and needs to be enabled through config or API that can be accessed only within the docker network or by privileged containers.
This way, we could limit our users to activate offline update capability through an interface thatās only available on the physical device itself and not through the local network.
If you choose to offer this option, I think itās fair to lay the burden of keeping the device safe on the developer. If physical access to a device by the customer is possible, thereās only so much that can reasonably achieved in regards of security, without involving actual HSMs.
Please allow me to repeat/rephrase my above questions in the hope for a comment from Balena folks:
Is Balena working actively on offline device updates?
Are there any tricks to preserve the persistant data through a reinstallation except keeping it on a second disk?
Is there any way to reestablish the former device ID after a reinstallation? Something like remounting file system with write-access and then changing the device ID wherever that may be stored?
Hi @krix. Currently, we arenāt actively working on offline updates but have a basic design of how we would like to do it but it has not been prioritized yet on our roadmap.
2.) I think for now your only option would be to copy the data onto an external drive as we donāt have a simple mechanism to preserve data.
3.) I believe could do this by copying the config.json from the /mnt/boot/ directory on the old device into the new one. Obviously you canāt have both of them running and connected to the server simultaneously.
Looks like a few people on this thread have shown interest, so I figure it is worth updating here. Please let us know if you have any questions or feedback.