Manage/update applications on an offline device

Can you elaborate more on:

  1. What exactly you mean by “offline”? Is it broken-offline, or merely intermittently offline?
  2. What your use case is?
  3. Which part do you want to update? The OS? Containers? Config?

I don’t really see a way to update a truly offline device, since there’d be no way of reaching it.

I would like to use balena for devices that always has a user interface and an operator on location.
Some of these machines may be required to be offline for security reasons.
I never use rolling updates, I point each single machine to a specific application release.

Most important would be to update the containers. It could be that someone inserts a USB stick in the device, logs on and performs some commands to apply an image from the USB stick.

I would this to be as close to a normal “online” upgrade as possible. So the persistent /data should be preserved, the device ID and Balena environment variables should be preserved. If the device later comes online it should not reappear as online in the balena backend, and of course report its current release.

Hi @krix, that is not a use case that we support at the moment. I’ve taken note, as this was something indeed that was requested before, and to signal towards the team that there are more people wanting it.

I have played around with a few ideas, but so far can’t think of a way that you could work around this in an offline environment, unfortunately…

What kind of security reasons the devices need to be completely offline for, by the way? Are they fully offline, or could e.g. connect through a proxy that is controlled by the location? Maybe we have some more ideas along the way.

The devices we have in the field will have data stored on the disk that is very confidential and sensitive for our end user.
We can promise and explain a lot about encryption and NDAs etc, but for some customers that just doesn’t make it. As soon as we ask for any internet connection they become concerned.

Can you explain more about what you mean by a proxy and how that could mitigate the concern?

It’s interesting that you bring this up, because we’ve been discussing such a feature internally. We have some ideas about how it could work, and a few customers interested in it. One option we’re considering is something USB-based, like you describe. It’s nice to know that you’re interested in this as well.

If you’d like, I can connect you with someone from our customer success team for further conversation.

2 Likes

We would also love the USB-based updating, just in case there is no internet connection at a customer and we would like to update their device.

So keep me posted!

Hi @brownjohnf, I’m interested in doing updates from a USB stick as well. Our use case is a fleet of devices installed in a field medical context where there is no Internet access or unreliable, low-bandwidth access that cannot handle large downloads. We’d like to be able to distribute updates to our devices via sneakernet, i.e. to provide update packages that people can download onto a USB stick and physically take into the field to apply the updates.

Could you please loop me into the conversation about this? Thank you!

If any open discussion will be offered I’d like to join too: by adding this feature balena offers a complete distribution mechanism ^^

Wauv, I am happy to see there is a big interest for this. I think this could be a killer feature for Balena. And without being too dramatic, the lack of offline upgrade options could potentially be a showstopper for us, forcing us to migrate to another solution.
Is there any thing we as a community can do to help or encourage work on this?

1 Like

Thank you for the feedback everyone, this feature request is in our radar and I’ve bumped it up for a product discussion. We will keep this thread updated.

Thank you for the feedback everyone, this feature request is in our radar and I’ve bumped it up for a product discussion. We will keep this thread updated.

Is it possible to (mis)use local mode to upgrade a device which is offline? I found that some new features have been introduced around local mode, so I wondered if I am overlooking a possible work around?

Besides that: Any news about this from your roadmap discussions?

1 Like

Its certainly possible to use local mode to upgrade a device that is not connected to our API, provided you can connect to it using a local IP. However local mode requires that you use a development image and for security reasons, development images should not be used in the field ( see https://www.balena.io/docs/reference/OS/overview/2.x/#dev-vs-prod-images ).
I’ve reached out to the rest of the team to see if they have suggestions that I’m not aware of and we’ll update you once we have more information

1 Like

Hi,
looks like offline updates - e.g. from USB - are on the road map but are not expected to be implemented this year.
So for now it seems your best chance would be local mode or getting the devices online for a short period of time.
Only other option I could think of would be using your own ‘trusted’ environment using open-balena: https://www.balena.io/open/. It is probably a bit of an overkill though…
Regards
Thomas

Yeah, I think local mode is a no-go, because of the security concerns.
Another work-around I am considering is to do a reinstall from a USB disk image, just like I normally do when setting up a new device.
The challenges here would be:

  1. How can I preserve or restore the /persistent partition in a safe way?
  2. Can I preserve/restore the device ID and other persistent Balena data as well?

Yeah, the reinstall would likely not be the best way.

  1. the device persistent data cannot really be saved/restored on another device easily. I believe this is on the roadmap as well, but further down
  2. the device ID (and thus the device) you can preserve, by generating a new config.json with the same UUID such as balena config generate --device UUID --generate-device-api-key (see balena config generate --help for more info) and adding that configuration to your image you are using to reinstall the device (e.g. with a preloaded image, so it runs your updated application). Note that the wifi settings of that option are legacy (for balenaOS 2.x need to have separate NetworkManager config files added, tose options by the cli are not effective, we are trying to make this clearer)

For further investigation, though:

  • you can add an ssh key to the config.json https://github.com/balena-os/meta-balena#sshkeys so you can have access to the device even if production device, with your own key locally
  • probably manually / with some scripting, could get the persistent data out of the device
  • if I recall, there was some work being done preloading data out of the device, will have to confirm whether that’s working and if does, how exactly?

With these pieces at hand, with an on the spot preloaded image generation might work as you describe. Will be checking it with our team and get back to you with more details.

Thanks for this input.

  1. Persistent data. Not so worried about this. Yes, I believe I can backup/restore this manually through some scripting.
  2. Device ID: generating a new config does not really sound like a scalable solution. Then I would have to provide a machine specific image to be able to upgrade offline. I would much rather have that the installation could just inherit existing device ID (and preferably inherit the persistent data as well).

But I am looking forward to hear whether your team has other input.

Hi @imrehg

I have tried this balena config generate which outputs the config.json file. Where must this be located?

I have tried to overwrite the /flash-boot/config.json in the base os image, but it seems to be the wrong config.

I am using:

balena config generate --device ${BalenaDeviceUUID} --network ethernet --generate-device-api-key --version 2.39.0 --appUpdatePollInterval 10 --output config.json

and

balena preload ./{IMAGE} --app {BalenaApplicationId} --pin-device-to-release --commit {BalenaCommitHash} --splash-image {SPLASH_SCREEN} --api-token “${token}”

The offline installation create a new device and do not reuse the old device in the balena cloud when connected again

Hi,
Depending on what you want to achieve you could use the CLI to put the config.json in the right place for you.
If you have the os image on the filesystem use: balena os configure <image> or if you have the image already flashed to disk you can use balena config inject <file>,
Kind regards,
Theodor

Tried the balena os configure command which allowed for a old balena cloud device to be reused without creating a new device