Lock file with Balena OS Update

Product support
1

Will the update lock file prevent a Balena OS Update? My experiments indicates it will not.
If it does not prevent the update it is problematic for us since we need to baby sit each device and check that is in a state where an update does not cause problems for our customers.

In our application we use the update lock file to prevent the computer from restarting when in use by the customer. This lock also needs to prevent an OS Update.

16

Hi, at the moment hostOS updates ignore application update locks. What we can suggest it to automate the hostOS updates using the SDK so that it can check whether application locks are in place before triggering the hostOS update by calling to the Supervisor APIā€™s /v1/update endpoint see that it doesnā€™t return a 432 Locked status code.

18

To me it seems the use case for application locks also applies to OS updates.

The purpose of the lock is to allow bulk updates of the fleet without having to worry that you are interrupting something critical that is running. For example you do not want to interrupt a running 3D-printer with an application or OS-update.

By not respecting the lock for OS-updates you are effectively forcing customers into one of 3 bad options:

  1. Do not update the OS
  2. Manually update the OS by baby sitting devices and seeing when they are not in use
  3. Develop custom solution working around the missing locks using the supervisor API

It would be much better if your product applied lock also to OS update, could you point this out to your product management?

1 Like
24

Thanks Olof, weā€™ll discuss this internally and let you know the outcome.

26

2nd vote for lock files applying to OS Updates as well. I just broke a few things in my fleet assuming this worked. Iā€™d argue if theyā€™re not going to follow the same pattern, at least put a warning up.

I was updating a fleet of 24 devices (3D Printers). I have them set to be locked while printing. I was updating the app across the fleet, so while I was there I wanted to push out the Supervisor and OS updates.

I did three very similar actions, and one didnt act like the rest. Kinda wonky from a UX standpoint.

  1. Select all devices, pin to new release. Unlocked devices updated, locked devices go into a waiting loop.

  2. Select all devices and update supervisor. Unlocked devices updated, locked devices go into a waiting loop.

  3. Select all devices and choose OS update. Different result from the previous two tasks. Devices were reset in the middle of doing work.

Seems to me the pattern should be the same across any updates youā€™re pushing out to the fleet.

1 Like
27

Josh, I appreciate the vivid explanation! Iā€™ve added it to our internal notes. Iā€™m happy to report we have a plan to bring these 3 actions into alignment. In the meantime, we welcome any more descriptions on how this is affecting your work so we can prioritise effectively and ensure the solution is broadly satisfactory.

1 Like
29

I lost probably 30 print hours of work across the 3d printer fleet in this incident, but I only had maybe 15% of the fleet printing at the time. At the very least Iā€™d prioritize a warning text or confirmation to prevent others/larger fleets from hitting this.

Longer term, Iā€™d really like to have a ā€˜draftā€™ and ā€˜finalā€™ fleet release setting for OS and Supervisor.

Iā€™m using the draft release feature from the GitHub deployment action and would love to see a similar concept for Supervisor/OS. Basically fleet-wide settings for the preferred version

  • Fleet OS / Supervisor Version - is the version the whole fleet should be on, if new devices are added or the version is changed, the devices should queue up that update (similar to when a release is deployed and devices are on track latest). Only one Fleet Version can be chosen per hardware type

  • Draft OS/Supervisor versions - List of versions to make available to the fleet. The current pattern of letting a device on the fleet pick from any version out there is a bit loose compared to the rigor of the app release process. Let me curate a list of versions that have passed my canary tests.

  • All other OS/Supervisor versions should be unavailable on the fleet.

  • I personally would like automatic minor and security updates on my fleet. But not sure if thereā€™s broad appeal for that in this market.

Finally, I think the communication around these OS/Sup versions could be better. My current update strategy for these is ā€œjust click them occasionally when I think about itā€. I never know which one is important/security related or just ā€˜fixed a wordā€™ type releases. Thereā€™s too many for me keep up with when they rarely affect me in an immediately noticeable way.

Find a way to force minor updates on me, or at least strongly encourage me to keep up. Press me hard if thereā€™s a security update. I suspect your support team will appreciate less variation out in the Balena-verse too.

30

Hi
Great ideas!
Iā€™ll add these to a product brainstorm to see if we can address some of these ideas around automatic updates.
Thanks for your input and good ideas!!
Rose

32

Hi
We brought this up in a meeting today and the current solution is to use autohupr block to do automated updates. The developer working on autohupr is adding functionality to combine the lockfile for balenaOS and supervisor. Having the lockfile cand be scheduled with the cron for now until we build out a UI scheduler.
Let us know if you try out autohupr and it fits your need for automated updates.
Thanks
Rose