Device not picking up changes to device variables

Our device arrived at the physical destination and our team powered it up. One of the problems discovered was that it had an incorrect setting for the device variables.

Change the variable value through the balena.io device variable page (this is a variable that does not have a definition for the whole fleet. Typically this results in the container being restarted but nope. No restart.

Hmm… ssh to the device through the balena.io dashboard works, device variables look fine for everything but this one variable. Change it a few times. No effect on the device. No container restart, no variable value change.

Restart the device through the balena dashboard. Variable value is still wrong.
Shutdown the device through the balena dashboard. Remote team powers it back up. Variable value is still wrong.

Try changing a different variable. Boom, device has picked up changes to both variables.

Huh?

Support access has been granted:

I forgot to push the commit to our repository so ignore this stuff…

Well, hmm, what source release is it running, was there something weird about it because the app isn’t up to date. Github cannot find the release hash for this device! It finds the release hash for other devices.

claims git hash 55e76bc8370321995d71453737ba0286620f7dd6 but that’s impossible (not in our repository) and since the device has updated to a more recent version of the app, it’s doubly impossible.

Hey @jason10

Apologies for the delay, it sounds to me like the device’s supervisor could not update it’s state with the API for whatever reason. This would explain why the variable was not updating.

It seems the device is offline now, but if you give us a ping in this thread once it’s back online (again ensuring that support access is enabled) we might be able to see exactly why this was occuring.

Hi Cameron,

I have set up an odroid-xu4 with the same software, and set device variables after having the device update it’s image from the balena cloud servers. Then I set the device variables. The local environment doesn’t have the device variables that I defined (printenv | grep -i zotac)

https://dashboard.balena-cloud.com/devices/c8aba0ee94cfc5e62a1a8e7fa4d97256/summary

I wonder if this is related to App lockfile or supervisor lockfile working too well

Hi @jason10 I just checked the device, and indeed the problem is that the update lock is taken, which means the supervisor can’t restart your app for the env vars to be updated. If you clear or override the lock, the changes should be applied.

Ok, thanks!