balena supervisor status stucked on "Downloaded"

Product support
1

Hey,

I want to use the finabler to be able to power down my balenaFIN. When I try to send the http request " sleep/5/120" to finabler service, I have the following error in finabler service log :

“Device is not Idle, likely updating, will not shutdown”

.

Based on the source code of finabler, before shutdown the device, the service check supervisor state. The supervisor state is getting via the supervisor api : Interacting with the balena Supervisor - Balena Documentation

In my device, the supervisor return :

{“api_port”:48484,“ip_address”:“192.168.1.116”,“os_version”:“balenaOS 2.80.3+rev1”,“mac_address”:“B8:27:EB:91:6C:61 48:A4:93:04:91:80 48:A4:93:04:90:80”,“supervisor_version”:“12.7.0”,“update_pending”:true,“update_failed”:false,“update_downloaded”:false,“status”:“Downloaded”,“download_progress”:null}

Returned status is “downloaded” while my device is not in update process. Moreover based on supervisor api documentation, status “Downloaded” is not in status list.

Can I have help to understand my supervisor status ?

Thanks,
Ben

3

Hi,

We’ve recently been seeing a bug in the Supervisor where the device won’t be running the target release, but the Supervisor logs say that the target state was applied successfully. Does your dashboard show anything the current commit not matching the target commit, or a status other than “Running” or a commit other than the target for any of the services in the service table? If so, this issue is because the Supervisor database’s image table contains outdated entries. We have a GH issue to track this here: Incorrect release commit · Issue #1579 · balena-os/balena-supervisor · GitHub. Please let us know if this is the case.

Regards,
Christina

5

I don’t have this problem in my side. But the status returned by the supervisor is always “Downloaded”.

Ben

6

Hello Ben,
does this “Downloaded” state persist even after you update your application? Do you run in local-mode?

8

Yes the status persists after a update operation. I’m not in local mode

11

Hi Ben, if the status is still “downloaded” can you provide support access to the device and I’ll take a look at the Supervisor database my colleague referred to earlier to see if there are any outdated entries that can be removed?

13

I activated the support access.
my device : balena dashboard

Ben

15

Hey Ben - I came to take a look at the device, but it’s now offline. I’m on support for the next couple of hours, if you want to get it back online and ping me. :slight_smile:

Phil

16

sorry. my device is now online

17

Hey,

Since your investigation do you have some news about my problem ? All my devices are in this case. I must be able to power down devices via the bgm111.

Ben

18

Hello @benat could you please grant support access to the device so we can try to help you? Thanks!

19

https://dashboard.balena-cloud.com/devices/ea70a90bbe3fe5d04b45de47f7bcec0d/summary
it’s done

21

Thanks benat! we are checking why it’s on Exited mode. Do you think you can try to make a new balena push and into the co-processor service make a tiny change? a comment or whatever? to see if then it gets Running?

23

My co-processor service is disable by default. I can enable it with environment variable. I add the env var . The service is now running.

25

Hi again, thanks for enabling support. We are actively working on a fix for this issue on the supervisor and it should hopefully be fixed soon. We know this happens when updating to a new release where some only some services change. This should be perfectly acceptable, but due to some supervisor optimizations this leads to data being improperly cleaned up on the supervisor and causing the issue you see under some conditions. We’ll keep you updated about the fix.

1 Like
28

Thanks for your feedback. I hope this issue will be fixed soon.

Ben

29

Ben we opened this issue in our Github repository Incorrect release commit · Issue #1579 · balena-os/balena-supervisor · GitHub and once this will be closed we will let you know! Thanks for your patience!

31

Hello, we are following up on this support ticket because your device experienced an issue where it contained some metadata about a previous release which would display confusing states in the dashboard: Incorrect release commit · Issue #1579 · balena-os/balena-supervisor · GitHub.

We resolved the issue with Use tags instead of dockerIds to track supervised images in docker by pipex · Pull Request #1749 · balena-os/balena-supervisor · GitHub and have made the fix available in the dashboard via self-service upgrades (Self-service Supervisor Upgrades - Balena Documentation). When upgrading devices simply select a Supervisor version 12.10.1 or greater. We will be releasing OS images with this version by default soon so freshly provisioned devices will already contain the fix.

33

Hey thanks for your job. After updating my device-supervisor, the supervisor status is correct (IDLE after a release update). We hope to have a new balenaOS version soon.

Nonetheless we found a bug on the ssh access via balenaCloud terminal after the supervisor updated.

image (1)
image (1)795×705 35.7 KB

34

Hey, there was an issue with our proxy service for < 24 hours which prevented devices with Supervisor 12.10.1 to SSH to the target containers via the web terminal. That regression was resolved yesterday so your devices should be fully operational by now.