Supervisor stuck in restart loop

Product support
1

A few hours ago the supervisor running on one of our devices started restarting endlessly. I’ve attached the logs below but they dont seem particularly relevant, save for the “Received SIGTERM. Exiting.” at the end

Currently the system is up and operating from a user-perspective, but I’m nervous to do anything to fix the supervisor that might affect the users. There is a window later today when I could do things like restart the system, assuming that might help.

Any help/insight/next steps would be appreciated!

Supervisor version: 17.0.3
balenaOS version: balenaOS 6.5.34
device type: Generic x86_64 (GPT)

2025-06-27T14:07:48.098132000Z [info]    Reported current state to the cloud
2025-06-27T14:08:28.338775000Z [event]   Event: Service restart {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:08:28.342263000Z [event]   Event: Service exit {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:09:29.163158000Z [event]   Event: Service restart {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:09:29.170253000Z [event]   Event: Service exit {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:10:29.745288000Z [event]   Event: Service exit {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:10:29.746690000Z [event]   Event: Service restart {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:11:30.659721000Z [event]   Event: Service exit {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:12:31.394823000Z [event]   Event: Service restart {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:12:31.398117000Z [event]   Event: Service exit {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:12:48.507114000Z [info]    Reported current state to the cloud
2025-06-27T14:13:32.071696000Z [event]   Event: Service restart {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:13:32.073828000Z [event]   Event: Service exit {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:14:32.976721000Z [event]   Event: Service restart {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:14:32.980270000Z [event]   Event: Service exit {"service":{"appId":2233736,"serviceId":3212766,"serviceName":"hive","commit":"8d1eb2d77ad67896956b602746bb1f69","releaseId":3545847}}
2025-06-27T14:15:08.660454000Z [info]    Received SIGTERM. Exiting.
2

I also am able to see balenad triggering the restart:
time="2025-06-27T14:15:08.476010415Z" level=info msg="Unhealthy container d2771f38c80a53d4d7109272428ad544983a6c6cd8f6fdad45d46d438dc4713c: restarting..."

3

This system appears to have a disk issue which the system diagnostics caught (check_localdisk) so thats almost certainly the problem.

4

Hi @MonKaiju ,

Thanks for updating with the root cause. As you’ve experienced, disk corruption can manifest as a bunch of strange behaviors on the device. Glad to hear you’ve resolved the issue! For reference, what storage media were you using and how old was it when you started to encounter the disk issues?

Regards,
Christina