Container Watchdog

Do the containers running on Balena have access to the systemd watchdog mechanism? I would like to setup a watchdog so that if my program doesn’t ping then the container can be restarted.

Thanks!

You’ll need to look into the “healthcheck” directive of docker-compose: Compose file version 2 reference | Docker Documentation which balenaOS fully supports.

Is there a way to get balenaOS to reboot entirely, not just the container?

Hey Jason, there’s no programmatic way via the docker-compose file to reboot the host OS, it would have to be a custom command in a new “watchdog” service of some kind. I think the supervisor API is likely the best approach for this.

Here’s some related documentation. Luckily the first example is a device reboot from within a service:

I don’t want a device reboot from within a service. I want to utilize the watchdog to perform a reboot if the container healthcheck fails. Devices are falling off of networks all the time and not recovering. This is a huge issue for me. I’m needing to ask people to go and manually perform hardware resets simply because the WiFi dropped for a bit, wifi-connect came up and they don’t rejoin.

Hey Jason, the container healthcheck is part of the engine and is not able to reboot the host on a failure, it can only take actions on containers. Is there an issue with creating a custom watchdog in a container that checks for network connectivity and reboots the device if needed? I understand your use case I just think it would be easiest to accomplish with a custom watchdog in a container.

Hello @jkridner i know this is an old message, however did you try to build the watchdog service on your use case? I was checking the balenaHub in case that any developer from the community would have created it, but no luck. I think that might we a great block for the Hub that would help a lot of developers/projects.

Let us know if we can help you more :slight_smile:

1 Like

I gave up. I created handlers that manually issued restarts. Not as robust as I’d like.

1 Like