Device offline on web dashboard and cli

After placing an Intel NUC device on the production site, the device went offline on the web dashboard and cli, but the device is still reporting logs, deploys container updates, shows the correct container status…

How can I debug this? I suspect the device is not capable of connecting to the vpn service, but it would be useful to check this assumption somewhere on the device.

That sounds like it. Do you happen to have another device in the same network that we can use to hop through and investigate this issue? Otherwise, are you able to deploy one for debugging purposes?

Currently there is not another balena device on the site, but we will install one in the near future. Thanks!

Also, was the device showing connected before and dropped out of nowhere? If yes, are you aware of any changes on the local network side (especially firewall related)?

The connection issues are firewall related, we did some testing with our own vpn servers and detected some connection resets from the third party firewall.

We integrated some basic supervisor api features shutdown/reboot/update in our main app container, but I wondered if balenaOS itself can be updated without the VPN?

Hi,

This is a list of network requirements to use BalenaOS
https://www.balena.io/docs/reference/OS/network/2.x/#network-requirements

If the VPN port443 is not open you won’t be able to access your boards.

Same problem here, but no firewall in place.

That is, one of my devices is marked offline since 21 hours, but it still communicate with my own server. The device is a Raspberry Pi 3 running balenaOS 2.32.0+rev1 and Supervisor 9.14.0. It connects to the Internet via a 3G USB dongle. It was been seen online before, but now the Balena VPN connection seems to be down and I cannot SSH into it, nor reboot it to see if the VPN comes back again.
Is there a kind of automatic retry on the VPN connection?
Other suggestions to get the VPN back?

Hi,

Thank you for the report. We’re looking into this, but in the short term, is the device running a development release, or is there another online device on the same network? Either would allow access to try and investigate what is occurring on the device.

Best regards,

Heds

No, sorry. The devices runs a production image a it only has a 3G USB dongle with a a common SIM with private addressing.

Is there a kind of automatic retry on the VPN connection?
That is, is the VPN expected to retry infinitely to connect, once every 24 hours or something like that?

@daghemo the VPN should reconnect almost immediately after a disconnect but we are currently seeing some instances where the client is not restarting when the connection times out. If you are able to power cycle your device this should get it back online, but I cannot currently provide any more information on the actual issue.