Can't Connected or Reboot Remote Device - Request error: tunneling socket could not be established, cause=socket hang up

Product support
1

Hi,

I’ve got 2 RPi3’s remotely deployed and both report as being online in the Cloud GUI. One is fully functional but the other has “frozen” in the logs, and won’t allow me to connect to any ssh session (Host or Application). It responds to ping locally (from the other deployed device).

Any actions I try in the GUI (including reboot) result in:
Request error: tunneling socket could not be established, cause=socket hang up

And attempting access to the Web GUI of my application fails also.

I’d love to get it rebooted and reconnected from here, as these are deployed in another country, is there anyway I can reboot one from the other? SSH as pi/root or something?

Thanks,

Nick

3

Hi,

Is the misbehaving device running a production or development build?

If it’s running a development build you can ssh on it as root on port 22222.

Regards!

6

Oh how I wish it was a development build. I did already try ssh’ing in that way. Can you switch from production to development without redeploying? If so I’ll get someone to reboot it on site next week and definitely make that change for future deployments.

7

Hi,

You can’t switch between development and production without redeploying.

There is a way for a support agent to ssh on a device that is offline(as long as it is connected to the network) using as a gateway an online device from the same network.

For this I would need you to grant support access to the device that is online and provide me with the long UUID of the working device and the long UUID of the device that seems offline.
The long UUIDs can be seen in the dashboard link when clicking on a specific device.

Please don’t paste them here as this is an open forum. Send them to me at sebastian balena.io

Regards!

10

That’s great news, here’s hoping it works. I’ll definitely try to work a remote reboot or scheduled reboot into my application (or even a cronjob in the host OS?) in future just in case.

Have sent you that email.

13

Hi,

I received the credentials.

Will let you know how investigation went.

17

Hi,

Unfortunately we still can’t ssh to your broken device, even using the working one as a gatewy.

Indeed, your broken device replies to ping on 192.168.2.4.

But it seems that the ssh port is closed. Probably ssh daemon is dead.
Tested with this command:
curl -m 2 192.168.2.4:22222
curl: (7) Failed to connect to 192.168.2.4 port 22222: Connection refused

For a working board the SSH daemon version should have been returned.

So your board seems to be connected to VPN , because it appears online, but can’t launch SSH server anymore.

This could mean SD card corruption or something else. We are curious too to find out what could it be.

If you say that the board is up on a pole, maybe the RPi is out of the working temperature range and it started misbehaving.

Through experimentation we found out that Sandisk Extreme Pro cards are less prone to corruption issues.

Also you could consider BalenaFin board which uses an eMMC instead of SD card to avoid such issues.

Regards!

19

Hmm, thanks for trying. What do you reckon the chances are of it rebooting normally on a power cycle? I might be able to get someone to trip a fuse or something.

Or do you think I’d be better off sending a new SD card over “in case”?

20

Definitely worth trying power cycling it first and if it still doesn’t work, then replacing the SD card.

22

It’s no longer reporting that its connected to the VPN which is interesting. I might prepare a replacement SD card, and test a unit here powered by a display (at least then I’d have a way of power cycling the device on failure from the ground). Not sure if the standard 1A will cut it, but will see.

I’ll report back what the reboot does if anything.