cannot ssh into host device

Hi,

I followed the getting started guide to create an openBalena instance on AWS EC2. Now I’m able to have my device(Raspberry Pi 4 with a development BalenaOS image) connected and succeed in deploying the example project. But I can’t ssh into the host device by following HowTo: SSH into host device. It shows

$ balena device 1974287
== YOUNG WOOD
ID:                 2
DEVICE TYPE:        raspberrypi4-64
STATUS:             inactive
IS ONLINE:          true
IP ADDRESS:         192.168.1.180
APPLICATION NAME:   myApp
UUID:               197428716162de80814f68f5ad39dc61
COMMIT:             ed47cfad28ca9fe4871b13fafa1db05b
SUPERVISOR VERSION: 10.2.2
OS VERSION:         balenaOS 2.41.0+rev4
DASHBOARD URL:      https://dashboard.****/devices/197428716162de80814f68f5ad39dc61/summary
$ ssh root@197428716162de80814f68f5ad39dc61.balena
error: connect() failed: [60] Operation timed out
ssh_exchange_identification: Connection closed by remote host

Notice that I’m using MacOS Catalina, balena-cli 11.21.4.
Any help is appreciated!

Hi,

Since you’re running openBalena on AWS EC2, can you please check if your inbound/outbound rules are properly configured? We need some specific ports for our services to work, you can find more details in our docs here https://www.balena.io/docs/reference/OS/network/2.x/#network-requirements.

Hi,

Thanks for your reply.

I set inbound and outbound rules to allow all traffic but I still can’t ssh into the host device. It comes with same error message.

@jacques7zhu could you try with the balena tunnel ... command please? It will make a port available on your localhost connect through the proxy to the device; so you can tunnel port 22222 from the device to your machine and connect with ssh -l root -p 22222 localhost

The tunnel command would be something like balena tunnel {uuid} 22222:22222

hi @richbayliss

I tried with balena tunnel {uuid} -p 22222:22222 and it’s stuck with waiting for connections

 balena tunnel  197428716162de80814f68f5ad39dc61 -p 22222:22222
[Info]    Opening a tunnel to 197428716162de80814f68f5ad39dc61...
[Info]     - tunnelling localhost:22222 to 197428716162de80814f68f5ad39dc61:22222
[Info]    Waiting for connections...

After several minutes, It comes to an error

[Info]    Opening a tunnel to 197428716162de80814f68f5ad39dc61...
[Info]     - tunnelling localhost:22222 to 197428716162de80814f68f5ad39dc61:22222
[Info]    Waiting for connections...
[Error]   127.0.0.1:50300 => 127.0.0.1:22222 ===> 10.240.0.3:22222 :: connect ETIMEDOUT 52.81.44.249:3128

Hi. I noticed that in your original post the device is marked as inactive. Can you check to see if that is still the case? It should show either as online or offline.

If it still inactive, the device probably never connected to your openBalena instance, and in that case you wouldn’t be able to connect to it through it.

Hi @Ereski. Running balena devices shows that the status is Idle but running balena device {uuid} shows that the status is inactive. However the IS ONLINE attribute shows true in both cases.

b.t.w balena logs 1974287 -t gives an error BalenaDeviceNotFound: Device not found: 1974287. I’ve checked the uuid.

I think the device is connected to my openBalena instance because after deploying some code by running balena deploy myApp --logs --source . --emulated, I ssh into the device from my laptop on the same network and I have verified the application code is correctly deployed on the device.

I tried with balena tunnel {uuid} -p 22222:22222 and it’s stuck with waiting for connections

That’s how it is supposed to work. :slight_smile: The balena tunnel command does not exit: it stays there “waiting for connections” on your laptop. Then you can open another terminal window/tab (on your laptop) and run something like ssh -p 22222 root@localhost. At that point, you should see activity on the balena tunnel output, as it will be tunnelling / forwarding the ssh connection over the VPN link to the device. But the output might be the same you had already seen:

[Error]   127.0.0.1:50300 => 127.0.0.1:22222 ===> 10.240.0.3:22222 :: connect ETIMEDOUT 52.81.44.249:3128

By the way, is 52.81.44.249 the IP address of your AWS EC2 instance running openBalena?

I ssh into the device from my laptop on the same network and I have verified the application code is correctly deployed on the device

Nice. But if I recall correctly, the application deployment on the device does not require a VPN link to succeed, whereas ssh (via cloud) requires it. This suggests that openvpn might be failing to establish the VPN tunnel. If you use that ssh session from your laptop to the device on the same local network, and run the following command on the device, does it show any errors?

# on the balenaOS device, host OS prompt:
journalctl -au openvpn

I set inbound and outbound rules to allow all traffic

Perhaps worth checking / remembering that AWS has security groups and network ACLs, and both would have to be configured to allow traffic through:

Hi @pdcastro

Thanks a lot for your explanation and help.

52.81.44.249 is the IP address of the AWS EC2 instance running openBalena.

The output of journalctl -au openvpn is https://pastebin.com/9thDUdTm.
I also attached the logs of dmesg and resin-supervisor. I hope this will help.
https://pastebin.com/fGtXdfpS
https://pastebin.com/patPZqjv

I checked the networks ACLs. It seems to allow all traffic too.

Hello and thanks for the detailed report. Did you also try opening the tunnel via balena tunnel {uuid} -p 22222:22222 and once you get to the waiting for connections... open up another terminal window and try ssh -p 22222 root@localhost? If so could you also provide the output of the two commands after the ssh connection fails?

@nazrhom

Yes, I tried this. Below are the output of the two commands.

$ balena tunnel 1974287  -p 22222:22222
[Info]    Opening a tunnel to 197428716162de80814f68f5ad39dc61...
[Info]     - tunnelling localhost:22222 to 197428716162de80814f68f5ad39dc61:22222
[Info]    Waiting for connections...
[Error]   127.0.0.1:62660 => 127.0.0.1:22222 ===> 10.240.0.3:22222 :: connect ETIMEDOUT 52.81.44.249:3128
$ ssh -p 22222 root@localhost
ssh_exchange_identification: Connection closed by remote host

Hi,

Could you run the ssh command with debug output please? The command would be ssh -vvv -p 22222 root@localhost, this should give us more clues. Thanks.