Troubleshooting Local Mode - Balena CLI Logs

I cannot connect to my Beaglebone Green device to retrieve logs. The device is configured for Local Mode (via BalenaCloud) and I’m using the Balena CLI.

e.g. see attached jpeg of logs command

I have a BalenaCLoud starter account. I have put my device into Local Mode. I am on a windows 10 laptop (Wired eth for BBG and laptop on the same switch). I have console port access to BBG and can login via root. I have confirmed that on the BBG the eth0 has a valid IP. I can ping out from the BBG.

BBG Balena versions:
Host OS BalenaOS 2.29.2+rev3 - DEVELOPMENT
Supervisor 9.0.1
Balena CLI version is 11.6.0 (using Windows installer)

I can ssh into device using balena ssh .local. I have nothing running on the device itself other than the base Host OS.

Hey there!

Can you confirm that the device is accessible from 73a5a9e.local? Is that address getting resolved within your network? When you say you could ping the device, were you using the IP, or the local name?

What method do you want me to use to confirm the device is accessible? In the new attachment I can balena ssh into it using .local but not using the IP address alone. I could not ping the device but from the device I can ping outbound to www.google.com.

THanks,

hi @vdsilva are you able to ssh directly using ssh root@192.168.50.114 -p22222 , its possible that network you are on is blocking interdevice communication, this is often done in coworking or university networks for security reasons

able to ssh directly using ssh root@192.168.50.114 -p22222 , its possible that network you are on is blocking interdevice communication, this is often done in coworking or university networks for security reasons

@shaunmulligan Trying that command gives me a Connection timed out. We are on a corporate network so I can have IT look into something but it is generally pretty open.

Hey @vdsilva,

That’s probably the issue. The 73a5a9e.local domain will resolve to 192.168.50.114, but then you will not be able to access it in that network anyways. You will probably need some help from IT to allow the connection to happen on that network

Hey @jviotti,
I talked to another developer in my office and our IT is not blocking anything specific like ssh. We have other devices that we can ssh into on our network (like standard Beaglebone Blacks running Debian distro).

Before you read next part I decided to try out a balena os image on my device. It booted successfully and has the same behavior I had with local mode device. The os image is 2.29.2.

I had my colleague (also on a Windows 10 laptop) try to use putty and he did successfully SSH once to the device (192.168.50.114). He ran a couple of bash commands just to poke around. After a few seconds the connection was lost. He then tried to ssh in from a BeagleBone Black device he had with him that is also on our wired corporate network. He did ssh in but again after about 15 seconds or so the connection was lost. Does this help with troubleshooting?

Hi @vdsilva looks like ssh to your device works nicely via the balean VPN but is not working or getting interrupted when trying locally.
Could you could enable support access to the device, then we can take a look if we can see anything from inside…

looks like ssh to your device works nicely via the balean VPN but is not working or getting interrupted when trying locally.
Could you could enable support access

Sure @samothx! Let me just reflash back to the BalenaCloud version from the standard Balena OS and grant access. I’ll message you again shortly in the ticket when I’ve done that.

Support access is granted for the next 3 hours. Device name is weathered-darkness @samothx

Thanks I will need the UUID though to access it.

Of course.
f3ceec58d980bbc3d391221a80702de6

Ok, for starters everything looks normal. I have a stable connection, IP address is 192.168.50.114 so you should be able to connect using
ssh root@192.168.50.114 -p 22222 from a linux host or using balena ssh 192.168.50.114
I am looking at logs to see if I see why things would be going wrong

Thanks @samothx! I’m heading off for the evening in 5 minutes. I will quickly try and do a couple of SSh sessions (putty, Balena CLI and Windows 10) just to see if the logs show you something before I leave

I saw

Jul 10 23:36:32 f3ceec5 systemd[1]: Starting SSH Per-Connection Server (52.4.252.97:57926)...
Jul 10 23:36:32 f3ceec5 systemd[1]: Started SSH Per-Connection Server (52.4.252.97:57926).
Jul 10 23:36:33 f3ceec5 dropbear[1608]: Failed loading /etc/dropbear/dropbear_dss_host_key
Jul 10 23:36:33 f3ceec5 dropbear[1608]: Failed loading /etc/dropbear/dropbear_ecdsa_host_key
Jul 10 23:36:33 f3ceec5 dropbear[1608]: Child connection from ::ffff:52.4.252.97:57926
Jul 10 23:36:34 f3ceec5 dropbear[1608]: Auth succeeded with blank password for 'root' from ::ffff:52.4.252.97:57926
Jul 10 23:36:50 f3ceec5 balenad[649]: time="2019-07-10T23:36:50.413188235Z" level=warning msg="failed to retrieve balena-engine-init version: exec: \"balena-engine-init\": executable file not found in $PATH"
Jul 10 23:36:57 f3ceec5 dropbear[1608]: Exit (root): Disconnect received

looks like a connection via the VPN

Over the VPN did work. My colleague and I both tried (1 from a Windows 10 box and then from a BeagleboneBlack Debian distro) and neither of those worked.

Could you ping the host ?
If not, have you got some ping targets I can try to ping in your network or even a http host I could try a curl to. Pinging the gateway on 192.168.50.1 works OK for me.

What would your host IP be ?

192.168.50.113 is my laptop

I’m leaving for the night. Thanks for your help @samothx!