SSH (using balena tunnel) gives 'No supported authentication methods available'

I am sorry for the silly questions lately.

The following used to work for me to SSH into production devices (based on this topic):

  1. Use balena tunnel to forward the ‘ssh port’:
    balena tunnel <uuid> -p 22222:22222

Fetching device by UUID fc0 (string)
[Info] Opening a tunnel to fc09dd5bcbcf6e81cda084c5e6f98ff0…
[Info] - tunnelling localhost:22222 to fc09dd5bcbcf6e81cda084c5e6f98ff0:22222
[Info] Waiting for connections…

  1. Use putty (or linux SSH) to the forwarded port using root@localhost and port 22222 => ===>

So the forwarding seems to work. I also ‘reach’ the device but there is something wrong with my public-key.

Putty shows this error:

Using WSL I get some more information:

I don’t really understand the problem. Before I could connect to my devices just fine. I even removed my devices and flashed them again. They are listed in my balena devices command. I can ping the devices inside my network and get responses. Also; ssh’ing to my development-devices does work. But I guess that’s because the security is lacking in that imagefor development purposes. You can connect as root without a password.

Maybe it is worth mentioning (as I think it is correlated): deploying an app to my devices using balena deploy <name> --logs --source . --emulated works. I see the images are being pushed to the registry and get the unicorn message etc. But those devices don’t actually start the application. My guess is that the application is not pushed correctly to those devices maybe due to the same error? My OpenBalena probably gets the same publickey error while pushing to those devices. I don’t know how to verify that though.

Hardware: RaspberryPi 4
Balena CLI: 11.36.0 (both linux and windows)

Hi ,
the error message from WSL points to a changed host key which is not a surprise if you go through localhost.
Edit your //home/gerben/.ssh./known_host file and delete line 2 which contains the offending host key, then try again with WSL.
Regards Thomas

I removed the entry on line 2.

Now it shows the error:

The authenticity of host ‘[localhost]:22222 ([]:22222)’ can’t be established.
ECDSA key fingerprint is SHA256:juXXXXXXXXzw.
Are you sure you want to continue connecting (yes/no/[fingerprint])? [fingerprint]
Please type ‘yes’, ‘no’

After typing yes I get:

Warning: Permanently added ‘[localhost]:22222’ (ECDSA) to the list of known hosts.
root@localhost: Permission denied (publickey).

Typing ‘yes’ is the reason the second entry is added into .ssh/known_hosts. I don’t fully understand why ‘The authenticity of host can’t be established’.

I’m having this problem on two development machines. I guess something changed on the OpenBalena instance.

the ‘The authenticity of host can’t be established’ message only pops up on first access for the device, after you type ‘yes’ and the entry is created in .ssh/known_hosts it should no longer appear.
The original error about remote host identification changing is because you are using port 22222 for multiple devices, the entry in .ssh/known_hostsis identified by a host and a port, which in your case is localhost and 22222. When you open the tunnel to a different device, the identification will obviously not match.
As for the ‘root@localhost: Permission denied (publickey).’ error, this is the expected behavior for production images. It should work without authentication for development images but for production images you need to provide the ssh key in the device’s config.json as pointed out here:
Could you please also confirm that balena ssh $DEVICE_UUID gives you access?

Thanks for helping me understand the problem and it makes sense now.
The balena ssh $DEVICE_UUID gives the same ‘Permission denied (publickey)’.
To my understanding the balena ssh mechanism is not compatible with Open Balena so I guess this result is expected.

What I don’t really get is this. So I need to add the ssh key to the device its config. But I can’t access it right now. Do I need to reinstall BalenaOS on my device with the config.json preconfigured with ssh keys?

you can set up the API service to provision the device with the public key. In order to do that you need to set the DEVICE_CONFIG_SSH_AUTHORIZED_KEYS environment variable on the API service with your public key (e.g. ssh-rsa AAAAB3Nz...) and then reboot the device. The device will then pull the public key and you should be able to connect. Please let us know whether that solves the issue or if you have any other questions.

So I’ve tried the steps you suggested but I’m kinda new to this. I still get the error so the following is what I did:

  • I went to my Open Balena instance and went into the balena/open-balena-api container using docker exec -it <containerid> /bin/bash
  • I added the environment variable using export DEVICE_CONFIG_SSH_AUTHORIZED_KEYS="ssh-rsa AABBCC....="
  • I rebooted the device
  • Started a tunnel with balena tunnel <uuid> -p 22222:22222
  • Tried to ssh to it using WSL with ssh -i id_rsa localhost -p 22222
  • Still the Permission denied (publickey) error

Did I enter the DEVICE_CONFIG_SSH_AUTHORIZED_KEYS environment variable correctly? The name suggests it can contain multiple keys. I am not sure what format the value should be in.

As openBalena is still in Beta, it would be best to try with 1 public key. Can you verify that you tried and it didn’t worked, following the steps provided by mtoman?

Yes I tried the steps in my previous post based on mtoman’s suggestion and yes I still got the permission denied (public key error)

Hi there, based on this guide and this template, you would need to export OPENBALENA_SSH_AUTHORIZED_KEYS='ssh-rsa ...' on the host before running the openBalena quickstart script.

I ended up adding the ssh-key manually to my config.json instead my device having it pulled from the server. Luckily I don’t have that many production-devices yet so I will make sure the images are flashed with the correct ssh-keys. Thanks for the help people.

good to hear you were able to make it work. Feel free to come back to us with any other questions.