Balena tunnel results in Unable to connect: 503 Service Unavailable

openBalena
#1

OS: 2.58.6+rev1
Supervisor: v11.14.0
open-balena-vpn: v9.16.1
open-balena-api: v0.109.2
open-balena-registry:v2.13.11
open-balena-db: v4.1.0
open-balena-s3:v2.9.9

I am having some trouble getting into my production devices using balena tunnel.

  1. I created a tunnel using: balena tunnel <uuid> -p 22222:22222

It opens a tunnel correctly:

[Info]    Opening a tunnel to ae2f4a8fbbf060e6d4f48589edc0cebe...
[Info]     - tunnelling localhost:22222 to ae2f4a8fbbf060e6d4f48589edc0cebe:22222
[Info]    Waiting for connections...
  1. Tried connecting to the tunnel using: ssh -p 22222 -i id_rsa root@localhost but it results in:
[Error]   127.0.0.1:51478 => 127.0.0.1:22222 ===> 100.64.0.3:22222 :: Unable to connect: 503 Service Unavailable

I checked if port 3128 is forwarded on the open-balena-server. It is. There is also nothing wrong with the id_rsa (key) because ssh’ing straight to the production device using ssh -p 22222 -i id_rsa root@<my-local-ip-to-device> does indeed work. The IP: 100.64.0.3 is unknown to me though as my open-balena-server has a different IP. Is that the problem?

This method used to work before I moved my vm from firebase to azure and before I updated to latest hostOS versions. I suspect there is some configuration missing in the open-balena-server. No alarming logs on docker logson the open-balena-vpn-container though. Also no logs on the supervisor on the device itself.

Anything obivious I am missing here?

#4

Hi, did you recently update balena-cli, could you share which version you’ve been trying with?

#8

did you recently update balena-cli, could you share which version you’ve been trying with?

We ask because of the following issue, although it is not clear that it is the cause: https://github.com/balena-io/balena-cli/issues/2042#issuecomment-765615362

Basically, balena CLI v12.38.5 or later requires openBalena v3.1.2 or later. Older CLI releases work with older openBalena releases.

#10

I indeed updated balena-cli as well to the latest version v12.38.5 (forgot that version to mention, sorry).

My intention was to upgrade everything to the latest version. So I pulled the latest version of open-balena from github and started all the containers. But if I understand you guys correctly I am still running an older openBalena release? Is this not how you upgrade openBalena?

#11

Hi there, if you pulled the latest you could update your openBalena installation using the scripts in ./scripts folder.

#15

But if I understand you guys correctly I am still running an older openBalena release?

Perhaps partially. I seem to have managed to reproduce the error you were getting when upgrading openBalena to v3.1.2. Then I fixed it by stopping openBalena and deleting the openbalena_haproxy image:

Before:

$ docker images
...
openbalena_haproxy             latest            8b71a403b4d4   4 months ago    23MB
haproxy                        1.9-alpine        9fa263174bad   5 months ago    21.1MB

After:

$ docker rmi openbalena_haproxy

$ ./scripts/compose up 
...
Building haproxy
Step 1/6 : FROM haproxy:1.9-alpine
 ---> 9fa263174bad
...
Successfully tagged openbalena_haproxy:latest
WARNING: Image for service haproxy was built because it did not already exist.
To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
...

$ docker images
openbalena_haproxy             latest            3edeae649eaa   2 minutes ago   23MB
haproxy                        1.9-alpine        9fa263174bad   5 months ago    21.1MB

Is this not how you upgrade openBalena?

The warning message quoted above has a hint of what might be appropriate when upgrading openBalena: adding the --build flag to the ./scripts/compose up command, i.e.:

$ ./scripts/compose up --build

This might do the trick, but I’ll ask a colleague to confirm.

#19

Is this not how you upgrade openBalena?

./scripts/compose up --build
This might do the trick, but I’ll ask a colleague to confirm.

We’ve come up with some upgrading steps that we’ve added to the openBalena Getting Started Guide, section Preparing a server for openBalena. These steps:

$ ./scripts/compose down
$ git pull
$ ./scripts/compose build
$ ./scripts/compose up -d

The compose down command will stop and remove the running containers, and the compose build command will rebuild images like openbalena_haproxy. Thanks for raising this and helping us improve the product! :+1:

#21

You guys rock! Thanks for the help and for the awesome product!

1 Like