Balena tunnel results in Unable to connect: 503 Service Unavailable

openBalena
1

OS: 2.58.6+rev1
Supervisor: v11.14.0
open-balena-vpn: v9.16.1
open-balena-api: v0.109.2
open-balena-registry:v2.13.11
open-balena-db: v4.1.0
open-balena-s3:v2.9.9

I am having some trouble getting into my production devices using balena tunnel.

  1. I created a tunnel using: balena tunnel <uuid> -p 22222:22222

It opens a tunnel correctly:

[Info]    Opening a tunnel to ae2f4a8fbbf060e6d4f48589edc0cebe...
[Info]     - tunnelling localhost:22222 to ae2f4a8fbbf060e6d4f48589edc0cebe:22222
[Info]    Waiting for connections...
  1. Tried connecting to the tunnel using: ssh -p 22222 -i id_rsa root@localhost but it results in:
[Error]   127.0.0.1:51478 => 127.0.0.1:22222 ===> 100.64.0.3:22222 :: Unable to connect: 503 Service Unavailable

I checked if port 3128 is forwarded on the open-balena-server. It is. There is also nothing wrong with the id_rsa (key) because ssh’ing straight to the production device using ssh -p 22222 -i id_rsa root@<my-local-ip-to-device> does indeed work. The IP: 100.64.0.3 is unknown to me though as my open-balena-server has a different IP. Is that the problem?

This method used to work before I moved my vm from firebase to azure and before I updated to latest hostOS versions. I suspect there is some configuration missing in the open-balena-server. No alarming logs on docker logson the open-balena-vpn-container though. Also no logs on the supervisor on the device itself.

Anything obivious I am missing here?

4

Hi, did you recently update balena-cli, could you share which version you’ve been trying with?

8

did you recently update balena-cli, could you share which version you’ve been trying with?

We ask because of the following issue, although it is not clear that it is the cause: tunnel command uses unencrypted connection to backend server · Issue #2042 · balena-io/balena-cli · GitHub

Basically, balena CLI v12.38.5 or later requires openBalena v3.1.2 or later. Older CLI releases work with older openBalena releases.

10

I indeed updated balena-cli as well to the latest version v12.38.5 (forgot that version to mention, sorry).

My intention was to upgrade everything to the latest version. So I pulled the latest version of open-balena from github and started all the containers. But if I understand you guys correctly I am still running an older openBalena release? Is this not how you upgrade openBalena?

11

Hi there, if you pulled the latest you could update your openBalena installation using the scripts in ./scripts folder.

15

But if I understand you guys correctly I am still running an older openBalena release?

Perhaps partially. I seem to have managed to reproduce the error you were getting when upgrading openBalena to v3.1.2. Then I fixed it by stopping openBalena and deleting the openbalena_haproxy image:

Before:

$ docker images
...
openbalena_haproxy             latest            8b71a403b4d4   4 months ago    23MB
haproxy                        1.9-alpine        9fa263174bad   5 months ago    21.1MB

After:

$ docker rmi openbalena_haproxy

$ ./scripts/compose up 
...
Building haproxy
Step 1/6 : FROM haproxy:1.9-alpine
 ---> 9fa263174bad
...
Successfully tagged openbalena_haproxy:latest
WARNING: Image for service haproxy was built because it did not already exist.
To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
...

$ docker images
openbalena_haproxy             latest            3edeae649eaa   2 minutes ago   23MB
haproxy                        1.9-alpine        9fa263174bad   5 months ago    21.1MB

Is this not how you upgrade openBalena?

The warning message quoted above has a hint of what might be appropriate when upgrading openBalena: adding the --build flag to the ./scripts/compose up command, i.e.:

$ ./scripts/compose up --build

This might do the trick, but I’ll ask a colleague to confirm.

19

Is this not how you upgrade openBalena?

./scripts/compose up --build
This might do the trick, but I’ll ask a colleague to confirm.

We’ve come up with some upgrading steps that we’ve added to the openBalena Getting Started Guide, section Preparing a server for openBalena. These steps:

$ ./scripts/compose down
$ git pull
$ ./scripts/compose build
$ ./scripts/compose up -d

The compose down command will stop and remove the running containers, and the compose build command will rebuild images like openbalena_haproxy. Thanks for raising this and helping us improve the product! :+1:

21

You guys rock! Thanks for the help and for the awesome product!

1 Like