Balena cli login expired very soon

openBalena
#1

Hi, I found a login issue in recent balena cli versions like 12.38.6 or 12.38.8:

Just after a successful login, I can run commands like balena devices or balena apps without any problem. But after about 1 hour, running any these commands will get error like:

> balena --debug apps
[debug] new argv=[/home/xxx/balena-cli-v12.38.8/balena,/snapshot/versioned-source/bin/balena,apps] length=3
BalenaRequestError: Request error: <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /whoami</pre>
</body>
</html>

BalenaRequestError: Request error: <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /whoami</pre>
</body>
</html>

    at Object.<anonymous> (/snapshot/versioned-source/node_modules/balena-request/build/request.js:190:27)
    at Generator.next (<anonymous>)
    at fulfilled (/snapshot/versioned-source/node_modules/tslib/tslib.js)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

For further help or support, visit:
https://www.balena.io/docs/reference/balena-cli/#support-faq-and-troubleshooting

From the log of HAProxy in open Balena server, I can see the whoami API URL path is /user/v1/whoami just after login. But after 1 hour, the URL path of whoami API becomes /whoami, which causing the 404 error, and cli existing after that.

The ~/.balena/token JWT token’s exp timestamp is 1 week later, so I think it is not due to the token expiration.

#5

Hello! Can you try v12.38.5, does the issue persist?

#7

Yes, same issue for cli v12.38.5.

#13

From the log of HAProxy in open Balena server, I can see the whoami API URL path is /user/v1/whoami just after login. But after 1 hour, the URL path of whoami API becomes /whoami , which causing the 404 error, and cli existing after that.

@yechaooo, thank you for these details. What version of openBalena are you using? I ask so that I can try and reproduce the issue. I was looking at the source code and I don’t see what could cause the CLI or the Node SDK to change the whoami URL from /user/v1/whoami to /whoami after one hour. I tested with balenaCloud as well, and I believe that both openBalena and balenaCloud should be checking the /user/v1/whoami endpoint only.

#14

Also:

  • When the request starts failing (after 1 hour), does the following command succeed? Replacing mydomain.com with your openBalena domain name:
$ curl -i api.mydomain.com/ping
...
OK

  • After 1 hour, when balena apps fails, does balena login succeed? I.e., are you able to login again after 1 hour with balena login, without having to restart the openBalena server?

  • You mentioned “From the log of HAProxy in open Balena server.” What about the logs of the api service, do they show anything at the time when the /whoami request fails? I understand that haproxy simply forwards the whoami request to the api service, so the api service logs may contain additional clues to the problem.

  • Is there any NAT router, port forwarding config, or “dynamic DNS” setup around the openBalena server? I am considering the possibility that some such configurations are expiring after one hour.

I am also considering that the /whoami message in logs or printed to the console may not really indicate that a request was made to /whoami instead of /user/v1/whoami. Perhaps it’s simply an imprecise error message. For example, I found the following CLI issue where another user reports the same error message “when not logged in”: Error handling when not logged in (openBalena) · Issue #2090 · balena-io/balena-cli · GitHub

#16

Hi. After Paulo’s comments, are you still having trouble with tokens?

#19

openBalena server version is v3.1.2.

#20

curl ping’s response is OK.

  1. When balena apps fails after 1 hour from last login, balena login can do login again with user/password.

  2. I think the log of HAProxy is actually from the API server.

  3. No.

#25

Hi there, there is a new version of openBalena v3.2.1 available, so you could try that, though looking at the commit logs, there isn’t anything in there that should affect the issue you are experiencing.

I think it would be good to verify after 1 hour has passed since login, so see if you can still make an authenticated API request with the token, as /ping is unauthenticated and will work regardless. For example, try running:

curl 'https://api.mydomain.com/device-types/v1' \
  -H "authorization: Bearer $(cat ~/.balena/token)"

If that returns a valid json response, it’s not an authentication/token expiry issue.

#26

Bump on this for me too. Expiry of session seems to be even shorter for me, roughly 20-30 minutes tops.

And I get this message which is how I know I need to balena login again:

BalenaRequestError: Request error: <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /whoami</pre>
</body>
</html>


Additional information may be available with the `--debug` flag.

For further help or support, visit:
https://www.balena.io/docs/reference/balena-cli/#support-faq-and-troubleshooting
#27

@barryjump how do you point the CLI to the open-balena instance? Via environment variables? A balena.yml file? Can you ensure the CLI is pointed to the correct address when this happens next time?

A simple way to do that would be to rerun the command that just failed but prepending BALENARC_BALENA_URL=<your-domain> to the command, eg:

$ BALENARC_BALENA_URL=... balena apps

If that works, then whatever method you used to point the CLI to your server didn’t work.

#28

Thanks for clarifying @dfunckt. I’m actually doing the following:

openbalena specific folder containing:

  • balenarc.yml

      balenaUrl: 'mydomain.com'
  dataDirectory: '.'

  • ca.crt

  • cachedUsername

  • token

And then I move a project folder inside to do a deploy to openbalena only, and move it back out when done to keep the folder clean.

As long as I’m inside that folder from the terminal all the standard balena commands work without a hitch. I posted elsewhere that I’m using both openbalen and balenacloud from the CLI so I thought that was the only way to use both environments on the same machine as needed.

Side note, I’ve been trying to just leave the terminal session open all the time inside that folder because if I close the window I have to do the export NODE_EXTRA_CA_CERTS='/path/to/ca.crt' each time I want to log in.

#30

@barryjump , @yechaooo, many thanks for reporting this issue and providing all the details that have helped us with the investigation. We believe we have found the problem in a module used by the balena CLI, which implemented authentication token refreshing through an old /whoami HTTP endpoint that still exists in balenaCloud but not in openBalena. The fix will first require a backend change to openBalena, and then the CLI can be updated. The following two GitHub issues provide further details:

You can subscribe for notifications on those issues, and we will also update this thread when the issue is resolved.

1 Like
#32

Excellent thank you. For me its not necessarily a major issue, just a minor annoyance. Btw, it’s even less of an issue now that I’ve got a pretty sweet CI pipeline to openbalena thanks to this thread: Balena push openBalena alternative

As a result I don’t need to rely on the local CLI as much to get app deployments done.