Balena deploy fails with error `unauthorized: authentication required` from the registry service

openBalena
1

I followed this official tutorial to set up openBalena.
The only thing I did differently was to include the -c flag to the ./scripts/quickstart command as it was suggested in the official screencast. Everything worked fine and I am able to add apps and devices to the system (this means that balena apps and balena devices commands work flawlessl.

However, when I try to push a new image to the app and execute balena deploy <appName>, I get the following error:

Retrying "registry.balena.mydomain.ee/v2/<token>:latest" after 2.00s (1 of 3) due to: Error: unauthorized: authentication required
Retrying "registry.balena.mydomain.ee/v2/<token>:latest" after 2.80s (2 of 3) due to: Error: unauthorized: authentication required
[Info]    Saving release...
[Error]   Deploy failed
unauthorized: authentication required

How can I fix this issue?
Are there any workarounds?

5

Hello @eeriksp,

Welcome to the forums! :slight_smile:

Can you please provide more details regarding the following:

  • Which openBalena and balenaCLI versions are you using?
  • Did you login using balena login after running the quickstart script?

Thanks

7

Hello, @rahul-thakoor,

Thank you for your quick response! :slight_smile:
I am using Balena CLI version 12.26.2 and openBalena version 3.1.0.
I logged in with balena login using the Credentials options.

Is there any other information I could provide?

9

Hello @eeriksp ,

Something probably went wrong in one of these three steps:

Did you experience any troubles here or saw something out of the ordinary? Have you tried running them again to see if something improves?

Also, are you able to ping your server from your local machine? curl -k https://api.mydomain.com/ping

let us know if it helps, or if you need more info here

regards,

Juan

11

Juhhu! I finally got it working. :partying_face:
I started from the very beginning with a fresh machine and strictly followed the tutorial. In particular compared to the previous attempt:

  • I did not use the -c flag for ./scripts/quickstart
  • I installed docker.io a shown in the tutorial (previously I used a version of Docker CE which was automatically installed by DigitalOcean)

Thank you for your quick responses! I must say I really love the developer experience of working with Balena (both cloud and open versions). Also, your open-source friendly business model is very inspiring for us and we want to have a similar policy with our own solutions as well.

12

Hey Eerik

Glad you got it working and thanks for the kind words.

Cheers

14

@JuanFRidano ,
Is possible to ping registry service as well ?
curl -k https://registry.mydomain.com/ping
I am able to ping api but not registry.

15

There is no /ping endpoint on the registry host, you can just try curl -k https://registry.mydomain.com/ instead.

17

@ab77 , Thanks. For - curl -k https://registry.mydomain.com/ I can see 200 response in registry logs. Though I don’t see ‘OK’ on client side.

18

Cool. That’s expected. I see the same result when I query balena-cloud registry.

20

Hi!

I am experiencing the same error… I am a openbalena v2.0.0 user and I recently update the root/vpn certificates through re-running the quickstart script on my server

I am using the balena.-cli version:

balena --version
12.2.2

I get this error while trying to deploy an application:

DEBUG=1 balena deploy myApp

I got this error:

[debug] original argv0="/usr/local/lib/balena-cli/bin/node" argv=[/usr/local/lib/balena-cli/bin/node,/usr/local/lib/balena-cli/bin/run,deploy,myApp] length=4
[Debug]   Parsing input...
(node:97932) ExperimentalWarning: The fs.promises API is experimental
[Debug]   Loading project...
[Debug]   Resolving project...
[Debug]   docker-compose.yml file found at "/Users/matteo/Desktop/iottacle/balenaApp/myApp"
[Debug]   Creating project...
[Info]    Everything is up to date (use --build to force a rebuild)
[Info]    Creating release...
[Debug]   Tagging images...
[Debug]   Authorizing push...
[Info]    Pushing images to registry...
Retrying "registry.mydomain.com/v2/token:latest" after 2.00s (1 of 3) due to: Error: unauthorized: authentication required
Retrying "registry.mydomain.com/v2/token:latest" after 2.00s (1 of 3) due to: Error: unauthorized: authentication required
Retrying "registry.mydomain.com/v2/token:latest" after 2.00s (1 of 3) due to: Error: unauthorized: authentication required
Retrying "registry.mydomain.com/v2/token:latest" after 2.80s (2 of 3) due to: Error: unauthorized: authentication required
Retrying "registry.mydomain.com/v2/token:latest" after 2.80s (2 of 3) due to: Error: unauthorized: authentication required
Retrying "registry.mydomain.com/v2/token:latest" after 2.80s (2 of 3) due to: Error: unauthorized: authentication required
Retrying "registry.mydomain.com/v2/token:latest" after 3.92s (3 of 3) due to: Error: unauthorized: authentication required
Retrying "registry.mydomain.com/v2/token:latest" after 3.92s (3 of 3) due to: Error: unauthorized: authentication required
Retrying "registry.mydomain.com/v2/token:latest" after 3.92s (3 of 3) due to: Error: unauthorized: authentication required
[Debug]   Saving image registry.mydomain.com/v2/token
[Debug]   Untagging images...
[Info]    Saving release...
[Error]   Deploy failed
unauthorized: authentication required

Error: unauthorized: authentication required
    at Stream.<anonymous> (/usr/local/lib/balena-cli/node_modules/docker-progress/index.js:53:19)
    at Stream.emit (events.js:198:13)
    at Stream.EventEmitter.emit (domain.js:448:20)
    at drain (/usr/local/lib/balena-cli/node_modules/through/index.js:36:16)
    at Stream.stream.queue.stream.push (/usr/local/lib/balena-cli/node_modules/through/index.js:45:5)
    at Parser.parser.onToken (/usr/local/lib/balena-cli/node_modules/JSONStream/index.js:132:18)
    at Parser.proto.write (/usr/local/lib/balena-cli/node_modules/jsonparse/jsonparse.js:135:34)
    at Stream.<anonymous> (/usr/local/lib/balena-cli/node_modules/JSONStream/index.js:23:12)
    at Stream.stream.write (/usr/local/lib/balena-cli/node_modules/through/index.js:26:11)
    at IncomingMessage.ondata (_stream_readable.js:693:20)
    at IncomingMessage.emit (events.js:198:13)
    at IncomingMessage.EventEmitter.emit (domain.js:448:20)
    at addChunk (_stream_readable.js:288:12)
    at readableAddChunk (_stream_readable.js:269:11)
    at IncomingMessage.Readable.push (_stream_readable.js:224:10)
    at HTTPParser.parserOnBody (_http_common.js:122:22)
    at awaitRegistryStream (/usr/local/lib/balena-cli/node_modules/docker-progress/index.js:43:12)
    at /usr/local/lib/balena-cli/node_modules/docker-progress/index.js:416:16
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
    at process.topLevelDomainCallback (domain.js:126:23)
    at DockerProgress.exports.DockerProgress.DockerProgress.push (/usr/local/lib/balena-cli/node_modules/docker-progress/index.js:415:56)
    at Bluebird.join.retry (/usr/local/lib/balena-cli/build/utils/compose.js:499:197)
    at retry (/usr/local/lib/balena-cli/build/utils/helpers.js:111:31)
    at Bluebird.delay.then (/usr/local/lib/balena-cli/build/utils/helpers.js:116:53)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
From previous event:
    at runCommand (/usr/local/lib/balena-cli/build/app-capitano.js:57:20)
    at Object.run (/usr/local/lib/balena-cli/build/app-capitano.js:67:42)
    at routeCliFramework (/usr/local/lib/balena-cli/build/preparser.js:44:79)
    at process._tickCallback (internal/process/next_tick.js:68:7)
    at Function.Module.runMain (internal/modules/cjs/loader.js:832:11)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)

I went on the server on the registry container and I found this error:

Oct 26 15:02:51 edbb2ad0bb03 registry[1013]: 172.22.0.9 - - [26/Oct/2021:15:02:51 +0000] "POST /v2/v2/token/blobs/uploads/ HTTP/1.1" 401 272 "" "docker/20.10.8 go/go1.16.6 git-commit/75249d8 kernel/5.10.47-linux
Oct 26 15:02:51 edbb2ad0bb03 registry[1013]: time="2021-10-26T15:02:51.610354925Z" level=warning msg="error authorizing context: invalid token" go.version=go1.11.2 http.request.host=registry.mydomain.com http.request.id=2e57bd7b-5dbb-40
Oct 26 15:02:51 edbb2ad0bb03 registry[1013]: 172.22.0.9 - - [26/Oct/2021:15:02:51 +0000] "POST /v2/v2/token/blobs/uploads/ HTTP/1.1" 401 272 "" "docker/20.10.8 go/go1.16.6 git-commit/75249d8 kernel/5.10.47-linux

Edit
I am noticing the same error also on a local device:

[826]: [event]   Event: Docker image download {"image":{"name":"registry.mydomain.com/v2/token@sha256:d1ed6efeed649f2956e2c3a93108ecf2147e200988671e718dfb704ab6298808","app>
	resin-supervisor[1791]: [event]   Event: Docker image download {"image":{"name":"registry.mydomain.com/v2/token@sha256:d1ed6efeed649f2956e2c3a93108ecf2147e200988671e718dfb704ab6298808">
	balenad[826]: time="2021-10-27T07:55:55.529681875Z" level=info msg="Attempting next endpoint for pull after error: errors:\nunauthorized: authentication required\nunauthorized: authentication required\n"
	balenad[826]: time="2021-10-27T07:55:55.531437742Z" level=info msg="Ignoring extra error returned from registry: unauthorized: authentication required"
	balenad[826]: time="2021-10-27T07:55:55.532203046Z" level=error msg="Handler for POST /images/create returned error: unauthorized: authentication required"

I am both able to ping the registry and the api correctly and I am able to use the balena cli commands which interact with the devices.

Any idea?
Thanks

Matteo

21

Hey matteopeluso
I found this question that’s been unanswered for nearly two months now… It must have got stuck at the bottom :confused:
I hope you did manage to sort out the issue.
For what it’s worth, I’ve noticed two things:
1st, the log is saying that you are using the mydomain.com. You should replace in the instructions this with your own domain
2nd, since you are already using openBalena, you should also update the devices’ certificates so they can in fact be authorised