balena deploy myapp -> Error: unauthorized: authentication required

openBalena
#1

When trying to do the first deploy to our openbalena instance, I get the following error:

$ balena deploy myapp --registry-secrets ../registry-secrets.yml
[Info]    Everything is up to date (use --build to force a rebuild)
[Info]    Creating release...
[Info]    Pushing images to registry...
Retrying "registry.balena.${domain}/v2/a0bd40c79a2756d16cd89b7b80959903:latest" after 2.00s (1 of 3) due to: Error: unauthorized: authentication required

We require the registry secrets due to private CRs being used to pull pre-built packages.

I have an authed session:

$ balena apps
ID APP NAME     SLUG         DEVICE TYPE ONLINE DEVICES DEVICE COUNT
1  provisioning provisioning jetson-nano 1              1
2  myapp        myapp        jetson-nano 0              0

$ balena devices
ID UUID    DEVICE NAME DEVICE TYPE APPLICATION NAME STATUS IS ONLINE SUPERVISOR VERSION OS VERSION
1  2027e82 still-bird  jetson-nano provisioning     Idle   true      11.14.0            balenaOS 2.56.0+rev1

On this same topic, is it possible to not utilise the CR within balena and use external ones?

#4

As strange as this is, after seeing this thread, I ended up deleting my entire server side install and starting again.

This time, the push works without issue.

Go figure.

#6

Annoyingly, I now get the error on the remote device:

Nov 18 04:27:54 9e9e126 balenad[2986]: time="2020-11-18T04:27:54.278028119Z" level=error msg="Handler for POST /images/create returned error: Get https://registry.balena.${domain}/v2/: x509: certificate signed by unknown authority"
Nov 18 04:27:54 9e9e126 balenad[2986]: time="2020-11-18T04:27:54.280341504Z" level=warning msg="Error getting v2 registry: Get https://registry.balena.${domain}/v2/: x509: certificate signed by unknown authority"
Nov 18 04:27:54 9e9e126 balenad[2986]: time="2020-11-18T04:27:54.280435098Z" level=info msg="Attempting next endpoint for pull after error: Get https://registry.balena.${domain}/v2/: x509: certificate signed by unknown authority"
Nov 18 04:27:54 9e9e126 balenad[2986]: time="2020-11-18T04:27:54.280582025Z" level=error msg="Handler for POST /images/create returned error: Get https://registry.balena.${domain}/v2/: x509: certificate signed by unknown authority"

I rebuilt the image via balena os configure balena-cloud-jetson-nano-2.56.0+rev1-dev-v11.14.0.img

Does this not set up the required SSL certs on the device image?

#13

Hey, the balena os configure balena-cloud-jetson-nano-2.56.0+rev1-dev-v11.14.0.img should indeed set up the certs but it does need the NODE_EXTRA_CA_CERTS environment variable set at the time, either via persisting it in your environment or adding it just for the command at the time of running, eg NODE_EXTRA_CA_CERTS=... balena os configure balena-cloud-jetson-nano-2.56.0+rev1-dev-v11.14.0.img

#15

Thanks for the reply - I did ssh into the device and started poking around… I did see the ca cert listed in config.json - base64 encoded. I ran it via base64 -d - and it did indeed match the cert present on my laptop which allows me to use balena login etc - so that cert does seem to be ok…

This leaves the question as to why with the same cert present, the actual device fails to recognise this cert.

#16

Ah, sorry I didn’t spot you were using v2.56.0, could you try v2.58.0 or higher please - that version ensures the extra root CA is respected system wide and should solve your issue

#18

It looks like there is only a v2.56.0 published for the nVidia Jetson Nano SD Card here: https://www.balena.io/os/

Is there somewhere else I should be looking for these images?

Failing that, can I attempt an update via ssh as per another thread I saw here? Would give me practice doing ‘OTA’ updates as well :slight_smile:

#22

Hi, we have opened https://github.com/balena-os/balena-jetson/issues/123 to request this update, please keep an eye on that ticket. We will also update this thread once the release is available.

#24

Hi, we’ve deployed BalenaOS v2.60.1 for the Jetson Nano SD-CARD. Let us know if you still encounter this issue.

#27

That looks like its doing more than it was before:

== STILL GLITTER
ID:                    35
DEVICE TYPE:           jetson-nano
STATUS:                updating
IS ONLINE:             true
IP ADDRESS:            10.1.1.194
MAC ADDRESS:           00:04:4B:EC:5A:6A
APPLICATION NAME:      triage
LAST SEEN:             2020-11-23T07:12:18.282Z
UUID:                  73414124ca8ab35699814c985d555a02
COMMIT:                N/a
SUPERVISOR VERSION:    11.14.0
IS WEB ACCESSIBLE:     false
OS VERSION:            balenaOS 2.60.1+rev1
DASHBOARD URL:         https://dashboard.balena.crc.id.au/devices/73414124ca8ab35699814c985d555a02/summary
CPU USAGE PERCENT:     30
CPU TEMP C:            35
MEMORY USAGE MB:       3455
MEMORY TOTAL MB:       3953
MEMORY USAGE PERCENT:  87
STORAGE BLOCK DEVICE:  /dev/mmcblk0
STORAGE USAGE MB:      3646
STORAGE TOTAL MB:      58381
STORAGE USAGE PERCENT: 6

I guess I can keep experimenting now :slight_smile:

Thanks for the help - I’ll let you know what other issues I find :slight_smile:

#29

Hi all,

I’m just trying to find version 2.60.1-rev1 or higher for this unit - I originally downloaded it, but now it seems that only v2.56.0-rev1 is available via the download links.

Have I just forgotten how I downloaded this 2.60.1 image? or has something been rolled back?

#30

Hi @Steve, indeed we rolled back 2.60.1 as we found an issue with devices updating to this version. Here’s the discussion about this issue: https://github.com/balena-io/balena-supervisor/issues/1514

So if I understand this right, it is fine to flash a new device with OS version 2.60.1. Feel free to use it. We saw the problem happening only on host OS updates.

Cheers…