Reboot requests to supervisor API fail with 401

I’m working on a BLE project running on Raspberry Pi 4s. Every once in a while, the RPi4 will lose the ability to connect to peripherals (perhaps the Bluetooth chip gets in a bad state), and all subsequent connection attempts fail. Rebooting the Pi usually solves this.

To fix this in an automated way, I am counting the number of consecutive failed connections, and after a certain number I am requesting a reboot via the device supervisor. Unfortunately, I often see that the reboot request fails with a 401 error:

Dec 15 16:25:04 75f3131  Error: syncqueue has failed 66 connections in a row. Performing reboot. 
Dec 15 16:25:04 75f3131  Requesting reboot at Wed Dec 15 2021 16:25:04 GMT+0000 (Coordinated Universal Time). 
Dec 15 16:25:04 75f3131  [api]     POST /v1/reboot 401 - 2.680 ms
Dec 15 16:25:04 75f3131  Error when requesting a reboot: HTTPError: Response code 401 (Unauthorized) 
Dec 15 16:25:04 75f3131      at Request.<anonymous> (/app/dist/index.js:1:598855) 
Dec 15 16:25:04 75f3131      at runMicrotasks (<anonymous>) 
Dec 15 16:25:04 75f3131      at processTicksAndRejections (internal/process/task_queues.js:95:5)

I’m using the credentials straight from the BALENA_SUPERVISOR_ADDRESS and BALENA_SUPERVISOR_API_KEY environment variables.

This behavior has been happening for a while, but what usually happened was that one reboot request would fail, but a subsequent one would work and the device would reboot and get itself back on track. More recently (possibly after upgrading to balenaOS 2.88.4 from a months-old version), reboot requests seem to fail every time, and I now have to manually log into the Balena Cloud Dashboard to reboot my devices. It’s worth noting that rebooting via the Balena Cloud Dashboard seems to work reliably.

Small update: I have added a failover to the Balena Cloud API (which in turn calls the Supervisor), which seems to be able to successfully trigger a reboot even when the Supervisor returns a 401. This is certainly not ideal, but I wanted to share it here in case anyone else is struggling with this (or if it helps with debugging).

Here’s some details for the working request:

  • URL:
  • Headers: { Authorization: 'Bearer ' + BALENA_API_KEY }
  • Body: { uuid: BALENA_DEVICE_UUID }
1 Like

Hello @Verigo_Tommy thank you for searching the work-around to restart your supervisor.

I think it’s interesting first of all to understand why you loose connectivity over BLE with your peripherals. Would you like to research more in there and share more logs?

Hi @mpous . Thanks for your reply.

First, let me give an update to the original problem: I contacted Balena Support directly (I’m on a paid plan), and quickly received help from someone who verified that reboots were working in the Supervisor on my devices. Their testing and logs led me to realize I was using apiKey as the auth query parameter, instead of apikey (note the lowercase k) like I should have been. I’m not sure why it used to work on older Supervisor versions, but I’ve got it fixed now and things seem to be working better.

Regarding losing connectivity via BLE - this is definitely something I want to dig into more. I believe there may have been a change for the worse in the newer (circa 2.88.4) Balena OS images for RPi 4s. I will start a separate topic or help chat for that.

1 Like

Hello @verigo-tommy,
Since we discussed/resolved this issue on paid-support I will go ahead and close this forum ticket.
Sharing the diagnosis for everyone’s benefit - in local mode the supervisor does not need/care about the API key param in the URL, hence both ‘apikey’ and ‘apiKey’ work. Once out of local mode however, the supervisor needs/verifies the API key param and we need to supply a param named apikey (i.e all lowercase letters). Using any other casing will not work.

Thanks and regards,

1 Like

@pranavpeshwe thanks again for your help!