Thanks @andrewnhem it actually turns out that in my case, at least for the post on this thread, I was trying to onboard a Pi Zero W which unfortunately only offers OS 2.54.2 as the latest, which is not compatible with openbalena. I tried again with a pi4 which has a newer OS and it works fine.
Any idea when openbalena users might be able to use Pi zeros?