Oh, let me try that again, I’m actually running a production image, let me change that and run the commands.
Here’s the log
Aug 13 13:15:20 4c4d4ef resin-supervisor[19611]: Supervisor API: GET /v1/healthy 200 - 0.935 ms
Aug 13 13:15:20 4c4d4ef resin-supervisor[19611]: Attempting container log timestamp flush…
Aug 13 13:15:20 4c4d4ef resin-supervisor[19611]: Container log timestamp flush complete
Aug 13 13:15:20 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap {}
Aug 13 13:15:20 4c4d4ef resin-supervisor[19611]: New device detected. Provisioning…
Aug 13 13:15:25 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Aug 13 13:15:55 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap {}
Aug 13 13:15:55 4c4d4ef resin-supervisor[19611]: New device detected. Provisioning…
Aug 13 13:16:00 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Aug 13 13:16:30 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap {}
Aug 13 13:16:30 4c4d4ef resin-supervisor[19611]: New device detected. Provisioning…
Aug 13 13:16:35 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Aug 13 13:17:05 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap {}
Aug 13 13:17:05 4c4d4ef resin-supervisor[19611]: New device detected. Provisioning…
Aug 13 13:17:10 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Aug 13 13:17:40 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap {}
Aug 13 13:17:40 4c4d4ef resin-supervisor[19611]: New device detected. Provisioning…
Aug 13 13:17:45 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Aug 13 13:18:15 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap {}
Aug 13 13:18:15 4c4d4ef resin-supervisor[19611]: New device detected. Provisioning…
Aug 13 13:18:20 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Aug 13 13:18:50 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap {}
Aug 13 13:18:50 4c4d4ef resin-supervisor[19611]: New device detected. Provisioning…
Aug 13 13:18:55 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Aug 13 13:19:25 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap {}
Aug 13 13:19:25 4c4d4ef resin-supervisor[19611]: New device detected. Provisioning…
Aug 13 13:19:30 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Aug 13 13:20:00 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap {}
Aug 13 13:20:00 4c4d4ef resin-supervisor[19611]: New device detected. Provisioning…
Aug 13 13:20:05 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Aug 13 13:20:20 4c4d4ef resin-supervisor[19611]: Supervisor API: GET /v1/healthy 200 - 1.008 ms
Aug 13 13:20:35 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap {}
Aug 13 13:20:35 4c4d4ef resin-supervisor[19611]: New device detected. Provisioning…
Aug 13 13:20:40 4c4d4ef resin-supervisor[19611]: Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Aha, this makes more sense now. So it looks like the device is having some issues registering with the openBalena API. When a device boots up for the first time, the supervisor will contact the API to finish the provisioning step, this is where it seems to be getting stuck. At this point we could try having a look at the openBalena API logs. In there you should be able to find the requests that the supervisor is sending the API, hopefully there should be a bit more information on what is going wrong.
I tried looking at the container logs, but there’s nothing there. Is there a way to view these logs?
From you open balena server you can run docker ps to get a list of the running containers. One of those should be the openBalena API. From there you can inspect its logs via docker logs <container-id>
I did that and there is just this
(base) $ docker logs openbalena_api_1
Systemd init system enabled.
Hi Anil, It seems like perhaps something is not working on your API side, can you run docker logs openbalena_api_1 -f and then from another terminal hit the /config and /ping endpoint of your openBalena instance. You should be able to do this with something like curl https://api.balena-cloud.com/config replacing balena-cloud.com with your domain name. You should then also see logs generated in your openbalena_api instance, but if not that means there is some problem with the instance.
There are no logs, how do I debug what’s wrong?
It sounds like connections/traffic are not getting to your openbalena instance. This could be for a number of reasons but might be that your domain is not properly setup and you might need to run through that part of the documentation again. It could be that it is still listening on openbalena.local so perhaps try the /config and /ping end point on the openbalena.local domain
Actually, when I run /ping, I get an OK response, even on the browser. I’ll revisit the documentation and check if I’ve done something differently
Can you also clarify whether visiting the /config API endpoint with a browser also returns back results?
It should also include a list of the device types available to your instance, so please also check whether your device’s type is also in there.
Kind regards,
Thodoris
Visiting the /config API however returns an error message Cannot GET /config
My bad on that since that’s not an open balena endpoint. Can you check /device-types/v1 instead?
There’s a huge json dump of what seems like devices along with some information, I’m assuming that is working fine
Should I add a hostname resolution inside the devices as well? In my case, should I have to add a hostname resolution inside the virtual machine? If yes, how should I add, I’ve tried editing /etc/hosts and /etc/resolv.conf.
Hi, can you do a quick test and use an older OpenBalena/cli combination? Can you use an OpenBalena 1.2.0 / CLI 11.2.1 and let us know if that works for you?
Even with these versions, I’m unable to see the device. I tried looking at the supervisor logs.
Supervisor API listening on allowed interfaces only
LogBackend: unexpected error: { Error: getaddrinfo ENOTFOUND api.openbalena.local api.openbalena.local:443
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:56:26)
errno: ‘ENOTFOUND’,
code: ‘ENOTFOUND’,
syscall: ‘getaddrinfo’,
hostname: ‘api.openbalena.local’,
host: ‘api.openbalena.local’,
port: 443 }
Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Looks like I have to add hostname resolution in the qemu virtual machine too, but I’m not sure how to do it as I’m unable to access /etc/hosts
Hi Anil, let’s rewind a bit. Can you please follow https://www.balena.io/open/docs/getting-started/ step by step (latest openBalena), and recreate your openBalena instance?
- Make sure that you can create an application, ping the api (
curl https://api.yourdomain/ping), and curlhttps://api.yourdomain/device-types/v1/qemux86-64and make sure that returns the expected result. - Then, make sure to download a development image, and provision it based on the instructions listed in the
balenaClouddashboard when you create a qemu application. - After the device is provisioned, confirm that
balena deviceslists the device. You can also SSH into the device and see the logs usingjournalctland try to see if there is anything unusual if the device still doesn’t show up.
If after doing all of these doesn’t work, we can try to narrow down the issue. Cheers!
Hello @sradevski, I’ve followed the steps in the getting started guide
Response for https://api.openbalena.local/ping
OK
Response for https://api.openbalena.local/device-types/v1/qemux86-64
{“slug”:“qemux86-64”,“version”:1,“name”:“QEMU X86 64bit (EXPERIMENTAL)”,“arch”:“amd64”,“state”:“EXPERIMENTAL”,“instructions”:[“Run the following to start the image:\n
\nOn systems with KVM support:\n
\nqemu-system-x86_64 -drive file=resin-image-qemux86-64.img,media=disk,cache=none,format=raw -net nic,model=virtio -net user -m 512 -nographic -machine type=pc,accel=kvm -smp 4 -cpu host\n
\nOn systems without KVM support:\n
\nqemu-system-x86_64 -drive file=resin-image-qemux86-64.img,media=disk,cache=none,format=raw -net nic,model=virtio -net user -m 512 -nographic -machine type=pc -smp 4\n
\nTweak-smpand-cpuparameters based on the CPU of the machine qemu is running on.-cpuparameter needs to be dropped on OSX and Windows.\n
\nTweak-nographicand-m 512to set the display of qemu and memory respectively.\n”],“gettingStartedLink”:{“windows”:“http://docs.resin.io/#/pages/installing/gettingStarted.md#windows",“osx”:“http://docs.resin.io/#/pages/installing/gettingStarted.md#on-mac-and-linux”,“linux”:“http://docs.resin.io/#/pages/installing/gettingStarted.md#on-mac-and-linux”},“supportsBlink”:true,“yocto”:{“machine”:“qemux86-64”,“image”:“resin-image”,“fstype”:“resinos-img”,“version”:“yocto-sumo”,“deployArtifact”:“resin-image-qemux86-64.resinos-img”,“compressed”:true},“configuration”:{“config”:{“partition”:{“primary”:1},“path”:“/config.json”}},“initialization”:{“options”:[{“message”:"Select a drive”,“type”:“drive”,“name”:“drive”}],“operations”:[{“command”:“burn”}]},“options”:[{“isGroup”:true,“isCollapsible”:true,“collapsed”:true,“name”:“advanced”,“message”:“Advanced”,“options”:[{“message”:“Check for updates every X minutes”,“name”:“appUpdatePollInterval”,“type”:“number”,“min”:10,“default”:10}]}],“buildId”:“2.38.0+rev1.prod”}
I’ve run the following command (provided in the balenaCloud)
sudo qemu-system-x86_64 -drive file=qemu.img,media=disk,cache=none,format=raw -net nic,model=virtio -net user -m 512 -nographic -machine type=pc,accel=kvm -smp 4 -cpu host
I’m still not seeing any devices.
I’ve tried running the VM with an image that was not configured and that was configured using the command balena os configure qemu.img --app myApp
Resin Supervisor logs without configuring:
root@balena:~# balena logs -f resin_supervisor
Starting system message bus: dbus.
* Starting Avahi mDNS/DNS-SD Daemon: avahi-daemon
…done.
(node:1) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
Starting event tracker
Starting up api binder
Unmanaged mode is set, skipping API client initialization
Starting logging infrastructure
Event: Supervisor start {}
Performing database cleanup for container log timestamps
Starting up in unmanaged mode, activating local mode
Switching logging backend to LocalLogBackend
Reporting initial state, supervisor version and API info
Attempting to load preloaded apps…
Starting API server
Applying target state
Unmanaged mode is set, skipping API binder initialization
Creating supervisor0 network
Supervisor API listening on all interfaces
Finished applying target state
Apply success!
Supervisor API: GET /v1/healthy 200 - 12.326 ms
Supervisor logs after configuring the image:
root@ee9f647:~# balena logs -f resin_supervisor
Starting system message bus: dbus.
* Starting Avahi mDNS/DNS-SD Daemon: avahi-daemon
…done.
Updating certificates in /etc/ssl/certs…
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d…
done.
(node:1) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
Starting event tracker
Starting up api binder
Starting logging infrastructure
Event: Supervisor start {}
Performing database cleanup for container log timestamps
Connectivity check enabled: true
Starting periodic check for IP addresses
Reporting initial state, supervisor version and API info
Waiting for connectivity…
Attempting to load preloaded apps…
Starting API server
Applying target state
Ensuring device is provisioned
Event: Device bootstrap {}
New device detected. Provisioning…
Event: Apply config change in progress {}
Event: Apply config change success {}
Supervisor API listening on all interfaces
Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Event: Device bootstrap {}
New device detected. Provisioning…
Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Event: Device bootstrap {}
New device detected. Provisioning…
Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Event: Device bootstrap {}
New device detected. Provisioning…
Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Event: Device bootstrap {}
New device detected. Provisioning…
Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Event: Device bootstrap {}
New device detected. Provisioning…
Event: Device bootstrap failed, retrying {“delay”:30000,“error”:{“message”:“”}}
Also, I tried to download the image using balena cli
Command: balena os download qemux86-64 -o qemu.img --version menu
After selecting version, I got:
No such version for the device type
Error: No such version for the device type
at /snapshot/versioned-source/node_modules/balena-image-manager/node_modules/balena-sdk/build/models/os.js:322:19
at runCallback (timers.js:705:18)
at tryOnImmediate (timers.js:676:5)
at processImmediate (timers.js:658:5)
at process.topLevelDomainCallback (domain.js:126:23)
at Object.getLastModified (/snapshot/versioned-source/node_modules/balena-image-manager/node_modules/balena-sdk/build/models/os.js:321:19)
at /snapshot/versioned-source/node_modules/balena-image-manager/build/cache.js:92:29
at Object.exports.isImageFresh (/snapshot/versioned-source/node_modules/balena-image-manager/build/cache.js:88:6)
at /snapshot/versioned-source/node_modules/balena-image-manager/build/manager.js:94:18
at Object.exports.get (/snapshot/versioned-source/node_modules/balena-image-manager/build/manager.js:93:59)
at /snapshot/versioned-source/build/actions/os.js:116:28
at runCallback (timers.js:705:18)
at tryOnImmediate (timers.js:676:5)
at processImmediate (timers.js:658:5)
at process.topLevelDomainCallback (domain.js:126:23)
at Command.action (/snapshot/versioned-source/build/actions/os.js:112:14)
at /snapshot/versioned-source/node_modules/capitano/build/command.js:98:37
From previous event:
at runCommand (/snapshot/versioned-source/build/app-capitano.js:57:20)
at Object.run (/snapshot/versioned-source/build/app-capitano.js:67:42)
at routeCliFramework (/snapshot/versioned-source/build/preparser.js:44:79)
at process._tickCallback (internal/process/next_tick.js:68:7)
at Function.Module.runMain (pkg/prelude/bootstrap.js:1381:13)
at startup (internal/bootstrap/node.js:320:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:660:3)For help, visit our support forums: https://forums.balena.io
For bug reports or feature requests, see: Issues · balena-io/balena-cli · GitHub
I’m using balena-cli v12.2.0 and openBalena v2.0.3
Hi Anil,
thanks for the effort. The unmanaged image looks fine, so let’s focus on the image configured for your app. Since this looks like a networking problem, could you please share some details about your setup? Are you running the VMs and the OpenBalena instance on the same host? Are they on the same local network? Are you able to ping the OpenBalena server from within the VM (directly to the IP address to rule out DNS)? Have you configured OpenBalena for a custom domain name or are you using the default openbalena.local? Did you configure any DNS records for your domain?
Thanks