Intermittent errors on Open Balena API container, in the journalctl logs. Running v3.2.1. It results in CPU spikes which is how I came across it, but haven’t noticed any other repercussions:
Apr 12 23:56:51 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Apr 13 01:53:38 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Apr 13 06:40:35 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Apr 14 06:40:40 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Apr 15 06:40:44 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Apr 16 06:40:49 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Apr 17 06:40:53 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Apr 19 03:19:11 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Apr 20 03:19:15 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Apr 21 03:19:19 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Apr 22 03:19:24 b4a37b483439 api[982]: } GET SyntaxError: Could not match resource
Hello, looks like that request line is truncated, but the device id begins with 9. Are you able to manually run this request from the command line using curl with the correct uuid and see what the API responds with please:
This is for OpenBalena. What/where would ‘your-api-token-or-session-token-from-dashboard’ be?
I also have a device id 964f2e39faea9920d7fa3574936f3bd6, which I assume is the one it is trying. The error isn’t caused by any particular action though, so not really sure what it is trying to do when the error occurs, or what device it is doing it to.
I’m not really sure what the command that is failing does. But a few unique environmental factors sprung to mind. This device is an orange pi zero using the image from the staging page run on open balena. Not sure if the fact it is a staging image makes a difference.
Another error, I think unrelated and covered by a coming pull request, but logging just in case relevant. Even with production mode it didn’t recover:
May 08 17:59:34 b4a37b483439 systemd[1]: open-balena-api.service: Job open-balena-api.service/start failed with result 'dependency'.
May 08 17:59:34 b4a37b483439 systemd[1]: Dependency failed for open-balena-api.
May 08 17:59:34 b4a37b483439 systemd[1]: Failed to start Confd.
May 08 17:59:34 b4a37b483439 systemd[1]: confd.service: Failed with result 'exit-code'.
May 08 17:59:34 b4a37b483439 systemd[1]: confd.service: Main process exited, code=exited, status=219/CGROUP
May 08 17:59:34 b4a37b483439 systemd[1523]: confd.service: Failed at step CGROUP spawning /usr/local/bin/confd: No such file or directory
May 08 17:59:34 b4a37b483439 systemd[1523]: confd.service: Failed to attach to cgroup /system.slice/containerd.service/system.slice/confd.service: No such file or directory
May 08 17:59:34 b4a37b483439 systemd[1]: Starting Confd...
May 08 17:59:34 b4a37b483439 systemd[1]: Stopped open-balena-api.
May 08 17:59:34 b4a37b483439 systemd[1]: open-balena-api.service: Scheduled restart job, restart counter is at 1.
May 08 17:59:34 b4a37b483439 systemd[1]: open-balena-api.service: Service RestartSec=100ms expired, scheduling restart.
May 08 17:59:33 b4a37b483439 systemd[1]: open-balena-api.service: Failed with result 'exit-code'.
May 08 17:59:33 b4a37b483439 systemd[1]: open-balena-api.service: Main process exited, code=exited, status=1/FAILURE
May 08 17:59:33 b4a37b483439 api[982]: at processTicksAndRejections (internal/process/task_queues.js:80:21)
May 08 17:59:33 b4a37b483439 api[982]: at endReadableNT (internal/streams/readable.js:1327:12)
May 08 17:59:33 b4a37b483439 api[982]: at IncomingMessage.EventEmitter.emit (domain.js:467:12)
May 08 17:59:33 b4a37b483439 api[982]: at IncomingMessage.emit (events.js:327:22)
May 08 17:59:33 b4a37b483439 api[982]: at Object.onceWrapper (events.js:421:28)
May 08 17:59:33 b4a37b483439 api[982]: at IncomingMessage.<anonymous> (/usr/src/app/node_modules/request/request.js:1076:12)
May 08 17:59:33 b4a37b483439 api[982]: at Request.EventEmitter.emit (domain.js:467:12)
May 08 17:59:33 b4a37b483439 api[982]: at Request.emit (events.js:327:22)
May 08 17:59:33 b4a37b483439 api[982]: at Request.<anonymous> (/usr/src/app/node_modules/request/request.js:1154:10)
May 08 17:59:33 b4a37b483439 api[982]: at Request.EventEmitter.emit (domain.js:467:12)
May 08 17:59:33 b4a37b483439 api[982]: at Request.emit (events.js:327:22)
May 08 17:59:33 b4a37b483439 api[982]: at Request.self.callback (/usr/src/app/node_modules/request/request.js:185:22)
May 08 17:59:33 b4a37b483439 api[982]: at Request.handleResponse [as _callback] (/usr/src/app/src/features/contracts/contracts-directory.ts:123:3)
May 08 17:59:33 b4a37b483439 api[982]: Error: Invalid response while fetching contracts: Bad Gateway
May 08 17:59:33 b4a37b483439 api[982]: ^
May 08 17:59:33 b4a37b483439 api[982]: new Error(
May 08 17:59:33 b4a37b483439 api[982]: /usr/src/app/src/features/contracts/contracts-directory.ts:123
@maggie0002, this looks to me (the missing resource) like a bug in openBalena. Something is expecting that resource to be there, and it is not. I’m checking to be sure. If so, I’ll open an issue and attach it.
@maggie0002, ok I have an idea what is happening. It seems that it is a minor bug in openBalena because a resource that the device wants is not present. However, improvements in device state handling that are in process right now will soon deprecate that resource anyway (at least as far as the device is concerned). So the error should eventually be going away with the latest Supervisor releases in a few months.
I’m aware Open Balena doesn’t currently have OS updates, but will supervisor updates still happen? In other words, will this bug/its related feature become deprecated on all existing devices or only new ones?
Hi there,
unfortunately there is still no easy way to update the OS or the supervisor in Open Balena, but you should be able to do it manually. I redirect you to an answer from another post that should explain this process: Roadmap for BalenaOS Updates in OpenBalena - #10 by pdcastro