Supervisor crashing when starting up

Product support
1

Hi guys,

A supervisor container is crashing out of the blue when starting up and we have no clue why.
During development, we created a temporary container on device using balena commit <container-id> and this container has been name focused_davinci.
Which check does the supervisor do?

[info]    Supervisor v13.1.3 starting up...
[info]    Setting host to discoverable
[warn]    Invalid firewall mode: . Reverting to state: off
[info]    Applying firewall mode: off
[debug]   Starting systemd unit: avahi-daemon.service
[debug]   Starting systemd unit: avahi-daemon.socket
[debug]   Starting logging infrastructure
[debug]   Performing database cleanup for container log timestamps
[info]    Starting firewall
[warn]    Ignoring unsupported or unknown compose fields: containerName
[info]    Previous engine snapshot was not stored. Skipping cleanup.
[debug]   Handling of local mode switch is completed
(node:1) UnhandledPromiseRejectionWarning: InternalInconsistencyError: Expected supervised container to have name '<serviceName>_<imageId>_<releaseId>_<commit>', got: /focused_davinci
    at Function.fromDockerContainer (/usr/src/app/dist/app.js:10:832269)
    at /usr/src/app/dist/app.js:6:292713
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 3)
(node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
[success] Firewall mode applied
[debug]   Starting api binder
(node:1) PromiseRejectionHandledWarning: Promise rejection was handled asynchronously (rejection id: 3)
(node:1) UnhandledPromiseRejectionWarning: InternalInconsistencyError: Expected supervised container to have name '<serviceName>_<imageId>_<releaseId>_<commit>', got: /focused_davinci
    at Function.fromDockerContainer (/usr/src/app/dist/app.js:10:832269)
    at /usr/src/app/dist/app.js:6:292713
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 4)
[debug]   Attempting container log timestamp flush...
[debug]   Container log timestamp flush complete

....
[info]    Supervisor v13.1.3 starting up...
[info]    Setting host to discoverable
[debug]   Starting systemd unit: avahi-daemon.service
[debug]   Starting systemd unit: avahi-daemon.socket
[debug]   Starting logging infrastructure
[warn]    Invalid firewall mode: . Reverting to state: off

Details on the environment:

balena --version
balenaEngine version 20.10.12, build 73c78258302d94f9652da995af6f65a621fac918
4

Hello,
Thanks for reaching out. This is a known issue being tracked here: `Error on docker event: InternalInconsistencyError: Expected supervised container to have name '<serviceName>_<imageId>_<releaseId>_<commit>', got: /$TEMPORARY_CONTAINER_NAME` · Issue #1930 · balena-os/balena-supervisor · GitHub
As a workaround for this issue, please delete the container called focused_davinci and restart your supervisor systemctl restart balena-supervisor. I think after this the device should function properly.

1 Like
7

Encountered same error on 1 device

2023-09-17T15:33:14.409392000Z [info]    Supervisor v14.11.14 starting up...
2023-09-17T15:33:14.567012000Z [info]    Setting host to discoverable
2023-09-17T15:33:14.590467000Z [debug]   Starting systemd unit: avahi-daemon.service
2023-09-17T15:33:14.593360000Z [debug]   Starting systemd unit: avahi-daemon.socket
2023-09-17T15:33:14.599474000Z [debug]   Starting logging infrastructure
2023-09-17T15:33:14.691202000Z [info]    Starting firewall
2023-09-17T15:33:14.700062000Z [warn]    Invalid firewall mode: . Reverting to state: off
2023-09-17T15:33:14.700484000Z [info]    Applying firewall mode: off
2023-09-17T15:33:15.127037000Z [success] Firewall mode applied
2023-09-17T15:33:15.130844000Z [debug]   Starting api binder
2023-09-17T15:33:15.302277000Z [debug]   Performing database cleanup for container log timestamps
2023-09-17T15:33:15.340773000Z [info]    Previous engine snapshot was not stored. Skipping cleanup.
2023-09-17T15:33:15.340964000Z [debug]   Handling of local mode switch is completed
2023-09-17T15:33:15.437695000Z [error]   Uncaught exception: InternalInconsistencyError: Expected supervised container to have name '<serviceName>_<imageId>_<releaseId>_<commit>', got: /priceless_elion
2023-09-17T15:33:15.438656000Z [error]         at Function.fromDockerContainer (/usr/src/app/dist/app.js:6:93419)
2023-09-17T15:33:15.438813000Z [error]       at /usr/src/app/dist/app.js:6:76004
2023-09-17T15:33:15.441269000Z [error]       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2023-09-17T15:33:16.821992000Z INFO: Found device /dev/sdb1 on current boot device sdb, using as mount for '(resin|balena)-boot'.
2023-09-17T15:33:16.889268000Z INFO: Found device /dev/sdb4 on current boot device sdb, using as mount for '(resin|balena)-state'.
2023-09-17T15:33:16.980804000Z INFO: Found device /dev/sdb5 on current boot device sdb, using as mount for '(resin|balena)-data'.
2023-09-17T15:33:19.557130000Z [info]    Supervisor v14.11.14 starting up...

My logs were not getting updated (streamed from supervisor API endpoint v2/journal-logs) and that’s how I found this error.

8

Actually, the priceless_elion container was created by me during testing and I totally forgot about it. Apologies.