balena engine failing after update with balena.service: Main process exited, code=exited, status=2/INVALIDARGUMENT

Product support balenaOS
1

Good day, I have several devices that is failing like below. Seems to be a BalenaOS issue.
balenaOS boots but non of the services does.

Sep 22 05:43:17 0c96958 balenad[25764]:         /yocto/resin-board/build/tmp/work/core2-64-poky-linux/balena/19.03.13-dev+git074a481789174b4b6fd2d706086e8ffceb72e924-r0/git/src/import/.gopath/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:852 +0x2ec
Sep 22 05:43:17 0c96958 balenad[25764]: github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).Execute(...)
Sep 22 05:43:17 0c96958 balenad[25764]:         /yocto/resin-board/build/tmp/work/core2-64-poky-linux/balena/19.03.13-dev+git074a481789174b4b6fd2d706086e8ffceb72e924-r0/git/src/import/.gopath/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:800
Sep 22 05:43:17 0c96958 balenad[25764]: github.com/docker/docker/cmd/dockerd.Main()
Sep 22 05:43:17 0c96958 balenad[25764]:         /yocto/resin-board/build/tmp/work/core2-64-poky-linux/balena/19.03.13-dev+git074a481789174b4b6fd2d706086e8ffceb72e924-r0/git/src/import/.gopath/src/github.com/docker/docker/cmd/dockerd/docker.go:98 +0x18f
Sep 22 05:43:17 0c96958 balenad[25764]: main.main()
Sep 22 05:43:17 0c96958 balenad[25764]:         /yocto/resin-board/build/tmp/work/core2-64-poky-linux/balena/19.03.13-dev+git074a481789174b4b6fd2d706086e8ffceb72e924-r0/git/src/import/.gopath/src/github.com/docker/docker/cmd/balena-engine/main.go:29 +0x129
Sep 22 05:43:17 0c96958 systemd[1]: balena.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Sep 22 05:43:17 0c96958 systemd[1]: Failed to start Balena Application Container Engine.
Sep 22 05:43:17 0c96958 systemd[1]: Failed to listen on Docker Socket for the API.
Sep 22 05:43:17 0c96958 systemd[1]: balena.service: Job balena.service/start failed with result 'dependency'.
Sep 22 05:43:18 0c96958 wpa_supplicant[1306]: wlp3s0: CTRL-EVENT-SCAN-FAILED ret=-95 retry=1
Sep 22 05:43:19 0c96958 wpa_supplicant[1306]: wlp3s0: CTRL-EVENT-SCAN-FAILED ret=-95 retry=1
Sep 22 05:43:20 0c96958 wpa_supplicant[1306]: wlp3s0: CTRL-EVENT-SCAN-FAILED ret=-95 retry=1
Sep 22 05:43:21 0c96958 balenad[25916]: time="2020-09-22T05:43:21.606280303Z" level=info msg="Starting up"
Sep 22 05:43:21 0c96958 balenad[25916]: time="2020-09-22T05:43:21.606317040Z" level=warning msg="Running experimental build"
Sep 22 05:43:21 0c96958 balenad[25916]: time="2020-09-22T05:43:21.607502146Z" level=info msg="libcontainerd: started new balena-engine-containerd process" pid=25934
Sep 22 05:43:21 0c96958 balenad[25916]: time="2020-09-22T05:43:21.647845839Z" level=info msg="starting containerd" revision= version=1.2.0+unknown
Sep 22 05:43:21 0c96958 balenad[25916]: time="2020-09-22T05:43:21.648419222Z" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
Sep 22 05:43:21 0c96958 balenad[25916]: time="2020-09-22T05:43:21.648465640Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
Sep 22 05:43:21 0c96958 balenad[25916]: time="2020-09-22T05:43:21.650207170Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="modprobe aufs failed: \"modprobe: FATAL: Module aufs not found in directory /lib/modules/5.2.10-yocto-standard\\n\": exit status 1: skip plugin" type=io.containerd.snapshotter.v1
5

Hi, thanks for reporting the issue. Which version of balenaOS are you using? Which device are you running it on? Are you using balenaOS image from balenaCloud or did you download a standalone image?

1 Like
6

This is balenaOS 2.54.2+rev1 Generic x86_64 on a dell 7212 from our cloud application.

1 Like
10

Hello,
Just curious if you recently performed an OS update on this device? Either way, could you please provide the device diagnostics snapshot file from the “device diagnostics” tab on the “diagnostics” page of the dashboard (after running the diagnostics)?

12

I ‘resolved’ the issue by removing the content of ‘/var/lib/balena/’