Balena Crash Loop

Product support balenaOS
1

One of our devices is in a balena engine rebooting loop. During which IO wait was through the roof (logging these errors?). Last 1000 lines of balena logs attached, not sure whats relevant - it’s very verbose…

Client:
 Version:           18.09.10-dev
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        7cb464a406748016f2df0c31a9851d20456a3d31
 Built:             Fri Nov  1 05:04:49 2019
 OS/Arch:           linux/arm
 Experimental:      false

Server:
 Engine:
  Version:          18.09.10-dev
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       7cb464a406748016f2df0c31a9851d20456a3d31
  Built:            Fri Nov  1 05:04:49 2019
  OS/Arch:          linux/arm
  Experimental:     true

Log: balena.log (43.1 KB)

Rebooting device solved issue

2

Hi @SplitIce,

Thank you for the report and the log! It looks as if balenaEngine faulted on an issue with a missing container reference. Would it be possible to answer a few questions for us?

  • balenaOS version and Supervisor version
  • Is this a balenaCloud attached device, unmanaged or using openBalena?
  • Have you seen this failure regularly?

We’d love to be able to investigate more if you see this again and the device is available.

Best regards,

Heds

4

@hedss

balenaOS 2.45.0+rev1
SUPERVISOR VERSION 10.3.7

It’s not the first time balena engine has behaved like this, although never investigated. Usually when balena engine gets faulty the device is just rebooted. This time it was on a development device - so I collected what information I could prior to reboot.

5

Hi @SplitIce
Thanks for the additional information, I’ve added this bug to our issue tracker ( https://github.com/balena-os/balena-engine/issues/191 ) and will update this thread once we have an update.