Containers in Inconsistent State After Local Push

Product support balenaEngine
1

While working on this issue React Front End with NGINX and an Express Backend. Can't Find Backend Server for some hours today we had the containers get in an inconsistent state quite a few times through out the day.

Unfortunately I don’t have the screen caps but when doing a balena images on the HostOS there were several images with tags of None. During the build process this message should in the console

Retrying "Supervisor API (GET http://192.168.1.68:48484/v2/state/status)" after 4.0s (2 of 5) due to: Error: connect ECONNREFUSED 192.168.1.68:48484
Retrying "Supervisor API (GET http://192.168.1.68:48484/v2/state/status)" after 8.0s (3 of 5) due to: Error: connect ECONNREFUSED 192.168.1.68:48484

It then keep saying Starting service 'server' over and over again.

Running balena ps -a showed several contains that were tied to the un-taged images that had exited .

Check journalctrl -f --all --no-pager didn’t show anything.

It seemed to magically fix it’s self when I updated the GPU memory size.

Thoughts on this or how to best troubleshoot this in the future?

Thanks,

3

Hi @westcoastdaz, to clarify a few points:

  • When you mean ‘an inconsistent state quite a few times through out the day’, does it occur randomly? Have you noticed any patterns for when they appear? Maybe if you push a release or something else?
  • When you experience it, do you update the GPU memory size to solve it everytime? or did it solve it for once?

Thanks!

5

@jaomaloy

  • Yes it seems to occur randomly. It seems to always happen when devices are in local mode.
  • No, this didn’t solve it all the time. We ended up just re-flashing the SD card most of the time.