I’ve been having trouble livepushing a 6 container app to my local mode fin.
It appears that the balena-engine / docker daemon may be crashing but it’s hard to tell because sometimes the device completely locks up.
Here’s some output that will hopefully help pin-point the issue.
balena push 192.168.0.11 --debug sometimes just locks up and stops streaming logs. Any other active ssh sessions also lock up.
The last thing I’ll see is something like this:
[Build] [map-service] Step 5/9 : RUN pip install -r requirements.txt
[Build] [hal] Step 5/9 : RUN pip install -r requirements.txt
[Build] [supervisor] —> Running in 2f88a7d31c5d
[Build] [scheduler] —> Running in 367ced99e0e8
[Build] [map-service] —> Running in 9272ba052ebb
[Build] [hal] —> Running in a8a578ed9e21
The device-api also goes down, which I verify by hitting http://192.168.0.11:48484/ping (no response)
Other times I see an actual error during
balena push 192.168.0.11 --debug
connect ECONNREFUSED 192.168.0.11:2375
[Error] Connection to device lost
ECONNRESET: socket hang up
And then I see errors in the systems logs like this:
Jul 01 20:28:43 07a2ee6 systemd: resin-supervisor.service: Main process exited, code=killed, status=15/TERM
Jul 01 20:28:54 07a2ee6 resin-supervisor: [info] Supervisor v11.4.10 starting up…
Jul 01 20:28:43 07a2ee6 balenad: time=“2020-07-01T20:28:43.778617677Z” level=error msg=“Handler for GET /images/json returned error: write unix /var/run/balena-engine.sock->@: write: broken pipe”
It appears to me that either the resin_supervisor container or balenad crashes (or gets terminated?) or both?
Any thoughts on what’s going on?
EDIT: I’ve uploaded another log over here with more details: https://gist.github.com/ebradbury/80537374db3471442033cc5b26c04abe
Thanks for the help,