Writing to stdout/stderror sometimes blocks the device

Product support
1

Hi, we started to see some issues logging to the console(stderr) in multiple devices, some containers started to lock, attaching strace to the process we saw that it was blocking when it was writing to stderr, doing some investigation we discover that the buffer in /dev/console(a pts device that I believe balena mounted) was full(seems that no other process is reading it), doing a flush manually(we used https://docs.python.org/3/library/termios.html#termios.tcflush with TCIOFLUSH to /dev/console) made the process that was blocked work again.

Any idea of what can be causing this problem?

4

Found out that it’s fixed in https://github.com/resin-os/balena/pull/101 so we need to wait for a new release in resinos

5

Thanks for following up, roger

6

@chrisys do you know if we have any ETA for the release to land in resin.io?

8

Hi @roger,
We don’t have an ETA to share at the moment.
What device type are you using and need this OS release for?

11

We have a fleet of 42 raspberrypi 3 that is having this issue from time to time. It keeps our devops team a little busy having to fix this manually either by flushing or restarting services.

So, it will be really awesome to have at least an ETA of the release :).

25

Hi, sorry to bother again, but we keep having this issue from day to day… any update for the ETA?

26

Hi @roger –

Sorry for the delay! The requisite balena-engine changes made it into balenaOS v2.18.0, so this issue should be fixed in the latest balenaOS release (2.26.0+rev1).

If you get a chance to test that OS version, please let us know the results!