zram: Decompression failed! err=-22

I’ve had some issues recently with my devices failing with a “zram: Decompression failed!” error in the dmesg logs.

Sometimes the unit runs for a while before this issue occurs, sometimes it can happen quite quickly. I’m running 4 containers on a BalenaFin v1.1.1, including InfluxDB and Telegraf to collect system metrics to check on the health of my devices (ironically to monitor the memory usage of my device to catch/prevent OOM errors with InfluxDB). The other 2 containers are a GUI and a python based data uploader (small packets of data in the 10’s of kB), which don’t consume much memory (10-20MB each reported by Balena)

As far as I can tell, I’m not maxing out the memory, unless something big is happening between logs (every 15seconds).

Setup:

  • BalenaFin v1.1.1
  • 4 Containers + Supervisor (And occasional health-check container the supervisor runs)
  • OS version: balenaOS 2.80.3+rev1
  • Supervisor version: 12.7.0
  • Typical memory used around 60-65% when issue occurs
  • Peak memory used has been as high as 75%

Below is a screen grab of the system metrics with a data gap in the middle (where the white markers are) where the unit stopped working, and then restarted a little later (although in this instance it didn’t seem to start 2 of the containers properly for some reason & I had to restart them manually)

Anyone run into this before?

I can provide some more dmesg output if needed.

Hi, there is a reported kernel problem that feels similar fixed in v5.10 (see kernel/git/torvalds/linux.git - Linux kernel source tree).
We are working on supporting the 5.10 kernel for our RaspberryPi family of devices including the balenaFin. I have linked the github issue tracking that work (Community user is requesting kernel 5.10 for RPI · Issue #601 · balena-os/balena-raspberrypi · GitHub) here and we will update here once it’s ready.

Hey @ST-Mono ,

Just to let you know that we’ve just merged support for kernel 5.10.

Cheers!

Great! I look forward to testing it once its available on the BalenaCloud dashboard to upgrade to.