Balena Memory Use

Hello,
We are having some confusion over memory use in our application running on a Balena Fin. Recently, the Balena dashboard added a really nice feature that lets us see memory use in the fins (“Memory usage”), which is great! We’ve noticed a couple devices get to a point where their memory use (as shown on the portal) is quite high - say 700MB used of 745MB, or 94% full, or more.

We did a deep dive on one of these units with 700MB used and found that the check_memory diagnostic test reported 26% memory available, which is quite different from the 700MB (94%) reported by the portal. We ran the “free” command in the application on the device, and this also reported about 45MB free memory (94% full). But based on my understanding of memory, a lot of the “used” memory from the “free” command might be reclaimable and could be used if more memory is needed. So we may not have a problem even when the “free” command shows little “free” remaining memory. Is that correct?

It appears the memory usage in the portal must come from the “free” or “top” command data. But there are times when the “free” command indicates there is very little free memory left, but the check_memory diagnostic seems to indicate the unit still has plenty of memory available (26% in this case).

Should we be concerned when the Balena portal memory usage gauge shows very little remaining memory on some devices? Do we need to somehow increase the memory allocation for our application, or will the OS allocate more from the 26% available memory if needed?

How does the check_memory command get the memory available number? Is there a command I could run from our application to get this number (26%)?

Ultimately I’d like to know if any of our units reach a point where we should be very concerned about low remaining memory, and I’m not sure how to figure that out.

Thanks for your help!

1 Like

Hi @dstewart,

You raise some important questions, and I’ve pinged our engineering teams to see if I can give you a deeper read on actual memory usage.

John

Hi @dstewart,

The memory display feature was recently updated. The information reported by the supervisor originally estimated the value of used memory as MemTotal - MemFree. However, as you point out, Linux systems will try to cache and buffer as much memory as possible, which will affect the output of MemFree (from /proc/meminfo) and in consequence the memory usage seen by the user on the dashboard, which will appear much greater than it is.

The correct calculation was changed to MemTotal - MemFree - Buffers - Cached, which is the calculation performed by the htop and free commands. This update was added in the v12.05 version of the supervisor.

Let us know if this answers your questions.

John

2 Likes

Thank you for the speedy reply John! Is it possible to get that version now, or will it be available in the next balena OS update (post 2.58.3+rev1)? If the latter, when will the next OS update be available for the Balena Fin?

1 Like

Hi

Our device support team is working on the next release, you can track the progress in this issue that I just created - https://github.com/balena-os/balena-raspberrypi/issues/578
I think it might be a couple weeks before we have a new release. Is this something that you require urgently?

We can wait.

Hi @dstewart, thank you for the wait :grinning_face_with_smiling_eyes:

Can you please try again with the latest version of balenaOS and report back your experience?

Cheers…