Jenkins Out of Memory on Balena Push

I’ve setup a Jenkins Instance to CI/CD my project. Previously on Jenkins I ran a script sh ‘build_ua.sh’ to push from a certain branch to my devices. This completed without issue.

I’ve just rewrote the build_ua.sh script within my Jenkinsfile, and now the Balena Push process (not Jenkins itself) is killed due to a Oom error.

I’m aware of Balena Deploy and Balena Build, and that might solve my issue, but I would really like to understand why this is happening when the same processes are being run as previously?

PS: I’m using --detached on the Balena Push command, and setting JENKINS_NODE_COOKIE to “dontKillMe”. Both of which I thought would assist with this situation.

Hey @spencerwf! Welcome to the forums. Out of curiosity, how much memory does the jenkins node have?

1 Like

The Out Of Memory (OOM) Killer is a function of the Linux kernel that kills user processes when free RAM is very low, in order to prevent the whole system from going down due to the lack of memory. It would appear at first glance that the combination of memory installed and memory being consumed is the issue. You can run htop while the process runs to see what processes are consuming memory and see if it’s what you expect.

Hello Toochevere,

Thank you for the responses. The device is on an instance with 2G of RAM, which I acknowledge is not much. However, I have ran Balena Push and Jenkins before on the same device, except Balena Push was in a separate .sh script which was called by Jenkins. Now Jenkins runs Balena Push directly, which was when it started being killed by the Oom killer.

Hi there, 2GB of memory is a little low for a build node/agent. Try enabling swap at the very least, to give the kernel space to page out. Ideally though, you should either consider ephemeral (on-demand) nodes with much more memory or upgrade existing nodes.