Hello
we are using the balenaEngine on Jetson Nano as part of our openBalena deployment.
On some occasions, processes that have been started within our application container seem to escape the container while still being active in the background and allocating resources. So far we haven’t figured out the exact context in which this behaviour occurs, but it seems to be independent of the processes themselves.
In order to reproduce this behaviour:
-
Enter container via balnea exec -it
-
Start long-running script in the background sudo python3 my_script.py &
-
Exit container via exit
-
Check for the process via ps | grep python3
Is there a reasonable explanation for this behaviour? Am I missing something?
Any input is highly appreciated!
Just as a followup information, we are running these scripts in privileged mode
Hey there @Langhalsdino
I’m not sure what you mean by “processes escape the container”, and the behaviour you describe seems nominal so I’ll try to give a bit of information about why it seems like that.
I’m reproducing what you describe so I’m entering an existing, running container using balena exec -it 07c /bin/bash where 07c is my container ID and /bin/bash is the command I want to run. At this point I’m able to start a script in the background as you’ve done, so I simply made a bash script with a sleep in it. I start this and send it to the background with ./test.sh &, and can confirm it’s running as PID 112 with the internal sleep command as PID 113
I now leave the container by issuing exit, but note that the container and all it’s processes are still running, I’ve just closed the bash instance I originally started with balena exec and hence am dropped back to the host OS leaving my test process behind.
Now if I run ps | grep sleep on the host OS I get:
This doesn’t mean the process has escaped the container though, we can confirm that by looking in the file /proc/<PID>/status for the NSpid line. If I look at this for my sleep process, I can see it shows the PID that the process is running under within the container:
Hopefully this helps but let us know if not!
P.S. love the work you’re doing at APIC.ai 