Balena Supervisor deleting my container after a few minutes

Product support
1

The supervisor seems to kill and delete my container MAIN at random. I don’t understand why.

Support access is enabled and the uuid is: a49451f78439e9a53876fb5a6172f6ee

The logs that seem relevant. This seems to happen almost randomly.

03.04.19 18:23:07 (-0700) Killing service ‘main sha256:ad06eec436661dbf4602c4cf413c6443cfe54b84b95ac10adfb1bddc56f59f48’
03.04.19 18:23:19 (-0700) Service exited ‘main sha256:ad06eec436661dbf4602c4cf413c6443cfe54b84b95ac10adfb1bddc56f59f48’
03.04.19 18:23:19 (-0700) Killed service ‘main sha256:ad06eec436661dbf4602c4cf413c6443cfe54b84b95ac10adfb1bddc56f59f48’
03.04.19 18:23:20 (-0700) Deleting image ‘registry2.balena-cloud.com/v2/b69b26628801bb587452baca9ded92dc@sha256:c406bf507637a7daf5aeffe12e5cf1e278b3dfd3edc7e5d91a2d2378489435f6’
03.04.19 18:23:23 (-0700) Deleted image ‘registry2.balena-cloud.com/v2/b69b26628801bb587452baca9ded92dc@sha256:c406bf507637a7daf5aeffe12e5cf1e278b3dfd3edc7e5d91a2d2378489435f6’
03.04.19 18:23:24 (-0700) Removing volume ‘resin-data’

4

Hi @tacLog,

Thanks for the support access. I took a look at the supervisor errors and I suspect something might be off with your docker compose file. Specifically, I remember seeing such errors when there is malformed environment variable including extra spaces for instance.

Could you please check if your variables are set correctly?

I’ll also send a private message to get your docker compose file. It’d be great if you could share the relevant bits.

5

I just sent you my Dockerfile @gelbal

If by environment variables you mean lines starting with: ENV
I don’t have any of those in my docker file.

I double checked everything in my Dockerfile and I am going to include what I can here for the sake of anyone googling this later:

FROM balenalib/raspberrypi3-alpine-python:3.6.8-3.9-build
MAINTAINER Thomas ‘tacLog’ MyLastNmae thomas.lastname@company.com

VOLUME [ “/pcaps” ]

WORKDIR /usr/src/app
RUN apk add --no-cache nameOfPrereqThatGivesTooMuchAway coreutils sshpass
RUN install_packages git

COPY nameOfMyScriptInThisFormat.py .

CMD [ “python3”, “nameOfMyScriptInThisFormat.py” ]

6

Hi
I am just following up on this thread because I cannot make out the final status. I looks to me like the issue has been solved and your service is running.
Is that correct ?
Regards
Thomas

7

Yes, it has been running all day.

I still want to nail down exactly what caused the issue in the first place. I will a message I sent to @gelbal public for context. (It contains what I did to try and resolve things on my end)

8

The solution that worked for me:

At least for now making the following changes helped: (it has run for about 2 hours now)

moved setting my WORDDIR to below the RUN commands.

COPY senseAndSend.py .

to

COPY senseAndSend.py senseAndSend.py

removed the volume:
VOLUME [ “/nameOfVolumue” ]

I understand the least about what removing the volume did, but I suspect that may have been the issue as I haven’t seemed that command in any of the balena example projects.

I just want to make it clear to anyone having this problem down the road what I did to solve it.

I would also love anyone’s perspective on what might have caused this issue.

9

Hey @samothx and @gelbal

I spoke too soon.
It just deleted it again :frowning:

Service exited 'main sha256:fe6e25c4010fc3507deb029d9cdc0d7df7edc0c0922ebfc4f3430c18db076dfa'
04.04.19 15:35:57 (-0700) Killed service 'main sha256:fe6e25c4010fc3507deb029d9cdc0d7df7edc0c0922ebfc4f3430c18db076dfa'
04.04.19 15:35:58 (-0700) Deleting image 'registry2.balena-cloud.com/v2/dac849672f016b2e6192284cf1b12344@sha256:d37e8ec371d001996a338d822d5de276340c007dfd7ce708c61aace63f8ba780'
04.04.19 15:36:02 (-0700) Deleted image 'registry2.balena-cloud.com/v2/dac849672f016b2e6192284cf1b12344@sha256:d37e8ec371d001996a338d822d5de276340c007dfd7ce708c61aace63f8ba780'
04.04.19 15:36:02 (-0700) Removing volume 'resin-data'

Current Dockerfile:

FROM balenalib/raspberrypi3-alpine-python:3.6.8-3.9-build
MAINTAINER Thomas 'tacLog' LastName <thomas.lastName@company.com>

RUN apk add --no-cache seePMforThisRequirement coreutils sshpass
RUN install_packages git

WORKDIR /usr/src/app

COPY scriptName.py scriptName.py

CMD [ "python3", "scriptName.py" ]
10

Hey @tacLog, thanks for the followup. I just checked your device and it seems like working fine now. Your main service up and running for the past 2 days. Then I don’t see these supervisor logs anymore which we saw earlier.

So shall we conclude that the issue resolved now?

11

Yes @gelbal

Thanks for you both of your help!

For the sake of anyone reading this in the future.
I don’t know what was causing the issue, but it has no showed up since I made my application more stable so I suspect that I was breaking off too many threads that were taxing the memory or cpu.

But I may never know for sure.

12

Thanks for the note @tacLog. Yes, high memory / cpu consumption by the application is a plausible reason.