Best way to set all GPIOs to zero before container shutdown?

Product support
1

I would like to ensure that, before a container is shutdown, during the OTA process, the outputs of the IO modules connected to the RPI are set to zero. I was thinking about something like https://www.npmjs.com/package/node-cleanup, but I would like to make sure that it will not tamper with the balena cloud logics. Will there be any side effects?

5

Hi

No, it shouldn’t cause any problem for the supervisor :slight_smile:

7

@kaisoz I have tested the issue with node-clean up mentioned above, and the code below:

const IO = require('./io');
var nodeCleanup = require('node-cleanup')

nodeCleanup(function (exitCode, signal) {
  if (signal) {
    console.log(`Running cleanup...`)

    try {
      IO.detach()
      IO.reset()
      nodeCleanup.uninstall();
    } catch(e) {
      console.log(e)
    }
  }
});

module.exports = nodeCleanup

But nothing seems to happen. I have tested the code in docker-compose without balena, and the script is called correctly when i run docker-compose down. How does balena shutdown the containers? Meaning, what command does it use?

8

Fyi, I have run a separate project to be able to test issue, where I am running an express server using docker compose in balena-cloud. When stopping the container, the cleanup code does not seem to get called, as the “Running cleanup…” message is not shown on the web console… or, I am thinking, is it possible that the code is actually getting called, but the console detached from the container already?

UPDATE: I have been doing more testing outside of balena, and the cleanup seems to work only on docker-compose down. If I docker kill a container, the cleanup code does not get called. By looking at the balena cloud web console, I see balena says “killing container”, which I am taking to mean that the docker kill command is being used to stop a container. @kaisoz can you confirm?

Assuming those thoughts are correct, and that balena uses SIGKILL to stop a container, then it’s impossible to run cleanup code, as the SIGKILL signal cannot be intercepted. What other options are there to ensure all pins are set to zero on container shutdown then?

10

Hi, the phrasing in the logs is a bit confusing, unfortunately. But no, sigkill is not used directly to shutdown a container.

When you see the supervisor logs saying about killing a service, the suprevisor actually sends a container stop request to the containers engine.

This is equivalent to running docker stop command.
The engine favors the container stop signal settings that can be defined on your image level[1] or on the container level[2] in the compose file.
1- https://docs.docker.com/engine/reference/builder/#stopsignal
2 - https://docs.docker.com/compose/compose-file/compose-file-v2/#stop_signal

So, the engine first sends the signal specified as the stop signal for the container and then can send a sigkill after a timeout.

12

Is it possible that the web console is just not showing it then?

13

Sorry, I did’t get the last question.
What is not shown on the dashboard?

14

In my cleanup code I am writing a string to the log that, on shutdown, should be displayed in the console:

console.log(`Running cleanup...`)

(see example above) to show that the clean up code is actually running. Unfortunately, it never appears in the log console in balena-cloud. Would you expect it to show?

UPDATE: I have just tested cleanup with docker by using the docker stop command and it seems to work. So it does not seem to be a docker issue.

15

Ah, got it.

Yeah, it might be an issue (probably the supervisor does not capture the logs after sending the stop command).
Would you test it with balena-engine ps -a, and then balena-engine logs on the device to check container logs (you could try issuing balena-engine stop command manually to test, just keep in mind that the supervisor will try to bring the service up again eventually, to reach the target state dictated by the. backend).

17

Mm I just ran a test using balena-engine, and no message is showing up on container stop… I also tried sending an HTTP request to a server in the callback, but nothing seems to be sent…

18

Hey, just to clarify did you try to run balena-engine stop <containerID> followed by balena-engine logs <containerID> and didn’t observe the Running cleanup message in the container logs?

19

That’s correct, I have used balena-engine logs -f <CONTAINER_ID>, which at some point stops (as expected), but nothing is showing up…

20

Thanks, let me check with the balena-engine maintainer and get back to you.

21

thanks a lot, I really appreciate your help.

23

Fyi, I have also tried setting the stop_signal in docker-compose to SIGINT, but no change…

27

@nazrhom I found something interesting in this stackoverflow question. Basically, it has to do with the PID. In fact, in order for a process running inside a container to receive POSIX signals, it needs to be running as PID 1. In order to achieve this, docker has to be started with the exec format, while mine was being run in shell format (ENTRYPOINT is being advised in the source mentioned above). In fact, I had to change my Docker file from

CMD node bin/www

to

ENTRYPOINT ["node", "bin/www"]

Now the PID of the process that’s running is 1, and when call balena-engine stop <CONTAINER_ID> the cleanup code gets called. Similarly, when I press the “stop” button from the web ui within balena cloud, the cleanup code seems to run correctly.

31

Hi

That’s a good lead! What base images are you using for your docker? Can you share that information?

33

This is the base image I am using

FROM balenalib/raspberrypi3-python:3-stretch-run

running on balenaOS 2.52.7+rev1

39

Thanks Tommaso, I’ll pass that along. Nice work chasing this down!