How to debug a container which is in a crash loop?

Product support
1

With running systemd inside the container deprecated, what is the best way to get shell access to a container which is in a crash loop?

4

hey @whm, my own debug process relies on a start script, instead of an initsystem.

In this case, there would be a start script, let’s call it start.sh that contains what’s needed for the application to be started, and add a section that if the application exitted, then it keeps the container running, so you can go in and investigate. For example something like this:

#!/usr/bin/bash

the-command-that-i-want-to-run

echo "Application exited."
while : ; do
  echo "Idling..."
  sleep 600
done

which will stick around sleeping if the application finished, so you can investigate. And instead of your Dockerfile using CMD the-command-that-i-want-to-run, would use CMD ["bash", "start.sh"] (don’t forget to make sure the start.sh is copied into the container).

A variant of this, to use for really debugging:

#!/usr/bin/bash

the-command-that-i-want-to-run

if [ -n "$DEBUG" ]; then
  echo "Application exited."
  while : ; do
    echo "Idling..."
    sleep 600
  done
fi

which would behave such, that if you set an environment variable (service variable) DEBUG to some value, it will end up in the idling state if the application exits/crashes, but if you don’t set a variable like that, it will not hold up the container restart (which might be bad in production devices).

Does this make sense? And is it any help? :slight_smile:

3 Likes
6

Yes, that is very helpful. Thank you!

1 Like