Docker: Stopping containers, softly

25 May 2016

Running a process inside a Docker container is not too hard. But there are some pitfalls with containers not reacting to SIGTERM and SIGINT, as sent by docker stop or by pressing Cmd + C on your keyboard when attached to a container.

We have seen processes not being stopped properly, and there are different solutions per exact use-case.

Hang on if your containers don’t stop like they should. Kill them softly.

ENTRYPOINT: exec form vs shell form

As stated in the official documentation, ENTRYPOINT supports two formats:

  1. exec form: ENTRYPOINT ["php", "run.php"]
  2. shell form: ENTRYPOINT php run.php

Shell form comes with a disadvantage, though:

The shell form prevents any CMD or run command line arguments from being used, but has the disadvantage that your ENTRYPOINT will be started as a subcommand of /bin/sh -c, which does not pass signals. This means that the executable will not be the container’s PID 1 - and will not receive Unix signals - so your executable will not receive a SIGTERM from docker stop <container>.

In general, you should prefer exec form, but we encountered a case where shell form was required in Dockerfile:

FROM java:8
ENV JAVA_OPTS=-Duser.timezone=Europe/Berlin
CMD java $JAVA_OPTS -jar /app/app.jar

This would run java $JAVA_OPTS -jar /app/app.jar as a sub-process of /bin/sh -c, thus not reacting properly to SIGTERM.

Simple fix: Use exec. It’s manual page says:

The exec() family of functions replaces the current process image with a new process image.

Our Dockerfile becomes:

FROM java:8
ENV JAVA_OPTS=-Duser.timezone=Europe/Berlin
CMD exec java $JAVA_OPTS -jar /app/app.jar

And now it correctly reacts to SIGTERM sent by docker stop, instead of being forcefully terminated as before.

ENTRYPOINT: Custom shell script

In a different case, a run.sh shell script wraps the main command. It handles some preparation and cleanup. Our ENTRYPOINT is as simple as:

FROM node:6
ADD run.sh /opt/run.sh
ENTRYPOINT ["/opt/run.sh"]
CMD [""]

With run.sh:

#!/bin/bash

# Preparation
# [...] 

# Main command
grunt "$*"

# Cleanup
# [...]

With this setup, the main command would not react to SIGTERM or SIGINT, making it impossible to stop long-running processes (in this case, grunt watch watching the file system for changes infinitely).

To allow passing signals to the main command, you can use a simple but rather verbose trap construct:

#!/bin/bash

_term() {
  kill -TERM "$child" 2>/dev/null
}

trap _term SIGINT SIGTERM

# Preparation
# [...] 

# Main command
grunt "$*" &
          
child=$!
wait "$child"

# Cleanup
# [...]

Other solutions

You can solve this problem on a higher level by using “a minimal init system for Linux containers”, dumb-init. You install it during build by adding commands in Dockerfile and then annotate in shebang line in run.sh to avoid the overhead of the above trap construct.

On the downside, it adds another installation step and a dependency to your Dockerfile, making it more verbose.

Wrap-up

Handling signals in Docker container child processes is important to gracefully shut down processes and have cleanup stages run successfully.

In local development, containers are properly stopped and do not block your IDE from shutting down.

The above methods might not be the most beautiful options out there to solve these issues, so let us know if you found better ways!