Profile Applicability:

  • Level 1

Description:

Docker containers should be configured to have their health checked at runtime to ensure the application running inside the container is functioning as expected. Docker provides a HEALTHCHECK instruction in Dockerfiles to define a command that can test the health of the container.

Rationale:

Health checks help monitor the state of the application inside the container. If a container becomes unhealthy, it can be restarted automatically or handled in other ways, reducing downtime and preventing problems from escalating.

Impact:

Pros:

  • Allows automated remediation of failed containers.

  • Reduces manual intervention and provides proactive monitoring.

Cons:

  • May slightly increase resource usage due to regular health checks.

  • Some applications might require specific configurations for proper health check commands.

Default Value:

Health checks are not defined by default in Docker images. They need to be manually configured in Dockerfiles or at runtime.

Pre-requisites:

  • Ensure Docker is installed and running on the host system.

  • Health check commands must be properly defined in the container's Dockerfile or specified when running the container.

Remediation:

Test Plan:

Using AWS Console:

  1. Navigate to the EC2 instance running Docker.

  2. Run the following command to inspect the container's health status:

docker inspect --format '{{.State.Health.Status}}' <container_name_or_id>

  1. Ensure the container has a valid health check configured.

Using AWS CLI:

  1. Connect to the EC2 instance.

  2. Run the following command to check the health status of a running container:

docker inspect --format '{{.State.Health.Status}}' <container_name_or_id>

Implementation Plan:

Using AWS Console:

  1. Connect to the EC2 instance where Docker is running.

    1. Modify the Dockerfile to include a HEALTHCHECK instruction. Example:

      HEALTHCHECK CMD curl --fail http://localhost:8080/health || exit 1

  2. Rebuild the container image and redeploy the container with the updated Dockerfile.

Using AWS CLI:

  1. Use SSM to update the Dockerfile remotely and redeploy the container:

Run the following command to apply the health check:

aws ssm send-command --document-name "AWS-RunShellScript" --targets "Key=instanceIds,Values=instance_id" --parameters 'commands=["echo \"HEALTHCHECK CMD curl --fail http://localhost:8080/health || exit 1\" >> /path/to/Dockerfile && docker build -t my-image . && docker run -d my-image"]'

Backout Plan:

Using AWS Console:

  1. Connect to the EC2 instance.

  2. Remove or comment out the HEALTHCHECK instruction from the Dockerfile.

  3. Rebuild and redeploy the container without the health check.

Using AWS CLI:

  1. Use SSM to revert the Dockerfile and redeploy the container:

Run the following command to remove the health check:

aws ssm send-command --document-name "AWS-RunShellScript" --targets "Key=instanceIds,Values=instance_id" --parameters 'commands=["echo \"#HEALTHCHECK CMD curl --fail http://localhost:8080/health || exit 1\" >> /path/to/Dockerfile && docker build -t my-image . && docker run -d my-image"]'

References: