Profile Applicability:
 • Level 1

Description:
 Kubernetes allows secrets to be mounted either as environment variables or as files in volumes. It is recommended to minimize the use of environment variables for storing secrets and instead use mounted files for improved security.

Rationale:
 When secrets are passed as environment variables, they are often logged by application code in error cases or for debugging purposes. These logs may inadvertently expose the sensitive values of secrets to anyone who can access the logs. Mounting secrets as files avoids this risk by not exposing them through the environment, where they could be logged or printed by mistake.

Impact:
 Pros:

  • Reduces the risk of secret exposure in logs or through debugging tools.

  • Improves security by ensuring secrets are not embedded in application runtime environments.

  • Allows secrets to be updated without restarting the pod (if mounted as files).

Cons:

  • Applications that expect to read secrets from environment variables will require modification to use files instead.

Default Value:
 By default, Kubernetes does not define secrets, and it does not assume any specific configuration for secret handling.

Pre-requisites:

  • Access to the Kubernetes cluster with sufficient privileges to review and modify secrets configurations.

  • Application modification may be required to read secrets from mounted volumes instead of environment variables.

Remediation

Test Plan:

Using AWS Console:

  1. Navigate to each namespace or pod configuration and review how secrets are referenced.

  2. Check if secrets are being used in environment variables. If they are, assess the feasibility of moving them to mounted files.

Using AWS CLI:

  1. Run the following command to find any references to secrets used as environment variables:

kubectl get all -o jsonpath='{range .items[?(@..secretKeyRef)]} {.kind} {.metadata.name} {"\n"}{end}' -A

  1. Review the output for any references to secrets being used as environment variables.

Implementation Plan

Using AWS Console:

  1. Update the pod or deployment configurations to use mounted secrets instead of environment variables.

    • For example, instead of defining a secret in the env section, use the volumeMounts and volumes sections to mount the secret.

  1. After modifying the application to read from the mounted secret file, apply the new configuration.

Using AWS CLI:

  1. To find the relevant secrets being used as environment variables, run:

kubectl get all -o jsonpath='{range .items[?(@..secretKeyRef)]} {.kind} {.metadata.name} {"\n"}{end}' -A

  1. Modify the pod/deployment definition to mount the secret as a volume and update your application code to read from files in /etc/secrets (or another appropriate path).

  2. After updating, reapply the configuration to the cluster:

kubectl apply -f updated-pod-definition.yaml

Backout Plan

Using AWS Console:

  1. If the changes cause issues, revert the application configuration to read secrets from environment variables instead of mounted files.

  2. Revert the changes made to the secret handling configuration.

Using AWS CLI:

  1. If the new configuration introduces problems, you can delete the volume definition and switch back to using environment variables for secrets.

kubectl delete -f updated-pod-definition.yaml

  1. Restore the environment variable configuration in the pod/deployment definition and reapply.

References:

  1. Kubernetes Documentation on Using Secrets

  2. AWS Best Practices for Kubernetes Secret