Profile Applicability:
 • Level 1

Description:
 Encrypt Kubernetes secrets, stored in etcd, using the secrets encryption feature during Amazon EKS cluster creation.

Rationale:
 Kubernetes stores secrets that pods can access via a mounted volume. While Kubernetes secrets are stored using Base64 encoding by default, encrypting them is a recommended practice. Amazon EKS clusters version 1.13 and higher support encrypting Kubernetes secrets using AWS Key Management Service (KMS) Customer Managed Keys (CMK).

Envelope encryption, which is the process of using a KMS-managed key to encrypt the secrets in etcd, is considered a security best practice for storing sensitive data. This additional layer of security protects against attackers if they gain access to etcd.

Impact:
 The secrets encryption feature must be enabled during EKS cluster creation. This encryption protects sensitive data, including service account keys and user-defined secrets, stored in etcd.

Default Value:
By default, Kubernetes secrets are stored in tmpfs and encrypted at rest when using Kubernetes with etcd encryption enabled. However, enabling encryption with CMKs for Kubernetes secrets must be configured during cluster creation.

Pre-requisites:
The EKS cluster must be created with KMS CMKs enabled for secrets encryption. This can only be done during the cluster creation process. If the cluster is already created, you must recreate it with encryption enabled.


Remediation

Test Plan:

Using AWS Console:

  1. Go to the EKS console and verify the cluster's encryption settings.

  2. Check if the cluster's secrets are encrypted with a valid KMS key by examining the cluster's configuration under Encryption Config.

Using AWS CLI:
Run the following command to describe the cluster and verify the encryption configuration:

  • aws eks describe-cluster --name <cluster-name> --query 'cluster.encryptionConfig' --region <region>
  • The output should show the keyArn with the KMS CMK used for encryption:

    • "encryptionConfig": [ { "provider": { "keyArn": "arn:aws:kms:region:account-id:key/key-id" }, "resources": ["secrets"] } ]


Implementation Plan

Using AWS Console:

  1. When creating a new Amazon EKS cluster, ensure that Secrets Encryption is enabled and configured with a valid KMS CMK.

  2. Verify that the keyArn points to a valid KMS-managed CMK used for secrets encryption during cluster creation.

Using AWS CLI:

  1. When creating a new EKS cluster, specify the encryption configuration:

    aws eks create-cluster --name <cluster-name> --region <region> --role-arn <role-arn> --resources-vpc-config subnetIds=<subnet-ids>,securityGroupIds=<sg-ids> --encryption-config resources=secrets,provider={keyArn=<kms-key-arn>}


Backout Plan

Using AWS Console:

  1. If encryption needs to be disabled or changed, recreate the cluster with the updated encryption configuration. Currently, this can only be done during cluster creation.

Using AWS CLI:

  1. To revert the encryption settings, you must recreate the EKS cluster with a new encryption configuration:

    aws eks create-cluster --name <cluster-name> --region <region> --role-arn <role-arn> --resources-vpc-config subnetIds=<subnet-ids>,securityGroupIds=<sg-ids> --encryption-config resources=secrets,provider={keyArn=<new-kms-key-arn>}


References:

  1. Amazon EKS Adds Envelope Encryption for Secrets with AWS KMS

  2. DescribeCluster API Reference