Profile Applicability:

  • Level 1

Description:
 Kubernetes allows secrets to be stored within the cluster using the Secret object. However, storing secrets in Kubernetes directly may not meet the highest security standards, especially when dealing with sensitive data such as passwords and API keys. This check encourages using external secret management solutions, like Azure Key Vault, HashiCorp Vault, or other third-party providers, to manage and securely store secrets outside of Kubernetes.

Rationale:
 While Kubernetes can encrypt secrets at rest, using an external secrets provider offers additional layers of security, auditability, and compliance features, such as automatic secret rotation, centralized management, and better integration with enterprise security policies. External secret storage solutions are designed to handle sensitive information with high levels of security and provide easy access controls.

Impact:

  • Pros:

    • Enhances security by centralizing secret management in a specialized service.

    • Provides better audit logging and access control for secrets.

    • Helps with compliance requirements for sensitive data handling.

  • Cons:

    • Requires additional setup and configuration to integrate external secret stores with Kubernetes.

    • Introduces external dependencies for managing secrets, which may increase operational complexity.

    • External storage solutions may incur additional costs.

Default Value:
 By default, Kubernetes stores secrets in the etcd database, which is unencrypted unless specifically configured. External secret storage must be manually configured.

Pre-requisites:
 Ensure that you have an external secret management service such as Azure Key Vault or HashiCorp Vault set up and ready to be integrated with Kubernetes.

Test Plan:

Using Azure Console:

  1. Navigate to the Azure portal and verify that Azure Key Vault (or another external secret provider) is properly configured.

  2. Check if the Kubernetes cluster is configured to use Azure Key Vault or another external secret provider to manage secrets.

  3. Review the integration setup between Kubernetes and the external secrets provider to ensure it is functioning as expected.

Using Azure CLI:

  1. Check if Azure Key Vault integration is enabled by running:

     az aks show --resource-group <resource-group-name> --name <aks-cluster-name> --query "addonProfiles.azureKeyvaultSecretsProvider.enabled"


  2. Verify that the correct secret store is being used to fetch and store secrets by running:

     kubectl get secrets --all-namespaces


Implementation Plan:

Using Azure Console:

  1. In the Azure portal, create an Azure Key Vault and configure it to store your secrets.

  2. Integrate Azure Key Vault with your AKS cluster by enabling the Azure Key Vault Secrets Provider under Add-ons.

  3. Use the Secrets Store CSI Driver to mount the secrets from Azure Key Vault into Kubernetes Pods as volumes or environment variables.

  4. Ensure that Kubernetes Pods access secrets only through the Azure Key Vault integration, not through native Kubernetes Secrets resources.

Using Azure CLI:

  1. Create a new Azure Key Vault using the following command:

     az keyvault create --name <keyvault-name> --resource-group <resource-group-name>


  2. Enable the Azure Key Vault Secrets Provider for your AKS cluster:

     az aks enable-addons --resource-group <resource-group-name> --name <aks-cluster-name> --addons azure-keyvault-secrets-provider


Create a SecretProviderClass to define how the Kubernetes Pods will retrieve secrets from Azure Key Vault:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: example-provider
spec:
  provider: azure
  secretObjects:
  - secretName: example-secret
    secretType: Opaque
    data:
    - objectName: secret1
      key: key1
Apply the SecretProviderClass to your cluster:
 kubectl apply -f secret-provider-class.yaml


Backout Plan:

Using Azure Console:

  1. If the integration with external secret storage causes issues, revert the configuration in the Azure portal by disabling Azure Key Vault Secrets Provider or switching back to Kubernetes-native secrets.

Using Azure CLI:

  1. Revert the integration by disabling the Azure Key Vault Secrets Provider with the following command:

     az aks disable-addons --resource-group <resource-group-name> --name <aks-cluster-name> --addons azure-keyvault-secrets-provider


  2. Restore the use of Kubernetes-native secrets if necessary.

References:

  1. Azure Key Vault Documentation

  2. Kubernetes Secrets Store CSI Driver Documentation

  3. Azure Kubernetes Service (AKS) Key Vault Integration