Profile Applicability

  • Level 2

Description:

Dataproc clusters store data on Persistent Disks (PDs) associated with Compute Engine VMs and in a Cloud Storage staging bucket. By default, this data is encrypted using Google-generated data encryption keys (DEK) and key encryption keys (KEK). However, enabling Customer-Managed Encryption Keys (CMEK) allows users to create, use, and revoke their own encryption keys for additional security and control. While Google still manages the DEK, CMEK gives users full control over the KEK.

Rationale:

CMEK provides an additional layer of security and control over the encryption process. By using customer-managed encryption keys, organizations can:

  • Protect sensitive data with encryption keys managed entirely by them.

  • Meet compliance and regulatory requirements by demonstrating control over data encryption.

  • Mitigate the risk of unauthorized access by maintaining strict control over the encryption keys.

Impact:

Implementing CMEK introduces administrative overhead as users are responsible for creating, protecting, and managing the lifecycle of encryption keys.

Default Value

Dataproc clusters use Google-managed keys for encryption by default.

Audit Steps:

Using Google Cloud Console

  1. Navigate to the Dataproc Clusters page.

  2. Select the desired project from the dropdown list.

  3. For each cluster:

    • Click the cluster name to open the details page.

    • Go to the Configurations tab.

    • Check the Encryption type attribute under the Configurations section. If it is set to Google-managed key, the cluster is not encrypted using CMEK.

  4. Repeat this process for all Dataproc clusters in the project.

  5. Switch projects using the dropdown list and repeat the procedure.

Using Google Cloud CLI

List all Dataproc clusters in a specific region:

gcloud dataproc clusters list --region='us-central1'

Get encryption details for a specific cluster:

gcloud dataproc clusters describe <CLUSTER_NAME> --region=us-central1 --flatten=config.encryptionConfig.gcePdKmsKeyName

  1. If the command output shows null, the cluster is not encrypted using CMEK.

Repeat these steps for all clusters in other regions and project

gcloud config set project <PROJECT_ID>

Remediation Steps:

Using Google Cloud Console

  1. Navigate to the Dataproc Clusters page.

                 

  1. Select the appropriate project.

           

  1. Create a new cluster:

    • Click Create Cluster.

                 

  • Under Set up cluster, specify:

    • A unique cluster name.

                                       

  • The desired location for deployment.

                                       

  • Under Manage security, select Customer-managed key and choose the desired CMEK from the dropdown list.

  • Ensure the KMS key has the CryptoKey Encrypter/Decrypter role assigned to the Dataproc service account (serviceAccount:service-<project_number>@computesystem.iam.gserviceaccount.com).

  1. Create the cluster and migrate workloads from the old cluster to the new one.

  2. Delete the old cluster:

    • Select the old cluster and click Delete cluster.

    • Confirm the deletion.

Using Google Cloud CLI

Ensure the KMS key has the CryptoKey Encrypter/Decrypter role assigned to the Dataproc service account:

gcloud kms keys add-iam-policy-binding <KEY_NAME> --keyring=<KEYRING_NAME> --location=<LOCATION> --member="serviceAccount:service-<PROJECT_NUMBER>@computesystem.iam.gserviceaccount.com" --role="roles/cloudkms.cryptoKeyEncrypterDecrypter

Create a new CMEK-enabled cluster:

gcloud dataproc clusters create <CLUSTER_NAME> --region=us-central1 --gce-pd-kms-key=<KMS_KEY_RESOURCE>

Migrate workloads to the new cluster and delete the old cluster:

gcloud dataproc clusters delete <OLD_CLUSTER_NAME> --region=us-central1

Switch projects and repeat the procedure:

gcloud config set project <PROJECT_ID>

Backout Plan:

Step 1: Revert Encryption to Google-Managed Keys

If CMEK causes service disruptions, create a new cluster without CMEK:

gcloud dataproc clusters create <NEW_CLUSTER_NAME> --region=us-central1

  •   Migrate workloads to the new cluster and delete the CMEK-encrypted cluster.

Step 2: Notify Stakeholders

  • Inform security and DevOps teams before making encryption changes.

References:

1. Dataproc Encryption with CMEK

CIS Controls:

Control

Description

IG 1

IG 2

IG 3

3.11 Encrypt Sensitive Data at Rest

Encrypt sensitive data stored in servers, applications, and databases. CMEK provides greater control over encryption compared to default Google-managed keys.

✅

✅


14.8 Encrypt Sensitive Information at Rest

Encrypt sensitive data at rest using encryption tools that require secondary authentication, enhancing data protection against unauthorized access.

✅

✅