Profile Applicability:
- Level 1
Description:
AWS Glue is a fully managed ETL (Extract, Transform, Load) service that allows users to discover, prepare, and load data for analytics. The Glue Data Catalog is the central metadata repository for all data assets in AWS Glue, and it contains the metadata for all tables, databases, and other objects.
Metadata encryption ensures that sensitive metadata stored in the Glue Data Catalog, such as database names, table names, column names, and schema information, is securely encrypted using AWS Key Management Service (KMS). This SOP ensures that metadata encryption is enabled for your Glue Data Catalog to protect sensitive information from unauthorized access.
Rationale:
Enabling metadata encryption for the Glue Data Catalog is critical for:
Data Security: Protects sensitive metadata information that might contain data schema, connections, or other confidential details.
Compliance: Helps meet regulatory requirements like PCI-DSS, HIPAA, and SOC 2 that require encryption of sensitive data.
Best Practice: Encrypting metadata aligns with industry security best practices for managing data assets and ensuring data privacy.
Impact:
Pros:
Enhanced Security: Ensures that all metadata stored in the Glue Data Catalog is protected from unauthorized access.
Compliance: Supports compliance with various data security and privacy standards.
Data Protection: Protects metadata, which can sometimes contain sensitive information about your data structure and business logic.
Cons:
Operational Overhead: Requires configuring and managing KMS keys for encryption.
Potential Performance Overhead: While the encryption overhead is typically minimal, it may affect performance when reading and writing metadata at scale.
Default Value:
By default, AWS Glue does not automatically enable metadata encryption for the Data Catalog. You must explicitly enable encryption using AWS KMS when setting up or modifying the Glue Data Catalog.
Pre-requisite:
AWS IAM Permissions:
glue:GetCatalogEncryptionSettings
glue:UpdateCatalogEncryptionSettings
kms:DescribeKey
AWS CLI installed and configured.
Basic knowledge of AWS Glue Data Catalog, KMS encryption, and IAM permissions.
Remediation:
Test Plan:
Using AWS Console:
Sign in to the AWS Management Console.
Navigate to AWS Glue under Services.
In the AWS Glue Dashboard, go to Data Catalog and select Encryption Settings.
Under Metadata Encryption, check if AWS KMS encryption is enabled. The KMS Key used for encryption should be displayed.
If metadata encryption is enabled, it will show that the KMS key is actively encrypting your Glue Data Catalog metadata.
If encryption is not enabled, it will show that no encryption is applied.
Using AWS CLI:
To describe the Glue Data Catalog encryption settings, run:
aws glue get-catalog-encryption-settings --query 'CatalogEncryptionSettings.EncryptionAtRest'
The output should indicate if metadata encryption is enabled. For example:
{
"CatalogEncryptionSettings": {
"EncryptionAtRest": {
"EncryptionMode": "DISABLED"
}
}
}If EncryptionMode is set to DISABLED, then metadata encryption is not enabled. You can then proceed to enable encryption.
Implementation Steps:
Using AWS Console:
Sign in to the AWS Management Console and navigate to AWS Glue.
In the AWS Glue Dashboard, go to Data Catalog and select Encryption Settings.
In the Encryption Settings section, select Enable Encryption for Metadata.
Choose an AWS KMS Key (either default AWS-managed key or a custom key from your account).
Click Save to apply the changes and enable metadata encryption for your Glue Data Catalog.
Using AWS CLI:
To enable metadata encryption using AWS KMS, run the following command:
aws glue update-catalog-encryption-settings \
--catalog-id <catalog-id> \
--encryption-at-rest "{\"EncryptionMode\": \"SSE-KMS\", \"KmsKeyId\": \"<kms-key-id>\"}"Verify that encryption is enabled by running the get-catalog-encryption-settings command again:
aws glue get-catalog-encryption-settings --query 'CatalogEncryptionSettings.EncryptionAtRest'
Ensure that EncryptionMode is set to SSE-KMS, and the KmsKeyId is the appropriate KMS key.
Backout Plan:
If enabling metadata encryption causes issues or disrupts the Glue Data Catalog operations:
Identify the affected KMS key and domain.
To disable metadata encryption, run:
aws glue update-catalog-encryption-settings \
--catalog-id <catalog-id> \
--encryption-at-rest "{\"EncryptionMode\": \"DISABLED\"}"Verify that the EncryptionMode is set to DISABLED and that the Glue Data Catalog is functioning normally.
Note :
KMS Key Management: Ensure the KMS key used for encryption is properly managed and rotated according to your organization’s key management policies.
Monitoring: Use CloudWatch Logs and CloudTrail to monitor any errors or unauthorized access attempts related to metadata encryption changes.
Backup: Ensure that you have backups of your Glue Data Catalog settings in case you need to revert changes.