Profile Applicability:
- Level 1
Description:
AWS Glue is a fully managed ETL (Extract, Transform, Load) service that facilitates data preparation and loading for analytics. One of the key features in AWS Glue is ML Transform (Machine Learning Transforms), which allows users to apply machine learning models to data as part of their ETL workflows. These transformations are used for tasks like anomaly detection, categorization, and other advanced data processing tasks.
To ensure the security of data processed by ML Transforms, it’s critical that the encryption at rest is enabled. This encryption ensures that all data processed by the ML models, including intermediate and final data, is encrypted while stored, protecting sensitive information from unauthorized access.
This SOP ensures that AWS Glue ML Transforms have encryption at rest enabled, ensuring that data processed by these transformations is securely stored.
Rationale:
Enabling encryption at rest for AWS Glue ML Transforms is important for:
Data Security: Protects sensitive data processed by machine learning models from unauthorized access.
Compliance: Helps meet compliance standards such as PCI-DSS, HIPAA, and SOC 2, which require encryption of data at rest.
Best Practices: Follows security best practices for protecting sensitive data in machine learning workflows.
Impact:
Pros:
Enhanced Data Security: Ensures that all data, including training data and model results, is encrypted and protected at rest.
Compliance: Helps with compliance requirements for data encryption, protecting sensitive information.
Data Integrity: Ensures that data used in machine learning transformations is secure, which is important for maintaining the integrity of the data and model.
Cons:
Performance Overhead: While encryption at rest has minimal impact on data processing, it may add overhead during data storage and retrieval.
Key Management: If using SSE-KMS, key management and permissions for KMS keys must be handled correctly, which adds some complexity.
Default Value:
By default, AWS Glue ML Transform Encryption at Rest is enabled using AWS-managed keys (SSE-S3). However, you can configure it to use AWS KMS for more granular control over encryption keys.
Pre-requisite:
AWS IAM Permissions:
glue:DescribeMLTransform
glue:GetMLTransform
glue:UpdateMLTransform
kms:DescribeKey (if using SSE-KMS)
AWS CLI installed and configured.
Basic knowledge of AWS Glue ML Transforms and S3 encryption.
Remediation:
Test Plan:
Using AWS Console:
Sign in to the AWS Management Console.
Navigate to AWS Glue under Services.
Go to Machine Learning Transforms.
Select the ML Transform you want to inspect.
In the Transform Details section, check the Encryption Settings:
If Encryption at Rest is enabled, it will display the encryption mode (either SSE-S3 or SSE-KMS).
If encryption is not enabled, it will either be set to None or not configured.
Using AWS CLI:
To describe the ML Transform and check if encryption at rest is enabled, run:
aws glue get-ml-transform --transform-id <transform-id> --query 'Transform.EncryptionMode'
The output should show the encryption mode:
SSE-S3 for AWS-managed encryption keys.
SSE-KMS for customer-managed encryption keys.
Example output
aws glue update-ml-transform \
--transform-id <transform-id> \
--transform-input '{"EncryptionMode": "SSE-KMS", "KmsKeyId": "<kms-key-id>"}'If EncryptionMode is None, then encryption at rest is not enabled.
Implementation Steps:
Using AWS Console:
Sign in to the AWS Management Console and navigate to AWS Glue.
In the AWS Glue Dashboard, go to Machine Learning Transforms and select the ML Transform to modify.
In the Encryption Settings, ensure that Encryption at Rest is enabled.
If using SSE-KMS, select or create a KMS key for encryption.
If using SSE-S3, ensure that the setting is configured to use the default AWS-managed keys.
Save the changes to apply encryption at rest.
Using AWS CLI:
To enable encryption at rest using SSE-KMS for an existing ML Transform, run:
aws glue update-ml-transform \
--transform-id <transform-id> \
--transform-input '{"EncryptionMode": "SSE-KMS", "KmsKeyId": "<kms-key-id>"}'To enable encryption at rest using SSE-S3, run:
aws glue update-ml-transform \
--transform-id <transform-id> \
--transform-input '{"EncryptionMode": "SSE-S3"}'Verify the update by running:
aws glue get-ml-transform --transform-id <transform-id> --query 'Transform.EncryptionMode'
Ensure that EncryptionMode is set to SSE-KMS or SSE-S3.
Backout Plan:
If enabling encryption at rest causes issues (e.g., connectivity problems or job failures):
Identify the affected ML Transform.
To disable encryption at rest, run:
aws glue update-ml-transform \
--transform-id <transform-id> \
--transform-input '{"EncryptionMode": "None"}'Verify that encryption at rest has been disabled and that the ML transform is functioning correctly.
Note:
Key Management: If using SSE-KMS, ensure that the KMS key is properly managed, rotated, and only accessible to authorized users and roles.
Performance Testing: After enabling encryption at rest, monitor the performance of the ML transform jobs to ensure that there are no significant delays in processing.
Security Monitoring: Set up CloudTrail and CloudWatch logs to monitor any unauthorized access attempts related to the KMS key or S3 bucket.