Profile Applicability:

  • Level 1

Description:

AWS Glue is a fully managed ETL (Extract, Transform, Load) service that simplifies the preparation and loading of data for analytics. Job bookmarks in AWS Glue help maintain the state between job runs, allowing the job to process only the new or modified data since the last run. This prevents reprocessing large amounts of data unnecessarily.

Job bookmark encryption ensures that job bookmark metadata, which tracks the state of data processing, is securely stored and encrypted. By enabling job bookmark encryption, sensitive information regarding job state transitions is protected from unauthorized access.

This SOP ensures that AWS Glue ETL jobs have job bookmark encryption enabled to maintain the security of job metadata.

Rationale:

Enabling job bookmark encryption for AWS Glue ETL jobs is essential for:

  • Data Security: Ensures that sensitive metadata about job states, transformations, and data processing is encrypted at rest.

  • Compliance: Helps meet compliance standards like PCI-DSS, HIPAA, and SOC 2, which require encryption of sensitive data, including job metadata.

  • Best Practices: Aligns with security best practices by ensuring that job bookmark information is not exposed or accessible by unauthorized parties.

Impact:

Pros:

  • Enhanced Security: Job bookmark encryption ensures that sensitive metadata is protected from unauthorized access.

  • Compliance: Meets regulatory compliance requirements for encrypted data storage.

  • Data Integrity: Ensures that job metadata, used for incremental data processing, is secure and protected from tampering.

Cons:

  • Key Management: If using SSE-KMS encryption, managing the KMS keys introduces additional overhead for permissions and key rotation.

  • Performance Overhead: There is a minimal overhead introduced during encryption and decryption processes, though this is generally negligible.

Default Value:

By default, AWS Glue ETL jobs do not have job bookmark encryption enabled unless explicitly configured during job setup or modification. You must explicitly enable encryption for job bookmarks when creating or editing an AWS Glue ETL job.

Pre-requisite:

  • AWS IAM Permissions:

    • glue:DescribeJob

    • glue:GetJob

    • glue:UpdateJob

    • kms:DescribeKey (if using SSE-KMS)

  • AWS CLI installed and configured.

  • Basic understanding of AWS Glue, job bookmarks, and SSE-S3 or SSE-KMS encryption.

Remediation:

Test Plan:

Using AWS Console:

  1. Sign in to the AWS Management Console.

  2. Navigate to AWS Glue under Services.

  3. In the AWS Glue Dashboard, go to Jobs and select the ETL job to check.

  4. Under the Job Details, check the Security Configuration section:

    • Look for Job Bookmark Encryption settings.

    • If encryption is enabled, it will show SSE-KMS or SSE-S3.

    • If encryption is not enabled, the encryption setting will either be absent or set to None.

Using AWS CLI:

To describe the ETL job and check if Job Bookmark Encryption is enabled, run:

aws glue get-job --job-name <job-name> --query 'Job.SecurityConfiguration'

  1. In the output, look for JobBookmarkEncryption:

    • If JobBookmarkEncryption is enabled, it will show SSE-KMS or SSE-S3.

    • If it’s not enabled, JobBookmarkEncryption will be absent or set to None.

Example output:

{

    "Job": {

        "SecurityConfiguration": {

            "JobBookmarkEncryption": "SSE-KMS",

            "KmsKeyId": "arn:aws:kms:us-east-1:123456789012:key/abcd-1234"

        }

    }

}

Implementation Steps:

Using AWS Console:

  1. Sign in to the AWS Management Console and navigate to AWS Glue.

  2. Go to Jobs and select the ETL job to modify.

  3. In the Security Configuration section, enable Job Bookmark Encryption:

    • Set Job Bookmark Encryption to SSE-KMS or SSE-S3.

    • If using SSE-KMS, select the KMS key to encrypt the job bookmark data.

  4. Save the changes to enable job bookmark encryption.

Using AWS CLI:

To enable job bookmark encryption for an existing job, run the following command:

aws glue update-job \

  --job-name <job-name> \

  --job-update '{"SecurityConfiguration": {"JobBookmarkEncryption": "SSE-KMS", "KmsKeyId": "<kms-key-id>"}}'

To verify the encryption setting, run:

aws glue get-job --job-name <job-name> --query 'Job.SecurityConfiguration'

  1. Ensure that JobBookmarkEncryption is set to SSE-KMS or SSE-S3.

Backout Plan:

If enabling job bookmark encryption causes issues (e.g., job failures or access issues):

  1. Identify the affected ETL job.

To disable job bookmark encryption, run:

aws glue update-job \

  --job-name <job-name> \

  --job-update '{"SecurityConfiguration": {"JobBookmarkEncryption": "None"}}'

  1. Verify that job bookmark encryption has been disabled and that the job is functioning as expected.

Note:

  • Key Management: If using SSE-KMS, ensure that the KMS key is properly managed, rotated, and only accessible to authorized users and roles.

  • Performance Monitoring: After enabling job bookmark encryption, monitor the performance of the ETL jobs to ensure minimal impact.

  • Security Monitoring: Set up CloudTrail and CloudWatch logs to monitor any unauthorized access attempts related to the KMS key or job metadata.

References:

CIS Controls Mapping:

Version

Control ID

Control Description

IG1

IG2

IG3

v8

3.4

Encrypt Data on End-User Devices – Ensure data encryption during file system access.

●

●

●

v8

6.7

Implement Application Layer Filtering and Content Control – Ensure appropriate content filtering is applied to sensitive files.

●

●

●

v8

6.8

Define and Maintain Role-Based Access Control – Implement and manage role-based access for file systems.

●

●

●

v8

14.6

Protect Information Through Access Control Lists – Apply strict access control to file systems.

●

●

●