Profile Applicability:

Level 1

Description:

AWS Batch is a fully managed batch processing service that allows you to run hundreds to thousands of batch computing jobs on AWS. Configuring AWS Batch with AWS CloudWatch Logs ensures that logs from your batch jobs are captured for monitoring, troubleshooting, and auditing purposes. By enabling CloudWatch Logs, you can view, search, and analyze log data generated by your batch jobs in real-time.

Rationale:

Configuring AWS Batch with CloudWatch Logs ensures that:

  • Logs generated by your batch jobs are automatically captured for further analysis.

  • You can monitor the health and status of batch jobs, which is essential for troubleshooting issues and ensuring smooth operations.

  • Logging helps in auditing batch job execution and adhering to compliance requirements.

Default Value:

By default, AWS Batch jobs are not configured to send logs to CloudWatch. You need to manually configure CloudWatch Logs as part of your job definitions.

Impact:

  • Pros:

    • Enhanced visibility into batch job executions and troubleshooting.

    • Real-time access to logs for monitoring performance, identifying errors, and tracking job progress.

    • Improved compliance by ensuring all job logs are stored and available for auditing.

  • Cons:

    • Requires proper configuration to ensure logs are being sent to CloudWatch.

    • CloudWatch logs can accumulate over time, leading to higher storage costs.

Remediation:

Test Plan:

Using AWS Console:

  1. Log in to the AWS Console at AWS Console.

  2. Navigate to AWS Batch under Compute.

  3. In the Job Definitions section, select the job definition for which you want to enable CloudWatch Logs.

  4. In the Logging section of the job definition, configure the Log Group and Log Stream to send job logs to CloudWatch.

  5. Make sure that CloudWatch Logs is enabled and that the correct IAM roles are assigned to allow access to CloudWatch.

  6. Save the job definition and ensure that the CloudWatch Logs are correctly configured and visible in the CloudWatch Logs Console.

Using AWS CLI:

  1. Run the following command to register a job definition with CloudWatch Logs enabled:

    aws batch register-job-definition --job-definition-name <job-definition-name> --type container --container-properties '{"image":"<container-image>","vcpus":1,"memory":1024,"command":["<your-command>"],"logConfiguration":{"logDriver":"awslogs","options":{"awslogs-group":"/aws/batch/job-logs","awslogs-stream-prefix":"batch"}}}'

  2. Verify the CloudWatch Logs configuration by running the following command to describe the job definition:

    aws batch describe-job-definitions --job-definition-name <job-definition-name>

Implementation Plan:

Using AWS Console:

  1. Log in to the AWS Console at AWS Console.

  2. Navigate to AWS Batch and go to Job Definitions.

  3. Select or create the job definition you want to modify.

  4. In the Logging section, enable CloudWatch Logs and configure the Log Group and Log Stream.

  5. Ensure that the correct IAM role with permission to access CloudWatch is assigned.

  6. Save and confirm that job logs are being sent to CloudWatch.

Using AWS CLI:

  1. Use the following command to register or update the job definition:

    aws batch register-job-definition --job-definition-name <job-definition-name> --type container --container-properties '{"image":"<container-image>","vcpus":1,"memory":1024,"command":["<your-command>"],"logConfiguration":{"logDriver":"awslogs","options":{"awslogs-group":"/aws/batch/job-logs","awslogs-stream-prefix":"batch"}}}'

  2. Check if the job definition is correctly updated by describing the job definition:

    aws batch describe-job-definitions --job-definition-name <job-definition-name>

Backout Plan:

Using AWS Console:

  1. If the job is not logging to CloudWatch as expected, log in to the AWS Console.

  2. Navigate to AWS Batch and check the Job Definitions section.

  3. Modify the job definition and ensure the correct CloudWatch Logs configuration is applied.

  4. Ensure that the IAM role has the necessary permissions to write to CloudWatch Logs.

Using AWS CLI:

  1. If the configuration needs to be reverted, update the job definition with the correct CloudWatch Logs settings:

    aws batch register-job-definition --job-definition-name <job-definition-name> --type container --container-properties '{"image":"<container-image>","vcpus":1,"memory":1024,"command":["<your-command>"],"logConfiguration":{"logDriver":"awslogs","options":{"awslogs-group":"/aws/batch/job-logs","awslogs-stream-prefix":"batch"}}}'


References:

CIS Controls:

Version

Control ID

Control Description

v8

3.4

Ensure AWS Batch jobs are configured to use CloudWatch Logs to track job execution and detect any issues.

v7

14.2

Implement centralized logging for batch jobs using CloudWatch Logs to monitor and audit job activity.