Description

Ensure that AWS CloudTrail events are sent to Amazon CloudWatch Logs and that the trail's log group has a metric filter with an associated CloudWatch alarm for AWS KMS customer managed keys being disabled or scheduled for deletion. It matches the DisableKey and ScheduleKeyDeletion events from AWS KMS. The alarm notifies responders whenever a matching event is recorded.

Rationale

Data encrypted with a KMS key cannot be decrypted while the key is disabled, and is lost permanently once the key is deleted. Without monitoring, an attacker or a mistake can disable a key or schedule its deletion without anyone noticing, causing outages, unusable backups and irreversible data loss. An alarm on these events gives the security team time to cancel a scheduled deletion during its waiting period. KMS key administration should also be limited to authorized administrators and made through change control.

Impact

  • Provides near real-time alerts when a KMS key is disabled or scheduled for deletion

  • Allows a scheduled key deletion to be cancelled before the waiting period ends

  • No effect on running workloads; the filter and alarm only read CloudTrail events

  • Adds a small cost for the CloudWatch alarm, and for CloudWatch Logs ingestion if CloudTrail was not already sending events to CloudWatch Logs

Default Value

No metric filters or CloudWatch alarms exist by default. A CloudTrail trail does not send events to CloudWatch Logs unless this is configured on the trail.

Pre-Requisites

  • IAM permissions: cloudtrail:DescribeTrails, cloudtrail:UpdateTrail, logs:DescribeMetricFilters, logs:PutMetricFilter, cloudwatch:DescribeAlarmsForMetric, cloudwatch:PutMetricAlarm, plus logs:DeleteMetricFilter and cloudwatch:DeleteAlarms for the backout plan

  • A CloudTrail trail, preferably multi-region, that sends its events to a CloudWatch Logs log group

  • An Amazon SNS topic with confirmed subscribers (for example, the security team's email address) to receive the alarm notifications

Remediation

Test Plan

Console:

  1. Sign in to the AWS Management Console at https://console.aws.amazon.com.

  2. Open the AWS CloudTrail console and select Trails.

  3. Open each trail and, in the CloudWatch Logs section, note the log group the trail sends events to.

  4. Open the Amazon CloudWatch console in the region of that log group, choose Log groups and select the log group.

  5. Open the Metric filters tab and check for a filter whose pattern includes the event source kms.amazonaws.com and all of these events: DisableKey and ScheduleKeyDeletion.

  6. Note the metric name of that filter, then choose Alarms, All alarms and check that an alarm exists for that metric.

  7. If no trail's log group has such a metric filter with an alarm, follow the implementation steps.

CLI:

List the trails and the CloudWatch Logs log group each one sends events to:

aws cloudtrail describe-trails --region <REGION> --query "trailList[].[Name,HomeRegion,IsMultiRegionTrail,CloudWatchLogsLogGroupArn]" --output table

List the metric filters of the log group (the name after log-group: in the ARN), in the region of the log group:

aws logs describe-metric-filters --region <REGION> --log-group-name <LOG_GROUP_NAME> --query "metricFilters[].[filterName,filterPattern,metricTransformations[0].metricName,metricTransformations[0].metricNamespace]" --output table

List the alarms on the metric of the matching filter:

aws cloudwatch describe-alarms-for-metric --region <REGION> --metric-name <METRIC_NAME> --namespace <METRIC_NAMESPACE> --query "MetricAlarms[].[AlarmName,StateValue,ActionsEnabled]" --output table

The account fails if no trail sends events to a log group, if no metric filter on a trail's log group has a pattern that includes $.eventSource = kms.amazonaws.com and every event listed above, or if no alarm exists for the metric of that filter.

Implementation Steps

Console:

  1. Sign in to the AWS Management Console at https://console.aws.amazon.com.

  2. If no trail sends events to CloudWatch Logs, open the AWS CloudTrail console, open a multi-region trail, choose Edit in the CloudWatch Logs section, enable it, select or create a log group and an IAM role, and save the changes.

  3. Open the Amazon CloudWatch console in the region of the trail's log group, choose Log groups and select the log group.

  4. Open the Metric filters tab and choose Create metric filter.

  5. In Filter pattern, enter {($.eventSource = kms.amazonaws.com) && (($.eventName=DisableKey)||($.eventName=ScheduleKeyDeletion)) } and choose Next.

  6. Enter the metric details and choose Next:

    • Filter name: disable_or_delete_cmk_changes

    • Metric namespace: CISBenchmark

    • Metric name: disable_or_delete_cmk_changes_metric

    • Metric value: 1

  7. Choose Create metric filter.

  8. On the Metric filters tab, select the new filter and choose Create alarm.

  9. Set the metric and conditions, then choose Next:

    • Statistic: Sum

    • Period: 5 minutes

    • Threshold type: Static, Greater/Equal, 1

  10. Under Notification, select the SNS topic that alerts the security team and choose Next.

  11. Enter the alarm name disable_or_delete_cmk_changes_alarm, choose Next, then Create alarm.

Backout Plan

  1. Sign in to the AWS Management Console at https://console.aws.amazon.com.

  2. Open the Amazon CloudWatch console, choose Alarms, All alarms, select the disable_or_delete_cmk_changes_alarm alarm and delete it.

  3. Choose Log groups, select the trail's log group, open the Metric filters tab, select the disable_or_delete_cmk_changes filter and delete it.

CLI:

aws cloudwatch delete-alarms --region <REGION> --alarm-names disable_or_delete_cmk_changes_alarm
aws logs delete-metric-filter --region <REGION> --log-group-name <LOG_GROUP_NAME> --filter-name disable_or_delete_cmk_changes

Note: removing the metric filter and alarm stops alerts for AWS KMS customer managed keys being disabled or scheduled for deletion, so these events can again go unnoticed.

Reference