Description:

CloudWatch is a monitoring and management service that provides data and actionable insights for AWS, hybrid, and on-premises applications and infrastructure resources. Using this service you can collect and access all your performance and operational data in form of logs and metrics from a single platform.

When performing any activities on Application and infrastructure resources it generates lots of operational and monitoring data in form of logs and metrics. On the Amazon CloudWatch correlate metrics and logs through the visualize data sets in a single platform. So, We can quickly diagnose the problem to understanding the root cause.


Rational:

Real-time monitoring of IAM policy change can be achieved by directing CloudTrail Logs to CloudWatch Logs and establishing corresponding metric filters and alarms. It is recommended that a metric filter and alarm be established for IAM policy change.


Impact:

The preceding rules together provide clear insight into  IAM policy change and will notify you if it breaks the rule.


Default value:

By default, Metric Filter for specific CloudTrail log events and a CloudWatch alarm for the usage of  IAM policy change will not exist.


Audit:

  1. Log in to the AWS Management Console and go to Cloudwatch dashboard at https://console.aws.amazon.com/cloudwatch/
  2. In the left navigation panel click on Log groups and select the log group you want to examine
  3. Go to the metric filter tab and Search for the metric filter related to IAM policy change

If the particular metric filter is not found it means IAM policy changes are not monitored by cloudwatch.


Via CLI:

To describe the Cloudwatch metrics

aws cloudwatch describe-metric-filters --log-group-name <value>


Remediation:

Pre-Requisite:

  • Cloud trails Must be enabled in your AWS account

  • You must contain SNS Topic to get notification

Implementation Steps:

  1. Log in to the AWS Management Console and go to Cloudwatch dashboard at https://console.aws.amazon.com/cloudwatch/

  2. In the left navigation, pane click on Log groups under Logs

  3. Select the log group for which you want to create a metric filter 

  4. Click on the Actions drop-down menu, select create a metric filter

  5. In the Create metric filter page, we define the filter pattern as {($.eventName = DeleteGroupPolicy) || ($.eventName = DeleteRolePolicy) || ($.eventName = DeleteUserPolicy) || ($.eventName = PutGroupPolicy) || ($.eventName = PutRolePolicy) || ($.eventName = PutUserPolicy) || ($.eventName = CreatePolicy) || ($.eventName = DeletePolicy) || ($.eventName = CreatePolicyVersion) || ($.eventName = DeletePolicyVersion) || ($.eventName = AttachRolePolicy) || ($.eventName = DetachRolePolicy) || ($.eventName = AttachUserPolicy) || ($.eventName = DetachUserPolicy) || ($.eventName = AttachGroupPolicy) || ($.eventName = DetachGroupPolicy) }and then click on Next button

  6. On Assign metric page, under the Create filter name give the filter name as IAM_policy_change

  7. Under the Metric Details section, provide the details required like Metric namespace, Metric name, metric value, and Default value(optional), and then click on the Next button 

  8. Review the details provided and click on Create metric filter button to create it

  9. After clicking on create metric filter You will see a metric filters page, Select the metric filter you have created and click on create alarm on the right side of the Metric filters section.

  10. After clicking on create alarm you will be redirected to the next tab to Create Alarm, define the following:-

    • Metric name (it automatically takes from the metric filter)

    • Statistic select Sum

    • Period as 5minutes

  11. In the conditions section select the Threshold type to Static, Define the alarm condition with a Greater(>) threshold and give the value of threshold as you defined in metric value then click on the Next button

  12. Next is Configure actions in this choose Alarm state trigger option as In alarm, For SNS topic select SNS if exist or can create a new topic, click on NEXT

  13. Give the Name of Alarm e.g. ApiCallMonitoring and Alarm description(optional), click on the Next button.

  14. Review all the entered details and click on create Alarm


Via CLI:

To create a metric filter

put-metric-filter
--log-group-name <value>
--filter-name <value>
--filter-pattern <value>
--metric-transformations <value>

To create Alarm

aws cloudwatch put-metric-alarm 
--alarm-name <value>
--metric-name <value> 
--statistic Sum --period 300 --threshold <value>
--comparison-operator <value>
--evaluation-periods 1 -namespace '<give the name space>' 
--alarm-actions <sns_topic_arn>


Backout plan: 

  1. Log in to the AWS Management Console and go to Cloudwatch dashboard at https://console.aws.amazon.com/cloudwatch/

  2. In the left navigation, pane click on Log groups under Logs

  3. Select the log group you want to modify, go to metric filter tab

  4. Choose the metric filter you want to delete and click on Delete

  5. Click on In Alarms in the left navigation pane

  6. Select the alarm you want to delete, click on the Actions drop-down, and select Delete


Reference:

Amazon CloudWatch concepts - Amazon CloudWatch