Profile Applicability:
Level 2
Description:
Amazon RDS (Relational Database Service) supports integration with Amazon CloudWatch Logs to capture database logs for various database engines such as MySQL, PostgreSQL, MariaDB, and Oracle. Integrating RDS with CloudWatch Logs provides better visibility into database performance, query execution, errors, and other important operational data. This integration allows you to monitor, store, and analyze logs for troubleshooting, auditing, and compliance purposes.
This SOP ensures that RDS clusters are properly configured to integrate with CloudWatch Logs to capture database logs for ongoing monitoring and analysis.
Rationale:
Security and Auditing: CloudWatch Logs provide a centralized place for auditing RDS database activity, helping with compliance and security monitoring.
Performance Monitoring: CloudWatch Logs allow for continuous tracking of database performance, enabling timely detection of anomalies or performance bottlenecks.
Troubleshooting: Integration with CloudWatch Logs enables faster identification of issues, such as query performance problems or errors, leading to quicker resolutions.
Operational Visibility: Logs provide operational insights and can be used for generating alerts or reports that help maintain service levels.
Impact:
Pros:
Improved Troubleshooting: Easy access to database logs allows for faster identification and resolution of issues.
Enhanced Monitoring: Provides a centralized view of the RDS instance's performance and activities.
Security and Compliance: Helps ensure that database activities are logged for auditing and security purposes.
Alerting: Integration with CloudWatch can allow for the setting of alarms based on specific log events.
Cons:
Storage Costs: Storing large amounts of logs in CloudWatch can incur additional costs depending on log volume and retention policies.
Management Overhead: Initial setup and ongoing management of log integration may require attention to ensure proper configuration and data retention.
Default Value:
By default, RDS clusters are not integrated with CloudWatch Logs unless explicitly configured. You need to enable the logging configuration when creating or modifying the DB instance or cluster to send logs to CloudWatch.
Pre-requisite:
AWS IAM Permissions:
rds:DescribeDBClusters
rds:ModifyDBCluster
logs:CreateLogGroup
logs:CreateLogStream
logs:PutLogEvents
AWS CLI installed and configured.
Ensure CloudWatch Logs group exists for RDS logs.
The RDS DB engine must support CloudWatch Logs integration (e.g., MySQL, PostgreSQL, MariaDB, Oracle).
Test Plan:
Using AWS Console:
Sign in to the AWS Management Console.
Navigate to Amazon RDS under Services.
In the RDS Dashboard, go to Databases and select the DB cluster you want to check for CloudWatch Logs integration.
Under the Configuration tab, check the Log exports section:
Ensure that CloudWatch Logs is listed as a log export option.
If CloudWatch Logs is not listed, the DB cluster is not integrated with CloudWatch Logs.
If not enabled, click Modify, and under the Log Exports section, select Enable CloudWatch Logs export for the desired log categories (e.g., error logs, general logs, slow query logs).
Save the changes and verify that logs are now being exported to CloudWatch Logs.
Using AWS CLI:
To check if RDS CloudWatch Logs integration is enabled, run:
aws rds describe-db-clusters --query 'DBClusters[*].{DBClusterIdentifier:DBClusterIdentifier,EnabledCloudWatchLogsExports:EnabledCloudWatchLogsExports}'
Review the output to ensure that CloudWatch Logs exports are enabled for the required log categories (e.g., error-log, general-log).
To enable CloudWatch Logs integration for a DB cluster, run:
aws rds modify-db-cluster --db-cluster-identifier <cluster-id> --cloudwatch-logs-export-configuration "ErrorLog,GeneralLog,SlowQueryLog" --apply-immediately
Verify the changes using:
aws rds describe-db-clusters --query 'DBClusters[*].{DBClusterIdentifier:DBClusterIdentifier,EnabledCloudWatchLogsExports:EnabledCloudWatchLogsExports}'
Implementation Steps:
Using AWS Console:
Log in to the AWS Management Console and navigate to Amazon RDS.
Select the RDS DB cluster for which you want to enable CloudWatch Logs.
In the Modify DB Cluster screen, locate the Log Exports section.
Select the log types (such as Error Logs, General Logs, or Slow Query Logs) you want to export to CloudWatch Logs.
Click Continue and then Apply Changes.
Using AWS CLI:
To enable CloudWatch Logs export for the RDS cluster, run:
aws rds modify-db-cluster --db-cluster-identifier <cluster-id> --cloudwatch-logs-export-configuration "ErrorLog,GeneralLog,SlowQueryLog" --apply-immediately
To verify that CloudWatch Logs integration is enabled, run:
aws rds describe-db-clusters --query 'DBClusters[*].{DBClusterIdentifier:DBClusterIdentifier,EnabledCloudWatchLogsExports:EnabledCloudWatchLogsExports}'
Backout Plan:
Using AWS Console:
If enabling CloudWatch Logs integration causes issues, sign in to the AWS Management Console.
Navigate to Amazon RDS, select the DB cluster, and click Modify.
Disable CloudWatch Logs integration by unselecting the log types.
Save the changes, and logs will no longer be exported to CloudWatch Logs.
Using AWS CLI:
To disable CloudWatch Logs export for an RDS cluster, run the following command:
aws rds modify-db-cluster --db-cluster-identifier <CLUSTER_ID> --cloudwatch-logs-export-configuration '{}' --apply-immediately
Verify that CloudWatch Logs integration has been disabled:
aws rds describe-db-clusters --db-cluster-identifier <CLUSTER_ID>