Profile Applicability:
Level 1
Description:
Amazon Athena is an interactive query service that makes it easy to analyze data directly in Amazon S3 using SQL. Workgroups in Athena allow you to organize users, queries, and resources into isolated groups, providing access control, query tracking, and resource management.
Enabling logging for Athena workgroups ensures that detailed records of query activity, including executed queries, errors, and performance metrics, are captured. These logs are stored in Amazon CloudWatch Logs or Amazon S3, allowing for monitoring, auditing, and troubleshooting of queries run within the workgroup.
This SOP ensures that query logging is enabled for Amazon Athena Workgroups, helping with monitoring and compliance.
Rationale:
Enabling query logging for Amazon Athena workgroups is important for:
Auditability: Capturing query logs helps in auditing query execution and monitoring user activity.
Security: Logs can be used to identify suspicious or unauthorized queries, providing better insights into potential security risks.
Troubleshooting: Helps in troubleshooting failed queries or performance issues by providing detailed logs.
Compliance: Logging query activities helps meet regulatory and compliance standards that require tracking of user actions and data access.
Impact:
Pros:
Enhanced Monitoring: Logs provide visibility into query execution, making it easier to track performance and troubleshoot issues.
Security and Compliance: Helps meet compliance requirements by ensuring query activity is logged.
Audit Trail: Provides a detailed audit trail of who ran what query, when, and its outcome.
Cons:
Cost: Storing logs in CloudWatch Logs or S3 can incur additional costs, especially for large datasets or high-frequency queries.
Management Overhead: Requires configuration and periodic review to ensure the correct logging settings are applied.
Default Value:
By default, logging is not enabled for Amazon Athena workgroups. You need to explicitly configure logging to capture query activity, either in CloudWatch Logs or S3.
Pre-requisite:
AWS IAM Permissions:
athena:UpdateWorkgroup
athena:GetWorkgroup
logs:CreateLogGroup
logs:CreateLogStream
logs:PutLogEvents
s3:PutObject
AWS CLI installed and configured.
Basic knowledge of Amazon Athena, CloudWatch Logs, and S3.
Remediation:
Test Plan:
Using AWS Console:
Sign in to the AWS Management Console.
Navigate to Amazon Athena under Services.
In the Athena Console, go to Workgroups.
Select the workgroup for which you want to enable query logging.
In the Workgroup Details section, verify if query logging is enabled:
Logging should be directed to either CloudWatch Logs or S3.
If CloudWatch Logs is used, the Log Group should be specified.
If S3 is used, the S3 bucket path should be specified for storing query logs.
If logging is not enabled, modify the workgroup settings to enable logging.
Using AWS CLI:
To describe the Athena workgroup and check if query logging is enabled, run:
aws athena get-workgroup --workgroup <workgroup-name> --query 'WorkGroup.QueryResultsConfiguration'
Verify the output for ResultConfiguration to check if the log destination is configured.
If CloudWatch Logs is used, it will display the CloudWatch Log Group ARN.
If S3 is used, it will display the S3 bucket used for storing query logs.
Example output for CloudWatch Logs:
{
"WorkGroup": {
"QueryResultsConfiguration": {
"OutputLocation": "s3://your-athena-query-logs-bucket/",
"EncryptionConfiguration": {
"EncryptionOption": "SSE_S3"
},
"CloudWatchLogsEnabled": true,
"CloudWatchLogGroupArn": "arn:aws:logs:us-east-1:123456789012:log-group:/aws/athena/query-logs"
}
}
}Implementation Steps:
Using AWS Console:
Sign in to the AWS Management Console and navigate to Amazon Athena.
In the Athena Dashboard, go to Workgroups and select the workgroup you want to modify.
In the Workgroup Details section, find the Query Results Configuration.
To enable logging:
Set the Query Results Location to either CloudWatch Logs or S3.
For CloudWatch Logs, specify the CloudWatch Log Group ARN.
For S3, provide the S3 bucket path.
Save the changes to enable logging for query activity.
Using AWS CLI:
To enable CloudWatch Logs for the workgroup, run:
{
"WorkGroup": {
"QueryResultsConfiguration": {
"OutputLocation": "s3://your-athena-query-logs-bucket/",
"EncryptionConfiguration": {
"EncryptionOption": "SSE_S3"
},
"CloudWatchLogsEnabled": true,
"CloudWatchLogGroupArn": "arn:aws:logs:us-east-1:123456789012:log-group:/aws/athena/query-logs"
}
}
}To enable S3 logging for the workgroup, run:
aws athena update-workgroup \
--workgroup <workgroup-name> \
--configuration-updates '{"ResultConfigurationUpdates": {"OutputLocation": "s3://your-athena-query-logs-bucket/"}}Verify the update by running:
aws athena get-workgroup --workgroup <workgroup-name> --query 'WorkGroup.QueryResultsConfiguration'
Ensure that the CloudWatch Logs or S3 configuration is applied.
Backout Plan:
If enabling query logging causes issues (e.g., increased costs or performance degradation):
Identify the affected workgroup.
To disable logging to CloudWatch Logs, run:
aws athena update-workgroup \
--workgroup <workgroup-name> \
--configuration-updates '{"ResultConfigurationUpdates": {"CloudWatchLoggingOptions": {"Enabled": false}}}'To disable S3 logging, run:
aws athena update-workgroup \
--workgroup <workgroup-name> \
--configuration-updates '{"ResultConfigurationUpdates": {"OutputLocation": ""}}'Verify that logging has been disabled and the workgroup is functioning correctly.
Note:
Cost Considerations: Be aware that enabling logging to CloudWatch Logs or S3 will incur additional costs. For CloudWatch, you will be charged for data ingestion, storage, and log retrieval. For S3, charges will apply for data storage.
Log Retention: Set appropriate retention policies for CloudWatch Logs or S3 to manage costs and comply with data retention policies.
Access Control: Ensure that only authorized users and services have access to the CloudWatch Logs or S3 bucket where the query logs are stored.
