Profile Applicability:
- Level 1
Description:
AWS Step Functions is a service that enables you to coordinate multiple AWS services into serverless workflows. When executing workflows, it is crucial to capture logs to track the state transitions, identify errors, and understand the function execution flow. Logging for Step Functions is essential for operational monitoring, debugging, and ensuring compliance.
Enabling logging for Step Functions state machines ensures that execution history, including all state transitions and any failures, is captured and can be stored in Amazon CloudWatch Logs. This allows better visibility into workflow performance and issues, helping in operational troubleshooting and security auditing.
Rationale:
Security and Auditing: Capturing logs helps ensure that state transitions and sensitive operations are logged for compliance with security standards (e.g., SOC 2, PCI-DSS).
Troubleshooting: Logs are crucial for debugging and identifying failures in the workflow, allowing for faster issue resolution.
Monitoring: Logs help track the performance of state machines, ensuring that processes are running smoothly and efficiently.
Compliance: Many compliance frameworks require that cloud services have logging enabled for operational transparency and auditing purposes.
Impact:
Pros:
Improved Troubleshooting: Logging enables quick identification of failures or issues in the workflow.
Better Monitoring: Logs help you track the execution of state machines and provide insights into their performance.
Security and Compliance: Ensures compliance by maintaining an audit trail of all state transitions and actions in workflows.
Operational Visibility: Provides visibility into workflows and allows you to track the status of each execution step.
Cons:
Storage Costs: Storing logs in CloudWatch Logs may incur costs, depending on the volume of logs generated.
Increased Complexity: Configuring and managing log settings for each state machine adds complexity to the setup.
Default Value:
By default, AWS Step Functions state machines do not have logging enabled. You must explicitly enable logging and specify a CloudWatch Logs group to capture the execution logs.
Pre-requisite:
AWS IAM Permissions:
stepfunctions:DescribeStateMachine
stepfunctions:UpdateStateMachine
logs:CreateLogGroup
logs:CreateLogStream
logs:PutLogEvents
AWS CLI installed and configured.
CloudWatch Logs group must exist for the state machine logs to be stored.
Test Plan:
Using AWS Console:
Sign in to the AWS Management Console.
Navigate to Step Functions under Services.
In the Step Functions Dashboard, select the state machine you want to check.
Under the State Machine Details section, look for the Logging tab.
If Logging is disabled, you will see a prompt to enable it.
Click Edit and enable logging by selecting an existing CloudWatch Logs group.
Ensure that all transitions and any relevant information are being logged.
Save the changes to apply the configuration.
Using AWS CLI:
To check if logging is enabled for a state machine, run:
aws stepfunctions describe-state-machine --state-machine-arn <state-machine-arn> --query 'loggingConfiguration'
If the output shows that logging is not enabled, enable logging by running:
aws stepfunctions update-state-machine --state-machine-arn <state-machine-arn> --logging-configuration '{"level":"ALL","includeExecutionData":true,"destinations":[{"cloudWatchLogsLogGroup":{"logGroupArn":"arn:aws:logs:<region>:<account-id>:log-group:<log-group-name>"}}]}'
Verify the logging configuration is enabled by running the describe-state-machine command again:
aws stepfunctions describe-state-machine --state-machine-arn <state-machine-arn> --query 'loggingConfiguration'
Implementation Steps:
Using AWS Console:
Log in to the AWS Management Console and navigate to Step Functions.
Select the state machine that requires logging.
Under State Machine Details, click Edit.
Enable Logging by selecting an appropriate CloudWatch Logs group.
Choose the log level (e.g., ALL, ERROR).
Include Execution Data to log additional information about state transitions.
Save the changes.
Using AWS CLI:
To enable logging, run:
aws stepfunctions update-state-machine --state-machine-arn <state-machine-arn> --logging-configuration '{"level":"ALL","includeExecutionData":true,"destinations":[{"cloudWatchLogsLogGroup":{"logGroupArn":"arn:aws:logs:<region>:<account-id>:log-group:<log-group-name>"}}]}'
To verify the logging configuration, run:
aws stepfunctions describe-state-machine --state-machine-arn <state-machine-arn> --query 'loggingConfiguration'
Backout Plan:
Using AWS Console:
If enabling logging causes issues, sign in to the AWS Management Console.
Navigate to AWS Step Functions, select the state machine, and click Edit.
Disable logging or change the logging level to ERRORS only.
Save the changes and verify that logging is no longer being collected.
Using AWS CLI:
To disable logging for a Step Functions state machine, run the following command:
aws stepfunctions update-state-machine --state-machine-arn <STATE_MACHINE_ARN> --logging-configuration '{"level": "OFF"}'
Verify that logging has been disabled:
aws stepfunctions describe-state-machine --state-machine-arn <STATE_MACHINE_ARN>