Description

Ensure that every AWS Lambda function is covered by a data event selector on an AWS CloudTrail trail, so that its invocations are recorded. Lambda Invoke calls are data events, which CloudTrail does not log by default. A trail must have an event selector for the resource type AWS::Lambda::Function, either for all functions or for the specific function.

Rationale

Without Lambda data events, there is no record of who invoked a function, when and from where. An attacker or misused automation could run function code without leaving an audit trail, which hides data exfiltration or unauthorized changes and makes incident investigation much harder. Logging Lambda data events in CloudTrail, with central storage, suitable retention and alerts on unusual invocations, provides this accountability.

Impact

  • Records every Lambda Invoke call with the caller identity, time and source

  • Supports incident investigation and detection of unusual invocations

  • CloudTrail charges for data events, and functions that are invoked often can generate a large volume of log records and storage costs

Default Value

CloudTrail trails log management events only. Data events, including Lambda Invoke calls, are not logged unless they are explicitly added to a trail.

Pre-Requisites

  • IAM permissions: cloudtrail:DescribeTrails, cloudtrail:GetEventSelectors and cloudtrail:PutEventSelectors

  • An existing trail that delivers logs to an Amazon S3 bucket (preferably a multi-region trail)

  • An estimate of the additional CloudTrail data event and storage costs

Remediation

Test Plan

Console:

  1. Sign in to the AWS Management Console at https://console.aws.amazon.com.

  2. Open the CloudTrail console and select Trails in the left menu.

  3. Select each trail and review the Data events section.

  4. Check that at least one trail logs data events for the Lambda resource type, for all functions or for the function being checked.

  5. If no trail logs Lambda data events for the function, follow the implementation steps.

CLI:

List the trails:

aws cloudtrail describe-trails --region <REGION> --query "trailList[].[Name,HomeRegion,IsMultiRegionTrail]" --output table

Show the event selectors of a trail:

aws cloudtrail get-event-selectors --region <REGION> --trail-name <TRAIL_NAME>

A function fails if no trail has either an advanced event selector with resources.type equal to AWS::Lambda::Function, or a basic event selector with a DataResources entry of type AWS::Lambda::Function whose Values contain the function ARN or arn:aws:lambda (all functions).

Implementation Steps

Console:

  1. Sign in to the AWS Management Console at https://console.aws.amazon.com.

  2. Open the CloudTrail console and select Trails in the left menu.

  3. Select the trail, and in the Data events section choose Edit.

  4. Select Data events if it is not already selected.

  5. Set Resource type to Lambda and Log selector template to Log all events.

  6. Choose Save changes.

  7. If the trail is not a multi-region trail, repeat for each trail in each region in use.

CLI:

The example below keeps management events logged. Without the management events selector, the command would replace all existing selectors and stop management event logging on the trail. If the trail has other selectors, add them to the file as well. Save the following as selectors.json:

[{"Name":"Management events","FieldSelectors":[{"Field":"eventCategory","Equals":["Management"]}]},{"Name":"Lambda data events","FieldSelectors":[{"Field":"eventCategory","Equals":["Data"]},{"Field":"resources.type","Equals":["AWS::Lambda::Function"]}]}]

Apply the event selectors to the trail:

aws cloudtrail put-event-selectors --region <REGION> --trail-name <TRAIL_NAME> --advanced-event-selectors file://selectors.json

Backout Plan

Before making changes, save the current event selectors:

aws cloudtrail get-event-selectors --region <REGION> --trail-name <TRAIL_NAME> > event-selectors-backup.json

To roll back:

  1. Sign in to the AWS Management Console at https://console.aws.amazon.com.

  2. Open the CloudTrail console and select Trails in the left menu.

  3. Select the trail, and in the Data events section choose Edit.

  4. Remove the Lambda data event selector, or restore the selectors from the saved backup.

  5. Choose Save changes.

CLI:

The put-event-selectors command replaces all selectors on the trail, so restore the exact selectors from the saved backup rather than writing new ones. Copy the AdvancedEventSelectors array from event-selectors-backup.json into restore-selectors.json (or copy all selectors except the Lambda one, to remove only the Lambda selector), then apply it:

aws cloudtrail put-event-selectors --region <REGION> --trail-name <TRAIL_NAME> --advanced-event-selectors file://restore-selectors.json

If the backup contains an EventSelectors array instead (basic event selectors), copy that array into restore-event-selectors.json and apply it:

aws cloudtrail put-event-selectors --region <REGION> --trail-name <TRAIL_NAME> --event-selectors file://restore-event-selectors.json

Note: rolling back stops the recording of Lambda invocations, so they can no longer be audited.

Reference