Profile Applicability:

  •  Level 2

Description:

Amazon FSx for Lustre requires EC2 instances to have a compatible kernel to interact with the Lustre file system. Ensuring that the EC2 kernel is compatible is essential for performance, reliability, and functionality when using Lustre for high-performance computing (HPC) or other large-scale data workloads.

Rationale:

Ensuring EC2 kernel compatibility with Lustre ensures:

  • Successful integration between EC2 instances and FSx for Lustre

  • Avoidance of errors or performance degradation due to kernel incompatibility

  • Full functionality of Lustre features, including high throughput and low latency for storage operations

Default Value:

Most Amazon Linux 2 and certain Ubuntu instances are compatible with Lustre by default. However, kernel updates or custom kernels may need to be validated for compatibility.

Impact:

Pros:
 • Ensures optimal performance and functionality when using Lustre
• Prevents kernel-related issues during data processing and I/O operations
 • Supports high-performance, scalable workloads

Cons:
 • Requires manual validation and potential kernel updates for compatibility
• Kernel incompatibility could lead to issues such as crashes, performance degradation, or inability to mount the Lustre file system
 • May involve downtime for instance reboots or reconfiguration

Pre-requisites:

IAM Permissions Required:
 ec2:DescribeInstances, ec2:DescribeImages, ec2:ModifyInstanceAttribute
 Access to EC2 instances and the ability to install or update kernel packages

Remediation:

Test Plan:

Using AWS Console:
 • Log in to the AWS Management Console
 • Navigate to EC2 > Instances
 • Select the EC2 instance you wish to use with Lustre
 • Check the Operating System and Kernel version (Lustre has compatibility requirements)
• Ensure that the EC2 instance type is supported by Lustre and that the correct kernel version is running
• For Amazon Linux, ensure the instance is using a version of the kernel compatible with Lustre
 • Run a test by mounting the Lustre file system on the instance and checking for any errors

Using AWS CLI:

aws ec2 describe-instances \
  --instance-id i-xxxxxxxxxxxxxxxxx
ssh ec2-user@<instance-ip> "uname -r"

Implementation Plan:

Using AWS Console:
 • Navigate to EC2 > Instances and select the appropriate EC2 instance
• For Amazon Linux 2, ensure that the instance is running the latest version of the kernel compatible with Lustre (Amazon Linux 2 typically supports Lustre out of the box)
• If using a custom AMI or another Linux distribution, verify that the required Lustre client kernel modules are installed and that the kernel version is compatible
 • For Ubuntu or other distributions, install the Lustre client and verify kernel compatibility

Using AWS CLI:
 Step 1: Check the EC2 instance's kernel version

ssh ec2-user@<instance-ip> "uname -r"

Step 2: Check if the instance uses an Amazon Linux AMI or if the required kernel modules are installed

ssh ec2-user@<instance-ip> "modinfo lustre"

Step 3: If the kernel is incompatible, update the kernel or launch a new EC2 instance with a supported kernel
 For Amazon Linux 2:

sudo yum update kernel

Step 4: Install Lustre client on the EC2 instance

sudo yum install lustre-client

Step 5: Confirm kernel compatibility with Lustre

uname -r

Backout Plan:

Using AWS Console:
 • If kernel compatibility issues are identified, roll back to a previously known working instance or kernel version
• Revert any kernel updates or configuration changes made during the process
 • Terminate the EC2 instance and launch a new one with a compatible kernel, if necessary

Using AWS CLI:
 To revert a kernel update or restore a compatible kernel:

sudo yum downgrade kernel

To launch a new EC2 instance with a supported AMI and kernel version:

aws ec2 run-instances --image-id ami-xxxxxxxxxxxxxxxxx --instance-type t3.medium

References: