Profile Applicability:

  • Level 2

Description:

The Lustre client is a software component that enables EC2 instances to interact with Amazon FSx for Lustre file systems. Ensuring that the Lustre client is installed and configured correctly allows EC2 instances to mount, read, and write to FSx for Lustre file systems. Proper installation and configuration are critical for high-performance computing and data-intensive workloads.

Rationale:

Ensuring that the Lustre client is installed and properly configured:

  • Enables EC2 instances to access data on Lustre file systems

  • Optimizes performance for workloads that require high throughput

  • Ensures compatibility between EC2 instances and FSx for Lustre

  • Minimizes errors during data processing and file system operations

Default Value:

Lustre client is not pre-installed on EC2 instances. It must be manually installed and configured to work with FSx for Lustre.

Impact:

Pros:
 • Provides high-performance storage for EC2 instances interacting with FSx for Lustre
• Ensures compatibility between Lustre and other AWS services for data processing
 • Optimizes access to large-scale datasets used in analytics and HPC

Cons:
 • Requires manual installation and configuration of the client on EC2 instances
• Kernel and OS compatibility must be verified for proper functionality
 • Misconfiguration can lead to failed mounts or performance issues

Pre-requisites:

IAM Permissions Required:
 ec2:DescribeInstances, ec2:RunInstances, fsx:DescribeFileSystems, fsx:DescribeMountTargets
 EC2 instance access with administrative privileges to install packages and configure the client

Remediation:

Test Plan:

Using AWS Console:
 • Log in to the AWS Management Console
 • Navigate to FSx > File systems
 • Verify that FSx for Lustre is correctly set up and accessible
 • Ensure that the mount targets are available for EC2 instances to connect to
• Confirm that the Lustre client is installed and running on the EC2 instances by checking system logs and performing file system operations
 • Ensure EC2 instances can successfully mount the FSx for Lustre file system and perform basic I/O operations

Using AWS CLI:

aws fsx describe-file-systems \
  --file-system-id fs-xxxxxxxxxxxxxxxxx
aws fsx describe-mount-targets \
  --file-system-id fs-xxxxxxxxxxxxxxxxx
ssh ec2-user@<instance-ip> "modinfo lustre"

Implementation Plan:

Using AWS Console:
 • Navigate to FSx > File systems and select the FSx for Lustre file system
 • Go to Mount targets and retrieve the mount target IP address for the file system
• SSH into the EC2 instance where Lustre will be mounted
 • Install the Lustre client on Amazon Linux 2 (or the appropriate AMI)

Run the following commands to install the Lustre client:

sudo yum install lustre-client
sudo yum install kernel-devel

• Mount the Lustre file system on the EC2 instance using the mount target IP

sudo mount -t lustre <mount-target-ip>:/fsx /mnt/fsx

• Verify that the Lustre file system is mounted and accessible by running basic file system commands

Using AWS CLI:
 Step 1: Install the Lustre client on the EC2 instance (for Amazon Linux 2):

sudo yum install lustre-client

Step 2: Verify the installation:

modinfo lustre

Step 3: Mount the Lustre file system:

sudo mount -t lustre <mount-target-ip>:/fsx /mnt/fsx

Step 4: Verify that the file system is mounted:

df -h

Step 5: Ensure that Lustre client is properly configured and running

sudo systemctl status lustre

Backout Plan:

Using AWS Console:
 • If there are issues with the Lustre client, remove it and reinstall
• Disconnect any incorrectly mounted Lustre file systems
 • Revert any changes made to the instance configuration or the mount target settings

Using AWS CLI:
 To unmount the file system:

sudo umount /mnt/fsx

To reinstall the Lustre client:

sudo yum remove lustre-client
sudo yum install lustre-client

References: