Profile Applicability:

  • Level 2

Description:

AWS Elastic File Cache is a fully managed, scalable, high-performance file system cache for applications that require low-latency file access. FSx for Lustre is used as the backend for storing and accessing data. By creating and managing FSx Elastic File Cache, you can accelerate applications that need fast and frequent access to large datasets, such as those used in machine learning, data analytics, and high-performance computing (HPC).

Rationale:

Creating and configuring an FSx Elastic File Cache ensures:

  • Low-latency file access and high throughput for data processing applications

  • Seamless integration between FSx for Lustre and Amazon Elastic File Cache

  • Efficient caching of frequently accessed data to reduce latency and offload backend storage

  • Scalable and cost-effective caching for large-scale file systems

Default Value:

Elastic File Cache is not automatically created. It must be manually configured and linked with a backend FSx for Lustre file system to provide caching capabilities.

Impact:

Pros:
 • Enhances performance by providing low-latency caching for data-intensive workloads
• Reduces pressure on backend storage by caching frequently accessed files
• Simplifies management with fully managed caching infrastructure
 • Seamless integration with FSx for Lustre for high-performance workloads

Cons:
 • Requires proper configuration and monitoring to avoid unnecessary caching costs
• Misconfigured cache settings may cause data inconsistency or application errors
 • Extra costs associated with caching and managing data across multiple systems

Pre-requisites:

IAM Permissions Required:
 fsx:DescribeFileSystems, elasticfilesystem:CreateFileCache, elasticfilesystem:DescribeFileCache, elasticfilesystem:DeleteFileCache
 Permissions to create, configure, and manage FSx file systems and Elastic File Cache

Remediation:

Test Plan:

Using AWS Console:

  1. Log in to the AWS Management Console
  2.  Navigate to FSx > File systems
  3. Ensure that an FSx for Lustre file system is already created and properly configured
  4. Go to Elastic File Cache and create a new cache
  5. Configure the cache by selecting the FSx for Lustre file system as the backend storage
  6. Set the cache size, network configuration, and other parameters
  7. Confirm the cache creation and monitor it in the Elastic File Cache Dashboard
  8. Test the cache by accessing and modifying data through the cache from an EC2 instance or application

Using AWS CLI:

aws fsx describe-file-systems \
  --file-system-id fs-xxxxxxxxxxxxxxxxx
aws efs create-file-cache \
  --file-cache-id <cache-id> \
  --resource-id <resource-id> \
  --network-configuration file://network-config.json
aws efs describe-file-caches

Implementation Plan

Using AWS Console:

  1. Navigate to FSx > File systems and select your FSx for Lustre file system
  2. In the Elastic File Cache section, click Create Cache
  3. Configure the cache by selecting the FSx file system to link with, specifying cache size, and network configuration
  4. Monitor the cache creation progress through the Elastic File Cache Dashboard
  5. Once the cache is created, mount it on an EC2 instance for file access and testing

Using AWS CLI:
 Step 1: Create the Elastic File Cache

aws efs create-file-cache \
  --file-cache-id <cache-id> \
  --resource-id <resource-id> \
  --network-configuration file://network-config.json

Step 2: Verify the cache creation

aws efs describe-file-caches

Step 3: Mount the cache to an EC2 instance

sudo mount -t efs <cache-id>:/ /mnt/efs

Step 4: Test cache functionality by reading and writing files

echo "test data" > /mnt/efs/testfile.txt
cat /mnt/efs/testfile.txt

Backout Plan:

Using AWS Console:

  1. If there are issues, delete the file cache via the Elastic File Cache section in the AWS Management Console
  2. Revert network and backend file system settings as required
  3. Ensure that any data stored in the cache is backed up before removal

Using AWS CLI:
 To delete the file cache:

aws efs delete-file-cache \
  --file-cache-id <cache-id>

To unmount the file system cache:

sudo umount /mnt/efs

References: