AWS Services:
Amazon Simple Storage Service (S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as data lakes, websites, mobile applications, backup and restore, archive, enterprise applications. S3 is designed for 99.999999999% (11 9's) of durability, and stores data for millions of applications for companies all around the world.
Description:
An S3 Lifecycle configuration is a set of rules that define actions that Amazon S3 applies to a group of objects. There are two types of actions:
Transition actions - It is when objects transition to another S3 storage class. For example, you might choose to transition objects to the S3 Standard-IA storage class 30 days after creating them or archive objects to the S3 Glacier storage class one year after creating them.
Expiration actions - It is when objects expire, S3 deletes expired objects on your behalf.
The lifecycle expiration costs depend on when you choose to expire the objects.
Implementation Plan:
Pre Requisites:
Have a clear cloud backup strategy:
How frequently data is accessed
Number of days for keeping data
Decommissioning plan
Detailed Steps:
“Rule applies to all objects in the bucket“ - Understanding with an Example:
Sign in to the AWS Management Console
Go to the Amazon S3 console at https://console.aws.amazon.com/s3/
In the bucket list available click on the bucket name you want to create lifecycle rules
Select the Management tab and click on create lifecycle rule
In the Lifecycle rule name, enter a name for your rule
Choose the scope of the lifecycle rule:
To apply this lifecycle rule to all objects with a specific prefix or tag, choose to Limit the scope to specific prefixes or tags.
To limit the scope by prefix, in Prefix, enter the prefix.
To limit the scope by tag, choose Add tag, and enter the tag key and value.
To apply this lifecycle rule to all objects in the bucket, choose This rule applies to all objects in the bucket, and choose I acknowledge that this rule applies to all objects in the bucket.
- Under Lifecycle rule actions, choose the actions that you want your lifecycle rule to perform:
Transition current versions of objects between storage classes
Transition previous versions of objects between storage classes
Expire current versions of objects
Permanently delete previous versions of objects
Delete expired delete markers or incomplete multipart uploads
Depending on the actions that you choose, different options appear.
Choose Create rule.
“Rule applies to all objects in the bucket“ - Understanding with an Example:
Let’s assume a backup scenario that
After 90 days I want to move data to the glacier
After 1 year I want to move it to the deep glacier
And after 10 years I want to delete it
We will see how we can proceed to create a lifecycle according to the above rules
Sign in to the AWS Management Console
Go to the Amazon S3 console at https://console.aws.amazon.com/s3/
In the bucket list available click on the bucket name you want to create lifecycle rules
Select the Management tab and click on create lifecycle rule
- In the Lifecycle rule name, enter a name for your rule.
- Choose the scope of the lifecycle rule:
- To apply this lifecycle rule to all objects with a specific prefix or tag, choose to Limit the scope to specific prefixes or tags.
To limit the scope by prefix, in Prefix, enter the prefix.
To limit the scope by tag, choose Add tag, and enter the tag key and value.
To apply this lifecycle rule to all objects in the bucket, choose This rule applies to all objects in the bucket, and choose I acknowledge that this rule applies to all objects in the bucket.
Note: Here we are selecting This rule applies to all objects in the bucket.
- To apply this lifecycle rule to all objects with a specific prefix or tag, choose to Limit the scope to specific prefixes or tags.
- Under Lifecycle rule actions, choose the actions that you want your lifecycle rule to perform:
- Select Transition current versions of objects between storage classes
Note: Here I am selecting the current version you can also select the previous version or both or any option available as per your requirement. - Select Glacier in Storage class transitions and enter 90 in Days after object creation column
- Click on Add transaction and select Glacier Deep Archive in Storage class transitions and enter 365 in Days after object creation column and acknowledge.
- Select Expire current versions of objects under Lifecycle rule actions
- Enter 3650(i.e, 10 years) in the Number of days after object creation option under Expire current versions of objects
- Select Transition current versions of objects between storage classes
- Review all the options selected in the summary shown at the bottom and click on Create rule
References:
Managing your storage lifecycle - Amazon Simple Storage Service