Efficiently Managing S3 Buckets: Implementing Lifecycle Rules for S3 Bucket

Introduction :-

In today’s cloud-driven world, efficient #data management is paramount. One key aspect of this is managing the lifecycle of objects stored in #Amazon S3 buckets. This document explores the process of adding S3 bucket lifecycle rules specifically tailored to development (dev) and production (prod) environments. #AWS S3 Bucket Lifecycle rules allow you to automate the transition and deletion of objects over time, reducing storage costs, and ensuring compliance. In this guide, we’ll walk through the steps to set up these rules for dev and prod environments in an AWS S3 bucket.

Pre-requestites :-

• Aws account with required permissions.

• #Aws cli should be installed and the credentials(access & secret keys) should be configured.

• #Terraform and #Terragrunt should be installed on your local system.

Step-1 :- First Login to your Aws management console and create a s3 bucket and dynamo-db table to storing the lock files. Next, Create a directory named aws-s3 in your home directory with-in the aws-s3 directory, create a file named terragrunt.hcl. Update the s3 bucket name and dynamo db table name in the main terragrunt.hcl file as shown below.

remote_state {
  backend = "s3"
  config = {
    bucket         = "YOUR-TERRAGRUNT-S3-BUCKET-NAME"
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "YOUR-DYNAMODB-TABLE"
  }
 generate = {
    path      = "backend.tf"
    if_exists = "overwrite_terragrunt"
  }
}

Step-2 :- Create a directory named aws-s3 in your home directory and within the aws-s3 directory create one more folder named life-cycle-rule. Create #Terragrunt configuration file (terragrunt.hcl) in your life-cycle-rule folder.

Step-3 :- Open the terragrunt.hcl file using visual editor and paste the below code into the file.

terraform {
  source = "tfr:///terraform-aws-modules/s3-bucket/aws?version=3.4.0"
}

include "root" {
  path = find_in_parent_folders()
}

locals {
  region      = "us-east-1"
}

inputs = {
  bucket = "prospect-uploads"

    lifecycle_rule = [
    {
      id      = "ChangeToGlacier"
      enabled = true

      transition = [
        {
          days          = 365
          storage_class = "GLACIER"
        }
      ]  
    }
  ]
}

Step-4 :- Open a command prompt or terminal window and locate to your aws-s3 directory .

Step-5 :- Now configure your aws credentials and run #Terragrunt commands.

terragrunt init

• Once the initialization is done , Run #Terragrunt apply command to deploy your load balancer configuration.

terragrunt apply

Step-6 :- After the deployment is done, Go a head and Login to your #AWS Management Console.

Step-7 :- Navigate to #S3 service and open your s3 bucket. As u can see that the life cycle rule has been configured for your s3 bucket like below.

Conclusion :-

Managing the lifecycle of objects in your S3 buckets is a crucial aspect of optimizing storage and ensuring data compliance. By adding specific S3 bucket lifecycle rules for your development and production environments, you can streamline your data management processes, reduce storage costs, and improve overall efficiency.Remember, these rules are customizable to suit your organization’s unique needs. As your data requirements evolve, so too can your #S3 bucket lifecycle rules. Stay agile, keep your data organized, and make the most of your AWS resources. Happy optimizing!