First steps with Terraform in AWS

Terraform is a cloud-agnostic provisioning tool created by Hashicorp. It allows you to manage your infrastructure in a sane, safe, and efficient manner by automating the provisioning of your cloud resources (server, databases, DNS) in a declarative way, as well as leverage version control systems to keep track of the history of changes.

In this post, we are going to go over how to set up Terraform to work with AWS. If you want to go straight to the code, then check it out at aws-terraform-examples/01-first-steps-with-terraform-in-aws.

Installing terraform

There are different ways to install terraform depending on you operating system

Chocolatey on Windows

choco install terraform

Homebrew on OS X

Using the new hashicorp tap (recommended)

brew install hashicorp/tap/terraform

Using the community tap

brew install terraform

Linux

Installing on Linux dependent on the distribution you are running. For Ubuntu/Debian:

1. Add the HashiCorp gpg key

curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -

2. Add the HashiCorp repository

sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"

3. Update and install

sudo apt-get update && sudo apt-get install terraform

For more information and different ways to install, check the installation pages

Commands overview

To view a list of the available commands, just run terraform and it will print its usage information. To show the help for a particular subcommand, just append -h/--help at the end. Some of the most common commands are:

• terraform plan: It is used to create an execution plan. This allows Terraform to determine what actions are necessary to achieve the desired state without making any changes to real resources or the state.

• terraform apply: It is used to apply the changes required to reach the desired state of the configuration.

• terraform import: It is used to import existing resources created outside of Terraform into it.

For a full rundown of the Terraform commands, check the Terraform commands docs

Connecting AWS with terraform

Now that we have the CLI installed, let’s get started connecting AWS with Terraform to manage the infrastructure.

Prerequisites

To execute the rest of the project, you’ll need to follow the next steps or use your existing configured AWS account.

1. An AWS account (a dedicated new one to execute terraform preferably, although it is OK to use your own AWS credentials for testing purposes)

2. The AWS CLI installed.

3. Configure your local aws cli with a dedicated terraform [profile

aws configure --profile terraform

Using the above command, input at the prompt your AWS Access Key ID and Secret Key

Defining the AWS provider

According to the Terraform documentation, a provider is essentially a plugin that offers a set of abstraction for certain API resources and their interactions. Usually, each provider focuses on a specific infrastructure platform.

Terraform needs to know which provider to download from the Terraform Registry. For that, we can use the terraform block to list all the providers that the code will use. For our scenario, we can list the aws provider as shown below:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
  required_version = ">= 0.13"
}

Now that we listed the aws provider, let’s add the configuration data for the AWS provider. Below, we are using the us-east-2 region and loading the credentials to connect to AWS from the terraform profile we created on the Prerequisites section.

provider "aws" {
  region = "us-east-2"
  profile = "terraform"
}

For alternative means of authentication, dive in the docs and always make sure that you don’t store your AWS credentials in plaintext on Terraform.

At this point, you are ready to use Terraform to automate your infrastructure. In the next section, we’ll dive into storing the backend information remotely to facilitate team workflows.

S3 for remote backend

First, I’ll briefly mention some of the advantages of using a remote backend:

• Improves working in a team since the state can be protected with locks to avoid multiple interactions at the same time and possible corruptions.
• It helps keep sensitive data off disk. Terraform stores the state in plaintext and if using a backend it will only be stored on the backend location (e.g. S3 bucket)
• If the backend supports remote operations, it means that terraform apply can be executed on the backend instead of your local machine, for an overall improved experience.

Before going through the code, let’s briefly look at what Terraform state entails.

Terraform state

Terraform stores the state of your infrastructure for several reasons:

• Mapping to the Real World: Terraform maps each configuration to a corresponding resource in the real world and it leverages the state to verify that no two configurations elements represent the same endpoint
• Metadata: It includes tracking knowledge such as resource dependencies or workflows that are necessary for resources to work as expected.
• Performance: Optionally, the state can be considered as a source of truth and therefore API requests to the providers are done only when specified. This helps improve performance for large teams.

This state is stored in a file, usually called terraform.tfstate using a JSON format. Let’s move on to the implementation

Implementation

To set up the remote backend, we need the following resources:

1. A S3 bucket where the terraform state will be stored
2. A DynamoDB table to lock the access to the state file
3. IAM policies to grant the user access to the S3 bucket and the DynamoDB table (required if you are using an IAM user with more restrictive credentials than just AdministratorAccess access)

S3 bucket

resource "aws_s3_bucket" "terraform_state_storage" {
  bucket = "terraform-remote-state-storage-s3"
  acl    = "private"

  tags = {
    name      = "Terraform Storage"
    dedicated = "infra"
  }

  versioning {
    enabled = true
  }

  lifecycle {
    prevent_destroy = true
  }
}

The S3 bucket, as shown above, gets created with the following settings:

• Private Access List Control (ACL) grant to limit access to the bucket.
• Versioning enabled to allow for state recovery in case of accidental deletion and file corruption
• Prevention of accidental destruction of the S3 bucket while running terraform operations

DynamoDB table

# create a dynamodb table for locking the state file.
# this is important when sharing the same state file across users
resource "aws_dynamodb_table" "terraform_state_lock" {
  name           = "terraform-state-lock"
  hash_key       = "LockID"
  read_capacity  = 20
  write_capacity = 20

  attribute {
    name = "LockID"
    type = "S"
  }

  tags = {
    name = "DynamoDB Terraform State Lock Table"
    dedicated = "infra"
  }

  lifecycle {
    prevent_destroy = true
  }
}

The DynamoDB table gets configured with the following properties:

• A LockID hash key of type string, so that all items created by terraform operations are stored together in the same bucket
• The read and write capacity per seconds for the table. This specifies how read/write operations are we allowed to execute against the table.
• Similar to the S3 bucket above, the resource is created with prevention of accidental destruction while running Terraform operations.

IAM policies

data "aws_iam_policy_document" "terraform_storage_state_access" {
  statement  {
    effect = "Allow"

    actions = [
      "s3:ListBucket"
    ]

    resources = [
      aws_s3_bucket.terraform_state_storage.arn
    ]
  }

  statement {
    effect = "Allow"

    actions = [
      "s3:GetObject",
      "s3:PutObject"
    ]

    resources = [
      "${aws_s3_bucket.terraform_state_storage.arn}/terraform.tfstate"
    ]
  }
}

# Creates the IAM policy to allow access to the bucket
resource "aws_iam_policy" "terraform_storage_state_access" {
  name = "terraform_storage_state_access"
  policy = data.aws_iam_policy_document.terraform_storage_state_access.json
}

# Assigns the policy to the terraform user
resource "aws_iam_user_policy_attachment" "terraform_storage_state_attachment" {
  user       = "terraform"
  policy_arn = aws_iam_policy.terraform_storage_state_access.arn
}

The above IAM policy describes the permission for a user to access the S3 bucket:

• Ability to list the bucket
• Get objects from the bucket
• Add new objects to it

As part of the setup, the policy is attached to the Terraform user, so it can have access to the S3 bucket.

data "aws_iam_policy_document" "dynamodb_access" {
  statement {
    effect = "Allow"

    actions = [
      "dynamodb:GetItem",
      "dynamodb:PutItem",
      "dynamodb:DeleteItem"
    ]

    resources = [
      "arn:aws:dynamodb:*:*:table/terraform-state-lock"
    ]
  }
}

# Creates the IAM policy to allow access to the dynamoDB
resource "aws_iam_policy" "dynamodb_access" {
  name = "dynamodb_access"
  policy = data.aws_iam_policy_document.dynamodb_access.json
}

# Assigns the policy to the terraform user
resource "aws_iam_user_policy_attachment" "dynamodb_attachment" {
  user       = local.terraform_user
  policy_arn = aws_iam_policy.dynamodb_access.arn
}

Similar to the previous IAM policy, this IAM policy describes the permission given to the user when accessing the DynamoDB table:

• Get items from the table
• Add items to the table
• Remove items from the table

The policy is then attached to the terraform user, so it can have access to the DynamoDB table.

Remote backend

Finally, to inform terraform to use the s3 bucket as remote backend, just add the following block:

terraform {
  backend "s3" {
    bucket = "terraform-state-storage"
    key    = "terraform.tfstate"
    region = "us-east-2"
    dynamodb_table = "terraform-state-lock"
    profile = "terraform"
  }
}

Resources

Below, it is a condensed list of all the resources mentioned throughout the posts as well as a few others I consider may be of interest to deepen your knowledge.

Providers:

• Providers Introduction
• Provider Requirements
• Terraform Registry
• AWS Provider Docs

Terraform remote storage:

• Terraform State
• S3 backend
• Remote State
• https://medium.com/@jessgreb01/how-to-terraform-locking-state-in-s3-2dc9a5665cb6

S3:

• Terraform S3 resource
• S3 AWS docs

DynamoDB:

• Terraform DynamoDB resource
• DynamoDB AWS docs

IAM:

• Terraform IAM policy
• Terraform IAM policy attachment
• Terraform IAM policy document
• IAM AWS docs

Conclusions

To sum up, this post provided an intro to Terraform in AWS. It covered how to install Terraform and configure it to manage AWS resources. As further steps, we went through the steps of configuring S3 and DynamoDB for managing the Terraform state remotely with locking included, which is a common setup, especially if working on a team.

Thank you so much for reading this post. Hope you enjoyed reading it as much as I did writing it. See you soon and stay tuned for more!!