Terraform State Management: Patterns for Teams

Terraform state is the source of truth for your infrastructure. Mismanage it, and you’ll have drift, conflicts, and 3 AM incidents. These patterns keep state safe and teams productive.

Why State Matters

Terraform state maps your configuration to real resources. Without it, Terraform can’t:

Know what already exists
Calculate what needs to change
Detect drift from desired state

Default local state (terraform.tfstate) breaks down quickly:

Can’t collaborate (who has the latest?)
No locking (concurrent runs = corruption)
No history (oops, we deleted production)

Remote Backend: S3 + DynamoDB

The standard pattern for AWS teams:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# backend.tf
terraform {
  backend "s3" {
    bucket         = "mycompany-terraform-state"
    key            = "prod/network/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"
  }
}

Create the Backend Resources

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
# bootstrap/main.tf - Run this once manually
provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "terraform_state" {
  bucket = "mycompany-terraform-state"
}

resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "aws:kms"
    }
  }
}

resource "aws_s3_bucket_public_access_block" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

resource "aws_dynamodb_table" "terraform_locks" {
  name         = "terraform-locks"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

State File Organization

By Environment and Component

Directory Structure to Match

Workspaces

Workspaces let you use the same config with different state files:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Create workspace
terraform workspace new staging

# List workspaces
terraform workspace list

# Switch workspace
terraform workspace select prod

# Current workspace
terraform workspace show

Use in configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
locals {
  environment = terraform.workspace
  
  instance_type = {
    dev     = "t3.micro"
    staging = "t3.small"
    prod    = "t3.large"
  }
}

resource "aws_instance" "app" {
  instance_type = local.instance_type[local.environment]
  
  tags = {
    Environment = local.environment
  }
}

When to Use Workspaces

Good for:

Same infrastructure, different sizes (dev/staging/prod)
Feature branches needing isolated environments
Quick environment duplication

Not good for:

Completely different architectures per environment
Different AWS accounts (use separate backends instead)
Long-lived, divergent environments

State Locking

DynamoDB provides locking for S3 backend:

1
2
3
4
5
6
7
8
terraform {
  backend "s3" {
    bucket         = "mycompany-terraform-state"
    key            = "prod/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-locks"  # Enables locking
  }
}

Force Unlock (Emergency)

1
2
# Only use when lock is stuck (crashed run, etc.)
terraform force-unlock LOCK_ID

State Operations

View State

1
2
3
4
5
6
7
8
# List all resources
terraform state list

# Show specific resource
terraform state show aws_instance.app

# Pull state to local file
terraform state pull > state.json

Move Resources

1
2
3
4
5
6
7
8
# Rename resource
terraform state mv aws_instance.old aws_instance.new

# Move to module
terraform state mv aws_instance.app module.compute.aws_instance.app

# Move between state files
terraform state mv -state-out=other.tfstate aws_instance.app aws_instance.app

Remove from State

1
2
# Remove without destroying (resource stays, Terraform forgets it)
terraform state rm aws_instance.app

Import Existing Resources

1
2
3
4
5
# Import existing resource into state
terraform import aws_instance.app i-1234567890abcdef0

# Import to module
terraform import module.compute.aws_instance.app i-1234567890abcdef0

State File Security

Sensitive Data

State contains sensitive values in plain text:

1
2
# See what's in state
terraform state pull | jq '.resources[].instances[].attributes | keys'

Protect state:

Enable S3 encryption (SSE-KMS)
Enable bucket versioning (recovery)
Restrict bucket access (IAM policies)
Never commit state to git

IAM Policy for State Access

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::mycompany-terraform-state/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::mycompany-terraform-state"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": "arn:aws:dynamodb:*:*:table/terraform-locks"
    }
  ]
}

Partial Configuration

Don’t hardcode backend config—pass at init:

1
2
3
4
5
6
# backend.tf
terraform {
  backend "s3" {
    # Configured via -backend-config
  }
}

1
2
3
4
5
6
# backend-prod.hcl
bucket         = "mycompany-terraform-state"
key            = "prod/terraform.tfstate"
region         = "us-east-1"
dynamodb_table = "terraform-locks"
encrypt        = true

1
terraform init -backend-config=backend-prod.hcl

Data Sources for Cross-State Reference

Reference outputs from other state files:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
# In compute/main.tf, reference network state
data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "mycompany-terraform-state"
    key    = "prod/network/terraform.tfstate"
    region = "us-east-1"
  }
}

resource "aws_instance" "app" {
  subnet_id = data.terraform_remote_state.network.outputs.private_subnet_ids[0]
  vpc_security_group_ids = [
    data.terraform_remote_state.network.outputs.app_security_group_id
  ]
}

In the network config, expose outputs:

1
2
3
4
5
6
7
8
# network/outputs.tf
output "private_subnet_ids" {
  value = aws_subnet.private[*].id
}

output "app_security_group_id" {
  value = aws_security_group.app.id
}

State Recovery

From S3 Versioning

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# List versions
aws s3api list-object-versions \
  --bucket mycompany-terraform-state \
  --prefix prod/terraform.tfstate

# Restore specific version
aws s3api get-object \
  --bucket mycompany-terraform-state \
  --key prod/terraform.tfstate \
  --version-id "abc123" \
  restored-state.tfstate

Recreate State

If state is lost but resources exist:

1
2
3
4
# Import each resource
terraform import aws_vpc.main vpc-12345
terraform import aws_subnet.public subnet-67890
# ... tedious but possible

CI/CD Integration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# GitHub Actions example
jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: hashicorp/setup-terraform@v3
      
      - name: Terraform Init
        run: terraform init -backend-config=backend-${{ env.ENVIRONMENT }}.hcl
        
      - name: Terraform Plan
        run: terraform plan -out=tfplan
        
      - name: Terraform Apply
        if: github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve tfplan

Best Practices

One state per component — Don’t put everything in one state file
Enable versioning — Always, for recovery
Enable locking — Always, for safety
Encrypt at rest — State contains secrets
Use remote state data sources — Not hardcoded IDs
Never edit state manually — Use terraform state commands
Plan before apply — Review changes, save plan file
Automate in CI/CD — Consistent, auditable applies

State management isn’t glamorous, but it’s foundational. Get it right once—remote backend, locking, encryption, versioning—and you’ll rarely think about it again.

Get it wrong, and you’ll think about it at 3 AM when someone ran terraform apply while you were mid-deploy.

Why State Matters#

Remote Backend: S3 + DynamoDB#

Create the Backend Resources#

State File Organization#

By Environment and Component#

Directory Structure to Match#

Workspaces#

When to Use Workspaces#

State Locking#

Force Unlock (Emergency)#

State Operations#

View State#

Move Resources#

Remove from State#

Import Existing Resources#

State File Security#

Sensitive Data#

IAM Policy for State Access#

Partial Configuration#

Data Sources for Cross-State Reference#

State Recovery#

From S3 Versioning#

Recreate State#

CI/CD Integration#

Best Practices#

📬 Get the Newsletter