Aws

Terraform state is where reality meets code. Get it wrong, and you’ll destroy production infrastructure or spend hours untangling drift. Here’s how to manage state like a pro. What Is State? Terraform state (terraform.tfstate) maps your configuration to real-world resources: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 { "resources": [ { "type": "aws_instance", "name": "web", "instances": [ { "attributes": { "id": "i-0abc123def456", "ami": "ami-12345678", "instance_type": "t3.medium" } } ] } ] } Without state, Terraform doesn’t know what exists. It would try to create everything fresh every time. ...

Your AWS bill is too high. Everyone’s is. The cloud makes it trivially easy to spin up resources and surprisingly hard to know what you’re actually paying for. Here’s how to stop the bleeding. The Low-Hanging Fruit Unused Resources The easiest savings come from things you’re not using. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 # Find unattached EBS volumes aws ec2 describe-volumes \ --filters Name=status,Values=available \ --query 'Volumes[*].[VolumeId,Size,CreateTime]' \ --output table # Find unused Elastic IPs aws ec2 describe-addresses \ --query 'Addresses[?AssociationId==`null`].[PublicIp,AllocationId]' \ --output table # Find idle load balancers (no healthy targets) aws elbv2 describe-target-health \ --query 'TargetHealthDescriptions[?TargetHealth.State!=`healthy`]' Run these monthly. You’ll find forgotten resources every time. ...

Your cloud bill is too high. It always is. Here’s how to actually reduce it without breaking things. Quick Wins 1. Find Unused Resources 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # AWS: Find unattached EBS volumes aws ec2 describe-volumes \ --filters Name=status,Values=available \ --query 'Volumes[*].[VolumeId,Size,CreateTime]' \ --output table # Find old snapshots (>90 days) aws ec2 describe-snapshots --owner-ids self \ --query 'Snapshots[?StartTime<=`2026-01-01`].[SnapshotId,VolumeSize,StartTime]' \ --output table # Find unattached Elastic IPs aws ec2 describe-addresses \ --query 'Addresses[?AssociationId==`null`].[PublicIp,AllocationId]' \ --output table Delete them. Unattached EBS volumes cost money. Unused EIPs cost $3.65/month each. ...

You’re still running a bastion host, aren’t you? That t3.micro sitting in a public subnet, port 22 open to… well, hopefully not 0.0.0.0/0, but let’s be honest — it’s probably close. Stop it. AWS Systems Manager Session Manager exists, and it’s better in every way. The Bastion Problem Bastion hosts have been the standard for decades. Jump box in a public subnet, SSH through it to reach private instances. Simple enough. ...

Clicking through cloud consoles doesn’t scale. Terraform lets you define infrastructure in code, track changes in git, and deploy the same environment repeatedly. Core Concepts Provider: Plugin for a platform (AWS, GCP, Azure, etc.) Resource: A thing to create (server, database, DNS record) State: Terraform’s record of what exists Plan: Preview of changes before applying Apply: Make the changes happen Basic Workflow 1 2 3 4 terraform init # Download providers terraform plan # Preview changes terraform apply # Create/update resources terraform destroy # Tear everything down First Configuration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 # main.tf # Configure the AWS provider terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } } } provider "aws" { region = "us-east-1" } # Create an EC2 instance resource "aws_instance" "web" { ami = "ami-0c55b159cbfafe1f0" instance_type = "t3.micro" tags = { Name = "WebServer" } } 1 2 3 terraform init # Downloads AWS provider terraform plan # Shows: 1 to add terraform apply # Creates the instance Variables 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 # variables.tf variable "environment" { description = "Deployment environment" type = string default = "dev" } variable "instance_type" { description = "EC2 instance type" type = string default = "t3.micro" } variable "allowed_ips" { description = "IPs allowed to SSH" type = list(string) default = ["0.0.0.0/0"] } # main.tf - use variables resource "aws_instance" "web" { ami = "ami-0c55b159cbfafe1f0" instance_type = var.instance_type tags = { Name = "web-${var.environment}" Environment = var.environment } } Setting Variables 1 2 3 4 5 6 7 8 9 # Command line terraform apply -var="environment=prod" # File (terraform.tfvars) environment = "prod" instance_type = "t3.small" # Environment variables export TF_VAR_environment="prod" Outputs 1 2 3 4 5 6 7 8 9 10 # outputs.tf output "instance_ip" { description = "Public IP of the instance" value = aws_instance.web.public_ip } output "instance_id" { description = "Instance ID" value = aws_instance.web.id } After apply: ...

Hardcoded credentials in your repository are a security incident waiting to happen. One leaked .env file, one accidental commit, and your database is exposed to the internet. Let’s do secrets properly. The Basics What’s a Secret? Anything that grants access: Database passwords API keys OAuth tokens TLS certificates SSH keys Encryption keys Where Secrets Don’t Belong 1 2 3 # ❌ Never do this DATABASE_URL = "postgres://admin:supersecret123@db.prod.internal/myapp" AWS_SECRET_KEY = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" Also bad: .env files committed to git Docker image layers CI/CD logs Chat messages Wikis or documentation Secret Storage Options Environment Variables Simple, but limited: ...

Clicking through cloud consoles doesn’t scale. Infrastructure as Code (IaC) lets you version, review, and automate your infrastructure just like application code. Terraform has become the de facto standard. Here’s how to use it effectively. The Basics Terraform uses HCL (HashiCorp Configuration Language) to declare resources: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # main.tf terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } } } provider "aws" { region = "us-east-1" } resource "aws_instance" "web" { ami = "ami-0c55b159cbfafe1f0" instance_type = "t3.micro" tags = { Name = "web-server" } } 1 2 3 4 terraform init # Download providers terraform plan # Preview changes terraform apply # Create resources terraform destroy # Tear down everything State Management Terraform tracks what it created in a state file. Never lose this file. ...

The AWS Console is fine for exploration. For real work—auditing, automation, bulk operations—the CLI is essential. Here’s how to use it effectively. Output Formats 1 2 3 4 5 6 7 8 9 10 11 # JSON (default, best for scripting) aws ec2 describe-instances --output json # Table (human readable) aws ec2 describe-instances --output table # Text (tab-separated, grep-friendly) aws ec2 describe-instances --output text # YAML aws ec2 describe-instances --output yaml Set default in ~/.aws/config: ...

Local Terraform state works for learning. Production requires remote state—for team collaboration, state locking, and not losing your infrastructure when your laptop dies. Here’s how to set it up properly. Why Remote State? Local state (terraform.tfstate) has problems: No collaboration - Team members overwrite each other’s changes No locking - Concurrent applies corrupt state No backup - Laptop dies, state is gone, orphaned resources everywhere Secrets in plain text - State contains sensitive data Remote backends solve all of these. ...

The AWS CLI is the fastest path from question to answer. These patterns cover the operations you’ll use daily. Setup and Configuration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 # Configure default profile aws configure # Configure named profile aws configure --profile production # Use specific profile aws --profile production ec2 describe-instances # Or set environment variable export AWS_PROFILE=production # Verify identity aws sts get-caller-identity Multiple Accounts 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # ~/.aws/credentials [default] aws_access_key_id = AKIA... aws_secret_access_key = ... [production] aws_access_key_id = AKIA... aws_secret_access_key = ... # ~/.aws/config [default] region = us-east-1 output = json [profile production] region = us-west-2 output = json EC2 Operations List Instances 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # All instances aws ec2 describe-instances # Just the essentials aws ec2 describe-instances \ --query 'Reservations[].Instances[].[InstanceId,State.Name,InstanceType,PrivateIpAddress,Tags[?Key==`Name`].Value|[0]]' \ --output table # Running instances only aws ec2 describe-instances \ --filters "Name=instance-state-name,Values=running" \ --query 'Reservations[].Instances[].[InstanceId,PrivateIpAddress]' \ --output text # By tag aws ec2 describe-instances \ --filters "Name=tag:Environment,Values=production" # By instance ID aws ec2 describe-instances --instance-ids i-1234567890abcdef0 Start/Stop/Reboot 1 2 3 4 5 6 7 8 9 10 11 # Stop aws ec2 stop-instances --instance-ids i-1234567890abcdef0 # Start aws ec2 start-instances --instance-ids i-1234567890abcdef0 # Reboot aws ec2 reboot-instances --instance-ids i-1234567890abcdef0 # Terminate (careful!) aws ec2 terminate-instances --instance-ids i-1234567890abcdef0 Get Console Output 1 aws ec2 get-console-output --instance-id i-1234567890abcdef0 --output text SSH Key Pairs 1 2 3 4 5 6 7 8 9 # List aws ec2 describe-key-pairs # Create aws ec2 create-key-pair --key-name mykey --query 'KeyMaterial' --output text > mykey.pem chmod 400 mykey.pem # Delete aws ec2 delete-key-pair --key-name mykey S3 Operations List and Navigate 1 2 3 4 5 6 7 8 9 10 11 # List buckets aws s3 ls # List bucket contents aws s3 ls s3://mybucket/ # Recursive listing aws s3 ls s3://mybucket/ --recursive # With human-readable sizes aws s3 ls s3://mybucket/ --recursive --human-readable --summarize Copy and Sync 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Upload file aws s3 cp myfile.txt s3://mybucket/ # Download file aws s3 cp s3://mybucket/myfile.txt ./ # Upload directory aws s3 cp ./mydir s3://mybucket/mydir --recursive # Sync (only changed files) aws s3 sync ./local s3://mybucket/remote # Sync with delete (mirror) aws s3 sync ./local s3://mybucket/remote --delete # Exclude patterns aws s3 sync ./local s3://mybucket/remote --exclude "*.log" --exclude ".git/*" Delete 1 2 3 4 5 6 7 8 # Single file aws s3 rm s3://mybucket/myfile.txt # Directory aws s3 rm s3://mybucket/mydir/ --recursive # Empty bucket aws s3 rm s3://mybucket/ --recursive Presigned URLs 1 2 3 4 5 # Generate download URL (expires in 1 hour) aws s3 presign s3://mybucket/myfile.txt --expires-in 3600 # Upload URL aws s3 presign s3://mybucket/upload.txt --expires-in 3600 Bucket Operations 1 2 3 4 5 6 7 8 # Create bucket aws s3 mb s3://mynewbucket # Delete bucket (must be empty) aws s3 rb s3://mybucket # Force delete (removes contents first) aws s3 rb s3://mybucket --force IAM Operations Users 1 2 3 4 5 6 7 8 9 10 11 # List users aws iam list-users # Create user aws iam create-user --user-name newuser # Delete user aws iam delete-user --user-name olduser # List user's access keys aws iam list-access-keys --user-name myuser Roles 1 2 3 4 5 6 7 8 # List roles aws iam list-roles # Get role details aws iam get-role --role-name MyRole # List attached policies aws iam list-attached-role-policies --role-name MyRole Policies 1 2 3 4 5 6 7 # List policies aws iam list-policies --scope Local # Get policy document aws iam get-policy-version \ --policy-arn arn:aws:iam::123456789012:policy/MyPolicy \ --version-id v1 CloudWatch Logs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # List log groups aws logs describe-log-groups # List log streams aws logs describe-log-streams --log-group-name /aws/lambda/myfunction # Get recent logs aws logs get-log-events \ --log-group-name /aws/lambda/myfunction \ --log-stream-name '2024/01/01/[$LATEST]abc123' \ --limit 100 # Tail logs (requires aws-cli v2) aws logs tail /aws/lambda/myfunction --follow # Filter logs aws logs filter-log-events \ --log-group-name /aws/lambda/myfunction \ --filter-pattern "ERROR" \ --start-time $(date -d '1 hour ago' +%s)000 Lambda 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # List functions aws lambda list-functions # Invoke function aws lambda invoke \ --function-name myfunction \ --payload '{"key": "value"}' \ output.json # Get function config aws lambda get-function-configuration --function-name myfunction # Update function code aws lambda update-function-code \ --function-name myfunction \ --zip-file fileb://function.zip # View recent invocations aws lambda get-function --function-name myfunction RDS 1 2 3 4 5 6 7 8 9 10 11 12 13 14 # List instances aws rds describe-db-instances # Instance details aws rds describe-db-instances --db-instance-identifier mydb \ --query 'DBInstances[0].[DBInstanceIdentifier,DBInstanceStatus,Endpoint.Address]' # Create snapshot aws rds create-db-snapshot \ --db-instance-identifier mydb \ --db-snapshot-identifier mydb-snapshot-$(date +%Y%m%d) # List snapshots aws rds describe-db-snapshots --db-instance-identifier mydb Secrets Manager 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # List secrets aws secretsmanager list-secrets # Get secret value aws secretsmanager get-secret-value --secret-id mysecret \ --query 'SecretString' --output text # Create secret aws secretsmanager create-secret \ --name mysecret \ --secret-string '{"username":"admin","password":"secret"}' # Update secret aws secretsmanager put-secret-value \ --secret-id mysecret \ --secret-string '{"username":"admin","password":"newsecret"}' SSM Parameter Store 1 2 3 4 5 6 7 8 9 10 11 12 # Get parameter aws ssm get-parameter --name /myapp/database/password --with-decryption # Put parameter aws ssm put-parameter \ --name /myapp/database/password \ --value "mysecret" \ --type SecureString \ --overwrite # List parameters by path aws ssm get-parameters-by-path --path /myapp/ --recursive --with-decryption Query and Filter Patterns JMESPath Queries 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Select specific fields aws ec2 describe-instances \ --query 'Reservations[].Instances[].[InstanceId,State.Name]' # Filter in query aws ec2 describe-instances \ --query 'Reservations[].Instances[?State.Name==`running`].[InstanceId]' # First result only aws ec2 describe-instances \ --query 'Reservations[0].Instances[0].InstanceId' # Flatten nested arrays aws ec2 describe-instances \ --query 'Reservations[].Instances[].Tags[?Key==`Name`].Value[]' Output Formats 1 2 3 4 5 6 7 8 9 10 11 # JSON (default) aws ec2 describe-instances --output json # Table (human readable) aws ec2 describe-instances --output table # Text (tab-separated, good for scripts) aws ec2 describe-instances --output text # YAML aws ec2 describe-instances --output yaml Scripting Patterns Loop Through Resources 1 2 3 4 5 6 7 8 # Stop all instances with specific tag for id in $(aws ec2 describe-instances \ --filters "Name=tag:Environment,Values=dev" \ --query 'Reservations[].Instances[].InstanceId' \ --output text); do echo "Stopping $id" aws ec2 stop-instances --instance-ids "$id" done Wait for State 1 2 3 4 5 6 7 8 # Wait for instance to be running aws ec2 wait instance-running --instance-ids i-1234567890abcdef0 # Wait for instance to stop aws ec2 wait instance-stopped --instance-ids i-1234567890abcdef0 # Wait for snapshot completion aws ec2 wait snapshot-completed --snapshot-ids snap-1234567890abcdef0 Pagination 1 2 3 4 5 6 7 # Auto-pagination (default in CLI v2) aws s3api list-objects-v2 --bucket mybucket # Manual pagination aws s3api list-objects-v2 --bucket mybucket --max-items 100 # Use NextToken from output for next page aws s3api list-objects-v2 --bucket mybucket --starting-token "token..." Useful Aliases 1 2 3 4 5 6 7 8 9 10 # ~/.bashrc or ~/.zshrc # Quick instance list alias ec2ls='aws ec2 describe-instances --query "Reservations[].Instances[].[InstanceId,State.Name,InstanceType,PrivateIpAddress,Tags[?Key==\`Name\`].Value|[0]]" --output table' # Who am I? alias awswho='aws sts get-caller-identity' # S3 bucket sizes alias s3sizes='aws s3 ls | while read _ _ bucket; do aws s3 ls s3://$bucket --recursive --summarize 2>/dev/null | tail -1; echo $bucket; done' Troubleshooting 1 2 3 4 5 6 7 8 9 10 11 # Debug mode aws ec2 describe-instances --debug # Dry run (check permissions without executing) aws ec2 run-instances --dry-run --image-id ami-12345 --instance-type t2.micro # Check CLI version aws --version # Clear credential cache rm -rf ~/.aws/cli/cache/* The AWS CLI rewards muscle memory. Start with the operations you do daily, build aliases for common patterns, and gradually expand. ...