Infrastructure

Terraform state is where infrastructure-as-code meets reality. It’s also where most Terraform disasters originate. Here’s how to manage state without losing sleep. The Problem Terraform tracks what it’s created in a state file. This file maps your HCL resources to real infrastructure. Without it, Terraform can’t update or destroy anything — it doesn’t know what exists. The default is a local file called terraform.tfstate. This works fine until: Someone else needs to run Terraform Your laptop dies Two people run apply simultaneously You accidentally commit secrets to Git Rule 1: Remote State from Day One Never use local state for anything beyond experiments: ...

GitOps sounds simple: put your infrastructure in Git, let a controller sync it to your cluster. In practice, there are a dozen ways to get it wrong. Here’s what works. The Core Principle Git is the source of truth. Not the cluster. Not a dashboard. Not someone’s kubectl session. D e v e l o p e r s → i n G g i l t ↑ e → s o C u o r n c t e r o l l e r → C l u s t e r If the cluster state doesn’t match Git, the controller fixes it. If someone manually changes the cluster, the controller reverts it. This is the contract. ...

“We’re deploying, please hold” is not an acceptable user experience. Whether you’re running a startup or enterprise infrastructure, users expect services to just work. Here’s how to ship code without the maintenance windows. The Goal: Invisible Deploys A zero-downtime deployment means users never notice you’re deploying. No error pages, no dropped connections, no “please refresh” messages. The old version serves traffic until the new version is proven healthy. Strategy 1: Rolling Deployments The simplest approach. Replace instances one at a time: ...

The promise of Ansible is simple: describe your desired state, run the playbook, and the system converges to that state. Run it again, nothing changes. That’s idempotency—and it’s harder to achieve than it sounds. Here’s how to write playbooks that won’t surprise you on the second run. The Problem: Commands That Lie The command and shell modules are where idempotency goes to die: 1 2 3 # ❌ BAD: Always reports "changed", even when nothing changed - name: Create database command: createdb myapp This fails on the second run because the database already exists. Worse, it always shows as “changed” even when it shouldn’t run at all. ...

When you deploy an LLM-powered agent in production, traditional APM dashboards only tell half the story. You can track latency, error rates, and throughput — but what about what the agent actually did? Did it hallucinate? Did it spiral into an infinite retry loop? Did it spend $47 on tokens chasing a dead end? Here’s how to build observability for autonomous agents that actually helps. The Three Pillars of Agent Observability Standard observability (logs, metrics, traces) still matters. But agents need three additional dimensions: ...

Git isn’t just for code anymore. In a GitOps workflow, your entire infrastructure lives in version control, and changes happen through pull requests, not SSH sessions. The principle is simple: the desired state of your system is declared in Git, and automated processes continuously reconcile actual state with desired state. No more “just SSH in and fix it.” No more tribal knowledge about what’s running where. The Core Loop GitOps operates on a continuous reconciliation loop: ...

Raw logs are noise. Processed telemetry is intelligence. The difference between them is your observability pipeline. Modern distributed systems generate enormous amounts of data—logs, metrics, traces, events. But data isn’t insight. The challenge isn’t collection; it’s transformation. How do you turn a firehose of JSON lines into something a human (or an AI) can actually act on? The Three Pillars, Unified You’ve heard the “three pillars of observability”: logs, metrics, and traces. What’s often missing from that conversation is how these pillars should connect. ...

Ansible playbooks can quickly become unwieldy spaghetti. Here are battle-tested patterns for writing infrastructure code that scales with your team and your infrastructure. The Role Structure That Actually Works Forget the minimal examples. Real roles need this structure: r └ o ─ l ─ e s w ├ │ ├ │ ├ │ │ │ │ ├ │ ├ │ ├ │ └ / e ─ ─ ─ ─ ─ ─ ─ b ─ ─ ─ ─ ─ ─ ─ s e d └ v └ t ├ ├ ├ └ h └ t └ f └ m └ r e ─ a ─ a ─ ─ ─ ─ a ─ e ─ i ─ e ─ v f ─ r ─ s ─ ─ ─ ─ n ─ m ─ l ─ t ─ e a s k d p e a r u m / m s m i c s l m l n s s / m / l a a / a n o e e a a g / s a t i i i s n r r i t i l i s n n n t f v s n e n - n / . . . a i i / . s x p . y y y l g c y / . a y m m m l u e m c r m l l l . r . l o a l y e y n m m . m f s l y l . . m j c l 2 o n # # # # # # # f # D R E P C S R D e o n a o e e e f l t c n r s p a e r k f v t e u y a i i a n l v g g c r d t a p e u e t e r o r / n v i i i a m r c a a n n t a e i r b t s i n l e i l t o a o s a e - a n g a b s l e d l j l f m e ( u a i e h s h s t l n a i t i e t n ( g o s d l h i n l o e n e w r c r e l s s p u t r d e e p c s r e e d c e e n d c e e n ) c e ) The key insight: tasks/main.yml should only contain includes: ...

You test your application code. Why not your infrastructure code? Infrastructure as Code (IaC) has the same failure modes as any software: bugs, regressions, unintended side effects. Yet most teams treat Terraform and Ansible like configuration files rather than code that deserves tests. Why Infrastructure Testing Matters A Terraform plan looks correct until it: Creates a security group that’s too permissive Deploys to the wrong availability zone Sets instance types that exceed your budget Breaks networking in ways that only manifest at runtime Manual review catches some issues. Automated testing catches more. ...

GitHub Actions has become the de facto CI/CD platform for many teams, but most only scratch the surface with pre-built actions from the marketplace. Building custom actions tailored to your infrastructure needs can dramatically reduce boilerplate and enforce consistency across repositories. Why Custom Actions? Every DevOps team has workflows that repeat across projects: Deploying to specific cloud environments Running security scans with custom policies Provisioning temporary environments for PR reviews Rotating secrets on a schedule Instead of copy-pasting YAML across repositories, custom actions encapsulate this logic once and reference it everywhere. ...