Kubernetes

Your pod is stuck in CrashLoopBackOff. Kubernetes keeps restarting it, each time waiting longer before trying again. Here’s how to diagnose and fix it. What CrashLoopBackOff Actually Means CrashLoopBackOff isn’t the error itself — it’s Kubernetes telling you “this container keeps crashing, so I’m backing off on restarts.” The actual problem is that your container exits with a non-zero exit code. Kubernetes notices, restarts it, it crashes again, and the backoff timer increases: 10s, 20s, 40s, up to 5 minutes. ...

Kubernetes networking confuses everyone at first. Pods, Services, Ingresses, CNIs — it’s a lot. Here’s how it actually works, layer by layer. The Fundamental Model Kubernetes networking has three simple rules: Every Pod gets its own IP address Pods can communicate with any other Pod without NAT Agents on a node can communicate with all Pods on that node That’s it. Everything else is implementation detail. Layer 1: Pod Networking Each Pod gets an IP from the cluster’s Pod CIDR (e.g., 10.244.0.0/16). This isn’t magic — your CNI plugin handles it. ...

Containers aren’t inherently secure. They share a kernel with the host. A container escape is a host compromise. Here’s how to not be the cautionary tale. Image Security Use Minimal Base Images Every package is attack surface. Minimize it. 1 2 3 4 5 6 7 8 # Bad: Full OS with thousands of packages FROM ubuntu:22.04 # Better: Minimal OS FROM alpine:3.19 # Best: Distroless (no shell, no package manager) FROM gcr.io/distroless/static-debian12 Distroless images contain only your app and runtime dependencies. No shell means attackers can’t get a shell. ...

GitOps sounds simple: Git is the source of truth, automation syncs it to reality. In practice, most teams get it wrong. Here’s how to get it right. The Core Principle GitOps isn’t “we use Git.” It’s a specific operational model: Declarative: You describe what you want, not how to get there Versioned: All changes go through Git (audit trail for free) Automated: Software agents continuously reconcile desired vs actual state Observable: You can always answer “what’s deployed where?” The magic is in reconciliation. Traditional CI/CD pushes changes. GitOps pulls desired state and converges toward it. The system heals itself. ...

Service mesh is either the solution to all your microservices problems or unnecessary complexity you don’t need. Here’s how to tell which. What a Service Mesh Does A service mesh handles cross-cutting concerns for service-to-service communication: Traffic management — Load balancing, routing, retries Security — mTLS, authorization policies Observability — Metrics, tracing, logging Resilience — Circuit breakers, timeouts, fault injection Instead of implementing these in every service, the mesh handles them at the infrastructure layer. ...

Hardcoded IPs are a maintenance nightmare. Here’s how to let services find each other dynamically. The Problem 1 2 3 4 5 6 7 # Bad: Hardcoded api_url = "http://192.168.1.50:8080" # What happens when: # - IP changes? # - Service moves to new host? # - You add a second instance? Service discovery solves this: services register themselves, and clients look them up by name. DNS-Based Discovery The simplest approach: use DNS. ...

Push to Git, watch your cluster update. That’s the GitOps promise. Here’s how to actually implement it. What GitOps Is GitOps means: Git is the source of truth for infrastructure and application state Changes happen through Git (PRs, not kubectl apply) A controller watches Git and reconciles cluster state Drift is automatically corrected The cluster converges to match what’s in Git, continuously. Why GitOps Over kubectl apply 1 2 3 4 5 6 # Bad: Who ran this? When? From where? kubectl apply -f deployment.yaml # Good: PR reviewed, approved, merged, tracked forever git commit -m "Scale API to 5 replicas" git push Over CI-Push Traditional CI/CD pushes to the cluster: ...

Your pod won’t start. The service isn’t routing. Something’s wrong but kubectl isn’t telling you what. Here’s how to actually debug Kubernetes problems. The Debugging Hierarchy Work from the outside in: Cluster level — Is the cluster healthy? Node level — Are nodes ready? Pod level — Is the pod running? Container level — Is the container healthy? Application level — Is the app working? Most problems are at levels 3-5. Start there. ...

Your service is running. Is it healthy? Can it handle requests? These are different questions with different answers. Kubernetes formalized this distinction with liveness and readiness probes. Even if you’re not on Kubernetes, the concepts matter everywhere. The Distinction Liveness: Is the process alive and not stuck? If NO → Restart the process Checks for: deadlocks, infinite loops, crashed but not exited Readiness: Can this instance handle traffic right now? ...

Kubernetes failures are rarely mysterious once you know where to look. The problem is knowing where to look. This guide covers the systematic approach to diagnosing common Kubernetes issues. The Diagnostic Hierarchy Start broad, drill down: C l u s t e r → N o d e → P o d → C o n t a i n e r → A p p l i c a t i o n At each level, the same questions apply: ...