Reliability

Your API will be called wrong. Clients will send garbage. Load will spike unexpectedly. Authentication will be misconfigured. The question isn’t whether these things happen — it’s whether your API degrades gracefully or explodes. Here’s how to build APIs that survive contact with the real world. Input Validation: Trust Nothing Every field, every header, every query parameter is hostile until proven otherwise. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 from pydantic import BaseModel, Field, validator from typing import Optional import re class CreateUserRequest(BaseModel): email: str = Field(..., max_length=254) name: str = Field(..., min_length=1, max_length=100) age: Optional[int] = Field(None, ge=0, le=150) @validator('email') def validate_email(cls, v): # Don't just regex — actually validate structure if not re.match(r'^[^@]+@[^@]+\.[^@]+$', v): raise ValueError('Invalid email format') return v.lower().strip() @validator('name') def sanitize_name(cls, v): # Remove control characters, normalize whitespace v = re.sub(r'[\x00-\x1f\x7f-\x9f]', '', v) return ' '.join(v.split()) Key principles: ...

When you’re building production systems that rely on LLM APIs, you quickly learn that “it works in development” doesn’t mean much. Rate limits hit at the worst times, APIs go down, and costs can spiral if you’re not careful. Here’s how to build integrations that actually survive the real world. The Problem with Naive Integrations Most tutorials show you something like this: 1 2 3 4 5 6 7 8 import anthropic client = anthropic.Anthropic() response = client.messages.create( model="claude-sonnet-4-20250514", max_tokens=1024, messages=[{"role": "user", "content": prompt}] ) This works great until: ...

Your load balancer routes traffic to a pod that’s crashed. Or kills a pod that’s just slow. Or restarts a pod that’s still initializing. Health checks prevent these failures — when configured correctly. Most teams get them wrong. Here’s how to get them right. The Three Probe Types Kubernetes offers three distinct probes, each with a different purpose: Probe Question Failure Action Liveness Is the process alive? Restart container Readiness Can it handle traffic? Remove from Service Startup Has it finished starting? Delay other probes Liveness: “Should I restart this?” Detects when your app is stuck — deadlocked, infinite loop, unrecoverable state. ...

Your pod got OOMKilled. Or throttled to 5% CPU. Or evicted because the node ran out of resources. The fix isn’t “add more resources” — it’s understanding how Kubernetes scheduling actually works. Requests vs Limits Requests: What you’re guaranteed. Kubernetes uses this for scheduling. Limits: The ceiling. Exceed this and bad things happen. 1 2 3 4 5 6 7 resources: requests: memory: "256Mi" cpu: "250m" # 0.25 cores limits: memory: "512Mi" cpu: "500m" # 0.5 cores What Happens When You Exceed Them Resource Exceed Request Exceed Limit CPU Throttled when node is busy Hard throttled always Memory Fine if available OOMKilled immediately CPU is compressible — you slow down but survive. Memory is not — you die. ...

Your deploy shouldn’t kill requests mid-flight. Every dropped connection is a failed payment, a lost form submission, or a frustrated user. Graceful shutdown ensures your application finishes what it started before dying. Here’s how to do it right. The Problem Without graceful shutdown: 1 1 1 1 1 3 3 3 3 3 : : : : : 0 0 0 0 0 0 0 0 0 0 : : : : : 0 0 0 0 0 0 1 1 1 1 - - - - - R D P C U e e r l s q p o i e u l c e r e o e n s y s t s t s e s g e s i k e s t g i t a n l s e r a l r t l e c r s d o o r n r ( e i n , e c m e x e m c r p i e t e e v d i t c e i o r t d a n i e t e d e r s l e , 2 y s e m s t a e y c b o e n d g i r v e e s s p o u n p s e ) With graceful shutdown: ...

Your application says it’s running. But is it actually working? Health checks answer that question. They’re the difference between “process exists” and “service is functional.” Get them wrong, and your orchestrator will either route traffic to broken instances or restart healthy ones. Three Types of Probes Liveness: “Is this process stuck?” Liveness probes detect deadlocks, infinite loops, and zombie processes. If liveness fails, the container gets killed and restarted. 1 2 3 4 5 6 7 livenessProbe: httpGet: path: /healthz port: 8080 initialDelaySeconds: 30 periodSeconds: 10 failureThreshold: 3 What to check: ...

Kill -9 is violence. Your application deserves a dignified death. Graceful shutdown means finishing in-flight work before terminating. Without it, deployments cause dropped requests, broken connections, and data corruption. With it, users never notice you restarted. The Problem When a process receives SIGTERM: Kubernetes/Docker sends the signal Your app has a grace period (default 30s) After the grace period, SIGKILL terminates forcefully If your app doesn’t handle SIGTERM, in-flight requests get dropped. Database transactions abort. WebSocket connections die mid-message. ...

Learn how circuit breakers prevent cascade failures in distributed systems by detecting failures early and failing fast instead of waiting for timeouts.

Learn how to design idempotent APIs that handle duplicate requests safely, making your systems more resilient to network failures and client retries.

Networks fail. Services go down. Databases get overwhelmed. The question isn’t whether your requests will fail—it’s how gracefully you handle it when they do. Naive retry logic can turn a minor hiccup into a catastrophic cascade. Smart retry logic can make your system resilient to transient failures. The difference is in the details. The Naive Approach (Don’t Do This) 1 2 3 4 5 6 7 8 9 # Bad: Immediate retry loop def fetch_data(url): for attempt in range(5): try: response = requests.get(url, timeout=5) return response.json() except requests.RequestException: continue raise Exception("Failed after 5 attempts") This code has several problems: ...