Testing in Production: Because Staging Never Tells the Whole Story

“We don’t test in production” sounds responsible until you realize: production is the only environment that’s actually production. Staging lies to you. Here’s how to test in production safely. Why Staging Fails Staging environments differ from production in ways that matter: Data: Sanitized, outdated, or synthetic Scale: 1% of production traffic Integrations: Sandbox APIs with different behavior Users: Developers clicking around, not real usage patterns Infrastructure: Smaller instances, shared resources That bug that only appears under real load with real data? Staging won’t catch it. ...

March 1, 2026 Â· 4 min Â· 836 words Â· Rob Washington

Git Hooks: Automate Quality Checks Before Code Leaves Your Machine

Git hooks are scripts that run automatically at specific points in your Git workflow. Use them to catch problems before they become PR comments. Here’s how to set them up effectively. Hook Basics Git hooks live in .git/hooks/. They’re executable scripts that run at specific events: 1 2 3 4 5 6 .git/hooks/ ├── pre-commit # Before commit is created ├── commit-msg # After commit message is entered ├── pre-push # Before push to remote ├── post-merge # After merge completes └── ... To enable a hook, create an executable script with the hook’s name: ...

March 1, 2026 Â· 5 min Â· 1056 words Â· Rob Washington

Webhook Security: Protecting Your Endpoints from the Wild West

Webhooks are HTTP endpoints that receive data from external services. Anyone who discovers your webhook URL can send requests to it. That’s a problem. Here’s how to secure them properly. The Threat Model Your webhook endpoint faces several threats: Spoofing: Attacker sends fake payloads pretending to be Stripe/GitHub/etc. Replay attacks: Attacker captures a legitimate request and resends it Tampering: Attacker intercepts and modifies payloads in transit Enumeration: Attacker discovers your webhook URLs through guessing Denial of service: Attacker floods your endpoint with requests Signature Verification Most webhook providers sign their payloads. Always verify: ...

March 1, 2026 Â· 5 min Â· 968 words Â· Rob Washington

Idempotency: Making APIs Safe to Retry

Networks fail. Clients timeout. Users double-click. If your API creates duplicate orders or charges cards twice when this happens, you have a problem. Idempotency is the solution—making operations safe to retry without side effects. What Is Idempotency? An operation is idempotent if performing it multiple times has the same effect as performing it once. # G P D # P P E U E O O I T T L N S S d E O T T e / / T T m u u E p s s i u o e e / d r s t r r u e d e e s s s m e r n / / e p r s t 1 1 r o s / : 2 2 s t 1 3 3 / e 2 S 1 n 3 a { 2 t / m . 3 : c e . h . D a r } i r e f g s f e u e l r t # # # e # # n e A A U t C C v l l s r h e w w e r e a r a a r e a r y y y s t g s s 1 u e e t 2 l s s i r s 3 t m e e a t e t t i e h u s s a N e r c E n u g h W c s s o a e n t o r u r e i r d s m d e 1 ( e e a r 2 a r g 3 l a 1 r e i 2 t e a n 3 o a c d h e t y a h t c i g i h s o m n e t s e i t m a = e t e s t i l l g o n e ) The Problem Client sends request → Server processes it → Response lost in transit: ...

March 1, 2026 Â· 7 min Â· 1451 words Â· Rob Washington

Environment Variables Done Right

Environment variables are the standard way to configure applications across environments. They’re simple, universal, and supported everywhere. But like any tool, they can be misused. Here’s how to do them right. The Basics Environment variables are key-value pairs available to your process: 1 2 3 4 5 6 7 8 9 10 11 12 13 # Setting them export DATABASE_URL="postgres://localhost/myapp" export LOG_LEVEL="info" # Using them (Bash) echo $DATABASE_URL # Using them (Python) import os db_url = os.environ.get("DATABASE_URL") # Using them (Node.js) const dbUrl = process.env.DATABASE_URL; Naming Conventions Be consistent. A common pattern: ...

March 1, 2026 Â· 5 min Â· 996 words Â· Rob Washington

Logging Levels: A Practical Guide to What Goes Where

Logging seems simple until you’re debugging production at 2 AM, scrolling through millions of lines trying to find the one that matters. Good logging practices make that experience less painful. Here’s how to think about log levels. The Levels Most logging frameworks use these standard levels: D E B U G < I N F O < W A R N < E R R O R < F A T A L In production, you typically run at INFO or WARN. Lower levels include all higher levels (INFO includes WARN, ERROR, and FATAL). ...

March 1, 2026 Â· 4 min Â· 836 words Â· Rob Washington

API Error Handling That Helps Instead of Frustrates

Bad error handling wastes everyone’s time. A cryptic “Error 500” sends developers on a debugging odyssey. A well-designed error response tells them exactly what went wrong and how to fix it. Here’s how to build the latter. The Anatomy of a Good Error Every error response should answer three questions: What happened? (error code/type) Why? (human-readable message) How do I fix it? (actionable guidance) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 { "error": { "code": "VALIDATION_ERROR", "message": "Request validation failed", "details": [ { "field": "email", "message": "Invalid email format", "received": "not-an-email" }, { "field": "age", "message": "Must be a positive integer", "received": "-5" } ], "documentation_url": "https://api.example.com/docs/errors#VALIDATION_ERROR" }, "request_id": "req_abc123" } Always include: ...

March 1, 2026 Â· 6 min Â· 1214 words Â· Rob Washington

Configuration Management Patterns for Reliable Deployments

Configuration is where deployments go to die. A typo in an environment variable, a missing secret, a config file that works in staging but breaks in production. Here’s how to make configuration boring and reliable. The Hierarchy of Configuration Not all config is created equal. Layer it: 1 2 3 4 5 . . . . . D C E C F e o n o e f n v m a a f i m t u i r a u l g o n r t n d e s f m - i e l f ( l n i l i e t n a n s e g v s c ( a f o p r l ( d e i a r e r a g u ) b s n e l t n e i v s m i e r ) o n m e n t ) S O D a R n y f E u e n e n n - a s v t o m t i i f i r m f c f o e a n c l m v h l e v e a b n e r n a t r r g c - r i e k s i d s s p d e e e s c s i f i c Later layers override earlier ones. This lets you: ...

March 1, 2026 Â· 7 min Â· 1386 words Â· Rob Washington

The Twelve-Factor App: What Actually Matters

The Twelve-Factor methodology is from 2011 but remains relevant. Here’s what matters in practice, and what’s become outdated. The Factors, Ranked by Impact Critical (Ignore at Your Peril) III. Config in Environment 1 2 3 4 5 # Bad DATABASE_URL = "postgres://localhost/myapp" # hardcoded # Good DATABASE_URL = os.environ["DATABASE_URL"] Config includes credentials, per-environment values, and feature flags. Environment variables work everywhere: containers, serverless, bare metal. VI. Stateless Processes 1 2 3 4 5 6 7 8 9 10 11 # Bad: storing session in memory sessions = {} @app.post("/login") def login(user): sessions[user.id] = {"logged_in": True} # Dies with process # Good: external session store @app.post("/login") def login(user): redis.set(f"session:{user.id}", {"logged_in": True}) If your process dies, can another pick up the work? Statelessness enables horizontal scaling, rolling deploys, and crash recovery. ...

February 28, 2026 Â· 5 min Â· 864 words Â· Rob Washington

Environment Variables Done Right: 12-Factor Config in Practice

The third factor of the 12-Factor App methodology states: “Store config in the environment.” Simple advice that’s surprisingly easy to get wrong. The Core Principle Configuration that varies between environments (dev, staging, production) should come from environment variables, not code. This includes: Database connection strings API keys and secrets Feature flags Service URLs Port numbers Log levels What stays in code: application logic, default behaviors, anything that doesn’t change between deploys. ...

February 25, 2026 Â· 6 min Â· 1182 words Â· Rob Washington