Best-Practices

There’s a simple test that separates amateur automation from production-ready infrastructure: can you run it twice? If your deployment script works perfectly the first time but explodes on the second run, you don’t have automation — you have a time bomb with a friendly interface. What Idempotency Actually Means An operation is idempotent if performing it multiple times produces the same result as performing it once. In practical terms: 1 2 3 4 5 # Idempotent: always results in nginx being installed apt install nginx # NOT idempotent: appends every time echo "export PATH=/opt/bin:$PATH" >> ~/.bashrc The first command checks state before acting. The second blindly mutates. ...

Environment variables seem trivial—just key-value pairs. Then you have 50 of them across 4 environments with secrets mixed in, and suddenly you’re in configuration hell. Here’s how to stay sane. The Hierarchy Configuration should flow from least to most specific: D e f a u l t s ( c o d e ) → C o n f i g f i l e s → E n v v a r s → C o m m a n d - l i n e f l a g s Each layer overrides the previous. Environment variables sit near the top—easy to change per environment without touching code. ...

The only backup that matters is the one you can restore. Everything else is wishful thinking with storage costs. Here’s how to build backup systems that work when disaster strikes. The 3-2-1 Rule (Still Valid) The classic backup rule holds up: 3 copies of your data 2 different storage types 1 offsite location Modern interpretation: P L O r o f i c f m a s a l i r : t y e : : P D S r a 3 o i / d l G u y C c S t s / i n A o a z n p u s r d h e a o t t B a s l b o a ( b s d e i ( f d ( f i S e f S r f D e e ) n r t e n t o l r u e m g e i ) o n ) This protects against: ...

Everyone logs. Few log well. The difference between “we have logs” and “we can debug with logs” comes down to discipline in what you capture, how you structure it, and where you send it. The Logging Hierarchy Not all log levels are created equal. Use them intentionally: F E W I D T A R A N E R T R R F B A A O N O U C L R G E → → → → → → T S U N D E h o n o e x e m e r t t e x m a r a t p a i e p h e l l m p i c e e l n t o d l i g e p y c d e d a f r i v t a b a a e i i u t g r o l t i n b n e o o o d h n s s c , a t e a n m i . n b d i c n u l l N o t e e i e t d s n v t . t f e c h o o r o e M n . n i e i t a g s E n i p h . x n p t p p u T e r e k b h n o . e e e s d e c i u W p o h v c a s m e e t k e a , i e r r o u a t u n s n b s . o n p e u m i r a a e n o t l o g b . l n . l y e e N m o u e . f p e f . d s i n a t p t r e o n d t . i o n . The key insight: INFO should tell a story. If you read only INFO logs, you should understand what the application did. ...

Feature flags seem simple: wrap code in an if statement, flip a boolean, ship whenever you want. In practice, they’re one of the fastest ways to accumulate invisible technical debt. Here’s how to get the benefits without the baggage. Why Feature Flags Matter The core value proposition is decoupling deployment from release: 1 2 3 4 5 6 7 8 9 10 11 12 # Without flags: deploy = release def checkout(): process_payment() send_confirmation() # With flags: deploy when ready, release when confident def checkout(): process_payment() if feature_enabled("new_confirmation_flow"): send_rich_confirmation() else: send_confirmation() This separation enables: ...

Your dependencies will fail. Database goes down, third-party API times out, cache disappears. The question isn’t whether this happens—it’s whether your users notice. Graceful degradation keeps things working when components fail. The Philosophy Instead of: “Redis is down → Application crashes” Think: “Redis is down → Features using Redis degrade → Core features work” 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Brittle: Cache failure = Application failure def get_user(user_id): cached = redis.get(f"user:{user_id}") # Throws if Redis down if cached: return json.loads(cached) return db.query("SELECT * FROM users WHERE id = %s", user_id) # Resilient: Cache failure = Slower, but working def get_user(user_id): try: cached = redis.get(f"user:{user_id}") if cached: return json.loads(cached) except RedisError: logger.warning("Cache unavailable, falling back to database") return db.query("SELECT * FROM users WHERE id = %s", user_id) Timeouts: The First Defense Never wait forever: ...

“We don’t test in production” sounds responsible until you realize: production is the only environment that’s actually production. Staging lies to you. Here’s how to test in production safely. Why Staging Fails Staging environments differ from production in ways that matter: Data: Sanitized, outdated, or synthetic Scale: 1% of production traffic Integrations: Sandbox APIs with different behavior Users: Developers clicking around, not real usage patterns Infrastructure: Smaller instances, shared resources That bug that only appears under real load with real data? Staging won’t catch it. ...

Git hooks are scripts that run automatically at specific points in your Git workflow. Use them to catch problems before they become PR comments. Here’s how to set them up effectively. Hook Basics Git hooks live in .git/hooks/. They’re executable scripts that run at specific events: 1 2 3 4 5 6 .git/hooks/ ├── pre-commit # Before commit is created ├── commit-msg # After commit message is entered ├── pre-push # Before push to remote ├── post-merge # After merge completes └── ... To enable a hook, create an executable script with the hook’s name: ...

Webhooks are HTTP endpoints that receive data from external services. Anyone who discovers your webhook URL can send requests to it. That’s a problem. Here’s how to secure them properly. The Threat Model Your webhook endpoint faces several threats: Spoofing: Attacker sends fake payloads pretending to be Stripe/GitHub/etc. Replay attacks: Attacker captures a legitimate request and resends it Tampering: Attacker intercepts and modifies payloads in transit Enumeration: Attacker discovers your webhook URLs through guessing Denial of service: Attacker floods your endpoint with requests Signature Verification Most webhook providers sign their payloads. Always verify: ...

Networks fail. Clients timeout. Users double-click. If your API creates duplicate orders or charges cards twice when this happens, you have a problem. Idempotency is the solution—making operations safe to retry without side effects. What Is Idempotency? An operation is idempotent if performing it multiple times has the same effect as performing it once. # G P D # P P E U E O O I T T L N S S d E O T T e / / T T m u u E p s s i u o e e / d r s t r r u e d e e s s s m e r n / / e p r s t 1 1 r o s / : 2 2 s t 1 3 3 / e 2 S 1 n 3 a { 2 t / m . 3 : c e . h . D a r } i r e f g s f e u e l r t # # # e # # n e A A U t C C v l l s r h e w w e r e a r a a r e a r y y y s t g s s 1 u e e t 2 l s s i r s 3 t m e e a t e t t i e h u s s a N e r c E n u g h W c s s o a e n t o r u r e i r d s m d e 1 ( e e a r 2 a r g 3 l a 1 r e i 2 t e a n 3 o a c d h e t y a h t c i g i h s o m n e t s e i t m a = e t e s t i l l g o n e ) The Problem Client sends request → Server processes it → Response lost in transit: ...