Webhook Reliability: Building Event Delivery That Actually Works

Webhooks are the internet’s way of saying “hey, something happened.” Simple in concept, surprisingly tricky in practice. The challenge: HTTP is unreliable, servers go down, networks flake out, and your webhook payload might arrive zero times, once, or five times. Building reliable webhook infrastructure means handling all of these gracefully. The Sender Side Retry with Exponential Backoff First attempt fails? Try again. But not immediately. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 RETRY_DELAYS = [60, 300, 900, 3600, 14400, 43200] # seconds def deliver_webhook(event, attempt=0): try: response = requests.post( event.webhook_url, json=event.payload, timeout=30, headers={ "Content-Type": "application/json", "X-Webhook-ID": event.id, "X-Webhook-Timestamp": str(int(time.time())), "X-Webhook-Signature": sign_payload(event.payload) } ) if response.status_code >= 200 and response.status_code < 300: mark_delivered(event) return if response.status_code >= 500: # Server error, retry schedule_retry(event, attempt) else: # Client error (4xx), don't retry mark_failed(event, response.status_code) except requests.Timeout: schedule_retry(event, attempt) except requests.ConnectionError: schedule_retry(event, attempt) def schedule_retry(event, attempt): if attempt >= len(RETRY_DELAYS): mark_failed(event, "max_retries") return delay = RETRY_DELAYS[attempt] # Add jitter to prevent thundering herd jitter = random.uniform(0, delay * 0.1) queue_at = time.time() + delay + jitter enqueue(event, attempt + 1, queue_at) Key decisions: ...

February 24, 2026 Â· 6 min Â· 1276 words Â· Rob Washington

API Pagination Patterns: Getting Large Result Sets Right

Every API eventually needs to return more data than fits in a single response. How you handle that pagination affects performance, reliability, and developer experience. Let’s look at the common patterns, their tradeoffs, and when to use each. The Three Main Approaches 1. Offset Pagination The classic approach: skip N records, return M records. G G G E E E T T T / / / a a a p p p i i i / / / i i i t t t e e e m m m s s s ? ? ? o o o f f f f f f s s s e e e t t t = = = 0 2 4 & 0 0 l & & i l l m i i i m m t i i = t t 2 = = 0 2 2 0 0 # # # P P P a a a g g g e e e 1 2 3 Implementation: ...

February 24, 2026 Â· 9 min Â· 1726 words Â· Rob Washington

Container Orchestration Patterns: Beyond 'Just Deploy It'

Running one container is easy. Running hundreds in production, reliably, at scale? That’s where patterns emerge. These aren’t Kubernetes-specific (though that’s where you’ll see them most). They’re fundamental approaches to composing containers into systems that actually work. The Sidecar Pattern A sidecar is a helper container that runs alongside your main application container, sharing the same pod/network namespace. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 apiVersion: v1 kind: Pod metadata: name: web-app spec: containers: # Main application - name: app image: myapp:1.0 ports: - containerPort: 8080 # Sidecar: log shipper - name: log-shipper image: fluentd:latest volumeMounts: - name: logs mountPath: /var/log/app volumes: - name: logs emptyDir: {} Common sidecar use cases: ...

February 24, 2026 Â· 6 min Â· 1153 words Â· Rob Washington

Blue-Green Deployments: Zero-Downtime Releases Without the Drama

The scariest moment in software delivery used to be clicking “deploy.” Will it work? Will it break? Will you be debugging at 2 AM? Blue-green deployments eliminate most of that fear. Instead of updating your production environment in place, you deploy to an identical standby environment and switch traffic over. If something’s wrong, you switch back. Done. The Core Concept You maintain two identical production environments: Blue: Currently serving live traffic Green: Idle, ready for the next release To deploy: ...

February 24, 2026 Â· 8 min Â· 1600 words Â· Rob Washington

Database Indexing Strategies: The Performance Lever You're Probably Misusing

Every junior developer learns that indexes make queries fast. What they don’t learn is that indexes also make writes slow, consume disk space, and can actively hurt performance when misused. Let’s fix that. What Indexes Actually Do An index is a separate data structure that maintains a sorted copy of specific columns, with pointers back to the full rows. Think of it like the index in a book — instead of reading every page to find “PostgreSQL,” you flip to the index, find the entry, and jump directly to page 247. ...

February 24, 2026 Â· 6 min Â· 1207 words Â· Rob Washington

Alerting That Doesn't Suck: From Noise to Signal

The worst oncall shift I ever had wasn’t the one with the outage. It was the one with 47 alerts, none of which mattered, followed by one that did — which I almost missed because I’d stopped paying attention. Alert fatigue is real, and it’s a systems problem, not a discipline problem. If your alerts are noisy, the fix isn’t “try harder to pay attention.” The fix is better alerts. ...

February 24, 2026 Â· 5 min Â· 1018 words Â· Rob Washington

Secrets Management in the Modern Stack

We’ve all done it. Committed an API key to git. Hardcoded a database password “just for testing.” Posted a screenshot with credentials visible in the corner. The security community has a name for this: Tuesday. But secrets management doesn’t have to be painful. Let’s walk through the progression from “please no” to “actually reasonable” in handling sensitive credentials. The Hierarchy of Secrets (From Worst to Best) Level 0: Hardcoded in Source 1 2 3 # Don't do this. Ever. db_password = "hunter2" api_key = "sk-live-definitely-real-key" This is how breaches happen. Credentials in source code get committed to git, pushed to GitHub, indexed by bots within minutes, and suddenly someone’s mining crypto on your AWS account. ...

February 24, 2026 Â· 5 min Â· 969 words Â· Rob Washington

Code Review Practices: Making Reviews Useful, Not Painful

Code review is a skill separate from coding. A great programmer can give terrible reviews — nitpicking style while missing logic bugs, or rubber-stamping everything to avoid conflict. Good reviews improve code quality and team knowledge. Bad reviews slow everything down. These practices help you do reviews that actually help. The Reviewer’s Mindset You’re Not the Gatekeeper ❌ ✅ " " I D o w e o s u l t d h n i ' s t w h o a r v k e c w o r r i r t e t c e t n l y i t a n t d h i m s a i w n a t y a " i n a b l y ? " Your job isn’t to make the code match your personal style. It’s to catch problems and improve clarity. ...

February 24, 2026 Â· 12 min Â· 2380 words Â· Rob Washington

Terminal Productivity: Tools and Techniques for Speed

The terminal is where developers live. A few minutes learning shortcuts and tools saves hours over a career. These are the techniques that make the biggest difference. Shell Basics That Pay Off History Navigation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # Search history with Ctrl+R # Type partial command, press Ctrl+R repeatedly to cycle # Run last command !! # Run last command with sudo sudo !! # Run last command starting with 'git' !git # Replace text in last command ^typo^fixed # If last command was: git psuh # ^psuh^push runs: git push # Last argument of previous command cd /very/long/path/to/directory ls $_ # $_ = /very/long/path/to/directory Keyboard Shortcuts 1 2 3 4 5 6 7 8 Ctrl+A # Move to beginning of line Ctrl+E # Move to end of line Ctrl+U # Delete from cursor to beginning Ctrl+K # Delete from cursor to end Ctrl+W # Delete word before cursor Alt+B # Move back one word Alt+F # Move forward one word Ctrl+L # Clear screen (keeps current line) Brace Expansion 1 2 3 4 5 6 7 8 9 10 11 # Create multiple directories mkdir -p project/{src,tests,docs} # Create multiple files touch file{1,2,3}.txt # Backup a file cp config.yml{,.bak} # Creates config.yml.bak # Rename with pattern mv file.{txt,md} # Renames file.txt to file.md Essential Tools fzf - Fuzzy Finder 1 2 3 4 5 6 7 8 9 10 11 12 13 # Install brew install fzf # macOS apt install fzf # Debian/Ubuntu # Search files fzf # Pipe anything to fzf cat file.txt | fzf ps aux | fzf # Ctrl+R replacement (add to .bashrc/.zshrc) # Much better history search ripgrep (rg) - Fast Search 1 2 3 4 5 6 7 8 9 10 11 12 13 # Install brew install ripgrep # Search in files (faster than grep) rg "pattern" rg "TODO" --type py rg "function" -g "*.js" # Case insensitive rg -i "error" # Show context rg -C 3 "exception" # 3 lines before and after fd - Better Find 1 2 3 4 5 6 7 8 9 10 # Install brew install fd # Find files fd "pattern" fd -e py # Find .py files fd -t d # Find directories only # Execute on results fd -e log -x rm # Delete all .log files bat - Better Cat 1 2 3 4 5 6 7 8 # Install brew install bat # Syntax highlighting, line numbers, git changes bat file.py # Use as man pager export MANPAGER="sh -c 'col -bx | bat -l man -p'" eza/exa - Better ls 1 2 3 4 5 6 7 8 # Install brew install eza # Tree view with git status eza --tree --level=2 --git # Long format with icons eza -la --icons jq - JSON Processing 1 2 3 4 5 6 7 8 9 10 11 # Pretty print JSON cat data.json | jq . # Extract field curl api.example.com/users | jq '.[0].name' # Filter array jq '.[] | select(.active == true)' # Transform jq '{name: .user.name, email: .user.email}' Tmux Essentials 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # Start new session tmux new -s project # Detach: Ctrl+B, then D # List sessions tmux ls # Attach to session tmux attach -t project # Split panes Ctrl+B % # Vertical split Ctrl+B " # Horizontal split Ctrl+B o # Switch pane Ctrl+B z # Zoom pane (toggle) # Windows Ctrl+B c # New window Ctrl+B n # Next window Ctrl+B p # Previous window Ctrl+B , # Rename window Git Shortcuts 1 2 3 4 5 6 7 8 9 10 11 # .gitconfig [alias] s = status -sb co = checkout br = branch ci = commit ca = commit --amend lg = log --oneline --graph --decorate -20 unstage = reset HEAD -- last = log -1 HEAD aliases = config --get-regexp alias 1 2 3 4 5 6 # Common workflows git s # Short status git lg # Pretty log git co - # Switch to previous branch git stash -u # Stash including untracked git stash pop # Apply and remove stash Shell Aliases 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 # Add to .bashrc or .zshrc # Navigation alias ..="cd .." alias ...="cd ../.." alias ~="cd ~" # Safety alias rm="rm -i" alias mv="mv -i" alias cp="cp -i" # Shortcuts alias ll="ls -lah" alias la="ls -la" alias g="git" alias k="kubectl" alias d="docker" alias dc="docker compose" # Quick edits alias zshrc="$EDITOR ~/.zshrc" alias reload="source ~/.zshrc" # Common tasks alias ports="netstat -tulanp" alias myip="curl ifconfig.me" alias weather="curl wttr.in" Functions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 # Make directory and cd into it mkcd() { mkdir -p "$1" && cd "$1" } # Extract any archive extract() { case $1 in *.tar.gz) tar xzf "$1" ;; *.tar.bz2) tar xjf "$1" ;; *.tar.xz) tar xJf "$1" ;; *.zip) unzip "$1" ;; *.gz) gunzip "$1" ;; *) echo "Unknown format: $1" ;; esac } # Find and kill process by name killnamed() { ps aux | grep "$1" | grep -v grep | awk '{print $2}' | xargs kill -9 } # Quick HTTP server serve() { python3 -m http.server ${1:-8000} } # Git clone and cd gclone() { git clone "$1" && cd "$(basename "$1" .git)" } Quick One-Liners 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 # Find large files find . -type f -size +100M # Disk usage by directory du -sh */ | sort -h # Watch command output watch -n 2 'kubectl get pods' # Run command on file change while inotifywait -e modify file.py; do python file.py; done # Parallel execution cat urls.txt | xargs -P 10 -I {} curl -s {} # Quick calculations echo $((2**10)) # 1024 python3 -c "print(2**100)" # Generate password openssl rand -base64 32 # Quick timestamp date +%Y%m%d_%H%M%S Environment Management 1 2 3 4 5 6 # .envrc with direnv # Auto-load environment per directory echo 'export API_KEY=xxx' > .envrc direnv allow # Now API_KEY is set when you cd into this directory SSH Config 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # ~/.ssh/config Host dev HostName dev.example.com User deploy IdentityFile ~/.ssh/dev_key Host prod HostName prod.example.com User deploy ProxyJump bastion Host * AddKeysToAgent yes IdentitiesOnly yes # Now just: ssh dev Prompt Customization 1 2 3 4 5 # Starship - cross-shell prompt brew install starship echo 'eval "$(starship init zsh)"' >> ~/.zshrc # Shows git status, language versions, errors, duration The Power Combo My essential setup: ...

February 24, 2026 Â· 7 min Â· 1294 words Â· Rob Washington

API Client Design: Building SDKs That Developers Love

A well-designed API client turns complex HTTP interactions into simple method calls. It handles authentication, retries, errors, and serialization — so users don’t have to. These patterns create clients that developers actually enjoy using. Basic Structure 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 import httpx from typing import Optional from dataclasses import dataclass @dataclass class APIConfig: base_url: str api_key: str timeout: float = 30.0 max_retries: int = 3 class APIClient: def __init__(self, config: APIConfig): self.config = config self._client = httpx.Client( base_url=config.base_url, timeout=config.timeout, headers={"Authorization": f"Bearer {config.api_key}"} ) def _request(self, method: str, path: str, **kwargs) -> dict: response = self._client.request(method, path, **kwargs) response.raise_for_status() return response.json() def close(self): self._client.close() def __enter__(self): return self def __exit__(self, *args): self.close() Resource-Based Design Organize by resource, not by HTTP method: ...

February 24, 2026 Â· 7 min Â· 1401 words Â· Rob Washington