Posts

Learn how to design idempotent APIs that handle duplicate requests safely, making your systems more resilient to network failures and client retries.

GitHub Actions has become the de facto CI/CD platform for many teams, but most only scratch the surface with pre-built actions from the marketplace. Building custom actions tailored to your infrastructure needs can dramatically reduce boilerplate and enforce consistency across repositories. Why Custom Actions? Every DevOps team has workflows that repeat across projects: Deploying to specific cloud environments Running security scans with custom policies Provisioning temporary environments for PR reviews Rotating secrets on a schedule Instead of copy-pasting YAML across repositories, custom actions encapsulate this logic once and reference it everywhere. ...

Every LLM API call costs money and takes time. When users ask variations of the same question, you’re paying for the same computation repeatedly. Semantic caching solves this by recognizing that “What’s the weather in NYC?” and “How’s the weather in New York City?” are functionally identical. The Problem with Traditional Caching Standard key-value caching uses exact string matching: 1 2 3 cache_key = hash(prompt) if cache_key in cache: return cache[cache_key] This fails for LLM applications because: ...

When your application spans multiple services, containers, and regions, print("something went wrong") doesn’t cut it anymore. Structured logging transforms your logs from walls of text into queryable data. Why Structured Logging? Traditional logs are strings meant for humans: [ 2 0 2 6 - 0 2 - 1 3 1 4 : 0 0 : 0 0 ] E R R O R : F a i l e d t o p r o c e s s o r d e r 1 2 3 4 5 f o r u s e r j o h n @ e x a m p l e . c o m Structured logs are data meant for machines (and humans): ...

One of the most underestimated challenges in building LLM-powered applications is context window management. You’ve got 128k tokens, but that doesn’t mean you should use them all on every request. The Problem Large context windows create a false sense of abundance. Yes, you can stuff 100k tokens into a request, but you’ll pay for it in: Latency: More tokens = slower responses Cost: You’re billed per token (input and output) Quality degradation: The “lost in the middle” phenomenon is real Practical Patterns 1. Rolling Window with Summary Keep a rolling window of recent conversation, but periodically summarize older context: ...

Every request to your microservices should pass through a single front door. That door is your API gateway—and getting it right determines whether your architecture scales gracefully or collapses under complexity. Why API Gateways? Without a gateway, clients must: Know the location of every service Handle authentication with each service Implement retry logic, timeouts, and circuit breaking Deal with different protocols and response formats An API gateway centralizes these concerns: ...

That 1.2GB Python image you’re pushing to production? It contains gcc, make, and half of Debian’s package repository. Your application needs none of it at runtime. Container image optimization isn’t just about saving disk space—it’s about security (smaller attack surface), speed (faster pulls and deploys), and cost (less bandwidth and storage). Let’s fix it. The Problem: Development vs Runtime A typical Dockerfile grows organically: 1 2 3 4 5 6 7 8 9 # The bloated approach FROM python:3.11 WORKDIR /app COPY requirements.txt . RUN pip install -r requirements.txt COPY . . CMD ["python", "app.py"] This image includes: ...

Networks fail. Services go down. Databases get overwhelmed. The question isn’t whether your requests will fail—it’s how gracefully you handle it when they do. Naive retry logic can turn a minor hiccup into a catastrophic cascade. Smart retry logic can make your system resilient to transient failures. The difference is in the details. The Naive Approach (Don’t Do This) 1 2 3 4 5 6 7 8 9 # Bad: Immediate retry loop def fetch_data(url): for attempt in range(5): try: response = requests.get(url, timeout=5) return response.json() except requests.RequestException: continue raise Exception("Failed after 5 attempts") This code has several problems: ...

Your container is about to die. It has 30 seconds to live. What happens next determines whether your users see a clean transition or a wall of 502 errors. Graceful shutdown is one of those things that seems obvious until you realize most applications do it wrong. The Problem When Kubernetes (or Docker, or systemd) decides to stop your application, it sends a SIGTERM signal. Your application has a grace period—usually 30 seconds—to finish what it’s doing and exit cleanly. After that, it gets SIGKILL. No negotiation. ...

There’s a subtle but powerful distinction in modern software delivery: deployment is not release. Deployment means your code is running in production. Release means your users can see it. Feature flags are the bridge between these two concepts—and mastering them changes how you think about shipping software. The Problem with Traditional Deployment In the old model, deploying code meant releasing features: 1 2 3 # Old way: deploy = release git push origin main # Boom, everyone sees the new feature immediately This creates pressure. You can’t deploy partially-complete work. You can’t test in production with real traffic. And if something breaks, your only option is another deploy to roll back. ...