Architecture

Some work doesn’t belong in a web request. Sending emails, processing uploads, generating reports, syncing with external APIs — these tasks are too slow, too unreliable, or too resource-intensive to run while a user waits. Background jobs solve this by moving work out of the request cycle and into a separate processing system. The Basic Architecture ┌ │ └ ─ ─ ─ W ─ ─ e ─ ─ b ─ ─ ─ │ │ └ ─ A ─ ─ ─ p ─ ─ ─ p ─ ─ ─ ─ ─ ─ ─ ─ ┐ │ ┘ ─ ─ ─ ─ ─ ─ ▶ ─ ┌ │ └ ▶ ─ ─ ┌ │ └ ─ R ─ ─ ─ ─ e ─ ─ Q ─ ─ s ─ ─ u ─ ─ u ─ ─ e ─ ─ l ─ ─ u ─ ─ t ─ ─ e ─ ─ s ─ ─ ─ ─ ─ ┐ │ ┘ ┐ │ ┘ ─ ◀ ─ ─ ─ ─ ─ ─ ▶ ─ ┌ │ └ ─ ─ ─ ─ ─ W ─ ─ ─ o ─ ─ ─ r ─ │ ┘ ─ k ─ │ ─ e ─ ─ r ─ ─ s ─ ─ ─ ─ ─ ┐ │ ┘ Producer: Web app enqueues jobs Queue: Stores jobs until workers are ready Workers: Process jobs independently Results: Optional storage for job outcomes Choosing a Queue Backend Redis (with Sidekiq, Bull, Celery) 1 2 3 4 5 6 7 8 9 # Celery with Redis from celery import Celery app = Celery('tasks', broker='redis://localhost:6379/0') @app.task def send_email(user_id, template): user = get_user(user_id) email_service.send(user.email, template) Pros: Fast, simple, good ecosystem Cons: Not durable by default (can lose jobs on crash) ...

The twelve-factor app methodology emerged from Heroku’s experience deploying thousands of applications. These principles create applications that work well in modern cloud environments — containerized, horizontally scalable, and continuously deployed. They’re not arbitrary rules. Each factor solves a real problem. I. Codebase One codebase tracked in version control, many deploys. ✅ m ├ ├ └ ❌ m m m y ─ ─ ─ y y y G a ─ ─ ─ B a a a o p a p p p o p . s d d p p p d g r e : - - - : i c p p s d t l r t e o o a v y d g e u i l t c n o o t g p : i m e s n n t / t a / g i n g , p r o d u c t i o n , f e a t u r e - b r a n c h e s Different environments come from the same codebase. Configuration, not code, varies between deploys. ...

Synchronous request-response is simple: client asks, server answers, everyone waits. But some operations don’t fit that model. Sending emails, processing images, generating reports — these take time, and your users shouldn’t wait. Message queues decouple the “request” from the “work,” letting you respond immediately while processing happens in the background. The Basic Pattern 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # API endpoint - returns immediately @app.post("/orders") def create_order(order_data): order = db.create_order(order_data) # Queue async work instead of doing it now queue.enqueue('process_order', order.id) return {"order_id": order.id, "status": "processing"} # Worker - processes queue in background def process_order(order_id): order = db.get_order(order_id) charge_payment(order) send_confirmation_email(order) notify_warehouse(order) update_inventory(order) The user gets a response in milliseconds. The actual work happens whenever the worker gets to it. ...

Every API eventually needs to change in ways that break existing clients. Field removed, response format changed, authentication updated. The question isn’t whether you’ll have breaking changes — it’s how you’ll manage them. Good versioning gives you freedom to evolve while giving clients stability and migration paths. The Three Schools of Versioning URL Path Versioning G G E E T T / / a a p p i i / / v 1 2 / / u u s s e e r r s s / / 1 1 2 2 3 3 Pros: ...

The fastest request is one you don’t make. Caching trades storage for speed, serving precomputed results instead of recalculating them. But caching done wrong is worse than no caching—stale data, inconsistencies, and debugging nightmares. When to Cache Cache when: Data is read more often than written Computing the result is expensive Slight staleness is acceptable The same data is requested repeatedly Don’t cache when: Data changes constantly Every request needs fresh data Storage cost exceeds compute savings Cache invalidation is harder than recomputation Cache Placement Client-Side Cache Browser cache, mobile app cache, CDN edge cache: ...

Request-response is synchronous. Events are not. That difference changes everything about how you build systems. In event-driven architecture, components communicate by producing and consuming events rather than calling each other directly. The producer doesn’t know who’s listening. The consumer doesn’t know who produced. This decoupling enables scale, resilience, and evolution that tight coupling can’t match. Why Events? Temporal decoupling: Producer and consumer don’t need to be online simultaneously. The order service publishes “OrderPlaced”; the shipping service processes it when ready. ...

One of the most underestimated challenges in building LLM-powered applications is context window management. You’ve got 128k tokens, but that doesn’t mean you should use them all on every request. The Problem Large context windows create a false sense of abundance. Yes, you can stuff 100k tokens into a request, but you’ll pay for it in: Latency: More tokens = slower responses Cost: You’re billed per token (input and output) Quality degradation: The “lost in the middle” phenomenon is real Practical Patterns 1. Rolling Window with Summary Keep a rolling window of recent conversation, but periodically summarize older context: ...

Every request to your microservices should pass through a single front door. That door is your API gateway—and getting it right determines whether your architecture scales gracefully or collapses under complexity. Why API Gateways? Without a gateway, clients must: Know the location of every service Handle authentication with each service Implement retry logic, timeouts, and circuit breaking Deal with different protocols and response formats An API gateway centralizes these concerns: ...

The twelve-factor methodology emerged from Heroku’s experience running millions of apps. These principles create applications that deploy cleanly, scale effortlessly, and minimize divergence between development and production. Let’s walk through each factor with practical examples. 1. Codebase: One Repo, Many Deploys One codebase tracked in version control, many deploys (dev, staging, prod). 1 2 3 4 5 6 7 8 9 # Good: Single repo, branch-based environments main → production staging → staging feature/* → development # Bad: Separate repos for each environment myapp-dev/ myapp-staging/ myapp-prod/ 1 2 3 4 5 # config.py - Same code, different configs import os ENVIRONMENT = os.getenv("ENVIRONMENT", "development") DATABASE_URL = os.getenv("DATABASE_URL") 2. Dependencies: Explicitly Declare and Isolate Never rely on system-wide packages. Declare everything. ...

When Service A needs to tell Service B something happened, the simplest approach is a direct HTTP call. But what happens when Service B is slow? Or down? Or overwhelmed? Message queues decouple your services, letting them communicate reliably even when things go wrong. Why Queues? Without a queue: U s e r R e q u e s t → A P I → P ( ( a i i y f f m e s d n l o t o w w n S , , e r u r v s e i e q c r u e e w s → a t i E t f m s a a ) i i l l s ) S e r v i c e → R e s p o n s e With a queue: ...