Architecture

The Twelve-Factor methodology is from 2011 but remains relevant. Here’s what matters in practice, and what’s become outdated. The Factors, Ranked by Impact Critical (Ignore at Your Peril) III. Config in Environment 1 2 3 4 5 # Bad DATABASE_URL = "postgres://localhost/myapp" # hardcoded # Good DATABASE_URL = os.environ["DATABASE_URL"] Config includes credentials, per-environment values, and feature flags. Environment variables work everywhere: containers, serverless, bare metal. VI. Stateless Processes 1 2 3 4 5 6 7 8 9 10 11 # Bad: storing session in memory sessions = {} @app.post("/login") def login(user): sessions[user.id] = {"logged_in": True} # Dies with process # Good: external session store @app.post("/login") def login(user): redis.set(f"session:{user.id}", {"logged_in": True}) If your process dies, can another pick up the work? Statelessness enables horizontal scaling, rolling deploys, and crash recovery. ...

The gap between “AI demo” and “AI that runs reliably” is enormous. Here are patterns that emerge when you actually deploy autonomous agents. The Heartbeat Pattern Agents need periodic check-ins, not just reactive responses. A heartbeat system provides: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 @dataclass class HeartbeatState: last_email_check: datetime last_calendar_check: datetime last_service_health: datetime async def heartbeat(state: HeartbeatState): now = datetime.now() if (now - state.last_service_health).hours >= 2: await check_services() state.last_service_health = now if (now - state.last_email_check).hours >= 4: await check_inbox() state.last_email_check = now The key insight: batch periodic tasks into a single heartbeat rather than creating dozens of scheduled jobs. This reduces API calls and keeps context coherent. ...

Some work doesn’t belong in a web request. Sending emails, processing uploads, generating reports, syncing with external APIs — these tasks are too slow, too unreliable, or too resource-intensive to run while a user waits. Background jobs solve this by moving work out of the request cycle and into a separate processing system. The Basic Architecture ┌ │ └ ─ ─ ─ W ─ ─ e ─ ─ b ─ ─ ─ │ │ └ ─ A ─ ─ ─ p ─ ─ ─ p ─ ─ ─ ─ ─ ─ ─ ─ ┐ │ ┘ ─ ─ ─ ─ ─ ─ ▶ ─ ┌ │ └ ▶ ─ ─ ┌ │ └ ─ R ─ ─ ─ ─ e ─ ─ Q ─ ─ s ─ ─ u ─ ─ u ─ ─ e ─ ─ l ─ ─ u ─ ─ t ─ ─ e ─ ─ s ─ ─ ─ ─ ─ ┐ │ ┘ ┐ │ ┘ ─ ◀ ─ ─ ─ ─ ─ ─ ▶ ─ ┌ │ └ ─ ─ ─ ─ ─ W ─ ─ ─ o ─ ─ ─ r ─ │ ┘ ─ k ─ │ ─ e ─ ─ r ─ ─ s ─ ─ ─ ─ ─ ┐ │ ┘ Producer: Web app enqueues jobs Queue: Stores jobs until workers are ready Workers: Process jobs independently Results: Optional storage for job outcomes Choosing a Queue Backend Redis (with Sidekiq, Bull, Celery) 1 2 3 4 5 6 7 8 9 # Celery with Redis from celery import Celery app = Celery('tasks', broker='redis://localhost:6379/0') @app.task def send_email(user_id, template): user = get_user(user_id) email_service.send(user.email, template) Pros: Fast, simple, good ecosystem Cons: Not durable by default (can lose jobs on crash) ...

The twelve-factor app methodology emerged from Heroku’s experience deploying thousands of applications. These principles create applications that work well in modern cloud environments — containerized, horizontally scalable, and continuously deployed. They’re not arbitrary rules. Each factor solves a real problem. I. Codebase One codebase tracked in version control, many deploys. ✅ m ├ ├ └ ❌ m m m y ─ ─ ─ y y y G a ─ ─ ─ B a a a o p a p p p o p . s d d p p p d g r e : - - - : i c p p s d t l r t e o o a v y d g e u i l t c n o o t g p : i m e s n n t / t a / g i n g , p r o d u c t i o n , f e a t u r e - b r a n c h e s Different environments come from the same codebase. Configuration, not code, varies between deploys. ...

Synchronous request-response is simple: client asks, server answers, everyone waits. But some operations don’t fit that model. Sending emails, processing images, generating reports — these take time, and your users shouldn’t wait. Message queues decouple the “request” from the “work,” letting you respond immediately while processing happens in the background. The Basic Pattern 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # API endpoint - returns immediately @app.post("/orders") def create_order(order_data): order = db.create_order(order_data) # Queue async work instead of doing it now queue.enqueue('process_order', order.id) return {"order_id": order.id, "status": "processing"} # Worker - processes queue in background def process_order(order_id): order = db.get_order(order_id) charge_payment(order) send_confirmation_email(order) notify_warehouse(order) update_inventory(order) The user gets a response in milliseconds. The actual work happens whenever the worker gets to it. ...

Every API eventually needs to change in ways that break existing clients. Field removed, response format changed, authentication updated. The question isn’t whether you’ll have breaking changes — it’s how you’ll manage them. Good versioning gives you freedom to evolve while giving clients stability and migration paths. The Three Schools of Versioning URL Path Versioning G G E E T T / / a a p p i i / / v 1 2 / / u u s s e e r r s s / / 1 1 2 2 3 3 Pros: ...

The fastest request is one you don’t make. Caching trades storage for speed, serving precomputed results instead of recalculating them. But caching done wrong is worse than no caching—stale data, inconsistencies, and debugging nightmares. When to Cache Cache when: Data is read more often than written Computing the result is expensive Slight staleness is acceptable The same data is requested repeatedly Don’t cache when: Data changes constantly Every request needs fresh data Storage cost exceeds compute savings Cache invalidation is harder than recomputation Cache Placement Client-Side Cache Browser cache, mobile app cache, CDN edge cache: ...

Request-response is synchronous. Events are not. That difference changes everything about how you build systems. In event-driven architecture, components communicate by producing and consuming events rather than calling each other directly. The producer doesn’t know who’s listening. The consumer doesn’t know who produced. This decoupling enables scale, resilience, and evolution that tight coupling can’t match. Why Events? Temporal decoupling: Producer and consumer don’t need to be online simultaneously. The order service publishes “OrderPlaced”; the shipping service processes it when ready. ...

One of the most underestimated challenges in building LLM-powered applications is context window management. You’ve got 128k tokens, but that doesn’t mean you should use them all on every request. The Problem Large context windows create a false sense of abundance. Yes, you can stuff 100k tokens into a request, but you’ll pay for it in: Latency: More tokens = slower responses Cost: You’re billed per token (input and output) Quality degradation: The “lost in the middle” phenomenon is real Practical Patterns 1. Rolling Window with Summary Keep a rolling window of recent conversation, but periodically summarize older context: ...

Every request to your microservices should pass through a single front door. That door is your API gateway—and getting it right determines whether your architecture scales gracefully or collapses under complexity. Why API Gateways? Without a gateway, clients must: Know the location of every service Handle authentication with each service Implement retry logic, timeouts, and circuit breaking Deal with different protocols and response formats An API gateway centralizes these concerns: ...