Bad APIs create support tickets. Good APIs create fans. Here’s how to design APIs that developers will actually enjoy using. The Fundamentals Use Nouns, Not Verbs # P G P # P G D O E O O E E B S T S G S T L a T T o T E d o T g / d u E c e d u s r t e s e / e U l e r u a s e r s s t e t s e e r e r U s U s s s / e e 1 r r 2 / 3 1 2 3 The HTTP method IS the verb. The URL is the noun. ...
Integrating LLM APIs into production systems is harder than the tutorials suggest. The API call works in development. Then you hit rate limits, latency spikes, context limits, and costs that scale faster than your revenue. Here’s how to build LLM integrations that actually work. The Basics Nobody Mentions Always Stream Non-streaming API calls block until complete. For a 500-token response, that’s 5-15 seconds of your user staring at nothing. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # Bad: User waits forever response = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": prompt}] ) print(response.choices[0].message.content) # Good: Tokens appear as generated stream = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": prompt}], stream=True ) for chunk in stream: if chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="", flush=True) Streaming also lets you abort early if the response is going off-rails. ...
Caching seems simple. Add Redis, cache everything, go fast. Then you get stale data, cache stampedes, and bugs that only happen in production. Here’s how to cache correctly. When to Cache Good candidates: Expensive database queries (aggregations, joins) External API responses Computed values that don’t change often Static content (templates, configs) Session data Bad candidates: Data that changes frequently User-specific data with many variations Security-sensitive data Data where staleness causes real problems Rule of thumb: Cache when read frequency » write frequency. ...
Your API is slow because it’s doing too much synchronously. Here’s when to reach for a message queue, and how to implement it without overcomplicating everything. When You Need a Queue Signs you need async processing: API response time dominated by side effects (emails, webhooks, analytics) Downstream service failures cascade to user-facing errors Traffic spikes overwhelm dependent services You need to retry failed operations automatically Work needs to happen on a schedule or with delay Signs you don’t: ...
You want to add LLM capabilities to your application. Not build a chatbot — actually integrate AI into your product. Here are the patterns that work. The Naive Approach (And Why It Fails) 1 2 3 4 5 6 def process_user_input(text): response = openai.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": text}] ) return response.choices[0].message.content Problems: No error handling No rate limiting No caching No fallbacks No cost control Prompt injection vulnerable Let’s fix each one. Pattern 1: The Robust Client Wrap your LLM calls in a proper client: ...
You shipped v1 of your API. Users integrated it. Now you need breaking changes. How do you evolve without breaking everyone? API versioning seems simple until you actually do it. Here’s what works, what doesn’t, and how to pick the right strategy. The Core Problem APIs are contracts. When you change the response format, rename fields, or alter behavior, you break that contract. Clients built against v1 stop working when you ship v2. ...
Event-driven architecture (EDA) sounds enterprise-y. Kafka clusters. Schema registries. Teams of platform engineers. But the core concepts? They’re surprisingly accessible—and incredibly useful—even for small teams. Why Events Matter (Even for Small Projects) The alternative to events is tight coupling. Service A calls Service B directly. Service B calls Service C. Soon you have a distributed monolith where everything needs to know about everything else. Events flip this model. Instead of “Service A tells Service B to do something,” it becomes “Service A announces what happened, and anyone who cares can respond.” ...
There are only two hard things in computer science: cache invalidation, naming things, and off-by-one errors. Caching is straightforward when your data never changes. Real systems aren’t that simple. Data changes, caches get stale, and suddenly your users see yesterday’s prices or last week’s profile pictures. Here’s how to build caching that scales without becoming a source of bugs and outages. Cache-Aside: The Default Pattern Most applications should start here: ...
Rate limiting is the immune system of your API. Without it, a single misbehaving client can take down your service for everyone. With poorly designed limits, you’ll frustrate legitimate users while sophisticated attackers route around you. The goal isn’t just protection—it’s fairness. Every user gets a reasonable share of your capacity. The Basic Algorithms Fixed Window The simplest approach: count requests per time window, reject when over limit. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 import time import redis def is_rate_limited(user_id: str, limit: int = 100, window: int = 60) -> bool: """Fixed window: 100 requests per minute.""" r = redis.Redis() # Window key based on current minute window_key = f"ratelimit:{user_id}:{int(time.time() // window)}" current = r.incr(window_key) if current == 1: r.expire(window_key, window) return current > limit Problem: Burst at window boundaries. A user can make 100 requests at 0:59 and 100 more at 1:00—200 requests in 2 seconds while technically staying under “100/minute.” ...
Every database connection has a cost. TCP handshake, TLS negotiation, authentication, session setup—all before your first query runs. For PostgreSQL, that’s typically 20-50ms. For a single request, barely noticeable. For thousands of requests per second, catastrophic. Connection pooling solves this by maintaining a set of pre-established connections that your application reuses. Done right, it’s one of the highest-impact performance optimizations you can make. The Problem: Connection Overhead Without pooling, every request cycle looks like this: ...