Rate-Limiting

Every public API needs rate limiting. Without it, one misbehaving client can take down your entire service—whether through malice, bugs, or just enthusiasm. Rate limiting protects your infrastructure, ensures fair usage, and creates predictable behavior for all clients. The Core Algorithms Fixed Window Count requests in fixed time intervals (e.g., per minute): 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 class FixedWindowLimiter { constructor(redis, limit, windowSeconds) { this.redis = redis; this.limit = limit; this.windowSeconds = windowSeconds; } async isAllowed(clientId) { const window = Math.floor(Date.now() / 1000 / this.windowSeconds); const key = `ratelimit:${clientId}:${window}`; const count = await this.redis.incr(key); if (count === 1) { await this.redis.expire(key, this.windowSeconds); } return count <= this.limit; } } Pros: Simple, memory-efficient. Cons: Burst at window boundaries. Client could hit 100 requests at 0:59 and 100 more at 1:00. ...

Every API needs rate limiting. Without it, a single misbehaving client can overwhelm your service, intentional attacks become trivial, and cost management becomes impossible. But implement it poorly, and you’ll frustrate legitimate users while barely slowing down bad actors. Let’s explore rate limiting patterns that actually work. The Fundamentals: Why Rate Limit? Rate limiting serves multiple purposes: Protection: Prevent abuse, DDoS attacks, and runaway scripts Fairness: Ensure one client can’t monopolize resources Cost control: Limit expensive operations (API calls, LLM tokens, etc.) Stability: Maintain consistent performance under load Algorithm 1: Token Bucket The token bucket is the most flexible approach. Imagine a bucket that fills with tokens at a steady rate. Each request consumes a token. If the bucket is empty, the request is denied. ...