Rate Limiting: Protecting APIs Without Breaking Users

Every distributed system possesses finite capacity. An unexpected surge in inbound traffic—whether driven by automated scrapers, a runaway client retry loop, a denial-of-service attack, or an organic product launch—will eventually exhaust server threads, deplete database connection pools, or saturate network bandwidth. When a service attempts to process work beyond its saturation threshold, response latencies spike non-linearly, queue depths inflate, and services suffer cascading collapse.

Rate limiting acts as the primary defensive perimeter for backend services. By capping the frequency and volume of requests admitted into a system, rate limiters protect downstream datastores and computational workers from saturation.

However, rate limiting introduces an inherent tension: how do you protect internal infrastructure without prematurely dropping legitimate requests or degrading client experience?

A naive rate limiting implementation can easily create severe operational problems:

• False rejections during normal mobile app startup bursts;
• Concurrency race conditions across distributed gateway nodes;
• Latency spikes caused by unoptimized datastore round-trips;
• Security vulnerabilities stemming from client IP spoofing;
• Complete API outages caused by an unhandled failure in the rate-limiting datastore.

This guide analyzes API rate limiting from a production engineering perspective: the mathematical mechanics of standard algorithms, distributed concurrency patterns in Redis, client identification security, failure handling modes, HTTP standards compliance, and a self-contained, type-safe implementation.

---

1. Core Rate Limiting Algorithms

Rate limiting algorithms differ in their memory overhead, burst tolerance, computational complexity, and computational precision. Selecting the correct algorithm requires understanding how each balances traffic smoothing against client flexibility.

┌─────────────────────────┐
│     Incoming Request    │
└────────────┬────────────┘
             │
             ▼
   [ Rate Limiting Engine ]
             │
   ┌─────────┴─────────┐
   │ Admitted          │ Rejected (Quota Exceeded)
   ▼                   ▼
[ Upstream Handlers ] [ HTTP 429 Too Many Requests ]
                      [ Retry-After & RateLimit Headers ]

1.1 Fixed Window Counter

The Fixed Window Counter partitions time into static, discrete intervals of duration $W$ (such as 60 seconds). A single counter tracks request volume for each unique client key within that window. When a request arrives, the system maps the current timestamp to the corresponding window bucket and increments the counter. If the counter exceeds the allocated threshold $L$, subsequent requests within that same window are rejected. At the conclusion of the interval, the counter resets to zero.

• Time Complexity: $O(1)$ per request check and increment.
• Space Complexity: $O(1)$ per client key (a single integer counter).

The $2\times$ Boundary Burst Vulnerability

The fundamental limitation of the Fixed Window algorithm is its vulnerability to boundary traffic spikes. Because windows reset at fixed intervals, a client can concentrate traffic immediately before and after the window boundary without exceeding the quota of either individual window.

Consider a rate limit configured for 100 requests per minute:

Window 1 [00:00 - 01:00]          Window 2 [01:00 - 02:00]
────────────────────────────┬────────────────────────────
... Idle ... 100 reqs       │ 100 reqs ... Idle ...
             [00:59]        │ [01:01]
             ───────────────┴───────────────
             200 requests within a 2-second span!

1. In Window 1, the client stays idle until 00:59, then transmits 100 requests.
2. In Window 2, beginning at 01:00, the counter resets. At 01:01, the client transmits another 100 requests.

Both windows record exactly 100 requests, satisfying their independent quotas. However, the downstream service was forced to absorb 200 requests within a two-second interval—twice the intended maximum rate. If the service's downstream database or third-party dependencies are sized strictly for 100 requests per minute, this $2\times$ burst can cause thread starvation and cascading timeouts.

---

1.2 Sliding Window Log

The Sliding Window Log eliminates boundary bursts by maintaining an exact historical log of request timestamps for each client key, typically implemented using a sorted set (such as a Redis ZSET).

When a request arrives at timestamp $t$:

1. Remove all entries with timestamps older than $t - W$ (e.g., via ZREMRANGEBYSCORE).
2. Count the number of remaining elements in the set (e.g., via ZCARD).
3. If the count is less than the limit $L$, insert the current timestamp $t$ into the set (e.g., via ZADD) and allow the request.
4. If the count equals or exceeds $L$, reject the request without inserting $t$.

• Time Complexity: $O(\log N + M)$ where $N$ is the number of timestamps in the sorted set and $M$ is the number of expired timestamps removed.
• Space Complexity: $O(N)$ where $N$ is the total request volume within the active window.

Trade-Off: Precision vs. Memory Explosion

While the Sliding Window Log provides mathematical accuracy and prevents boundary bursts, its memory footprint scales linearly with request throughput.

If an API gateway enforces a limit of 10,000 requests per hour for 50,000 active client keys, storing 64-bit Unix timestamps alongside sorted set metadata in memory consumes tens of megabytes per key. Under heavy traffic or denial-of-service conditions, this memory overhead can exhaust cache instance capacity and degrade Redis throughput.

---

1.3 Sliding Window Counter (Approximation)

The Sliding Window Counter combines the bounded memory footprint of the Fixed Window algorithm with the boundary smoothing of the Sliding Window Log. Rather than logging individual timestamps, it stores only two integer counters per client: one for the current fixed window and one for the previous fixed window.

When a request arrives at timestamp $t$ during the current window:

1. Determine the proportion of the current window that has elapsed: $\text{Progress} = \frac{\text{TimeElapsed}_{\text{current}}}{W}$
2. Calculate the overlap factor for the previous window: $\text{Weight}_{\text{previous}} = 1 - \text{Progress}$
3. Estimate the total request volume across the rolling window using a weighted sum: $\text{Estimated Count} = \text{Count}_{\text{current}} + \lfloor \text{Count}_{\text{previous}} \times \text{Weight}_{\text{previous}} \rfloor$
4. If $\text{Estimated Count} + 1 \le L$, increment $\text{Count}_{\text{current}}$ and admit the request. Otherwise, reject.

Previous Window (Count = 80)     Current Window (Count = 30)
[═══════════════════════════════][═══════════                   ]
                     ◄──────── Rolling 60s Window ────────►
                     Overlap: 30%                         Elapsed: 70%
                     
Estimated Count = 30 + (80 * (1 - 0.70)) = 30 + 24 = 54 requests

• Time Complexity: $O(1)$ arithmetic operations and key lookups.
• Space Complexity: $O(1)$ per client key (two integer counters).

Mathematical Assumptions and Error Bounds

The algorithm assumes that requests in the previous window were distributed uniformly across its duration. Under realistic web traffic, this assumption introduces a negligible error margin:

• If previous-window traffic was evenly distributed, the calculated count is exact.
• Under worst-case synthetic traffic—where all requests in the previous window occurred in its final millisecond—the algorithm underestimates the rolling total by a bounded margin (empirically within $5\%$ for standard web workloads).

Because it eliminates the catastrophic $2\times$ boundary spikes of fixed windows while requiring only $O(1)$ memory, the Sliding Window Counter is widely regarded as the industry standard for general API rate limiting (utilized by infrastructure providers such as Cloudflare).

---

1.4 Token Bucket

The Token Bucket algorithm models rate limiting around a token repository with a maximum capacity $B$ (burst allowance) that continuously refills at a sustained rate $r$ (tokens per second).

Each incoming request requires $k$ tokens (typically $k = 1$). If the bucket holds at least $k$ tokens, $k$ tokens are deducted and the request is processed immediately. If fewer than $k$ tokens are available, the request is rejected or placed in a queue.

Lazy Evaluation Mechanics

Running an asynchronous background timer or daemon to increment token counters for millions of active keys would waste substantial CPU cycles. Instead, production systems compute token replenishment lazily upon request arrival:

$\text{Replenished Tokens} = \lfloor (t_{\text{current}} - t_{\text{last}}) \times r \rfloor$ $\text{Tokens}_{\text{available}} = \min(B, \text{Tokens}_{\text{last}} + \text{Replenished Tokens})$

Where:

• $t_{\text{current}}$ is the current system timestamp.
• $t_{\text{last}}$ is the timestamp of the last recorded request.
• $r$ is the constant replenishment rate.
• $B$ is the maximum bucket capacity.

If $\text{Tokens}_{\text{available}} \ge k$, the state updates to $\text{Tokens}_{\text{available}} - k$ with timestamp $t_{\text{current}}$.

• Time Complexity: $O(1)$ per evaluation.
• Space Complexity: $O(1)$ per client key (storing token count and last timestamp).
• Behavior: Permits short, controlled bursts up to capacity $B$ while strictly enforcing the long-term consumption rate $r$.

---

1.5 Leaky Bucket

The Leaky Bucket algorithm models incoming requests as liquid poured into a bucket with a perforated base. Regardless of the arrival rate or burstiness of incoming requests, liquid leaks out of the base at a strictly constant rate $r$.

1. Requests enter a FIFO buffer (bucket) of fixed capacity $B$.
2. If the buffer is full, newly arriving requests overflow and are dropped immediately.
3. A background process or scheduler dispatches requests from the buffer at the fixed outflow rate $r$.

Token Bucket vs. Leaky Bucket: Traffic Smoothing vs. Burst Allowance

While often conflated, Token Bucket and Leaky Bucket serve different architectural purposes:

• Token Bucket allows requests to proceed immediately without delay as long as tokens are available. It accommodates bursty traffic patterns (such as a web page loading multiple API assets concurrently) while capping sustained consumption.
• Leaky Bucket flattens bursts into an unyielding, constant outbound flow. It introduces queuing delay to smooth traffic spikes.

Leaky Bucket is particularly suited for egress rate limiting, such as dispatching webhooks or calling third-party payment gateways with strict concurrency thresholds.

---

Algorithm Comparison Matrix

| Algorithm | Time Complexity | Space Complexity | Burst Tolerance | Boundary Spike Vulnerability | Ideal Production Context | | :--- | :--- | :--- | :--- | :--- | :--- | | Fixed Window | $O(1)$ | $O(1)$ (1 counter) | Low | High ($2\times$ boundary burst) | Simple internal tools, coarse daily/monthly quotas. | | Sliding Window Log | $O(\log N)$ | $O(N)$ (all timestamps) | Zero (exact) | None (exact calculation) | Low-volume, high-security endpoints (e.g. password resets). | | Sliding Window Counter | $O(1)$ | $O(1)$ (2 counters) | Low to Medium | Low (bounded error $\le 5\%$) | High-throughput public and tenant-facing APIs. | | Token Bucket | $O(1)$ | $O(1)$ (tokens + time) | High (up to capacity $B$) | None (bounded by $B$ and $r$) | User-facing REST/GraphQL APIs permitting bursty actions. | | Leaky Bucket | $O(1)$ | $O(B)$ (queue size) | None (smooths all bursts) | None (constant output rate) | Egress queue dispatching, legacy database protection. |

---

2. Distributed Concurrency and State Synchronization

In a modern horizontal architecture, client requests are distributed across multiple API gateway instances behind an L4/L7 load balancer. Local, in-memory counters on individual gateway nodes cannot enforce global quotas: if a client's requests hit ten different nodes, an in-memory limit of 100 requests per minute could allow the client to execute 1,000 requests per minute.

Enforcing consistent global limits requires a shared, low-latency datastore. Redis is the standard choice due to its in-memory data structures, sub-millisecond execution times, and atomic scripting capabilities.

                 ┌──────────────────┐
                 │  Load Balancer   │
                 └────────┬─────────┘
          ┌───────────────┼───────────────┐
          ▼               ▼               ▼
   ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
   │ API Node 1  │ │ API Node 2  │ │ API Node 3  │
   └──────┬──────┘ └──────┬──────┘ └──────┬──────┘
          │               │               │
          └───────────────┼───────────────┘
                          ▼  Atomic Lua Execution (EVALSHA)
                 ┌──────────────────┐
                 │   Redis Cluster  │
                 │ (Sliding Window) │
                 └──────────────────┘

2.1 The Check-Then-Set (TOCTOU) Concurrency Hazard

A common distributed systems pitfall is executing rate limit checks via separate read and write commands from the application layer:

// ANTI-PATTERN: Prone to race conditions under concurrency
const current = await redis.get(`ratelimit:${clientId}`);
if (parseInt(current || "0", 10) < LIMIT) {
  // Concurrency window: Another node can read the same value here!
  await redis.incr(`ratelimit:${clientId}`);
  return allow();
}
return reject();

If Client A sends 20 concurrent requests across multiple nodes when its counter is at 99 (limit: 100), all 20 nodes may read current = 99 simultaneously. Each node evaluates 99 < 100 as true, increments the key, and approves the request. The client successfully bypasses the rate limit by $20\%$.

2.2 Atomic Execution via Redis Lua Scripts

To prevent Time-of-Check to Time-of-Use (TOCTOU) race conditions, the read, calculation, increment, and expiration must execute atomically.

Redis executes Lua scripts as a single atomic operation within its single-threaded execution context for a given shard. While a script executes, no other command or script can interleave.

Here is a production-grade Lua script implementing an atomic Sliding Window Counter:

-- KEYS[1]: Current window key (e.g., "rl:usr_101:win_1715697600")
-- KEYS[2]: Previous window key (e.g., "rl:usr_101:win_1715697540")
-- ARGV[1]: Rate limit threshold (integer)
-- ARGV[2]: Previous window weight scaled by 10000 (integer: (1 - elapsed/window) * 10000)
-- ARGV[3]: Key TTL in seconds (integer: window_size * 2)

local current_key = KEYS[1]
local prev_key = KEYS[2]
local limit = tonumber(ARGV[1])
local weight_scaled = tonumber(ARGV[2])
local ttl = tonumber(ARGV[3])

-- 1. Read counters atomically
local current_count = tonumber(redis.call('GET', current_key) or '0')
local prev_count = tonumber(redis.call('GET', prev_key) or '0')

-- 2. Compute sliding window approximation
local estimated = current_count + math.floor((prev_count * weight_scaled) / 10000)

-- 3. Evaluate threshold
if estimated < limit then
    local new_count = redis.call('INCR', current_key)
    if new_count == 1 then
        redis.call('EXPIRE', current_key, ttl)
    end
    -- Return: 1 (allowed), remaining tokens, estimated count
    return {1, limit - (estimated + 1), estimated + 1}
else
    -- Return: 0 (rejected), 0 remaining, estimated count
    return {0, 0, estimated}
end

Optimization: Using EVALSHA in Production

Sending the entire Lua script text over the network on every HTTP request consumes unnecessary bandwidth and CPU. Production clients pre-load the script into Redis using SCRIPT LOAD, obtain its SHA-1 hash, and subsequently execute it using EVALSHA <sha1>. If Redis returns a NOSCRIPT error (such as following a cluster failover or restart), the client falls back to EVAL once to re-cache the script.

2.3 Key TTL Discipline and Memory Reclamation

In a system processing millions of unique IP addresses or API keys, rate limit keys must have deterministic Time-To-Live (TTL) values. Without TTLs, abandoned keys permanently occupy Redis memory, eventually triggering memory limits or volatile evictions.

• For fixed and sliding window counters, set the key TTL to twice the window duration ($2 \times W$). This ensures the previous window key remains available for weighted calculations during the subsequent window, but is automatically garbage-collected shortly thereafter.
• Never issue INCR without ensuring an EXPIRE is set. As shown in the Lua script above, check if new_count == 1 to atomically bind the TTL upon key creation.

---

3. Client Identification and Trust Boundaries

A rate limiter is only as reliable as the identifier it keys against. In production, choosing and extracting that identifier introduces subtle security vulnerabilities and architectural trade-offs.

3.1 The IP Address Vulnerability: Header Spoofing

Public APIs frequently rate limit unauthenticated requests by client IP address. However, extracting client IP addresses in cloud environments is fraught with risk.

When traffic traverses reverse proxies, CDNs (e.g., Cloudflare), or load balancers (e.g., AWS ALB), the client's original TCP source IP is appended to the X-Forwarded-For HTTP header.

Attacker Client
  │ Sends header: "X-Forwarded-For: 203.0.113.1"
  ▼
Edge Proxy (e.g., Cloudflare IP: 198.41.128.1)
  │ Appends real TCP IP (198.51.100.25)
  ▼
API Gateway
  │ Reads: "X-Forwarded-For: 203.0.113.1, 198.51.100.25"

If the API gateway naively reads the first entry in X-Forwarded-For (req.headers['x-forwarded-for'].split(',')[0]), an attacker can bypass all rate limits by injecting arbitrary, spoofed IP addresses with each outgoing request:

# Spoofing an arbitrary IP on each request
curl -H "X-Forwarded-For: 192.0.2.$(($RANDOM % 255))" https://api.example.com/v1/data

Mitigating Header Spoofing

1. Verify Immediate TCP Socket Peer: Only inspect X-Forwarded-For or vendor headers (CF-Connecting-IP, True-Client-IP) if the immediate upstream TCP connection originates from a trusted reverse proxy CIDR block.
2. Right-to-Left Parsing: When traversing X-Forwarded-For, parse addresses from right to left, discarding hops corresponding to known internal proxies. The first unverified IP encountered represents the authentic client connection.

3.2 The Shared IP Pitfall (CGNAT and Corporate VPNs)

IP-based rate limiting inherently struggles with shared network infrastructure:

• Carrier-Grade NAT (CGNAT): Mobile telecommunication providers often map thousands of cellular mobile devices to a handful of public IPv4 addresses.
• Corporate Firewalls and VPNs: Entire enterprise campuses route employee traffic through centralized egress proxies.
• Cloud NAT Gateways: Developers testing integrations from cloud instances (e.g., AWS Lambda or GitHub Actions) share egress IP pools.

Imposing aggressive IP-based rate limits on endpoints will inevitably produce false positives for legitimate users sharing egress infrastructure. Where possible:

• Reserve IP-based rate limiting for coarse edge DDoS mitigation and unauthenticated authentication endpoints (login, forgot password).
• Transition to authenticated identity tokens immediately after authentication.

3.3 Authenticated Identifiers and Cryptographic Hashing

For authenticated traffic, rate limits should key against:

• User ID (sub claim in JWTs)
• Organization / Tenant ID (for multi-tenant B2B architectures)
• API Key ID

Security Hygiene: Never Cache Raw API Keys

Never use raw client API keys as Redis cache keys (e.g., ratelimit:api_sec_live_98f7a2...). If Redis command logs, slow query logs, or memory dumps are compromised, sensitive client credentials will be exposed in plaintext.

Always compute a fast cryptographic hash of the key before querying the datastore:

import { createHash } from "node:crypto";

function getRateLimitKey(tenantId: string, apiKey: string): string {
  const hashedKey = createHash("sha256").update(apiKey).digest("hex").slice(0, 16);
  return `rl:tenant:${tenantId}:key:${hashedKey}`;
}

3.4 Multi-Tiered Rate Limiting

Robust systems rarely apply a single monolithic rate limit. Instead, they enforce limits across multiple hierarchical dimensions:

Incoming Request
  │
  ├─► [ Tier 1: Global Edge Limit ]   (e.g., 2,000 req/min per IP - stops L7 floods)
  │
  ├─► [ Tier 2: Tenant Quota ]        (e.g., 500 req/min per Org ID - contract quota)
  │
  └─► [ Tier 3: Operation Cost ]      (e.g., 10 req/min for POST /v1/reports/export)

1. Tier 1 (Edge/IP): Coarse rate limiting at the API gateway or CDN to prevent infrastructure exhaustion.
2. Tier 2 (Tenant/User): Enforces business contractual tiers (Free vs. Pro vs. Enterprise) to ensure fair resource allocation across tenants.
3. Tier 3 (Cost-Weighted Endpoints): Assigns token costs proportional to computational complexity. A lightweight GET /health may cost 1 token, while an expensive POST /ai/generate or large database aggregation may consume 25 tokens.

---

4. Resilience and Failure Handling: Fail-Open vs. Fail-Closed

In any production architecture, the rate limiting datastore (Redis) is a dependency that can experience network partitions, cluster failovers, connection pool exhaustion, or severe latency degradation.

The critical architectural question is: how should the API gateway behave when the rate limiter fails?

                           [ Rate Limiter Engine ]
                                      │
                         Redis Outage / Network Partition?
                                      │
                     ┌────────────────┴────────────────┐
                     ▼                                 ▼
             [ Fail-Open Mode ]               [ Fail-Closed Mode ]
                     │                                 │
     Allow Request to Pass Downstream          Reject Request (HTTP 503/429)
     Priority: Service Availability            Priority: Protect Fragile Core
     Risk: Downstream Overload                 Risk: Outage on Cache Failure

4.1 Fail-Open (Prioritizing Availability)

Under a Fail-Open policy, if the rate limiter encounters an error or timeout communicating with Redis, it catches the exception, logs a high-severity alert, and allows the client request to proceed to the application logic.

• Advantage: A transient Redis failure does not trigger a cascading outage for end users. Availability of core services remains intact.
• Risk: If the Redis failure coincides with a volumetric DDoS attack or scraper surge, the unthrottled traffic directly hits downstream relational databases and application servers, potentially causing systemic collapse.
• Ideal Context: Standard read-heavy web applications, e-commerce storefronts, and customer-facing consumer apps.

4.2 Fail-Closed (Prioritizing Infrastructure Integrity)

Under a Fail-Closed policy, if the rate limiter cannot verify quota, it rejects the incoming request with an HTTP 503 Service Unavailable or HTTP 429 Too Many Requests.

• Advantage: Protects fragile downstream dependencies (such as legacy core banking systems or resource-constrained relational databases) from being overwhelmed.
• Risk: Converts a non-fatal cache degradation into a total platform outage.
• Ideal Context: High-cost compute endpoints, LLM inference APIs, financial transaction processors, and services where downstream failure carries catastrophic data corruption risks.

4.3 Adaptive Resilience: Circuit Breaker with Local Fallback

A mature production architecture does not treat fail-open and fail-closed as static constants. Instead, it incorporates a circuit breaker with local in-memory degradation:

1. Normal State (Circuit Closed): The gateway executes atomic checks against the shared Redis cluster.
2. Degraded State (Circuit Open): If Redis client timeouts exceed a configured error threshold (e.g., 5 consecutive timeouts or error rate $> 15\%$), the circuit trips.
3. Local Fallback: Rather than allowing completely unconstrained traffic, the gateway degrades to local in-process memory rate limiting (using an LRU cache) with scaled-down quotas per instance, while asynchronously probing Redis health in the background.

This prevents the gateway from adding a 2,000ms Redis connection timeout to every incoming HTTP request during an infrastructure incident.

---

5. HTTP Standards: Status Codes and Response Headers

When an API enforces rate limits, communicating quota state transparently enables well-behaved client libraries to throttle their requests before encountering errors.

5.1 The Standard Response: 429 Too Many Requests

RFC 6585 (Section 4) defines HTTP status code 429 specifically for rate limiting:

"The 429 status code indicates that the user has sent too many requests in a given amount of time ('rate limiting'). The response representations SHOULD include details explaining the condition, and MAY include a Retry-After header indicating how long to wait before making a new request."

Avoid using generic error codes such as 400 Bad Request or 403 Forbidden for rate limit events. HTTP 403 signals an authorization deficiency that will not change upon retrying, whereas HTTP 429 signals a transient condition that resolves over time.

5.2 Standard RateLimit Headers (IETF Draft)

Historically, API providers invented inconsistent custom headers (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset).

The IETF HTTP API Working Group has standardized rate limiting headers under draft-ietf-httpapi-ratelimit-headers. Production APIs should prioritize these standard headers:

HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 37
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 37
RateLimit-Policy: 100;w=60

{
  "type": "https://api.example.com/errors/rate-limit-exceeded",
  "title": "Too Many Requests",
  "status": 429,
  "detail": "Quota exceeded. Maximum allowed is 100 requests per 60 seconds.",
  "instance": "/v1/reports",
  "retry_after_seconds": 37
}

• RateLimit-Limit: The maximum request quota allocated in the current window (e.g., 100).
• RateLimit-Remaining: The number of requests remaining in the active window (e.g., 0).
• RateLimit-Reset: The number of seconds remaining until the quota window resets (e.g., 37).
• RateLimit-Policy: Expresses quota policy (e.g., 100;w=60 means 100 requests per 60-second window).
• Retry-After (RFC 7231): Communicates the exact wait duration in delta-seconds before a retry can be attempted.

---

6. Type-Safe TypeScript Implementation

Below is a self-contained, production-grade TypeScript implementation of a distributed rate limiting engine. It features an atomic Sliding Window Counter, an in-memory fallback store simulating atomic operations, a circuit breaker for resilience, and automated RFC-compliant header generation.

import { createHash } from "node:crypto";

/**
 * Configuration options for the Rate Limiter
 */
export interface RateLimitOptions {
  windowSeconds: number;
  maxRequests: number;
  failOpen: boolean;
  keyPrefix?: string;
  circuitBreakerThreshold?: number;
}

/**
 * Standardized evaluation result returned by the limiter
 */
export interface RateLimitResult {
  allowed: boolean;
  limit: number;
  remaining: number;
  resetSeconds: number;
  retryAfterSeconds?: number;
  headers: Record<string, string>;
}

/**
 * Storage interface matching atomic Redis Lua execution semantics
 */
export interface IRateLimitStore {
  evalSlidingWindow(
    currentKey: string,
    previousKey: string,
    limit: number,
    weightScaled: number,
    ttlSeconds: number
  ): Promise<[allowed: number, remaining: number, count: number]>;
}

/**
 * In-Memory atomic store implementation (ideal for testing, single-node,
 * or degraded circuit-breaker fallback)
 */
export class InMemoryRateLimitStore implements IRateLimitStore {
  private counters = new Map<string, { count: number; expiresAt: number }>();

  async evalSlidingWindow(
    currentKey: string,
    previousKey: string,
    limit: number,
    weightScaled: number,
    ttlSeconds: number
  ): Promise<[allowed: number, remaining: number, count: number]> {
    const now = Date.now();
    this.reapExpired(now);

    const currentEntry = this.counters.get(currentKey);
    const prevEntry = this.counters.get(previousKey);

    const currentCount = currentEntry && currentEntry.expiresAt > now ? currentEntry.count : 0;
    const prevCount = prevEntry && prevEntry.expiresAt > now ? prevEntry.count : 0;

    const estimated = currentCount + Math.floor((prevCount * weightScaled) / 10000);

    if (estimated < limit) {
      const newCount = currentCount + 1;
      const expiresAt = currentEntry ? currentEntry.expiresAt : now + ttlSeconds * 1000;
      this.counters.set(currentKey, { count: newCount, expiresAt });

      return [1, Math.max(0, limit - (estimated + 1)), estimated + 1];
    }

    return [0, 0, estimated];
  }

  private reapExpired(now: number): void {
    for (const [key, val] of this.counters.entries()) {
      if (val.expiresAt <= now) {
        this.counters.delete(key);
      }
    }
  }
}

/**
 * Production Rate Limiting Engine with Circuit Breaker and Header Builder
 */
export class RateLimiter {
  private store: IRateLimitStore;
  private options: Required<RateLimitOptions>;
  private consecutiveFailures = 0;
  private circuitOpenUntil = 0;

  constructor(store: IRateLimitStore, options: RateLimitOptions) {
    this.store = store;
    this.options = {
      windowSeconds: options.windowSeconds,
      maxRequests: options.maxRequests,
      failOpen: options.failOpen,
      keyPrefix: options.keyPrefix ?? "rl",
      circuitBreakerThreshold: options.circuitBreakerThreshold ?? 5,
    };
  }

  /**
   * Consistently hashes identity to prevent key leakage and collisions
   */
  public generateKey(identifier: string): string {
    const hash = createHash("sha256").update(identifier).digest("hex").slice(0, 16);
    return `${this.options.keyPrefix}:${hash}`;
  }

  /**
   * Evaluates request quota against the atomic sliding window
   */
  public async check(identifier: string): Promise<RateLimitResult> {
    const nowSeconds = Math.floor(Date.now() / 1000);
    const windowSize = this.options.windowSeconds;

    const currentWindowIndex = Math.floor(nowSeconds / windowSize);
    const prevWindowIndex = currentWindowIndex - 1;

    const elapsedInCurrentWindow = nowSeconds % windowSize;
    const weightScaled = Math.floor(
      ((windowSize - elapsedInCurrentWindow) / windowSize) * 10000
    );
    const resetSeconds = Math.max(1, windowSize - elapsedInCurrentWindow);

    const baseKey = this.generateKey(identifier);
    const currentKey = `${baseKey}:win_${currentWindowIndex}`;
    const previousKey = `${baseKey}:win_${prevWindowIndex}`;
    const ttl = windowSize * 2;

    // Check circuit breaker status
    if (this.isCircuitOpen()) {
      return this.handleFallback(resetSeconds, "Circuit breaker tripped");
    }

    try {
      const [allowedNum, remaining] = await this.store.evalSlidingWindow(
        currentKey,
        previousKey,
        this.options.maxRequests,
        weightScaled,
        ttl
      );

      // Reset failure counter on successful execution
      this.consecutiveFailures = 0;

      const allowed = allowedNum === 1;
      return this.buildResult(allowed, remaining, resetSeconds);
    } catch (error) {
      this.recordFailure();
      return this.handleFallback(resetSeconds, error instanceof Error ? error.message : "Unknown error");
    }
  }

  private isCircuitOpen(): boolean {
    return Date.now() < this.circuitOpenUntil;
  }

  private recordFailure(): void {
    this.consecutiveFailures++;
    if (this.consecutiveFailures >= this.options.circuitBreakerThreshold) {
      // Trip circuit for 30 seconds
      this.circuitOpenUntil = Date.now() + 30_000;
    }
  }

  private handleFallback(resetSeconds: number, reason: string): RateLimitResult {
    if (this.options.failOpen) {
      return {
        allowed: true,
        limit: this.options.maxRequests,
        remaining: 1,
        resetSeconds,
        headers: {
          "RateLimit-Limit": String(this.options.maxRequests),
          "RateLimit-Remaining": "1",
          "RateLimit-Reset": String(resetSeconds),
          "X-RateLimit-Fallback": `fail-open (${reason})`,
        },
      };
    }

    // Fail-Closed
    return {
      allowed: false,
      limit: this.options.maxRequests,
      remaining: 0,
      resetSeconds,
      retryAfterSeconds: resetSeconds,
      headers: {
        "Retry-After": String(resetSeconds),
        "RateLimit-Limit": String(this.options.maxRequests),
        "RateLimit-Remaining": "0",
        "RateLimit-Reset": String(resetSeconds),
        "X-RateLimit-Fallback": `fail-closed (${reason})`,
      },
    };
  }

  private buildResult(allowed: boolean, remaining: number, resetSeconds: number): RateLimitResult {
    const headers: Record<string, string> = {
      "RateLimit-Limit": String(this.options.maxRequests),
      "RateLimit-Remaining": String(remaining),
      "RateLimit-Reset": String(resetSeconds),
      "RateLimit-Policy": `${this.options.maxRequests};w=${this.options.windowSeconds}`,
      // Legacy compatibility headers
      "X-RateLimit-Limit": String(this.options.maxRequests),
      "X-RateLimit-Remaining": String(remaining),
      "X-RateLimit-Reset": String(Math.floor(Date.now() / 1000) + resetSeconds),
    };

    if (!allowed) {
      headers["Retry-After"] = String(resetSeconds);
    }

    return {
      allowed,
      limit: this.options.maxRequests,
      remaining,
      resetSeconds,
      retryAfterSeconds: allowed ? undefined : resetSeconds,
      headers,
    };
  }
}

---

7. Client IP Resolution: Secure Header Parsing

To prevent X-Forwarded-For injection attacks, application servers must parse proxy headers cautiously. The following helper extracts client IPs securely by validating proxy trust:

import type { IncomingMessage } from "node:http";

export interface ProxyTrustConfig {
  /**
   * Number of trusted reverse proxy hops between client and application server
   */
  trustedHopCount: number;
  /**
   * Set of verified proxy IP addresses
   */
  trustedProxyIps: Set<string>;
}

/**
 * Safely resolves the authentic client IP address by parsing proxy headers from right to left
 */
export function resolveClientIp(
  req: IncomingMessage,
  config: ProxyTrustConfig
): string {
  const socketIp = req.socket.remoteAddress || "127.0.0.1";

  // If the immediate connection does not originate from a trusted proxy,
  // do not trust any forwarded headers
  if (!config.trustedProxyIps.has(socketIp)) {
    return socketIp;
  }

  const headerValue = req.headers["x-forwarded-for"];
  if (!headerValue) {
    return socketIp;
  }

  const rawHops = Array.isArray(headerValue) ? headerValue.join(",") : headerValue;
  const hops = rawHops
    .split(",")
    .map((ip) => ip.trim())
    .filter((ip) => ip.length > 0);

  if (hops.length === 0) {
    return socketIp;
  }

  // Parse from right to left: skip configured trusted hops
  const targetIndex = hops.length - config.trustedHopCount;
  if (targetIndex >= 0 && targetIndex < hops.length) {
    return hops[targetIndex];
  }

  // Fallback to the first hop if fewer hops exist than configured
  return hops[0];
}

---

8. Production Verification and Operational Checklist

Before promoting rate limiting infrastructure to production, verify each of the following operational dimensions:

8.1 Verification Checklist

• [ ] Concurrency Testing (Race Conditions): Execute synthetic load tests transmitting 100 concurrent requests within a 5ms window against a client key configured with a limit of 10. Confirm that exactly 10 requests succeed and 90 are rejected.
• [ ] Boundary Burst Testing: In sliding window implementations, simulate peak load at $t = W - 1\text{s}$ and $t = W + 1\text{s}$. Confirm that combined throughput across the boundary does not exceed the allowed threshold.
• [ ] Proxy Header Spoofing Drill: Send curl requests with injected X-Forwarded-For: 1.2.3.4 headers. Confirm that the rate limiter evaluates the authentic upstream client IP rather than the spoofed value.
• [ ] Failover Drill (Chaos Testing): Sever the network connection to the Redis rate limiting shard. Verify that the gateway triggers the configured fail-open or fail-closed policy without hanging client request connections.
• [ ] Clock Drift Synchronization: Ensure all API gateway instances and Redis nodes maintain Network Time Protocol (NTP) synchronization within $\pm 20\text{ms}$. Significant clock drift can distort window index calculations.
• [ ] Redis Memory & Eviction Policy: Verify that Redis instances running rate limiters use an appropriate maxmemory eviction policy:
  • Prefer volatile-ttl or volatile-lru so transient rate limit keys with defined TTLs can be evicted under memory pressure without dropping permanent data.
  • Set explicit TTLs on every created key.
• [ ] Standard Header Conformance: Validate that all rejected responses return HTTP 429 with Retry-After, RateLimit-Limit, RateLimit-Remaining, and RateLimit-Reset headers.

---

Conclusion: Strategic Rate Limiting as Architecture

Rate limiting is not merely an operational utility for blocking malicious actors; it is a fundamental architectural mechanism for ensuring predictable performance, system stability, and fair multi-tenant resource sharing.

When designing rate limiting for modern distributed APIs:

1. Match the Algorithm to the Traffic Pattern: Use the Sliding Window Counter for general-purpose API protection to balance $O(1)$ memory efficiency with boundary burst protection. Use the Token Bucket when applications require burst tolerance, and Leaky Bucket for strict egress traffic shaping.
2. Enforce Atomicity at the Data Layer: Never implement check-then-set logic across separate application round-trips. Use atomic Redis Lua scripts (EVALSHA) to prevent concurrency race conditions under parallel load.
3. Secure the Trust Boundary: Never trust client-supplied IP headers blindly. Parse headers from right to left against verified proxy hops, and prefer cryptographically hashed API keys for authenticated endpoints.
4. Plan for Datastore Degradation: Choose an explicit fail-open or fail-closed strategy, and back it with circuit breakers to prevent datastore hiccups from escalating into customer-facing platform outages.
5. Honor Web Standards: Provide transparency through HTTP 429 Too Many Requests, Retry-After, and standard RateLimit-* headers to allow legitimate API clients to self-regulate cleanly.