Build a Stateless MCP Gateway Workflow with Header-Based Routing & Cacheable Tool Lists in 2026

By Deepak Bagada, CEO at SaaSNext & Principal AI Architect.

The 2026-07-28 Model Context Protocol specification ended the era of the session. MCP went from a bidirectional stateful protocol to a request/response stateless core: every request is self-describing, method and tool names travel in the Mcp-Method and Mcp-Name HTTP headers, list responses carry cache hints, and any request can land on any server instance behind a plain round-robin load balancer. That single change turns MCP from something you babysit into something you can operate like the rest of the web — and it is the difference between a fleet of MCP servers and an actual MCP platform. The protocol's maintainers report nearly half a billion SDK downloads a month, with the TypeScript and Python SDKs each crossing a billion total downloads, because statelessness is what production teams asked for.

This guide builds the piece that makes a stateless MCP fleet work: a stateless MCP gateway workflow that routes tools/call requests by header to the right server pool, serves cached tools/list catalogs with ttlMs and cacheScope hints, handles multi-round-trip requests (MRTR) where a tool needs an answer before it can act, and meters every call for rate limits and billing. It is the routing and caching layer for the servers catalogued in the MCP directory, and the operational patterns it encodes are part of our AI workflows library.

Architecture Overview

graph TD
  A[MCP Client] --> B[Round-Robin LB]
  B --> G[Stateless Gateway]
  G --> C{Header: Mcp-Method}
  C -- tools/list --> D[Catalog Cache]
  D --> G
  C -- tools/call --> E[Route by Mcp-Name]
  E --> P1[Server Pool A]
  E --> P2[Server Pool B]
  E --> P3[Server Pool C]
  P1 --> G
  P2 --> G
  P3 --> G
  G --> F[MRTR Handler: input_required]
  F --> G

A stateless gateway is a pure function from request to response: parse the Mcp-Method and Mcp-Name headers, decide the destination pool, apply cache and rate-limit policy, forward, and return. No session stickiness, no shared storage, no state to drain on deploy. That is the entire point — and it is why the same gateway can scale horizontally behind a load balancer without a single coordination point.

Part 1 — Configuration

.env

LISTEN_PORT=8443
UPSTREAM_POOLS=./pools.json
TOOLS_LIST_TTL_MS=60000
TOOLS_LIST_CACHE_SCOPE=shared
RATE_LIMIT_PER_NAME=100
MRTR_ENABLED=true
OAUTH_INTROSPECT_URL=https://auth.example.com/introspect
IDEMPOTENCY_TTL_SECONDS=300

pools.json

{
  "search": {"servers": ["https://mcp-a.internal:9443", "https://mcp-b.internal:9443"]},
  "docs":    {"servers": ["https://docs-mcp.internal:9443"]},
  "billing": {"servers": ["https://billing-mcp.internal:9443"], "rate_limit": 20}
}

The pools file is the routing table: every tool name maps to a pool of identical server instances. Because the protocol is stateless, any instance can serve any request in its pool — round-robin distribution just works. Pools can carry their own rate limits, which is how a billing tool gets a stricter budget than a search tool without any special-case code.

Part 2 — Routing by header

gateway.py

import os, json, httpx, time

async def route_request(headers: dict, body: dict) -> dict:
    method = headers.get("mcp-method", body.get("method"))
    name = headers.get("mcp-name") or (body.get("params") or {}).get("name")

    if method == "tools/list":
        return await serve_catalog(headers)
    if method == "tools/call":
        return await forward_call(name, headers, body)
    if method == "server/discover":
        return await discover(headers)
    return {"jsonrpc": "2.0", "id": body.get("id"),
            "error": {"code": -32601, "message": "method not found"}}

def pick_instance(pool_name: str) -> str:
    servers = POOLS[pool_name]["servers"]
    return servers[int(time.time()) % len(servers)]  # round-robin

async def forward_call(name: str, headers: dict, body: dict) -> dict:
    pool = POOLS.get(name) or POOLS.get(default_pool(name))
    if pool and not rate_ok(name):
        return {"jsonrpc": "2.0", "id": body.get("id"),
                "error": {"code": -429, "message": "rate limit exceeded"}}
    upstream = pick_instance(name)
    async with httpx.AsyncClient() as c:
        r = await c.post(f"{upstream}/mcp", headers=headers, json=body)
        return r.json()

Header-based routing is the spec's gift to operators: the gateway never parses the JSON body to decide where a request goes. Mcp-Method and Mcp-Name arrive as plain HTTP headers, so a WAF, a rate limiter, or an API gateway can route and meter on them directly — and the gateway itself stays a thin, fast passthrough. The round-robin instance picker needs no shared state, so the gateway scales by adding replicas.

Part 3 — Cacheable tool catalogs

cache.py

import time, json

_CATALOG = {}

def get_catalog(key: str):
    entry = _CATALOG.get(key)
    if entry and entry["expires"] > time.time():
        return entry["value"]
    return None

async def serve_catalog(headers: dict) -> dict:
    # Deterministic order + cache hints per SEP-2549
    cached = get_catalog("tools/list")
    if cached:
        return cached
    upstream = pick_instance("catalog")
    async with httpx.AsyncClient() as c:
        r = await c.post(f"{upstream}/mcp", headers=headers,
                         json={"jsonrpc": "2.0", "id": 1, "method": "tools/list"})
    result = r.json()
    result["result"]["_meta"] = {
        "ttlMs": int(os.environ["TOOLS_LIST_TTL_MS"]),
        "cacheScope": os.environ["TOOLS_LIST_CACHE_SCOPE"],
        "ordered": True,
    }
    _CATALOG["tools/list"] = {"value": result,
                               "expires": time.time() + int(os.environ["TOOLS_LIST_TTL_MS"]) / 1000}
    return result

Catalog caching is where stateless MCP gets its cost win. tools/list results now carry ttlMs and cacheScope hints and a deterministic order, so the gateway can serve the tool catalog from cache instead of fanning every tools/list to every server. A fleet of agents re-discovering tools on reconnect stops hammering upstream — the catalog is served in microseconds from the cache layer, and cacheScope: shared tells every client the same catalog is safe to reuse.

Part 4 — MRTR and the human-in-the-loop

mrtr.py

async def handle_mrtr(headers: dict, body: dict, server_response: dict) -> dict:
    # SEP-2322: server returns input_required with pending requests
    if server_response.get("result", {}).get("resultType") == "input_required":
        pending = server_response["result"]["requests"]
        # Route the elicitation to the human-facing channel (approval UI, chat, email)
        answers = await elicit(pending)          # blocks until human answers
        body["params"]["inputResponses"] = answers
        return await forward_call(
            headers.get("mcp-name") or body["params"]["name"], headers, body)
    return server_response

MRTR replaces the old held-open bidirectional stream for server-to-client requests. When a tool needs a confirmation mid-call — approve a payment, provide a missing parameter — the server returns resultType: "input_required" with the unanswered requests, and the client retries the original call with the answers attached in inputResponses. The gateway's job is to ferry that round trip: the elicit step is where the human approval UI plugs in, which is exactly the governance pattern we detail in our AI workflows library. Retry rules are explicit: MRTR round trips retry up to twice with backoff, and an idempotency key on the original request guarantees a retried approval is never applied twice.

Part 5 — Running the gateway

main.py

import os
from fastapi import FastAPI, Request, Response
from gateway import route_request

app = FastAPI()

@app.post("/mcp")
async def mcp_endpoint(request: Request):
    headers = {k.lower(): v for k, v in request.headers.items()}
    body = await request.json()
    result = await route_request(headers, body)
    return Response(content=json.dumps(result),
                    media_type="application/json")

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=int(os.environ["LISTEN_PORT"]))

The gateway is a single POST /mcp endpoint; everything else is policy. Deploy it as a stateless service, put it behind the load balancer, and scale replicas to meet demand — there is nothing to drain because there is no session state anywhere in the path. Upgrade your servers to the 2026-07-28 spec and the same gateway keeps working with the headers the new SDKs send.

Production checklist

1. Route on headers, never on body JSON. Mcp-Method and Mcp-Name keep the gateway fast and let edge infrastructure meter the traffic.
2. Cache tool catalogs with hints. Honor ttlMs/cacheScope; serve tools/list from cache with deterministic order.
3. Pools are the routing table. Every tool maps to a pool of identical stateless instances; round-robin needs no shared state.
4. MRTR is the approval channel. input_required round trips plug the human approval UI into the request path.
5. Retry with idempotency. Retry transient 5xx with backoff; never apply the same MRTR approval twice.

Frequently Asked Questions

Q: What does "stateless MCP" mean for operators?

A: The initialize handshake and Mcp-Session-Id are gone. Every request carries its protocol version, client identity, and capabilities, so any instance behind a load balancer can serve any request without shared storage — deploys become instant and scaling is trivial.

Q: How does the gateway know which server handles which tool?

A: The Mcp-Name header names the tool, and pools.json maps tool names to server pools. The gateway never parses the body to route; the headers are the routing table.

Q: Why are tool lists cacheable now?

A: tools/list responses carry ttlMs and cacheScope hints with deterministic ordering (SEP-2549), so clients and gateways can cache catalogs and stop re-fetching on every reconnect — a large cost win for agent fleets.

Q: How do approvals work without sessions?

A: Via MRTR: the server returns resultType: input_required with pending requests, the client (or gateway) gets the answers from the human, and retries the original call with inputResponses attached. No held-open stream needed.

Q: Does this gateway work with older MCP servers?

A: Yes. The spec gives a twelve-month deprecation window, so a stateless gateway can front mixed fleets while older servers migrate — but new deployments should target the 2026-07-28 spec directly.