By Deepak Bagada, CEO at SaaSNext & Principal AI Architect.

Build a Cerebras Fast-Inference MCP Server for AI Agents

On August 13, 2026, OpenAI previewed "Ultrafast," a service tier that flips the latency conversation on its head: a preview build of GPT-5.6 Sol answering at up to 750 output tokens per second — roughly 14x the Standard tier — with the model running entirely on Cerebras wafer-scale engines. The reason this matters to builders in Bengaluru, Pune, or the enterprise SaaS corridors of Delhi NCR is not the headline number. It is that latency has become a product feature. Streaming UI, voice agents, real-time copilots, and trading-style decision loops all amortise differently when the first token lands in well under a second and the last token arrives fourteen times sooner. The platform launched as a limited preview with no public pricing at announcement, but the design pattern it introduces — a fast tier that agents can route to, measure, and fall back from — is worth building against today.

This dispatch builds cerebras-fast-mcp: a production-grade Model Context Protocol server (FastMCP, Python) that exposes Cerebras / wafer-scale fast inference to agents as four governed tools — complete_fast, tokens_per_second, route_fast_vs_standard, and failover_standard. We cover the full build: a runnable server.py with tool decorators, the JSON inputSchema the model actually sees, the mcpServers config for Claude Desktop and Cursor, OAuth 2.0 / scoped-token security over a localhost transport, a per-request cost-and-jitter meter, and a retry table for the failure modes you will actually meet in production. For the surrounding ecosystem, the MCP Directory is the map; this guide is the build.

Why wafer-scale changed the latency calculus

The physics behind the preview matters more than the model name. A conventional GPU inference server spends a large share of every request moving weights from high-bandwidth memory into compute across a PCIe or NVLink boundary — the weight-transfer bottleneck. Cerebras engines sidestep it entirely: a wafer-scale engine (WSE) packs roughly 900,000 cores with 44GB of on-wafer SRAM onto a single silicon slab, so model weights live on the same die that computes on them. No weight shuttling, no interconnect stalls, no memory-bandwidth ceiling — just tokens leaving at wafer speed.

Cerebras already runs that architecture as a service for open models, and the numbers are agentic speed today: Llama 4 Maverick above 2,500 tokens per second, gpt-oss-120B above 2,700 TPS, and Gemma 4 around 1,851 TPS. That is the difference between an agent that "thinks" for ten seconds and one that completes a multi-step generation in under a second. When Ultrafast ships beyond preview, GPT-5.6-class models ride the same class of silicon at up to 750 TPS with a 14x latency spread over Standard. For latency-sensitive work, "the model I chose" matters less than "the tier I routed to" — and that is exactly the decision an MCP tool should own.

Why an MCP server is the right wrapper

Fast inference solves a throughput problem, not an integration problem. The integration problem is: which requests are allowed on the fast tier, who pays for them, what happens when the tier is throttled or down, and how do you prove the tier paid for itself. A raw API call answers none of those. An MCP server answers all of them, because:

• the model gets typed, governed tools instead of raw endpoints — it can only call complete_fast with the exact inputs the inputSchema allows;
• routing policy (fast vs standard), failover, and metering live in one auditable place instead of being scattered through prompt engineering;
• every agent client that speaks MCP — Claude Desktop, Cursor, VS Code, custom orchestrators — reuses the same server and the same policy;
• cost and jitter telemetry ride along with every response, so you can prove the 14x number is real on your workload, not just on the marketing slide.

The pattern is simple enough to reuse across providers: a fast tier, a standard tier, a router, and a failover — four tools, one policy. It also fits naturally into the multi-agent patterns catalogued in our Workflows section, where latency budgets are a first-class planning input.

The tool surface

Every tool returns telemetry, not just text. That telemetry is the difference between "fast" as a vendor claim and "fast" as a measured, reportable SLA.

Building the server (FastMCP, Python)

Create a project, declare one dependency group, and ship server.py.

[project]
name = "cerebras-fast-mcp"
version = "0.1.0"
description = "Wafer-scale fast inference as governed MCP tools"
requires-python = ">=3.11"
dependencies = [
  "mcp[cli]>=1.9.0",
  "httpx>=0.27.0",
]

[project.scripts]
cerebras-fast-mcp = "cerebras_fast_mcp.server:main"

Then the server itself:

# server.py - cerebras-fast-mcp: wafer-scale fast inference as governed MCP tools
import json
import os
import statistics
import time
from dataclasses import dataclass

import httpx
from mcp.server.fastmcp import FastMCP

FAST_URL = os.environ.get("CEREBRAS_FAST_URL", "https://api.cerebras.ai/v1/chat/completions")
FAST_API_KEY = os.environ.get("CEREBRAS_API_KEY")
STANDARD_URL = os.environ.get("STANDARD_URL", "https://api.openai.com/v1/chat/completions")
STANDARD_API_KEY = os.environ.get("STANDARD_API_KEY")
FAST_MODEL = os.environ.get("FAST_MODEL", "gpt-oss-120b")
STANDARD_MODEL = os.environ.get("STANDARD_MODEL", "gpt-5.6-standard")
TTFT_GUARD_MS = float(os.environ.get("TTFT_GUARD_MS", "1500"))
MAX_OUTPUT_TOKENS = int(os.environ.get("MAX_OUTPUT_TOKENS", "4096"))
RATE_PER_1K_OUT = float(os.environ.get("FAST_RATE_PER_1K", "0.012"))

mcp = FastMCP("cerebras-fast-inference")
client = httpx.AsyncClient(timeout=60.0)

@dataclass
class RequestMeter:
    model: str = ""
    ttft_ms: float = 0.0
    elapsed_ms: float = 0.0
    output_tokens: int = 0
    cost_usd: float = 0.0
    failed_over: bool = False

    def tokens_per_second(self) -> float:
        secs = self.elapsed_ms / 1000.0
        return self.output_tokens / secs if secs > 0 else 0.0

_jitter_hist: dict[str, list[float]] = {}

async def _stream(prompt: str, model: str, url: str, api_key: str) -> tuple[str, RequestMeter]:
    """Stream one completion, recording client-side TTFT, elapsed, output tokens and cost."""
    meter = RequestMeter(model=model)
    started = time.monotonic()
    payload = {
        "model": model,
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": MAX_OUTPUT_TOKENS,
        "stream": True,
    }
    headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
    out: list[str] = []
    async with client.stream("POST", url, headers=headers, json=payload) as resp:
        resp.raise_for_status()
        async for line in resp.aiter_lines():
            if not line or not line.startswith("data:"):
                continue
            data = line[5:].strip()
            if data == "[DONE]":
                break
            evt = json.loads(data)
            if meter.ttft_ms == 0.0 and evt.get("choices"):
                meter.ttft_ms = (time.monotonic() - started) * 1000.0
            delta = evt["choices"][0]["delta"].get("content")
            if delta:
                out.append(delta)
            usage = evt.get("usage")
            if usage:
                meter.output_tokens = usage.get("completion_tokens", meter.output_tokens)
    meter.elapsed_ms = (time.monotonic() - started) * 1000.0
    meter.cost_usd = (meter.output_tokens / 1000.0) * RATE_PER_1K_OUT
    return "".join(out), meter

@mcp.tool()
async def complete_fast(
    prompt: str,
    model: str = FAST_MODEL,
    max_tokens: int = MAX_OUTPUT_TOKENS,
    temperature: float = 0.7,
    auto_failover: bool = True,
) -> dict:
    """Stream a fast-tier completion; auto-failover to standard if TTFT exceeds the guard."""
    text, meter = await _stream(prompt, model, FAST_URL, FAST_API_KEY)
    if auto_failover and meter.ttft_ms > TTFT_GUARD_MS:
        meter.failed_over = True
        text, meter = await _stream(prompt, STANDARD_MODEL, STANDARD_URL, STANDARD_API_KEY)
    _jitter_hist.setdefault(meter.model, []).append(meter.ttft_ms)
    return {
        "text": text,
        "model": meter.model,
        "ttft_ms": round(meter.ttft_ms, 1),
        "output_tokens": meter.output_tokens,
        "tokens_per_second": round(meter.tokens_per_second(), 1),
        "cost_usd": round(meter.cost_usd, 6),
        "failed_over": meter.failed_over,
    }

@mcp.tool()
async def tokens_per_second(model: str = FAST_MODEL, iterations: int = 3) -> dict:
    """Benchmark a fast-tier model: mean & p95 TPS, mean TTFT and TTFT jitter."""
    tps, ttfts = [], []
    for _ in range(iterations):
        _, meter = await _stream("Reply with the single word 'ping'.", model, FAST_URL, FAST_API_KEY)
        tps.append(meter.tokens_per_second())
        ttfts.append(meter.ttft_ms)
    tps.sort()
    ttfts.sort()
    p95 = lambda xs: xs[min(len(xs) - 1, int(len(xs) * 0.95)) or 0]
    return {
        "model": model,
        "iterations": iterations,
        "mean_tokens_per_second": round(sum(tps) / len(tps), 1),
        "p95_tokens_per_second": round(p95(tps), 1),
        "mean_ttft_ms": round(sum(ttfts) / len(ttfts), 1),
        "ttft_jitter_ms": round(statistics.pstdev(ttfts), 1) if len(ttfts) > 1 else 0.0,
    }

@mcp.tool()
async def route_fast_vs_standard(
    prompt: str = "",
    max_output_tokens: int = 200,
    estimated_context_tokens: int = 0,
    prefer_latency: bool = True,
) -> dict:
    """Decide whether a request qualifies for the fast tier (size, latency budget, cost)."""
    eligible = max_output_tokens <= MAX_OUTPUT_TOKENS and estimated_context_tokens < 32000
    route = "fast" if eligible and prefer_latency else "standard"
    reason = (
        "Within output and context budgets; latency-sensitive."
        if route == "fast"
        else "Exceeds the fast-tier ceiling, or cost-optimised route chosen."
    )
    return {
        "route": route,
        "reason": reason,
        "estimated_fast_tps": 2700,
        "estimated_standard_tps": 54,
        "suggested_tool": "complete_fast" if route == "fast" else "failover_standard",
    }

@mcp.tool()
async def failover_standard(
    prompt: str,
    fast_model: str = FAST_MODEL,
    standard_model: str = STANDARD_MODEL,
) -> dict:
    """Probe the fast tier; on any error or TTFT breach, retry the same prompt on standard."""
    try:
        text, meter = await _stream(prompt, fast_model, FAST_URL, FAST_API_KEY)
    except httpx.HTTPError:
        text, meter = "", RequestMeter(model=fast_model, ttft_ms=TTFT_GUARD_MS + 1)
    if meter.ttft_ms > TTFT_GUARD_MS:
        meter.failed_over = True
        text, meter = await _stream(prompt, standard_model, STANDARD_URL, STANDARD_API_KEY)
    return {
        "text": text,
        "failed_over": meter.failed_over,
        "ttft_ms": round(meter.ttft_ms, 1),
        "tokens_per_second": round(meter.tokens_per_second(), 1),
        "cost_usd": round(meter.cost_usd, 6),
    }

if __name__ == "__main__":
    mcp.run(transport="stdio")

Two design notes worth keeping: TTFT is measured client-side from the first data: chunk — the same number your users feel, which is the only honest input to a latency guard. And failover_standard is deliberately a separate tool rather than a hidden branch inside complete_fast, so the policy is explicit, testable, and visible to whoever audits the agent's spend.

The inputSchema the model actually sees

FastMCP derives each tool's JSON Schema from the Python signature. This is the shape an MCP client receives at handshake time:

{
  "complete_fast": {
    "type": "object",
    "properties": {
      "prompt": {"type": "string", "description": "User prompt to complete"},
      "model": {"type": "string", "default": "gpt-oss-120b", "enum": ["gpt-oss-120b", "llama-4-maverick", "gemma-4"]},
      "max_tokens": {"type": "integer", "default": 4096, "maximum": 4096},
      "temperature": {"type": "number", "default": 0.7},
      "auto_failover": {"type": "boolean", "default": true, "description": "Fail over to the standard tier when TTFT exceeds the guard"}
    },
    "required": ["prompt"]
  },
  "tokens_per_second": {
    "type": "object",
    "properties": {
      "model": {"type": "string", "default": "gpt-oss-120b"},
      "iterations": {"type": "integer", "default": 3, "minimum": 1, "maximum": 10}
    },
    "required": []
  },
  "route_fast_vs_standard": {
    "type": "object",
    "properties": {
      "prompt": {"type": "string"},
      "max_output_tokens": {"type": "integer", "default": 200},
      "estimated_context_tokens": {"type": "integer", "default": 0},
      "prefer_latency": {"type": "boolean", "default": true}
    },
    "required": []
  },
  "failover_standard": {
    "type": "object",
    "properties": {
      "prompt": {"type": "string"},
      "fast_model": {"type": "string", "default": "gpt-oss-120b"},
      "standard_model": {"type": "string", "default": "gpt-5.6-standard"}
    },
    "required": ["prompt"]
  }
}

Registering with Claude Desktop and Cursor

Add the entry to claude_desktop_config.json (or Cursor's MCP settings) and restart the client:

{
  "mcpServers": {
    "cerebras-fast": {
      "command": "uvx",
      "args": ["cerebras-fast-mcp"],
      "env": {
        "CEREBRAS_API_KEY": "${CEREBRAS_API_KEY}",
        "STANDARD_API_KEY": "${STANDARD_API_KEY}",
        "TTFT_GUARD_MS": "1500",
        "FAST_MODEL": "gpt-oss-120b",
        "FAST_RATE_PER_1K": "0.012"
      }
    }
  }
}

Cursor reads the same shape, and the VS Code MCP extension follows too. Keep secrets in the environment or the client's keychain, never inside a checked-in config file.

Using it (quickstart)

pip install -e .
export CEREBRAS_API_KEY="ck-..."
export STANDARD_API_KEY="sk-..."
cerebras-fast-mcp                          # stdio server, ready for any MCP client

# Smoke-test with the Inspector before wiring an agent
npx @modelcontextprotocol/inspector cerebras-fast-mcp

# Benchmark before you route anything
# in any MCP chat: tokens_per_second(model="gpt-oss-120b", iterations=5)

Security: OAuth 2.0, localhost transport, scoped tokens

This server holds real inference secrets, so treat it as a privileged component:

• Localhost / stdio transport. FastMCP defaults to stdio — the client spawns the process and no network socket is opened. If a remote deployment is unavoidable, serve HTTP/SSE bound to 127.0.0.1 only, never 0.0.0.0, and put it behind a reverse proxy with mTLS or a corporate VPN, which most Indian enterprise and SaaS firms already run across regions.
• OAuth 2.0 client credentials for the standard tier. The OpenAI-compatible standard endpoint accepts OAuth 2.0 client-credentials tokens. Exchange the client ID and secret in a trusted secrets manager (AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault) at startup and refresh short-lived tokens in-process. The agent never sees the raw token — the server only presents it to the upstream API.
• Scoped API keys for the fast tier. Use dedicated Cerebras keys with the narrowest scope the platform offers, rotated every 90 days, with per-key quotas so a runaway agent burns its own budget rather than the company's. In a SaaS context, map one key per customer or product line to keep chargeback and audit simple.
• Never leak secrets to the model. Because tools return telemetry, make sure no error string interpolates a bearer token or header. Log request IDs and status codes only.
• Meter everything. Every tool response carries cost_usd and latency fields. Ship those to your observability stack so the 14x claim is measured on your traffic — and so finance sees the fast-tier line item.

Retry Rules & Error Handling

The production checklist

Before you route real traffic through the wafer tier, close these loops: (1) benchmark with tokens_per_second on your actual prompt mix, because TPS varies with output length and content; (2) set the TTFT guard from real p95 measurements, not the vendor's best case; (3) decide routing policy per workload in route_fast_vs_standard — batch summarization rarely needs 750 TPS, voice and search do; (4) wire the cost meter into billing so "fast" never becomes an unmanaged cost center; (5) rehearse the failover path the way you rehearse a database outage, because the tier will go down at the worst moment.

The fast-inference tier is the first time in two years that the provider's hardware choice has been an agent-relevant product decision. Wrap it in MCP, measure it, govern it, and fail over from it — then the wafer-scale advantage becomes yours instead of only OpenAI's and Cerebras's. For the wider catalogue of servers worth building, see the MCP Directory, and track the Latest AI News for the moment Ultrafast pricing and general availability land.