By Deepak Bagada, CEO at SaaSNext & Principal AI Architect.

OpenAI Ultrafast: 750 Tokens/s Ends the Fast-vs-Smart Tradeoff

On August 13, 2026, OpenAI previewed Ultrafast, a serving tier that streams GPT-5.6 Sol at up to 750 output tokens per second — as much as 14x faster than the Standard tier, on hardware that removes the exact bottleneck that made fast frontier inference impossible. For a decade, the operating rule in applied AI has been: want it fast, pick a smaller, specialized model; want it smart, stand in line. Ultrafast is the first credible signal that the tradeoff is ending: frontier-grade reasoning at interactive speeds, in a limited preview of the OpenAI API.

The preview is powered by Cerebras wafer-scale engines (WSE). Each WSE is a single silicon slab holding roughly 900,000 AI cores and 44GB of on-die SRAM. In a conventional GPU, model weights live in HBM and stream to compute on every forward pass; that weight-transfer bandwidth — not raw FLOPs — is the wall that caps tokens per second. Put the weights on-chip and the transfer tax disappears, which is why a wafer-scale design can claim an order-of-magnitude speedup without inventing a smarter model. Pricing has not been disclosed and the tier is limited preview, so treat every number below as vendor-reported preview data until independent benchmarks land.

What the vendor-reported numbers look like

The Humanity's Last Exam run is a useful sanity check: it is a multiple-choice benchmark, answered question by question, and 11 hours vs 78 hours is a genuine end-to-end throughput gap. But it is not an agentic workload — no tool calls, no branching. The headline I would hold on to is the simpler one: more useful work per second, which is exactly how a latency-sensitive agent consumes tokens.

Why wafer-scale breaks the latency wall

GPU inference has a dirty secret: for every token you generate, the full weight matrix must stream from HBM to the compute units. At 8B+ parameters this means the accelerator is mostly a memory pipe, not a math engine. The Cerebras WSE reverses that design: with ~44GB of SRAM on the same slab, a 30B-class model fits entirely on-die, and token generation becomes a compute problem instead of a bandwidth problem. That is why the speed does not come from a bigger or smarter model — it comes from deleting the memory-bandwidth tax. If you are building infrastructure for a company that serves thousands of concurrent agent loops, that tax is your real cost driver, not the model card.

The latency-budget analysis: why agent loops compound

A multi-step agent does not pay latency once; it pays it per hop. Each hop is model call, tool call, model call — and the wait multiplies by the number of hops. The table below assumes 400 output tokens per hop at roughly 55 tokens/s for Standard versus 750 tokens/s for Ultrafast:

This is the latency-budget view: interactive systems have a deadline (a human waiting on a chat reply, a pager responder staring at a screen, an API caller with a 30-second gateway timeout). Standard is under budget for a one-hop answer but blows the budget on the eighth hop. Ultrafast stays under budget for the whole chain. The compounding is what matters — a linear speedup in tokens per second becomes a near-linear win in end-to-end completion, and that is what makes real-time, multi-step agent loops viable at frontier quality for the first time.

ROI math for a support and incident-response agent

Let us price the win in throughput rather than tokens, since pricing is undisclosed. If the tier costs the same per token, the same budget simply buys more decisions per minute. If it carries a premium, you must switch to a cost-per-useful-decision view — but the capacity math below is the number to argue with your CFO:

For an incident-response system, the ROI framing is even simpler: a 12-hop loop that used to take 87 seconds now takes 6.4. In the same minute you can run nine full loops instead of one — which is the difference between answering the pager and writing the postmortem. The secondary effect is structural: teams no longer have to downgrade to a smaller, cheaper model to meet a latency SLO, so quality goes up at roughly the same spend. The latest AI news desk covered the preview the day it dropped; this article digs into what it changes for production systems.

Code: calling the API with a service tier, plus a benchmark harness

import json
import time

from openai import OpenAI

client = OpenAI()


def timed_completion(model: str, tier: str, prompt: str) -> dict:
    messages = [{"role": "user", "content": prompt}]
    started = time.perf_counter()
    stream = client.chat.completions.create(
        model=model,
        messages=messages,
        service_tier=tier,  # "ultrafast" | "standard" | "auto"
        stream=True,
    )
    tokens = 0
    for chunk in stream:
        tokens += len(chunk.choices[0].delta.content or "")
    elapsed = time.perf_counter() - started
    return {
        "tokens": tokens,
        "elapsed_s": round(elapsed, 2),
        "tokens_per_s": round(tokens / elapsed, 1) if elapsed else 0.0,
    }


prompts = [
    "Triage this page: billing DB error rate spiked to 9 percent. Give the top 3 hypotheses and a check order.",
    "Summarize these 12 support tickets and group them by root cause.",
    "Draft a containment plan for this suspicious login burst, in two paragraphs.",
]
for prompt in prompts:
    for tier in ("standard", "ultrafast"):
        result = timed_completion("gpt-5.6-sol", tier, prompt)
        print(json.dumps({"tier": tier, **result}))

Build the harness first, benchmark your own prompt mix across tiers for at least a week, and only then decide. If your workload is bursty rather than latency-critical, the win shows up as batch completion time instead of p95 — measure both.

What I would watch before betting production on it

Four unknowns keep Ultrafast from being a slam dunk. First, pricing: undisclosed today, and if the tier carries a per-token premium, the ROI table flips to cost-per-useful-decision. Second, independence: the 11x and 5x comparisons are Cerebras-reported; wait for third-party replication. Third, concurrency: 750 tokens/s per stream is not aggregate throughput; we do not yet know how the WSE behaves under 1,000 concurrent streams, and per-stream vs pooled behavior is the difference between a fast endpoint and a fast gridlock. Fourth, preview caps: expect rate limits, and do not put an SLO on a tier that is still a limited preview.

Where Ultrafast changes agent architecture

The durable architectural shift is for workloads where speed is the missing variable: outage response (a responder that must re-run scenarios live), cyber threat detection (classify-and-contain loops inside a detection window), financial analysis (model-in-the-loop checks before a trade), and real-time multi-step agent loops where each tool call re-enters the model. If your agent is already a chain of calls, a 14x token rate collapses the chain's latency, and you can afford to run more planning depth for the same wall-clock budget. Pair that with a workflow library that checkpoints each hop, and the reliability argument stops being about speed at all — it is about the fact that a 6-second loop is retriable, where an 87-second one was not.

The fast-vs-smart tradeoff is not dead everywhere — small models still win on cost per token and cold-start latency. But for the first time, the frontier itself can be interactive. Teams that benchmark now will own the operating playbook when the tier exits preview.

Practical adoption steps

If you are evaluating Ultrafast for a production system, run a disciplined week-long evaluation rather than a demo. Start with the benchmark harness above over your real prompt mix, capturing both per-stream tokens per second and end-to-end latency for your actual agent chains — remember that a single response time is meaningless; what matters is the full multi-hop completion distribution. Then record three numbers: p95 end-to-end latency per agent loop, cost per completed task, and task success rate against your golden set. Compare the Standard and Ultrafast tiers on all three, and only adopt if the speed win survives at your concurrency level rather than collapsing under load. Finally, design for tier failure: wire a fallback that routes to the Standard tier or a smaller model the moment Ultrafast returns an error, hits a rate cap, or drifts out of your latency SLO. In a preview-era deployment, the fallback is not a nice-to-have; it is the deployment.

Disclaimer: All speed and benchmark figures in this article are vendor-reported from an OpenAI/Cerebras preview and have not been independently verified. Pricing and general availability were not disclosed at the time of writing.