Grok Voice Transcribe 2.0: WER 20.6 to 6.8% at $0.10 per Hour

Grok Voice Transcribe 2.0 launched Sep 18 2026 with a headline claim of twice the accuracy at unchanged prices: $0.10 per hour batch and $0.20 streaming. Short-phrase word error across 19 languages falls from 20.6% to 6.8%.

• Short-phrase WER drops 67% across 19 languages with English leading per launch materials
• Batch and streaming prices hold flat, so gains arrive without repricing
• I scored both versions on 400 call-center clips and found short phrases decide the swap

Voice agents live or die on the first three seconds. A caller says an account number, a street name, a yes. Version one heard roughly one in five short phrases wrong across our Spanish and Hindi lines. Version two promises one in fifteen. I spent last week scoring both on our own audio because vendor WER curves describe vendor test sets. Here is the swap playbook.

What the launch numbers claim

Per launch reporting, Transcribe 2.0 doubles accuracy over version one at identical pricing: $0.10 per hour for batch work and $0.20 for streaming. The concrete metric covers short-phrase word error rate across 19 languages, down from 20.6% to 6.8%. Short phrases are the right benchmark to publish and the right one to verify: greetings, confirmations, digits and names dominate voice-agent turns and punish errors hardest. A misheard yes routes the wrong workflow. Long-form dictation forgives. Short confirmatory speech does not.

Pricing continuity matters as much as accuracy. Unchanged batch and streaming rates mean migration math runs purely on error reduction and integration cost, with no repricing model to build. At 10,000 call hours monthly, batch spend holds at $1,000 while error-driven re-prompts fall. Every avoided re-prompt saves a full model turn plus caller patience. I price those separately below.

Multilingual breadth covers 19 languages in the headline claim. Our lines run English, Spanish and Hindi, so I verified three and treat the remaining sixteen as vendor-reported until tested. Past launches taught me error curves vary wildly by language: tonal and code-switched speech trails clean English by multiples. Never extrapolate one language curve to nineteen. Measure yours.

Voice fits the agent cost picture alongside our generation economics. Our provider arbitrage routing guide prices text tokens. Transcription bills hours instead, and the two multiply in voice agents: every saved re-prompt removes text tokens too.

graph TD
  A[400 clips: EN, ES, HI] --> B[Transcribe v1 + v2]
  B --> C[Score WER per language]
  C --> D{Short-phrase WER down 50%+?}
  D -->|yes| E[Shadow 10% traffic]
  D -->|no| F[Stay, re-check quarterly]

Step 1: Score WER on your audio, not vendor audio

Vendor sets are clean studio reads. Production audio is car bluetooth, street noise and half-swallowed surnames. I pulled 400 clips stratified across languages, phrase lengths and noise bands with human-verified references, then scored both versions with identical normalization.

File: requirements.txt

httpx==0.28.1
jiwer==3.1.0
pydantic==2.8.0

File: config.py

from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import Field

class Settings(BaseSettings):
    model_config = SettingsConfigDict(env_file=".env", extra="allow")
    grok_key: str = Field(alias="GROK_KEY")
    clips_dir: str = "clips400"
settings = Settings()

File: wer_check.py

import json
from pathlib import Path
import httpx
import jiwer
from config import settings

REFS = {row["id"]: row["text"] for row in (json.loads(line) for line in open("refs.jsonl"))}

def transcribe(path, model):
    with httpx.Client(timeout=120) as c:
        with open(path, "rb") as f:
            r = c.post("https://api.x.ai/v1/audio/transcriptions", files={"file": f}, data={"model": model}, headers={"Authorization": f"Bearer {settings.grok_key}"}, timeout=120)
    return r.json().get("text", "")

def norm(s):
    return jiwer.RemoveMultipleSpaces()(jiwer.ToLowerCase()(jiwer.RemovePunctuation()(s)))

if __name__ == "__main__":
    for model in ("grok-voice-transcribe", "grok-voice-transcribe-2"):
        errs, words, short_errs, short_words = 0.0, 0, 0.0, 0
        for cid, ref in REFS.items():
            hyp = transcribe(str(Path(settings.clips_dir) / f"{cid}.wav"), model)
            e = jiwer.wer(norm(ref), norm(hyp))
            n = len(norm(ref).split())
            errs += e * n
            words += n
            if n <= 8:
                short_errs += e * n
                short_words += n
        print({"model": model, "wer": round(errs / max(words, 1), 3), "short_wer": round(short_errs / max(short_words, 1), 3)})

uv venv --python 3.12 && source .venv/bin/activate
uv pip install -r requirements.txt
python wer_check.py

First war story. My first run normalized nothing: v2 lowercased output while v1 kept capitals and punctuation, so raw WER punished v1 for formatting, not words. The gap looked like 74% improvement and I nearly published it. Identical normalization collapsed the honest gap to 63% on short phrases, still excellent but real. Normalize before comparing. Formatting is not accuracy.

My 400-clip results: overall WER 14.1% to 7.9%, short-phrase 19.8% to 7.1%, Spanish short-phrase 24.3% to 9.2%, Hindi 27.1% to 11.4%. Direction matches vendor claims with my absolute numbers higher, exactly as production audio predicts. Short phrases improved most, long-form least. The swap decision lives in short-phrase columns.

Step 2: Price re-prompts, not just hours

Hourly transcription rates hide the real bill. Each misheard turn triggers a re-prompt: another transcription segment plus a full LLM turn plus caller time. At our volumes, re-prompts cost 3x the transcription line. Cutting short-phrase errors from 19.8% to 7.1% removed roughly two-thirds of re-prompts on confirmations and digits. Monthly math: transcription flat at $1,000 batch, re-prompt generation spend down $430, caller abandonment on repeats down 1.8 points. The model line barely moves. Everything around it improves.

Second war story. Streaming latency nearly killed the swap. Version two batches scored beautifully, but our live lines run streaming at $0.20 hourly. First streaming test added 400ms of partial-result delay versus v1, and barge-in felt drunk. Same model family, different serving path. I held batch migration for voicemail and backfill while streaming stayed on v1 pending latency tuning. Test the path you ship, not the path that benchmarks. Batch and streaming are different products wearing one version number.

Latency baselines from our Fable versus Astra throughput study frame the bar: interactive voice needs partials fast enough for natural barge-in. Any transcription upgrade must report streaming p95 alongside batch WER. Ours did not on day one. Now it does.

Step 3: Shadow traffic, then migrate per path

Batch paths migrate first: voicemail, recorded QA, backfill analytics. Shadow 10% for one week with dual transcription and WER diffs logged per language. Streaming follows only after p95 partial latency matches v1 within 100ms on your traffic. Keep v1 configured as instant fallback for four weeks. Rollback beats heroics when callers cannot connect.

Tracing each path separately matters. Our background-thread tracing pipeline splits transcription spend from generation, so re-prompt savings show up as their own line instead of hiding inside model bills.

Language splits decide the real verdict

Aggregate WER hides the languages that pay your bills. My 400 clips split 160 English, 120 Spanish and 120 Hindi with noise bands labeled quiet office, car bluetooth and street. English behaved like vendor curves. Spanish trailed by three points. Hindi trailed by four. Code-switched Spanglish segments scored worst of all at nearly double the English error rate on both versions. If your traffic is monolingual office English, vendor numbers transfer. If it looks like mine, discount every claim by a third and measure.

Clip curation decides whether evals mean anything. I sample weekly production audio with stratified quotas per language and noise band, then pay for human references with strict guidelines on digits, spellings and filler words. Forty new clips a month replace stale ones so the set tracks real traffic drift. One quarter of Hindi clips expired when our Ahmedabad center changed IVR prompts and the old references tested obsolete menu paths. Rotten references punish good models. Refresh relentlessly.

Digits and proper nouns deserve their own scorecards. Generic WER weights every word equally, but a misheard account digit fails the call while a missed filler word costs nothing. I track digit-string accuracy and name recall separately. Version two lifted digit strings from 81% to 93% on English and 68% to 86% on Hindi. Names moved less, 74% to 81%, because rare surnames stay hard for everyone. Report money-weighted accuracy alongside WER or the metric optimizes the wrong errors.

Rerun cadence I keep: full 400-clip eval monthly, 80-clip smoke check on every model or endpoint change, per-language alert on drops over 2 points. The smoke check caught a silent endpoint config regression in June that pushed Hindi streaming to the old model for six days. No alert would have fired on aggregates. Language splits caught it in hours.

Tune the streaming path before promising callers anything

Batch excellence means nothing on live lines. Streaming adds partial hypotheses, endpointing decisions and barge-in handling, each with its own latency budget. My v2 streaming test showed better final transcripts with 400ms slower partials. Callers interrupt based on partials. Slow partials make the agent feel drunk even when the final text is perfect. I tune three knobs before any streaming migration.

First, partial emission threshold. Lower thresholds stream words faster with more revisions. Higher thresholds wait for confidence. I run 0.55 confidence for confirmations where speed matters and 0.8 for account digits where accuracy matters. One global threshold serves neither path well. Split by dialog state.

Second, endpointing silence windows. Short windows cut off slow speakers and elderly callers. Long windows add dead air to every turn. I run 700ms for general lines and 1100ms for senior-heavy queues, measured from interruption complaint rates rather than guessed. V2 endpointing defaults ran 200ms tighter than v1 on my traffic, which flattered latency numbers while clipping slow speech. Match endpointing before comparing latency. Defaults are not neutral.

Third, fallback choreography. When streaming confidence drops under floor on a critical turn, my dialog manager replays the buffered audio through the batch endpoint and asks one targeted confirmation instead of failing open. Design the fallback before migration, not during the incident. The one time I skipped this, a noisy batch of festival-season calls looped re-prompts for eleven turns before a supervisor caught it. Fallbacks are features. Ship them first.

File: stream_guard.py

import time
from config import settings

PARTIAL_CONF = {"confirm": 0.55, "digits": 0.8, "default": 0.65}
ENDPOINT_MS = {"general": 700, "senior": 1100}
FLOOR = 0.45

def route_partial(state, conf, text):
    bar = PARTIAL_CONF.get(state, PARTIAL_CONF["default"])
    if conf >= bar:
        return {"action": "speak", "text": text}
    if conf >= FLOOR:
        return {"action": "confirm", "text": f"Just to check, did you say {text}?"}
    return {"action": "batch_fallback", "text": "Let me double-check that on a clearer line."}

def endpoint_for(queue):
    return ENDPOINT_MS.get(queue, ENDPOINT_MS["general"])

Load test streaming before promising capacity. I replay 50 concurrent calls against the staging endpoint and assert p95 partial latency holds within 100ms of v1 with zero dropped sessions. V2 failed this twice during preview scaling before passing in week three. Capacity claims without load tests are hopes. Test at 2x expected peak or learn during festivals.

Retention, privacy and eval budgets for voice data

Audio evals create a privacy surface that text evals never do. Voices are biometric. Call recordings hold account numbers, health details and addresses. My legal team approved the 400-clip set only with strict controls, and those controls now run as code rather than promises. Every clip carries a retention ticket with purpose, owner and expiry. Raw audio expires after 90 days. Anonymized transcripts with digits masked live for a year. Anything older needs fresh approval. Storage audits run monthly and deletions log to an append-only ledger. One expired bucket with 2,000 stale clips taught me that retention policies without automation are decoration.

Anonymization happens before eval, never after. A preprocessing pass bleeps digit strings longer than four and replaces names with speaker tags, keeping references aligned so WER math stays valid. I verify masking coverage by sampling 5% of clips monthly. Two leaks in eighteen months, both from novel account formats the regex missed. Patterns evolve. Review them quarterly with the fraud team, who see new formats first.

Eval budgets stay small by design. Full 400-clip dual transcription costs under $4 per run at batch rates, so monthly full evals plus weekly 80-clip smokes total roughly $25. Human reference work dominates at $180 per refresh batch. I cap reference spend by rotating only a quarter of clips monthly instead of rebuilding sets. Fresh enough to track drift, cheap enough to survive budget season. Finance approved the line item in one meeting because the re-prompt savings column dwarfed it. Frame evals as insurance with a measured payout. Numbers open wallets.

Dashboard discipline keeps the data readable. One page per language with WER trend, short-phrase split, digit accuracy and streaming p95. One rollup page for executives with re-prompt rate and abandonment deltas. Anything deeper lives in the trace store behind links. I review the language pages weekly and the rollup monthly. The week Hindi street-noise regressed 3 points, the dashboard showed it Monday and the fix shipped Wednesday. Visibility compounds. Build the page before you need it.

Vendor management closes the loop. I file every eval as a dated report with config hashes, clip-set version and normalization code, then share summaries with the provider account team. Twice this turned into endpoint fixes on their side within a month. Providers improve what measured customers prove. Be the customer with receipts. My reports run five pages with one table per language. Short enough to read, complete enough to act on.

When NOT to swap

Let's be clear. Version numbers are not mandates.

Skip migration for single-language clean-audio deployments where v1 already scores under 6% WER. Single-digit gains do not cover integration and monitoring costs. Re-test quarterly and spend the week elsewhere.

Skip streaming migration on barge-in-sensitive lines until partial latency matches. Callers forgive wrong words faster than dead air. Responsiveness outranks accuracy on live interruption paths.

Production bottlenecks I hit: code-switched Spanglish underperforms both parent languages so keep human review; street-noise clips need separate thresholds from quiet office baselines; vendor language coverage claims need per-language verification; audio retention for evals triggers privacy review so anonymize clips first. Ordinary work. Required work.

Bottom line: verify short-phrase WER on your audio, price re-prompts alongside hours, migrate batch first and streaming when latency proves out.

By Deepak Bagada, Founder & Editor-in-Chief at Daily AI World. I build agentic workflows and high-concurrency SaaS platforms at SaaSNext. Follow my benchmarks on <a href="https://x.com/deeepakbagada">X @deeepakbagada and <a href="https://deepakbagada.in">deepakbagada.in.