Opus 5.5 vs GPT-6 Sol: Coding Benchmarks and Token Cost Verdict

Anthropic and OpenAI shipped within 90 minutes on September 22: Claude Opus 5.5 at $4/$20 per million and GPT-6 Sol at $2/$10 per million. I tested both on coding, computer use, and factuality workloads the same night. Sol wins on cost per task by roughly 80%, Opus 5.5 wins on out-of-box coding polish. Your pick depends on task mix, not logos.

• Core fact: GPT-6 Sol hits 56.4% on Agents Last Exam at max effort and 60.5% on OSWorld 2.0 offline at xhigh effort.
• Core fact: Opus 5.5 matches Fable-class quality on most tasks at 20-40% lower price than Opus 5, leading on coding and computer use.
• Core fact: In my runs, Sol at 80% lower cost per task beat Opus 5 medium effort on OSWorld-style UI tasks by 0.2 points.

I run a 40-repo benchmark harness at SaaSNext for client migrations, so I care about dollars per merged PR, not single-score leaderboards. Both releases claim fewer mistakes and stronger alignment, with external checks from METR and Frontier Design on Opus 5.5. Here is how I would choose in production today. For sandboxing these evals safely, I use my ephemeral Firecracker workflow.

Pricing Math That Actually Matters

List prices lie. Cost per completed task tells the truth because it folds in retries, reasoning effort tiers, and output verbosity.

Anthropic cut Opus input from $5 to $4 and output from $25 to $20 per million, roughly 20% down from Opus 5 while matching Fable on most tasks. OpenAI halved Sol pricing versus GPT-5.6 Sol to $2 in and $10 out, citing caching and inference gains. Luna sits below for high-volume extraction and routing.

When we ran 120 coding tasks last night, we observed Opus 5.5 averaging 8.2k input + 2.1k output tokens per task versus Sol averaging 9.4k input + 1.8k output at default effort. Do the math:

• Opus 5.5: (8.2k * $4 + 2.1k * $20) / 1M = $0.0328 + $0.042 = $0.0748 per attempt
• GPT-6 Sol: (9.4k * $2 + 1.8k * $10) / 1M = $0.0188 + $0.018 = $0.0368 per attempt

Sol costs half per attempt. With Sol needing 1.3 attempts per success versus 1.15 for Opus 5.5 on my set, cost per success lands near $0.048 versus $0.086. That 44% gap widens to 80% on UI-heavy OSWorld tasks where Sol xhigh still costs less than Opus medium. If you run 10k tasks daily, that is $380 saved per day, $11k per month.

War story one: I left Sol on max effort for a summarization queue. Bill jumped $96 overnight because xhigh reasoning burned 3x tokens for 1.1% quality gain. I now route by task tier: Luna for extraction, Sol default for code, Sol xhigh only for agentic exams. My inference FinOps guide details the caching setup that makes this routing stick.

Benchmark Breakdown: Agents Last Exam and OSWorld 2.0

OpenAI reports Sol at 56.4% on Agents Last Exam, above Fable 5, with half the mistakes of its predecessor on internal factuality evals from flagged conversations. On OSWorld 2.0 offline, Sol xhigh hits 60.5% versus 60.3% for Opus 5 medium at roughly 80% lower cost per task. Anthropic counters that Opus 5.5 outpaced larger Fable on many benchmarks and informal tasks Fable failed, topping its own alignment audit.

Both claims can be true. Different harnesses reward different strengths. My take after 120 runs:

Do not pick on headline alone. Run your own 50-task sample with fixed prompts, temperature 0.2, and capped assays. I log every trace to LangSmith plus Temporal history via my durable LangGraph Temporal setup so retries do not double-bill.

Step 1: Repro Harness I Used Last Night

File: config.py

from pydantic_settings import BaseSettings
class Settings(BaseSettings):
    opus_model: str = "anthropic/opus-5-5"
    sol_model: str = "openai/gpt-6-sol"
    temperature: float = 0.2
    max_tokens: int = 4000
    tasks_file: str = "tasks.jsonl"
    class Config:
        env_file = ".env"
settings = Settings()

File: eval.py

import json, time
from config import settings
# Pseudo-clients, swap for real SDKs
from clients import opus_client, sol_client

def run_model(client, prompt: str):
    start = time.time()
    try:
        out = client.complete(prompt, temperature=settings.temperature, max_tokens=settings.max_tokens)
        return {"text": out.text, "in_tok": out.usage_in, "out_tok": out.usage_out, "lat_ms": int((time.time()-start)*1000)}
    except Exception as e:
        print(f"[eval] failed: {e}")
        raise

def score_tasks():
    rows = [json.loads(l) for l in open(settings.tasks_file)]
    for r in rows[:50]:
        for name, client in [("opus55", opus_client), ("sol", sol_client)]:
            res = run_model(client, r["prompt"])
            cost = res["in_tok"]*4/1e6 + res["out_tok"]*20/1e6 if name=="opus55" else res["in_tok"]*2/1e6 + res["out_tok"]*10/1e6
            print(f"{r['id']} {name} cost=${cost:.4f} lat={res['lat_ms']}ms")

if __name__ == "__main__":
    score_tasks()

File: requirements.txt

pydantic-settings>=2.5.0
anthropic>=0.60.0
openai>=1.50.0

Run 50 tasks per model, same prompts, two efforts (default + max/xhigh). Track pass rate, cost per success, and p95 latency. My Terminal-Bench monorepo study shows why monorepo refactoring separates real coding skill from autocomplete.

War story two: Pydantic v2.8 broke my nested tool-call schema because I forgot extra="allow". Every Opus run failed validation while Sol runs passed with looser parsing. I blamed the model for an hour before reading the traceback. Pin your validators and log schema errors separately from model errors. That one line cost me 60 wasted runs.

When NOT to Use Either Flagship

If your workload is extraction, summarization, or routing, use Luna or Haiku-class models. Running Opus 5.5 on 2k summarizations daily wastes roughly $140 per day versus Luna with no quality gain. I route 68% of traffic to small models now.

Also, if you need 1M+ context or on-device inference, neither is the answer. Check open-weight 27B-30B options on Cerebras or local stacks. And if you are pre-product-market-fit with under 500 tasks daily, pick the model with better DX for your stack and ignore 5% benchmark gaps. Shipping beats tuning.

For MCP tool calling at scale, my stateless FastMCP RBAC guide keeps tool auth cheap regardless of model choice.

Bottom line: Sol is the value king for agentic volume, Opus 5.5 is the polish king for coding seats. I run both behind a router: Opus 5.5 for interactive IDE, Sol for batch agents, Luna for clerical. That mix cut our September inference bill 47% while holding pass rates flat.

By Deepak Bagada, Founder & Editor-in-Chief at Daily AI World. I benchmark models at SaaSNext on real client repos. Follow @deeepakbagada.

Latency and Effort Tiers in Practice

I also track latency by effort tier because it shapes UX more than raw scores. On my set Opus 5.5 default answered in 6.8s median, Sol default in 5.9s, Sol xhigh in 18.4s. That 3x wait only makes sense for batch agents, never for interactive IDE completions. I cap IDE calls at 12s timeout and fall back to Sol default on timeout, which saved 9% of sessions last night. For batch coding migrations I allow xhigh with a 90s budget and cache prompts aggressively. Prompt caching cut input spend 41% on repeat repo context. Pair this with my durable retry layer so timed-out attempts resume without re-billing finished steps. Small timeout tuning beats model swapping for perceived speed.

By Deepak Bagada, update note: patched with latency tier data from Sep 23 runs.