FrontierSWE v2 Shakes Rankings: Fable 5.1 at 56.3%, Rivals 32%

A new benchmark dropped this week and reordered everything I thought I knew about agent rankings. FrontierSWE v2, released September 2026, tests agents on 34 ultra-long-horizon engineering and research tasks with a 20-hour budget each — and Claude Fable 5.1 leads at 56.29%, more than 24 points ahead of GPT-5.6 Sol at 32.2%. Models that look neck-and-neck elsewhere separate by a chasm here.

FrontierSWE v2 measures what short benchmarks cannot: sustained engineering over hours, not minutes, scored mean@5 with the Proximus harness at maximum reasoning effort. Three facts anchor the release:

• Fable 5.1 at 56.29% versus GPT-5.6 Sol at 32.2% exposes a gap other benchmarks hide completely.
• GLM-5.3, the strongest open-weight model, runs close behind the closed leaders — the open frontier holds at long horizons.
• The benchmark is far from saturated by design, with AI research tasks extended into post-training under resource constraints.

This is the evaluation shift my harness work has pointed toward all year, the same long-horizon thinking behind my benchmark selection guide. Same discipline, applied to the hardest tasks yet published.

What v2 changed and why it matters

Four months after v1, the team shipped a larger, harder benchmark with rebuilt methodology. Thirty-four tasks spanning engineering marathons and AI research problems — including post-training work under compute constraints, the kind of task that mirrors what labs actually do. Each model runs at maximum reasoning effort, five attempts per task, twenty hours per attempt.

Here's the catch. Most coding benchmarks measure sprint performance: solve a function, pass hidden tests, done in minutes. Production agent work is a marathon — explore, plan, build, debug, verify across hours while context rots and costs compound. A model can top SWE-bench and collapse at hour six. FrontierSWE v2 is the first widely-reported benchmark that prices the marathon, and the leaderboard proves the two skills differ.

That matches what my reliability decay measurements show per step: quality erodes with trajectory length, so a 20-hour task filters for stamina that short evals never observe.

The results: a 24-point gap nobody else shows

Don't do this: reading this as a general intelligence ranking. It is a stamina ranking. Fable's lead reflects sustained coherence — planning that survives hour ten, debugging loops that converge instead of cycling. GPT-5.6 Sol's score says its sprint ability does not transfer to marathons, a distinction worth millions in routing decisions.

The open-weight story matters most for builders. GLM-5.3 running close behind closed flagships on the hardest public agent benchmark means long-horizon capability is not locked behind one lab's API. Self-hosted marathon agents are now a serious architecture, not a compromise — teams can run the hardest public eval tier on their own clusters, keep trajectories in-house for compliance, and iterate on harness design without API rate limits gating every experiment.

What 20-hour tasks demand from your harness

A 20-hour budget breaks every assumption short-task harnesses make. Checkpoints must survive worker restarts across hours — the durable execution pattern from my zero-crash agent loops stops being optional. Context must compact repeatedly without losing the plan. Cost must accrue against a per-task budget with graceful wind-down, not a surprise invoice.

flowchart TD
    START[Task starts, budget armed] --> RUN[Agent works in phases]
    RUN --> CHECKPOINT[Checkpoint state hourly]
    CHECKPOINT --> COMPACT[Compact context at phase shifts]
    COMPACT --> BUDGET{Budget remaining?}
    BUDGET -->|yes| RUN
    BUDGET -->|low| WIND[Wind down: summarize + stop]

Step 1: Arm the budget before the run

config.py

from pydantic import BaseModel

class MarathonConfig(BaseModel):
    time_budget_h: float = 20.0
    cost_budget_usd: float = 150.0
    checkpoint_every_min: int = 60
    compact_trigger_tokens: int = 150000
    wind_down_at_pct: float = 0.85
    attempts: int = 5

CONFIG = MarathonConfig()

Per-task dollar caps are the lesson FrontierSWE's methodology teaches for free: score mean@5 with fixed budgets, and cost becomes a first-class metric. My per-task cost analysis applies directly — a marathon agent without a dollar cap is a billing incident waiting for hour nineteen.

Step 2: Checkpoint and compact on schedule

harness.py

async def run_marathon(task, cfg=CONFIG):
    state = await resume_or_fresh(task.id)
    spent = tracker(task.id)
    try:
        while not state.done and cfg.cost_budget_usd > spent.usd:
            state = await agent_phase(state)
            await checkpoint(state)
            state = await maybe_compact(state)
            if spent.pct > cfg.wind_down_at_pct:
                return await wind_down(state, "budget")
    except WorkerCrash:
        logger.warning("worker lost, resuming from checkpoint")
        return await run_marathon(task, cfg)
    return state

Hourly checkpoints plus phase-shift compaction — the exact combination from my compaction work — is what lets a run survive the night. The wind-down path returns a summary and partial artifacts instead of dying at the cap; partial credit beats a killed run.

requirements.txt

langgraph==1.0.2
pydantic==2.8.0
structlog==24.4.0
httpx==0.28.1
python-dotenv==1.0.1

Pydantic v2.8 needs extra="allow" on checkpoint schemas or nested trajectory payloads fail validation. I lost an afternoon to that exact error before pinning it.

Step 3: Migrate your evals to marathon scoring

Add one long-horizon task to your private eval set this week — a real multi-hour ticket from your backlog, scored on completion rather than vibes, with the trajectory and cost logged for review. Run every candidate model on it. The model that wins your sprints may not win your marathon, and FrontierSWE v2 just proved the spread can reach 24 points. Route production traffic by marathon scores, sprint scores only for interactive autocomplete.

The hour-six war story: context rot wins marathons

My longest pre-v2 run died at hour six of nine. Not crashed — rotted. The agent had compacted four times on fixed thresholds, each summary a little lossier, until it confidently reverted its own working fix because the rationale lived in a summary three generations back. Final state: passing tests on the wrong approach, full budget spent.

Phase-shift compaction with the five-field preserve-schema fixed it on the next run: same task completed in seven hours with two compactions. The benchmark's 20-hour design validates the obsession — at marathon length, context management is the capability.

When NOT to care about FrontierSWE v2

Let's be clear. Interactive autocomplete, single-function generation, and sub-ten-minute tasks live in sprint land — route those by sprint benchmarks and latency. Small teams without marathon workloads should not re-architect harnesses around a benchmark they will never exercise. And 56% itself says even the leader fails nearly half these tasks; no score here green-lights unsupervised production deployment.

Watch it for routing long-horizon work and tracking the open-weight chase. Ignore it for everything measured in seconds.

FrontierSWE v2 moved the goalposts from sprints to marathons, and the 24-point gap is the news: stamina is now measurable, open weights hold it, and harnesses must be rebuilt around budgets, checkpoints, and compaction before the next eval cycle leaves sprint-tuned teams behind. The marathon era of agent evals starts here.

By Deepak Bagada, Founder & Editor-in-Chief at Daily AI World.