GPT-5.4 Pro Tops FrontierScience at 36.7%: Research Bends

The September 2026 FrontierScience leaderboard carries the number that matters for science copilots: GPT-5.4 Pro on top at 36.7% on research-level physics, chemistry, and biology tasks. Not a multiple-choice quiz — PhD-written open-ended research subtasks graded against 10-point expert rubrics, where 7 points marks success.

FrontierScience splits into two tracks with two stories. Olympiad: 100 medalist-written theory problems, already at 77% for top models. Research: 60 open-ended subtasks at PhD difficulty, where the same flagships manage roughly a third. Three facts anchor the release:

• GPT-5.4 Pro leads the blended board at 36.7% with a 1.05M-token context window, per the September 17 BenchLM publication.
• The Research track grades intermediate reasoning steps, not just final answers — rubric items with explicit pass/fail conditions per step.
• Judge-model grading stays within 0.5 points of human experts by design, making rubric evals scalable without losing calibration.

This is the evaluation frontier my benchmark coverage tracks quarter to quarter, the same rubric-first thinking as my judge-reliability analysis. Same verdict discipline, applied to science instead of tools.

Why the Research track is the real news

Olympiad saturation tells you models do structured theory well. The Research track tells you something rarer: can a model carry a multi-step investigation a PhD scientist would recognize — deriving the equation, checking the limiting case, interpreting the result? Each subtask took experts 3–5 hours to draft, with solution paths and independent rubric items. That construction cost is the moat against saturation.

Here's the catch. At 36.7% blended with Research far lower, frontier models still fail most genuine research subtasks. The failure taxonomy from the paper reads like my own lab notes: reasoning and logic errors first, then niche-concept gaps where the model confidently misstates specialized knowledge, calculation slips in multi-step derivations, and plain factual inaccuracy. Chemistry leads, biology follows, physics trails — a subject split worth mirroring in any science-adjacent eval you build. Structured reasoning works; open-ended judgment lags. Any team buying science-copilot capability on Olympiad scores alone is measuring the wrong track.

That matches my harness-selection guidance: match the eval to the workload, and research assistance lives on the Research track, not the Olympiad one.

The leaderboard: what 36.7% actually means

Don't do this: reading 36.7% as failure. Rubric thresholds set 7/10 as success, and partial credit across ten independent items means the number compounds differently from pass/fail benchmarks. A model earning 5–6 points consistently is a useful assistant with verification; the board measures distance to autonomy, and 36.7% says supervised copilots, not independent researchers.

The context footnote matters: 1.05M tokens lets the model hold full papers alongside the question. My long-context measurements warn that effective windows run far below advertised — science tasks with whole-paper context sit exactly where position bias bites.

The pattern: rubric evals for your own domain

flowchart TD
    TASK[Domain task] --> RUBRIC[Expert rubric: 10 pts, pass/fail items]
    RUBRIC --> RUN[Model attempts with solution path]
    RUN --> JUDGE[Judge model grades per item]
    JUDGE --> CAL[Human audit: within 0.5 pts?]
    CAL -->|yes| SHIP[Rubric certified, scale it]
    CAL -->|no| FIX[Rewrite ambiguous items]

The transferable invention is not the score but the instrument: independent, objectively assessable rubric items with explicit conditions, calibrated until human-model gaps stay under half a point. Any team evaluating expert work — code review, medical notes, legal drafts — can steal the architecture wholesale.

Step 1: Write rubrics like the paper demands

rubric_guide.md

- Affirmative pass/fail per item (Writes equation X)
- All variables and acronyms defined
- Independent items, no double-counting
- 3-5 hours drafting difficulty per task
- Human-model gap audited under 0.5 pts

Each item must survive a judge that treats the rubric as gold — even when the rubric is wrong. That strictness is deliberate: it forces item quality instead of grader cleverness. My terminal-bench harness applies the same gold-standard discipline to coding tasks.

Step 2: Grade with calibrated judges

grade.py

async def grade_rubric(problem, rubric, answer) -> float:
    try:
        verdict = await judge.ainvoke({
            "problem": problem, "rubric": rubric,
            "answer": answer, "strict": True,
        })
    except RateLimitError as e:
        logger.warning("judge 429", extra={"err": str(e)})
        raise
    return parse_verdict(verdict)  # VERDICT: points_total, last line

Judge prompts return step-by-step reasoning per item then a single VERDICT line — parseable, auditable, and comparable across runs. I pin the judge model version alongside results because judge upgrades silently rescore history; an unevaluated judge swap once moved my suite four points overnight with zero product changes. Human experts audit a sample monthly; any item drifting past 0.5 points gets rewritten, not the judge reprompted. Fix the instrument, not the observer.

requirements.txt

httpx==0.28.1
pydantic==2.8.0
structlog==24.4.0
python-dotenv==1.0.1
numpy==2.1.0

Pydantic v2.8 needs extra="allow" on rubric schemas or nested item payloads fail validation. I lost an afternoon to that exact error before pinning it.

Step 3: Deploy copilots supervised, not autonomous

The 36.7% verdict sets the operating model: models draft, experts dispose. My science-adjacent pipelines require human sign-off on every model-derived claim heading into reports, with rubric scores attached as confidence metadata. Supervised drafting at 5–6 rubric points per subtask already compresses literature review cycles measurably — my team clears survey drafts in roughly half the expert-hours, with the rubric items doubling as review checklists. Autonomy can wait for the board to move; leverage is available now.

The overclaim war story: 77% sold as research-ready

A vendor deck once quoted the Olympiad number as proof of research capability — 77% on theory problems sold as 77% on research. The Research track at a third of that is the rebuttal I now keep bookmarked. Track confusion is the oldest trick in benchmark marketing, and dual-track boards exist precisely to prevent it. I now require every vendor claim in writing with the track named — Olympiad-cited research promises get sent back, and two vendors have quietly revised decks after the question. Always ask which track a claim cites.

When NOT to care

Let's be clear. Teams outside science and engineering can ignore this board — general assistants live on other evals. Small labs cannot afford rubric construction at 3–5 hours per task; use the open gold set instead of building your own. And nobody should route production on a single blended number without reading the track split.

Watch it for copilot capability planning and rubric-eval design. Ignore it for anything the Research track does not resemble.

FrontierScience at 36.7% says the Olympiad era is ending and the research era is hard: supervised copilots today, rubric instruments for every expert domain, and autonomy still over the horizon. The teams that internalize this split — buying copilots for leverage while building rubric evals for their own hardest tasks — will be calibrated when the board finally moves. Everyone else will keep mistaking theory scores for research ability.

By Deepak Bagada, Founder & Editor-in-Chief at Daily AI World.