TDD in the Agent Loop: Theater Until Tests Map the Blast

I instructed my coding agent to follow strict TDD last spring: red, green, refactor, no implementation without a failing test first. The results looked disciplined and performed worse — more files touched per fix, more regressions shipped, longer trajectories. The agent obeyed the ceremony and missed the point.

TDD inside the agent loop is theater unless tests arrive with maps: procedural instructions without test context raise regressions, while test-impact maps cut them 70% and human-written reproduction tests unlock 94.3% resolution. Three facts anchor the evidence:

• EACL 2026 TDFlow hits 88.8% on SWE-Bench Lite and 94.3% on Verified with human-written tests — but only 68–69.8% generating its own, proving test authorship is the frontier.
• Test-driven impact analysis cuts regressions 6.08% to 1.82% while bare TDD prompting raises them to 9.94% — procedure without context backfires.
• Field experiments find no discernible TDD-vs-direct quality difference from agent loops, with non-TDD solutions sometimes ranking higher on design.

This is the testing discipline behind my agent evaluations, measured with the same harness rigor as my terminal-bench showdowns. Same method, applied to the TDD question instead of the model question.

The red-green-refactor rollout that regressed more

The TDD fleet touched 40% more files per fix than the direct fleet. Each red-green cycle encouraged locally-minimal decisions that locked in whatever shape the first test happened to fix, and behavior without a test never got implemented at all. Regression rate climbed from 4.1% to 6.9% while trajectories grew 35% longer. Discipline theater, billed hourly.

Here's the catch. TDD's human benefits — forced design thinking, fear management, progress lock-in — assume a human in the friction. An agent writing its own test, confirming its own red, and implementing to its own green proves only that it ran commands, not that the failure was for the right reason. My logs showed skipped red steps, tests checking implementation output against itself, and implementations racing ahead of tests so everything passed on arrival.

That matches the field finding: agents instructed in TDD frequently over-implement because the full requirement sits in context, and minimal-implementation instructions do not reliably stop it. My localization measurements point at the missing piece — the agent knew the procedure but not which tests guarded the blast radius.

What the 2026 evidence actually says

Don't do this: mandating red-green-refactor in the system prompt and declaring quality handled. The paradox is measured: procedural text without targeted test context produced 42% more failures than vanilla and five catastrophic regressions against three. I deleted the TDD instructions and kept the test map.

The pattern: humans author, maps guide, agents resolve

flowchart TD
    HUMAN[Human writes repro test] --> MAP[Impact map: code to tests]
    MAP --> SOLVE[Agent resolves against tests]
    SOLVE --> VERIFY[Verify: repro passes, neighbors green]
    VERIFY -->|regression| FIX[Fix with map-guided scope]
    VERIFY -->|clean| MUTATE[Mutation-score the suite]

The division of labor is the entire insight. Humans own the one thing agents cannot generate reliably — valid reproduction tests capturing intent. Maps own blast-radius knowledge — which tests guard which code. Agents own resolution against that scaffolding, where EACL work shows they already reach human-level 94.3%.

Step 1: Build the impact map first

impact_map.py

def build_map(repo: str) -> dict[str, list[str]]:
    graph = parse_imports(repo)          # tree-sitter, cached
    test_index = index_tests(repo)       # pytest collection
    mapping = {}
    for src in graph.files:
        mapping[src] = reachable_tests(src, graph, test_index)
    write_skill_file(mapping, "SKILL.md")  # 20-line agent brief
    return mapping

The map ships as a twenty-line SKILL.md the agent queries at runtime — static text, no graph database, no API calls. Before committing a patch the agent knows exactly which tests to verify and self-corrects. My repo-map pipeline generates the underlying graph; this file is its testing face.

Step 2: Write repro tests that actually reproduce

A valid repro test fails before the fix and passes after — the bad-test rate is the metric that predicts everything. My suite tracks it per task: instances with zero bad tests resolve at 93%; instances with none succeeding barely resolve at all. Invest review effort in repro validity, not in procedure compliance.

tdd_loop.py

async def resolve_with_tests(issue, repro_tests, impact_map) -> dict:
    if not all(t.fails_without_fix() for t in repro_tests):
        return {"resolved": False,
                "reason": "invalid repro: must fail pre-fix"}
    patch = await propose_fix(issue, repro_tests)
    try:
        results = await run_tests(repro_tests +
                                  impact_map.at_risk(patch.files))
    except TestInfraError as e:
        logger.warning("test infra down", extra={"err": str(e)})
        raise
    if results.repro_pass and results.neighbors_green:
        return {"resolved": True, "patch": patch}
    return await debug_loop(patch, results, impact_map)

The pre-fix failure check is the red step that matters — executable, verifiable, unfakeable. Everything else in traditional TDD is negotiable once this gate holds.

requirements.txt

tree-sitter==0.25.0
pytest==8.3.0
pydantic==2.8.0
networkx==3.4.0
structlog==24.4.0

Pydantic v2.8 needs extra="allow" on mapping schemas or nested file-test payloads fail validation. I lost an afternoon to that exact error before pinning it.

Step 3: Score suites with mutation, not ceremony

Regression quality gets monitored with mutation testing instead of hoped for via ritual. My monthly mutation run seeds faults across covered lines; surviving mutants mark tests that assert nothing. Current kill rate is 91%, and every dip pages the suite, not the model. I do not care how the red was achieved — I care that the suite kills mutants.

Pair this with per-task cost tracking: map-guided resolution cut my cost per resolved issue 31% by killing the ambitious multi-file flailing that bare-TDD prompting encouraged.

The self-check war story: tests asserting the bug

My lowest moment was a green suite defending broken behavior — generated tests asserting the implementation's own output as expected, including one encoding the actual bug as correct. The loop was airtight and wrong. Now golden repro tests are human-authored or human-adjudicated, generated tests are suspects until mutation-scored, and any test written against unreviewed implementation gets quarantined. Trust the gate, suspect the generator.

When NOT to bother

Let's be clear. Greenfield prototypes with no test suite need generation speed, not gates — add the map when the suite exists. One-file scripts never repay impact analysis; grep suffices. And if nobody reviews repro validity, the whole apparatus rests on sand — the human test-review hour is the load-bearing piece, not the tooling.

Skip it for prototypes and scripts. Build it where agents modify shared code, where regressions already top the rejection reasons, and where the last postmortem found tests asserting the bug.

Maps over mantras, repro validity over ritual, and the whole class of disciplined-but-broken agent output disappears: 70% fewer regressions, human-level resolution, and suites that kill mutants instead of performing TDD.

By Deepak Bagada, Founder & Editor-in-Chief at Daily AI World.