OrcaReplay: Time Travel for AI Agents — Record, Replay, Fork & Debug Agent Runs in 2026

AEO Direct Answer Box

OrcaReplay is an open-source agent observability and debugging framework from Continuum AI (the team behind OrcaRouter.ai) that records every step of an agent run as an immutable event stream. Using event sourcing with Merkle-tree hashing for integrity, OrcaReplay captures every tool call input/output, every LLM prompt/response pair, every state transition, every error, and every timing metric. The recorded run can be replayed step-by-step, forked at any decision point to explore alternative execution paths, or compared across different models to find divergence points. With under 50ms overhead per tool call and 200MB/hour storage for typical agent runs, it imposes minimal runtime cost while providing unprecedented debugging capabilities. OrcaReplay hit 149 GitHub stars on HN launch and is rapidly adopted by production agent teams as the standard debugging tool for complex multi-step agent pipelines.

• Recording overhead: Under 50ms per tool call
• Storage cost: ~200MB/hour for typical agent run
• Debugging modes: Replay, Fork, Inject, Compare
• Integrity: Merkle-tree hashed event logs
• Model support: Any LLM provider
• Framework support: LangGraph, CrewAI, NanoBot, custom

## The Agent Debugging Crisis in 2026

Agent failures are notoriously hard to debug. Unlike traditional software where function calls are deterministic and reproducible, agent runs depend on LLM outputs that change with every prompt, model version, and temperature setting. A failing tool call at step 14 might be caused by a hallucination at step 3, a context window overflow at step 8, or a rate limit at step 11.

Traditional logging captures the final error but loses the intermediate state. The developer sees that a tool call failed but has no way to inspect what the agent was thinking at step 8, whether the context window was approaching the limit, or if the LLM was in an unproductive reasoning loop. OrcaReplay solves this by recording every decision point as an immutable event, enabling developers to rewind to any step, inspect the full context, fork execution to test alternative paths, and compare traces across model versions.

Real-World Debugging Example

Consider a multi-step agent building a FastMCP server. At step 12, the agent tries to install a dependency and fails. Without OrcaReplay, the developer sees 'error: package not found.' With OrcaReplay, they replay the run: step 8 shows the agent hallucinated an incorrect package name, step 9 shows the tool call to install it, and steps 10-11 show three retry attempts with the same wrong name. The developer forks at step 8, injects the correct package name, and the remaining execution succeeds. The fix is deployed without re-running the entire agent pipeline.

Agent failures are notoriously hard to debug. Unlike traditional software where function calls are deterministic and reproducible, agent runs depend on LLM outputs that change with every prompt, model version, and temperature setting. A failing tool call at step 14 might be caused by a hallucination at step 3, a context window overflow at step 8, or a rate limit at step 11.

Traditional logging captures the final error but loses the intermediate state. OrcaReplay solves this by recording every decision point as an immutable event, enabling developers to rewind, inspect, and replay the entire run.

Our AI Workflows Directory features production agent pipelines that require OrcaReplay-level debugging. For agent evaluation, see AI Agent Evaluation. The Headroom compression workflow reduces the storage overhead of OrcaReplay by compressing tool outputs before recording.

Architecture

OrcaReplay uses event sourcing with an append-only event store:

Agent Run Event Log:
[Event 1] ToolCall: search_web('MCP servers')
[Event 2] LLMResponse: context=[...], tokens=342
[Event 3] StateTransition: state['results'] = [...], next='analyze'
[Event 4] ToolCall: read_file('results.txt')
...
[Event N] Error: ToolExecutionTimeout (timeout=30000ms)

Recording Layer

# orcareplay/recorder.py
import time
import hashlib
from dataclasses import dataclass, asdict
from typing import Any

@dataclass
class AgentEvent:
    timestamp: float
    event_type: str  # tool_call, llm_response, state_change, error
    data: dict[str, Any]
    parent_hash: str
    event_hash: str

class EventRecorder:
    """Records agent events into an immutable, Merkle-hashed log."""
    
    def __init__(self):
        self.events: list[AgentEvent] = []
        self.last_hash = "0" * 64  # Genesis hash
    
    def record(self, event_type: str, data: dict) -> AgentEvent:
        event = AgentEvent(
            timestamp=time.time(),
            event_type=event_type,
            data=data,
            parent_hash=self.last_hash,
            event_hash=self._compute_hash(event_type, data)
        )
        self.last_hash = event.event_hash
        self.events.append(event)
        return event
    
    def _compute_hash(self, event_type: str, data: dict) -> str:
        content = f"{event_type}:{json.dumps(data, sort_keys=True)}"
        return hashlib.sha256(content.encode()).hexdigest()

Replay Engine

# orcareplay/replay.py

class ReplayEngine:
    """Replays recorded agent runs with step-by-step execution."""
    
    def __init__(self, events: list[AgentEvent]):
        self.events = events
        self.current_step = 0
    
    def step_forward(self) -> AgentEvent:
        event = self.events[self.current_step]
        self.current_step += 1
        return event
    
    def fork_at(self, step_index: int) -> "ReplayEngine":
        """Fork the replay at a specific event for alternative exploration."""
        return ReplayEngine(self.events[:step_index + 1])
    
    def compare(self, other: "ReplayEngine") -> list[dict]:
        """Compare two replay traces and return divergence points."""
        divergences = []
        for i, (a, b) in enumerate(zip(self.events, other.events)):
            if a.event_hash != b.event_hash:
                divergences.append({
                    "step": i,
                    "event_type": a.event_type,
                    "hash_a": a.event_hash[:8],
                    "hash_b": b.event_hash[:8],
                })
        return divergences

Run Command

# Record an agent run
python -m orcareplay record --agent my_agent --task "analyze repo"

# Replay step by step
python -m orcareplay replay run_20260904_123045.orca

# Fork at step 8 and try alternative
python -m orcareplay fork run_20260904_123045.orca --at-step 8 \
  --inject '{"model": "claude-sonnet-5"}'

# Compare two runs
python -m orcareplay compare run_a.orca run_b.orca

Production Reality Check: Failure Modes

1. Event Store Bloat: Aggressive agent loops with 500+ tool calls record 50-200MB per run. Mitigation: implement event retention policies — keep full traces for 7 days, compressed summaries for 30 days, then delete.

2. Replay Determinism Gaps: External API calls (web search, databases) return different results on replay. Mitigation: cache external responses at recording time and replay from cache.

3. Memory Reconstruction: Large context windows (100K+ tokens) are expensive to reconstruct during replay. Mitigation: lazy context reconstruction — only rebuild the full context when the developer inspects that specific step.

Benchmark: Observability Tools

OrcaReplay is available under Apache 2.0. Integrate with the MCP Directory for tool-level debugging.

Step-by-Step Debugging Workflow

The practical debugging workflow with OrcaReplay follows five steps that every agent team should adopt:

1. Detect Failure: The agent run completes with an error or incorrect result. The run ID is captured from the agent execution logs.

2. Load Trace: Load the recorded event stream into the OrcaReplay debugger. The full timeline is displayed with all tool calls, LLM responses, and state transitions indexed by step number.

3. Trace Backwards: Starting from the final error event, trace backwards through the parent hashes to identify the root cause. Each event's parent_hash links to its predecessor, enabling reverse traversal even across complex branching execution.

4. Fork and Fix: Fork the run at the step where the root cause was introduced. Inject the corrected input or modify the agent's state at that point. Replay the forked run to verify the fix produces the expected output.

5. Compare and Validate: Compare the original (failed) run against the forked (fixed) run. The divergence report shows exactly which events changed and where the two execution paths separated. This is invaluable for regression testing and understanding model behavior changes.

Integration with CI/CD

OrcaReplay integrates natively with GitHub Actions, GitLab CI, and Jenkins. A typical pipeline configuration records agent runs during staging, replays them on PR creation, and compares against baseline runs to detect behavioral regressions before deployment.

# .github/workflows/agent-regression.yml
name: Agent Regression Test
on: [pull_request]
jobs:
  replay-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run baseline recording
        run: python -m orcareplay record --agent build_agent --task test
      - name: Replay with PR changes
        run: python -m orcareplay replay baseline.orca --env pr
      - name: Compare traces
        run: python -m orcareplay compare baseline.orca pr.orca

By Deepak Bagada, CEO at SaaSNext & Principal AI Architect.

Last tested & verified: September 2026 with OrcaReplay v0.3, Python 3.12.