pi-crew: Durable Multi-Agent Teams with Async Background Execution

pi-crew is a Pi extension by baphuongna that orchestrates autonomous multi-agent workflows with durable state, parallel execution, worktree isolation, and async background runs. It provides a single team tool that handles routing, planning, execution, review, and cleanup across 10 built-in agents (analyst, critic, executor, explorer, planner, reviewer, security-reviewer, test-engineer, verifier, writer) and 6 built-in teams (default, fast-fix, implementation, review, research, parallel-research). The agentic reasoning step occurs through adaptive planning — the implementation workflow lets a planner agent decide the subagent fanout, choosing the smallest effective crew for each task. This is agentic because pi-crew dynamically determines team composition based on task complexity rather than using fixed teams. The system supports 4 runtime modes (auto, child-process, scaffold, live-session), durable persistent state, and Prometheus/OTLP observability.

BUSINESS PROBLEM

Multi-agent Pi workflows lack durability. A code review that spawns 5 subagents — security scan, lint check, architecture review, test coverage analysis, documentation audit — cannot survive a Pi session crash or reload. If any agent fails mid-task, the entire workflow restarts from scratch. According to pi-crew's architecture documentation, state loss in multi-agent Pi sessions is the #1 reported issue in the Pi extension community. Additionally, parallel agents editing the same files create race conditions and merge conflicts. Most Pi extensions ignore this because they run serialized agents with no shared state. pi-crew solves both: durable disk-persisted state ensures crash recovery, and git worktree isolation ensures parallel agents never conflict.

WHO BENEFITS

Pi CLI users running complex code review workflows: you need 5 agents analyzing security, style, architecture, tests, and docs in parallel without stepping on each other. pi-crew's worktree isolation makes this safe. Teams running Pi in CI/CD pipelines: a crashed agent means a failed pipeline. pi-crew's durable state and async background execution ensure workflows complete even across session boundaries. Developers needing Prometheus-observable Pi workflows: pi-crew ships Prometheus and OTLP exporters for metrics registry, heartbeat watching, and deadletter queues — essential for production deployments.

HOW IT WORKS

1. Team Selection: The user selects a team (default, fast-fix, implementation, review, research, or parallel-research) or defines a custom team in .crew/teams/. Each team has a workflow with phases. Output: validated team configuration.
2. Adaptive Planning: The team's workflow starts. For the implementation team, a planner agent analyzes the task and decides the optimal subagent fanout — how many agents, what roles, and what concurrency. The planner considers task complexity, file count, and dependency graph.
3. Parallel Agent Spawn: pi-crew spawns child Pi processes for each task in the current phase. Tasks run concurrently with configurable max concurrency. Each task runs in its own git worktree (optional, enabled per workflow).
4. State Persistence: Every task's state is written to disk — manifest, tasks, events, and artifacts. If the Pi session crashes, the run survives. On reconnect, pi-crew loads the durable state and resumes.
5. Quality Gates: Each phase has completion criteria. The verifier agent evaluates outputs against the task rubric. Tasks that complete without calling submit_result get a needs_attention status instead of completed — allowing retry without blocking downstream phases.
6. Async Completion: Background runs survive session switches. pi-crew notifies the original session when async tasks complete. Users can switch to other work while agents run in the background.
7. Observability Dashboard: Running and completed workflows are visible through pi-crew's live dashboard widget. Metrics export to Prometheus for production monitoring.

TOOL INTEGRATION

pi-crew (baphuongna, MIT): Multi-agent orchestration for Pi CLI. Install: pi install npm:pi-crew. 10 built-in agents, 6 teams, 4 runtime modes. GitHub: github.com/baphuongna/pi-crew. Gotcha: pi-crew's async background mode requires Pi v0.70+ for session switch notification support.

Git Worktree (built-in): Isolated working directories for parallel agent edits. pi-crew creates worktrees under .worktrees/ in the project root. Gotcha: Worktree mode is opt-in and must be enabled per team config. Each worktree consumes ~1x repo disk space.

Prometheus / OTLP (optional): Production observability stack. pi-crew exports metrics via Prometheus endpoint or OTLP exporter. Enables Grafana dashboards for agent workflows. Gotcha: OTLP exporter is experimental in current pi-crew release.

ROI METRICS

1. Crash recovery time: 100% manual restart without state → 0-second resume with durable persistence
2. Parallel review throughput: 1 sequential review → 5 parallel agents in worktree isolation
3. Code conflict rate: 30-40% with shared-directory parallel agents → 0% with git worktree isolation
4. Production observability setup: 2-3 weeks custom → 10 minutes with pi-crew's Prometheus exporter
5. Time to first ROI: first multi-agent review completes 4x faster than sequential (Source: pi-crew documentation, June 2026)

CAVEATS

1. pi-crew uses real Pi child processes. Each agent spawns a separate Pi instance, consuming memory proportional to agent count. With 10 parallel agents, expect 2-5GB RAM usage.
2. Worktree isolation uses significant disk. For large repos (1GB+), each worktree adds 1GB+. Monitor disk usage when running many parallel agents.
3. The adaptive planner makes decisions about agent fanout. In complex or unfamiliar codebases, the planner may under-fanout (too few agents) or over-fanout (unnecessary agents). Tuning requires experimentation.
4. pi-crew is a heavy extension with many dependencies. Install time is longer than zero-dep alternatives. Consider pi-taskflow for simpler DAG needs.