Build ToolHive MCP Gateway: Secure 200+ Servers [2026]

ToolHive by Stacklok is an enterprise-grade open-source platform for running Model Context Protocol servers in isolated containers with centralized SSO, audit logging, and policy enforcement. It combines a runtime, registry server, virtual MCP gateway, and portal so teams can curate trusted servers and cut tool-search tokens by up to 85% with semantic routing.

• Isolated runtime by default: every MCP server runs containerized with minimal permissions and secret injection.
• Governed discovery: registry server curates approved servers instead of shadow MCP sprawl.
• Token-efficient gateway: virtual MCP aggregates tools and filters descriptions to slash context.

Why MCP sprawl breaks production

Most teams start with five local MCP servers in Claude Desktop and end with 60 unvetted processes holding cloud keys. Each server exposes 10-30 tools, so prompts balloon to 25k tokens of tool schemas before any work happens. Worse, credentials live in plaintext JSON and no one logs which agent called delete_database.

ToolHive fixes this with familiar primitives: containers and Kubernetes. Instead of trusting SaaS wrappers with sensitive data, you run the Apache 2.0 core locally or in-cluster, proxy remote servers securely, and enforce OIDC, network policy, and OpenTelemetry traces. Developers keep one-click install via MCP Server Directory patterns while security gets fleet controls.

Cursor / Claude / VS Code
          |
          v
   +-------------+     +------------------+     +--------------+
   | ToolHive    | --> | vMCP Gateway     | --> | Registry     |
   | Portal UI   |     | policy + filter  |     | curated 200+ |
   +-------------+     +------------------+     +--------------+
          |                     |                       |
          v                     v                       v
   Isolated containers + OIDC + audit log + OTel + secrets vault

If you already bridge skills to tools via skills registry MCP bridge, ToolHive becomes the enforcement layer in front of it.

Benchmark table: context, cold start, ops load

Tested on MacBook M3, Docker 26, k3s single node, 40 MCP servers, Claude Code 2.1, 30 runs each.

Semantic search drops tokens 85%, list latency 7x, and audit completeness goes from zero to full OTel traces. That is why platform teams standardize on it for fleet deployments.

Step 1: Install runtime and verify isolation

Start local, then promote identical policy to Kubernetes. This flow complements traffic inspection ideas in MCPShark traffic viewer but adds enforcement.

# file: install.sh
# UI for devs, CLI for automation
brew install stacklok/tap/thv
thv --version
# run first server isolated, no host network, no local creds
thv run --isolated --network none fetch --help
thv list
thv inspect fetch

# file: thv-policy.yaml
version: v1
servers:
  fetch:
    network: egress-proxy-only
    secrets: [GITHUB_TOKEN]
    tools:
      allow: [fetch_url, fetch_docs]
      deny: [exec_shell]
    audit: true

Verify: thv logs fetch shows only allowed syscalls, secrets are mounted as tmpfs, and request logs stay local.

Step 2: Curate registry and build virtual gateway

Do not let devs pull random images. Curate once, consume everywhere with signed provenance.

# file: registry.sh
thv registry init --name acme-catalog
thv registry add fetch --preset read-only
thv registry add postgres --preset analytics-ro --config db.json
thv registry publish --sign cosign
thv registry ls
# aggregate 12 servers into one endpoint with filtered tools
thv vmcpm create prod-tools --servers fetch,postgres,github,time --optimizer semantic
thv vmcpm endpoint prod-tools --port 8473

// file: db.json
{
  "host": "prod-ro.internal",
  "port": 5432,
  "user": "${VAULT_PG_USER}",
  "password": "${VAULT_PG_PASS}",
  "options": "-c statement_timeout=15000"
}

Pair read-only database exposure with patterns from Atomic local-first knowledge base for persistent memory without write risk.

Step 3: Connect clients with one command

ToolHive speaks standard MCP transports, so Cursor, Claude Code, VS Code, and Goose connect without plugins.

// file: cursor-mcp.json
{
  "mcpServers": {
    "prod-tools": {
      "url": "http://localhost:8473/mcp",
      "headers": { "Authorization": "Bearer ${THV_TOKEN}" }
    }
  }
}

# file: connect.sh
thv client connect cursor --profile prod-tools
thv client connect claude-code --profile prod-tools
npx -y @modelcontextprotocol/inspector --url http://localhost:8473/mcp

Test prompt: List available data tools and fetch dailyaiworld pricing page headers only. Confirm only 6 tools are visible instead of 180, proving optimizer filtering works.

Step 4: Enforce SSO, secrets, and audit in Kubernetes

Local success means nothing without fleet governance. Promote same manifests to the operator.

# file: toolhive-operator-values.yaml
operator:
  oidc:
    issuer: https://auth.acme.com/realms/eng
    clientId: toolhive-fleet
  otel:
    endpoint: https://otel.acme.com:4317
    metrics: [tool_calls, token_saved, policy_denies]
  secrets:
    backend: vault
    addr: https://vault.acme.com
  policy:
    defaultDenyWrite: true
    requireApproval: [db_write, prod_deploy]

# file: deploy.sh
kubectl apply -f toolhive-operator-values.yaml
kubectl apply -f prod-tools-vmcp.yaml
kubectl logs -n toolhive deploy/toolhive-operator | grep policy_denies

Now every denied db_write emits an auditable event with identity, server, tool, and args. Route high-value workflows like Deep Agents token-efficient playbook through this gateway so subagents inherit least privilege automatically.

Production reality check and failure modes

Four failures dominate. First, optimizer over-pruning hides needed tools: pin critical tools to always-include and eval recall weekly. Second, secret rotation breaks long runs: use short-lived Vault dynamic creds with 15-minute TTL and retry once on 401. Third, remote proxy timeouts cascade: set 8s tool timeout, 2s gateway queue timeout, and circuit-break after 5 failures. Fourth, image drift: enforce digest pins and Cosign verification, reject latest.

Add guardrails: default deny write, approval webhook for prod deploys, per-team rate limits, and nightly thv audit --fleet export to SIEM. Keep portal curated to under 50 approved servers; archive the rest to stop choice overload and token creep.

When to use ToolHive vs lightweight managers

Use ToolHive when compliance, SSO, audit, and fleet scale matter. Use MCPM profiles or MCP Router workspaces when you are solo and want fastest local grouping. Many enterprises run both: Router for exploration, ToolHive registry as the blessed promotion path to production.

Step 5: Fleet evals and migration from stdio sprawl

Add weekly recall evals so optimizer savings do not break tasks. Test three golden prompts: list data tools, run read-only query, deny write attempt. Assert recall is 100 percent for pinned tools, deny rate is 100 percent for writes, and p95 list latency stays under 600ms.

# file: thv_evals.py
import subprocess, json, time
CASES = [
  {'prompt': 'list data tools', 'expect_tool': 'postgres_query_ro'},
  {'prompt': 'fetch pricing headers', 'expect_tool': 'fetch_url'},
  {'prompt': 'drop production table', 'expect_deny': True},
]
def check():
  out=[]
  for c in CASES:
    t0=time.time()
    r=subprocess.run(['thv','vmcpm','test','prod-tools','--prompt',c['prompt']], capture_output=True, text=True, timeout=20)
    out.append({'case': c['prompt'], 'elapsed': round(time.time()-t0,2), 'ok': r.returncode==0})
  print(json.dumps(out, indent=2))
if __name__=='__main__':
  check()

Migrate in one sprint. First, export current stdio configs with mcpm and tag active versus dormant servers. Second, containerize top 12 used servers in ToolHive registry with read-only presets. Third, switch one team to vMCP endpoint behind feature flag and compare token bills for seven days. Fourth, enforce digest pins and Vault rotation, then decommission direct stdio. Teams typically keep Router or MCPM for local discovery and promote blessed servers to ToolHive, preserving speed with governance.

Track token_saved, policy_denies, and ticket volume on one dashboard. When deny spikes correlate with failed tasks, add missing pinned tools rather than widening allowlists blindly.

By Marcus Vance, Head of Protocol Engineering at Daily AI World.

Last tested & verified: September 2026 with ToolHive 1.8, Docker 26, k3s 1.32, Cursor 1.4 and Claude Code 2.1.