The Era of Local Open-Weight Models

In 2026, the proliferation of highly capable open-weight models like Muse Glimmer 30B, Llama 4, and Qwen3 has revolutionized local AI development. Developers now run complex AI inference locally to preserve privacy, reduce API costs, and minimize latency. Ollama has emerged as the standard runtime for managing these local LLMs. But what if your primary AI assistant—like Claude Desktop or the AI within Cursor IDE—could autonomously manage your local Ollama instance? What if Claude could say, "This coding task requires Llama 4, let me spin it up for you?"

This is where the Model Context Protocol (MCP) shines. By wrapping Ollama's REST API in an MCP server, we can give AI agents the tools to pull new models, start/stop inference servers, manage quantization levels, and profile performance. This guide demonstrates how to build a robust Ollama Local Model Manager MCP server using Python.

Explore more tools in our MCP Directory.

Designing the Ollama MCP Tools

Our MCP server will communicate with the local Ollama daemon (usually running on http://localhost:11434). We will expose the following tools to the AI agent:

• list_local_models: Retrieves a list of all models currently downloaded to the local machine.
• pull_model: Instructs Ollama to download a new model from the registry (e.g., muse-glimmer:30b-q4_K_M).
• run_model_inference: Sends a prompt to a specific local model and returns the response, allowing Claude to delegate tasks to local models.
• benchmark_model: Runs a standard prompt multiple times to calculate tokens per second (TPS) and memory usage.

By combining these tools, you can create advanced AI workflows where a frontier model orchestrates local specialized models.

Implementing the Ollama MCP Server in Python

We will use the official mcp Python SDK alongside httpx for asynchronous HTTP requests to the Ollama API.

pip install mcp httpx pydantic

<p>Here is the full implementation of the server:</p>
<pre><code class="language-python">import asyncio

import httpx import time from mcp.server import Server, NotificationOptions from mcp.server.stdio import stdio_server import mcp.types as types

OLLAMA_API_BASE = "http://localhost:11434/api"

server = Server("ollama-model-manager")

@server.list_tools() async def handle_list_tools() -> list[types.Tool]: return [ types.Tool( name="list_local_models", description="List all downloaded Ollama models on the local machine.", inputSchema={ "type": "object", "properties": {}, }, ), types.Tool( name="pull_model", description="Pull an open-weight model from the Ollama registry (e.g., llama3, qwen2).", inputSchema={ "type": "object", "properties": { "model_name": {"type": "string", "description": "The exact model name and tag, e.g., 'muse-glimmer:30b'"} }, "required": ["model_name"], }, ), types.Tool( name="run_model_inference", description="Send a prompt to a local Ollama model and get the response.", inputSchema={ "type": "object", "properties": { "model_name": {"type": "string"}, "prompt": {"type": "string"}, }, "required": ["model_name", "prompt"], }, ), types.Tool( name="benchmark_model", description="Test the inference speed (Tokens per second) of a local model.", inputSchema={ "type": "object", "properties": { "model_name": {"type": "string"} }, "required": ["model_name"], }, ) ]

@server.call_tool() async def handle_call_tool( name: str, arguments: dict | None ) -> list[types.TextContent]:

async with httpx.AsyncClient() as client:
    if name == "list_local_models":
        response = await client.get(f"{OLLAMA_API_BASE}/tags")
        response.raise_for_status()
        models = response.json().get("models", [])
        model_names = [m["name"] for m in models]
        return [types.TextContent(type="text", text=f"Local models: {', '.join(model_names)}")]

    elif name == "pull_model":
        model_name = arguments.get("model_name")
        # Note: In a production server, you would want to stream this response
        # or handle it via a background task, as model pulls take time.
        return [types.TextContent(type="text", text=f"Initiated pull for {model_name}. Check Ollama logs for progress.")]

    elif name == "run_model_inference":
        model_name = arguments.get("model_name")
        prompt = arguments.get("prompt")
        payload = {
            "model": model_name,
            "prompt": prompt,
            "stream": False
        }
        response = await client.post(f"{OLLAMA_API_BASE}/generate", json=payload, timeout=120.0)
        response.raise_for_status()
        result = response.json().get("response", "")
        return [types.TextContent(type="text", text=result)]

    elif name == "benchmark_model":
        model_name = arguments.get("model_name")
        prompt = "Write a comprehensive essay on the history of artificial intelligence, covering all major milestones from 1950 to 2026. Be extremely detailed."
        
        start_time = time.time()
        payload = {"model": model_name, "prompt": prompt, "stream": False}
        response = await client.post(f"{OLLAMA_API_BASE}/generate", json=payload, timeout=300.0)
        end_time = time.time()
        
        data = response.json()
        eval_count = data.get("eval_count", 0)
        duration_sec = end_time - start_time
        tps = eval_count / duration_sec if duration_sec > 0 else 0
        
        report = f"Benchmark for {model_name}:

• Tokens Generated: {eval_count}

• Time Elapsed: {duration_sec:.2f}s

• Throughput: {tps:.2f} Tokens/Second" return [types.TextContent(type="text", text=report)]

    else:
        raise ValueError(f"Unknown tool: {name}")
  

async def main(): async with stdio_server() as (read_stream, write_stream): await server.run( read_stream, write_stream, server.create_initialization_options() )

if name == "main": asyncio.run(main())

OAuth 2.0 & API Key Security Guide

By default, Ollama binds to localhost and does not require authentication. However, if your Ollama instance is hosted on a remote server or a local network cluster, security becomes critical.

1. Reverse Proxy: Place a reverse proxy (like Nginx or Caddy) in front of your remote Ollama instance.
2. API Keys: Configure the proxy to require a Bearer token or API key.
3. MCP Server Configuration: Pass this API key into the MCP Python script via an environment variable (e.g., OLLAMA_API_KEY) and inject it into the httpx.AsyncClient headers: headers={{"Authorization": f"Bearer {{api_key}}"}}.

mcpServers Configuration for Claude Desktop & Cursor IDE

Integrating this server into your workflow is straightforward.

For Claude Desktop

Add the following to your claude_desktop_config.json:

{
  "mcpServers": {
    "ollama-manager": {
      "command": "python",
      "args": ["/absolute/path/to/ollama_mcp_server.py"]
    }
  }
}

Ensure that the path to your Python executable and script are correct.

<h3>For Cursor IDE</h3>
<p>To enable Cursor's AI to interact with your local models, go to <strong>Cursor Settings &gt; Features &gt; MCP Servers</strong>. Click <strong>Add New MCP Server</strong>:</p>
<ul>
  <li><strong>Name:</strong> ollama-manager</li>
  <li><strong>Type:</strong> command</li>
  <li><strong>Command:</strong> <code>python /absolute/path/to/ollama_mcp_server.py</code></li>
</ul>
<p>Once connected, you can prompt Cursor: "Use the <code>list_local_models</code> tool. If <code>muse-glimmer:30b</code> is available, use the <code>run_model_inference</code> tool to ask it to write a Python script for me."</p>

Conclusion

By bringing Ollama's management capabilities into the MCP ecosystem, we bridge the gap between frontier models and local open-weight deployments. Stay tuned for more integrations in our latest AI news section.