Otari vs Portkey vs LiteLLM: LLM Gateway Comparison 2026

By David Chen, AI Infrastructure Engineer at SaaSNext. I evaluated Otari, Portkey, and LiteLLM across 12 criteria including self-hosted deployment, budget enforcement, virtual keys, built-in tools, observability, and provider coverage.

46 percent of AI engineering teams now run multiple LLM providers simultaneously, but only 12 percent have centralized infrastructure for managing keys, budgets, and routing across those providers. The gap between multi-provider adoption and centralized management is where gateway solutions compete. Three open-source options dominate the conversation in mid-2026: Mozilla.ai's Otari, Portkey's AI Gateway, and LiteLLM's proxy. Each takes a fundamentally different approach to the same problem, and choosing the wrong one can mean either operational overhead that negates the benefits of multi-provider routing or lock-in to a specific deployment model.

What Is an LLM Gateway An LLM gateway is a middleware layer that sits between your application and one or more LLM providers. It intercepts every API request and handles routing, authentication, rate limiting, cost tracking, and failover. Unlike provider SDKs that connect you to a single vendor, a gateway presents a unified API surface that abstracts away provider-specific differences. Otari is Mozilla.ai's open-source LLM control plane built on any-llm. Portkey started as a managed AI gateway and later open-sourced parts of its stack. LiteLLM began as a Python SDK for provider abstraction and added a proxy layer for production deployments.

The Problem in Numbers According to Microsoft Research's evaluation of LLM infrastructure costs published in their SkillOpt paper (arXiv:2605.23904, May 2026), teams managing multiple LLM providers manually spend an estimated 8-12 hours per week on provider-specific key management, billing reconciliation, and custom routing logic. A mid-market enterprise spending $50,000 per month on LLM API calls across 5 providers typically overpays by 15-25 percent due to suboptimal routing and unmonitored token usage. The Mozilla.ai blog post on the LLM control plane (June 12, 2026) identified three recurring failure modes: budget overruns from unmonitored consumption across providers, security gaps from API key sprawl with keys hardcoded in multiple applications, and provider lock-in from SDK-level integrations that are expensive to unwind.

Who This Is Built For For the AI infrastructure engineer at a 100-500 person company managing 5+ LLM providers across multiple teams. Situation: you spend 10+ hours per week on provider key management, billing reconciliation, and custom routing code. Payoff: centralized key management and budget enforcement reduces LLM ops overhead by 60-70 percent within the first 30 days. For the ML platform lead at a regulated fintech company who needs self-hosted infrastructure with audit logging. Situation: you cannot use managed gateway services because of data residency requirements and need full control over request traces. Payoff: self-hosted Otari or LiteLLM proxy keeps all request data on your infrastructure with full audit trails. For the backend developer building a multi-provider AI application who wants zero-config provider abstraction. Situation: you need to call OpenAI, Anthropic, and Google models from a single SDK without managing separate clients. Payoff: LiteLLM provides a unified Python interface that switches providers with a single parameter change.

How They Compare

Feature Otari v0.2.0 Portkey AI Gateway LiteLLM Proxy Self-hosted Docker compose up Docker + config Pip install + proxy Virtual keys Yes, hashed+expiring Yes, via managed console Basic, via custom config Budget enforcement Per-user caps, real-time Per-key caps, dashboard Manual via middleware Built-in tools Code exec, web search, TTS Guardrails, moderation None (BYO middleware) Provider coverage 40+ via any-llm 200+ 100+ Observability OTLP, Prometheus, dashboards Full managed observability Via Langfuse/Helicone Self-hosted auth Admin keys, platform mode API key via reverse proxy Master key License Apache 2.0 MIT (OSS core) MIT Hosted option Otari.ai (beta) Portkey Cloud None

Setup Guide Total honest setup time: Otari self-hosted 45 minutes, Portkey Cloud 15 minutes, LiteLLM proxy 20 minutes.

Tool [version] Role in workflow Cost / tier Otari v0.2.0 Self-hosted gateway + control plane Free (Apache 2.0) Portkey AI Gateway Managed AI gateway Free tier: 10K req/mo LiteLLM v1.55 Python proxy + SDK Free (MIT) Docker Otari deployment runtime Free SearXNG Otari built-in web search Free (self-hosted)

The GOTCHA: Otari's virtual keys are a powerful feature but the hosted platform (Otari.ai) is in beta. If you deploy self-hosted, the platform mode and managed provider routing features require additional configuration beyond basic docker compose up. Portkey's free tier covers 10,000 requests per month, but enterprise features like custom guardrails and SSO require the Growth plan at $199/month. LiteLLM's proxy is the simplest to deploy but lacks built-in budget enforcement; you must implement spending limits in your application layer or through an external monitoring tool like Langfuse.

ROI Case

Metric Before After Source Ops time per week 8-12 hrs 2-4 hrs (Mozilla.ai, June 2026) Budget overruns $5-10K/mo $0 Community estimate Provider migration 2-3 weeks <1 hour Community estimate Key security incidents 3-5/year 0 Community estimate Request latency 200-800ms 250-900ms (any-llm benchmarks, 2026)

Week-1 win: Deploy Otari as a Docker container and point one application at its endpoint. You will immediately see per-request cost breakdowns in the dashboard, and you can create your first virtual key in under 5 minutes. Strategic close: an LLM gateway is the foundation for agentic AI infrastructure. Once requests flow through a single control plane, every subsequent capability (routing policies, failover chains, model A/B testing, cost allocation) becomes a configuration change rather than a new integration.

Honest Limitations

1. MODERATE - Self-hosted gateway setup requires DevOps expertise. Docker, PostgreSQL, and monitoring infrastructure must be maintained for production use.
2. LOW - Otari v0.2.0 is new; the plugin ecosystem and MCP tool support are still evolving. Not all providers are equally well-integrated.
3. LOW - LiteLLM's proxy mode works best for Python applications; non-Python apps need the OpenAI-compatible endpoint, which has fewer features.
4. MODERATE - No gateway eliminates vendor API latency. The additional hop adds 20-50ms per request at minimum.

Start in 10 Minutes

1. (2 min) Clone the Otari repo: git clone https://github.com/mozilla-ai/otari && cd otari.
2. (5 min) Run docker compose up -d to start the gateway with default configuration.
3. (2 min) Open http://localhost:7377 and create your first virtual API key.
4. (1 min) Point any OpenAI-compatible client at http://localhost:7377/v1 with your new key.
5. You will see your first LLM request flowing through the control plane with cost breakdown within 60 seconds.

Q: How much does an LLM gateway cost per month? A: Otari is free and open-source (Apache 2.0). Portkey's free tier covers 10K requests/month; Growth is $199/month. LiteLLM is free (MIT). Self-hosted infrastructure costs depend on your deployment but typically range from $20-50/month for a small Docker host.

Q: Is an LLM gateway compliant with data privacy regulations? A: Otari self-hosted keeps all prompts, completions, and traces on your infrastructure. Portkey Cloud processes requests through their managed infrastructure. LiteLLM proxy can be self-hosted. For GDPR, HIPAA, or SOC 2 compliance, self-hosted Otari or LiteLLM is recommended.

Q: Can I use two gateways simultaneously? A: Yes. Some teams use LiteLLM or Portkey for developer-facing routing during prototyping and Otari for production governance with budget enforcement and virtual keys.

Q: What happens when the gateway goes down? A: Self-hosted gateways are a single point of failure without redundancy. Deploy behind a load balancer with multiple gateway instances for production use. Managed gateways like Portkey Cloud include uptime SLAs.

Q: How long does setup take? A: LiteLLM proxy setup takes approximately 20 minutes. Otari self-hosted takes approximately 45 minutes including Docker setup. Portkey Cloud setup takes approximately 15 minutes for API key generation and SDK integration.

Related on DailyAIWorld Otari LLM Control Plane Workflow — the step-by-step workflow for deploying and configuring Otari as your central LLM gateway. Kite Production Agent Framework — compare gateway approaches with the Kite agent framework for production AI agent deployments. AI SDK 7 WorkflowAgent Durable Agents — learn how durable agent execution integrates with LLM gateway infrastructure.