diegosouzapw/OmniRoute: OmniRoute is an AI gateway for multi-provider LLMs: an OpenAI-compatible endpoint with smart routing, load balancing, retries, and fallbacks. Add policies, rate limits, caching, and observability for reliable, cost-aware inference.

OmniRoute is an AI gateway for multi-provider LLMs: an OpenAI-compatible endpoint with smart routing, load balancing, retries, and fallbacks. Add policies, rate limits, caching, and observability f...

Summary

OmniRoute is an MIT-licensed, TypeScript AI gateway that exposes a single OpenAI-compatible endpoint (default http://localhost:20128/v1) in front of 357+ AI providers and 1200+ model IDs, including Kimi, Claude, GPT, Gemini, GLM, and DeepSeek. It is designed to plug into existing coding CLIs and agents — Claude Code, Codex, Cursor, Cline, Copilot — without rewriting them. The project focuses on fallback across provider tiers, quota awareness, and prompt compression rather than replacing the upstream APIs.

What it is useful for

OmniRoute is useful when an engineer or team wants to consolidate API access across many LLM providers behind one local URL, so tools like Cursor, Cline, or Claude Code don't each need their own provider configuration. The README documents a four-tier fallback ladder — Subscription → API Key → Cheap → Free — with circuit breakers and key-cooldown logic, so a request that fails on one provider can be retried on another when a healthy target exists. It also documents a "Quota-Share" scheduler (v3.8.50+) and live quota telemetry, so you can see which free pool is exhausted. An RTK + Caveman compression pipeline is described as reducing prompt tokens by 15–95%. Local-first operation is highlighted: credentials are stored with AES-256-GCM encryption and requests can stay on-machine, with optional 3-level proxying and TLS stealth for restricted networks.

How engineers can use it

The README's documented zero-config path is npm i -g omniroute, after which the server boots on localhost:20128. A curl example is included:

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'

Because the gateway speaks OpenAI's Chat Completions schema, pointing Claude Code, Cursor, Cline, Codex, or Copilot at http://localhost:20128/v1 is the documented integration path; the README lists 19 routing strategies, MCP/A2A tool surfaces, and 36 CLI/agent integrations configured through one endpoint. Docker images are published (diegosouzapw/omniroute), and copy-paste quickstart scripts for Python, Node.js, PHP, and cURL live in examples/quickstart/.

A few caveats the README itself flags: the "~1.62B free tokens/month" headline is recomputed from 35 documented recurring pool keys and can move in either direction as providers change terms; 13 providers are listed as "avoid" in its terms-risk catalog; and some pools (e.g., ModelScope) only open after regional identity verification and are shown separately rather than summed into the headline. Full methodology is in docs/reference/FREE_TIERS.md.