Submatter/docs/omniroute-setup.md

OmniRoute Setup Guide

Status: Draft
Date: 2026-09-07
Purpose: Step-by-step guide for operators to register OmniRoute as a model provider in Echodo, configure role routes, and test that everything works.

---

What is OmniRoute?

OmniRoute is an OpenAI-compatible model routing gateway. It provides dynamic model profiles (auto, auto/coding, auto/coding-pro, auto/reasoning, auto/reasoning-pro) that automatically resolve to the best available model based on your OmniRoute plan and current load.

Benefits for Echodo:

• Single API key, 929+ models available
• Auto-routing: no need to pick individual models per role
• Fallback/resilience: if one model is down, OmniRoute switches automatically
• Works through Echodo's existing openai_compatible provider type — zero code changes required

---

Prerequisites

1. Echodo is running locally (pnpm dev) or deployed (Coolify/Dokploy)
2. Database is initialized (pnpm db:push or pnpm db:migrate)
3. You have an OmniRoute account/API key (or plan to use the local instance)

---

Step 1: Get OmniRoute Credentials

Option A: OmniRoute Cloud (recommended)

1. Sign up at omniroute.example.com
2. Create an API key in Settings → API Keys
3. Note your:
   • Base URL: https://omniroute.example.com/v1 (check your dashboard)
   • API Key: sk-... (save securely — not committed to git)

Option B: Self-hosted OmniRoute (local)

If running OmniRoute locally (e.g., via Docker):

1. Clone/run the OmniRoute server
2. Note your:
   • Base URL: http://localhost:11434/v1 (or whatever port)
   • API Key: any-non-empty-string (self-hosted often ignores auth)

---

Step 2: Set Environment Variables

Add to your .env (or Coolify environment):

# OmniRoute credentials
OMNIROUTE_BASE_URL=https://omniroute.example.com/v1
OMNIROUTE_API_KEY=sk-your-key-here

# (Already set by default) Database
DATABASE_URL=postgres://echodo:echodo@localhost:5432/echodo

# (Already set by default) Redis (for orchestrator + worker)
REDIS_URL=redis://localhost:6379

Important: These env vars must be available to:

• The web app (apps/web) — for Settings UI
• The worker (apps/worker) — for inference
• The orchestrator (apps/orchestrator) — for dispatch

---

Step 3: Register OmniRoute Providers in Echodo

Via Settings UI (recommended)

1. Open http://localhost:3001 (web app)
2. Go to Settings → AI → Providers
3. Click Add Provider and create one for each profile:

Provider 1: omni-auto

Field	Value
Display Name	OmniRoute Auto
Alias	omni-auto
Provider Type	openai_compatible
Model ID	auto
Base URL	${OMNIROUTE_BASE_URL} (your actual URL)
API Key Env Var	OMNIROUTE_API_KEY
Metadata	{"tier": "auto", "provider": "omniroute"}
Enabled	✅

Provider 2: omni-coding

Field	Value
Display Name	OmniRoute Coding
Alias	omni-coding
Provider Type	openai_compatible
Model ID	auto/coding
Base URL	(same as above)
API Key Env Var	OMNIROUTE_API_KEY
Metadata	{"tier": "auto", "provider": "omniroute"}
Enabled	✅

Provider 3: omni-coding-pro

Field	Value
Display Name	OmniRoute Coding Pro
Alias	omni-coding-pro
Provider Type	openai_compatible
Model ID	auto/coding-pro
Base URL	(same)
API Key Env Var	OMNIROUTE_API_KEY
Metadata	{"tier": "auto", "provider": "omniroute"}
Enabled	✅

Provider 4: omni-reasoning

Field	Value
Display Name	OmniRoute Reasoning
Alias	omni-reasoning
Provider Type	openai_compatible
Model ID	auto/reasoning
Base URL	(same)
API Key Env Var	OMNIROUTE_API_KEY
Metadata	{"tier": "auto", "provider": "omniroute"}
Enabled	✅

Provider 5: omni-reasoning-pro

Field	Value
Display Name	OmniRoute Reasoning Pro
Alias	omni-reasoning-pro
Provider Type	openai_compatible
Model ID	auto/reasoning-pro
Base URL	(same)
API Key Env Var	OMNIROUTE_API_KEY
Metadata	{"tier": "auto", "provider": "omniroute"}
Enabled	✅

Via tRPC (for automation)

// In a script or test
await trpc.ai.models.createProvider.mutate({
  alias: "omni-coding",
  displayName: "OmniRoute Coding",
  providerType: "openai_compatible",
  modelId: "auto/coding",
  baseUrl: "https://omniroute.example.com/v1",
  apiKeyEnvVar: "OMNIROUTE_API_KEY",
  metadata: { tier: "auto", provider: "omniroute" }
});

---

Step 4: Map Roles to Profiles

Via Settings UI

1. Go to Settings → AI → Role Routing
2. For each role, click Edit and select the provider:

Role ID	Select Provider
role-conductor	omni-reasoning
role-architect	omni-reasoning
role-ia-architect	omni-reasoning
role-implementer	omni-coding
role-reviewer	omni-coding-pro
role-ux-reviewer	omni-coding
role-doc-writer	omni-coding
role-design-system-auditor	omni-coding
role-a11y-auditor	omni-coding
gate-copilot	omni-reasoning
chat-default	omni-auto

Via tRPC

// Example: route implementer to omni-coding
await trpc.ai.models.upsertRoleRoute.mutate({
  roleId: "role-implementer",
  providerAliasId: "omni-coding-provider-uuid"  // get from listProviders
});

---

Step 5: Test Connectivity

Via Settings UI

1. Go to Settings → AI → Providers
2. Click the Test button next to omni-coding
3. You should see: ✅ Success — latency: 320ms

Via MCP (if worker is running)

// testProvider
{
  "tool": "testProvider",
  "arguments": { "alias": "omni-coding" }
}

Via direct curl

curl ${OMNIROUTE_BASE_URL}/chat/completions \
  -H "Authorization: Bearer ${OMNIROUTE_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto/coding",
    "messages": [{"role": "user", "content": "Say hello in 3 words"}],
    "max_tokens": 20
  }'

Expected: A response like {"choices": [{"message": {"content": "Hello world!"}}]}

---

Step 6: Test Model Resolution

Verify a specific role resolves correctly

Use the debug endpoint (or MCP resolveForRole):

resolveForRole({ roleId: "role-implementer" })
// Expected output:
// {
//   ok: true,
//   model: {
//     alias: "omni-coding",
//     providerType: "openai_compatible",
//     modelId: "auto/coding",
//     baseUrl: "https://omniroute.example.com/v1",
//     apiKey: "<redacted>",
//     tier: "auto"
//   },
//   source: "workspace",
//   resolvedAlias: "omni-coding"
// }

Verify resolution order works

Create a test convoy with a model override:

# .convoys/test-resolver.md
---
name: test-resolver
model_policy:
  roles:
    role-implementer: omni-reasoning-pro   # overrides workspace route
---

Then resolveForRole with convoyContext:

resolveForRole({
  roleId: "role-implementer",
  convoyContext: { model_policy: { "role-implementer": "omni-reasoning-pro" } }
})
// → source: "convoy", alias: "omni-reasoning-pro"

---

Step 7: Verify Worker Picks Up the Model

1. Start the worker:

   pnpm --filter @tasks/worker dev
   # or
   WORKER_DRY_RUN=0 ORCHESTRATOR_DRY_RUN=0 REDIS_URL=redis://localhost:6379 pnpm --filter @tasks/worker dev
   

2. Start the orchestrator:

   pnpm --filter @tasks/orchestrator dev
   

3. Enqueue a test job (via MCP create_convoy → transition to ready)

4. Watch worker logs:

   • Should see: Dequeued job for role-implementer, resolved model: omni-coding (auto/coding)
   • Should see: inference call to OmniRoute endpoint

5. Verify model metadata in agent_runs:

   SELECT model_alias, provider_type, model_id, model_tier
   FROM agent_runs
   ORDER BY created_at DESC
   LIMIT 5;
   

   Expected: omni-coding | openai_compatible | auto/coding | auto

---

Troubleshooting

"Provider alias 'omni-coding' is not registered in this workspace"

Cause: Provider not created yet, or alias doesn't match.

Fix:

1. Check Settings → AI → Providers for the alias name
2. Re-run createProvider with exact alias

"Environment variable OMNIROUTE_API_KEY is not set"

Cause: Worker process doesn't have the env var loaded.

Fix:

1. Verify OMNIROUTE_API_KEY is in .env at repo root
2. Restart the worker process (env vars aren't hot-reloaded)
3. Check: echo $OMNIROUTE_API_KEY in the worker terminal

"Connection refused" or "ECONNREFUSED"

Cause: OmniRoute server not reachable at base_url.

Fix:

1. Verify OmniRoute is running: curl ${OMNIROUTE_BASE_URL}/models
2. Check base URL ends with /v1
3. For local OmniRoute: check port (11434 vs 8000 vs other)

"Unsupported provider type"

Cause: providerType value not in providerTypeSchema.

Fix: Must be one of: anthropic, openai, google, openai_compatible, azure_openai. For OmniRoute, use openai_compatible.

Model returns empty or unexpected response

Cause: OmniRoute profile not available or rate-limited.

Fix:

1. Check OmniRoute dashboard for rate limits
2. Try a different profile (e.g., auto instead of auto/coding)
3. Check OmniRoute status page

Resolution returns wrong source

Cause: Convoy or brief override taking precedence unexpectedly.

Fix: Remember the priority order:

1. Brief recommended_model (highest)
2. Convoy model_policy.roles[roleId]
3. Workspace role_model_routes row
4. Pipeline defaults (lowest)

If you want workspace route to win, remove model_policy from the convoy.

---

Security Notes

• API key never in DB: api_key_env_var stores the name of the env var, not the value
• API key never sent to browser: Settings UI shows env var name only
• Worker reads env var at dispatch time: Not stored in job payload
• Test endpoint runs server-side: Rate-limited per workspace

---

Related Docs

• Role mapping: docs/omniroute-role-mapping.md (profile → role matrix, resolution flow)
• ADR: docs/adr-model-routing.md (architecture decision record)
• Skill: .agents/skills/model-routing/SKILL.md (developer reference)
• Provider schema: packages/database/src/schema/model_providers.ts
• Route schema: packages/database/src/schema/role_model_routes.ts