Submatter/docs/omniroute-setup.md

# OmniRoute Setup Guide

**Status:** Draft  
**Date:** 2026-09-07  
**Purpose:** Step-by-step guide for operators to register OmniRoute as a model provider in Echodo, configure role routes, and test that everything works.

---

## What is OmniRoute?

OmniRoute is an OpenAI-compatible model routing gateway. It provides **dynamic model profiles** (`auto`, `auto/coding`, `auto/coding-pro`, `auto/reasoning`, `auto/reasoning-pro`) that automatically resolve to the best available model based on your OmniRoute plan and current load.

**Benefits for Echodo:**
- Single API key, 929+ models available
- Auto-routing: no need to pick individual models per role
- Fallback/resilience: if one model is down, OmniRoute switches automatically
- Works through Echodo's existing `openai_compatible` provider type — zero code changes required

---

## Prerequisites

1. Echodo is running locally (`pnpm dev`) or deployed (Coolify/Dokploy)
2. Database is initialized (`pnpm db:push` or `pnpm db:migrate`)
3. You have an OmniRoute account/API key (or plan to use the local instance)

---

## Step 1: Get OmniRoute Credentials

### Option A: OmniRoute Cloud (recommended)

1. Sign up at [omniroute.example.com](https://omniroute.example.com)
2. Create an API key in Settings → API Keys
3. Note your:
   - **Base URL:** `https://omniroute.example.com/v1` (check your dashboard)
   - **API Key:** `sk-...` (save securely — not committed to git)

### Option B: Self-hosted OmniRoute (local)

If running OmniRoute locally (e.g., via Docker):

1. Clone/run the OmniRoute server
2. Note your:
   - **Base URL:** `http://localhost:11434/v1` (or whatever port)
   - **API Key:** `any-non-empty-string` (self-hosted often ignores auth)

---

## Step 2: Set Environment Variables

Add to your `.env` (or Coolify environment):

```bash
# OmniRoute credentials
OMNIROUTE_BASE_URL=https://omniroute.example.com/v1
OMNIROUTE_API_KEY=sk-your-key-here

# (Already set by default) Database
DATABASE_URL=postgres://echodo:echodo@localhost:5432/echodo

# (Already set by default) Redis (for orchestrator + worker)
REDIS_URL=redis://localhost:6379
```

**Important:** These env vars must be available to:
- The **web app** (`apps/web`) — for Settings UI
- The **worker** (`apps/worker`) — for inference
- The **orchestrator** (`apps/orchestrator`) — for dispatch

---

## Step 3: Register OmniRoute Providers in Echodo

### Via Settings UI (recommended)

1. Open `http://localhost:3001` (web app)
2. Go to **Settings → AI → Providers**
3. Click **Add Provider** and create one for each profile:

**Provider 1: omni-auto**

| Field | Value |
|-------|-------|
| Display Name | `OmniRoute Auto` |
| Alias | `omni-auto` |
| Provider Type | `openai_compatible` |
| Model ID | `auto` |
| Base URL | `${OMNIROUTE_BASE_URL}` (your actual URL) |
| API Key Env Var | `OMNIROUTE_API_KEY` |
| Metadata | `{"tier": "auto", "provider": "omniroute"}` |
| Enabled | ✅ |

**Provider 2: omni-coding**

| Field | Value |
|-------|-------|
| Display Name | `OmniRoute Coding` |
| Alias | `omni-coding` |
| Provider Type | `openai_compatible` |
| Model ID | `auto/coding` |
| Base URL | (same as above) |
| API Key Env Var | `OMNIROUTE_API_KEY` |
| Metadata | `{"tier": "auto", "provider": "omniroute"}` |
| Enabled | ✅ |

**Provider 3: omni-coding-pro**

| Field | Value |
|-------|-------|
| Display Name | `OmniRoute Coding Pro` |
| Alias | `omni-coding-pro` |
| Provider Type | `openai_compatible` |
| Model ID | `auto/coding-pro` |
| Base URL | (same) |
| API Key Env Var | `OMNIROUTE_API_KEY` |
| Metadata | `{"tier": "auto", "provider": "omniroute"}` |
| Enabled | ✅ |

**Provider 4: omni-reasoning**

| Field | Value |
|-------|-------|
| Display Name | `OmniRoute Reasoning` |
| Alias | `omni-reasoning` |
| Provider Type | `openai_compatible` |
| Model ID | `auto/reasoning` |
| Base URL | (same) |
| API Key Env Var | `OMNIROUTE_API_KEY` |
| Metadata | `{"tier": "auto", "provider": "omniroute"}` |
| Enabled | ✅ |

**Provider 5: omni-reasoning-pro**

| Field | Value |
|-------|-------|
| Display Name | `OmniRoute Reasoning Pro` |
| Alias | `omni-reasoning-pro` |
| Provider Type | `openai_compatible` |
| Model ID | `auto/reasoning-pro` |
| Base URL | (same) |
| API Key Env Var | `OMNIROUTE_API_KEY` |
| Metadata | `{"tier": "auto", "provider": "omniroute"}` |
| Enabled | ✅ |

### Via tRPC (for automation)

```typescript
// In a script or test
await trpc.ai.models.createProvider.mutate({
  alias: "omni-coding",
  displayName: "OmniRoute Coding",
  providerType: "openai_compatible",
  modelId: "auto/coding",
  baseUrl: "https://omniroute.example.com/v1",
  apiKeyEnvVar: "OMNIROUTE_API_KEY",
  metadata: { tier: "auto", provider: "omniroute" }
});
```

---

## Step 4: Map Roles to Profiles

### Via Settings UI

1. Go to **Settings → AI → Role Routing**
2. For each role, click **Edit** and select the provider:

| Role ID | Select Provider |
|---------|-----------------|
| `role-conductor` | `omni-reasoning` |
| `role-architect` | `omni-reasoning` |
| `role-ia-architect` | `omni-reasoning` |
| `role-implementer` | `omni-coding` |
| `role-reviewer` | `omni-coding-pro` |
| `role-ux-reviewer` | `omni-coding` |
| `role-doc-writer` | `omni-coding` |
| `role-design-system-auditor` | `omni-coding` |
| `role-a11y-auditor` | `omni-coding` |
| `gate-copilot` | `omni-reasoning` |
| `chat-default` | `omni-auto` |

### Via tRPC

```typescript
// Example: route implementer to omni-coding
await trpc.ai.models.upsertRoleRoute.mutate({
  roleId: "role-implementer",
  providerAliasId: "omni-coding-provider-uuid"  // get from listProviders
});
```

---

## Step 5: Test Connectivity

### Via Settings UI

1. Go to **Settings → AI → Providers**
2. Click the **Test** button next to `omni-coding`
3. You should see: ✅ `Success — latency: 320ms`

### Via MCP (if worker is running)

```typescript
// testProvider
{
  "tool": "testProvider",
  "arguments": { "alias": "omni-coding" }
}
```

### Via direct curl

```bash
curl ${OMNIROUTE_BASE_URL}/chat/completions \
  -H "Authorization: Bearer ${OMNIROUTE_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto/coding",
    "messages": [{"role": "user", "content": "Say hello in 3 words"}],
    "max_tokens": 20
  }'
```

Expected: A response like `{"choices": [{"message": {"content": "Hello world!"}}]}`

---

## Step 6: Test Model Resolution

### Verify a specific role resolves correctly

Use the debug endpoint (or MCP `resolveForRole`):

```typescript
resolveForRole({ roleId: "role-implementer" })
// Expected output:
// {
//   ok: true,
//   model: {
//     alias: "omni-coding",
//     providerType: "openai_compatible",
//     modelId: "auto/coding",
//     baseUrl: "https://omniroute.example.com/v1",
//     apiKey: "<redacted>",
//     tier: "auto"
//   },
//   source: "workspace",
//   resolvedAlias: "omni-coding"
// }
```

### Verify resolution order works

Create a test convoy with a model override:

```yaml
# .convoys/test-resolver.md
---
name: test-resolver
model_policy:
  roles:
    role-implementer: omni-reasoning-pro   # overrides workspace route
---
```

Then `resolveForRole` with `convoyContext`:

```typescript
resolveForRole({
  roleId: "role-implementer",
  convoyContext: { model_policy: { "role-implementer": "omni-reasoning-pro" } }
})
// → source: "convoy", alias: "omni-reasoning-pro"
```

---

## Step 7: Verify Worker Picks Up the Model

1. **Start the worker:**
   ```bash
   pnpm --filter @tasks/worker dev
   # or
   WORKER_DRY_RUN=0 ORCHESTRATOR_DRY_RUN=0 REDIS_URL=redis://localhost:6379 pnpm --filter @tasks/worker dev
   ```

2. **Start the orchestrator:**
   ```bash
   pnpm --filter @tasks/orchestrator dev
   ```

3. **Enqueue a test job** (via MCP `create_convoy` → transition to `ready`)

4. **Watch worker logs:**
   - Should see: `Dequeued job for role-implementer, resolved model: omni-coding (auto/coding)`
   - Should see: inference call to OmniRoute endpoint

5. **Verify model metadata in `agent_runs`:**
   ```sql
   SELECT model_alias, provider_type, model_id, model_tier
   FROM agent_runs
   ORDER BY created_at DESC
   LIMIT 5;
   ```
   Expected: `omni-coding | openai_compatible | auto/coding | auto`

---

## Troubleshooting

### "Provider alias 'omni-coding' is not registered in this workspace"

**Cause:** Provider not created yet, or `alias` doesn't match.

**Fix:**
1. Check Settings → AI → Providers for the alias name
2. Re-run `createProvider` with exact alias

### "Environment variable OMNIROUTE_API_KEY is not set"

**Cause:** Worker process doesn't have the env var loaded.

**Fix:**
1. Verify `OMNIROUTE_API_KEY` is in `.env` at repo root
2. Restart the worker process (env vars aren't hot-reloaded)
3. Check: `echo $OMNIROUTE_API_KEY` in the worker terminal

### "Connection refused" or "ECONNREFUSED"

**Cause:** OmniRoute server not reachable at `base_url`.

**Fix:**
1. Verify OmniRoute is running: `curl ${OMNIROUTE_BASE_URL}/models`
2. Check base URL ends with `/v1`
3. For local OmniRoute: check port (11434 vs 8000 vs other)

### "Unsupported provider type"

**Cause:** `providerType` value not in `providerTypeSchema`.

**Fix:** Must be one of: `anthropic`, `openai`, `google`, `openai_compatible`, `azure_openai`. For OmniRoute, use `openai_compatible`.

### Model returns empty or unexpected response

**Cause:** OmniRoute profile not available or rate-limited.

**Fix:**
1. Check OmniRoute dashboard for rate limits
2. Try a different profile (e.g., `auto` instead of `auto/coding`)
3. Check OmniRoute status page

### Resolution returns wrong source

**Cause:** Convoy or brief override taking precedence unexpectedly.

**Fix:** Remember the priority order:
1. Brief `recommended_model` (highest)
2. Convoy `model_policy.roles[roleId]`
3. Workspace `role_model_routes` row
4. Pipeline defaults (lowest)

If you want workspace route to win, remove `model_policy` from the convoy.

---

## Security Notes

- **API key never in DB:** `api_key_env_var` stores the *name* of the env var, not the value
- **API key never sent to browser:** Settings UI shows env var name only
- **Worker reads env var at dispatch time:** Not stored in job payload
- **Test endpoint runs server-side:** Rate-limited per workspace

---

## Related Docs

- **Role mapping:** `docs/omniroute-role-mapping.md` (profile → role matrix, resolution flow)
- **ADR:** `docs/adr-model-routing.md` (architecture decision record)
- **Skill:** `.agents/skills/model-routing/SKILL.md` (developer reference)
- **Provider schema:** `packages/database/src/schema/model_providers.ts`
- **Route schema:** `packages/database/src/schema/role_model_routes.ts`