491 lines
27 KiB
Markdown
491 lines
27 KiB
Markdown
|
# Autonomous Convoys with OpenHands & OmniRoute
|
|||
|
|
|
|||
|
|
**Status:** Draft (planning phase)
|
|||
|
|
**Date:** 2026-09-07
|
|||
|
|
**Goal:** Enable OpenHands + omniroute to autonomously execute Echodo convoys by mapping roles to model profiles and integrating Agent Skills format.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Executive Summary
|
|||
|
|
|
|||
|
|
Echodo's convoy system (defined in `.convoys/*.md` with role-based `model_policy`) currently requires Cursor to execute per-role briefs. This plan replaces Cursor with OpenHands as the runtime harness, using omniroute's model profiles (`auto`, `auto/coding`, `auto/reasoning`, `auto/coding-pro`, `auto/reasoning-pro`) to route each role to the appropriate model.
|
|||
|
|
|
|||
|
|
The main insight: **Your convoy structure already works. We're just swapping the agent harness and making skills discoverable.**
|
|||
|
|
|
|||
|
|
**Key changes:**
|
|||
|
|
1. Convert role-prompt rules to `.agents/skills/` in SKILL.md format (Agent Skills spec)
|
|||
|
|
2. Map omniroute profiles → roles (conductor/architect/implementer/reviewer)
|
|||
|
|
3. Wire OpenHands session bootstrap to inherit `agent_prompt` + skills + model policy
|
|||
|
|
4. Document the flow so agents executing convoys follow consistent patterns
|
|||
|
|
|
|||
|
|
**Outcome:** By end of v0.1, a human can:
|
|||
|
|
- Define a convoy in `.convoys/<slug>.md` with `model_policy` roles mapped to omniroute profiles
|
|||
|
|
- Trigger the convoy from Mission Control
|
|||
|
|
- OpenHands picks up each brief, inherits the task context, loads the right model via omniroute, and executes
|
|||
|
|
- After success, worker logs role completion + model metadata back to `agent_runs` and `convoy_events`
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Architecture: The Convoy-to-OpenHands Flow
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
┌─── Human / Mission Control ──────────────────────────────────────┐
|
|||
|
|
│ "Start convoy: brief-1-implement-feature" │
|
|||
|
|
│ (.convoys/feature-x.md, brief-1-*.md exists, ready to go) │
|
|||
|
|
└────────────────────────┬────────────────────────────────────────┘
|
|||
|
|
│
|
|||
|
|
Orchestrator
|
|||
|
|
(apps/orchestrator)
|
|||
|
|
│
|
|||
|
|
┌────────────────┼────────────────┐
|
|||
|
|
│ │ │
|
|||
|
|
▼ ▼ ▼
|
|||
|
|
┌─────────────────────────────────────────────────────────────┐
|
|||
|
|
│ Redis Job Queue (echodo:orchestrator:jobs) │
|
|||
|
|
│ { roleId, harnessAlias, briefPath, workspaceId, ...} │
|
|||
|
|
└──────────────────────┬──────────────────────────────────────┘
|
|||
|
|
│
|
|||
|
|
Worker dequeues
|
|||
|
|
(apps/worker)
|
|||
|
|
│
|
|||
|
|
┌─────────────────┼──────────────────┐
|
|||
|
|
│ │ │
|
|||
|
|
▼ ▼ ▼
|
|||
|
|
┌────────────┐ ┌──────────────┐ ┌──────────────┐
|
|||
|
|
│ Resolve │ │ Resolve │ │ Load │
|
|||
|
|
│ model via │ │ harness + │ │ session │
|
|||
|
|
│ omniroute │ │ worktree │ │ bootstrap: │
|
|||
|
|
│ profile │ │ │ │ • agent_prompt
|
|||
|
|
│ (coding, │ │ (code/doc/ │ │ • skills from
|
|||
|
|
│ reasoning) │ │ research │ │ .agents/ │
|
|||
|
|
│ │ │ harness) │ │ • role rules │
|
|||
|
|
└────────────┘ └──────────────┘ └──────────────┘
|
|||
|
|
│ │ │
|
|||
|
|
└────────────────┼──────────────────┘
|
|||
|
|
│
|
|||
|
|
OpenHands Session Start
|
|||
|
|
(apps/web calls MCP claim_task)
|
|||
|
|
│
|
|||
|
|
┌────────────────┼────────────────┐
|
|||
|
|
│ │ │
|
|||
|
|
▼ ▼ ▼
|
|||
|
|
┌──────────────────────────────────────────────────────────────┐
|
|||
|
|
│ Session context ready: │
|
|||
|
|
│ • Model resolved (e.g. qwen-coder or claude-3-5-sonnet) │
|
|||
|
|
│ • Task/brief context loaded (agent_prompt, file paths) │
|
|||
|
|
│ • Skills available (.agents/skills/*.md files) │
|
|||
|
|
│ • Tool allowlist set (code vs doc harness) │
|
|||
|
|
│ • MCP tools: read_file, write_file, grep, run_command │
|
|||
|
|
│ │
|
|||
|
|
│ Agent now thinks: "I am role-implementer on brief-1. │
|
|||
|
|
│ My success metric is: [from convoy frontmatter]. │
|
|||
|
|
│ I should follow: [agent_prompt from task]. │
|
|||
|
|
│ I can use these skills: [loaded from .agents/skills/]." │
|
|||
|
|
└────────────────────┬─────────────────────────────────────────┘
|
|||
|
|
│
|
|||
|
|
Agent tool loop
|
|||
|
|
(read_file, write_file,
|
|||
|
|
grep, run_command)
|
|||
|
|
│
|
|||
|
|
┌───────────────┼───────────────┐
|
|||
|
|
│ │ │
|
|||
|
|
Success Failure Stall
|
|||
|
|
│ │ │
|
|||
|
|
▼ ▼ ▼
|
|||
|
|
┌──────────┐ ┌──────────┐ ┌──────────────┐
|
|||
|
|
│ Log │ │ Log │ │ Orchestrator │
|
|||
|
|
│ complete │ │ failure │ │ detects │
|
|||
|
|
│ + model │ │ + │ │ stall, │
|
|||
|
|
│ metadata │ │ tokens │ │ resumes from │
|
|||
|
|
│ │ │ │ │ checkpoint │
|
|||
|
|
└────┬─────┘ └────┬─────┘ └──────┬───────┘
|
|||
|
|
│ │ │
|
|||
|
|
└─────────────┼───────────────┘
|
|||
|
|
│
|
|||
|
|
MCP complete_task + convoy_events.role_complete
|
|||
|
|
│
|
|||
|
|
Orchestrator ticks
|
|||
|
|
(determines next role)
|
|||
|
|
│
|
|||
|
|
┌─────────────┼──────────────┐
|
|||
|
|
│ │ │
|
|||
|
|
Next Brief Human Gate End Convoy
|
|||
|
|
│ │ │
|
|||
|
|
▼ ▼ ▼
|
|||
|
|
┌──────────┐ ┌─────────┐ ┌──────────┐
|
|||
|
|
│ Back to │ │ Mission │ │ Released │
|
|||
|
|
│ Redis │ │ Control │ │ or │
|
|||
|
|
│ (loop) │ │ approve │ │ merged │
|
|||
|
|
└──────────┘ └─────────┘ └──────────┘
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Four Concrete Work Phases
|
|||
|
|
|
|||
|
|
### Phase 1: Agent Skills Format Conversion (M estimate)
|
|||
|
|
|
|||
|
|
**Goal:** Make repo-integrated skills discoverable by OpenHands via SKILL.md format in `.agents/skills/`.
|
|||
|
|
|
|||
|
|
**Why:** Your current `.cursor/rules/*.mdc` files are Cursor-specific. OpenHands reads `.agents/skills/*/SKILL.md` (per Agent Skills spec). Converting moves you from Cursor-only to Cursor + OpenHands + any harness that supports Agent Skills.
|
|||
|
|
|
|||
|
|
**Tasks:**
|
|||
|
|
|
|||
|
|
1. **Create `.agents/skills/backlog-sync/SKILL.md`**
|
|||
|
|
- **When to invoke:** When working with `plans/`, `markdown_backlog_items`, or the Echodo markdown import
|
|||
|
|
- **What it teaches:** How backlog frontmatter maps to DB, how `pnpm --filter @tasks/database watch:markdown-backlog` works, the contract between `.md` and `markdown_backlog_items`
|
|||
|
|
- **Reference:** `.cursor/rules/markdown-backlog.mdc` + `packages/database/src/markdown-backlog/*.ts` + `docs/Glossary.md`
|
|||
|
|
- **Output format:** `SKILL.md` with frontmatter (name, description) + sections (When to Activate, Core Rules, Common Patterns, Examples)
|
|||
|
|
|
|||
|
|
2. **Create `.agents/skills/convoy-lifecycle/SKILL.md`**
|
|||
|
|
- **When to invoke:** When working with convoys, briefs, transitions, gates, or the state machine
|
|||
|
|
- **What it teaches:** Status enum (draft→ready→in_progress→pr_review→merged), gate actors (human-gate-1/2/3), MCP tools (create_convoy, transition_convoy_status, log_convoy_event), checkpoint/resume pattern
|
|||
|
|
- **Reference:** `docs/convoy-lifecycle.md` + `packages/shared/src/convoy-lifecycle/*.ts` + `apps/mcp-server/src/tools/*convoy*`
|
|||
|
|
- **Output format:** Same as backlog-sync
|
|||
|
|
|
|||
|
|
3. **Create `.agents/skills/model-routing/SKILL.md`**
|
|||
|
|
- **When to invoke:** When routing models, configuring aliases, or understanding how the orchestrator selects models
|
|||
|
|
- **What it teaches:** The resolution order (brief override → convoy policy → workspace routes → pipeline defaults), omniroute profile mapping, how `resolveModel(workspace, roleId)` works, how model metadata flows to `agent_runs` and `convoy_events`
|
|||
|
|
- **Reference:** `docs/adr-model-routing.md` + `packages/shared/src/orchestrator/*.ts` + this doc's **Phase 2** section
|
|||
|
|
- **Output format:** Same as above
|
|||
|
|
|
|||
|
|
4. **Create `.agents/skills/code-review-two-axis/SKILL.md`** (adapted from mattpocock/skills)
|
|||
|
|
- **When to invoke:** When `role-reviewer` runs or a human asks for a PR review in Mission Control
|
|||
|
|
- **What it teaches:** Two-axis review (Standards vs Spec), running both as parallel sub-agents, how to aggregate findings, the code-smell baseline
|
|||
|
|
- **Reference:** https://github.com/mattpocock/skills/blob/main/skills/engineering/code-review/SKILL.md (with Echodo-specific tweaks for Standards against `AGENTS.md` + repo rules)
|
|||
|
|
- **Output format:** Same as above
|
|||
|
|
|
|||
|
|
5. **Create `.agents/skills/tdd-red-green/SKILL.md`** (adapted from mattpocock/skills)
|
|||
|
|
- **When to invoke:** When implementing features or fixing bugs in a test-first flow
|
|||
|
|
- **What it teaches:** Red-green-refactor loop, seams, anti-patterns, the tracer-bullet concept
|
|||
|
|
- **Reference:** https://github.com/mattpocock/skills/blob/main/skills/engineering/tdd/SKILL.md
|
|||
|
|
- **Output format:** Same as above
|
|||
|
|
|
|||
|
|
**Acceptance criteria:**
|
|||
|
|
- [ ] Each SKILL.md has frontmatter (name, description) + at least 3 sections
|
|||
|
|
- [ ] `skil --validate .agents/skills/*/SKILL.md` passes (or equiv. local validation)
|
|||
|
|
- [ ] Skills are referenced in `AGENTS.md` under the "Skills available" section
|
|||
|
|
- [ ] Agent can invoke them via `/skill` command or model-invocation within OpenHands
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Phase 2: OmniRoute Profile Mapping & Model Policy Doc (S estimate)
|
|||
|
|
|
|||
|
|
**Goal:** Document how omniroute profiles map to roles and how the orchestrator uses `model_policy` to route.
|
|||
|
|
|
|||
|
|
**Why:** Your `model_policy.roles` in convoy frontmatter lists role names + model aliases. Those aliases need to resolve to omniroute **profiles** (e.g., `auto/coding`, `auto/reasoning`) so the worker can ask omniroute "give me the model for `claude-planning`" and get back the current endpoint.
|
|||
|
|
|
|||
|
|
**Tasks:**
|
|||
|
|
|
|||
|
|
1. **Create `docs/omniroute-role-mapping.md`**
|
|||
|
|
- **Section 1: Profile inventory**
|
|||
|
|
```
|
|||
|
|
- auto (default, fastest)
|
|||
|
|
- auto/coding (code-specialized)
|
|||
|
|
- auto/coding-pro (complex, high-quality)
|
|||
|
|
- auto/reasoning (reasoning/planning)
|
|||
|
|
- auto/reasoning-pro (complex reasoning, high-cost)
|
|||
|
|
```
|
|||
|
|
- **Section 2: Role → Profile mapping (recommended)**
|
|||
|
|
```
|
|||
|
|
role-conductor → auto/reasoning (kickoff, classify, skip flags)
|
|||
|
|
role-architect → auto/reasoning-pro (design, brief decomposition)
|
|||
|
|
role-implementer → auto/coding (default) or auto/coding-pro (escalation)
|
|||
|
|
role-reviewer → auto/coding-pro (two-axis review)
|
|||
|
|
role-doc-writer → auto/coding (docs are simpler than code)
|
|||
|
|
gate-copilot → auto/reasoning (summary for human gate)
|
|||
|
|
```
|
|||
|
|
- **Section 3: How to configure in a convoy**
|
|||
|
|
```yaml
|
|||
|
|
model_policy:
|
|||
|
|
roles:
|
|||
|
|
role-conductor: claude-planning # resolves to auto/reasoning profile
|
|||
|
|
role-implementer: qwen-coder # resolves to auto/coding profile
|
|||
|
|
escalate_to: claude-planning # if implementer fails 2x, re-dispatch with this
|
|||
|
|
```
|
|||
|
|
- **Section 4: How the orchestrator uses this**
|
|||
|
|
- Reads `model_policy.roles[roleId]` from the convoy
|
|||
|
|
- Calls `resolveModel(workspace, roleId)` in `packages/shared/src/orchestrator/*.ts`
|
|||
|
|
- Resolution order: brief override → convoy policy → workspace routes → pipeline defaults
|
|||
|
|
- Result: `{ alias, provider_type, model_id, base_url, api_key_env_var }`
|
|||
|
|
- Worker receives model spec in the job payload
|
|||
|
|
- **Section 5: Escalation & retry logic**
|
|||
|
|
- If implementer brief fails, orchestrator can re-dispatch with `escalate_to: role-architect` (higher-tier model)
|
|||
|
|
- Log tier change on `agent_runs`
|
|||
|
|
|
|||
|
|
2. **Create `docs/omniroute-setup.md`** (operational guide)
|
|||
|
|
- **How to register omniroute in the workspace** (via Settings → AI → Providers or tRPC `model_providers.create`)
|
|||
|
|
- **How to set up the 5 profiles** (assuming omniroute cloud endpoint; or local Ollama + MLX)
|
|||
|
|
- **How to test** (`testProvider` MCP call, test a single model)
|
|||
|
|
- **How to debug** (logs in `agent_runs`, `worker_checkpoints` metadata)
|
|||
|
|
|
|||
|
|
3. **Update `docs/adr-model-routing.md`** (add OmniRoute section)
|
|||
|
|
- Keep the existing conceptual model
|
|||
|
|
- Add subsection: "OmniRoute Cloud Integration (v0.1)"
|
|||
|
|
- Explain how `resolveModel` calls omniroute's API to get the actual endpoint + auth
|
|||
|
|
|
|||
|
|
**Acceptance criteria:**
|
|||
|
|
- [ ] `docs/omniroute-role-mapping.md` covers all 6 roles (conductor, architect, implementer, reviewer, doc-writer, gate-copilot)
|
|||
|
|
- [ ] Each role has a recommended default profile + an override example
|
|||
|
|
- [ ] `docs/omniroute-setup.md` has step-by-step instructions (create alias, test, configure workspace route)
|
|||
|
|
- [ ] ADR updated with omniroute context
|
|||
|
|
- [ ] Convoy examples in `.convoys/*.md` use the recommended aliases (no hardcoded model names)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Phase 3: OpenHands Session Bootstrap & Worker Integration (L estimate)
|
|||
|
|
|
|||
|
|
**Goal:** Wire up OpenHands session to inherit task context, load skills, and respect model policy.
|
|||
|
|
|
|||
|
|
**Why:** When the worker dequeues a job and spawns an OpenHands session, the agent needs to know:
|
|||
|
|
- What task it's working on (brief path, success metric, acceptance criteria)
|
|||
|
|
- What role it is (role-implementer, role-reviewer, etc.)
|
|||
|
|
- Which model to use (resolved from omniroute)
|
|||
|
|
- Which skills are available (from `.agents/skills/`)
|
|||
|
|
- Which tools are allowed (code harness: `pnpm lint`, `run_command`; doc harness: read/write/grep only)
|
|||
|
|
|
|||
|
|
**Tasks:**
|
|||
|
|
|
|||
|
|
1. **Extend `apps/worker/src/session/minimal-session.ts`** (or create `openhands-session.ts`)
|
|||
|
|
- **Input:** `{ jobSpec, resolvedModel, harnessAlias, briefPath }`
|
|||
|
|
- **On session init:**
|
|||
|
|
- Load `brief-*.md` from `.convoys/<slug>/` (task context)
|
|||
|
|
- Extract `agent_prompt` (from task → epic → plan inheritance, per AGENTS.md)
|
|||
|
|
- Inherit `title`, `success_metric`, acceptance criteria from frontmatter
|
|||
|
|
- Construct a **session bootstrap** (initial system + first user message)
|
|||
|
|
- **Session bootstrap structure:**
|
|||
|
|
```
|
|||
|
|
System message (shared context):
|
|||
|
|
- You are an AI assistant helping implement a software task
|
|||
|
|
- Your task: [title from brief frontmatter]
|
|||
|
|
- Success means: [success_metric]
|
|||
|
|
- You are playing the role of: [roleId]
|
|||
|
|
- Acceptance criteria: [from brief or inherited]
|
|||
|
|
- Constraints:
|
|||
|
|
- Use the [tool allowlist] only
|
|||
|
|
- If you make a file change, it will be pushed to a branch
|
|||
|
|
- After you succeed, claim_task + complete_task will be called
|
|||
|
|
- Principles: [karpathy-guidelines from AGENTS.md]
|
|||
|
|
|
|||
|
|
First user message:
|
|||
|
|
- [inherited agent_prompt from task frontmatter]
|
|||
|
|
- [brief body content — the actual work description]
|
|||
|
|
- "Here are the skills you have available: [list from .agents/skills/]"
|
|||
|
|
- "Your model is: [resolved model name from omniroute]"
|
|||
|
|
```
|
|||
|
|
- **On session end:**
|
|||
|
|
- Collect metrics: `{ model, tokens, duration, outcome }`
|
|||
|
|
- Return to worker for logging
|
|||
|
|
|
|||
|
|
2. **Wire OpenHands integration into Worker job handler** (`apps/worker/src/handle-job.ts`)
|
|||
|
|
- **Current flow:** `MinimalSession` → tool loop → checkpoint
|
|||
|
|
- **New flow:**
|
|||
|
|
```
|
|||
|
|
1. Dequeue job from Redis
|
|||
|
|
2. resolveHarnessForJob(job.roleId) → harness config
|
|||
|
|
3. resolveModel(workspace, job.roleId) → model spec
|
|||
|
|
4. Create OpenHands session with bootstrap (brief context + skills)
|
|||
|
|
5. Run tool loop (agent interacts with codebase)
|
|||
|
|
6. On success: push branch, call forge PR endpoint
|
|||
|
|
7. Log complete_task + convoy_events.role_complete with model metadata
|
|||
|
|
8. Return to step 1
|
|||
|
|
```
|
|||
|
|
- **Config:** Add env var `WORKER_SESSION_TYPE=openai-client | minimal | openhands` (default: minimal for backward compat, set to `openhands` to use new flow)
|
|||
|
|
|
|||
|
|
3. **Create `.agents/` integration test** (`apps/worker/src/openhands-integration.test.ts`)
|
|||
|
|
- **Test:** Spawn a dry-run OpenHands session with a toy brief, verify:
|
|||
|
|
- [ ] Brief context loads correctly
|
|||
|
|
- [ ] Skills are discoverable
|
|||
|
|
- [ ] agent_prompt is inherited
|
|||
|
|
- [ ] Tool allowlist is respected
|
|||
|
|
- [ ] Session logs metrics on exit
|
|||
|
|
|
|||
|
|
**Acceptance criteria:**
|
|||
|
|
- [ ] `WORKER_SESSION_TYPE=openhands` env var enables the flow
|
|||
|
|
- [ ] Integration test passes (dry-run with mock brief)
|
|||
|
|
- [ ] Live E2E test (real brief, real repo) succeeds with model logging
|
|||
|
|
- [ ] `agent_runs` row includes `{ model_alias, model_tier, tokens_in, tokens_out }`
|
|||
|
|
- [ ] `convoy_events` has `role_complete` entry with `metadata.harnessAlias`
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Phase 4: End-to-End E2E Test & Dogfood (L estimate)
|
|||
|
|
|
|||
|
|
**Goal:** Prove one convoy executes autonomously from start to finish using OpenHands + omniroute.
|
|||
|
|
|
|||
|
|
**Why:** This is the acceptance test for the whole milestone. A human should be able to:
|
|||
|
|
1. Define a convoy with roles mapped to omniroute profiles
|
|||
|
|
2. Click "Start" in Mission Control
|
|||
|
|
3. Watch OpenHands agents pick up and complete briefs without Cursor
|
|||
|
|
4. See model metadata logged correctly
|
|||
|
|
|
|||
|
|
**Tasks:**
|
|||
|
|
|
|||
|
|
1. **Create a test convoy** (`.convoys/test-e2e-autonomous.md`)
|
|||
|
|
- Classification: `server-only` (simple, no IA/UX/design)
|
|||
|
|
- 2–3 briefs (implementer + reviewer)
|
|||
|
|
- Roles: `role-conductor` (kickoff only), `role-implementer` (fix a small bug or add a minor feature), `role-reviewer` (review the PR)
|
|||
|
|
- Model policy: use omniroute profiles (e.g., `role-implementer: auto/coding`, `role-reviewer: auto/coding-pro`)
|
|||
|
|
- Success metric: "Bug is fixed, PR is drafted, review is complete"
|
|||
|
|
|
|||
|
|
2. **Run the full loop** (Manual walkthrough)
|
|||
|
|
- [ ] Human creates convoy via MCP `create_convoy` or Mission Control UI
|
|||
|
|
- [ ] Orchestrator picks up `role-conductor` (dry-run or light task)
|
|||
|
|
- [ ] Orchestrator enqueues `role-implementer` to Redis
|
|||
|
|
- [ ] Worker dequeues, spawns OpenHands session
|
|||
|
|
- [ ] OpenHands loads brief, runs tool loop, fixes the bug, commits
|
|||
|
|
- [ ] Worker logs `complete_task` + `role_complete`
|
|||
|
|
- [ ] Orchestrator transitions convoy to `pr_review` (or next gate)
|
|||
|
|
- [ ] Orchestrator enqueues `role-reviewer` brief
|
|||
|
|
- [ ] Worker dequeues, spawns OpenHands session, runs two-axis review
|
|||
|
|
- [ ] Worker logs `complete_task` + `role_complete` with review metadata
|
|||
|
|
- [ ] Convoy reaches `merged` or `published` gate
|
|||
|
|
- [ ] Check `agent_runs`: model aliases logged for both implementer + reviewer
|
|||
|
|
- [ ] Check `convoy_events`: role_complete entries present with metadata
|
|||
|
|
|
|||
|
|
3. **Dogfood the pipeline** (Optional: use this convoy to close a real task)
|
|||
|
|
- Pick a backlog task that fits the convoy (small feature or bug)
|
|||
|
|
- Map task to a convoy brief
|
|||
|
|
- Run the E2E flow on real code
|
|||
|
|
- Validate that the fix/feature is correct and the PR is mergeable
|
|||
|
|
|
|||
|
|
**Acceptance criteria:**
|
|||
|
|
- [ ] Convoy completes all briefs without Cursor
|
|||
|
|
- [ ] `agent_runs` has entries for implementer + reviewer with model aliases
|
|||
|
|
- [ ] `convoy_events` has `role_complete` entries
|
|||
|
|
- [ ] PR is draft or merged (depending on gates)
|
|||
|
|
- [ ] No manual intervention after "Start Convoy"
|
|||
|
|
- [ ] Logs are clean (no auth errors, model resolution errors)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Dependencies & Ordering
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
Phase 1 (Skills) ──┐
|
|||
|
|
Phase 2 (Routing) ─┼──→ Phase 3 (Worker) ──→ Phase 4 (E2E)
|
|||
|
|
Phase 3 unblocked │ (can start once 1 + 2 done)
|
|||
|
|
as soon as 1 + 2 │
|
|||
|
|
complete
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**Critical path:**
|
|||
|
|
- Phase 1 + 2 are sequential (skills first, then routing docs)
|
|||
|
|
- Phase 3 can start once Phase 1 + 2 are done
|
|||
|
|
- Phase 4 requires Phase 3 complete
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Agent Instructions for Each Phase
|
|||
|
|
|
|||
|
|
### For Agents Working on Phase 1 (Skills)
|
|||
|
|
|
|||
|
|
1. **Before writing:** Read the existing `.cursor/rules/*.mdc` files in this repo + the referenced SKILL.md files from mattpocock/skills, anthropics/skills
|
|||
|
|
2. **Structure:** Follow the Agent Skills spec (frontmatter + markdown sections)
|
|||
|
|
3. **Validation:** Each skill should be invokable by model (e.g., `/skill backlog-sync` or implied when needed) and have 3–4 substantive sections
|
|||
|
|
4. **Test:** Run locally with OpenHands or Claude Code to verify skills load
|
|||
|
|
5. **Acceptance:** PR reviewer checks that skills have frontmatter + clear When-to-Activate section
|
|||
|
|
|
|||
|
|
### For Agents Working on Phase 2 (Routing)
|
|||
|
|
|
|||
|
|
1. **Before writing:** Read `docs/adr-model-routing.md` fully + understand how `model_policy` works in `.convoys/*.md`
|
|||
|
|
2. **Assumption:** OmniRoute cloud is the target (not local-only), and omniroute has 5 profiles (auto, auto/coding, auto/coding-pro, auto/reasoning, auto/reasoning-pro)
|
|||
|
|
3. **Output:** Two markdown files (mapping + setup), both human-readable and copy-pasteable examples
|
|||
|
|
4. **Validation:** Walk through the flow: convoy sets `model_policy.roles`, orchestrator calls `resolveModel`, worker receives model spec
|
|||
|
|
5. **Test:** No automated test needed; human walkthrough in Phase 4
|
|||
|
|
|
|||
|
|
### For Agents Working on Phase 3 (Worker Integration)
|
|||
|
|
|
|||
|
|
1. **Before writing:** Read `apps/worker/src/session/minimal-session.ts` + `apps/worker/src/handle-job.ts` + understand the worker dequeue loop
|
|||
|
|
2. **Pattern:** Don't replace MinimalSession; extend it or wrap it in an adapter (prefer wrapping for backward compat)
|
|||
|
|
3. **Assumption:** OpenHands agent-server is running and reachable (via env var `AGENT_SERVER_URL`)
|
|||
|
|
4. **Validation:**
|
|||
|
|
- Type-check passes (`pnpm type-check`)
|
|||
|
|
- New env var `WORKER_SESSION_TYPE` is optional (defaults to `minimal`)
|
|||
|
|
- Integration test dry-runs without errors
|
|||
|
|
5. **Test:** Phase 4 (E2E) will exercise this
|
|||
|
|
|
|||
|
|
### For Agents Working on Phase 4 (E2E)
|
|||
|
|
|
|||
|
|
1. **Before starting:** Phase 1 + 2 + 3 must be done
|
|||
|
|
2. **Pre-flight:**
|
|||
|
|
- [ ] `.agents/skills/` SKILL.md files exist + load correctly
|
|||
|
|
- [ ] `docs/omniroute-role-mapping.md` + `docs/omniroute-setup.md` exist
|
|||
|
|
- [ ] `WORKER_SESSION_TYPE=openhands` env var works
|
|||
|
|
- [ ] Orchestrator + worker are running (or can dry-run)
|
|||
|
|
3. **Test convoy:** Create `.convoys/test-e2e-autonomous.md` with 2 briefs (implementer + reviewer)
|
|||
|
|
4. **Manual walkthrough:** Follow the flow in "Phase 4: Tasks" section, step by step
|
|||
|
|
5. **Validation:** Check `agent_runs`, `convoy_events` tables for logs; verify PR created + reviewed
|
|||
|
|
6. **Report:** Summarize the run (timeline, model aliases used, any errors, fixes applied)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Success Criteria (Overall)
|
|||
|
|
|
|||
|
|
By end of Phase 4:
|
|||
|
|
|
|||
|
|
- [ ] One full convoy (2+ briefs) executes autonomously via OpenHands without Cursor
|
|||
|
|
- [ ] Each role uses the model from omniroute (via model_policy)
|
|||
|
|
- [ ] `agent_runs` + `convoy_events` log model metadata for observability
|
|||
|
|
- [ ] No secrets or API keys leaked in logs
|
|||
|
|
- [ ] Docs cover setup + usage (omniroute-role-mapping.md, omniroute-setup.md)
|
|||
|
|
- [ ] Skills are discoverable + usable by agents (.agents/skills/*/SKILL.md)
|
|||
|
|
- [ ] Worker integration test passes
|
|||
|
|
- [ ] Code is linted + type-checked
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Rollout & Scaling
|
|||
|
|
|
|||
|
|
Once Phase 4 succeeds:
|
|||
|
|
|
|||
|
|
1. **Publish to v0.1** — Mark the orchestrator + worker as production-ready for autonomous convoys
|
|||
|
|
2. **Scale briefs** — Extend to 5–6 briefs per convoy (architect → implementer → reviewer → ops-copilot → etc.)
|
|||
|
|
3. **Scale roles** — Add `role-docs-writer`, `role-security-auditor`, `role-performance-reviewer` as needed
|
|||
|
|
4. **Scale models** — Extend omniroute integration to support Claude/OpenAI/Gemini APIs side-by-side
|
|||
|
|
5. **Migrate Cursor convoys** — For convoys that have historically run in Cursor, run them via OpenHands instead (backward compat: human-initiated, not automated)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Risk Mitigations
|
|||
|
|
|
|||
|
|
| Risk | Mitigation |
|
|||
|
|
|------|-----------|
|
|||
|
|
| Model tier mismatches (expensive model for simple task) | OmniRoute profiles + explicit role mapping in docs |
|
|||
|
|
| Agent hallucinations in brief context | Agent_prompt inheritance from task → epic → plan keeps context tight; karpathy-guidelines skill |
|
|||
|
|
| Network issues between worker + OpenHands | Checkpoint/resume logic already in place; retry logic in orchestrator |
|
|||
|
|
| Secrets leaked in logs | Review `agent_runs.metadata` + `worker_checkpoints.tool_history` before logging; never log full API responses |
|
|||
|
|
| Convoy state desync (DB vs `.convoys/*.md`) | `reconcile_from_files` MCP tool keeps them in sync; convoy_events is the source of truth |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## References
|
|||
|
|
|
|||
|
|
- **Agent Skills Spec:** https://agentskills.io/specification
|
|||
|
|
- **Mattpocock Skills (code-review, tdd):** https://github.com/mattpocock/skills
|
|||
|
|
- **Superpowers (bootstrap pattern):** https://github.com/obra/superpowers
|
|||
|
|
- **Karpathy Guidelines (4 behavioral rules):** https://github.com/multica-ai/andrej-karpathy-skills
|
|||
|
|
- **Echodo Convoys:** `.convoys/README.md`
|
|||
|
|
- **Model Routing ADR:** `docs/adr-model-routing.md`
|
|||
|
|
- **Worker Docs:** `docs/worker.md`
|
|||
|
|
- **Orchestrator Docs:** `docs/orchestrator.md`
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Questions for Review
|
|||
|
|
|
|||
|
|
Before agents start Phase 1, answer these:
|
|||
|
|
|
|||
|
|
1. **OmniRoute target:** Are we assuming omniroute cloud endpoint, or should we support local Ollama + MLX as a fallback?
|
|||
|
|
2. **Model fallback:** If omniroute is down, should the worker use a hardcoded fallback model, or fail closed?
|
|||
|
|
3. **Role coverage:** Are 6 roles (conductor, architect, implementer, reviewer, doc-writer, gate-copilot) enough, or do we need more?
|
|||
|
|
4. **Cost tracking:** Should Phase 3 also wire up `cost_budget` checks from `model_providers`, or leave that for a follow-on?
|
|||
|
|
5. **Backward compat:** Should Phase 3 preserve the MinimalSession adapter, or fully replace it with OpenHands?
|
|||
|
|
6. **v0.1 scope:** Phase 4 tests one convoy. Should we aim for 2+ convoys running in parallel by end of Phase 4, or save that for post-v0.1?
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**Document Version:** 1.0
|
|||
|
|
**Last Updated:** 2026-09-07
|
|||
|
|
**Next Review:** After Phase 1 completion
|