Submatter/plans/Plan-automation-foundation/Plan-automation-foundation.md

324 lines
17 KiB
Markdown
Raw Normal View History

---
kind: plan
slug: automation-foundation
title: Automation foundation — mission control, model routing, orchestrator path
status: ready
priority: P1
tenant_id: global
cursor_plan_id: null
updated_at: "2026-08-12"
agent_prompt: |
You are implementing Echodo's automation foundation. Read docs/adr-model-routing.md
before touching model routing code. This repo uses pnpm, Drizzle, tRPC, zod in
packages/shared, and multitenancy via workspace_id on every tenant table.
Dogfooding (required for every task in this plan):
1. At session start: call MCP claim_task with this task's backlogItemId (import
plans/ if the row is missing).
2. At session end: call MCP complete_task with outcome, tokensTotal, and
modelAlias/modelTier when you used a routed model.
3. Stay within the task's acceptance criteria; do not widen scope.
4. Run pnpm lint && pnpm type-check && pnpm test before declaring done.
Stack: Next.js App Router, packages/database for schema, apps/web for UI.
Never store API keys in committed files or DB — only api_key_env_var names.
---
# Plan overview
Echodo / **Submatter** is evolving from a multitenant task app with MCP coordination into an **owner cockpit** for agent-built personal projects: fleet home, roadmap + Start, mission-control gates, first-party worker (leave Cursor for convoy builds), forge PRs, completion summaries/demos, and ops copilot.
Epics **1–6 shipped**. Epic **7 in progress** (tool loop + checkpoints in code; draft PR + live E2E verify still open). Epics **8–9** (webhooks, fleet) and **10–12** (plans UI, completion artifacts, ops copilot) carry the full owner vision. Autensa/Builderz-style ideas are benchmarks only — no OpenClaw dependency.
## Owner vision phases
| Phase | Goal | Epics |
|-------|------|-------|
| **A** Engine without Cursor | Go → worker → PR → gates | 7 → 8 (+ draft PR, live E2E) |
| **B** Fleet attention home | All projects, needs-you, activity | 9 |
| **C** Roadmap + Start | Plans tree, Start → convoy | 10 |
| **D** Owner review | Summaries, screenshots/demos | 11 |
| **E** Ops chat | Status / triage / draft work | 12 |
### A1 status check (2026-08-12)
| Claim | Reality in repo |
|-------|-----------------|
| Worker tool loop exists | Yes — `apps/worker` dequeue, session, worktree, claim/complete |
| Checkpoints | Yes — task done, `worker_checkpoints` path documented |
| Live Go → unattended job on your deploy | **Operator-verify** — [Task-worker-live-e2e-dogfood](./Epic-apps-worker-v1/Task-worker-live-e2e-dogfood.md) |
| Draft PR from worker | **Missing** — no forge PR call in worker; [Task-worker-draft-pr](./Epic-apps-worker-v1/Task-worker-draft-pr.md) |
| Minimal-tool-loop marked `done` | Code landed; several AC checkboxes in the task file still unchecked — treat live E2E + draft PR as the real close-out |
**If you “finished A1”** meaning services run and a job completed: mark criteria on the live E2E task and keep going. **If you only shipped code:** run the live E2E checklist before calling Phase A done.
## Dogfooding the agent pipeline
**Yes — this plan is designed to dogfood the agent-pipeline end to end.**
| Pipeline artifact | How we use it |
|-------------------|---------------|
| L2 roles | Implementation runs as `role-implementer` per task brief (or ad hoc with implementer rules) |
| `claim_task` / `complete_task` | Every task session claims and completes via Echodo MCP |
| `agent_prompt` (plan → epic → task) | First-message context for agents; inheritance from this frontmatter |
| `docs/adr-model-routing.md` | Architect contract — agents read before coding |
| `.convoys/model-routing-registry.md` | **Created at kickoff** via `role-conductor` — tracks IA/UX/architecture for the initiative (not stored under `plans/`) |
| `model_policy` in convoy | Dogfoods alias-based routing (Claude planning, Qwen local) once registry exists |
| `scripts/log-convoy-event.sh` | Log each role completion to `.convoys/.metrics.jsonl` when L3 convoy scaffolding is present |
**Kickoff ritual (human + conductor):**
1. Prompt: *"Start a new convoy: Implement workspace-scoped model provider registry and role routing per docs/adr-model-routing.md. Success = operators can configure Anthropic + Ollama aliases and assign them to pipeline roles from Settings."*
2. Conductor writes `.convoys/model-routing-registry.md` with `classification: server-only` (skip ia, ux, design, a11y).
3. Architect decomposes into briefs that **map 1:1 to tasks in `Epic-model-provider-registry`** (below).
4. Each implementer session: `claim_task` on the matching `Task-*.md` → implement → `complete_task`.
## Getting started (operator checklist)
Do these **before or in parallel with** Epic 1 implementation. Nothing here blocks writing code; some items unblock dogfooding and `testProvider` against real endpoints.
### A. Repo and backlog (one time)
- [ ] `pnpm install` at repo root
- [ ] `.env` from `.env.example` — `DATABASE_URL`, `REDIS_URL`, `NEXTAUTH_*` at minimum
- [ ] `pnpm db:migrate` if schema is behind
- [ ] `pnpm --filter @tasks/database import:markdown-backlog` — loads `plans/Plan-automation-foundation/**` into DB for `claim_task`
- [ ] Confirm Echodo MCP in `.cursor/mcp.json` works (restart Cursor after MCP changes)
### B. Local inference on Mac (pick one or both)
See ADR § **Local MLX on Apple Silicon** and **Local Ollama**.
| Goal | Action |
|------|--------|
| Simplest local coder | Install [Ollama](https://ollama.com), `ollama pull qwen2.5-coder:32b`, listen on LAN if Coolify must reach Mac |
| Best Apple Silicon perf / MLX models | `pip install mlx-openai-server` (preferred if you expect **parallel implementers**), launch with `--host 0.0.0.0` |
| Quick MLX smoke test | `pip install mlx-lm` → `mlx_lm.server --model mlx-community/...` on port 8080 |
Note your Mac LAN IP and port for registry aliases later (`http://<ip>:8000/v1` or `:11434/v1`).
### C. Cloud / free-tier aliases (optional until Epic 1 UI exists)
Configure env vars on the machine that runs `apps/web` / worker (Coolify or local):
- `ANTHROPIC_API_KEY` — planning roles (or use NIM/Gemini free tier per ADR references)
- `GOOGLE_GENERATIVE_AI_API_KEY`, `NVIDIA_API_KEY` — optional free-tier planning/gates
Until Settings UI ships, `OPENAI_BASE_URL` + `OPENAI_API_KEY` still power the existing chat route.
### D. Agent-pipeline convoy (when you start building)
- [ ] Run **role-conductor** (Cursor): convoy for model registry per kickoff ritual above → `.convoys/model-routing-registry.md`
- [ ] **role-architect** → briefs map to Epic 1 tasks
- [ ] Epic 2 (optional parallel): install `.convoys/` scaffolding + `log-convoy-event.sh`
### E. First implementation task
Start with **`Task-shared-schemas-resolve-model`** — no DB migration, pure `packages/shared` + tests. Then migration, tRPC, UI.
### F. Not needed yet
- Forgejo / connected repos (Epic 3)
- Orchestrator or worker (Epics 6–7)
- LiteLLM (optional gateway later)
- Symphony
## Objectives and success criteria
- Workspace admins can CRUD model provider aliases (Anthropic, OpenAI, Google, OpenAI-compatible for Ollama/NIM/LiteLLM).
- Workspace admins can map any `role_id` (nine L2 defaults + custom) to a provider alias.
- `resolveModel()` implements ADR resolution order (brief → convoy → workspace → template).
- Settings UI: Providers + Role routing pages with connection test.
- `agent_runs` records `model_alias`, `provider_type`, `model_id`, `model_tier` on completion.
- Documented in ADR; `.env.example` updated with referenced env var names.
## Scope and boundaries
**In scope (full plan)**
- Twelve epics — see [Epic roadmap](#epic-roadmap-complete) below (engine + owner cockpit).
- Epics 1–6 **done**; Epic 7 **in progress**; Epics 8–12 **draft/ready** with bead-scale tasks.
- [Autensa benchmark](#autensa-benchmark-2026-08) follow-ons remain mapped under 1, 3, 5–9.
- ADR at `docs/adr-model-routing.md`.
**Out of scope (separate future plans)**
- Agent-pipeline Phase 2b deliverable templates.
- Agent-pipeline Phase 4 drift/rollup workers (replace GHA cron).
- LiteLLM deployment guide (operators use OpenAI-compatible alias).
- Symphony adoption.
- Autensa research → swipe ideation.
## Cross-team collaborators
| Name / group | Role |
|--------------|------|
| Solo operator | Product, implement, dogfood pipeline |
## Epic roadmap (complete)
**Current focus: close Epic 7** (draft PR + live E2E), then **Epic 8 → 9 → 10 → 11 → 12**. Follow-ons on done epics can run in parallel when they do not block the critical path.
```mermaid
flowchart LR
E1[1 Model registry]
E2[2 Convoy scaffolding]
E3[3 Connected repos]
E4[4 Convoy lifecycle MCP]
E5[5 Gates UI]
E6[6 Orchestrator]
E7[7 Worker v1]
E8[8 Forge webhooks]
E9[9 Fleet dashboard]
E10[10 Plans roadmap UI]
E11[11 Completion artifacts]
E12[12 Ops copilot]
E1 --> E6
E1 --> E7
E2 --> E4
E3 --> E6
E4 --> E5
E4 --> E6
E5 --> E6
E6 --> E7
E7 --> E8
E5 --> E9
E4 --> E9
E4 --> E10
E5 --> E10
E8 --> E11
E9 --> E12
E10 --> E12
```
| # | Epic | Status | Delivers | Depends on |
|---|------|--------|----------|------------|
| 1 | [Model provider registry](./Epic-model-provider-registry/Epic-model-provider-registry.md) | done (+1 follow-on) | `resolveModel()`, Settings AI; cost caps task | — |
| 2 | [Pipeline convoy scaffolding](./Epic-pipeline-convoy-scaffolding/Epic-pipeline-convoy-scaffolding.md) | done | `.convoys/`, metrics, planning rule | — |
| 3 | [Connected repos](./Epic-connected-repos/Epic-connected-repos.md) | done (+1 follow-on) | `repo_links`, forge adapters; preflight | — |
| 4 | [Convoy lifecycle MCP](./Epic-convoy-lifecycle-mcp/Epic-convoy-lifecycle-mcp.md) | done | create/transition/events | 2, 3 |
| 5 | [Mission control gates UI](./Epic-mission-control-gates-ui/Epic-mission-control-gates-ui.md) | done (+2 follow-ons) | Kanban, gates, co-pilot; DAG, tiers | 4 |
| 6 | [Orchestrator](./Epic-apps-orchestrator/Epic-apps-orchestrator.md) | done (+1 follow-on) | Redis dispatch; auto-nudge | 1, 3, 4 |
| 7 | [Worker v1](./Epic-apps-worker-v1/Epic-apps-worker-v1.md) | **in progress** | Loop, checkpoints, **draft PR**, **live E2E**, notes | 1, 3, 6 |
| 8 | [Forge webhooks](./Epic-forge-webhooks/Epic-forge-webhooks.md) | draft | Webhooks + PR checks recovery | 3, 4, 7 |
| 9 | [Fleet dashboard](./Epic-fleet-dashboard/Epic-fleet-dashboard.md) | draft | Cross-repo home + SSE feed | 4, 5 |
| 10 | [Plans roadmap UI](./Epic-plans-roadmap-ui/Epic-plans-roadmap-ui.md) | draft | Tree, status, Start → convoy | 4, 5 |
| 11 | [Owner completion artifacts](./Epic-owner-completion-artifacts/Epic-owner-completion-artifacts.md) | draft | Summaries, gallery, screenshots | 7, 8 |
| 12 | [Ops copilot](./Epic-ops-copilot/Epic-ops-copilot.md) | draft | Project-aware status chat | 9, 10 |
### Overlap with existing plans
| Existing artifact | How this roadmap relates |
|-------------------|-------------------------|
| [Plan-agent-coordination](../Plan-agent-coordination/Plan-agent-coordination.md) | Epics 1 (logging) and 4–7 **extend** it; `claim_task` / `complete_task` stay canonical |
| [Epic-optional-orchestrator](../Plan-agent-coordination/Epic-optional-orchestrator/Epic-optional-orchestrator.md) | **Superseded in intent** by Epic 6 here — richer spec, Echodo-native, no Symphony dependency |
| Agent-pipeline v0.4 Phase 2a | Epic 4 implements the MCP tools described there |
| Agent-pipeline v0.4 Phase 4 workers | Out of scope until Epic 8+; drift/rollup is a future Plan, not this roadmap |
### End-state (what “done” looks like for the full plan)
Owner opens **fleet** → sees gates/PRs/releases across projects → opens a repo **Plans** roadmap → **Start** a ready task → Mission Control convoy → **Go** → worker (local or server models) builds without Cursor → draft PR → approve gates → **completion summary + screenshots/demos** → ask **ops copilot** what needs attention. `plans/` stays git SoT; DB + UI are how owners and agents query without dumping whole trees into context.
### Autensa benchmark (2026-08)
Studied [crshdn/mission-control](https://github.com/crshdn/mission-control) (Autensa) as a UX/reliability benchmark — **not** as a runtime dependency. Ideas mapped to tasks:
| Autensa concept | Echodo task | Epic |
|-----------------|-------------|------|
| Checkpoint & crash recovery | [Task-worker-checkpoint-resume](./Epic-apps-worker-v1/Task-worker-checkpoint-resume.md) | 7 |
| Operator chat mid-build | [Task-worker-operator-notes](./Epic-apps-worker-v1/Task-worker-operator-notes.md) | 7 |
| Agent health / auto-nudge | [Task-orchestrator-auto-nudge](./Epic-apps-orchestrator/Task-orchestrator-auto-nudge.md) | 6 |
| Repo setup preflight | [Task-repo-setup-preflight](./Epic-connected-repos/Task-repo-setup-preflight.md) | 3 |
| PR checks recovery | [Task-pr-checks-recovery-panel](./Epic-forge-webhooks/Task-pr-checks-recovery-panel.md) | 8 |
| Convoy DAG visualization | [Task-convoy-dag-visualization](./Epic-mission-control-gates-ui/Task-convoy-dag-visualization.md) | 5 |
| Automation tiers | [Task-automation-tiers](./Epic-mission-control-gates-ui/Task-automation-tiers.md) | 5 |
| Cost tracking & budget caps | [Task-cost-budget-ui-and-caps](./Epic-model-provider-registry/Task-cost-budget-ui-and-caps.md) | 1 |
| Live activity feed (SSE) | [Task-convoy-activity-feed-sse](./Epic-fleet-dashboard/Task-convoy-activity-feed-sse.md) | 9 |
| Fleet rollup | [Task-fleet-rollup-dashboard](./Epic-fleet-dashboard/Task-fleet-rollup-dashboard.md) | 9 |
**Explicitly not adopted:** OpenClaw Gateway, autopilot research/ideation/swipe, preference learning, agent skills learner, SQLite single-tenant model.
## Epics (links)
| Epic | Link |
|------|------|
| 1 Model provider registry | `./Epic-model-provider-registry/Epic-model-provider-registry.md` |
| 2 Pipeline convoy scaffolding | `./Epic-pipeline-convoy-scaffolding/Epic-pipeline-convoy-scaffolding.md` |
| 3 Connected repos | `./Epic-connected-repos/Epic-connected-repos.md` |
| 4 Convoy lifecycle MCP | `./Epic-convoy-lifecycle-mcp/Epic-convoy-lifecycle-mcp.md` |
| 5 Mission control gates UI | `./Epic-mission-control-gates-ui/Epic-mission-control-gates-ui.md` |
| 6 Orchestrator | `./Epic-apps-orchestrator/Epic-apps-orchestrator.md` |
| 7 Worker v1 | `./Epic-apps-worker-v1/Epic-apps-worker-v1.md` |
| 8 Forge webhooks | `./Epic-forge-webhooks/Epic-forge-webhooks.md` |
| 9 Fleet dashboard | `./Epic-fleet-dashboard/Epic-fleet-dashboard.md` |
| 10 Plans roadmap UI | `./Epic-plans-roadmap-ui/Epic-plans-roadmap-ui.md` |
| 11 Owner completion artifacts | `./Epic-owner-completion-artifacts/Epic-owner-completion-artifacts.md` |
| 12 Ops copilot | `./Epic-ops-copilot/Epic-ops-copilot.md` |
## Metrics and milestones
| Milestone | Epic | Metric |
|-----------|------|--------|
| M0 ADR | — | `docs/adr-model-routing.md` merged |
| M1 Model registry | 1 | Settings: configure + test Anthropic + Ollama aliases |
| M2 Dogfood loop | 1–2 | One task via claim_task → complete_task with model fields |
| M3 Repos connected | 3 | Paste Forge/GitHub URL; clone succeeds from Echodo |
| M4 Convoy state | 4 | `transition_convoy_status` enforces gate actors |
| M5 Human surface | 5 | Gate inbox shows pending `convoy_review` |
| M6 Automation | 6–7 | Unattended implementer brief (live E2E task) |
| M6b Draft PR | 7 | Worker opens draft PR on success |
| M7 PR loop | 8 | Merge webhook moves convoy to `merged` |
| M8 Fleet | 9 | Single view across ≥2 connected repos |
| M9 Reliability | 7 | Checkpoint resume after kill |
| M10 Observability | 9 | Activity feed shows role_complete without refresh |
| M11 Spend control | 1 follow-on | Month-to-date spend + dispatch pause on cap |
| M12 Roadmap UI | 10 | Plans tree + Start → convoy |
| M13 Owner package | 11 | Completion summary (+ screenshot when UI) |
| M14 Ops chat | 12 | “What needs me?” answered with links |
## Recommended execution order (owner vision)
### Phase A — Engine (do now)
1. [Task-worker-live-e2e-dogfood](./Epic-apps-worker-v1/Task-worker-live-e2e-dogfood.md) — confirm A1 on Coolify/local (**ops checklist**)
2. [Task-worker-draft-pr](./Epic-apps-worker-v1/Task-worker-draft-pr.md) — **next coding chat** if PR not already working
3. [Task-forge-webhook-receiver](./Epic-forge-webhooks/Task-forge-webhook-receiver.md) → [Task-pr-checks-recovery-panel](./Epic-forge-webhooks/Task-pr-checks-recovery-panel.md)
4. *Parallel optional:* operator notes, auto-nudge, repo preflight
### Phase B — Fleet
5. [Task-fleet-rollup-dashboard](./Epic-fleet-dashboard/Task-fleet-rollup-dashboard.md) → SSE feed
### Phase C — Roadmap
6. Epic 10: backlog tree tRPC → Plans page → Start → convoy
### Phase D — Owner review
7. Epic 11: completion summary → artifact gallery → screenshot capture
### Phase E — Ops chat
8. Epic 12: context tools → UI → draft work from chat
### Polish (anytime after blockers clear)
- Epic 5: DAG viz, automation tiers (tiers after Epic 8)
- Epic 1: cost budget caps
- Epic 7: operator notes if not done in Phase A
## Risks and mitigations
| Risk | Mitigation |
|------|------------|
| Mac Ollama unreachable from Coolify-hosted web | `testProvider` surfaces error; document LAN URL; dev can run web locally |
| Cursor slug drift in old convoys | Legacy alias map in ADR; conductor uses registry aliases for new convoys |
| Scope creep into orchestrator | ADR + plan explicitly defer daemon/worker |
## Links
- [ADR: Model routing](../../docs/adr-model-routing.md)
- [Plan: Agent coordination](../Plan-agent-coordination/Plan-agent-coordination.md)
- Agent-pipeline: `docs/model-routing-policy.md`, `docs/role-reference.md`