rstillwell/Submatter
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
Submatter/plans/Plan-automation-foundation/Plan-automation-foundation.md
Randall Stillwell d1b2b9695e docs(plans): expand automation foundation for owner cockpit vision 
Add epics 10–12 (plans UI, completion artifacts, ops copilot), worker
draft-PR and live E2E verify tasks, and phase A–E execution order after
A1 gap review.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-12 14:11:20 -05:00

323 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
kind: plan
slug: automation-foundation
title: Automation foundation — mission control, model routing, orchestrator path
status: ready
priority: P1
tenant_id: global
cursor_plan_id: null
updated_at: "2026-08-12"
agent_prompt: |
You are implementing Echodo's automation foundation. Read docs/adr-model-routing.md
before touching model routing code. This repo uses pnpm, Drizzle, tRPC, zod in
packages/shared, and multitenancy via workspace_id on every tenant table.
Dogfooding (required for every task in this plan):
1. At session start: call MCP claim_task with this task's backlogItemId (import
plans/ if the row is missing).
2. At session end: call MCP complete_task with outcome, tokensTotal, and
modelAlias/modelTier when you used a routed model.
3. Stay within the task's acceptance criteria; do not widen scope.
4. Run pnpm lint && pnpm type-check && pnpm test before declaring done.
Stack: Next.js App Router, packages/database for schema, apps/web for UI.
Never store API keys in committed files or DB — only api_key_env_var names.
---
# Plan overview
Echodo / **Submatter** is evolving from a multitenant task app with MCP coordination into an **owner cockpit** for agent-built personal projects: fleet home, roadmap + Start, mission-control gates, first-party worker (leave Cursor for convoy builds), forge PRs, completion summaries/demos, and ops copilot.
Epics **1–6 shipped**. Epic **7 in progress** (tool loop + checkpoints in code; draft PR + live E2E verify still open). Epics **8–9** (webhooks, fleet) and **10–12** (plans UI, completion artifacts, ops copilot) carry the full owner vision. Autensa/Builderz-style ideas are benchmarks only — no OpenClaw dependency.
## Owner vision phases
| Phase | Goal | Epics |
|-------|------|-------|
| **A** Engine without Cursor | Go → worker → PR → gates | 7 → 8 (+ draft PR, live E2E) |
| **B** Fleet attention home | All projects, needs-you, activity | 9 |
| **C** Roadmap + Start | Plans tree, Start → convoy | 10 |
| **D** Owner review | Summaries, screenshots/demos | 11 |
| **E** Ops chat | Status / triage / draft work | 12 |
### A1 status check (2026-08-12)
| Claim | Reality in repo |
|-------|-----------------|
| Worker tool loop exists | Yes — `apps/worker` dequeue, session, worktree, claim/complete |
| Checkpoints | Yes — task done, `worker_checkpoints` path documented |
| Live Go → unattended job on your deploy | **Operator-verify** — [Task-worker-live-e2e-dogfood](./Epic-apps-worker-v1/Task-worker-live-e2e-dogfood.md) |
| Draft PR from worker | **Missing** — no forge PR call in worker; [Task-worker-draft-pr](./Epic-apps-worker-v1/Task-worker-draft-pr.md) |
| Minimal-tool-loop marked `done` | Code landed; several AC checkboxes in the task file still unchecked — treat live E2E + draft PR as the real close-out |
**If you “finished A1”** meaning services run and a job completed: mark criteria on the live E2E task and keep going. **If you only shipped code:** run the live E2E checklist before calling Phase A done.
## Dogfooding the agent pipeline
**Yes — this plan is designed to dogfood the agent-pipeline end to end.**
| Pipeline artifact | How we use it |
|-------------------|---------------|
| L2 roles | Implementation runs as `role-implementer` per task brief (or ad hoc with implementer rules) |
| `claim_task` / `complete_task` | Every task session claims and completes via Echodo MCP |
| `agent_prompt` (plan → epic → task) | First-message context for agents; inheritance from this frontmatter |
| `docs/adr-model-routing.md` | Architect contract — agents read before coding |
| `.convoys/model-routing-registry.md` | **Created at kickoff** via `role-conductor` — tracks IA/UX/architecture for the initiative (not stored under `plans/`) |
| `model_policy` in convoy | Dogfoods alias-based routing (Claude planning, Qwen local) once registry exists |
| `scripts/log-convoy-event.sh` | Log each role completion to `.convoys/.metrics.jsonl` when L3 convoy scaffolding is present |
**Kickoff ritual (human + conductor):**
1. Prompt: *"Start a new convoy: Implement workspace-scoped model provider registry and role routing per docs/adr-model-routing.md. Success = operators can configure Anthropic + Ollama aliases and assign them to pipeline roles from Settings."*
2. Conductor writes `.convoys/model-routing-registry.md` with `classification: server-only` (skip ia, ux, design, a11y).
3. Architect decomposes into briefs that **map 1:1 to tasks in `Epic-model-provider-registry`** (below).
4. Each implementer session: `claim_task` on the matching `Task-*.md` → implement → `complete_task`.
## Getting started (operator checklist)
Do these **before or in parallel with** Epic 1 implementation. Nothing here blocks writing code; some items unblock dogfooding and `testProvider` against real endpoints.
### A. Repo and backlog (one time)
- [ ] `pnpm install` at repo root
- [ ] `.env` from `.env.example` — `DATABASE_URL`, `REDIS_URL`, `NEXTAUTH_*` at minimum
- [ ] `pnpm db:migrate` if schema is behind
- [ ] `pnpm --filter @tasks/database import:markdown-backlog` — loads `plans/Plan-automation-foundation/**` into DB for `claim_task`
- [ ] Confirm Echodo MCP in `.cursor/mcp.json` works (restart Cursor after MCP changes)
### B. Local inference on Mac (pick one or both)
See ADR § **Local MLX on Apple Silicon** and **Local Ollama**.
| Goal | Action |
|------|--------|
| Simplest local coder | Install [Ollama](https://ollama.com), `ollama pull qwen2.5-coder:32b`, listen on LAN if Coolify must reach Mac |
| Best Apple Silicon perf / MLX models | `pip install mlx-openai-server` (preferred if you expect **parallel implementers**), launch with `--host 0.0.0.0` |
| Quick MLX smoke test | `pip install mlx-lm` → `mlx_lm.server --model mlx-community/...` on port 8080 |
Note your Mac LAN IP and port for registry aliases later (`http://<ip>:8000/v1` or `:11434/v1`).
### C. Cloud / free-tier aliases (optional until Epic 1 UI exists)
Configure env vars on the machine that runs `apps/web` / worker (Coolify or local):
- `ANTHROPIC_API_KEY` — planning roles (or use NIM/Gemini free tier per ADR references)
- `GOOGLE_GENERATIVE_AI_API_KEY`, `NVIDIA_API_KEY` — optional free-tier planning/gates
Until Settings UI ships, `OPENAI_BASE_URL` + `OPENAI_API_KEY` still power the existing chat route.
### D. Agent-pipeline convoy (when you start building)
- [ ] Run **role-conductor** (Cursor): convoy for model registry per kickoff ritual above → `.convoys/model-routing-registry.md`
- [ ] **role-architect** → briefs map to Epic 1 tasks
- [ ] Epic 2 (optional parallel): install `.convoys/` scaffolding + `log-convoy-event.sh`
### E. First implementation task
Start with **`Task-shared-schemas-resolve-model`** — no DB migration, pure `packages/shared` + tests. Then migration, tRPC, UI.
### F. Not needed yet
- Forgejo / connected repos (Epic 3)
- Orchestrator or worker (Epics 6–7)
- LiteLLM (optional gateway later)
- Symphony
## Objectives and success criteria
- Workspace admins can CRUD model provider aliases (Anthropic, OpenAI, Google, OpenAI-compatible for Ollama/NIM/LiteLLM).
- Workspace admins can map any `role_id` (nine L2 defaults + custom) to a provider alias.
- `resolveModel()` implements ADR resolution order (brief → convoy → workspace → template).
- Settings UI: Providers + Role routing pages with connection test.
- `agent_runs` records `model_alias`, `provider_type`, `model_id`, `model_tier` on completion.
- Documented in ADR; `.env.example` updated with referenced env var names.
## Scope and boundaries
**In scope (full plan)**
- Twelve epics — see [Epic roadmap](#epic-roadmap-complete) below (engine + owner cockpit).
- Epics 1–6 **done**; Epic 7 **in progress**; Epics 8–12 **draft/ready** with bead-scale tasks.
- [Autensa benchmark](#autensa-benchmark-2026-08) follow-ons remain mapped under 1, 3, 5–9.
- ADR at `docs/adr-model-routing.md`.
**Out of scope (separate future plans)**
- Agent-pipeline Phase 2b deliverable templates.
- Agent-pipeline Phase 4 drift/rollup workers (replace GHA cron).
- LiteLLM deployment guide (operators use OpenAI-compatible alias).
- Symphony adoption.
- Autensa research → swipe ideation.
## Cross-team collaborators
| Name / group | Role |
|--------------|------|
| Solo operator | Product, implement, dogfood pipeline |
## Epic roadmap (complete)
**Current focus: close Epic 7** (draft PR + live E2E), then **Epic 8 → 9 → 10 → 11 → 12**. Follow-ons on done epics can run in parallel when they do not block the critical path.
```mermaid
flowchart LR
E1[1 Model registry]
E2[2 Convoy scaffolding]
E3[3 Connected repos]
E4[4 Convoy lifecycle MCP]
E5[5 Gates UI]
E6[6 Orchestrator]
E7[7 Worker v1]
E8[8 Forge webhooks]
E9[9 Fleet dashboard]
E10[10 Plans roadmap UI]
E11[11 Completion artifacts]
E12[12 Ops copilot]
E1 --> E6
E1 --> E7
E2 --> E4
E3 --> E6
E4 --> E5
E4 --> E6
E5 --> E6
E6 --> E7
E7 --> E8
E5 --> E9
E4 --> E9
E4 --> E10
E5 --> E10
E8 --> E11
E9 --> E12
E10 --> E12
```
| # | Epic | Status | Delivers | Depends on |
|---|------|--------|----------|------------|
| 1 | [Model provider registry](./Epic-model-provider-registry/Epic-model-provider-registry.md) | done (+1 follow-on) | `resolveModel()`, Settings AI; cost caps task | — |
| 2 | [Pipeline convoy scaffolding](./Epic-pipeline-convoy-scaffolding/Epic-pipeline-convoy-scaffolding.md) | done | `.convoys/`, metrics, planning rule | — |
| 3 | [Connected repos](./Epic-connected-repos/Epic-connected-repos.md) | done (+1 follow-on) | `repo_links`, forge adapters; preflight | — |
| 4 | [Convoy lifecycle MCP](./Epic-convoy-lifecycle-mcp/Epic-convoy-lifecycle-mcp.md) | done | create/transition/events | 2, 3 |
| 5 | [Mission control gates UI](./Epic-mission-control-gates-ui/Epic-mission-control-gates-ui.md) | done (+2 follow-ons) | Kanban, gates, co-pilot; DAG, tiers | 4 |
| 6 | [Orchestrator](./Epic-apps-orchestrator/Epic-apps-orchestrator.md) | done (+1 follow-on) | Redis dispatch; auto-nudge | 1, 3, 4 |
| 7 | [Worker v1](./Epic-apps-worker-v1/Epic-apps-worker-v1.md) | **in progress** | Loop, checkpoints, **draft PR**, **live E2E**, notes | 1, 3, 6 |
| 8 | [Forge webhooks](./Epic-forge-webhooks/Epic-forge-webhooks.md) | draft | Webhooks + PR checks recovery | 3, 4, 7 |
| 9 | [Fleet dashboard](./Epic-fleet-dashboard/Epic-fleet-dashboard.md) | draft | Cross-repo home + SSE feed | 4, 5 |
| 10 | [Plans roadmap UI](./Epic-plans-roadmap-ui/Epic-plans-roadmap-ui.md) | draft | Tree, status, Start → convoy | 4, 5 |
| 11 | [Owner completion artifacts](./Epic-owner-completion-artifacts/Epic-owner-completion-artifacts.md) | draft | Summaries, gallery, screenshots | 7, 8 |
| 12 | [Ops copilot](./Epic-ops-copilot/Epic-ops-copilot.md) | draft | Project-aware status chat | 9, 10 |
### Overlap with existing plans
| Existing artifact | How this roadmap relates |
|-------------------|-------------------------|
| [Plan-agent-coordination](../Plan-agent-coordination/Plan-agent-coordination.md) | Epics 1 (logging) and 4–7 **extend** it; `claim_task` / `complete_task` stay canonical |
| [Epic-optional-orchestrator](../Plan-agent-coordination/Epic-optional-orchestrator/Epic-optional-orchestrator.md) | **Superseded in intent** by Epic 6 here — richer spec, Echodo-native, no Symphony dependency |
| Agent-pipeline v0.4 Phase 2a | Epic 4 implements the MCP tools described there |
| Agent-pipeline v0.4 Phase 4 workers | Out of scope until Epic 8+; drift/rollup is a future Plan, not this roadmap |
### End-state (what “done” looks like for the full plan)
Owner opens **fleet** → sees gates/PRs/releases across projects → opens a repo **Plans** roadmap → **Start** a ready task → Mission Control convoy → **Go** → worker (local or server models) builds without Cursor → draft PR → approve gates → **completion summary + screenshots/demos** → ask **ops copilot** what needs attention. `plans/` stays git SoT; DB + UI are how owners and agents query without dumping whole trees into context.
### Autensa benchmark (2026-08)
Studied [crshdn/mission-control](https://github.com/crshdn/mission-control) (Autensa) as a UX/reliability benchmark — **not** as a runtime dependency. Ideas mapped to tasks:
| Autensa concept | Echodo task | Epic |
|-----------------|-------------|------|
| Checkpoint & crash recovery | [Task-worker-checkpoint-resume](./Epic-apps-worker-v1/Task-worker-checkpoint-resume.md) | 7 |
| Operator chat mid-build | [Task-worker-operator-notes](./Epic-apps-worker-v1/Task-worker-operator-notes.md) | 7 |
| Agent health / auto-nudge | [Task-orchestrator-auto-nudge](./Epic-apps-orchestrator/Task-orchestrator-auto-nudge.md) | 6 |
| Repo setup preflight | [Task-repo-setup-preflight](./Epic-connected-repos/Task-repo-setup-preflight.md) | 3 |
| PR checks recovery | [Task-pr-checks-recovery-panel](./Epic-forge-webhooks/Task-pr-checks-recovery-panel.md) | 8 |
| Convoy DAG visualization | [Task-convoy-dag-visualization](./Epic-mission-control-gates-ui/Task-convoy-dag-visualization.md) | 5 |
| Automation tiers | [Task-automation-tiers](./Epic-mission-control-gates-ui/Task-automation-tiers.md) | 5 |
| Cost tracking & budget caps | [Task-cost-budget-ui-and-caps](./Epic-model-provider-registry/Task-cost-budget-ui-and-caps.md) | 1 |
| Live activity feed (SSE) | [Task-convoy-activity-feed-sse](./Epic-fleet-dashboard/Task-convoy-activity-feed-sse.md) | 9 |
| Fleet rollup | [Task-fleet-rollup-dashboard](./Epic-fleet-dashboard/Task-fleet-rollup-dashboard.md) | 9 |
**Explicitly not adopted:** OpenClaw Gateway, autopilot research/ideation/swipe, preference learning, agent skills learner, SQLite single-tenant model.
## Epics (links)
| Epic | Link |
|------|------|
| 1 Model provider registry | `./Epic-model-provider-registry/Epic-model-provider-registry.md` |
| 2 Pipeline convoy scaffolding | `./Epic-pipeline-convoy-scaffolding/Epic-pipeline-convoy-scaffolding.md` |
| 3 Connected repos | `./Epic-connected-repos/Epic-connected-repos.md` |
| 4 Convoy lifecycle MCP | `./Epic-convoy-lifecycle-mcp/Epic-convoy-lifecycle-mcp.md` |
| 5 Mission control gates UI | `./Epic-mission-control-gates-ui/Epic-mission-control-gates-ui.md` |
| 6 Orchestrator | `./Epic-apps-orchestrator/Epic-apps-orchestrator.md` |
| 7 Worker v1 | `./Epic-apps-worker-v1/Epic-apps-worker-v1.md` |
| 8 Forge webhooks | `./Epic-forge-webhooks/Epic-forge-webhooks.md` |
| 9 Fleet dashboard | `./Epic-fleet-dashboard/Epic-fleet-dashboard.md` |
| 10 Plans roadmap UI | `./Epic-plans-roadmap-ui/Epic-plans-roadmap-ui.md` |
| 11 Owner completion artifacts | `./Epic-owner-completion-artifacts/Epic-owner-completion-artifacts.md` |
| 12 Ops copilot | `./Epic-ops-copilot/Epic-ops-copilot.md` |
## Metrics and milestones
| Milestone | Epic | Metric |
|-----------|------|--------|
| M0 ADR | — | `docs/adr-model-routing.md` merged |
| M1 Model registry | 1 | Settings: configure + test Anthropic + Ollama aliases |
| M2 Dogfood loop | 1–2 | One task via claim_task → complete_task with model fields |
| M3 Repos connected | 3 | Paste Forge/GitHub URL; clone succeeds from Echodo |
| M4 Convoy state | 4 | `transition_convoy_status` enforces gate actors |
| M5 Human surface | 5 | Gate inbox shows pending `convoy_review` |
| M6 Automation | 6–7 | Unattended implementer brief (live E2E task) |
| M6b Draft PR | 7 | Worker opens draft PR on success |
| M7 PR loop | 8 | Merge webhook moves convoy to `merged` |
| M8 Fleet | 9 | Single view across ≥2 connected repos |
| M9 Reliability | 7 | Checkpoint resume after kill |
| M10 Observability | 9 | Activity feed shows role_complete without refresh |
| M11 Spend control | 1 follow-on | Month-to-date spend + dispatch pause on cap |
| M12 Roadmap UI | 10 | Plans tree + Start → convoy |
| M13 Owner package | 11 | Completion summary (+ screenshot when UI) |
| M14 Ops chat | 12 | “What needs me?” answered with links |
## Recommended execution order (owner vision)
### Phase A — Engine (do now)
1. [Task-worker-live-e2e-dogfood](./Epic-apps-worker-v1/Task-worker-live-e2e-dogfood.md) — confirm A1 on Coolify/local (**ops checklist**)
2. [Task-worker-draft-pr](./Epic-apps-worker-v1/Task-worker-draft-pr.md) — **next coding chat** if PR not already working
3. [Task-forge-webhook-receiver](./Epic-forge-webhooks/Task-forge-webhook-receiver.md) → [Task-pr-checks-recovery-panel](./Epic-forge-webhooks/Task-pr-checks-recovery-panel.md)
4. *Parallel optional:* operator notes, auto-nudge, repo preflight
### Phase B — Fleet
5. [Task-fleet-rollup-dashboard](./Epic-fleet-dashboard/Task-fleet-rollup-dashboard.md) → SSE feed
### Phase C — Roadmap
6. Epic 10: backlog tree tRPC → Plans page → Start → convoy
### Phase D — Owner review
7. Epic 11: completion summary → artifact gallery → screenshot capture
### Phase E — Ops chat
8. Epic 12: context tools → UI → draft work from chat
### Polish (anytime after blockers clear)
- Epic 5: DAG viz, automation tiers (tiers after Epic 8)
- Epic 1: cost budget caps
- Epic 7: operator notes if not done in Phase A
## Risks and mitigations
| Risk | Mitigation |
|------|------------|
| Mac Ollama unreachable from Coolify-hosted web | `testProvider` surfaces error; document LAN URL; dev can run web locally |
| Cursor slug drift in old convoys | Legacy alias map in ADR; conductor uses registry aliases for new convoys |
| Scope creep into orchestrator | ADR + plan explicitly defer daemon/worker |
## Links
- [ADR: Model routing](../../docs/adr-model-routing.md)
- [Plan: Agent coordination](../Plan-agent-coordination/Plan-agent-coordination.md)
- Agent-pipeline: `docs/model-routing-policy.md`, `docs/role-reference.md`
Reference in a new issue View git blame Copy permalink