Submatter/plans/Plan-automation-foundation/Plan-automation-foundation.md
Randall Stillwell 35def387ae docs(plans): add Autensa benchmark tasks to automation foundation roadmap
Map reliability, operator UX, and spend-control follow-ons onto epics 1–9
so worker, webhooks, and fleet work have bead-scale task briefs.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-03 10:48:36 -05:00

15 KiB
Raw Blame History

kind slug title status priority tenant_id cursor_plan_id updated_at agent_prompt
plan automation-foundation Automation foundation — mission control, model routing, orchestrator path ready P1 global null 2026-08-03 You are implementing Echodo's automation foundation. Read docs/adr-model-routing.md before touching model routing code. This repo uses pnpm, Drizzle, tRPC, zod in packages/shared, and multitenancy via workspace_id on every tenant table. Dogfooding (required for every task in this plan): 1. At session start: call MCP claim_task with this task's backlogItemId (import plans/ if the row is missing). 2. At session end: call MCP complete_task with outcome, tokensTotal, and modelAlias/modelTier when you used a routed model. 3. Stay within the task's acceptance criteria; do not widen scope. 4. Run pnpm lint && pnpm type-check && pnpm test before declaring done. Stack: Next.js App Router, packages/database for schema, apps/web for UI. Never store API keys in committed files or DB — only api_key_env_var names.

Plan overview

Echodo is evolving from a multitenant task app with MCP coordination into mission control for an agent pipeline: fleet view, human gates, per-role model routing, and (later) a built-in orchestrator and worker. This plan delivers the first concrete slice — the model provider registry and role routing system defined in docs/adr-model-routing.md.

Epics 1–6 and the mission-control UI are shipped. Epic 7 (worker) is in progress — skeleton exists in apps/worker. Epics 8–9 remain draft with decomposed tasks. Follow-on tasks from the Autensa benchmark are injected into epics 1, 3, 5, 6, 7, 8, and 9 (concepts only — no OpenClaw dependency).

Dogfooding the agent pipeline

Yes — this plan is designed to dogfood the agent-pipeline end to end.

Pipeline artifact How we use it
L2 roles Implementation runs as role-implementer per task brief (or ad hoc with implementer rules)
claim_task / complete_task Every task session claims and completes via Echodo MCP
agent_prompt (plan → epic → task) First-message context for agents; inheritance from this frontmatter
docs/adr-model-routing.md Architect contract — agents read before coding
.convoys/model-routing-registry.md Created at kickoff via role-conductor — tracks IA/UX/architecture for the initiative (not stored under plans/)
model_policy in convoy Dogfoods alias-based routing (Claude planning, Qwen local) once registry exists
scripts/log-convoy-event.sh Log each role completion to .convoys/.metrics.jsonl when L3 convoy scaffolding is present

Kickoff ritual (human + conductor):

  1. Prompt: "Start a new convoy: Implement workspace-scoped model provider registry and role routing per docs/adr-model-routing.md. Success = operators can configure Anthropic + Ollama aliases and assign them to pipeline roles from Settings."
  2. Conductor writes .convoys/model-routing-registry.md with classification: server-only (skip ia, ux, design, a11y).
  3. Architect decomposes into briefs that map 1:1 to tasks in Epic-model-provider-registry (below).
  4. Each implementer session: claim_task on the matching Task-*.md → implement → complete_task.

Getting started (operator checklist)

Do these before or in parallel with Epic 1 implementation. Nothing here blocks writing code; some items unblock dogfooding and testProvider against real endpoints.

A. Repo and backlog (one time)

  • pnpm install at repo root
  • .env from .env.example — DATABASE_URL, REDIS_URL, NEXTAUTH_* at minimum
  • pnpm db:migrate if schema is behind
  • pnpm --filter @tasks/database import:markdown-backlog — loads plans/Plan-automation-foundation/** into DB for claim_task
  • Confirm Echodo MCP in .cursor/mcp.json works (restart Cursor after MCP changes)

B. Local inference on Mac (pick one or both)

See ADR § Local MLX on Apple Silicon and Local Ollama.

Goal Action
Simplest local coder Install Ollama, ollama pull qwen2.5-coder:32b, listen on LAN if Coolify must reach Mac
Best Apple Silicon perf / MLX models pip install mlx-openai-server (preferred if you expect parallel implementers), launch with --host 0.0.0.0
Quick MLX smoke test pip install mlx-lm → mlx_lm.server --model mlx-community/... on port 8080

Note your Mac LAN IP and port for registry aliases later (http://<ip>:8000/v1 or :11434/v1).

C. Cloud / free-tier aliases (optional until Epic 1 UI exists)

Configure env vars on the machine that runs apps/web / worker (Coolify or local):

  • ANTHROPIC_API_KEY — planning roles (or use NIM/Gemini free tier per ADR references)
  • GOOGLE_GENERATIVE_AI_API_KEY, NVIDIA_API_KEY — optional free-tier planning/gates

Until Settings UI ships, OPENAI_BASE_URL + OPENAI_API_KEY still power the existing chat route.

D. Agent-pipeline convoy (when you start building)

  • Run role-conductor (Cursor): convoy for model registry per kickoff ritual above → .convoys/model-routing-registry.md
  • role-architect → briefs map to Epic 1 tasks
  • Epic 2 (optional parallel): install .convoys/ scaffolding + log-convoy-event.sh

E. First implementation task

Start with Task-shared-schemas-resolve-model — no DB migration, pure packages/shared + tests. Then migration, tRPC, UI.

F. Not needed yet

  • Forgejo / connected repos (Epic 3)
  • Orchestrator or worker (Epics 6–7)
  • LiteLLM (optional gateway later)
  • Symphony

Objectives and success criteria

  • Workspace admins can CRUD model provider aliases (Anthropic, OpenAI, Google, OpenAI-compatible for Ollama/NIM/LiteLLM).
  • Workspace admins can map any role_id (nine L2 defaults + custom) to a provider alias.
  • resolveModel() implements ADR resolution order (brief → convoy → workspace → template).
  • Settings UI: Providers + Role routing pages with connection test.
  • agent_runs records model_alias, provider_type, model_id, model_tier on completion.
  • Documented in ADR; .env.example updated with referenced env var names.

Scope and boundaries

In scope (full plan)

  • Nine epics — see Epic roadmap below.
  • Epics 1–6 done; Epic 7 in progress; Epics 8–9 draft with bead-scale tasks.
  • Autensa benchmark follow-ons: reliability (checkpoints, auto-nudge), operator UX (activity feed, DAG, PR recovery), cost caps.
  • ADR at docs/adr-model-routing.md.

Out of scope (separate future plans)

  • Agent-pipeline Phase 2b deliverable templates.
  • Agent-pipeline Phase 4 drift/rollup workers (replace GHA cron).
  • LiteLLM deployment guide (operators use OpenAI-compatible alias).
  • Symphony adoption.

Cross-team collaborators

Name / group Role
Solo operator Product, implement, dogfood pipeline

Epic roadmap (complete)

Sequential order with parallel opportunities noted. Current focus: Epic 7 (worker end-to-end dogfood), then Epic 8 → 9. Follow-on tasks on done epics can run in parallel when they do not block the critical path.

flowchart LR
  E1[1 Model registry]
  E2[2 Convoy scaffolding]
  E3[3 Connected repos]
  E4[4 Convoy lifecycle MCP]
  E5[5 Gates UI]
  E6[6 Orchestrator]
  E7[7 Worker v1]
  E8[8 Forge webhooks]
  E9[9 Fleet dashboard]

  E1 --> E6
  E1 --> E7
  E2 --> E4
  E3 --> E6
  E4 --> E5
  E4 --> E6
  E5 --> E6
  E6 --> E7
  E7 --> E8
  E5 --> E9
  E4 --> E9
# Epic Status Delivers Depends on
1 Model provider registry done (+1 follow-on) resolveModel(), Settings AI pages, run model logging; cost caps task —
2 Pipeline convoy scaffolding done .convoys/, log-convoy-event.sh, convoy-planning.mdc — dogfood metrics — (parallel with 1)
3 Connected repos done (+1 follow-on) repo_links table, paste URL, forge adapters; repo setup preflight —
4 Convoy lifecycle MCP done create_convoy, transition_convoy_status, state machine §7, convoy_events 2, 3
5 Mission control gates UI done (+2 follow-ons) Kanban, gate inbox, co-pilot; DAG viz + automation tiers 4
6 Orchestrator done (+1 follow-on) apps/orchestrator, Redis queue, role dispatch; auto-nudge 1, 3, 4
7 Worker v1 in progress (3 tasks) Tool loop, checkpoints, operator notes 1, 3, 6
8 Forge webhooks draft (2 tasks) Webhooks + PR checks recovery panel 3, 4, 7
9 Fleet dashboard draft (2 tasks) Cross-repo rollup + SSE activity feed 4, 5

Overlap with existing plans

Existing artifact How this roadmap relates
Plan-agent-coordination Epics 1 (logging) and 4–7 extend it; claim_task / complete_task stay canonical
Epic-optional-orchestrator Superseded in intent by Epic 6 here — richer spec, Echodo-native, no Symphony dependency
Agent-pipeline v0.4 Phase 2a Epic 4 implements the MCP tools described there
Agent-pipeline v0.4 Phase 4 workers Out of scope until Epic 8+; drift/rollup is a future Plan, not this roadmap

End-state (what “done” looks like for the full plan)

Operator opens Echodo fleet view → starts convoy chat → hits Go → orchestrator runs pipeline roles with per-role models → worker executes briefs against connected repos → gates surface in inbox with AI co-pilot → approve → repeat until released. Live activity feed, spend rollups, and PR recovery are visible without leaving mission control.

Autensa benchmark (2026-08)

Studied crshdn/mission-control (Autensa) as a UX/reliability benchmark — not as a runtime dependency. Ideas mapped to tasks:

Autensa concept Echodo task Epic
Checkpoint & crash recovery Task-worker-checkpoint-resume 7
Operator chat mid-build Task-worker-operator-notes 7
Agent health / auto-nudge Task-orchestrator-auto-nudge 6
Repo setup preflight Task-repo-setup-preflight 3
PR checks recovery Task-pr-checks-recovery-panel 8
Convoy DAG visualization Task-convoy-dag-visualization 5
Automation tiers Task-automation-tiers 5
Cost tracking & budget caps Task-cost-budget-ui-and-caps 1
Live activity feed (SSE) Task-convoy-activity-feed-sse 9
Fleet rollup Task-fleet-rollup-dashboard 9

Explicitly not adopted: OpenClaw Gateway, autopilot research/ideation/swipe, preference learning, agent skills learner, SQLite single-tenant model.

Epic Link
1 Model provider registry ./Epic-model-provider-registry/Epic-model-provider-registry.md
2 Pipeline convoy scaffolding ./Epic-pipeline-convoy-scaffolding/Epic-pipeline-convoy-scaffolding.md
3 Connected repos ./Epic-connected-repos/Epic-connected-repos.md
4 Convoy lifecycle MCP ./Epic-convoy-lifecycle-mcp/Epic-convoy-lifecycle-mcp.md
5 Mission control gates UI ./Epic-mission-control-gates-ui/Epic-mission-control-gates-ui.md
6 Orchestrator ./Epic-apps-orchestrator/Epic-apps-orchestrator.md
7 Worker v1 ./Epic-apps-worker-v1/Epic-apps-worker-v1.md
8 Forge webhooks ./Epic-forge-webhooks/Epic-forge-webhooks.md
9 Fleet dashboard ./Epic-fleet-dashboard/Epic-fleet-dashboard.md

Metrics and milestones

Milestone Epic Metric
M0 ADR — docs/adr-model-routing.md merged
M1 Model registry 1 Settings: configure + test Anthropic + Ollama aliases
M2 Dogfood loop 1–2 One task via claim_task → complete_task with model fields
M3 Repos connected 3 Paste Forge/GitHub URL; clone succeeds from Echodo
M4 Convoy state 4 transition_convoy_status enforces gate actors
M5 Human surface 5 Gate inbox shows pending convoy_review
M6 Automation 6–7 Orchestrator dispatches one implementer brief unattended
M7 PR loop 8 Merge webhook moves convoy to merged
M8 Fleet 9 Single view across ≥2 connected repos
M9 Reliability 7 Worker completes brief; checkpoint resume after kill
M10 Observability 9 Activity feed shows role_complete without refresh
M11 Spend control 1 follow-on Month-to-date spend + dispatch pause on cap

Recommended execution order (post-benchmark)

  1. Epic 7 critical path — Task-worker-minimal-tool-loop → live dogfood with orchestrator
  2. Epic 7 reliability — checkpoint/resume, then operator notes
  3. Epic 8 — webhook receiver → PR checks panel (needs worker PR output)
  4. Epic 6 follow-on — auto-nudge (pairs with checkpoints)
  5. Epic 3 follow-on — repo preflight (before unattended prod dogfood)
  6. Epic 9 — fleet rollup → SSE feed
  7. Epic 5 follow-ons — DAG viz, automation tiers (after Epic 8 for semi-auto)
  8. Epic 1 follow-on — cost caps (after worker records token totals)

Risks and mitigations

Risk Mitigation
Mac Ollama unreachable from Coolify-hosted web testProvider surfaces error; document LAN URL; dev can run web locally
Cursor slug drift in old convoys Legacy alias map in ADR; conductor uses registry aliases for new convoys
Scope creep into orchestrator ADR + plan explicitly defer daemon/worker