Submatter/plans/Plan-automation-foundation/Plan-automation-foundation.md

kind	slug	title	status	priority	tenant_id	cursor_plan_id	updated_at	agent_prompt
plan	automation-foundation	Automation foundation — mission control, model routing, orchestrator path	ready	P1	global	null	2026-08-25	You are implementing Echodo's automation foundation. Read docs/adr-model-routing.md before touching model routing code. This repo uses pnpm, Drizzle, tRPC, zod in packages/shared, and multitenancy via workspace_id on every tenant table. Dogfooding (required for every task in this plan): 1. At session start: call MCP claim_task with this task's backlogItemId (import plans/ if the row is missing). 2. At session end: call MCP complete_task with outcome, tokensTotal, and modelAlias/modelTier when you used a routed model. 3. Stay within the task's acceptance criteria; do not widen scope. 4. Run pnpm lint && pnpm type-check && pnpm test before declaring done. Stack: Next.js App Router, packages/database for schema, apps/web for UI. Never store API keys in committed files or DB — only api_key_env_var names.

Plan overview

Echodo / Submatter is evolving from a multitenant task app with MCP coordination into an owner cockpit for agent-built personal projects: fleet home, roadmap + Start, mission-control gates, first-party worker (leave Cursor for convoy builds), forge PRs, completion summaries/demos, and ops copilot.

Epics 1–6 shipped. Epic 7 in progress (tool loop + checkpoints in code; draft PR + live E2E verify still open). Epics 8–9 (webhooks, fleet) and 10–12 (plans UI, completion artifacts, ops copilot) carry the full owner vision. Autensa/Builderz-style ideas are benchmarks only — no OpenClaw dependency.

2026-08-25 pivot — open self-hosted "Second Brain OS" (decision: docs/decisions/2026-08-25-second-brain-os-pivot.md)

After deep dive vs Grok Bot ($300/mo Heavy, no standalone) / OpenBot (AG-UI) / OpenClaw 347k stars / CrewAI 57k / LangChain 97k+ and two real-life walkthroughs (solo Maya Clarity offline notes, Studio North RouteMap), the plan pivots:

• Run for self (homelab dogfood) + publish as open MIT self-hosted Obsidian for crews — docker compose -f docker/docker-compose.yml up, BYO LLMs (qwen-coder 11434 / qwen-coder-mlx 8000 / NIM https://integrate.api.nvidia.com/v1 / LiteLLM), any harness via AG-UI/MCP. Do not clone LangChain/CrewAI; be the gated control plane they lack (No native policy gate).
• Harness-agnostic execution: WorkerAdapter packages/shared/src/worker/executor.ts dispatches minimal | crewai | langgraph | remote-ag-ui per role_harness_routes (mirror model_providers docs/adr-model-routing.md:78). handleOrchestratorJob apps/worker/src/handle-job.ts:40 stops hard-calling MinimalSession apps/worker/src/session/minimal-session.ts:87; job spec packages/shared/src/orchestrator/job.ts adds harnessAlias.
• Web-first: almost everything from echodo.stillwell.cloud — Fleet rollup + Plans tree CRUD for plans/ (backlog.createPlan/createEpic/createTask → writes .md → sync.ts), CreateConvoyDialog apps/web/components/mission-control/create-convoy-dialog.tsx → convoys.create apps/web/server/routers/convoys.ts:173, gates gateAction convoys.ts:237 + gateCopilot convoys.ts:267. MCP create-backlog-item is shortcut for natural "OpenCode, add Epic with 3 tasks".
• Non-code ready: worktree optional (docs/SOPs write straight to docs/ no branch apps/worker/src/git/worktree.ts:1), allowlist.ts per-harness, doc_review→published alongside pr_review→merged packages/shared/src/convoy-lifecycle/state-machine.ts, templates Settings → Templates for book/course/research.
• Monetize only if signal: >1k stars or 50 self-hosters or 3 managed fleet inquiries in 30 days post-publish; else keep homelab.

v0.1 for second brain (next 2 weeks) — second-brain-first order

1. MCP create-backlog-item (0.5d) — agents can author Task-*.md naturally
2. Optional worktree + per-harness allowlist (1d) — unlocks book/SOPs/research
3. Harness registry + WorkerAdapter (2-3d) — harness_providers table + Settings → Harnesses + AG-UI /api/ag-ui/[workspace] + testHarness
4. Plans tree (1w) — apps/web/app/(app)/[workspaceSlug]/plans/page.tsx Epic 10 (Task-plans-roadmap-page + Task-trpc-backlog-tree + Task-start-convoy-from-task)
5. Fleet rollup (3-4d) — /fleet Epic 9 (Task-fleet-rollup-dashboard + SSE) reusing ConvoyKanban

Owner vision phases (pre-pivot, kept for traceability)

Phase	Goal	Epics
A Engine without Cursor	Go → worker → PR → gates	7 → 8 (+ draft PR, live E2E)
B Fleet attention home	All projects, needs-you, activity	9
C Roadmap + Start	Plans tree, Start → convoy	10
D Owner review	Summaries, screenshots/demos	11
E Ops chat	Status / triage / draft work	12

A1 status check (2026-08-12)

Claim	Reality in repo
Worker tool loop exists	Yes — apps/worker dequeue, session, worktree, claim/complete
Checkpoints	Yes — task done, worker_checkpoints path documented
Live Go → unattended job on your deploy	Operator-verify — Task-worker-live-e2e-dogfood
Draft PR from worker	Missing — no forge PR call in worker; Task-worker-draft-pr
Minimal-tool-loop marked done	Code landed; several AC checkboxes in the task file still unchecked — treat live E2E + draft PR as the real close-out

If you “finished A1” meaning services run and a job completed: mark criteria on the live E2E task and keep going. If you only shipped code: run the live E2E checklist before calling Phase A done.

Dogfooding the agent pipeline

Yes — this plan is designed to dogfood the agent-pipeline end to end.

Pipeline artifact	How we use it
L2 roles	Implementation runs as role-implementer per task brief (or ad hoc with implementer rules)
claim_task / complete_task	Every task session claims and completes via Echodo MCP
agent_prompt (plan → epic → task)	First-message context for agents; inheritance from this frontmatter
docs/adr-model-routing.md	Architect contract — agents read before coding
.convoys/model-routing-registry.md	Created at kickoff via role-conductor — tracks IA/UX/architecture for the initiative (not stored under plans/)
model_policy in convoy	Dogfoods alias-based routing (Claude planning, Qwen local) once registry exists
scripts/log-convoy-event.sh	Log each role completion to .convoys/.metrics.jsonl when L3 convoy scaffolding is present

Kickoff ritual (human + conductor):

1. Prompt: "Start a new convoy: Implement workspace-scoped model provider registry and role routing per docs/adr-model-routing.md. Success = operators can configure Anthropic + Ollama aliases and assign them to pipeline roles from Settings."
2. Conductor writes .convoys/model-routing-registry.md with classification: server-only (skip ia, ux, design, a11y).
3. Architect decomposes into briefs that map 1:1 to tasks in Epic-model-provider-registry (below).
4. Each implementer session: claim_task on the matching Task-*.md → implement → complete_task.

Getting started (operator checklist)

Do these before or in parallel with Epic 1 implementation. Nothing here blocks writing code; some items unblock dogfooding and testProvider against real endpoints.

A. Repo and backlog (one time)

• pnpm install at repo root
• .env from .env.example — DATABASE_URL, REDIS_URL, NEXTAUTH_* at minimum
• pnpm db:migrate if schema is behind
• pnpm --filter @tasks/database import:markdown-backlog — loads plans/Plan-automation-foundation/** into DB for claim_task
• Confirm Echodo MCP in .cursor/mcp.json works (restart Cursor after MCP changes)

B. Local inference on Mac (pick one or both)

See ADR § Local MLX on Apple Silicon and Local Ollama.

Goal	Action
Simplest local coder	Install Ollama, ollama pull qwen2.5-coder:32b, listen on LAN if Coolify must reach Mac
Best Apple Silicon perf / MLX models	pip install mlx-openai-server (preferred if you expect parallel implementers), launch with --host 0.0.0.0
Quick MLX smoke test	pip install mlx-lm → mlx_lm.server --model mlx-community/... on port 8080

Note your Mac LAN IP and port for registry aliases later (http://<ip>:8000/v1 or :11434/v1).

C. Cloud / free-tier aliases (optional until Epic 1 UI exists)

Configure env vars on the machine that runs apps/web / worker (Coolify or local):

• ANTHROPIC_API_KEY — planning roles (or use NIM/Gemini free tier per ADR references)
• GOOGLE_GENERATIVE_AI_API_KEY, NVIDIA_API_KEY — optional free-tier planning/gates

Until Settings UI ships, OPENAI_BASE_URL + OPENAI_API_KEY still power the existing chat route.

D. Agent-pipeline convoy (when you start building)

• Run role-conductor (Cursor): convoy for model registry per kickoff ritual above → .convoys/model-routing-registry.md
• role-architect → briefs map to Epic 1 tasks
• Epic 2 (optional parallel): install .convoys/ scaffolding + log-convoy-event.sh

E. First implementation task

Start with Task-shared-schemas-resolve-model — no DB migration, pure packages/shared + tests. Then migration, tRPC, UI.

F. Not needed yet

• Forgejo / connected repos (Epic 3)
• Orchestrator or worker (Epics 6–7)
• LiteLLM (optional gateway later)
• Symphony

Objectives and success criteria — updated 2026-08-25 for second brain OS

Original (Epic 1) still holds + new for pivot:

• Workspace admins can CRUD model provider aliases (Anthropic, OpenAI, Google, OpenAI-compatible for Ollama/NIM/LiteLLM) — docs/adr-model-routing.md, Epic 1 done.
• Workspace admins can map any role_id (nine L2 defaults + custom) to a provider alias — role_model_routes.
• New: Workspace admins can CRUD harness providers (minimal | crewai | langgraph | remote-ag-ui) and map role_id → harness_alias (mirror of model routing, harness_providers + role_harness_routes, write-only auth_env_var).
• resolveModel() + resolveHarnessForRole() implement ADR resolution order (brief → convoy → workspace → template).
• Settings UI: Providers + Role routing + Harnesses pages with connection test (testProvider + testHarness).
• agent_runs + convoy_events + worker_checkpoints record harness_alias, model_alias, provider_type, model_id, model_tier on completion.
• Web-first: Fleet shows gates/PRs across workspaces, Plans tree shows plans/Plan-*/Epic-*/Task-*.md without dumping vault into context (Plan-automation-foundation.md:222), Start → Convoy without CLI.
• Non-code: docs/ second-brain + book/course/research pipelines runnable as doc_review→published without git worktree; allowlist per-harness; personal tasks status:ready pillable by any agent via claim_task.
• Dogfooded end-to-end via claim_task → complete_task with harness+model fields.
• Documented in ADRs (adr-model-routing.md + new adr-harness-routing.md / this decision log); .env.example updated.

Scope and boundaries

In scope (full plan)

• Twelve epics — see Epic roadmap below (engine + owner cockpit).
• Epics 1–6 done; Epic 7 in progress; Epics 8–12 draft/ready with bead-scale tasks.
• Autensa benchmark follow-ons remain mapped under 1, 3, 5–9.
• ADR at docs/adr-model-routing.md.

Out of scope (separate future plans)

• Agent-pipeline Phase 2b deliverable templates.
• Agent-pipeline Phase 4 drift/rollup workers (replace GHA cron).
• LiteLLM deployment guide (operators use OpenAI-compatible alias).
• Symphony adoption.
• Autensa research → swipe ideation.

Cross-team collaborators

Name / group	Role
Solo operator	Product, implement, dogfood pipeline

Epic roadmap (updated 2026-08-25 — second brain OS)

Current focus (v0.1, per pivot): MCP create-backlog-item (0.5d) → make worker non-code optional worktree + per-harness allowlist (1d) → harness-agnostic harness registry + WorkerAdapter (2-3d) → Plans tree Epic 10 (1w) → Fleet rollup Epic 9 (3-4d) → then close Epic 7 draft PR + live E2E and Epic 8 webhooks. Follow-ons on done epics can run in parallel when they do not block the critical path.

Legacy order (pre-pivot, kept for history): close Epic 7 (draft PR + live E2E), then Epic 8 → 9 → 10 → 11 → 12.

flowchart LR
  E1[1 Model registry]
  E2[2 Convoy scaffolding]
  E3[3 Connected repos]
  E4[4 Convoy lifecycle MCP]
  E5[5 Gates UI]
  E6[6 Orchestrator]
  E7[7 Worker v1]
  E8[8 Forge webhooks]
  E9[9 Fleet dashboard]
  E10[10 Plans roadmap UI]
  E11[11 Completion artifacts]
  E12[12 Ops copilot]

  E1 --> E6
  E1 --> E7
  E2 --> E4
  E3 --> E6
  E4 --> E5
  E4 --> E6
  E5 --> E6
  E6 --> E7
  E7 --> E8
  E5 --> E9
  E4 --> E9
  E4 --> E10
  E5 --> E10
  E8 --> E11
  E9 --> E12
  E10 --> E12

#	Epic	Status	Delivers	Depends on
1	Model provider registry	done (+1 follow-on)	resolveModel(), Settings AI; cost caps task	—
2	Pipeline convoy scaffolding	done	.convoys/, metrics, planning rule	—
3	Connected repos	done (+1 follow-on)	repo_links, forge adapters; preflight	—
4	Convoy lifecycle MCP	done	create/transition/events	2, 3
5	Mission control gates UI	done (+2 follow-ons)	Kanban, gates, co-pilot; DAG, tiers	4
6	Orchestrator	done (+1 follow-on)	Redis dispatch; auto-nudge	1, 3, 4
7	Worker v1	in progress → v0.1: add optional worktree + per-harness allowlist + draft PR	Loop, checkpoints, draft PR, live E2E, notes; doc_review→published	1, 3, 6
7b	Harness-agnostic (new)	planned (v0.1)	harness_providers + role_harness_routes, `WorkerAdapter(minimal	crewai
8	Forge webhooks	draft (post-v0.1)	Webhooks + PR checks recovery	3, 4, 7
9	Fleet dashboard	draft → next after harnesses (v0.1)	Cross-repo home + SSE feed (/fleet)	4, 5
10	Plans roadmap UI	draft → next after harnesses (v0.1)	Tree, status, Start → convoy (web plans/ CRUD)	4, 5
11	Owner completion artifacts	draft	Summaries, gallery, screenshots	7, 8
12	Ops copilot	draft	Project-aware status chat	9, 10

Overlap with existing plans

Existing artifact	How this roadmap relates
Plan-agent-coordination	Epics 1 (logging) and 4–7 extend it; claim_task / complete_task stay canonical
Epic-optional-orchestrator	Superseded in intent by Epic 6 here — richer spec, Echodo-native, no Symphony dependency
Agent-pipeline v0.4 Phase 2a	Epic 4 implements the MCP tools described there
Agent-pipeline v0.4 Phase 4 workers	Out of scope until Epic 8+; drift/rollup is a future Plan, not this roadmap

End-state (what “done” looks like for the full plan)

Owner opens fleet → sees gates/PRs/releases across projects → opens a repo Plans roadmap → Start a ready task → Mission Control convoy → Go → worker (local or server models) builds without Cursor → draft PR → approve gates → completion summary + screenshots/demos → ask ops copilot what needs attention. plans/ stays git SoT; DB + UI are how owners and agents query without dumping whole trees into context.

Autensa benchmark (2026-08)

Studied crshdn/mission-control (Autensa) as a UX/reliability benchmark — not as a runtime dependency. Ideas mapped to tasks:

Autensa concept	Echodo task	Epic
Checkpoint & crash recovery	Task-worker-checkpoint-resume	7
Operator chat mid-build	Task-worker-operator-notes	7
Agent health / auto-nudge	Task-orchestrator-auto-nudge	6
Repo setup preflight	Task-repo-setup-preflight	3
PR checks recovery	Task-pr-checks-recovery-panel	8
Convoy DAG visualization	Task-convoy-dag-visualization	5
Automation tiers	Task-automation-tiers	5
Cost tracking & budget caps	Task-cost-budget-ui-and-caps	1
Live activity feed (SSE)	Task-convoy-activity-feed-sse	9
Fleet rollup	Task-fleet-rollup-dashboard	9

Explicitly not adopted: OpenClaw Gateway, autopilot research/ideation/swipe, preference learning, agent skills learner, SQLite single-tenant model.

Epics (links)

Epic	Link
1 Model provider registry	./Epic-model-provider-registry/Epic-model-provider-registry.md
2 Pipeline convoy scaffolding	./Epic-pipeline-convoy-scaffolding/Epic-pipeline-convoy-scaffolding.md
3 Connected repos	./Epic-connected-repos/Epic-connected-repos.md
4 Convoy lifecycle MCP	./Epic-convoy-lifecycle-mcp/Epic-convoy-lifecycle-mcp.md
5 Mission control gates UI	./Epic-mission-control-gates-ui/Epic-mission-control-gates-ui.md
6 Orchestrator	./Epic-apps-orchestrator/Epic-apps-orchestrator.md
7 Worker v1	./Epic-apps-worker-v1/Epic-apps-worker-v1.md
8 Forge webhooks	./Epic-forge-webhooks/Epic-forge-webhooks.md
9 Fleet dashboard	./Epic-fleet-dashboard/Epic-fleet-dashboard.md
10 Plans roadmap UI	./Epic-plans-roadmap-ui/Epic-plans-roadmap-ui.md
11 Owner completion artifacts	./Epic-owner-completion-artifacts/Epic-owner-completion-artifacts.md
12 Ops copilot	./Epic-ops-copilot/Epic-ops-copilot.md

Metrics and milestones — updated 2026-08-25

Milestone	Epic	Metric
M0 ADR	—	docs/adr-model-routing.md merged
M1 Model registry	1	Settings: configure + test Anthropic + Ollama aliases
M1b Harness registry	7b	Settings → Harnesses CRUD + testHarness + resolveHarnessForRole
M2 Dogfood loop	1–2	One task via claim_task → complete_task with model + harness fields
M2b Author via MCP	7b	create-backlog-item creates Task-*.md from OpenCode natural language
M3 Repos connected	3	Paste Forge/GitHub URL; clone succeeds from Echodo
M4 Convoy state	4	transition_convoy_status enforces gate actors
M5 Human surface	5	Gate inbox shows pending convoy_review
M6 Automation	6–7	Unattended implementer brief (live E2E task) — now optional-worktree for docs
M6b Draft PR	7	Worker opens draft PR on success (code) / published on docs
M7 PR loop	8	Merge webhook moves convoy to merged
M8 Fleet	9	/fleet single view across ≥2 connected repos (v0.1)
M9 Reliability	7	Checkpoint resume after kill
M10 Observability	9	Activity feed shows role_complete without refresh
M11 Spend control	1 follow-on	Month-to-date spend + dispatch pause on cap
M12 Roadmap UI	10	Plans tree + Start → convoy in-web, no CLI (v0.1)
M13 Owner package	11	Completion summary (+ screenshot when UI)
M14 Ops chat	12	“What needs me?” answered with links
M15 Open publish	—	docker compose -f docker/docker-compose.yml up star-ready, BYO LLM headline

Recommended execution order — updated 2026-08-25 (v0.1 second brain OS first)

v0.1 — Publishable second brain (next 2 weeks, per docs/decisions/2026-08-25-second-brain-os-pivot.md)

1. MCP create-backlog-item (0.5d) — apps/mcp-server/src/tools/create-backlog-item.ts — agents author Plan/Epic/Task naturally
2. Worker non-code (1d) — apps/worker/src/git/worktree.ts:1 optional worktree + apps/worker/src/tools/allowlist.ts:1 per-harness + doc_review→published packages/shared/src/convoy-lifecycle/state-machine.ts
3. Harness registry + WorkerAdapter (2-3d) — packages/database/src/schema/harness_providers.ts + role_harness_routes + packages/shared/src/worker/executor.ts + Settings → Harnesses + AG-UI /api/ag-ui/[workspace] + testHarness
4. Plans tree (1w) — Epic 10 Task-plans-roadmap-page + Task-trpc-backlog-tree + Task-start-convoy-from-task → apps/web/app/(app)/[workspaceSlug]/plans/page.tsx — web CRUD for plans/ without CLI
5. Fleet rollup (3-4d) — Epic 9 Task-fleet-rollup-dashboard + SSE → /fleet

Then close original A1: Task-worker-draft-pr + Task-worker-live-e2e-dogfood (now harness-aware).

Post-v0.1 — legacy engine order (kept for traceability)

6. Task-forge-webhook-receiver → Task-pr-checks-recovery-panel (Epic 8)
7. Epic 11: completion summary → artifact gallery → screenshot capture
8. Epic 12: context tools → UI → draft work from chat

Polish (anytime after blockers clear)

• Epic 5: DAG viz, automation tiers (tiers after Epic 8)
• Epic 1: cost budget caps
• Epic 7: operator notes if not done in Phase A

Risks and mitigations

Risk	Mitigation
Mac Ollama unreachable from Coolify-hosted web	testProvider surfaces error; document LAN URL; dev can run web locally
Cursor slug drift in old convoys	Legacy alias map in ADR; conductor uses registry aliases for new convoys
Scope creep into orchestrator	ADR + plan explicitly defer daemon/worker

Links

• ADR: Model routing
• Decision 2026-08-25 second brain pivot
• Plan: Agent coordination
• Agent-pipeline: docs/model-routing-policy.md, docs/role-reference.md
• Examples: Maya Clarity + Studio North RouteMap walkthroughs in decision log