rstillwell/Submatter
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
Submatter/docs/decisions/2026-08-25-second-brain-os-pivot.md
Randall Stillwell 6ccb060bea feat(second-brain): open self-hosted Obsidian for crews — harness-agnostic v0.1 
- MCP create-backlog-item for natural language Plan/Epic/Task authoring
- Worker non-code: optional worktree + per-harness allowlist + doc_review→published
- Harness registry: harness_providers/role_harness_routes, WorkerAdapter (minimal|crewai|langgraph|remote-ag-ui), AG-UI POST /api/ag-ui, Settings → Harnesses with Test
- Plans tree: backlog.listTree + createPlan/Epic/Task + Start→Convoy, web CRUD without CLI
- Fleet: /fleet rollup across workspaces + SSE /api/fleet/events

Decision: docs/decisions/2026-08-25-second-brain-os-pivot.md
Convoy: .convoys/second-brain-os-v01

Co-authored-by: orchestrator
2026-08-25 16:20:53 -05:00

7.1 KiB
Raw Blame History

Decision Log 2026-08-25 — Second Brain OS Pivot

Date: 2026-08-25
Participants: solo operator + deep-dive analysis of Echodo (tasks repo) vs Grok Bot / OpenBot / Agents Squads / Agent Teams / OpenClaw
Status: Accepted
Scope: Product positioning, roadmap, and harness architecture for remaining epics.

TL;DR

Echodo stays homelab dogfooded, publishes as open MIT self-hosted "Obsidian for agent crews" — docker compose up, BYO LLMs (Ollama 11434 / MLX 8000 / NIM / LiteLLM), any harness via AG-UI/MCP. Do not chase Grok Bot SaaS ($300/mo Heavy) or clone LangChain/CrewAI/AutoGen frameworks. Win by being the file-first, gated control plane they lack.

Context

Full deep dive 2026-08-25 covered:

  • What Echodo is: tasks repo package.json:1, @tasks/*, DB/domain echodo, plans/Plan-*/Epic-*/Task-*.md git SoT → markdown_backlog_items + cursor_sync_mappings → tRPC 19 routers + MCP claim_task/complete_task + model_providers/role_model_routes + convoy state machine + orchestrator Redis tick + worker worktree+MinimalSession.
  • Current status: Epics 1-6 done, 7 in_progress (draft PR + live E2E missing), 8-12 draft. 14 migrations, apps/worker 4-tool allowlist, uncommitted checkpoint/worktree code.
  • Market: $6-17B orchestration 2026 → $38-121B 2035, 23-25% CAGR. Human-in-loop hybrid 18% ($1B). Grok Bot has no standalone price (bundled Heavy/Cursor Ultra). OpenClaw 347k stars > React in 60 days. LangChain 97-135k, CrewAI 57k, AutoGen maintenance → Microsoft Agent Framework.

Examples walked: solo Maya Clarity offline notes + local tags (M4 Mac); small team Studio North RouteMap delivery planner (CrewAI crew + Nim). Both driven from web Mission control apps/web/app/(app)/[workspaceSlug]/mission-control/page.tsx:12 or via MCP claim_task in Cursor/OpenCode.

Decisions Made

1. Positioning: Open self-hosted second brain, not managed SaaS

  • Internal tool first: Run for self, fleet of personal projects, dogfood every bead via claim_task → complete_task.
  • Publish as open self-hosted Obsidian for agent crews: git clone && cp .env.example .env && docker compose -f docker/docker-compose.yml up must work without CT102 IPs. BYO LLM headline.
  • Monetize later as managed fleet (SSE, copilot hosting) if >1k stars or 50 self-hosters or 3 agencies in 30 days post-publish. Not per-seat SaaS.

Rejected: competing head-on with Grok Bot appliance; cloning LangChain/CrewAI (223M dl/mo ecosystem tax, No native policy gate gap already exists to fill).

2. Harness-agnostic execution (the big change)

  • Add WorkerAdapter interface packages/shared/src/worker/executor.ts — handleOrchestratorJob apps/worker/src/handle-job.ts:40 dispatches to minimal | crewai | langgraph | remote-ag-ui (AG-UI).
  • Add harness_providers + role_harness_routes (mirror model_providers docs/adr-model-routing.md:78), harness_type enum, auth_env_var write-only like OpenBot.
  • Job spec packages/shared/src/orchestrator/job.ts adds harnessAlias. AG-UI endpoint apps/web/app/api/ag-ui/[workspace]/route.ts; MCP stays apps/mcp-server/src/index.ts:12.
  • One agent can author tasks (doc-writer), another harness implements them — role-architect → claude-planning + langgraph, role-implementer → qwen-coder-mlx http://mac:8000/v1 + crewai.

3. Web-first, almost everything from UI

Goal: never leave echodo.stillwell.cloud.

  • Fleet: new /fleet rollup across workspace_id + connected_repos + convoy_events SSE (Epic 9).
  • Plans: tree CRUD for plans/*.md packages/database/src/markdown-backlog/sync.ts:38 via backlog.createPlan/createEpic/createTask + Start → Convoy (Epic 10). Replaces hand markdown + pnpm import:markdown-backlog loop.
  • Run: CreateConvoyDialog apps/web/components/mission-control/create-convoy-dialog.tsx → convoys.create apps/web/server/routers/convoys.ts:173 → orchestrator fans out; gates gateAction convoys.ts:237 + gateCopilot convoys.ts:267 in same panel; history agent_runs/worker_checkpoints.
  • MCP remains shortcut for natural language ("OpenCode, add Epic with 3 tasks") via new create-backlog-item tool.

4. Non-code support (book/course/research/SOPs)

  • Make worktree optional apps/worker/src/git/worktree.ts:1 — doc tasks write straight to docs//plans/ no branch.
  • Make allowlist.ts per-harness — research harness allows search, doc harness allows no pnpm.
  • Add doc_review→published transition alongside pr_review→merged packages/shared/src/convoy-lifecycle/state-machine.ts.
  • Templates Settings → Templates apps/web/app/(app)/[workspaceSlug]/settings/templates/page.tsx for research pipeline/book without migration.

5. What stays vs. what goes

Keep: workspace_id tenancy packages/database/src/schema/objects.ts:32, plans/ hierarchy docs/Glossary.md:50, markdown contentHash idempotency, claim_task/complete_task primitives Plan-agent-coordination.md:12, model routing resolveModel docs/adr-model-routing.md:63, gates/kanban apps/web/app/(app)/[workspaceSlug]/mission-control/page.tsx:12.

Add: harness registry, Plans tree, Fleet rollup, create-backlog-item.

Not building now: full Cursor API push/pull (config/CursorSync.md:5 modes doc-only), LiteLLM required, per-user model overrides adr-model-routing.md:354.

Examples Consolidated

  • Maya solo Clarity: starts from web Plans "local tag prompt Ready" → Start→Convoy clarity-tag-v1 → gate convoy_review → approve → worker qwen-coder + tag-crew worktree → checkpoint toolHistory → draft PR feat(tags) → pr_review → merged. Or same via Cursor claim_task repoPath:plans/... natural language.
  • Studio North RouteMap: Ana triages Fleet → sees routemap-optimize-v2 pr_review 2d Carlos, creates Epic-driver-offline in Plans; Ben claims via Cursor; Carlos gate Request changes via Gate co-pilot summary. Ops copilot answers "what needs me?".

Both show markdown as token-cheap second brain Plan-automation-foundation.md:222 "DB+UI are how agents query without dumping whole trees" — not Obsidian vault scan.

Roadmap Impact

Updates plans/Plan-automation-foundation/Plan-automation-foundation.md — see "2026-08-25 pivot" section. Priorities v0.1 for second-brain OS: (1) MCP create item (0.5d), (2) optional worktree + per-harness allowlist (1d), (3) harness registry + WorkerAdapter (2-3d), (4) Plans tree (1w), (5) Fleet rollup. Epics 9/10 remain draft but now unblock non-code. Epic 11/12 unchanged.

Go/no-go: 30 days post-open: >1k stars / 50 self-hosters / 3 managed inquiries else keep homelab-only.

Risks

  • Mac Ollama unreachable from Coolify → testProvider/testHarness surface LAN error; run web locally fallback docs/adr-model-routing.md:314.
  • Scope creep into framework — mitigate WorkerAdapter is adapter, not framework.

Links

  • Plan: plans/Plan-automation-foundation/Plan-automation-foundation.md
  • ADR: docs/adr-model-routing.md
  • Convoy lifecycle: docs/convoy-lifecycle.md
  • Worker: docs/worker.md
  • Plans layout: plans/README.md, docs/Glossary.md, config/CursorSync.md