6ccb060bea
- MCP create-backlog-item for natural language Plan/Epic/Task authoring - Worker non-code: optional worktree + per-harness allowlist + doc_review→published - Harness registry: harness_providers/role_harness_routes, WorkerAdapter (minimal|crewai|langgraph|remote-ag-ui), AG-UI POST /api/ag-ui, Settings → Harnesses with Test - Plans tree: backlog.listTree + createPlan/Epic/Task + Start→Convoy, web CRUD without CLI - Fleet: /fleet rollup across workspaces + SSE /api/fleet/events Decision: docs/decisions/2026-08-25-second-brain-os-pivot.md Convoy: .convoys/second-brain-os-v01 Co-authored-by: orchestrator
56 lines
3.4 KiB
Markdown
56 lines
3.4 KiB
Markdown
# Worker (`apps/worker`)
|
|
|
|
Minimal first-party agent runtime for convoy automation. Consumes orchestrator jobs from Redis, resolves models via the workspace registry, runs a tool loop against a git worktree, and records `claim_task` / `complete_task` + `convoy_events.role_complete`.
|
|
|
|
## Run locally
|
|
|
|
```bash
|
|
# Dry-run (no Redis consumer — exits after banner)
|
|
WORKER_DRY_RUN=1 pnpm --filter @tasks/worker dev
|
|
|
|
# Live consumer (requires REDIS_URL + DATABASE_URL)
|
|
pnpm --filter @tasks/worker dev
|
|
```
|
|
|
|
Pair with the orchestrator:
|
|
|
|
```bash
|
|
ORCHESTRATOR_DRY_RUN=0 REDIS_URL=... pnpm --filter @tasks/orchestrator dev
|
|
WORKER_DRY_RUN=0 REDIS_URL=... pnpm --filter @tasks/worker dev
|
|
```
|
|
|
|
## Architecture
|
|
|
|
- **Queue** — `BLPOP` on `echodo:orchestrator:jobs` (same key as `apps/orchestrator`)
|
|
- **Session** — `WorkerSession` interface in `@tasks/shared/worker`; v1 implementation uses Vercel AI SDK `generateText` + tools
|
|
- **Tools** — `read_file`, `write_file`, `grep` (ripgrep), `run_command` (allowlist per harness: code harness `pnpm lint`, `pnpm type-check`, `pnpm test`, `git status`, `git diff`; doc harness `[]` — no `run_command`, only `read`/`write`/`grep`)
|
|
- **Worktree** — one git worktree per brief job under `WORKER_WORKTREE_BASE` when `harness.needsWorktree === true` (code tasks). Doc tasks (`harness: doc-writer`) set `needsWorktree: false` → worker skips `ensureWorktree` and runs straight against `monorepoRoot` (`docs/`/`plans/` writes without a branch). See `apps/worker/src/harness.ts` and `apps/worker/src/git/worktree.ts` `needsWorktree` param.
|
|
- **Lifecycle** — DB writes mirroring MCP `claim_task` / `complete_task`; `role_complete` convoy event on finish (includes `metadata.harnessAlias`)
|
|
- **Harnesses** — `resolveHarnessForJob` maps `job.harnessAlias` or `roleId` to `{ alias, needsWorktree, allowedCommands }`; `allowedCommands` is threaded to `runCommandTool` via `MinimalSession`. Brief 3 will promote this to DB-backed `harness_providers` / `role_harness_routes`.
|
|
|
|
## Env
|
|
|
|
| Variable | Default | Purpose |
|
|
|----------|---------|---------|
|
|
| `DATABASE_URL` | required | Convoy, backlog, model registry |
|
|
| `REDIS_URL` | optional | Job queue; missing ⇒ dry-run |
|
|
| `WORKER_DRY_RUN` | `false` | Skip dequeue loop |
|
|
| `WORKER_BLPOP_TIMEOUT_S` | `5` | Blocking pop timeout |
|
|
| `WORKER_MAX_TOOL_STEPS` | `12` | Max tool rounds per session |
|
|
| `WORKER_CHECKPOINT_EVERY_N_STEPS` | `1` | Persist a checkpoint every N tool rounds |
|
|
| `WORKER_REPO_ROOT` | monorepo root | Repo for role prompts + worktree base |
|
|
| `WORKER_WORKTREE_BASE` | `/tmp/echodo-worker-worktrees` | Worktree parent directory |
|
|
|
|
## Checkpoints & crash recovery
|
|
|
|
After each tool round (or every `WORKER_CHECKPOINT_EVERY_N_STEPS` rounds), the worker writes a row to `worker_checkpoints` (workspace-scoped): bounded `tool_history`, `files_touched` as **paths + content hashes only** (never API keys or full file bodies), plus `branch` / `worktree_path`.
|
|
|
|
On re-dispatch with `resumeFromCheckpoint: true` (orchestrator stall detector), the worker:
|
|
|
|
1. Loads the latest checkpoint for the open `agent_runs` row (filtered by `workspace_id`)
|
|
2. Reuses the saved worktree when still present — no re-clone
|
|
3. Continues the tool loop with prior steps summarized in the system prompt
|
|
|
|
Mission control convoy detail shows checkpoint count and last checkpoint time.
|
|
|
|
Provider API keys are read from env vars named in `model_providers.api_key_env_var` (same as web settings).
|