86155455a7
- Draft PR via forge (apps/worker/src/git/pr.ts) push branch + createPullRequest draft with echodo-convoy marker, expose prUrl in Mission Control (convoy-service, convoys router, detail panel) - Operator notes mid-build: convoy_operator_messages (0016), tRPC post/list, worker poll between tool rounds (queued) and before generateText (direct), detail panel thread for in_progress - Doc-writer: harness doc-writer examples (docs/examples/research-pipeline.md), SOP template (docs/templates/doc-sop.md), role-doc-writer wiring Convoy: .convoys/worker-closeout-docwriter Co-authored-by: orchestrator
72 lines
4.2 KiB
Markdown
72 lines
4.2 KiB
Markdown
# Worker (`apps/worker`)
|
|
|
|
Minimal first-party agent runtime for convoy automation. Consumes orchestrator jobs from Redis, resolves models via the workspace registry, runs a tool loop against a git worktree, and records `claim_task` / `complete_task` + `convoy_events.role_complete`.
|
|
|
|
## Run locally
|
|
|
|
```bash
|
|
# Dry-run (no Redis consumer — exits after banner)
|
|
WORKER_DRY_RUN=1 pnpm --filter @tasks/worker dev
|
|
|
|
# Live consumer (requires REDIS_URL + DATABASE_URL)
|
|
pnpm --filter @tasks/worker dev
|
|
```
|
|
|
|
Pair with the orchestrator:
|
|
|
|
```bash
|
|
ORCHESTRATOR_DRY_RUN=0 REDIS_URL=... pnpm --filter @tasks/orchestrator dev
|
|
WORKER_DRY_RUN=0 REDIS_URL=... pnpm --filter @tasks/worker dev
|
|
```
|
|
|
|
## Architecture
|
|
|
|
- **Queue** — `BLPOP` on `echodo:orchestrator:jobs` (same key as `apps/orchestrator`)
|
|
- **Session** — `WorkerSession` interface in `@tasks/shared/worker`; v1 implementation uses Vercel AI SDK `generateText` + tools
|
|
- **Tools** — `read_file`, `write_file`, `grep` (ripgrep), `run_command` (allowlist per harness: code harness `pnpm lint`, `pnpm type-check`, `pnpm test`, `git status`, `git diff`; doc harness `[]` — no `run_command`, only `read`/`write`/`grep`)
|
|
- **Worktree** — one git worktree per brief job under `WORKER_WORKTREE_BASE` when `harness.needsWorktree === true` (code tasks). Doc tasks (`harness: doc-writer`) set `needsWorktree: false` → worker skips `ensureWorktree` and runs straight against `monorepoRoot` (`docs/`/`plans/` writes without a branch). See `apps/worker/src/harness.ts` and `apps/worker/src/git/worktree.ts` `needsWorktree` param.
|
|
- **Lifecycle** — DB writes mirroring MCP `claim_task` / `complete_task`; `role_complete` convoy event on finish (includes `metadata.harnessAlias`)
|
|
- **Harnesses** — `resolveHarnessForJob` maps `job.harnessAlias` or `roleId` to `{ alias, needsWorktree, allowedCommands }`; `allowedCommands` is threaded to `runCommandTool` via `MinimalSession`. Brief 3 will promote this to DB-backed `harness_providers` / `role_harness_routes`.
|
|
|
|
### `role-doc-writer` via `doc-writer` example
|
|
|
|
```ts
|
|
// DB seed (or via tRPC harnesses.create / upsertRoute)
|
|
await harnesses.create({ alias: "doc-writer", displayName: "Doc writer", harnessType: "minimal" });
|
|
await harnesses.upsertRoute({ roleId: "role-doc-writer", harnessProviderId: docWriter.id });
|
|
|
|
// Job dispatch
|
|
{ roleId: "role-doc-writer", harnessAlias: "doc-writer" } // resolveHarnessForJob → { alias: "doc-writer", needsWorktree: false, allowedCommands: [] }
|
|
|
|
// Session trace: read_file / grep / write_file only — no run_command, no worktree under /tmp/echodo-worker-worktrees.
|
|
// Gate: in_progress → doc_review (automation or role-doc-writer), then doc_review → published via human-gate-2 / doc-approver.
|
|
```
|
|
|
|
See also `docs/examples/research-pipeline.md` (fixture) and `docs/templates/doc-sop.md` (`kind: doc-sop`).
|
|
|
|
## Env
|
|
|
|
| Variable | Default | Purpose |
|
|
|----------|---------|---------|
|
|
| `DATABASE_URL` | required | Convoy, backlog, model registry |
|
|
| `REDIS_URL` | optional | Job queue; missing ⇒ dry-run |
|
|
| `WORKER_DRY_RUN` | `false` | Skip dequeue loop |
|
|
| `WORKER_BLPOP_TIMEOUT_S` | `5` | Blocking pop timeout |
|
|
| `WORKER_MAX_TOOL_STEPS` | `12` | Max tool rounds per session |
|
|
| `WORKER_CHECKPOINT_EVERY_N_STEPS` | `1` | Persist a checkpoint every N tool rounds |
|
|
| `WORKER_REPO_ROOT` | monorepo root | Repo for role prompts + worktree base |
|
|
| `WORKER_WORKTREE_BASE` | `/tmp/echodo-worker-worktrees` | Worktree parent directory |
|
|
|
|
## Checkpoints & crash recovery
|
|
|
|
After each tool round (or every `WORKER_CHECKPOINT_EVERY_N_STEPS` rounds), the worker writes a row to `worker_checkpoints` (workspace-scoped): bounded `tool_history`, `files_touched` as **paths + content hashes only** (never API keys or full file bodies), plus `branch` / `worktree_path`.
|
|
|
|
On re-dispatch with `resumeFromCheckpoint: true` (orchestrator stall detector), the worker:
|
|
|
|
1. Loads the latest checkpoint for the open `agent_runs` row (filtered by `workspace_id`)
|
|
2. Reuses the saved worktree when still present — no re-clone
|
|
3. Continues the tool loop with prior steps summarized in the system prompt
|
|
|
|
Mission control convoy detail shows checkpoint count and last checkpoint time.
|
|
|
|
Provider API keys are read from env vars named in `model_providers.api_key_env_var` (same as web settings).
|