6ccb060bea
- MCP create-backlog-item for natural language Plan/Epic/Task authoring - Worker non-code: optional worktree + per-harness allowlist + doc_review→published - Harness registry: harness_providers/role_harness_routes, WorkerAdapter (minimal|crewai|langgraph|remote-ag-ui), AG-UI POST /api/ag-ui, Settings → Harnesses with Test - Plans tree: backlog.listTree + createPlan/Epic/Task + Start→Convoy, web CRUD without CLI - Fleet: /fleet rollup across workspaces + SSE /api/fleet/events Decision: docs/decisions/2026-08-25-second-brain-os-pivot.md Convoy: .convoys/second-brain-os-v01 Co-authored-by: orchestrator
66 lines
2 KiB
Markdown
66 lines
2 KiB
Markdown
---
|
|
kind: task
|
|
slug: worker-checkpoint-resume
|
|
title: Worker checkpoint and crash recovery
|
|
plan_slug: automation-foundation
|
|
epic_slug: apps-worker-v1
|
|
status: done
|
|
priority: P1
|
|
tenant_id: global
|
|
owner: unassigned
|
|
cursor_todo_id: null
|
|
updated_at: "2026-08-12"
|
|
---
|
|
|
|
# Task summary
|
|
|
|
Persist worker session progress at configurable checkpoints so a crashed or stalled job resumes from the last checkpoint instead of restarting the brief from scratch.
|
|
|
|
## Description
|
|
|
|
**Benchmark:** [Autensa mission-control](https://github.com/crshdn/mission-control) checkpoint & crash recovery — study their checkpoint restore API and per-task history UX; implement Echodo-native equivalents on Postgres.
|
|
|
|
### Schema
|
|
|
|
Add `worker_checkpoints` (or JSONB on `agent_runs`) workspace-scoped:
|
|
|
|
- `agent_run_id`, `convoy_id`, `brief_id`
|
|
- `checkpoint_seq`, `tool_history` (bounded), `files_touched`, `branch`, `worktree_path`
|
|
- `occurred_at`
|
|
|
|
### Worker behavior
|
|
|
|
- Save checkpoint after each tool round (configurable via `WORKER_CHECKPOINT_EVERY_N_STEPS`, default 1).
|
|
- On job start: if open run has latest checkpoint, resume tool loop from saved state.
|
|
- Orchestrator stall detector can trigger re-dispatch with `resume_from_checkpoint: true`.
|
|
|
|
### Operator surface (minimal v1)
|
|
|
|
- Mission control detail panel: checkpoint count + last checkpoint time (full history deferred).
|
|
|
|
## Dependencies
|
|
|
|
- [Task-worker-minimal-tool-loop](./Task-worker-minimal-tool-loop.md) (baseline loop must exist).
|
|
|
|
## Subtasks
|
|
|
|
- [x] Zod schemas in `packages/shared/worker`
|
|
- [x] Drizzle migration + workspace-scoped queries
|
|
- [x] Checkpoint write in `minimal-session` / `handle-job`
|
|
- [x] Resume path on re-dispatch
|
|
- [x] Env docs in `docs/worker.md`
|
|
|
|
## Estimation
|
|
|
|
M
|
|
|
|
## Acceptance criteria
|
|
|
|
- [x] Kill worker mid-brief; re-dispatch resumes from last tool step without re-cloning
|
|
- [x] Checkpoints filtered by `workspace_id`
|
|
- [x] Checkpoint payload does not store API keys or full file contents (paths + hashes only)
|
|
|
|
## Links
|
|
|
|
- Epic: `./Epic-apps-worker-v1.md`
|
|
- Blocks: reliable unattended dogfood for Epic 8
|