ubiquitous-invention/plans/Plan-agent-coordination/Epic-task-as-runnable-unit/Task-export-db-to-markdown-frontmatter.md

82 lines
5 KiB
Markdown
Raw Normal View History

---
kind: task
slug: export-db-to-markdown-frontmatter
title: Export DB state back to markdown frontmatter (close the sync loop)
plan_slug: agent-coordination
epic_slug: task-as-runnable-unit
feat(markdown-backlog): close the sync loop with DB → frontmatter export Until now the markdown importer was one-way (plans/*.md → DB). Any agent-driven status flip via claim_task / complete_task would be clobbered on the next importer sweep. This change closes the loop: the DB now projects status, priority, agent_prompt, and updated_at back into the file's frontmatter, preserving body bytes, key order, and every other frontmatter key. New: packages/database/src/markdown-backlog/export.ts - `rewriteFrontmatter()` — pure function, covered by 10 Vitest cases (round-trip identity, status flip, priority flip, agent_prompt null/block-scalar/single-line variants, body preservation, trailing-newline preservation, idempotent re-application). - `exportBacklogItemToMarkdown()` — DB-loading wrapper with atomic write (tmp + rename) and tenant fencing. Returns a structured result so callers can surface what happened in their response. Wired into: - `claim_task` MCP tool — exports on the ready → in_progress flip. - `complete_task` MCP tool — exports on any finalStatus transition. - `backlog.updateWorkflowPrompt` tRPC mutation — exports on prompt edits made through the app UI. Robust repo-root resolution (`apps/{mcp-server,web}/src/lib/repo-root.ts`, plus a copy in `import-markdown-backlog.ts`): walk up from the source file looking for `pnpm-workspace.yaml`, falling back to env var or cwd. This fixes a class of bug where `pnpm --filter <pkg>` cd's into the package directory and breaks naive cwd-based path resolution — the importer was deleting all 44 rows during smoke testing before this fix because it found zero files in `packages/database/plans/`. `config/CursorSync.md`: documents the new two-way contract, the DB-wins-on-allow-list conflict policy, and the MARKDOWN_BACKLOG_REPO_ROOT=off escape hatch for production deployments where `plans/` isn't checked out. Smoke verified end-to-end against the homelab DB: claim flips file status to in_progress, complete flips it back to ready, importer round-trips with stable content_hash (true no-op), agent identity preserved throughout. Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-03 11:21:22 -04:00
status: done
priority: P1
tenant_id: global
owner: unassigned
cursor_todo_id: null
updated_at: "2026-06-03"
# agent_prompt: |
# Treat this task as a correctness-critical change to the sync layer. The
# importer writes one direction (md → DB); after this task the system MUST
# also write the other direction (DB → md frontmatter) so MCP-driven status
# transitions survive the next importer pass. Be careful with diff noise,
# trailing newlines, and YAML key ordering — the markdown files are
# human-authored.
---
# Task summary
The markdown importer at `packages/database/src/markdown-backlog/sync.ts` is one-way: it reads `plans/**/*.md` and upserts into `markdown_backlog_items`. There is no path back from the DB to the markdown frontmatter. As soon as an agent calls `complete_task` (status → `done`), running the importer would clobber that transition back to whatever the markdown still says.
For now we paper over this by running the importer once at bootstrap and ad-hoc thereafter (`pnpm --filter @tasks/database import:markdown-backlog`). That's tolerable for a single-operator dogfood loop, but it doesn't scale and it's a correctness bug: any change made through the app — status flips, prompt edits, new tasks created in the UI — is at risk of being silently reverted.
This task closes that loop by adding a DB → markdown export.
## Description
### Scope
Implement `exportBacklogItemToMarkdown(db, { workspaceId, backlogItemId })` that:
1. Loads the `markdown_backlog_items` row.
2. Locates the corresponding `.md` file on disk using `repo_relative_path`.
3. Reads the existing file (must preserve the body verbatim).
4. Rewrites the YAML frontmatter so the canonical fields match the DB row:
- `status`
- `priority`
- `title` (only when changed; titles are usually human-authored)
- `updated_at` (bumped to today on any rewrite)
- `workflow_prompt` → frontmatter `agent_prompt:` (omit if null; serialize as a block scalar `|` if multi-line)
5. Preserves every other frontmatter key untouched, in the original order.
6. Writes the file back atomically (write to tmp, rename) to avoid half-written reads from any watcher.
Wire it into:
- `complete_task` (after the status transition commits): export the touched row.
- `claim_task`: optional — probably yes when status flipped to `in_progress`.
- `backlog.updateWorkflowPrompt` tRPC mutation (`apps/web/server/routers/backlog.ts`).
- Maybe `workspaces.archive` cascade for backlog items (lower priority — archive is a UI-driven verb and the markdown may not need to reflect it).
Out of scope:
- Creating brand-new `.md` files from DB rows (we never UI-create backlog items today; punt to a separate task).
- Two-way conflict resolution. If someone edits both the file and the DB row between importer runs we just take the DB. Document this clearly in `config/CursorSync.md`.
### Implementation notes
- Use a small, well-tested YAML rewriter, not a regex. The `yaml` package (already a dep of `@tasks/database`) supports preserving comments and key order via `Document` parsing. Use `Document.parse(source)`, mutate scalar values in place, and `String(doc)` to serialize.
- File I/O lives in `packages/database/src/markdown-backlog/export.ts`. Keep this module free of tRPC / Next imports so the MCP server can call it directly.
- Add Vitest cases covering:
- Round-trip: read → no change → write produces identical bytes.
- Status update: `ready``done` only mutates `status` and `updated_at`.
- Setting `agent_prompt: null` removes the key (does not write `agent_prompt: null`).
- Block-scalar serialization for multi-line prompts.
- Preserves trailing newline and body separator (`---\n\n` vs `---\n`).
- Behind a workspace-level setting `markdown_export_enabled` (default `true`). Some operators may not want the app rewriting their `plans/` tree; let them opt out.
### Acceptance
1. After calling `complete_task` with `finalStatus: done`, the corresponding `.md` file's frontmatter status reads `done` on disk.
2. Running `pnpm --filter @tasks/database import:markdown-backlog` immediately after produces zero new upserts (the file already matches the DB).
3. Vitest coverage of the export helper.
4. `config/CursorSync.md` is updated to describe the new two-way contract and the conflict policy (DB wins on conflict between importer runs).
5. No diff noise: re-exporting an unchanged row leaves the file byte-identical.
### Risks
- YAML formatting is finicky; agents may produce huge accidental diffs the first time this runs. Mitigate with the round-trip test and an `--dry-run` flag on the export helper for first verification.
- Concurrent writes: an operator editing the `.md` in their editor at the moment of export could lose changes. The atomic write helps but doesn't fully solve it. Document the contract: while agents are running, treat the `.md` files as derived state.