ubiquitous-invention/config/CursorSync.md

126 lines
7.9 KiB
Markdown
Raw Permalink Normal View History

# Cursor sync configuration
This document describes how the application and Cursor should keep **plans, epics, and tasks** aligned. Implementation is incremental; treat this as the contract the data layer and jobs will follow.
## Goals
1. **App → Cursor**: Tasks and status updates in the app appear as Cursor to-dos / plan items where configured.
2. **Cursor → App**: To-dos created or completed in Cursor are mirrored into the correct plan/epic in the app.
3. **Markdown as source of truth (optional mode)**: Repo markdown can be authoritative; the app imports on change, or the app exports on change—policy is per tenant (see Modes).
## Multitenancy
- Each **tenant** has its own:
- API credentials or OAuth connection to Cursor (when available).
- Mapping table: internal plan/epic/task id ↔ Cursor identifiers ↔ filesystem paths under `plans/`.
- No cross-tenant sync or shared Cursor workspace.
## Modes (to implement)
| Mode | Behavior |
|------|----------|
| `markdown_authoritative` | Watch `plans/**`; import on save; push summaries to Cursor. |
| `app_authoritative` | UI/API edits win; export markdown + update Cursor on commit or interval. |
| `bidirectional` | Reconcile by `updated_at` and explicit conflict rules (last-write-wins per field or manual resolution). |
feat(markdown-backlog): close the sync loop with DB → frontmatter export Until now the markdown importer was one-way (plans/*.md → DB). Any agent-driven status flip via claim_task / complete_task would be clobbered on the next importer sweep. This change closes the loop: the DB now projects status, priority, agent_prompt, and updated_at back into the file's frontmatter, preserving body bytes, key order, and every other frontmatter key. New: packages/database/src/markdown-backlog/export.ts - `rewriteFrontmatter()` — pure function, covered by 10 Vitest cases (round-trip identity, status flip, priority flip, agent_prompt null/block-scalar/single-line variants, body preservation, trailing-newline preservation, idempotent re-application). - `exportBacklogItemToMarkdown()` — DB-loading wrapper with atomic write (tmp + rename) and tenant fencing. Returns a structured result so callers can surface what happened in their response. Wired into: - `claim_task` MCP tool — exports on the ready → in_progress flip. - `complete_task` MCP tool — exports on any finalStatus transition. - `backlog.updateWorkflowPrompt` tRPC mutation — exports on prompt edits made through the app UI. Robust repo-root resolution (`apps/{mcp-server,web}/src/lib/repo-root.ts`, plus a copy in `import-markdown-backlog.ts`): walk up from the source file looking for `pnpm-workspace.yaml`, falling back to env var or cwd. This fixes a class of bug where `pnpm --filter <pkg>` cd's into the package directory and breaks naive cwd-based path resolution — the importer was deleting all 44 rows during smoke testing before this fix because it found zero files in `packages/database/plans/`. `config/CursorSync.md`: documents the new two-way contract, the DB-wins-on-allow-list conflict policy, and the MARKDOWN_BACKLOG_REPO_ROOT=off escape hatch for production deployments where `plans/` isn't checked out. Smoke verified end-to-end against the homelab DB: claim flips file status to in_progress, complete flips it back to ready, importer round-trips with stable content_hash (true no-op), agent identity preserved throughout. Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-03 11:21:22 -04:00
## Two-way markdown contract (current implementation)
The repo today operates in a **hybrid markdown / DB authoritative** mode that approximates `bidirectional` for the small set of fields agents touch. The contract is:
| Direction | Trigger | Code path | Fields it can rewrite |
|---|---|---|---|
| `plans/**.md` → DB | `pnpm --filter @tasks/database import:markdown-backlog` (one-shot) or the optional `watch:markdown-backlog` script | `packages/database/src/markdown-backlog/sync.ts` | **All** frontmatter fields plus body bytes. The importer overwrites the DB row from the file. |
| DB → `plans/**.md` | `claim_task` (status flip only), `complete_task` (always when `finalStatus` is set), `backlog.updateWorkflowPrompt` tRPC mutation | `packages/database/src/markdown-backlog/export.ts` | A tight allow-list: `status`, `priority`, `agent_prompt`, `updated_at`. **Body bytes and all other frontmatter keys are preserved verbatim.** |
### Conflict policy: DB wins on the allow-list, file wins on everything else
If a human edits `status: ready``status: blocked` in the `.md` between two MCP calls, the next `complete_task` will rewrite it (DB → file). If they edit the body, the title, the `tenant_id`, or any other frontmatter key, the exporter leaves their change in place. The next `import:markdown-backlog` run will then pull those file-side changes back into the DB.
This is a single-operator contract. In a multi-operator setup you'd need a true reconciliation policy with conflict detection (see `Task-export-db-to-markdown-frontmatter` for the deferred design notes); for now it's enough that any one operator picks "edit the file" OR "drive the app" per session.
### Idempotence guarantees the exporter must keep
- **No-diff re-export is a no-op.** Running the export with desired state equal to current file state must produce the original bytes byte-for-byte. The pure rewriter in `export.ts` is covered by 10 Vitest cases for this.
- **`updated_at` only bumps when something else changes.** A "phantom" rewrite that bumps `updated_at` and nothing else is treated as a bug.
- **Atomic writes.** Write to `.tmp` next to the target file, then `rename()`. On POSIX inside the same directory this is atomic — partial reads are impossible.
- **Block-literal preservation for `agent_prompt`.** Multi-line prompts get `agent_prompt: |` so diffs stay readable. Single-line stays plain.
### How to disable the export side
Set `MARKDOWN_BACKLOG_REPO_ROOT=off` (or `0`, or empty string) in the environment of the MCP server or the Next dev server. Useful when:
- You're running the app in production (Coolify) where `plans/` isn't checked out.
- You're testing a destructive change and want to confirm the file write is the cause without rolling back.
When unset, the exporter defaults to `process.cwd()` — which works in dev because Cursor spawns the MCP from the repo root.
## MCP server lifecycle (dev gotcha)
The MCP server is spawned **once per Cursor chat session** via [.cursor/mcp.json](../.cursor/mcp.json) as `pnpm -s --filter @tasks/mcp-server mcp`, which runs `tsx --env-file=../../.env src/index.ts`. It is a long-lived stdio process for the duration of the chat. **It does not auto-reload on source changes.**
Consequences when iterating on agent-side code:
- Editing any file under `apps/mcp-server/**` or any imported library used by a tool (e.g. `packages/database/src/markdown-backlog/export.ts`, `packages/database/src/schema/**`, `packages/database/src/client.ts`) has **no effect on tool calls** until the server is restarted and Cursor reconnects.
- A new Cursor chat does not always spawn a new server. Cursor may bind to an existing stale process, which means you can end up talking to code committed before that process started. Multiple stale servers from prior sessions can accumulate. Symptom: a tool that should return a new field returns the old shape, or status flips happen in the DB but the markdown file doesn't get re-exported.
### How to recover
```bash
# 1. Find stale MCP servers (each pair is pnpm wrapper + tsx child)
pgrep -fa "apps/mcp-server.*src/index.ts"
# 2. Kill them
pkill -f "apps/mcp-server.*src/index.ts"
# 3. Open a new Cursor chat. Cursor will spawn a fresh server against current source.
```
When debugging "did the tool definitely run against the new code?", a useful check is whether the tool's response shape includes a field that only exists in the new version (e.g. the `export` field on `claim_task` / `complete_task` was added in commit `56b697b`). Missing field = stale server.
### Why we don't run `tsx watch`
A watch-mode restart would drop the stdio JSON-RPC connection mid-call, which the MCP client (Cursor) wouldn't gracefully recover from. A real fix would need a graceful-reload protocol or out-of-process tool execution. Documented gotcha is the cheap, correct first step.
## API surface (target)
Lightweight endpoints or jobs (names indicative):
- `POST /api/v1/tenants/:tenantId/plans` — create plan + optional seed markdown paths.
- `GET/PATCH /api/v1/tenants/:tenantId/plans/:planId` — read/update metadata and Cursor mapping.
- `GET/PATCH /api/v1/.../epics/:epicId`, `.../tasks/:taskId` — same for epics and tasks.
- `POST /api/v1/tenants/:tenantId/sync/cursor/pull` — ingest Cursor to-dos into tasks.
- `POST /api/v1/tenants/:tenantId/sync/cursor/push` — export task state to Cursor.
- Webhook receiver (future): `POST /webhooks/cursor` for push notifications when Cursor exposes them; until then **polling** on a tenant schedule.
## Mapping record (logical schema)
Implemented in Postgres as `markdown_backlog_items` plus `cursor_sync_mappings` (see `docs/Glossary.md`). Logical fields:
- `tenant_id``markdown_backlog_items.workspace_id` (workspace object UUID)
- `plan_slug`, `epic_slug`, `slug` (filesystem / frontmatter alignment)
- `cursor_plan_id` / `cursor_item_id``cursor_sync_mappings` (nullable until connected)
- `last_pulled_at`, `last_pushed_at`, `sync_content_hash` on the mapping row
- `content_hash` on the backlog row (file body hash for import idempotency)
## Environment variables (placeholder)
Document only; wire in app config when implementing.
| Variable | Purpose |
|----------|---------|
| `CURSOR_SYNC_ENABLED` | `true` / `false` per environment. |
| `CURSOR_SYNC_POLL_INTERVAL_SEC` | Polling fallback interval. |
| `CURSOR_API_BASE_URL` | When a stable API exists for your integration tier. |
| `CURSOR_WEBHOOK_SECRET` | Verify inbound webhooks. |
## Security
- Store tokens in tenant-scoped secrets (env, vault, or DB encrypted column)—never in markdown.
- Audit log for every push/pull with actor (user id or system job).
## References
- Backlog layout: `plans/README.md`
- Terminology: `docs/Glossary.md`
- Templates: `docs/templates/`