convoy). It uses ESM, is idempotent, and is UNIQUE-collision-safe —
the right shape for a single migration script, but it is NOT wrapped
by any migration tool, so re-running it is the operator's
responsibility and the operator gets no signal about whether the
migration has already been applied to a given env.
The `.cursor/rules/no-go-zones.mdc` rule (pre-convoy) documented the
ad-hoc `scripts/migrations/YYYY-MM-DD-<slug>.js` convention as a
placeholder *"until a real migration tool is adopted"*. This convoy
adopts that tool.
## Decisions
Seven architect decisions. D1-D2 + D4-D5 + D7 are parent-self-ratified
following the convoy spec's recommended choices verbatim (the
recommendations included tuning evidence and matched the project's
existing patterns). D3 + D6 are mechanically determined by D1-D2.
This is a single-implementer convoy; no operator gate was needed
because the spec pre-ratified each Decision's recommended path.
### D1 — Tool: `node-pg-migrate@^8`
**Ratified: `node-pg-migrate@^8.0.4`** (the version that resolves at
install time; pinned to `^8` in `devDependencies`).
Considered alternatives:
| Tool | Why rejected |
| --- | --- |
| `drizzle-kit` | Schema-as-code in TypeScript; would force broader TS adoption than `AGENTS.md` Gotcha #9 allows (TS is a devDep only for the `eslint-config-next` chain). Out of scope. |
| `prisma migrate` | Whole-ORM adoption is a multi-month surface change. Out of scope. |
| `kysely` migrations | TypeScript-first; same TS-adoption issue as drizzle-kit. Out of scope. |
| Hand-rolled in-house tool | Reinvents the tracking-table + locking + dry-run + idempotency-mark surface; pure cost, zero novel value. |
`node-pg-migrate` matches the repo's existing style:
- JavaScript-only (no `tsconfig.json` required; ESM migrations work
out of the box because `package.json` has `"type": "module"`).
- Raw SQL-friendly via `pgm.sql(...)` — no schema-as-code DSL to
learn. The pre-existing `2026-05-24-rename-admin-email.js`
migration is hand-written SQL-shaped JS; the new tool wraps that
exact shape.
- Lightweight: ~533 KB unpacked; only `glob@~11.1.0` + `yargs@~17.7.0`
as direct deps.
- Lowest-magic option in the candidate set: no codegen, no
declarative diff, no opinions about file layout beyond a
conventional `migrations/` folder.
Cost: brings in `pg@^8.21.0` as a peer dep (needed because
`node-pg-migrate` uses the standard TCP `pg` client, not Neon's HTTP
driver). That's an extra ~3 MB of `node_modules`; no runtime cost
because the migration tool is dev-only / scripts-only and never
loads in the Next.js bundle.
### D2 — Migrations directory: `migrations/` at repo root
**Ratified: `migrations/` at the repo root.**
`scripts/migrations/` already exists as the historical placeholder
folder (housing `2026-05-24-rename-admin-email.js`). Per the convoy
spec's suggested split, the new tool-wrapped migrations live in a
fresh top-level `migrations/` directory:
- Separates the tool-wrapped artifacts from the historical
placeholder. The placeholder folder is preserved for the audit
trail of the lone pre-tool migration; agents reading
`scripts/migrations/` see "this is history, NOT how new schema
1. Validates `ADMIN_INITIAL_PASSWORD` is set (fail loud BEFORE touching the DB).
2. Validates `POSTGRES_URL` is set (new — was previously implicit).
3. Spawns `npm run migrate up` via `node:child_process.spawn` with
`stdio: 'inherit'`. If migrate exits non-zero, setup-db exits non-zero
with a wrapping error message that surfaces the failing exit code +
signal and points the operator at `npm run migrate up` to re-try just
the migration step.
4. Connects to Neon (via the existing `@neondatabase/serverless` HTTP driver)
and runs the admin-row INSERT with `ON CONFLICT (email) DO NOTHING`.
The seven `await sql\`CREATE TABLE IF NOT EXISTS ...\`` blocks are
removed from `setup-neon-db.js` — they now live in
`migrations/1779853647564_initial-schema.js`. The script's success/error
message copy is updated to mention the migration step explicitly so
the operator's mental model matches the new pipeline.
Rejected alternative: keep all DDL in `setup-neon-db.js`, have the
migration runner be a separate `npm run migrate` invocation operators
remember to call. This would leave the DDL-ownership ambiguity intact
(two sources of truth) and require operators to know to call both
scripts in the right order — exactly the kind of operator-burden that
the convoy is trying to remove.
### D6 — CI integration: defer
**Ratified: defer to `wire-migrate-into-ci` follow-up convoy.**
Adding a CI job that runs `npm run migrate up` against a test Neon
branch (or against a temporary Postgres container) requires either:
- A dedicated test Neon branch + a `MIGRATE_TEST_DATABASE_URL` secret
in GitHub Actions, plus branch-reset logic so successive PRs don't
see each other's migrations.
- A Postgres service container in the workflow YAML, which means a
~30s container-start tax on every PR and a CI-specific code path for
the migration runner.
Both are real work. The convoy spec explicitly authorizes deferring
this to a follow-up — surfaced as `wire-migrate-into-ci` in the
follow-ups list below. **Risk acknowledged:** until that follow-up
lands, new migration files are validated at code-review time only
(via `node --check`, the implementer's local `npm run migrate up`
against their dev branch, and any operator-led pre-deploy run).
### D7 — Down-migration shape on the initial backfill: hard stub
**Ratified: hard stub that throws.** Per the convoy spec's recommended
"too risky to drop prod schema" path.
Rolling back the initial schema would drop every users / cards /
collections / decks row in the database — including the
`pgmigrations` row itself. There is no realistic ops scenario where
this is the right thing to do; if a developer genuinely needs a
clean schema for testing, branching the Neon database is instant +
cheap and produces a strictly better outcome (zero data loss for
other developers using the same branch).
The stub's error message names the migration explicitly, explains the
risk, and points at the recommended alternative (Neon branch +
forward-apply). Future migrations that touch one of the seven
bootstrap tables should write their OWN dated migration with a real
`down()` — they do NOT need to re-enable the down on this initial
backfill.
## The fix
| File | Action | Purpose |
| --- | --- | --- |
| `migrations/1779853647564_initial-schema.js` | new | The backfill migration. `up()` runs seven `pgm.sql(\`CREATE TABLE IF NOT EXISTS ...\`)` blocks reproducing `scripts/setup-neon-db.js`'s 7-table DDL verbatim (users / cards / user_cards / collections / collection_cards / decks / deck_cards). `down()` throws (D7). `shorthands` is `undefined`. |
| `package.json` | modified | Adds `"migrate": "node-pg-migrate --database-url-var POSTGRES_URL --envPath .env.local --migrations-dir migrations --verbose"` to `scripts`. Adds `node-pg-migrate@^8.0.4` and `pg@^8.21.0` to `devDependencies`. `setup-db` script unchanged at the YAML layer; its behavior changes per D5. |
| `scripts/setup-neon-db.js` | modified | Splits DDL (now in the migration) from seed (admin-row INSERT). Adds a `POSTGRES_URL` env-var check (was previously implicit). Adds a `runMigrations()` helper that spawns `npm run migrate up` and rejects with a wrapped error on non-zero exit. Updates the success/error message copy to mention the migration step. |
| `README.md` | modified | § Installation step 4 now mentions that `setup-db` chains the migration runner. New § "Schema changes" documents the `npm run migrate create ...` → edit → `npm run migrate up` → commit flow. § "First-time admin setup" mentions the migrate step. |
| `AGENTS.md` | modified | § 3 Conventions: new "Schema changes (post-`migration-tool`)" bullet pointing at the new flow. § 4 Gotchas: Gotcha #6 flipped from open → RESOLVED with the as-shipped paragraph (tool / dir / tracking-table / idempotency notes). |
| `.cursor/rules/no-go-zones.mdc` | modified | "Schema changes" rule now describes the `node-pg-migrate` flow. The legacy `scripts/migrations/YYYY-MM-DD-<slug>.js` convention is documented as "preserved for the lone existing pre-tool migration; not used for new work". |
| `.cursor/rules/db-and-schema.mdc` | modified | § "Schema source of truth" now points at `migrations/` and the `npm run migrate create ...` workflow. The "Until a proper migration tool is adopted" preamble is replaced. |
| `docs/SCHEMA_MAP.md` | modified | Preamble paragraph re-scopes the file: tool-managed schema lives in `migrations/`; this file remains the curated reference for the full prod shape (which still includes the historical `add-*.js` columns the initial backfill doesn't replay). § Regeneration updated to mention `migrations/`. |
Down-migration is a hard stub that throws with a long-form error
message naming the alternative (Neon branch + forward-apply).
Lint-clean (no new ignore patterns in `eslint.config.mjs`).
- **`scripts/setup-neon-db.js`** refactored. The seven `await sql\`CREATE TABLE IF NOT EXISTS\`` blocks are removed (DDL now lives in the migration). A `runMigrations()` helper is added that spawns `npm run migrate up` via `node:child_process.spawn({ stdio: 'inherit', shell: false })` and rejects with a wrapped error on non-zero exit. A `POSTGRES_URL` pre-flight check is added (was previously implicit). The success/error message copy is updated to mention the migration step explicitly so the operator's mental model matches the new pipeline. The admin-row INSERT (with `ON CONFLICT (email) DO NOTHING`) and the `ADMIN_INITIAL_PASSWORD` env-var check are preserved verbatim.
- **`README.md`** updated: § Installation step 4 documents the migration chain; new § "Schema changes (post-`migration-tool` convoy)" explains the create/edit/up/commit flow; § "First-time admin setup" mentions the migrate step.
- **`AGENTS.md`** updated: § 3 Conventions gains a "Schema changes" bullet pointing at the new flow + `.convoys/migration-tool.md`; § 4 Gotcha #6 flipped from open → RESOLVED with the as-shipped paragraph (tool / dir / tracking-table / idempotency notes).
- **`.cursor/rules/no-go-zones.mdc`** updated: "Schema changes" rule rewritten to describe the `node-pg-migrate` flow; legacy `scripts/migrations/YYYY-MM-DD-<slug>.js` convention documented as "preserved for the lone existing pre-tool migration; not used for new work".
- **`.cursor/rules/db-and-schema.mdc`** updated: § "Schema source of truth" rewritten to point at `migrations/` and the `npm run migrate create ...` workflow.
- **`docs/SCHEMA_MAP.md`** updated: preamble re-scopes the file (tool-managed schema lives in `migrations/`; this file remains the curated reference for the full prod shape including historical add-*.js columns); § Regeneration updated.
### Verification (all gates green pre-PR)
-`npm run lint` → exit 1 with **128 problems** (baseline preserved, zero regression). The new migration file is lint-clean; no new ignore patterns added.
-`npm run test:run` → **21/21 pass** in ~1.3s (4 test files, no rate-limit / migration touchpoint).
- CI on PR #32: Lint ✓ (128 problems at convoy time; the new lint baseline post-PR-#31 is 125 — see § What did NOT change below for the version note) | Vitest 21/21 ✓ | Playwright smoke 3/3 ✓ | `forbidden-endpoints` ✓ | `forbidden-cors-headers` ✓ | Vercel preview deploy ✓ | Aggregate gate ✓
-`Screenshot diff`: not triggered (PR #32's diff is `migrations/**` + `scripts/**` + `package.json` + docs / rules / `package-lock.json` — none of which matches the visual-diff `paths:` filter; the post-PR-#26 `!pages/api/**` exclusion is not even relevant here).
**Spec deviation:** none. All seven decisions landed verbatim from
the convoy spec's recommendations at gate 1. The verification gates
-`lib/database.js`, `lib/permission-middleware.js`, `lib/auth-secret.js`, `lib/rate-limit.js` — no auth or runtime surface touched.
-`pages/api/**/*.js` — no API handlers touched.
-`test/**` — no test surface touched (Decision-6-equivalent: per-route handler tests for the migration runner would be valuable but are out of scope; the migration's correctness is verified via the operator runbook's manual sequence).
-`scripts/migrations/2026-05-24-rename-admin-email.js` — preserved verbatim; not migrated into the new `migrations/` directory because it's already-applied history (the no-go-zones rule covers it).
-`.github/workflows/**` — no new CI job (Decision 6 defers `wire-migrate-into-ci` to a follow-up convoy).
-`package-lock.json` semantics — only adds the new transitive deps; no version bumps to existing deps.
- The legacy 27 `scripts/add-*.js` / `scripts/fix-*.js` / `scripts/seed-*.js` files — all preserved as append-only history per the no-go-zones rule.
### Subagent / multitask footnote
This convoy ran with the parent owning architecture + implementation
end-to-end (no architect or implementer subagent dispatch). The
convoy spec pre-ratified each Decision's recommended path, and the
implementation surface was a small set of well-bounded file edits
following the spec's "The change (implementation shape)" checklist
verbatim. A mid-execution worktree migration was required (the parent
opened the convoy from the main worktree, which was incidentally
checked out on a sibling branch `convoy/purge-weak-creds-from-helpers`
due to a parallel agent's activity; the parent stashed its work,
created `tcg-vault-worktrees/migration-tool` as a fresh worktree on
the canonical `convoy/migration-tool` branch, re-applied the work
there via a tracked-changes patch + an untracked-files tar, and
continued cleanly). No file content was lost in that migration; the
git history is single-branch from the worktree's perspective.