deckhearth/.convoys/migration-tool.md
Randall Stillwell 68bf75f18c feat(infra): adopt node-pg-migrate + backfill initial schema migration
Closes P1 #11 of .convoys/ship-readiness.md (launch sequence step 7) —
"No migration tool — scripts/add-*.js graveyard". Schema changes
post-this-convoy ship as node-pg-migrate migrations under migrations/
at the repo root; the legacy 27 scripts/add-*.js / scripts/fix-*.js /
scripts/seed-*.js jobs remain append-only history per the no-go-zones
rule.

Decisions (full record in .convoys/migration-tool.md § Decisions):

D1 — Tool: node-pg-migrate@^8. Rejected drizzle-kit / prisma migrate /
kysely because each forces broader TypeScript surface than AGENTS.md
Gotcha #9 allows (TS is a devDep only). node-pg-migrate is
JavaScript-native, raw-SQL-friendly via pgm.sql(), and ESM-clean for
the post-bump-next-js "type": "module" repo. Brings pg@^8.21.0 as a
peer dep (dev-only; never loaded in the Next.js bundle).

D2 — Migrations directory: migrations/ at the repo root. Separates
the tool-wrapped artifacts from the historical scripts/migrations/
placeholder folder (which housed the lone pre-tool
2026-05-24-rename-admin-email.js migration and remains preserved for
the audit trail). Matches node-pg-migrate's default flag.

D3 — Tracking table: default pgmigrations (no name collision with
the existing 7-table bootstrap; zero CLI noise).

D4 — Backfill strategy: hand-translate scripts/setup-neon-db.js's
DDL into the initial migration verbatim. Each await sql`...` block
becomes one pgm.sql(`...`) call. Each CREATE uses IF NOT EXISTS, so
the migration is idempotent against fresh AND pre-existing envs —
re-running setup-db on an env that already has the schema is a no-op
DDL-wise (only records the pgmigrations row). Documented assumption:
prod has drifted via the 27 historical add-*.js scripts; reconciling
those into the migration history is the queued
reconcile-historical-add-scripts follow-up convoy.

D5 — Bootstrap reconciliation: split. setup-neon-db.js now (1)
validates ADMIN_INITIAL_PASSWORD + POSTGRES_URL, (2) spawns
`npm run migrate up` via child_process with stdio inherited, (3)
seeds the admin row with ON CONFLICT (email) DO NOTHING. The seven
DDL blocks are deleted from setup-neon-db.js; success/error message
copy is updated to mention the migration step explicitly.

D6 — CI integration: defer. Wiring a CI job that runs migrate up
against a test DB needs either a dedicated Neon branch + secret OR a
Postgres service container; both are real work. Surface as
wire-migrate-into-ci follow-up. Risk acknowledged in
.convoys/migration-tool.md § R3.

D7 — Down-migration on the initial backfill: hard stub. Rolling back
the initial schema would drop every user / card / collection / deck
row in the DB. The stub throws with a long-form error pointing at
the recommended alternative (branch the Neon database + forward-apply).
Future migrations that touch one of the seven bootstrap tables write
their own dated migration with a real down().

Verification (pre-PR):
- npm run lint → 128 problems (baseline preserved, zero regression;
  migration file is lint-clean, no new ignore patterns)
- npm run test:run → 21/21 pass
- node --check on migrations/1779853647564_initial-schema.js + on
  scripts/setup-neon-db.js → exit 0
- Module load + down() throw verified via dynamic import
- npm run migrate -- --help reaches the node-pg-migrate CLI through
  the wrapper

Live verification against a Neon branch is deferred (no throwaway
branch available); the operator's optional post-merge sequence is
documented in .convoys/migration-tool.md § Operator runbook.

See .convoys/migration-tool.md § Follow-ups for the queued
wire-migrate-into-ci / reconcile-historical-add-scripts /
retire-graveyard-scripts-after-audit / audit-node-pg-migrate-transitive-deps
/ add-migration-template follow-up convoys.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-26 22:55:09 -05:00

31 KiB

name classification priority success_metric skip status created shipped parent addresses depends_on
migration-tool quality P1 (launch sequence step 7) Schema changes ship as a migration file under `migrations/` at the repo root and are applied by `npm run migrate up`. The initial backfill migration reproduces `scripts/setup-neon-db.js`'s 7-table bootstrap DDL verbatim. `npm run setup-db` invokes the migration runner before seeding the admin user. `.convoys/ship-readiness.md` P1 #11 ("No migration tool — `scripts/add-*.js` graveyard") flips from open → RESOLVED.
role-design-system-auditor
role-a11y-auditor
role-ux-reviewer
role-ia-architect
shipped 2026-05-26 2026-05-26 ship-readiness P1
drop-public-setup (setup-neon-db.js was previously CJS — already ESM post-Brief 2; this convoy assumes ESM)
fix-reset-db-script (precedent for the ESM/env-var/seed-line shape that setup-neon-db.js now shares)

Convoy: migration-tool

Adopt node-pg-migrate as the schema-change tool. Backfill a single initial migration that reproduces scripts/setup-neon-db.js's documented bootstrap DDL. Modify setup-neon-db.js to invoke the migration runner before seeding the admin user. Document the new convention.

Background — the graveyard problem

AGENTS.md Gotcha #6 (pre-convoy) flagged 27+ scripts in scripts/ of the form add-*.js / fix-*.js / seed-*.js, each a one-shot DDL change applied once to prod with no idempotency tracking and no rollback. Onboarding a fresh env required re-running every script in the right order; there was no way to know what had been run on a given Neon branch; every new column was at risk of being missed in prod.

The lone existing "tool-shaped" migration was scripts/migrations/2026-05-24-rename-admin-email.js (pick-a-name convoy). It uses ESM, is idempotent, and is UNIQUE-collision-safe — the right shape for a single migration script, but it is NOT wrapped by any migration tool, so re-running it is the operator's responsibility and the operator gets no signal about whether the migration has already been applied to a given env.

The .cursor/rules/no-go-zones.mdc rule (pre-convoy) documented the ad-hoc scripts/migrations/YYYY-MM-DD-<slug>.js convention as a placeholder "until a real migration tool is adopted". This convoy adopts that tool.

Decisions

Seven architect decisions. D1-D2 + D4-D5 + D7 are parent-self-ratified following the convoy spec's recommended choices verbatim (the recommendations included tuning evidence and matched the project's existing patterns). D3 + D6 are mechanically determined by D1-D2. This is a single-implementer convoy; no operator gate was needed because the spec pre-ratified each Decision's recommended path.

D1 — Tool: node-pg-migrate@^8

Ratified: node-pg-migrate@^8.0.4 (the version that resolves at install time; pinned to ^8 in devDependencies).

Considered alternatives:

Tool Why rejected
drizzle-kit Schema-as-code in TypeScript; would force broader TS adoption than AGENTS.md Gotcha #9 allows (TS is a devDep only for the eslint-config-next chain). Out of scope.
prisma migrate Whole-ORM adoption is a multi-month surface change. Out of scope.
kysely migrations TypeScript-first; same TS-adoption issue as drizzle-kit. Out of scope.
Hand-rolled in-house tool Reinvents the tracking-table + locking + dry-run + idempotency-mark surface; pure cost, zero novel value.

node-pg-migrate matches the repo's existing style:

  • JavaScript-only (no tsconfig.json required; ESM migrations work out of the box because package.json has "type": "module").
  • Raw SQL-friendly via pgm.sql(...) — no schema-as-code DSL to learn. The pre-existing 2026-05-24-rename-admin-email.js migration is hand-written SQL-shaped JS; the new tool wraps that exact shape.
  • Lightweight: ~533 KB unpacked; only glob@~11.1.0 + yargs@~17.7.0 as direct deps.
  • Lowest-magic option in the candidate set: no codegen, no declarative diff, no opinions about file layout beyond a conventional migrations/ folder.

Cost: brings in pg@^8.21.0 as a peer dep (needed because node-pg-migrate uses the standard TCP pg client, not Neon's HTTP driver). That's an extra ~3 MB of node_modules; no runtime cost because the migration tool is dev-only / scripts-only and never loads in the Next.js bundle.

D2 — Migrations directory: migrations/ at repo root

Ratified: migrations/ at the repo root.

scripts/migrations/ already exists as the historical placeholder folder (housing 2026-05-24-rename-admin-email.js). Per the convoy spec's suggested split, the new tool-wrapped migrations live in a fresh top-level migrations/ directory:

  • Separates the tool-wrapped artifacts from the historical placeholder. The placeholder folder is preserved for the audit trail of the lone pre-tool migration; agents reading scripts/migrations/ see "this is history, NOT how new schema changes ship".
  • Matches node-pg-migrate's default --migrations-dir migrations (zero CLI noise for the common case).
  • Keeps tool-managed state out of scripts/, which already houses 27 append-only legacy jobs that are flagged as no-go-zones.

D3 — Tracking table: default pgmigrations

Ratified: accept the default. No existing table in the schema collides with pgmigrations (verified by reading scripts/setup-neon-db.js's 7-table DDL and by cross-referencing docs/SCHEMA_MAP.md). Zero CLI flag, zero documentation overhead.

D4 — Backfill strategy: hand-translate setup-neon-db.js

Ratified: hand-translate scripts/setup-neon-db.js's DDL into the initial backfill migration.

Considered alternative: run pg_dump --schema-only against a fresh npm run setup-db to capture the bootstrap shape mechanically. The mechanical approach would be marginally more "guaranteed correct" versus the hand-translated copy, but requires standing up a throwaway Neon branch, dotenv-wiring pg_dump, sanitizing the output (stripping owner/permissions lines that pg_dump adds), and manually accepting the resulting normalized SQL into the migration file. That's more moving parts than a verbatim copy from a single documented file.

The hand-translation is mechanically simple: each await sql\`block insetup-neon-db.jsbecomes onepgm.sql(`...`)call in the migration'sup(). The seven CREATE TABLE IF NOT EXISTSblocks are preserved verbatim (including column order, types, defaults, FK clauses, and the UNIQUE constraints). The admin-row INSERT insetup-neon-db.jsis NOT replicated into the migration — that's the seed step, which stays insetup-neon-db.js` (per D5).

Documented assumption: setup-neon-db.js is the bootstrap source-of-truth that the project has shipped with since first commit. If the prod schema has drifted from that bootstrap shape (which it HAS — the 27 historical scripts/add-*.js jobs added many columns that the bootstrap script never created), the drift is out-of-scope for this convoy. The backfill migration captures the setup-neon-db.js-shape only; reconciling the full prod shape into the migration history is the queued reconcile-historical-add-scripts follow-up convoy.

The choice is documented as a known limitation in docs/SCHEMA_MAP.md's new preamble: "the initial backfill captures only the post-setup-neon-db.js shape ... if a fresh env needs the full historical column set, a follow-up convoy ... will fold the historical effects into the migration history; until then this file remains the curated reference for the full prod shape."

The use of CREATE TABLE IF NOT EXISTS in the backfill (matching setup-neon-db.js verbatim) means the initial migration is safe to run against existing envs — every CREATE is a no-op on tables that already exist (the historical add-*.js columns are preserved). Only the pgmigrations row changes.

D5 — Bootstrap reconciliation: split DDL (migrations) from seed (setup-neon-db.js)

Ratified: split. setup-neon-db.js now:

  1. Validates ADMIN_INITIAL_PASSWORD is set (fail loud BEFORE touching the DB).
  2. Validates POSTGRES_URL is set (new — was previously implicit).
  3. Spawns npm run migrate up via node:child_process.spawn with stdio: 'inherit'. If migrate exits non-zero, setup-db exits non-zero with a wrapping error message that surfaces the failing exit code + signal and points the operator at npm run migrate up to re-try just the migration step.
  4. Connects to Neon (via the existing @neondatabase/serverless HTTP driver) and runs the admin-row INSERT with ON CONFLICT (email) DO NOTHING.

The seven await sql\CREATE TABLE IF NOT EXISTS ...`blocks are removed fromsetup-neon-db.js— they now live inmigrations/1779853647564_initial-schema.js`. The script's success/error message copy is updated to mention the migration step explicitly so the operator's mental model matches the new pipeline.

Rejected alternative: keep all DDL in setup-neon-db.js, have the migration runner be a separate npm run migrate invocation operators remember to call. This would leave the DDL-ownership ambiguity intact (two sources of truth) and require operators to know to call both scripts in the right order — exactly the kind of operator-burden that the convoy is trying to remove.

D6 — CI integration: defer

Ratified: defer to wire-migrate-into-ci follow-up convoy.

Adding a CI job that runs npm run migrate up against a test Neon branch (or against a temporary Postgres container) requires either:

  • A dedicated test Neon branch + a MIGRATE_TEST_DATABASE_URL secret in GitHub Actions, plus branch-reset logic so successive PRs don't see each other's migrations.
  • A Postgres service container in the workflow YAML, which means a ~30s container-start tax on every PR and a CI-specific code path for the migration runner.

Both are real work. The convoy spec explicitly authorizes deferring this to a follow-up — surfaced as wire-migrate-into-ci in the follow-ups list below. Risk acknowledged: until that follow-up lands, new migration files are validated at code-review time only (via node --check, the implementer's local npm run migrate up against their dev branch, and any operator-led pre-deploy run).

D7 — Down-migration shape on the initial backfill: hard stub

Ratified: hard stub that throws. Per the convoy spec's recommended "too risky to drop prod schema" path.

Rolling back the initial schema would drop every users / cards / collections / decks row in the database — including the pgmigrations row itself. There is no realistic ops scenario where this is the right thing to do; if a developer genuinely needs a clean schema for testing, branching the Neon database is instant + cheap and produces a strictly better outcome (zero data loss for other developers using the same branch).

The stub's error message names the migration explicitly, explains the risk, and points at the recommended alternative (Neon branch + forward-apply). Future migrations that touch one of the seven bootstrap tables should write their OWN dated migration with a real down() — they do NOT need to re-enable the down on this initial backfill.

The fix

File Action Purpose
migrations/1779853647564_initial-schema.js new The backfill migration. up() runs seven pgm.sql(\CREATE TABLE IF NOT EXISTS ...`)blocks reproducingscripts/setup-neon-db.js's 7-table DDL verbatim (users / cards / user_cards / collections / collection_cards / decks / deck_cards). down()throws (D7).shorthandsisundefined`.
package.json modified Adds "migrate": "node-pg-migrate --database-url-var POSTGRES_URL --envPath .env.local --migrations-dir migrations --verbose" to scripts. Adds node-pg-migrate@^8.0.4 and pg@^8.21.0 to devDependencies. setup-db script unchanged at the YAML layer; its behavior changes per D5.
scripts/setup-neon-db.js modified Splits DDL (now in the migration) from seed (admin-row INSERT). Adds a POSTGRES_URL env-var check (was previously implicit). Adds a runMigrations() helper that spawns npm run migrate up and rejects with a wrapped error on non-zero exit. Updates the success/error message copy to mention the migration step.
README.md modified § Installation step 4 now mentions that setup-db chains the migration runner. New § "Schema changes" documents the npm run migrate create ... → edit → npm run migrate up → commit flow. § "First-time admin setup" mentions the migrate step.
AGENTS.md modified § 3 Conventions: new "Schema changes (post-migration-tool)" bullet pointing at the new flow. § 4 Gotchas: Gotcha #6 flipped from open → RESOLVED with the as-shipped paragraph (tool / dir / tracking-table / idempotency notes).
.cursor/rules/no-go-zones.mdc modified "Schema changes" rule now describes the node-pg-migrate flow. The legacy scripts/migrations/YYYY-MM-DD-<slug>.js convention is documented as "preserved for the lone existing pre-tool migration; not used for new work".
.cursor/rules/db-and-schema.mdc modified § "Schema source of truth" now points at migrations/ and the npm run migrate create ... workflow. The "Until a proper migration tool is adopted" preamble is replaced.
docs/SCHEMA_MAP.md modified Preamble paragraph re-scopes the file: tool-managed schema lives in migrations/; this file remains the curated reference for the full prod shape (which still includes the historical add-*.js columns the initial backfill doesn't replay). § Regeneration updated to mention migrations/.
package-lock.json modified Reflects node-pg-migrate + pg + transitive deps.

File diff highlights

  • migrations/1779853647564_initial-schema.js is ~155 lines. Seven pgm.sql(...) blocks plus a docstring explaining the idempotency guarantee, plus a down() stub with a long-form error message.
  • scripts/setup-neon-db.js net diff: ~+40 / -50. The seven DDL blocks are deleted; the spawn helper + the POSTGRES_URL guard + the updated message copy are added.
  • package.json adds one scripts line + two devDependencies entries.
  • The doc / rule edits are 1-2 paragraphs each.

Verification plan

  1. npm run lint → exit 1 with 128 problems (baseline preserved). The new migration file MUST be lint-clean (no new ignore patterns in eslint.config.mjs).
  2. npm run test:run → 21/21 pass. Vitest does not touch the migration surface; the run must stay green.
  3. node --check migrations/1779853647564_initial-schema.js → exit 0 (syntax-valid).
  4. node --check scripts/setup-neon-db.js → exit 0.
  5. Dynamic import of the migration file: node -e "import('./migrations/...').then(m => m.down())" should throw with the documented error message (proves D7 is wired correctly).
  6. npm run migrate -- --help returns the node-pg-migrate help text through the wrapper (proves the wrapper's flag chain is shell-parseable).

Live test against a Neon branch: NOT performed in this convoy. The parent did not have a throwaway Neon branch available, and the convoy spec authorizes documenting the gap. Operator's optional post-merge verification:

DATABASE_URL=<neon-branch> npm run migrate up
# expect: applies the initial migration; pgmigrations row appears
DATABASE_URL=<neon-branch> npm run migrate down   # expect: throws hard stub
ADMIN_INITIAL_PASSWORD=<...> POSTGRES_URL=<neon-branch> npm run setup-db
# expect: migrate up logs (no-op since the migration is already applied), then admin user seeded

(Note the --database-url-var POSTGRES_URL wrapper means POSTGRES_URL is the var the operator sets; the DATABASE_URL calls above are illustrative of how a stock node-pg-migrate would be invoked. For the wrapper, set POSTGRES_URL in .env.local and just run npm run migrate up.)

Risks

R1 — Prod schema drift from setup-neon-db.js DDL

The 27 historical scripts/add-*.js jobs added columns + tables to prod that setup-neon-db.js never created (user_settings, user_avatars, user_favorites, collection_permissions, collection_activity, invitations, plus many users and collections columns — see docs/SCHEMA_MAP.md). The initial backfill captures only the post-setup-neon-db.js shape.

Why this is safe: CREATE TABLE IF NOT EXISTS is a no-op on existing tables. Running npm run migrate up against the prod DB applies the backfill migration (recording it in pgmigrations) without touching the existing schema. Future migrations can ALTER / CREATE freely from this baseline.

Why this is not safe for fresh-env onboarding: a brand-new Neon branch onboarded via npm installnpm run setup-db will end up with the bootstrap 7-table shape only — none of the historical add-*.js columns will be present. Most code paths assume those columns exist (see docs/SCHEMA_MAP.md for the full shape). Mitigation: the queued reconcile-historical-add-scripts follow-up convoy will fold the historical effects into the migration history. Until then, fresh-env onboarding still requires either a Neon branch of an existing prod-shaped DB, or a manual replay of the historically-applied scripts (which is the pre-convoy status quo — this convoy does not regress that situation).

R2 — Down-migration is a hard stub on the initial backfill

By design (D7). Documented in the migration file's down() docstring and in this convoy. Future migrations (alter-foo-add-bar, etc.) should have real down()s for safe rollback; the stub is specific to the initial backfill.

R3 — CI does not exercise migrations

By design (D6). The first signal that a new migration is broken is the developer's local npm run migrate up against their dev branch. Mitigation: surface as wire-migrate-into-ci follow-up convoy. Operator-visible consequence: a PR that adds a syntactically-valid but logically-broken migration (e.g. ALTER TABLE non_existent_table) will pass CI; the breakage shows up the first time npm run setup-db or npm run migrate up is run against an env. Pre-deploy, this is caught by the developer's own dev-loop. Post-deploy, the migration's failure on prod is setup-neon-db.js exiting non-zero with the wrapped error message — visible in the Vercel deploy logs.

R4 — Operator must seed POSTGRES_URL for any migrate invocation

npm run migrate up requires POSTGRES_URL in .env.local (or set in the calling environment). If unset, node-pg-migrate errors with its standard "No database connection string" message; the wrapper doesn't add a friendlier pre-flight check (matching node-pg-migrate's default behavior is fine — operators running migrations are by definition operating on a known DB). setup-neon-db.js does add an explicit POSTGRES_URL check before spawning the migration runner, so the npm run setup-db happy path produces a useful error.

R5 — New npm audit vulnerabilities surface via node-pg-migrate's

glob + yargs transitive deps

node-pg-migrate@8.0.4 pulls in glob@~11.1.0 + yargs@~17.7.0, which transitively bring in older brace-expansion, minimatch, and picomatch versions with known advisories. npm audit reports 11 vulnerabilities (6 moderate, 5 high) at install time. These are all in dev-only paths (the migration tool runs in scripts/CI, never in the deployed Next.js bundle) and the affected APIs (glob's shell-injection CLI; brace-expansion's ReDoS) are not exercised by node-pg-migrate's call sites. The convoy spec says "do NOT bump unrelated deps", so the audit fix is deferred. Surface as a follow-up if a security audit flags this surface specifically.

R6 — Migration 1779853647564_initial-schema timestamp is fixed

Once committed and applied anywhere, the file name + the migration's contents are pinned in the pgmigrations table. Mutating either silently corrupts any env that has the prior version recorded. This is node-pg-migrate's standard contract; documented inline in the migration file's docstring. A future schema correction needs a NEW dated migration, not an edit to this file.

Operator runbook (new flow)

Schema change

# 1. Generate a migration file (JS template, timestamp-prefixed)
npm run migrate create add-foo-column -- -j js

# 2. Edit the generated file under migrations/<timestamp>_add-foo-column.js
#    - Put DDL in up() via pgm.sql(`ALTER TABLE ...`)
#    - Write a real down() if rollback is safe; otherwise a throwing stub.

# 3. Apply locally against the dev DB (POSTGRES_URL from .env.local)
npm run migrate up

# 4. Update docs/SCHEMA_MAP.md to reflect the schema change.

# 5. Commit the migration file + docs together. CI runs lint + vitest
#    only (D6); the migration itself isn't exercised in CI yet.

# 6. After merge + deploy, an operator runs the migration against prod:
#    POSTGRES_URL=<prod> npm run migrate up
#    (Or wait for the next setup-db invocation; it chains migrate up.)

Onboarding a new env

# 1. Clone + install
git clone <repo>
cd tcg-vault
npm install

# 2. Seed .env.local (POSTGRES_URL, JWT_SECRET, ADMIN_INITIAL_PASSWORD).

# 3. Run setup-db — chains migrate up, then seeds the admin user.
npm run setup-db

Re-running setup against an existing env

Idempotent on both halves:

  • The initial backfill migration uses CREATE TABLE IF NOT EXISTS, so it's a no-op on tables that already exist.
  • The admin-row INSERT uses ON CONFLICT (email) DO NOTHING.

The operator caveat from drop-public-setup Brief 1 still applies: re-running setup-db does NOT rotate an existing admin row's password. See README § "First-time admin setup" → operator-rotation note.

Recovering from a failed migration

If npm run migrate up fails partway through (unlikely for the initial backfill since each CREATE is independent; possible for future migrations with multi-statement up()s), the operator's options:

  1. Fix forward: edit the failing migration file, re-run npm run migrate up. node-pg-migrate's default --single-transaction true flag means a failure rolls back the transaction, so the DB is left in the pre-migration state and the pgmigrations row is NOT recorded. Re-running picks up cleanly.

  2. Skip a broken migration (last resort): npm run migrate up -- --fake marks pending migrations as applied without running them. Use this ONLY if the schema state is already correct outside the tool's view (e.g., the migration was applied manually via psql). Document any --fake use in the convoy/PR that caused it.

Follow-ups

  • wire-migrate-into-ci (priority: P2 CI infra). Add a CI job that runs npm run migrate up against a test DB (either a dedicated Neon branch + secret, or a Postgres service container). Catches syntactically-invalid migrations + most logical errors at PR time. Deferred per D6.
  • reconcile-historical-add-scripts (priority: P1 quality — needed for fresh-env onboarding). Fold the effects of the 27 historical scripts/add-*.js / scripts/fix-*.js / scripts/seed-*.js jobs into the migration history so a brand-new Neon branch can be onboarded by npm installnpm run setup-db alone (without manually replaying the historical scripts). Multi-PR: one migration per logical change, ideally generated by reading the scripts' SQL and re-shaping into idempotent pgm.sql(...) blocks (with IF NOT EXISTS / IF EXISTS guards so re-application is safe).
  • retire-graveyard-scripts-after-audit (priority: P3 polish, blocked on reconcile-historical-add-scripts). Once the migration history captures all historical effects, the legacy scripts/add-*.js / scripts/fix-*.js / scripts/seed-*.js files can be deleted (or moved to scripts/historical/). They remain no-go-zones until that cleanup convoy lands.
  • audit-node-pg-migrate-transitive-deps (priority: P3 hygiene). npm audit reports 11 vulnerabilities (6 moderate, 5 high) coming from node-pg-migrate's glob + yargs transitive deps. All in dev-only paths; not exercised by node-pg-migrate's call sites. Surface only if a security audit specifically flags this surface, or if node-pg-migrate ships a v9 that updates the transitive tree.
  • add-migration-template (priority: P3 DX). Add a custom template via --template-file-name so generated migrations include the project's preferred docstring shape + a reminder about docs/SCHEMA_MAP.md updates. Surface if migration authoring proves inconsistent.

As-shipped

Shipped 2026-05-26 on convoy/migration-tool.

Decisions ratified at gate 1

All seven decisions landed verbatim from the convoy spec's recommendations. No mid-execution surprises that would have routed back through an architect bounce.

As-shipped surface

  • migrations/ directory created at repo root (was: empty / nonexistent).
  • migrations/1779853647564_initial-schema.js added. ~155 lines. Seven pgm.sql(\CREATE TABLE IF NOT EXISTS ...`)blocks reproducingscripts/setup-neon-db.js's 7-table DDL verbatim. Down-migration is a hard stub that throws with a long-form error message naming the alternative (Neon branch + forward-apply). Lint-clean (no new ignore patterns in eslint.config.mjs`).
  • scripts/setup-neon-db.js refactored. The seven await sql\CREATE TABLE IF NOT EXISTS`blocks are removed (DDL now lives in the migration). ArunMigrations()helper is added that spawnsnpm run migrate upvianode:child_process.spawn({ stdio: 'inherit', shell: false })and rejects with a wrapped error on non-zero exit. APOSTGRES_URLpre-flight check is added (was previously implicit). The success/error message copy is updated to mention the migration step explicitly so the operator's mental model matches the new pipeline. The admin-row INSERT (withON CONFLICT (email) DO NOTHING) and the ADMIN_INITIAL_PASSWORD` env-var check are preserved verbatim.
  • package.json: scripts.migrate added (node-pg-migrate --database-url-var POSTGRES_URL --envPath .env.local --migrations-dir migrations --verbose). devDependencies adds node-pg-migrate@^8.0.4 + pg@^8.21.0.
  • README.md updated: § Installation step 4 documents the migration chain; new § "Schema changes (post-migration-tool convoy)" explains the create/edit/up/commit flow; § "First-time admin setup" mentions the migrate step.
  • AGENTS.md updated: § 3 Conventions gains a "Schema changes" bullet pointing at the new flow + .convoys/migration-tool.md; § 4 Gotcha #6 flipped from open → RESOLVED with the as-shipped paragraph (tool / dir / tracking-table / idempotency notes).
  • .cursor/rules/no-go-zones.mdc updated: "Schema changes" rule rewritten to describe the node-pg-migrate flow; legacy scripts/migrations/YYYY-MM-DD-<slug>.js convention documented as "preserved for the lone existing pre-tool migration; not used for new work".
  • .cursor/rules/db-and-schema.mdc updated: § "Schema source of truth" rewritten to point at migrations/ and the npm run migrate create ... workflow.
  • docs/SCHEMA_MAP.md updated: preamble re-scopes the file (tool-managed schema lives in migrations/; this file remains the curated reference for the full prod shape including historical add-*.js columns); § Regeneration updated.

Verification (all gates green pre-PR)

  • npm run lint → exit 1 with 128 problems (baseline preserved, zero regression). The new migration file is lint-clean; no new ignore patterns added.
  • npm run test:run21/21 pass in ~1.3s (4 test files, no rate-limit / migration touchpoint).
  • node --check migrations/1779853647564_initial-schema.js → exit 0.
  • node --check scripts/setup-neon-db.js → exit 0.
  • Module-load + down() throw verification: node -e "import('./migrations/1779853647564_initial-schema.js').then(m => m.down())" → throws with the documented [migration:1779853647564_initial-schema] Refusing to drop the initial schema. ... message.
  • npm run migrate -- --help → returns the standard node-pg-migrate help text through the wrapper.

Live verification status

Deferred per the convoy spec. The parent did not have a throwaway Neon branch available. The operator's optional post-merge verification sequence is documented in § Operator runbook → "Re-running setup against an existing env".

Operator action required going forward

None for the convoy itself. The migration is idempotent against the existing prod schema (CREATE TABLE IF NOT EXISTS no-ops on existing tables; the pgmigrations row is the only DB-side change). No new env vars; no new secrets. The existing POSTGRES_URL + ADMIN_INITIAL_PASSWORD contract is preserved.

Optional but recommended post-merge:

  1. The next deploy that runs setup-neon-db.js will silently apply the backfill migration (recording it in pgmigrations). No operator action; this is just-in-time chained.

  2. To pre-apply the migration without re-running the seed step: POSTGRES_URL=<...> npm run migrate up.

Cross-validation finding

Same shape as the add-rate-limiting cross-validation finding: the existing Playwright smoke 3/3 spec defends the post-migration-tool deployment without anyone writing a dedicated test. Smoke calls /api/health once per run; /api/health reads from the users table (via lib/database.js health-check query — verify with the smoke run on the PR) which is created by both the initial backfill AND the pre-existing scripts/setup-neon-db.js DDL, so smoke's signal is invariant to which mechanism populated the schema. Sixth consecutive convoy where the same 3-test smoke spec defends the auth surface through a sweeping change (PR #15 Layout default-user → PR #19 CORS-tighten → PR #20 rate-limiting → PR #21 pick-a-name → PR #25 reset-db-fix → this convoy).

What did NOT change

  • lib/database.js, lib/permission-middleware.js, lib/auth-secret.js, lib/rate-limit.js — no auth or runtime surface touched.
  • pages/api/**/*.js — no API handlers touched.
  • test/** — no test surface touched (Decision-6-equivalent: per-route handler tests for the migration runner would be valuable but are out of scope; the migration's correctness is verified via the operator runbook's manual sequence).
  • scripts/migrations/2026-05-24-rename-admin-email.js — preserved verbatim; not migrated into the new migrations/ directory because it's already-applied history (the no-go-zones rule covers it).
  • .github/workflows/** — no new CI job (Decision 6 defers wire-migrate-into-ci to a follow-up convoy).
  • package-lock.json semantics — only adds the new transitive deps; no version bumps to existing deps.
  • The legacy 27 scripts/add-*.js / scripts/fix-*.js / scripts/seed-*.js files — all preserved as append-only history per the no-go-zones rule.

Subagent / multitask footnote

This convoy ran with the parent owning architecture + implementation end-to-end (no architect or implementer subagent dispatch). The convoy spec pre-ratified each Decision's recommended path, and the implementation surface was a small set of well-bounded file edits following the spec's "The change (implementation shape)" checklist verbatim. A mid-execution worktree migration was required (the parent opened the convoy from the main worktree, which was incidentally checked out on a sibling branch convoy/purge-weak-creds-from-helpers due to a parallel agent's activity; the parent stashed its work, created tcg-vault-worktrees/migration-tool as a fresh worktree on the canonical convoy/migration-tool branch, re-applied the work there via a tracked-changes patch + an untracked-files tar, and continued cleanly). No file content was lost in that migration; the git history is single-branch from the worktree's perspective.