Randall Stillwell 68bf75f18c feat(infra): adopt node-pg-migrate + backfill initial schema migration

Closes P1 #11 of .convoys/ship-readiness.md (launch sequence step 7) —
"No migration tool — scripts/add-*.js graveyard". Schema changes
post-this-convoy ship as node-pg-migrate migrations under migrations/
at the repo root; the legacy 27 scripts/add-*.js / scripts/fix-*.js /
scripts/seed-*.js jobs remain append-only history per the no-go-zones
rule.

Decisions (full record in .convoys/migration-tool.md § Decisions):

D1 — Tool: node-pg-migrate@^8. Rejected drizzle-kit / prisma migrate /
kysely because each forces broader TypeScript surface than AGENTS.md
Gotcha #9 allows (TS is a devDep only). node-pg-migrate is
JavaScript-native, raw-SQL-friendly via pgm.sql(), and ESM-clean for
the post-bump-next-js "type": "module" repo. Brings pg@^8.21.0 as a
peer dep (dev-only; never loaded in the Next.js bundle).

D2 — Migrations directory: migrations/ at the repo root. Separates
the tool-wrapped artifacts from the historical scripts/migrations/
placeholder folder (which housed the lone pre-tool
2026-05-24-rename-admin-email.js migration and remains preserved for
the audit trail). Matches node-pg-migrate's default flag.

D3 — Tracking table: default pgmigrations (no name collision with
the existing 7-table bootstrap; zero CLI noise).

D4 — Backfill strategy: hand-translate scripts/setup-neon-db.js's
DDL into the initial migration verbatim. Each await sql`...` block
becomes one pgm.sql(`...`) call. Each CREATE uses IF NOT EXISTS, so
the migration is idempotent against fresh AND pre-existing envs —
re-running setup-db on an env that already has the schema is a no-op
DDL-wise (only records the pgmigrations row). Documented assumption:
prod has drifted via the 27 historical add-*.js scripts; reconciling
those into the migration history is the queued
reconcile-historical-add-scripts follow-up convoy.

D5 — Bootstrap reconciliation: split. setup-neon-db.js now (1)
validates ADMIN_INITIAL_PASSWORD + POSTGRES_URL, (2) spawns
`npm run migrate up` via child_process with stdio inherited, (3)
seeds the admin row with ON CONFLICT (email) DO NOTHING. The seven
DDL blocks are deleted from setup-neon-db.js; success/error message
copy is updated to mention the migration step explicitly.

D6 — CI integration: defer. Wiring a CI job that runs migrate up
against a test DB needs either a dedicated Neon branch + secret OR a
Postgres service container; both are real work. Surface as
wire-migrate-into-ci follow-up. Risk acknowledged in
.convoys/migration-tool.md § R3.

D7 — Down-migration on the initial backfill: hard stub. Rolling back
the initial schema would drop every user / card / collection / deck
row in the DB. The stub throws with a long-form error pointing at
the recommended alternative (branch the Neon database + forward-apply).
Future migrations that touch one of the seven bootstrap tables write
their own dated migration with a real down().

Verification (pre-PR):
- npm run lint → 128 problems (baseline preserved, zero regression;
  migration file is lint-clean, no new ignore patterns)
- npm run test:run → 21/21 pass
- node --check on migrations/1779853647564_initial-schema.js + on
  scripts/setup-neon-db.js → exit 0
- Module load + down() throw verified via dynamic import
- npm run migrate -- --help reaches the node-pg-migrate CLI through
  the wrapper

Live verification against a Neon branch is deferred (no throwaway
branch available); the operator's optional post-merge sequence is
documented in .convoys/migration-tool.md § Operator runbook.

See .convoys/migration-tool.md § Follow-ups for the queued
wire-migrate-into-ci / reconcile-historical-add-scripts /
retire-graveyard-scripts-after-audit / audit-node-pg-migrate-transitive-deps
/ add-migration-template follow-up convoys.

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-05-26 22:55:09 -05:00

8.1 KiB

Raw Blame History

SCHEMA_MAP.md

Hand-curated reference for the Neon Postgres schema. As of the migration-tool convoy (2026-05-26), the formal source of truth is the migration history under migrations/ at the repo root (managed by node-pg-migrate). The initial backfill migration migrations/1779853647564_initial-schema.js reproduces scripts/setup-neon-db.js's 7-table bootstrap DDL verbatim. The historical scripts/add-*.js / scripts/fix-*.js jobs are preserved as append-only history; their effects are baked into the production schema but are NOT replayed by npm run migrate up on a fresh env — the initial backfill captures only the post-setup-neon-db.js shape. If a fresh env needs the full historical column set, a follow-up convoy (reconcile-historical-add-scripts) will fold the historical effects into the migration history; until then this file remains the curated reference for the full prod shape.

Last reviewed: 2026-05-22 against scripts/setup-neon-db.js + every scripts/add-*.js and scripts/fix-*.js in repo HEAD.

Quick model groups

Group	Tables	Purpose
Identity	`users`, `user_settings`, `user_avatars`	Accounts, profile, preferences
Catalog	`cards`	Master card list across MTG / Pokémon / Lorcana
Ownership	`user_cards`, `user_favorites`	What a user owns / has favorited
Collections	`collections`, `collection_cards`, `collection_permissions`, `collection_activity`	Curated card lists with sharing
Decks	`decks`, `deck_cards`	Playable deck definitions
Invitations	`invitations` (referenced; verify)	Pending share requests

Tables

users

Column	Type	Notes
`id`	`SERIAL PK`
`email`	`VARCHAR(255) UNIQUE NOT NULL`	Lowercase before query/insert (no CI collation set)
`password`	`VARCHAR(255) NOT NULL`	bcrypt hash, cost 12
`role`	`VARCHAR(50)` default `'user'`	`'user' \| 'admin'`
`created_at`, `updated_at`	`TIMESTAMP` default now
`first_name`, `last_name`	`VARCHAR(255)`	From `add-user-profile-columns.js`
`username`	`VARCHAR(255) UNIQUE`
`profile_image_url`, `avatar_url`	`TEXT`	Two redundant columns; verify which is canonical
`bio`	`TEXT`
`favorite_games`	`JSONB` default `'["MTG"]'`	Per-user game preference array
`collection_visibility`	`VARCHAR(20)` default `'private'`
`preferred_currency`	`VARCHAR(3)` default `'USD'`
`cards_per_page`	`INTEGER` default `50`
`default_view`	`VARCHAR(10)` default `'grid'`
`notifications_email`	`BOOLEAN` default `true`
`notifications_marketing`	`BOOLEAN` default `false`
`two_factor_enabled`	`BOOLEAN` default `false`	Not implemented yet
`theme`	`VARCHAR(10)` default `'system'`	`'light' \| 'dark' \| 'system'`
`language`	`VARCHAR(5)` default `'en'`
`is_pending`	`BOOLEAN` default `false`	Set by invitation flow before signup completes

cards

Column	Type	Notes
`id`	`SERIAL PK`
`name`	`VARCHAR(255) NOT NULL`
`set_name`, `set_code`, `card_number`	`VARCHAR`
`rarity`	`VARCHAR(50)`
`game`	`VARCHAR(50) NOT NULL`	`'mtg' \| 'pokemon' \| 'lorcana'`
`mana_cost`	`VARCHAR(50)`	MTG only
`cmc`	`INTEGER`	MTG only
`card_type`	`VARCHAR(255)`
`colors`	`JSONB`	MTG color array
`oracle_text`	`TEXT`	LARGE — never `SELECT *`
`power`, `toughness`	`VARCHAR(10)`	MTG creatures
`image_url`, `stock_image_url`	`TEXT`
`current_price`, `market_price`	`DECIMAL(10,2)`
`scryfall_id`	`VARCHAR(255) UNIQUE`	Use for dedupe on MTG import
`verified`	`BOOLEAN` default `false`	Admin-edited cards
`quantity`	`INTEGER` default `0`	Unused; consider dropping — quantity lives in `user_cards`
`favorited`	`BOOLEAN` default `false`	Unused; favorites live in `user_favorites`
`created_at`, `updated_at`	`TIMESTAMP` default now

user_cards

Column	Type	Notes
`id`	`SERIAL PK`
`user_id`	`INTEGER FK users(id) ON DELETE CASCADE`
`card_id`	`INTEGER FK cards(id) ON DELETE CASCADE`
`quantity`	`INTEGER` default `1`
`condition`	`VARCHAR(50)` default `'NM'`	NM / LP / MP / HP / DMG
`is_foil`	`BOOLEAN` default `false`
`notes`	`TEXT`
		UNIQUE(user_id, card_id, is_foil)

user_favorites

Column	Type	Notes
`id`	`SERIAL PK`
`user_id`, `card_id`	FKs cascade
`created_at`	`TIMESTAMP`
		UNIQUE(user_id, card_id) (verify constraint exists)

collections

Column	Type	Notes
`id`	`SERIAL PK`
`user_id`	`INTEGER FK users(id) ON DELETE CASCADE`
`name`	`VARCHAR(255) NOT NULL`
`description`	`TEXT`
`is_public`	`BOOLEAN` default `false`	Used by `withCollectionPermission('viewer')`
`slug`	`VARCHAR(100) UNIQUE`	Format constraint: lowercase kebab, ≤50 chars
`image`	`TEXT`	Cover image
`visibility`	`VARCHAR(20)` default `'private'`	Coexists with `is_public`; verify single source of truth
`tcg`	`VARCHAR(50)` default `'MTG'`
`tags`	`TEXT`	Comma-separated; consider migrating to JSONB array
`is_system_collection`	`BOOLEAN` default `false`	E.g. "All My Cards" auto-collection
`created_at`, `updated_at`	`TIMESTAMP`

collection_cards

Column	Type	Notes
`id`	`SERIAL PK`
`collection_id`, `card_id`	FKs cascade
`quantity`	`INTEGER` default `1`
		UNIQUE(collection_id, card_id)

collection_permissions

Column	Type	Notes
`id`	`SERIAL PK`
`collection_id`, `user_id`	FKs cascade
`role`	`VARCHAR`	`'viewer' \| 'editor' \| 'owner'`
`status`	`VARCHAR`	`'pending' \| 'active' \| 'declined'` — used by invite flow
`created_at`	`TIMESTAMP`

collection_activity

Column	Type	Notes
`id`	`SERIAL PK`
`collection_id`, `user_id`	FKs
`action`	`VARCHAR`	e.g. `'card_added'`, `'permission_granted'`
`details`	`JSONB`
`created_at`	`TIMESTAMP`

decks

Column	Type	Notes
`id`	`SERIAL PK`
`user_id`	FK
`name`, `description`	text
`game`	`VARCHAR(50)`
`is_public`	`BOOLEAN` default `false`
`created_at`, `updated_at`	`TIMESTAMP`

deck_cards

Column	Type	Notes
`id`	`SERIAL PK`
`deck_id`, `card_id`	FKs cascade
`quantity`	`INTEGER` default `1`
		UNIQUE(deck_id, card_id)

user_settings (split from users.*; verify which is canonical)

Defined in add-user-profile-fields.js. Mirrors several users.* columns — there's redundancy that needs to be reconciled.

user_avatars

Tracks uploaded avatar history. Older avatars are typically deleted from blob storage; verify the cleanup job runs.

Known schema smells

Two users avatar columns — profile_image_url and avatar_url. Pick one.
Two collection-visibility flags — collections.is_public (BOOLEAN) and collections.visibility (VARCHAR). Pick one.
cards.quantity + cards.favorited — these belong on user_cards / user_favorites, not the catalog. Drop them.
user_settings ↔ users.* — split-brain. Reconcile.
No formal constraints on enums — role, condition, theme, language, game, visibility are all VARCHAR. Consider CHECK constraints or proper ENUMs.
collections.tags is TEXT — should be JSONB or a join table.

Regeneration

For new schema changes (post-migration-tool), look at migrations/<timestamp>_<slug>.js files and update the matching table section here in the same PR. For the historical state captured before the migration tool landed:

rg "ALTER TABLE|CREATE TABLE|ADD COLUMN" scripts/ migrations/

…then update this file by hand. A npm run schema:map script regenerated from the migration history is in .convoys/.

8.1 KiB Raw Blame History