de9f3348f6
Closes P1 #11 of .convoys/ship-readiness.md (launch sequence step 7) — "No migration tool — scripts/add-*.js graveyard". Schema changes post-this-convoy ship as node-pg-migrate migrations under migrations/ at the repo root; the legacy 27 scripts/add-*.js / scripts/fix-*.js / scripts/seed-*.js jobs remain append-only history per the no-go-zones rule. Decisions (full record in .convoys/migration-tool.md § Decisions): D1 — Tool: node-pg-migrate@^8. Rejected drizzle-kit / prisma migrate / kysely because each forces broader TypeScript surface than AGENTS.md Gotcha #9 allows (TS is a devDep only). node-pg-migrate is JavaScript-native, raw-SQL-friendly via pgm.sql(), and ESM-clean for the post-bump-next-js "type": "module" repo. Brings pg@^8.21.0 as a peer dep (dev-only; never loaded in the Next.js bundle). D2 — Migrations directory: migrations/ at the repo root. Separates the tool-wrapped artifacts from the historical scripts/migrations/ placeholder folder (which housed the lone pre-tool 2026-05-24-rename-admin-email.js migration and remains preserved for the audit trail). Matches node-pg-migrate's default flag. D3 — Tracking table: default pgmigrations (no name collision with the existing 7-table bootstrap; zero CLI noise). D4 — Backfill strategy: hand-translate scripts/setup-neon-db.js's DDL into the initial migration verbatim. Each await sql`...` block becomes one pgm.sql(`...`) call. Each CREATE uses IF NOT EXISTS, so the migration is idempotent against fresh AND pre-existing envs — re-running setup-db on an env that already has the schema is a no-op DDL-wise (only records the pgmigrations row). Documented assumption: prod has drifted via the 27 historical add-*.js scripts; reconciling those into the migration history is the queued reconcile-historical-add-scripts follow-up convoy. D5 — Bootstrap reconciliation: split. setup-neon-db.js now (1) validates ADMIN_INITIAL_PASSWORD + POSTGRES_URL, (2) spawns `npm run migrate up` via child_process with stdio inherited, (3) seeds the admin row with ON CONFLICT (email) DO NOTHING. The seven DDL blocks are deleted from setup-neon-db.js; success/error message copy is updated to mention the migration step explicitly. D6 — CI integration: defer. Wiring a CI job that runs migrate up against a test DB needs either a dedicated Neon branch + secret OR a Postgres service container; both are real work. Surface as wire-migrate-into-ci follow-up. Risk acknowledged in .convoys/migration-tool.md § R3. D7 — Down-migration on the initial backfill: hard stub. Rolling back the initial schema would drop every user / card / collection / deck row in the DB. The stub throws with a long-form error pointing at the recommended alternative (branch the Neon database + forward-apply). Future migrations that touch one of the seven bootstrap tables write their own dated migration with a real down(). Verification (pre-PR): - npm run lint → 128 problems (baseline preserved, zero regression; migration file is lint-clean, no new ignore patterns) - npm run test:run → 21/21 pass - node --check on migrations/1779853647564_initial-schema.js + on scripts/setup-neon-db.js → exit 0 - Module load + down() throw verified via dynamic import - npm run migrate -- --help reaches the node-pg-migrate CLI through the wrapper Live verification against a Neon branch is deferred (no throwaway branch available); the operator's optional post-merge sequence is documented in .convoys/migration-tool.md § Operator runbook. See .convoys/migration-tool.md § Follow-ups for the queued wire-migrate-into-ci / reconcile-historical-add-scripts / retire-graveyard-scripts-after-audit / audit-node-pg-migrate-transitive-deps / add-migration-template follow-up convoys. Co-authored-by: Cursor <cursoragent@cursor.com>
62 lines
3.4 KiB
Text
62 lines
3.4 KiB
Text
---
|
|
description: Neon Postgres conventions + the until-we-have-migrations workflow
|
|
globs: pages/api/**/*.js,lib/database.js,scripts/**/*.js
|
|
---
|
|
|
|
# DB + schema
|
|
|
|
## Clients
|
|
|
|
Two are installed (`@vercel/postgres` + `@neondatabase/serverless`); they point at the same Neon Postgres. New code uses `@vercel/postgres` (tagged-template SQL).
|
|
|
|
```js
|
|
import { sql } from '@vercel/postgres';
|
|
|
|
const { rows } = await sql`
|
|
SELECT id, name, set_name
|
|
FROM cards
|
|
WHERE game = ${game}
|
|
AND set_code = ${setCode}
|
|
LIMIT 50
|
|
`;
|
|
```
|
|
|
|
**Never** use `lib/database.js`'s `db.query(string, params)` API for new code — it interpolates params into a string and then calls `sql.unsafe()`, which is a SQL-injection vector. Marked for removal in `.convoys/`.
|
|
|
|
## Schema source of truth
|
|
|
|
`migrations/` at the repo root owns the schema (post-`migration-tool` convoy, 2026-05-26). The initial backfill `migrations/1779853647564_initial-schema.js` reproduces the 7-table bootstrap shape verbatim from `scripts/setup-neon-db.js`. The tool is `node-pg-migrate@^8`; tracking table is the default `pgmigrations`. To add a column:
|
|
|
|
- **Adding a column**: `npm run migrate create add-<table>-<column> -- -j js`, edit the generated file in `migrations/`, then `npm run migrate up` to apply.
|
|
- **Document** the change in `docs/SCHEMA_MAP.md`.
|
|
- **Never** edit a migration that has already been applied (the `pgmigrations` row pins the file's contents — mutating it silently corrupts every env that has the prior version recorded).
|
|
- **Never** edit historical `scripts/add-*.js` / `scripts/fix-*.js` / `scripts/seed-*.js` jobs — they're append-only history per the no-go-zones rule.
|
|
|
|
`scripts/setup-neon-db.js` now spawns `npm run migrate up` before seeding the admin user; do not put DDL back into it.
|
|
|
|
## Tables (current)
|
|
|
|
| Table | Owner | Notes |
|
|
| --- | --- | --- |
|
|
| `users` | core | `(id, email UNIQUE, password, role, created_at, …)` |
|
|
| `cards` | core | Big — `oracle_text TEXT`, `colors JSONB`. Don't `SELECT *`. |
|
|
| `user_cards` | per-user | `(user_id, card_id, quantity, condition, is_foil)` — UNIQUE on tuple |
|
|
| `collections` | per-user | `is_public BOOLEAN`, `slug`, `tags`, `image_url`, `system_collection` |
|
|
| `collection_cards` | join | `(collection_id, card_id, quantity)` UNIQUE |
|
|
| `collection_permissions` | per-user | `(collection_id, user_id, role, status)` — `viewer\|editor\|owner` |
|
|
| `collection_activity` | log | `(collection_id, user_id, action, details JSONB, created_at)` |
|
|
| `decks` / `deck_cards` | per-user | Mirror of collections |
|
|
| `favorites` | per-user | `(user_id, card_id)` |
|
|
| `invitations` | per-collection | Pending share requests |
|
|
|
|
Full map: [`docs/SCHEMA_MAP.md`](../../docs/SCHEMA_MAP.md). Regenerate by re-reading `setup-neon-db.js` + every `add-*.js` script that's been run.
|
|
|
|
## Indexing reminders
|
|
|
|
- `cards.scryfall_id` is UNIQUE — use it for dedupe on import.
|
|
- `users.email` is UNIQUE — case-insensitive collation NOT set; lowercase before query/insert.
|
|
- `collections.slug` should be UNIQUE per user; verify with `lib/slug-utils.js::generateUniqueSlug` before insert.
|
|
|
|
## Transactions
|
|
|
|
Neon HTTP doesn't support multi-statement transactions across separate `sql` calls — each call is its own connection. For multi-table writes that need atomicity, use Neon's `sql.transaction([query1, query2])` array form OR refactor to a single SQL statement with CTEs. The current codebase has several non-atomic multi-step inserts that should be flagged.
|