Commit graph

9 commits

Author SHA1 Message Date
Randall Stillwell
a000c59ec4 fix(migrate): pre-check pg_constraint instead of catching duplicate_object
ADD CONSTRAINT UNIQUE creates a supporting index under the hood; when
the index name already exists from initial-schema's inline UNIQUE,
Postgres raises SQLSTATE 42P07 (duplicate_table), not 42710
(duplicate_object) — so the EXCEPTION block didn't catch it and CI's
Migrations apply gate failed with `relation
"user_cards_user_id_card_id_is_foil_key" already exists`.

Swap to a pg_constraint pre-check: bulletproof against both SQLSTATEs
without overreaching to WHEN OTHERS.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 08:08:47 -05:00
Randall Stillwell
75ba7c751d feat(migrations): reconcile user_cards UNIQUE constraint per Option A (B6)
Convoy: reconcile-historical-add-scripts Brief 6.

Operator decision on Finding 1 (ratified 2026-06-14): Option A — keep
the canonical 3-col UNIQUE(user_id, card_id, is_foil) declared by
migrations/1779853647564_initial-schema.js. Drop / treat-as-no-op the
stricter 2-col UNIQUE(user_id, card_id) that the historical
scripts/fix-user-cards-constraints.js job would have installed. Foil
and non-foil copies of the same card are semantically separate rows.

Defensive idempotent shape; safe against all three prod states (fresh
Neon branch, long-lived env that never ran the script, long-lived env
that did run it). down() is a hard-stub throw — re-installing the
2-col constraint would forbid the foil distinction runtime code
relies on.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 08:03:12 -05:00
Randall Stillwell
67073aab7f feat(catalog): Scryfall bulk data import + Tagger community tags
Add full Scryfall bulk data pipeline:

- Migration: 13 new columns on `cards` (oracle_id, illustration_id,
  color_identity, keywords, legalities, flavor_text, artist, released_at,
  layout, edhrec_rank, reserved, reprint, finishes) with GIN indexes
  for JSONB search.
- Migration: `tags` + `card_tags` tables for Tagger community data.
- Script: `bulk-import-scryfall.js` — downloads Oracle Cards bulk file
  (168 MB) and upserts all 36k+ MTG cards with rich metadata.
- Script: `import-scryfall-tags.js` — imports oracle tags (4.5k tags,
  227k taggings) and art tags (11k tags, 458k taggings).
- Lib: `bulk-sync.js` — runtime bulk sync callable from the admin API.
- Admin UI: mode toggle (incremental vs bulk) on catalog sync panel.

Enables Commander deck validation (color_identity), format legality
checks, keyword search, EDHREC popularity ranking, and functional
card tagging ("removal", "ramp", "draw") for deck building assistance.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 07:51:39 -05:00
varutasu
e78f78e3f6
ci: run migrations against Postgres service container in CI (#64)
* ci: run migrations against Postgres service container in CI

Add a migrate job that applies node-pg-migrate against an ephemeral
Postgres 16 service container so broken migrations fail at PR time.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(migrations): retimestamp scan tables after initial-schema.

Fresh CI/postgres runs failed because 1748365200000 sorted before
initial-schema. Renamed to 1779853647566 with IF NOT EXISTS guards intact.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(migrations): ensure is_system_collection exists before description backfill.

Fresh CI Postgres runs initial-schema without this column (added historically
via scripts); ADD COLUMN IF NOT EXISTS makes the data migration safe on new
and existing envs.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-02 11:01:18 -05:00
varutasu
00193651aa
Backfill system collection descriptions to current vocabulary. (#60)
Existing is_system_collection rows may carry stale copy; align them with VOCAB.SYSTEM_COLLECTION_SEED_DESCRIPTION so UI matches new signups.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-02 00:42:23 -05:00
varutasu
673af83519
feat(scanner): persist scan captures to Blob (Brief 3) (#44)
Upload confirmed scan frames to Vercel Blob and store the URL on user_cards
when routing to owned cards, completing the redesign-scanner-flow convoy.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-27 13:57:26 -05:00
varutasu
d798e284c3
feat(scanner): AI Gateway vision + Layer-1 Tesseract/pg_trgm OCR (#38)
Route Layer-2 identification through Vercel AI Gateway (AI_GATEWAY_API_KEY,
default google/gemini-2.5-flash-lite). Add Layer-1 browser Tesseract name-strip
OCR with pg_trgm fuzzy catalog match via /api/cards/identify-by-text before
escalating to vision.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-27 12:59:59 -05:00
varutasu
e81dd49752
feat(scanner): server-side scan pipeline (convoy #2) (#35)
* feat(scanner): move card identification server-side (convoy #2)

Replace browser Gemini/OCR with POST /api/scan/identify, add card_submissions
review queue, remove user-writable cards INSERT, and surface disambiguation
when catalog matching is ambiguous.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: allowlist server-only lib/scan-gemini.js in LLM key gate

The scan pipeline helper lives under lib/ but is imported exclusively
from pages/api/scan/identify — exclude it from the client-side URL scan.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-27 08:47:05 -05:00
varutasu
de9f3348f6
feat(infra): adopt node-pg-migrate + backfill initial schema migration (#32)
Closes P1 #11 of .convoys/ship-readiness.md (launch sequence step 7) —
"No migration tool — scripts/add-*.js graveyard". Schema changes
post-this-convoy ship as node-pg-migrate migrations under migrations/
at the repo root; the legacy 27 scripts/add-*.js / scripts/fix-*.js /
scripts/seed-*.js jobs remain append-only history per the no-go-zones
rule.

Decisions (full record in .convoys/migration-tool.md § Decisions):

D1 — Tool: node-pg-migrate@^8. Rejected drizzle-kit / prisma migrate /
kysely because each forces broader TypeScript surface than AGENTS.md
Gotcha #9 allows (TS is a devDep only). node-pg-migrate is
JavaScript-native, raw-SQL-friendly via pgm.sql(), and ESM-clean for
the post-bump-next-js "type": "module" repo. Brings pg@^8.21.0 as a
peer dep (dev-only; never loaded in the Next.js bundle).

D2 — Migrations directory: migrations/ at the repo root. Separates
the tool-wrapped artifacts from the historical scripts/migrations/
placeholder folder (which housed the lone pre-tool
2026-05-24-rename-admin-email.js migration and remains preserved for
the audit trail). Matches node-pg-migrate's default flag.

D3 — Tracking table: default pgmigrations (no name collision with
the existing 7-table bootstrap; zero CLI noise).

D4 — Backfill strategy: hand-translate scripts/setup-neon-db.js's
DDL into the initial migration verbatim. Each await sql`...` block
becomes one pgm.sql(`...`) call. Each CREATE uses IF NOT EXISTS, so
the migration is idempotent against fresh AND pre-existing envs —
re-running setup-db on an env that already has the schema is a no-op
DDL-wise (only records the pgmigrations row). Documented assumption:
prod has drifted via the 27 historical add-*.js scripts; reconciling
those into the migration history is the queued
reconcile-historical-add-scripts follow-up convoy.

D5 — Bootstrap reconciliation: split. setup-neon-db.js now (1)
validates ADMIN_INITIAL_PASSWORD + POSTGRES_URL, (2) spawns
`npm run migrate up` via child_process with stdio inherited, (3)
seeds the admin row with ON CONFLICT (email) DO NOTHING. The seven
DDL blocks are deleted from setup-neon-db.js; success/error message
copy is updated to mention the migration step explicitly.

D6 — CI integration: defer. Wiring a CI job that runs migrate up
against a test DB needs either a dedicated Neon branch + secret OR a
Postgres service container; both are real work. Surface as
wire-migrate-into-ci follow-up. Risk acknowledged in
.convoys/migration-tool.md § R3.

D7 — Down-migration on the initial backfill: hard stub. Rolling back
the initial schema would drop every user / card / collection / deck
row in the DB. The stub throws with a long-form error pointing at
the recommended alternative (branch the Neon database + forward-apply).
Future migrations that touch one of the seven bootstrap tables write
their own dated migration with a real down().

Verification (pre-PR):
- npm run lint → 128 problems (baseline preserved, zero regression;
  migration file is lint-clean, no new ignore patterns)
- npm run test:run → 21/21 pass
- node --check on migrations/1779853647564_initial-schema.js + on
  scripts/setup-neon-db.js → exit 0
- Module load + down() throw verified via dynamic import
- npm run migrate -- --help reaches the node-pg-migrate CLI through
  the wrapper

Live verification against a Neon branch is deferred (no throwaway
branch available); the operator's optional post-merge sequence is
documented in .convoys/migration-tool.md § Operator runbook.

See .convoys/migration-tool.md § Follow-ups for the queued
wire-migrate-into-ci / reconcile-historical-add-scripts /
retire-graveyard-scripts-after-audit / audit-node-pg-migrate-transitive-deps
/ add-migration-template follow-up convoys.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-26 23:01:58 -05:00