Commit graph

8 commits

Author SHA1 Message Date
Randall Stillwell
ec9bb2b93e feat(catalog): unified multi-game bulk sync + schema map update
Ship Pokemon and Lorcana bulk import libs/scripts, unified weekly cron
sync across MTG/Pokemon/Lorcana with per-game error isolation and
catalog_sync_log telemetry. Admin UI adds Unified/Incremental/Bulk MTG modes.

- Rename reconcile migrations to 1781442330* timestamps so they apply
  after bulk-data migrations without node-pg-migrate ordering conflicts
- Add Lorcana set-code normalization + orphan cleanup migrations
- Drop stricter user_cards_user_card_unique (keep 3-column foil unique)
- Update docs/SCHEMA_MAP.md for tags, card_tags, catalog_sync_log, bulk columns

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 08:22:19 -05:00
Randall Stillwell
67073aab7f feat(catalog): Scryfall bulk data import + Tagger community tags
Add full Scryfall bulk data pipeline:

- Migration: 13 new columns on `cards` (oracle_id, illustration_id,
  color_identity, keywords, legalities, flavor_text, artist, released_at,
  layout, edhrec_rank, reserved, reprint, finishes) with GIN indexes
  for JSONB search.
- Migration: `tags` + `card_tags` tables for Tagger community data.
- Script: `bulk-import-scryfall.js` — downloads Oracle Cards bulk file
  (168 MB) and upserts all 36k+ MTG cards with rich metadata.
- Script: `import-scryfall-tags.js` — imports oracle tags (4.5k tags,
  227k taggings) and art tags (11k tags, 458k taggings).
- Lib: `bulk-sync.js` — runtime bulk sync callable from the admin API.
- Admin UI: mode toggle (incremental vs bulk) on catalog sync panel.

Enables Commander deck validation (color_identity), format legality
checks, keyword search, EDHREC popularity ranking, and functional
card tagging ("removal", "ramp", "draw") for deck building assistance.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 07:51:39 -05:00
Randall Stillwell
59e2ca4ce4 fix(catalog-sync): add retry logic and pagination for Scryfall API
Scryfall was returning 429/503 transiently, causing catalog sync to
fail immediately with no recovery. Adds exponential-backoff retry
(3 attempts) for rate-limit and service-unavailable responses in both
the set discovery and card import paths. Also adds proper pagination
support for sets with 175+ cards and URL-encodes set codes.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-13 10:49:50 -05:00
varutasu
c51ec6a04c
Normalize collector numbers in catalog match and harden scanner adds. (#53)
Share card-number normalization across reconcile and identify paths, retry set/name matches when OCR uses leading-zero collector numbers, and extend in-flight locks to all scanner destination actions with disabled Mark Owned feedback.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-28 14:45:55 -05:00
varutasu
174a370fc3
Switch Pokémon catalog import to pokemon-tcg-data on GitHub. (#52)
Replace pokemontcg.io API discovery and import with raw JSON from PokemonTCG/pokemon-tcg-data; format collector numbers as number/printedTotal and drop the API key dependency for catalog sync.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-28 11:56:01 -05:00
varutasu
cf5c0558f1
Auto-link pending scan submissions after catalog sync imports. (#51)
When a set lands via runCatalogSync, match pending card_submissions by set/name/number to catalog rows and approve them with promoted_card_id instead of leaving them in the admin queue.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-28 09:47:59 -05:00
varutasu
bec0a7abbd
Prioritize newest missing sets in catalog sync queue. (#49)
The cron was importing oldest MTG sets first and never reaching recent Pokémon releases like Perfect Order; merge MTG and Pokémon by release date descending instead.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-27 21:49:58 -05:00
varutasu
0a47362103
feat(catalog): weekly Vercel Cron sync for MTG and Pokémon sets (#48)
Extract shared import logic into lib/card-import, discover missing sets via
Scryfall/Pokémon TCG APIs, and expose GET /api/cron/sync-catalog protected
by CRON_SECRET (max 3 sets/run, paced imports).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-27 14:59:59 -05:00