diff --git a/.cursor/rules/api-routes.mdc b/.cursor/rules/api-routes.mdc index 9846cf5..3b99133 100644 --- a/.cursor/rules/api-routes.mdc +++ b/.cursor/rules/api-routes.mdc @@ -111,7 +111,7 @@ await logCollectionActivity(collectionId, userId, 'card_added', { cardId, quanti | `search` | 60 | 1 min | IP | `/api/users/search`, `/api/cards/search` | `checkSearchRateLimit(req)` | | `upload` | 10 | 1 hour | user | `/api/user/avatar` | `checkUploadRateLimit(req, userId)` | | `generate` | 5 | 1 hour | user | `/api/user/avatar/generate` | `checkGenerateRateLimit(req, userId)` | -| `import` | 5 | 1 hour | user | `/api/cards/import-mtg`, `/api/cards/import-pokemon`, `/api/cards/import-lorcana` | `checkImportRateLimit(req, userId)` | +| `import` | 5 | 1 hour | user | `/api/cards/import-mtg`, `/api/cards/import-pokemon` | `checkImportRateLimit(req, userId)` | | `scan` | 5 | 1 min | user | `/api/scan/identify` | `checkScanRateLimit(req, userId)` | **Verbatim call shape** (identical across all six classes — only the helper name and the optional `userId` argument differ): @@ -145,7 +145,7 @@ export default async function handler(req, res) { 1. **Method check first.** Reject the wrong verb with 405 before doing any limiter work. 2. **Auth check before any user-keyed limiter.** `extractUserIdentifier(userId)` THROWS when `userId` is null/undefined/empty (defensive). For `upload`, `generate`, `import`, and `scan`, the handler MUST call `getUserFromRequest(req)` (or equivalent JWT verification) and confirm a non-null user BEFORE calling the limiter. Wrong order = anonymous user bypasses (the THROW surfaces immediately during dev; do not catch and silently fall back to IP). 3. **For IP-keyed limiters (`auth`, `search`), gate placement is flexible** — either at the top of the handler (after the method check) or after a separate auth check that the route happens to also have (e.g. `users/search` JWT-verifies before rate-limiting, both are correct). The limiter only needs `req` for IP extraction. -4. **Admin-role check, if applicable, goes between auth and rate-limit.** Used by all three `/api/cards/import-*` routes: `if (user.role !== 'admin') return res.status(403).json({ error: 'Admin access required' })` sits between the `if (!user)` 401 and the import rate-limit call. +4. **Admin-role check, if applicable, goes between auth and rate-limit.** Used by both `/api/cards/import-*` routes: `if (user.role !== 'admin') return res.status(403).json({ error: 'Admin access required' })` sits between the `if (!user)` 401 and the import rate-limit call. **Identifier extraction:** diff --git a/AGENTS.md b/AGENTS.md index a34a4fd..5c3fb9b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -101,13 +101,13 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560 | `checkSearchRateLimit(req)` | `search` | 60 / 1 min | IP | `deckhearth:search` | `/api/users/search`, `/api/cards/search` | | `checkUploadRateLimit(req, userId)` | `upload` | 10 / 1 hour | user | `deckhearth:upload` | `/api/user/avatar` | | `checkGenerateRateLimit(req, userId)` | `generate` | 5 / 1 hour | user | `deckhearth:generate` | `/api/user/avatar/generate` | - | `checkImportRateLimit(req, userId)` | `import` | 5 / 1 hour | user (admin-only) | `deckhearth:import` | `/api/cards/import-mtg`, `/api/cards/import-pokemon`, `/api/cards/import-lorcana` | + | `checkImportRateLimit(req, userId)` | `import` | 5 / 1 hour | user (admin-only) | `deckhearth:import` | `/api/cards/import-mtg`, `/api/cards/import-pokemon` | Prefixes renamed `tcgvault:*` → `deckhearth:*` in `pick-a-name` (squash `9abbab6`, 2026-05-24); accepted one-time per-15-min / per-1-hour counter reset; existing Upstash state at `tcgvault:*` keys is now stale and will TTL out naturally. All five return the same `{ allowed, remaining, reset }` shape; on `!allowed`, set `Retry-After: Math.ceil((reset - Date.now()) / 1000)` and return 429 with the uniform message `'Too many attempts. Try again later.'` (per-class variation would fingerprint the limits to an attacker — explicitly rejected). - **Defensive THROW pattern.** `extractUserIdentifier(userId)` THROWS with a named error when `userId` is `null` / `undefined` / `''` / `NaN`. Surfaces gate-ordering bugs at dev time rather than silently falling back to IP and converting a per-user limit into a per-IP limit (which would lock household members out for one user's behavior). Numeric `0` is intentionally accepted (returns `'user:0'`) for forward-compat. **Gate-ordering rule: per-user rate-limit gates (`upload`, `generate`, `import`) MUST sit AFTER the auth check.** For the three `/api/cards/import-*` routes, the ordering is also `auth → admin-role check (403 if not admin) → rate-limit`; the admin-role check sits between auth and rate-limit. IP-keyed gates (`auth`, `search`) can sit anywhere after the method check. + **Defensive THROW pattern.** `extractUserIdentifier(userId)` THROWS with a named error when `userId` is `null` / `undefined` / `''` / `NaN`. Surfaces gate-ordering bugs at dev time rather than silently falling back to IP and converting a per-user limit into a per-IP limit (which would lock household members out for one user's behavior). Numeric `0` is intentionally accepted (returns `'user:0'`) for forward-compat. **Gate-ordering rule: per-user rate-limit gates (`upload`, `generate`, `import`) MUST sit AFTER the auth check.** For the two `/api/cards/import-*` routes, the ordering is also `auth → admin-role check (403 if not admin) → rate-limit`; the admin-role check sits between auth and rate-limit. IP-keyed gates (`auth`, `search`) can sit anywhere after the method check. Adding a sixth class is a one-line `LIMITER_CONFIG` addition + one new exported function (no `init()` restructuring needed). Tuning an existing class is a one-line `LIMITER_CONFIG` edit. The full verbatim call shape + gate-ordering rules + identifier-extraction documentation live in `.cursor/rules/api-routes.mdc` § Rate limiting. diff --git a/pages/api/cards/import-lorcana.js b/pages/api/cards/import-lorcana.js deleted file mode 100644 index 705e266..0000000 --- a/pages/api/cards/import-lorcana.js +++ /dev/null @@ -1,185 +0,0 @@ -import { sql } from '@vercel/postgres'; -import { getUserFromRequest } from '../../../lib/permission-middleware'; -import { checkImportRateLimit } from '../../../lib/rate-limit.js'; - -// Helper function to delay execution -const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms)); - -// Helper function to fetch with retry logic -async function fetchWithRetry(url, maxRetries = 3, delayMs = 2000) { - for (let attempt = 1; attempt <= maxRetries; attempt++) { - try { - const response = await fetch(url, { - headers: { - 'User-Agent': 'Deck-Hearth/1.0', - 'Accept': 'application/json' - }, - timeout: 30000 // 30 second timeout - }); - - if (response.ok) { - return response; - } - - // If it's a 404, don't retry - if (response.status === 404) { - throw new Error(`Set not found: ${response.status}`); - } - - // If it's a 504 or 503, wait longer before retry - if (response.status === 504 || response.status === 503) { - console.log(`Attempt ${attempt}: Got ${response.status}, waiting ${delayMs * attempt}ms before retry...`); - await delay(delayMs * attempt); - continue; - } - - throw new Error(`Lorcana API error: ${response.status}`); - } catch (error) { - if (attempt === maxRetries) { - throw error; - } - console.log(`Attempt ${attempt} failed:`, error.message); - await delay(delayMs * attempt); - } - } -} - -export default async function handler(req, res) { - if (req.method !== 'POST') { - return res.status(405).json({ error: 'Method not allowed' }); - } - - const user = await getUserFromRequest(req); - if (!user) { - return res.status(401).json({ error: 'Authentication required' }); - } - if (user.role !== 'admin') { - return res.status(403).json({ error: 'Admin access required' }); - } - - const { allowed, reset } = await checkImportRateLimit(req, user.userId); - if (!allowed) { - res.setHeader('Retry-After', Math.ceil((reset - Date.now()) / 1000)); - return res.status(429).json({ error: 'Too many attempts. Try again later.' }); - } - - try { - const { setCode } = req.body; - - if (!setCode) { - return res.status(400).json({ error: 'Set code is required' }); - } - - console.log(`Starting import for Lorcana set: ${setCode}`); - - // Map set codes to Lorcast set codes - const setCodeMap = { - 'tfc': '1', // The First Chapter - 'rotf': '2', // Rise of the Floodborn - 'ink': '3' // Into the Inklands - }; - - const lorcastSetCode = setCodeMap[setCode]; - if (!lorcastSetCode) { - return res.status(400).json({ error: `Unknown set code: ${setCode}` }); - } - - // Fetch cards from Lorcast API - const response = await fetchWithRetry(`https://api.lorcast.com/v0/cards/search?q=set:${lorcastSetCode}&unique=prints`); - - const data = await response.json(); - const cards = data.results || []; - - if (cards.length === 0) { - return res.status(200).json({ - success: true, - message: `No cards found for set ${setCode}`, - imported: 0, - skipped: 0, - total: 0 - }); - } - - console.log(`Found ${cards.length} cards for set ${setCode}`); - - let importedCount = 0; - let skippedCount = 0; - - for (const card of cards) { - try { - // Check if card already exists - const existingCard = await sql` - SELECT id FROM cards WHERE scryfall_id = ${card.id} - `; - - if (existingCard.rows.length > 0) { - skippedCount++; - continue; - } - - // Extract price data - let currentPrice = null; - if (card.prices?.usd) { - currentPrice = parseFloat(card.prices.usd); - } else if (card.prices?.usd_foil) { - currentPrice = parseFloat(card.prices.usd_foil); - } - - // Determine rarity - let rarity = card.rarity || 'Common'; - if (rarity === 'Super_rare') { - rarity = 'Super Rare'; - } - - // Extract card type and text - const cardType = card.type?.[0] || 'Character'; - const cardText = card.text || null; - const flavorText = card.flavor_text || null; - - // Extract stats - const strength = card.strength || null; - const willpower = card.willpower || null; - const lore = card.lore || null; - - // Insert card into database - await sql` - INSERT INTO cards ( - name, set_name, set_code, card_number, rarity, game, - mana_cost, cmc, card_type, colors, oracle_text, - power, toughness, image_url, stock_image_url, - current_price, market_price, scryfall_id, verified - ) VALUES ( - ${card.name}, ${card.set.name}, ${card.set.code}, ${card.collector_number}, - ${rarity}, 'Lorcana', ${card.cost}, ${card.cost}, ${cardType}, - ${JSON.stringify(card.ink ? [card.ink] : [])}, ${cardText || flavorText}, - ${strength}, ${willpower || lore}, - ${card.image_uris?.digital?.large || card.image_uris?.digital?.small || null}, ${card.image_uris?.digital?.small || null}, - ${currentPrice}, null, ${card.id}, true - ) - `; - - importedCount++; - } catch (error) { - console.error(`Error importing card ${card.name}:`, error); - skippedCount++; - } - } - - console.log(`Import completed for set ${setCode}: ${importedCount} imported, ${skippedCount} skipped`); - - res.status(200).json({ - success: true, - message: `Import completed for set ${setCode}`, - imported: importedCount, - skipped: skippedCount, - total: cards.length - }); - - } catch (error) { - console.error('Card import error:', error); - res.status(500).json({ - error: 'Import failed', - details: error.message - }); - } -} \ No newline at end of file diff --git a/scripts/README.md b/scripts/README.md index 2a1f0a6..d253733 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -17,7 +17,6 @@ npm run import-popular **What it imports:** - **Magic: The Gathering**: ~100+ sets from Alpha to recent releases - **Pokemon**: ~100+ sets from Base Set to current Scarlet & Violet -- **Lorcana**: All 3 available sets **Estimated time:** 2-4 hours (depending on API response times) @@ -32,7 +31,6 @@ npm run import-all **What it imports:** - **Magic: The Gathering**: 100+ sets (Alpha to current) - **Pokemon**: 100+ sets (Base Set to current) -- **Lorcana**: All available sets **Estimated time:** 4-8 hours (depending on API response times) @@ -171,13 +169,11 @@ You can stop the script with `Ctrl+C` and restart it later. The scripts will sta - `POST /api/cards/import-mtg` - Magic: The Gathering imports - `POST /api/cards/import-pokemon` - Pokemon imports -- `POST /api/cards/import-lorcana` - Lorcana imports ## Data Sources - **Magic: The Gathering**: Scryfall API - **Pokemon**: [PokemonTCG/pokemon-tcg-data](https://github.com/PokemonTCG/pokemon-tcg-data) on GitHub (sets + per-set JSON; images from linked CDNs). Optional override: `POKEMON_TCG_DATA_BASE_URL` (defaults to `master` branch raw URLs). The legacy Pokemon TCG API is no longer used by catalog sync. -- **Lorcana**: Lorcana API (limited availability) ## Performance Notes diff --git a/scripts/import-lorcana.js b/scripts/import-lorcana.js deleted file mode 100644 index a3eeefa..0000000 --- a/scripts/import-lorcana.js +++ /dev/null @@ -1,160 +0,0 @@ -import fetch from 'node-fetch'; -import fs from 'fs'; - -// Lorcana sets to import -const LORCANA_SETS = [ - { code: 'tfc', name: 'The First Chapter', lorcastCode: '1' }, - { code: 'rotf', name: 'Rise of the Floodborn', lorcastCode: '2' }, - { code: 'ink', name: 'Into the Inklands', lorcastCode: '3' } -]; - -// Helper function to delay execution -const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms)); - -// Helper function to fetch with retry logic -async function fetchWithRetry(url, maxRetries = 3, delayMs = 2000) { - for (let attempt = 1; attempt <= maxRetries; attempt++) { - try { - const response = await fetch(url, { - headers: { - 'User-Agent': 'Deck-Hearth/1.0', - 'Accept': 'application/json' - }, - timeout: 30000 // 30 second timeout - }); - - if (response.ok) { - return response; - } - - // If it's a 404, don't retry - if (response.status === 404) { - throw new Error(`Set not found: ${response.status}`); - } - - // If it's a 504 or 503, wait longer before retry - if (response.status === 504 || response.status === 503) { - console.log(`Attempt ${attempt}: Got ${response.status}, waiting ${delayMs * attempt}ms before retry...`); - await delay(delayMs * attempt); - continue; - } - - throw new Error(`Lorcana API error: ${response.status}`); - } catch (error) { - if (attempt === maxRetries) { - throw error; - } - console.log(`Attempt ${attempt} failed:`, error.message); - await delay(delayMs * attempt); - } - } -} - -async function importLorcanaSet(setCode, setName, lorcastCode) { - try { - console.log(`\n🎨 Importing Lorcana set: ${setName} (${setCode})`); - - // Fetch cards directly from Lorcast API - const response = await fetchWithRetry(`https://api.lorcast.com/v0/cards/search?q=set:${lorcastCode}&unique=prints`); - const data = await response.json(); - const cards = data.results || []; - - if (cards.length === 0) { - console.log(`❌ No cards found for set ${setName}`); - return { success: false, error: 'No cards found', importedCount: 0 }; - } - - console.log(`📊 Found ${cards.length} cards for set ${setName}`); - - let importedCount = 0; - let skippedCount = 0; - - // Import all cards via our API in a single batch - const importResponse = await fetch('http://localhost:3000/api/cards/import-lorcana', { - method: 'POST', - headers: { - 'Content-Type': 'application/json', - }, - body: JSON.stringify({ - setCode: setCode, - setName: setName - }) - }); - - const importResult = await importResponse.json(); - - if (importResult.success) { - importedCount = importResult.imported || 0; - skippedCount = importResult.skipped || 0; - console.log(`✅ Import completed for ${setName}: ${importedCount} imported, ${skippedCount} skipped`); - } else { - console.log(`❌ Failed to import ${setName}: ${importResult.error}`); - return { success: false, error: importResult.error }; - } - - return { success: true, importedCount, skippedCount, total: cards.length }; - - } catch (error) { - console.log(`❌ Error importing ${setName}: ${error.message}`); - return { success: false, error: error.message }; - } -} - -async function importAllLorcana() { - console.log('🎨 Starting Lorcana card import...\n'); - - const results = { - total: 0, - successful: 0, - failed: 0, - sets: [] - }; - - for (const set of LORCANA_SETS) { - const result = await importLorcanaSet(set.code, set.name, set.lorcastCode); - results.sets.push({ ...set, ...result }); - - if (result.success) { - results.successful++; - results.total += result.importedCount || 0; - } else { - results.failed++; - } - - // Add a delay between sets - await delay(2000); - } - - // Print summary - console.log('\n📊 Lorcana Import Summary:'); - console.log('========================'); - console.log(`Total sets: ${LORCANA_SETS.length}`); - console.log(`Successful: ${results.successful}`); - console.log(`Failed: ${results.failed}`); - console.log(`Total cards imported: ${results.total}`); - console.log(`Success rate: ${((results.successful / LORCANA_SETS.length) * 100).toFixed(1)}%`); - - // Save detailed results - const timestamp = new Date().toISOString().replace(/[:.]/g, '-'); - const filename = `lorcana-import-results-${timestamp}.json`; - - fs.writeFileSync(filename, JSON.stringify(results, null, 2)); - console.log(`\n📄 Detailed results saved to: ${filename}`); - - return results; -} - -// Run the import if this script is executed directly -if (import.meta.url === `file://${process.argv[1]}`) { - importAllLorcana() - .then(() => { - console.log('\n🎉 Lorcana import completed!'); - process.exit(0); - }) - .catch((error) => { - console.error('❌ Import failed:', error); - process.exit(1); - }); -} - -export { importAllLorcana, importLorcanaSet }; \ No newline at end of file