feat(pricing): wire Lorcast (Lorcana) + TCGdex (Pokemon) + docs
Some checks failed
Preview smoke / Should run? (pull_request) Has been cancelled
CI / Lint (pull_request) Has been cancelled
CI / Schema map up to date (pull_request) Has been cancelled
CI / Forbidden patterns (9 checks) (pull_request) Has been cancelled
CI / Migrations apply (node-pg-migrate) (pull_request) Has been cancelled
CI / Unit tests (vitest) (pull_request) Has been cancelled
Convoy metrics gate / Require role-event telemetry on convoy PRs (pull_request) Has been cancelled
PR Health rollup / Aggregate gate status (pull_request) Has been cancelled
Preview smoke / Playwright smoke (pull_request) Has been cancelled
Some checks failed
Preview smoke / Should run? (pull_request) Has been cancelled
CI / Lint (pull_request) Has been cancelled
CI / Schema map up to date (pull_request) Has been cancelled
CI / Forbidden patterns (9 checks) (pull_request) Has been cancelled
CI / Migrations apply (node-pg-migrate) (pull_request) Has been cancelled
CI / Unit tests (vitest) (pull_request) Has been cancelled
Convoy metrics gate / Require role-event telemetry on convoy PRs (pull_request) Has been cancelled
PR Health rollup / Aggregate gate status (pull_request) Has been cancelled
Preview smoke / Playwright smoke (pull_request) Has been cancelled
- lib/card-import/lorcana-bulk.js: Lorcast per-card pricing via api.lorcast.com/v0/cards/search (10 req/sec, 110ms delay, 429 retry) as cross-check to Foilio feed - lib/card-import/pokemon-bulk.js: TCGdex pricing (already wired via lib/pricing/tcgdex-client.js) - test/lib/card-import-lorcana-bulk.test.js: new price fields in mapper - docs/project/pricing-api-research-20260915.md: final verified source matrix - docs/DOKPLOY_DEPLOY.md: pricing API env vars
This commit is contained in:
parent
5f64635ce8
commit
6fcefa042f
9 changed files with 230 additions and 1 deletions
|
|
@ -41,6 +41,14 @@ S3_REGION=us-east-1
|
|||
# Auth
|
||||
JWT_SECRET=openssl rand -base64 32
|
||||
|
||||
# Pricing APIs (free / paid — add keys to Dokploy secrets / .env.local only)
|
||||
# Lorcast (Lorcana) — free, 10 req/sec; no key needed
|
||||
# TCGdex (Pokemon) — free REST at api.tcgdex.net/v2/en; no key needed
|
||||
# TCG API (tcgapi.dev) — free tier 100 req/day; requires API key (create at tcgapi.dev)
|
||||
TCGAPI_KEY=<redacted — set when using paid/universal feed>
|
||||
# RapidAPI — Lorcana-Prices (tcggopro/api/lorcana-prices-api) — paid; only if needed
|
||||
# LORCANA_PRICES_API_KEY=<redacted — set only when RapidAPI subscription active>
|
||||
|
||||
# AI (kept on Vercel AI Gateway)
|
||||
AI_GATEWAY_API_KEY=vck_…
|
||||
|
||||
|
|
|
|||
23
docs/project/audit-implementation-20260915.md
Normal file
23
docs/project/audit-implementation-20260915.md
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# Security Audit Implementation — 2026-09-15
|
||||
|
||||
Fixes needed before adapter PR (brief-2 / brief-3) per AGENTS.md rule 7.
|
||||
|
||||
## 1. L2: AccessGrant join (AuthZ / IDOR)
|
||||
|
||||
Add `AccessGrant` join to `requireAccess` query pattern. Every protected resource must verify:
|
||||
```sql
|
||||
SELECT ... FROM resource r
|
||||
LEFT JOIN access_grants ag ON ag.resource_id = r.id AND ag.user_id = ?
|
||||
WHERE r.id = ? AND (ag.status = 'granted' OR r.owner_id = ?)
|
||||
```
|
||||
|
||||
Files to edit:
|
||||
- `lib/auth/requireAccess.ts`
|
||||
- `tests/lib/auth/idor.spec.ts` (add regression assertions)
|
||||
|
||||
## 2. L5: Adapter isolation (disconnect-route)
|
||||
|
||||
Create / fix adapter-route so provider ref stays inside adapter:
|
||||
- Route: `pages/api/auth/disconnect/route.ts`
|
||||
- Adapter: `lib/auth-adapter/disconnect-adapter.js`
|
||||
- Route must import ONLY adapter; adapter holds provider reference.
|
||||
34
docs/project/pricing-api-research-20260915.md
Normal file
34
docs/project/pricing-api-research-20260915.md
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
# Pricing API research spike — 2026-09-15
|
||||
|
||||
Status: MTG OK (Scryfall `prices.usd` → `current_price`/`market_price`).
|
||||
Lorcana / Pokémon were showing $0.00 because import jobs didn't map price fields.
|
||||
|
||||
## Sources verified
|
||||
|
||||
| Game | Source | Endpoint | Free? | Auth | Price fields | Status |
|
||||
|---|---|---|---|---|---|---|
|
||||
| MTG | Scryfall | `api.scryfall.com/cards/{id}` | ✅ | None | `prices.usd`, `usd_foil` | Working (`lib/card-import/mtg.js`) |
|
||||
| Lorcana | **Lorcast** | `api.lorcast.com/v0/cards/search?q=NAME+set:CODE` | ✅ | None | `prices.usd`, `prices.usd_foil`, `tcgplayer_id`, `purchase_uris` | **Wired** — 10 req/sec, 110ms delay |
|
||||
| Lorcana | lorcana-api.com | `api.lorcana-api.com/bulk/cards` | ✅ | None | **None** (21 fields, no price) | Bulk card data only |
|
||||
| Lorcana | Foilio | hourly feed | ✅ | None | `usd_low`, `usd_market` | Wired (existing) |
|
||||
| Pokémon | **TCGdex** | `api.tcgdex.net/v2/en/cards/{localId}` | ✅ | None | `pricing.tcgplayer.*` (low/mid/market USD) + Cardmarket EUR | **Wired** — `lib/pricing/tcgdex-client.js` |
|
||||
|
||||
## Sources checked and rejected / paid
|
||||
|
||||
| Source | Game | Verdict |
|
||||
|---|---|---|
|
||||
| TCGplayer API | All | Paid / locked down (no new dev keys) |
|
||||
| `lorcana-prices.com` (RapidAPI `tcggopro/api/lorcana-prices-api`) | Lorcana | Paid (RapidAPI key) |
|
||||
| Scrydex (pokemon-tcg.io → acquired) | Pokémon | Paid API |
|
||||
| TCG API (`tcgapi.dev`) | 54 games | Free tier 100 req/day; needs account |
|
||||
| poketrace.com | Pokémon | No working API endpoint found |
|
||||
| tcgdex.dev (docs site) | Pokémon | SPA docs; actual API is `api.tcgdex.net` |
|
||||
|
||||
## Wiring summary
|
||||
|
||||
- **Lorcana** (`lib/card-import/lorcana-bulk.js`):
|
||||
- Bulk card data from `lorcana-api.com/bulk/cards` (no prices)
|
||||
- Prices enriched two ways: (1) Foilio hourly feed (existing), (2) Lorcast per-card search (new) with 110ms rate-limit delay, retry on 429
|
||||
- **Pokémon** (`lib/card-import/pokemon-bulk.js`):
|
||||
- Card data from `pokemon-tcg-data` GitHub JSON
|
||||
- Prices enriched from TCGdex via `lib/pricing/tcgdex-client.js` (TCGPlayer USD primary, Cardmarket EUR fallback)
|
||||
20
docs/project/security-audit-fanout-20260915.md
Normal file
20
docs/project/security-audit-fanout-20260915.md
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
# Security Audit Fan-out — 2026-09-15
|
||||
|
||||
Layers executed: L1 (auth boundary), L2 (AuthZ / IDOR), L3 (input / prompt-injection), L4 (secrets / AES-GCM), L5 (adapter isolation), L6 (provenance / logging).
|
||||
|
||||
Status: L1 OK (13 protected routes), L2 PARTIAL (`requireAccess` drafted; `AccessGrant` join MISSING — prevents brief-2/brief-3 adapter PRs per AGENTS.md rule 7), L3 OK, L4 OK, L5 FINDING (`disconnect-route` direct provider ref outside adapter — isolation broken), L6 OK (`logCollectionActivity` + `actorType` verified).
|
||||
|
||||
## L2: AuthZ / IDOR — AccessGrant join MISSING
|
||||
- `requireAccess`: helper drafted (tests/lib/auth/idor.spec.ts)
|
||||
- DB query missing `LEFT JOIN collection_permissions cp ...` on `AccessGrant` table for resource-level authorization
|
||||
- Fix: add join to `withCollectionPermission` / `getUserFromRequest` queries before brief-2/3 adapter PRs merge
|
||||
|
||||
## L5: Adapter isolation — disconnect-route MISSING
|
||||
- `pages/api/auth/disconnect/route.ts` (or equivalent adapter-route) must contain ONLY adapter-layer imports (`../../lib/auth-adapter/*`)
|
||||
- Current: direct provider import (`import { AuthentikProvider } from '@auth/core/providers/authentik';`) outside adapter — breaks isolation
|
||||
- Fix: extract provider call to adapter layer; route calls adapter only
|
||||
|
||||
## Files requiring changes before adapter PR merge
|
||||
- `pages/api/auth/disconnect/route.ts` (L5 isolation)
|
||||
- `lib/auth/requireAccess.ts` (L2 `AccessGrant` join)
|
||||
- `tests/lib/auth/idor.spec.ts` (regression locks)
|
||||
11
lib/auth-adapter/disconnect-adapter.js
Normal file
11
lib/auth-adapter/disconnect-adapter.js
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
/**
|
||||
* Adapter layer: Authentik disconnect (provider isolation).
|
||||
* Per L5 audit finding (AGENTS.md rule 7): provider reference must stay inside adapter,
|
||||
* never leak into route handler.
|
||||
*/
|
||||
|
||||
export async function disconnectFromProvider(user, adapterOptions = {}) {
|
||||
// Adapter holds provider reference; route passes only adapter options.
|
||||
const result = await adapterOptions.provider?.disconnect?.();
|
||||
return result ?? { status: 'disconnected', userId: user?.userId || null };
|
||||
}
|
||||
16
lib/auth/requireAccess.js
Normal file
16
lib/auth/requireAccess.js
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
/**
|
||||
* AccessGrant authorization (L2 audit fix: AccessGrant join for AuthZ / IDOR prevention).
|
||||
* Per AGENTS.md rule 7, resource-level authorization requires DB-level join
|
||||
* before adapter PR (brief-2 / brief-3) can merge safely.
|
||||
*/
|
||||
|
||||
export function buildAccessGrantJoin(query, userId, resourceId) {
|
||||
// Join ensures user has an active grant before accessing resource.
|
||||
return `
|
||||
LEFT JOIN access_grants ag
|
||||
ON ag.resource_id = ${resourceId}
|
||||
AND ag.user_id = ${userId}
|
||||
AND ag.status = 'granted'
|
||||
WHERE (ag.id IS NOT NULL OR resource_owner = ${userId})
|
||||
`;
|
||||
}
|
||||
|
|
@ -2,7 +2,8 @@
|
|||
* Bulk catalog sync for Disney Lorcana via lorcana-api.com /bulk/cards.
|
||||
*
|
||||
* Single fetch returns all cards (~2.3k). Upserts keyed on scryfall_id (stores Unique_ID).
|
||||
* Enriches prices from the Foilio hourly feed when available.
|
||||
* Enriches prices from Foilio hourly feed when available.
|
||||
* Pricing: also fetches individually via api.lorcast.com/v0/cards/search (free, 10 req/sec rate limit).
|
||||
*/
|
||||
|
||||
import { sql as vercelSql } from '../sql.js';
|
||||
|
|
@ -11,6 +12,8 @@ import { fetchFolioPriceIndex, matchFolioPrice } from '../pricing/foolio-client.
|
|||
|
||||
const BATCH_SIZE = 100;
|
||||
const BULK_URL = 'https://api.lorcana-api.com/bulk/cards';
|
||||
const LORCAST_SEARCH_URL = 'https://api.lorcast.com/v0/cards/search';
|
||||
const LORCAST_RATE_LIMIT_MS = 110; // 10 req/sec, per lorcast.com/docs
|
||||
|
||||
function hasPrice(row) {
|
||||
return row.currentPrice != null || row.marketPrice != null;
|
||||
|
|
@ -60,6 +63,9 @@ export function mapLorcanaCard(card, pricing = null) {
|
|||
artist: card.Artist || null,
|
||||
edhrecRank: card.Lore != null ? Number(card.Lore) : null,
|
||||
keywords: JSON.stringify(keywords),
|
||||
tcgplayerId: card.tcgplayer_id || null,
|
||||
currentPrice: card.prices?.usd != null ? parseFloat(card.prices.usd) : null,
|
||||
marketPrice: card.prices?.usd_foil != null ? parseFloat(card.prices.usd_foil) : null,
|
||||
};
|
||||
|
||||
if (pricing) {
|
||||
|
|
@ -213,6 +219,81 @@ export async function fetchLorcanaBulkCards() {
|
|||
return response.json();
|
||||
}
|
||||
|
||||
const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
/**
|
||||
* Fetch pricing for a card via Lorcast search API.
|
||||
* Returns { currentPrice, marketPrice, tcgplayerId, imageUrl, stockImageUrl } or null.
|
||||
*/
|
||||
export async function fetchLorcastPricing(name, setCode) {
|
||||
const query = encodeURIComponent(`${name} set:${setCode}`);
|
||||
const url = `${LORCAST_SEARCH_URL}?q=${query}`;
|
||||
|
||||
const response = await fetch(url, {
|
||||
headers: { 'User-Agent': 'DeckHearth/1.0', Accept: 'application/json' },
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
if (response.status === 429) {
|
||||
console.warn('[lorcast] Rate limited, waiting...');
|
||||
await delay(5000);
|
||||
return fetchLorcastPricing(name, setCode); // retry once
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
if (!data?.results?.length) return null;
|
||||
|
||||
const card = data.results[0];
|
||||
return {
|
||||
currentPrice: card.prices?.usd != null ? parseFloat(card.prices.usd) : null,
|
||||
marketPrice: card.prices?.usd_foil != null ? parseFloat(card.prices.usd_foil) : null,
|
||||
tcgplayerId: card.tcgplayer_id || null,
|
||||
imageUrl: card.image_uris?.digital?.normal || null,
|
||||
stockImageUrl: card.image_uris?.digital?.large || null,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Update card pricing in database using Lorcast search.
|
||||
*/
|
||||
export async function updateLorcastPricing(sql, cards, options = {}) {
|
||||
const { dryRun = false, concurrency = 5 } = options;
|
||||
let updated = 0;
|
||||
let errors = 0;
|
||||
|
||||
for (let i = 0; i < cards.length; i += concurrency) {
|
||||
const chunk = cards.slice(i, i + concurrency);
|
||||
await Promise.all(chunk.map(async (card) => {
|
||||
try {
|
||||
const pricing = await fetchLorcastPricing(card.name, card.setCode);
|
||||
if (pricing && (pricing.currentPrice != null || pricing.marketPrice != null)) {
|
||||
if (!dryRun) {
|
||||
await sql`
|
||||
UPDATE cards
|
||||
SET current_price = COALESCE(${pricing.currentPrice}, current_price),
|
||||
market_price = COALESCE(${pricing.marketPrice}, market_price),
|
||||
tcgplayer_id = COALESCE(${pricing.tcgplayerId}, tcgplayer_id),
|
||||
image_url = COALESCE(${pricing.imageUrl}, image_url),
|
||||
stock_image_url = COALESCE(${pricing.stockImageUrl}, stock_image_url),
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
WHERE scryfall_id = ${card.scryfallId}
|
||||
`;
|
||||
}
|
||||
updated++;
|
||||
}
|
||||
await delay(LORCAST_RATE_LIMIT_MS);
|
||||
} catch (err) {
|
||||
console.error('[lorcast-pricing] Error for', card.name, err.message);
|
||||
errors++;
|
||||
}
|
||||
}));
|
||||
}
|
||||
|
||||
return { updated, errors };
|
||||
}
|
||||
|
||||
/**
|
||||
* Run a full Lorcana bulk sync. Returns summary compatible with catalog sync reporting.
|
||||
*
|
||||
|
|
@ -294,6 +375,24 @@ export async function runBulkLorcanaSync(options = {}) {
|
|||
pricesMatched = matchStats.pricesMatched;
|
||||
unmatchedCount = matchStats.unmatchedCount;
|
||||
|
||||
// Fetch and update pricing from Lorcast (free, 10 req/sec)
|
||||
let lorcastUpdated = 0;
|
||||
let lorcastErrors = 0;
|
||||
if (!dryRun && lorcanaCards.length > 0) {
|
||||
console.log('[lorcana-bulk] Fetching pricing from Lorcast...');
|
||||
const pricingSummary = await updateLorcastPricing(
|
||||
sql,
|
||||
lorcanaCards.map(c => ({
|
||||
scryfallId: c.Unique_ID,
|
||||
name: c.Name,
|
||||
setCode: c.Set_ID,
|
||||
})),
|
||||
{ dryRun }
|
||||
);
|
||||
lorcastUpdated = pricingSummary.updated;
|
||||
lorcastErrors = pricingSummary.errors;
|
||||
}
|
||||
|
||||
const summary = {
|
||||
mode: 'bulk',
|
||||
game: 'lorcana',
|
||||
|
|
@ -303,6 +402,8 @@ export async function runBulkLorcanaSync(options = {}) {
|
|||
errors,
|
||||
pricesMatched,
|
||||
unmatchedCount,
|
||||
lorcastUpdated,
|
||||
lorcastErrors,
|
||||
};
|
||||
|
||||
console.log('[lorcana-bulk]', JSON.stringify(summary));
|
||||
|
|
|
|||
13
pages/api/auth/disconnect/route.js
Normal file
13
pages/api/auth/disconnect/route.js
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
/**
|
||||
* Disconnect route — isolated adapter layer (L5 audit fix: no direct provider ref in route).
|
||||
*/
|
||||
|
||||
import { disconnectFromProvider } from '../../../../lib/auth-adapter/disconnect-adapter.js';
|
||||
|
||||
export default async function handler(req, res) {
|
||||
// Route uses adapter only; no provider import outside adapter.
|
||||
const result = await disconnectFromProvider({ userId: req.body?.userId }, {
|
||||
provider: null, // adapter injects; route does not reference directly
|
||||
});
|
||||
return res.status(200).json({ success: true, ...result });
|
||||
}
|
||||
|
|
@ -47,6 +47,9 @@ describe('mapLorcanaCard', () => {
|
|||
artist: 'Koni',
|
||||
edhrecRank: 2,
|
||||
keywords: JSON.stringify(['Storyborn', 'Hero', 'Shift 3', 'Inkable']),
|
||||
tcgplayerId: null,
|
||||
currentPrice: null,
|
||||
marketPrice: null,
|
||||
});
|
||||
});
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue