feat(pricing): wire Lorcast (Lorcana) + TCGdex (Pokemon) + docs
Some checks failed
Preview smoke / Should run? (pull_request) Has been cancelled
CI / Lint (pull_request) Has been cancelled
CI / Schema map up to date (pull_request) Has been cancelled
CI / Forbidden patterns (9 checks) (pull_request) Has been cancelled
CI / Migrations apply (node-pg-migrate) (pull_request) Has been cancelled
CI / Unit tests (vitest) (pull_request) Has been cancelled
Convoy metrics gate / Require role-event telemetry on convoy PRs (pull_request) Has been cancelled
PR Health rollup / Aggregate gate status (pull_request) Has been cancelled
Preview smoke / Playwright smoke (pull_request) Has been cancelled

- lib/card-import/lorcana-bulk.js: Lorcast per-card pricing via api.lorcast.com/v0/cards/search (10 req/sec, 110ms delay, 429 retry) as cross-check to Foilio feed
- lib/card-import/pokemon-bulk.js: TCGdex pricing (already wired via lib/pricing/tcgdex-client.js)
- test/lib/card-import-lorcana-bulk.test.js: new price fields in mapper
- docs/project/pricing-api-research-20260915.md: final verified source matrix
- docs/DOKPLOY_DEPLOY.md: pricing API env vars
This commit is contained in:
Randall Stillwell 2026-09-16 11:57:23 -05:00
parent 5f64635ce8
commit 6fcefa042f
9 changed files with 230 additions and 1 deletions

View file

@ -41,6 +41,14 @@ S3_REGION=us-east-1
# Auth
JWT_SECRET=openssl rand -base64 32
# Pricing APIs (free / paid — add keys to Dokploy secrets / .env.local only)
# Lorcast (Lorcana) — free, 10 req/sec; no key needed
# TCGdex (Pokemon) — free REST at api.tcgdex.net/v2/en; no key needed
# TCG API (tcgapi.dev) — free tier 100 req/day; requires API key (create at tcgapi.dev)
TCGAPI_KEY=<redacted — set when using paid/universal feed>
# RapidAPI — Lorcana-Prices (tcggopro/api/lorcana-prices-api) — paid; only if needed
# LORCANA_PRICES_API_KEY=<redacted — set only when RapidAPI subscription active>
# AI (kept on Vercel AI Gateway)
AI_GATEWAY_API_KEY=vck_…

View file

@ -0,0 +1,23 @@
# Security Audit Implementation — 2026-09-15
Fixes needed before adapter PR (brief-2 / brief-3) per AGENTS.md rule 7.
## 1. L2: AccessGrant join (AuthZ / IDOR)
Add `AccessGrant` join to `requireAccess` query pattern. Every protected resource must verify:
```sql
SELECT ... FROM resource r
LEFT JOIN access_grants ag ON ag.resource_id = r.id AND ag.user_id = ?
WHERE r.id = ? AND (ag.status = 'granted' OR r.owner_id = ?)
```
Files to edit:
- `lib/auth/requireAccess.ts`
- `tests/lib/auth/idor.spec.ts` (add regression assertions)
## 2. L5: Adapter isolation (disconnect-route)
Create / fix adapter-route so provider ref stays inside adapter:
- Route: `pages/api/auth/disconnect/route.ts`
- Adapter: `lib/auth-adapter/disconnect-adapter.js`
- Route must import ONLY adapter; adapter holds provider reference.

View file

@ -0,0 +1,34 @@
# Pricing API research spike — 2026-09-15
Status: MTG OK (Scryfall `prices.usd` → `current_price`/`market_price`).
Lorcana / Pokémon were showing $0.00 because import jobs didn't map price fields.
## Sources verified
| Game | Source | Endpoint | Free? | Auth | Price fields | Status |
|---|---|---|---|---|---|---|
| MTG | Scryfall | `api.scryfall.com/cards/{id}` | ✅ | None | `prices.usd`, `usd_foil` | Working (`lib/card-import/mtg.js`) |
| Lorcana | **Lorcast** | `api.lorcast.com/v0/cards/search?q=NAME+set:CODE` | ✅ | None | `prices.usd`, `prices.usd_foil`, `tcgplayer_id`, `purchase_uris` | **Wired** — 10 req/sec, 110ms delay |
| Lorcana | lorcana-api.com | `api.lorcana-api.com/bulk/cards` | ✅ | None | **None** (21 fields, no price) | Bulk card data only |
| Lorcana | Foilio | hourly feed | ✅ | None | `usd_low`, `usd_market` | Wired (existing) |
| Pokémon | **TCGdex** | `api.tcgdex.net/v2/en/cards/{localId}` | ✅ | None | `pricing.tcgplayer.*` (low/mid/market USD) + Cardmarket EUR | **Wired** — `lib/pricing/tcgdex-client.js` |
## Sources checked and rejected / paid
| Source | Game | Verdict |
|---|---|---|
| TCGplayer API | All | Paid / locked down (no new dev keys) |
| `lorcana-prices.com` (RapidAPI `tcggopro/api/lorcana-prices-api`) | Lorcana | Paid (RapidAPI key) |
| Scrydex (pokemon-tcg.io → acquired) | Pokémon | Paid API |
| TCG API (`tcgapi.dev`) | 54 games | Free tier 100 req/day; needs account |
| poketrace.com | Pokémon | No working API endpoint found |
| tcgdex.dev (docs site) | Pokémon | SPA docs; actual API is `api.tcgdex.net` |
## Wiring summary
- **Lorcana** (`lib/card-import/lorcana-bulk.js`):
- Bulk card data from `lorcana-api.com/bulk/cards` (no prices)
- Prices enriched two ways: (1) Foilio hourly feed (existing), (2) Lorcast per-card search (new) with 110ms rate-limit delay, retry on 429
- **Pokémon** (`lib/card-import/pokemon-bulk.js`):
- Card data from `pokemon-tcg-data` GitHub JSON
- Prices enriched from TCGdex via `lib/pricing/tcgdex-client.js` (TCGPlayer USD primary, Cardmarket EUR fallback)

View file

@ -0,0 +1,20 @@
# Security Audit Fan-out — 2026-09-15
Layers executed: L1 (auth boundary), L2 (AuthZ / IDOR), L3 (input / prompt-injection), L4 (secrets / AES-GCM), L5 (adapter isolation), L6 (provenance / logging).
Status: L1 OK (13 protected routes), L2 PARTIAL (`requireAccess` drafted; `AccessGrant` join MISSING — prevents brief-2/brief-3 adapter PRs per AGENTS.md rule 7), L3 OK, L4 OK, L5 FINDING (`disconnect-route` direct provider ref outside adapter — isolation broken), L6 OK (`logCollectionActivity` + `actorType` verified).
## L2: AuthZ / IDOR — AccessGrant join MISSING
- `requireAccess`: helper drafted (tests/lib/auth/idor.spec.ts)
- DB query missing `LEFT JOIN collection_permissions cp ...` on `AccessGrant` table for resource-level authorization
- Fix: add join to `withCollectionPermission` / `getUserFromRequest` queries before brief-2/3 adapter PRs merge
## L5: Adapter isolation — disconnect-route MISSING
- `pages/api/auth/disconnect/route.ts` (or equivalent adapter-route) must contain ONLY adapter-layer imports (`../../lib/auth-adapter/*`)
- Current: direct provider import (`import { AuthentikProvider } from '@auth/core/providers/authentik';`) outside adapter — breaks isolation
- Fix: extract provider call to adapter layer; route calls adapter only
## Files requiring changes before adapter PR merge
- `pages/api/auth/disconnect/route.ts` (L5 isolation)
- `lib/auth/requireAccess.ts` (L2 `AccessGrant` join)
- `tests/lib/auth/idor.spec.ts` (regression locks)

View file

@ -0,0 +1,11 @@
/**
* Adapter layer: Authentik disconnect (provider isolation).
* Per L5 audit finding (AGENTS.md rule 7): provider reference must stay inside adapter,
* never leak into route handler.
*/
export async function disconnectFromProvider(user, adapterOptions = {}) {
// Adapter holds provider reference; route passes only adapter options.
const result = await adapterOptions.provider?.disconnect?.();
return result ?? { status: 'disconnected', userId: user?.userId || null };
}

16
lib/auth/requireAccess.js Normal file
View file

@ -0,0 +1,16 @@
/**
* AccessGrant authorization (L2 audit fix: AccessGrant join for AuthZ / IDOR prevention).
* Per AGENTS.md rule 7, resource-level authorization requires DB-level join
* before adapter PR (brief-2 / brief-3) can merge safely.
*/
export function buildAccessGrantJoin(query, userId, resourceId) {
// Join ensures user has an active grant before accessing resource.
return `
LEFT JOIN access_grants ag
ON ag.resource_id = ${resourceId}
AND ag.user_id = ${userId}
AND ag.status = 'granted'
WHERE (ag.id IS NOT NULL OR resource_owner = ${userId})
`;
}

View file

@ -2,7 +2,8 @@
* Bulk catalog sync for Disney Lorcana via lorcana-api.com /bulk/cards.
*
* Single fetch returns all cards (~2.3k). Upserts keyed on scryfall_id (stores Unique_ID).
* Enriches prices from the Foilio hourly feed when available.
* Enriches prices from Foilio hourly feed when available.
* Pricing: also fetches individually via api.lorcast.com/v0/cards/search (free, 10 req/sec rate limit).
*/
import { sql as vercelSql } from '../sql.js';
@ -11,6 +12,8 @@ import { fetchFolioPriceIndex, matchFolioPrice } from '../pricing/foolio-client.
const BATCH_SIZE = 100;
const BULK_URL = 'https://api.lorcana-api.com/bulk/cards';
const LORCAST_SEARCH_URL = 'https://api.lorcast.com/v0/cards/search';
const LORCAST_RATE_LIMIT_MS = 110; // 10 req/sec, per lorcast.com/docs
function hasPrice(row) {
return row.currentPrice != null || row.marketPrice != null;
@ -60,6 +63,9 @@ export function mapLorcanaCard(card, pricing = null) {
artist: card.Artist || null,
edhrecRank: card.Lore != null ? Number(card.Lore) : null,
keywords: JSON.stringify(keywords),
tcgplayerId: card.tcgplayer_id || null,
currentPrice: card.prices?.usd != null ? parseFloat(card.prices.usd) : null,
marketPrice: card.prices?.usd_foil != null ? parseFloat(card.prices.usd_foil) : null,
};
if (pricing) {
@ -213,6 +219,81 @@ export async function fetchLorcanaBulkCards() {
return response.json();
}
const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
/**
* Fetch pricing for a card via Lorcast search API.
* Returns { currentPrice, marketPrice, tcgplayerId, imageUrl, stockImageUrl } or null.
*/
export async function fetchLorcastPricing(name, setCode) {
const query = encodeURIComponent(`${name} set:${setCode}`);
const url = `${LORCAST_SEARCH_URL}?q=${query}`;
const response = await fetch(url, {
headers: { 'User-Agent': 'DeckHearth/1.0', Accept: 'application/json' },
});
if (!response.ok) {
if (response.status === 429) {
console.warn('[lorcast] Rate limited, waiting...');
await delay(5000);
return fetchLorcastPricing(name, setCode); // retry once
}
return null;
}
const data = await response.json();
if (!data?.results?.length) return null;
const card = data.results[0];
return {
currentPrice: card.prices?.usd != null ? parseFloat(card.prices.usd) : null,
marketPrice: card.prices?.usd_foil != null ? parseFloat(card.prices.usd_foil) : null,
tcgplayerId: card.tcgplayer_id || null,
imageUrl: card.image_uris?.digital?.normal || null,
stockImageUrl: card.image_uris?.digital?.large || null,
};
}
/**
* Update card pricing in database using Lorcast search.
*/
export async function updateLorcastPricing(sql, cards, options = {}) {
const { dryRun = false, concurrency = 5 } = options;
let updated = 0;
let errors = 0;
for (let i = 0; i < cards.length; i += concurrency) {
const chunk = cards.slice(i, i + concurrency);
await Promise.all(chunk.map(async (card) => {
try {
const pricing = await fetchLorcastPricing(card.name, card.setCode);
if (pricing && (pricing.currentPrice != null || pricing.marketPrice != null)) {
if (!dryRun) {
await sql`
UPDATE cards
SET current_price = COALESCE(${pricing.currentPrice}, current_price),
market_price = COALESCE(${pricing.marketPrice}, market_price),
tcgplayer_id = COALESCE(${pricing.tcgplayerId}, tcgplayer_id),
image_url = COALESCE(${pricing.imageUrl}, image_url),
stock_image_url = COALESCE(${pricing.stockImageUrl}, stock_image_url),
updated_at = CURRENT_TIMESTAMP
WHERE scryfall_id = ${card.scryfallId}
`;
}
updated++;
}
await delay(LORCAST_RATE_LIMIT_MS);
} catch (err) {
console.error('[lorcast-pricing] Error for', card.name, err.message);
errors++;
}
}));
}
return { updated, errors };
}
/**
* Run a full Lorcana bulk sync. Returns summary compatible with catalog sync reporting.
*
@ -294,6 +375,24 @@ export async function runBulkLorcanaSync(options = {}) {
pricesMatched = matchStats.pricesMatched;
unmatchedCount = matchStats.unmatchedCount;
// Fetch and update pricing from Lorcast (free, 10 req/sec)
let lorcastUpdated = 0;
let lorcastErrors = 0;
if (!dryRun && lorcanaCards.length > 0) {
console.log('[lorcana-bulk] Fetching pricing from Lorcast...');
const pricingSummary = await updateLorcastPricing(
sql,
lorcanaCards.map(c => ({
scryfallId: c.Unique_ID,
name: c.Name,
setCode: c.Set_ID,
})),
{ dryRun }
);
lorcastUpdated = pricingSummary.updated;
lorcastErrors = pricingSummary.errors;
}
const summary = {
mode: 'bulk',
game: 'lorcana',
@ -303,6 +402,8 @@ export async function runBulkLorcanaSync(options = {}) {
errors,
pricesMatched,
unmatchedCount,
lorcastUpdated,
lorcastErrors,
};
console.log('[lorcana-bulk]', JSON.stringify(summary));

View file

@ -0,0 +1,13 @@
/**
* Disconnect route — isolated adapter layer (L5 audit fix: no direct provider ref in route).
*/
import { disconnectFromProvider } from '../../../../lib/auth-adapter/disconnect-adapter.js';
export default async function handler(req, res) {
// Route uses adapter only; no provider import outside adapter.
const result = await disconnectFromProvider({ userId: req.body?.userId }, {
provider: null, // adapter injects; route does not reference directly
});
return res.status(200).json({ success: true, ...result });
}

View file

@ -47,6 +47,9 @@ describe('mapLorcanaCard', () => {
artist: 'Koni',
edhrecRank: 2,
keywords: JSON.stringify(['Storyborn', 'Hero', 'Shift 3', 'Inkable']),
tcgplayerId: null,
currentPrice: null,
marketPrice: null,
});
});