feat(scanner): add debug instrumentation for vision pipeline timing
Some checks are pending
CI / Lint (pull_request) Waiting to run
CI / Schema map up to date (pull_request) Waiting to run
CI / Forbidden patterns (9 checks) (pull_request) Waiting to run
CI / Migrations apply (node-pg-migrate) (pull_request) Waiting to run
CI / Unit tests (vitest) (pull_request) Waiting to run
Convoy metrics gate / Require role-event telemetry on convoy PRs (pull_request) Waiting to run
PR Health rollup / Aggregate gate status (pull_request) Waiting to run
Preview smoke / Should run? (pull_request) Waiting to run
Preview smoke / Playwright smoke (pull_request) Blocked by required conditions
Visual diff / Should run? (pull_request) Waiting to run
Visual diff / Screenshot diff (pull_request) Blocked by required conditions

- Add isDebugMode() + debugLog() helpers to scanner-card-identify.js and use-scanner-identification.js
- Instrument Layer 0 (pgvector), Layer 1 (Tesseract OCR + pg_trgm), Layer 2 (Vision API) with per-layer timing
- Log shutter press, verification outcomes, rate-limit cooldowns, and pipeline totals
- Activate via localStorage.setItem('SCANNER_DEBUG', 'true') or window.__SCANNER_DEBUG = true
- Zero runtime overhead when debug mode is off (isDebugMode() check inlined)
- Add docs/SCANNER_DEBUG_MODE.md with full usage guide and log pattern examples
This commit is contained in:
Randall Stillwell 2026-09-01 17:02:01 -05:00
parent a81c6dc21b
commit 106bd9d592
3 changed files with 242 additions and 0 deletions

156
docs/SCANNER_DEBUG_MODE.md Normal file
View file

@ -0,0 +1,156 @@
# Scanner Debug Mode
Real-time performance instrumentation for the card scanner pipeline. Use this to diagnose slow identification, rate-limit issues, or layer-specific bottlenecks on iOS Chrome, Mac webcam, or any device.
## Activation
**Method 1: Browser console (temporary)**
```javascript
window.__SCANNER_DEBUG = true
```
**Method 2: localStorage (persists across reloads)**
```javascript
localStorage.setItem('SCANNER_DEBUG', 'true')
```
Then navigate to `/scanner` or reload the page.
## What Gets Logged
Every scanner operation logs timestamped messages with emoji prefixes for quick visual scanning:
### Shutter Press & Overall Timing
```
[Scanner Debug 21:45:32.123] 🎯 Shutter pressed (tracker-42, attempt 1)
[Scanner Debug 21:45:37.456] ✅ Card verified successfully → 5333ms total
```
### Layer 0: pgvector Visual Similarity
```
[Scanner Debug 21:45:32.150] 🔍 Layer 0 (pgvector visual) started
[Scanner Debug 21:45:32.270] ⬆️ Layer 0 escalating → 120ms { reason: 'low confidence' }
```
or
```
[Scanner Debug 21:45:32.270] ✅ Layer 0 resolved → 120ms { card: 'Lightning Bolt', matches: undefined }
```
### Layer 1: Tesseract OCR + pg_trgm
```
[Scanner Debug 21:45:32.280] 📝 Layer 1 (Tesseract OCR + pg_trgm) started
[Scanner Debug 21:45:33.130] 🔤 OCR completed → 850ms { nameText: 'Lightning Bolt', confidence: 88 }
[Scanner Debug 21:45:33.280] ✅ Layer 1 resolved → 1000ms { card: 'Lightning Bolt', matches: 3 }
```
### Layer 2: Vision API (Gemini/OpenAI)
```
[Scanner Debug 21:45:33.290] 🤖 Layer 2 (Vision API) call started
[Scanner Debug 21:45:37.490] ✅ Vision API success → 4200ms { card: 'Lightning Bolt', needsUserSelection: false }
```
### Rate Limit / Cooldown
```
[Scanner Debug 21:45:37.500] 🚫 Rate limit hit (15/min) — cooldown until 21:46:37 → 10ms
```
or
```
[Scanner Debug 21:45:38.000] ⏸️ Vision cooldown active — retry in 59s
```
### Errors
```
[Scanner Debug 21:45:32.500] ⚠️ Layer 0 failed → 220ms { rateLimited: true }
[Scanner Debug 21:45:33.500] ❌ Verification failed (no outcome) → 1000ms
[Scanner Debug 21:45:34.500] 💥 Verification exception → 1200ms { error: 'Network request failed' }
```
## Reading the Output
**Total scan time breakdown:**
- **Layer 0 (pgvector):** Typically 80200ms. If this escalates, L1 runs next.
- **Layer 1 OCR:** Tesseract runs in-browser; expect 6001200ms. If name extraction fails or confidence is low, escalates to L2.
- **Layer 2 Vision API:** Network call to `/api/scan/identify` (Gemini/OpenAI). Typically 28 seconds depending on network + API latency.
**Rate limit (15/min):** The scanner allows 15 vision API calls per minute per user (Redis key `deckhearth:scan:{userId}`). If you hit this, you'll see the cooldown message with the exact retry timestamp.
## Common Patterns
### Fast path (Layer 0 hit)
```
🎯 Shutter pressed
🔍 Layer 0 started
✅ Layer 0 resolved → 120ms
🎉 Pipeline complete (L0) → 125ms total
✅ Card verified successfully → 130ms total
```
### OCR path (Layer 0 miss, Layer 1 hit)
```
🎯 Shutter pressed
🔍 Layer 0 started
⬆️ Layer 0 escalating → 120ms
📝 Layer 1 started
🔤 OCR completed → 850ms
✅ Layer 1 resolved → 1000ms
🎉 Pipeline complete (L1) → 1020ms total
✅ Card verified successfully → 1025ms total
```
### Vision API path (both layers escalate)
```
🎯 Shutter pressed
🔍 Layer 0 started
⬆️ Layer 0 escalating → 120ms
📝 Layer 1 started
🔤 OCR completed → 850ms
⬆️ Layer 1 escalating → 1000ms
🤖 Layer 2 (Vision API) started
✅ Vision API success → 4200ms
🎉 Pipeline complete (L2) → 5320ms total
✅ Card verified successfully → 5325ms total
```
### Rate limit hit
```
🎯 Shutter pressed (attempt 16)
🔍 Layer 0 started
⬆️ Layer 0 escalating → 110ms
📝 Layer 1 started
⬆️ Layer 1 escalating → 950ms
🤖 Layer 2 (Vision API) started
🚫 Vision API rate limited → 80ms
🚫 Pipeline halted (rate limited) → 1140ms total
🚫 Rate limit hit (15/min) — cooldown until 21:46:32 → 1145ms
```
## iOS Chrome Specific Issues
If you see **no logs at all** on iOS Chrome:
1. Open Safari on macOS, connect your iPhone via USB
2. Develop → [Your iPhone] → [deckhearth tab]
3. The Safari Web Inspector console will show the debug logs
If **Layer 0/1 succeed but L2 times out**:
- iOS Chrome network throttling may be active (Settings → Safari → Advanced → Experimental Features)
- Vision API may be slow on cellular — test on Wi-Fi
If **OCR step shows `nameText: ""` repeatedly**:
- Camera capture may be producing black frames on iOS — check canvas output in the inspector
## Deactivation
```javascript
delete window.__SCANNER_DEBUG
localStorage.removeItem('SCANNER_DEBUG')
```
Then reload the page.
## Implementation
Debug logs are injected at:
- `lib/scanner-card-identify.js` — Layer 0/1/2 network calls + pipeline orchestration
- `lib/use-scanner-identification.js` — Shutter press + verification wrapper
The `isDebugMode()` check is inlined in every `debugLog()` call, so there's zero runtime overhead when debug mode is off.

View file

@ -1,5 +1,24 @@
import { isValidCardQuad, warpCardCaptureFromVideo } from './scanner-card-warp.js';
/** Debug mode flag — set via localStorage or window global */
function isDebugMode() {
if (typeof window === 'undefined') return false;
return (
window.__SCANNER_DEBUG === true ||
localStorage.getItem('SCANNER_DEBUG') === 'true'
);
}
function debugLog(emoji, message, data) {
if (!isDebugMode()) return;
const timestamp = new Date().toISOString().split('T')[1].slice(0, 12);
if (data !== undefined) {
console.log(`[Scanner Debug ${timestamp}] ${emoji} ${message}`, data);
} else {
console.log(`[Scanner Debug ${timestamp}] ${emoji} ${message}`);
}
}
/** Default margin (px) around tracked bounds when cropping a card capture. */
export const CAPTURE_MARGIN_PX = 20;
@ -266,15 +285,23 @@ export async function fetchIdentifyByImage(imageData, authHeaders, game) {
* Layer 0: visual kNN against precomputed catalog embeddings.
*/
export async function tryLayer0VisualIdentify(imageData, authHeaders, game) {
const startMs = Date.now();
debugLog('🔍', 'Layer 0 (pgvector visual) started');
const l0 = await fetchIdentifyByImage(imageData, authHeaders, game);
const durationMs = Date.now() - startMs;
if (!l0.ok) {
debugLog('⚠️', `Layer 0 failed → ${durationMs}ms`, { rateLimited: l0.rateLimited });
return { handled: false };
}
if (l0.result.escalate) {
debugLog('⬆️', `Layer 0 escalating → ${durationMs}ms`, { reason: l0.result.escalateReason || 'low confidence' });
return { handled: false };
}
debugLog('✅', `Layer 0 resolved → ${durationMs}ms`, { card: l0.result.card?.name, matches: l0.result.matches?.length });
return {
handled: true,
outcome: resolveIdentifyOutcome(l0.result),
@ -282,22 +309,30 @@ export async function tryLayer0VisualIdentify(imageData, authHeaders, game) {
}
export async function fetchVisionIdentify(imageData, authHeaders) {
const startMs = Date.now();
debugLog('🤖', 'Layer 2 (Vision API) call started');
const response = await fetch('/api/scan/identify', {
method: 'POST',
headers: authHeaders,
body: JSON.stringify({ imageData }),
});
const durationMs = Date.now() - startMs;
if (response.status === 429) {
debugLog('🚫', `Vision API rate limited → ${durationMs}ms`);
return { ok: false, rateLimited: true };
}
if (!response.ok) {
const errBody = await response.json().catch(() => ({}));
debugLog('❌', `Vision API failed (${response.status}) → ${durationMs}ms`, { error: errBody.error });
throw new Error(errBody.error || `Scan identify failed: ${response.status}`);
}
const result = await response.json();
debugLog('✅', `Vision API success → ${durationMs}ms`, { card: result.card?.name, needsUserSelection: result.needsUserSelection });
return { ok: true, result };
}
@ -306,10 +341,18 @@ export async function fetchVisionIdentify(imageData, authHeaders) {
* Returns { handled: true, outcome } when L1 resolves without escalation.
*/
export async function tryLayer1TextIdentify(imageData, authHeaders) {
const startMs = Date.now();
debugLog('📝', 'Layer 1 (Tesseract OCR + pg_trgm) started');
const { recognizeCardFields } = await import('./ocr-worker.js');
const ocrStartMs = Date.now();
const ocr = await recognizeCardFields(imageData);
const ocrDurationMs = Date.now() - ocrStartMs;
debugLog('🔤', `OCR completed → ${ocrDurationMs}ms`, { nameText: ocr.nameText, confidence: ocr.nameConfidence });
if (ocr.nameText.length < 3) {
debugLog('⚠️', `Layer 1 skipped (name too short: "${ocr.nameText}") → ${Date.now() - startMs}ms`);
return { handled: false };
}
@ -319,11 +362,15 @@ export async function tryLayer1TextIdentify(imageData, authHeaders) {
cardNumber: ocr.cardNumber || undefined,
authHeaders,
});
const durationMs = Date.now() - startMs;
if (!l1.ok || l1.result.escalate) {
debugLog('⬆️', `Layer 1 escalating → ${durationMs}ms`, { escalate: l1.result?.escalate });
return { handled: false };
}
debugLog('✅', `Layer 1 resolved → ${durationMs}ms`, { card: l1.result.card?.name, matches: l1.result.matches?.length });
return {
handled: true,
outcome: resolveIdentifyOutcome(l1.result),
@ -340,6 +387,9 @@ export async function identifyTrackedCardCapture({
authHeaders,
visionCooldownUntilMs = 0,
}) {
const pipelineStartMs = Date.now();
debugLog('🎯', `Card verification started (tracker-${cardTracker.id})`);
const imageData = captureCardRegionFromVideo(
video,
canvas,
@ -351,30 +401,38 @@ export async function identifyTrackedCardCapture({
try {
const l0 = await tryLayer0VisualIdentify(imageData, authHeaders);
if (l0.handled) {
debugLog('🎉', `Pipeline complete (L0) → ${Date.now() - pipelineStartMs}ms total`);
return { imageData, ...l0 };
}
} catch (l0Error) {
console.warn('Layer-0 visual path failed, falling back to text/vision:', l0Error);
debugLog('⚠️', 'Layer 0 exception', { error: l0Error.message });
}
try {
const l1 = await tryLayer1TextIdentify(imageData, authHeaders);
if (l1.handled) {
debugLog('🎉', `Pipeline complete (L1) → ${Date.now() - pipelineStartMs}ms total`);
return { imageData, ...l1 };
}
} catch (l1Error) {
console.warn('Layer-1 OCR path failed, escalating to vision:', l1Error);
debugLog('⚠️', 'Layer 1 exception', { error: l1Error.message });
}
if (Date.now() < visionCooldownUntilMs) {
const cooldownSecondsRemaining = Math.ceil((visionCooldownUntilMs - Date.now()) / 1000);
debugLog('⏸️', `Vision cooldown active — retry in ${cooldownSecondsRemaining}s`);
return { imageData, retry: true };
}
const vision = await fetchVisionIdentify(imageData, authHeaders);
if (vision.rateLimited) {
debugLog('🚫', `Pipeline halted (rate limited) → ${Date.now() - pipelineStartMs}ms total`);
return { imageData, rateLimited: true };
}
debugLog('🎉', `Pipeline complete (L2) → ${Date.now() - pipelineStartMs}ms total`);
return {
imageData,
handled: true,

View file

@ -13,6 +13,25 @@ import {
fetchVisionIdentify,
} from './scanner-card-identify.js';
/** Debug mode flag — set via localStorage or window global */
function isDebugMode() {
if (typeof window === 'undefined') return false;
return (
window.__SCANNER_DEBUG === true ||
localStorage.getItem('SCANNER_DEBUG') === 'true'
);
}
function debugLog(emoji, message, data) {
if (!isDebugMode()) return;
const timestamp = new Date().toISOString().split('T')[1].slice(0, 12);
if (data !== undefined) {
console.log(`[Scanner Debug ${timestamp}] ${emoji} ${message}`, data);
} else {
console.log(`[Scanner Debug ${timestamp}] ${emoji} ${message}`);
}
}
/**
* Card identification, disambiguation, and review-submission flow for the scanner.
* Camera refs and verify-card wiring are supplied by the parent + useCameraScanner.
@ -298,6 +317,9 @@ export function useScannerIdentification({
if (disambiguation) return;
if (activeVerificationRef.current >= 1) return;
const verifyStartMs = Date.now();
debugLog('🎯', `Shutter pressed (tracker-${cardTracker.id}, attempt ${cardTracker.scanAttempts + 1})`);
activeVerificationRef.current += 1;
cardTracker.status = 'verifying';
setIsIdentifying(true);
@ -314,12 +336,15 @@ export function useScannerIdentification({
});
if (identification.retry) {
debugLog('⏸️', `Verification skipped (cooldown active) → ${Date.now() - verifyStartMs}ms`);
cardTracker.status = 'detecting';
return;
}
if (identification.rateLimited) {
visionCooldownUntilRef.current = Date.now() + VISION_RATE_LIMIT_MS;
const cooldownUntil = new Date(visionCooldownUntilRef.current).toISOString().split('T')[1].slice(0, 8);
debugLog('🚫', `Rate limit hit (15/min) — cooldown until ${cooldownUntil}${Date.now() - verifyStartMs}ms`);
reportScannerError('Too many scan attempts. Please wait a moment and try again.');
cardTracker.status = 'negative';
cardTracker.negativeAt = Date.now();
@ -328,13 +353,16 @@ export function useScannerIdentification({
if (identification.handled && identification.outcome) {
cardTracker.status = 'confirmed';
debugLog('✅', `Card verified successfully → ${Date.now() - verifyStartMs}ms total`, { outcome: identification.outcome.type });
await applyIdentifyOutcome(cardTracker, identification.imageData, identification.outcome);
} else {
debugLog('❌', `Verification failed (no outcome) → ${Date.now() - verifyStartMs}ms`);
cardTracker.status = 'negative';
cardTracker.negativeAt = Date.now();
}
} catch (error) {
console.error(`Error verifying card ${cardTracker.id}:`, error);
debugLog('💥', `Verification exception → ${Date.now() - verifyStartMs}ms`, { error: error.message });
cardTracker.status = 'negative';
cardTracker.negativeAt = Date.now();
reportScannerError(error.message || 'Scan failed');