deckhearth/.convoys/scanner-identify-upgrade.md

name	classification	success_metric	skip	status	created	depends_on	model_policy
scanner-identify-upgrade	feature	Of legitimate card scans (excluding not_a_card), ≥50% auto-match a catalog printing with no picker; Layer-1 escalate rate falls from 76.7% to ≤40%; median /api/scan/identify latency stays near today's 1.7s p50 or improves. Measured on scan_attempts after Phase 1 ships.		open	2026-08-14	add-real-ocr-layer server-side-scan-pipeline	default_session roles escalate_to escalate_to_premium never_premium auto role-conductor role-architect role-ia-architect role-ux-reviewer role-ui-designer role-implementer role-reviewer role-security-auditor role-design-system-auditor role-a11y-auditor role-doc-writer composer-2.5-fast composer-2.5 composer-2.5-fast composer-2.5-fast composer-2.5-fast composer-2.5-fast cursor-grok-4.5-high gpt-5.6-terra-medium cursor-grok-4.5-high cursor-grok-4.5-high auto claude-sonnet-5-thinking-medium claude-4.6-opus-high-thinking role-reviewer role-security-auditor role-design-system-auditor role-a11y-auditor role-ui-designer role-doc-writer

Scanner identify upgrade — epic

Umbrella for making the camera scanner faster and more accurate without replacing the two-layer shape (cheap local path, then server-owned Gemini). Planning-only in this file. Each numbered sub-convoy is its own gated PR stream.

Worktree: tcg-vault-worktrees/scanner-identify-upgrade on convoy/scanner-identify-upgrade (branched from main @ c6c1364). Do not land these docs on feat/scanner-mobile-checkout — that convoy owns scanner chrome/cart and explicitly leaves identify out of scope.

Why

add-real-ocr-layer (PR #38) targeted ≥70% of scans resolving at Layer-1 (Tesseract + pg_trgm) with zero Gemini calls. Live scan_attempts from the local Neon (2026-05-27 → 2026-08-12, n=224, 1 user) shows that target was missed by a wide margin:

Signal	Value	Target / note
L1 auto-match (result_kind=matched)	5.8% of L1 / 2.7% of all	≥70% L1 resolve
L1 escalate	76.7% of L1 (79/103)	Should be the minority path
L1 disambiguation	17.5% of L1 (18/103)	Name-only matcher cannot pick printings
End-to-end auto-match	10.3% (23/224)	User still picks or retries most cards
L2 not_a_card	44.6% of L2 (54/121)	Detector fires on non-cards
L2 p50 / p90 latency	1655 / 2349 ms	Plus 3.5s hold-still before identify
L1 p50 latency	158 ms	Fast, but almost never uniquely matches
card_submissions	27 pending, 0 reviewed	Catalog-gap queue is unread

The six L1 "matches" include noisy Tesseract strips (J ——, Rock Jockey ©) at confidence 36–68. L1 is a cheap filter that rarely identifies a printing. Meanwhile every card waits DETECTION_START_DELAY_MS (1s) + MIN_FIRST_SEEN_MS_FOR_VERIFY (2.5s) before OCR starts, and L2 is capped at 5 Gemini calls / user / minute.

Users feel this as: hold still forever, then pick from a list, or get "not a card" / "saved for review."

Scope

In scope (this epic)

1. tighten-scan-identify-hot-path — Phase 1. Cut the hold-still gate, OCR collector number on L1, structured Gemini JSON, optional SCAN_VISION_MODEL bump, stop the automatic second Gemini refine.
2. improve-scan-card-detection — Phase 2. Replace the 320×240 Sobel brute-force detector with a real card crop + perspective warp. This was queued in add-real-ocr-layer and never opened.
3. scan-visual-catalog-search — Phase 3. Embed catalog image_urls; nearest-neighbor the warped crop. Gemini becomes fallback. This is the accuracy leap.

Out of scope (this epic)

• Scanner chrome, cart, checkout (scanner-mobile-checkout / scanner-rebuild). Identify libs only.
• Training a custom card CNN or standing up a Python/CUDA OCR service.
• Swapping Tesseract for EasyOCR / PaddleOCR as the primary identifier.
• Raising the L2 rate limit until Phase 1 hit-rate is re-measured.
• Admin review of the 27 pending card_submissions (ops, not this epic).

Baseline (do not re-query to "start" Phase 1)

Pulled 2026-08-14 from local .env.local → Neon scan_attempts.

layer | result_kind    | n  | pct
    1 | escalate       | 79 | 35.3
    1 | disambiguation | 18 |  8.0
    1 | matched        |  6 |  2.7
    2 | not_a_card     | 54 | 24.1
    2 | submitted      | 26 | 11.6
    2 | disambiguation | 22 |  9.8
    2 | matched        | 17 |  7.6
    2 | needs_input    |  2 |  0.9

L1 escalate text-length buckets (all had ≥3 chars — Tesseract is emitting text that pg_trgm cannot match): 3–7 chars n=28; 8–19 n=30; 20+ n=21.

Re-measure with the same grouping after each sub-convoy ships.

Dependency graph

  [baseline pulled 2026-08-14]
              │
              ▼
  ┌───────────────────────────────────┐
  │ 1. tighten-scan-identify-hot-path │
  │    gates, collector #, schema JSON│
  └───────────────┬───────────────────┘
                  │
                  ▼
  ┌───────────────────────────────────┐
  │ 2. improve-scan-card-detection    │
  │    detect + homography crop       │
  └───────────────┬───────────────────┘
                  │
                  ▼
  ┌───────────────────────────────────┐
  │ 3. scan-visual-catalog-search     │
  │    embeddings + pgvector kNN      │
  └───────────────────────────────────┘

Strict-blockers: #1 before #2 only if #2 would retune the same stability constants — otherwise #1 (identify files) and #2 (detection files) are file-disjoint and may run in parallel after Architect confirms. #3 needs a stable crop (#2) to be worth the embedding job; do not start #3 until #2 has a warped JPEG.

Sibling: scanner-mobile-checkout (other worktree / branch) must not edit lib/ocr-worker.js, lib/scan-vision.js, lib/card-text-match.js, lib/scanner-card-identify.js, lib/scanner-card-detection.js, or /api/scan/identify.

Roles invoked

Umbrella is planning-only. Each sub-convoy lists its own roles. Default for #1: Architect → Implementer → audit fan-out (reviewer + security-auditor). UX reviewer on #1 and #2. Skip IA and UI Designer on all three (no new routes or visual language).

Todos

• Pull scan_attempts baseline (2026-08-14)
• Open worktree scanner-identify-upgrade from main
• Seed sub-convoys #1–#3
• Architect: pick up tighten-scan-identify-hot-path first
• Re-measure scan_attempts after all phases ship (2026-08-15 — see below)
• Architect: improve-scan-card-detection
• Architect: scan-visual-catalog-search after warped crops exist
• Operator: finish embedding backfill (npm run backfill-embeddings) — 14.9% done (9,865 / 66,211)
• Re-measure after backfill + L0 traffic (layer = 0 rows)

Post-ship telemetry (2026-08-15)

Pulled from Neon scan_attempts + cards.embedding coverage. Single user (user_id = 3); treat post-ship windows as directional, not statistically significant.

All-time (n=250, 2026-05-27 → 2026-08-15)

Signal	Value	Baseline (2026-08-14)	Target
L1 auto-match	5.4% of L1 (6/111)	5.8%	≥25%
L1 escalate	78.4% of L1 (87/111)	76.7%	≤40%
L2 not_a_card	43.9% of L2 (61/139)	44.6%	≤15%
End-to-end auto-match	13.2% excl. not_a_card (25/189)	13.5%	≥50%
L1 p50 latency	158 ms	158 ms	—
L2 p50 / p90	1642 / 2297 ms	1655 / 2349 ms	≤1800 p50
L0 attempts	0	—	(Phase 3 code live; index sparse)
card_submissions pending	31	27	ops

Aug 15 post-deploy session (n=26)

First scan burst after #156–#158 merged (~06:23–07:43 UTC). No L0 rows yet (#160 landed same evening; backfill incomplete).

Signal	Aug 15 (n=26)	Baseline
L1 auto-match	0% (0/8 L1)	5.8%
L1 escalate	100% (8/8 L1)	76.7%
L2 not_a_card	38.9% (7/18 L2)	44.6%
End-to-end auto-match	10.5% excl. not_a_card (2/19)	13.5%

Qualitative notes: L1 OCR strips post-warp are shorter garbage ("7", "Tey", "yr'") — collector-number path cannot help. L2 structured JSON (isCard: false) works. Two L2 auto-matches (Hunger Hawk, Hog-Monkey). Disambiguation still common (Seel printings).

Phase verdict

Phase	PR	Code	Success metrics
1 tighten-scan-identify-hot-path	#156	Shipped	❌ L1 match/escalate targets missed
2 improve-scan-card-detection	#158	Shipped	❌ not_a_card still ~44%; slight Aug 15 improvement
3 scan-visual-catalog-search	#160, #162	Shipped	⏳ Awaiting backfill + L0 traffic

Recommendation: Finish catalog embedding backfill, scan 50+ cards on preview, then re-query layer = 0 distribution before tuning thresholds.

Embedding backfill status (2026-08-15)

Metric	Value
Cards with image_url	66,211
Rows with embedding	9,865 (14.9%)
pgvector on Neon	✅ v0.8.0
Operator command	npm run backfill-embeddings

Worktree

Checkout	Branch	Purpose
tcg-vault-worktrees/scanner-identify-upgrade	convoy/scanner-identify-upgrade	These convoy docs
(later) Agents Window worktree per brief	brief/scanner-identify-upgrade/<n>-<title>	Implementer slices

scripts/wt.sh is a deprecation stub. Create implementer worktrees from the Agents Window after Architect writes slice_dependencies:.

Multitask dispatch

No implementer fan-out from this umbrella. After #1 Architect marks parallel-safe briefs (depends_on: [] + disjoint files:):

/multitask role-implementer briefs <ids>

After each PR draft:

/multitask role-reviewer + role-security-auditor

Add design-system + a11y auditors only if the PR touches components/ or pages/scanner.js (not expected in #1).

Group ids: audit-tighten-scan-identify-hot-path-<pr>, audit-improve-scan-card-detection-<pr>, audit-scan-visual-catalog-search-<pr>.