208 lines
8 KiB
Markdown
208 lines
8 KiB
Markdown
|
---
|
|||
|
|
name: scanner-identify-upgrade
|
|||
|
|
classification: feature
|
|||
|
|
success_metric: |
|
|||
|
|
Of legitimate card scans (excluding not_a_card), ≥50% auto-match a
|
|||
|
|
catalog printing with no picker; Layer-1 escalate rate falls from 76.7%
|
|||
|
|
to ≤40%; median /api/scan/identify latency stays near today's 1.7s p50
|
|||
|
|
or improves. Measured on scan_attempts after Phase 1 ships.
|
|||
|
|
skip: []
|
|||
|
|
status: open
|
|||
|
|
created: 2026-08-14
|
|||
|
|
depends_on:
|
|||
|
|
- add-real-ocr-layer
|
|||
|
|
- server-side-scan-pipeline
|
|||
|
|
model_policy:
|
|||
|
|
default_session: auto
|
|||
|
|
roles:
|
|||
|
|
role-conductor: composer-2.5-fast
|
|||
|
|
role-architect: composer-2.5
|
|||
|
|
role-ia-architect: composer-2.5-fast
|
|||
|
|
role-ux-reviewer: composer-2.5-fast
|
|||
|
|
role-ui-designer: composer-2.5-fast
|
|||
|
|
role-implementer: composer-2.5-fast
|
|||
|
|
role-reviewer: cursor-grok-4.5-high
|
|||
|
|
role-security-auditor: gpt-5.6-terra-medium
|
|||
|
|
role-design-system-auditor: cursor-grok-4.5-high
|
|||
|
|
role-a11y-auditor: cursor-grok-4.5-high
|
|||
|
|
role-doc-writer: auto
|
|||
|
|
escalate_to: claude-sonnet-5-thinking-medium
|
|||
|
|
escalate_to_premium: claude-4.6-opus-high-thinking
|
|||
|
|
never_premium:
|
|||
|
|
- role-reviewer
|
|||
|
|
- role-security-auditor
|
|||
|
|
- role-design-system-auditor
|
|||
|
|
- role-a11y-auditor
|
|||
|
|
- role-ui-designer
|
|||
|
|
- role-doc-writer
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Scanner identify upgrade — epic
|
|||
|
|
|
|||
|
|
Umbrella for making the camera scanner **faster and more accurate**
|
|||
|
|
without replacing the two-layer shape (cheap local path, then
|
|||
|
|
server-owned Gemini). Planning-only in this file. Each numbered
|
|||
|
|
sub-convoy is its own gated PR stream.
|
|||
|
|
|
|||
|
|
Worktree: `tcg-vault-worktrees/scanner-identify-upgrade` on
|
|||
|
|
`convoy/scanner-identify-upgrade` (branched from `main` @ `c6c1364`).
|
|||
|
|
Do **not** land these docs on `feat/scanner-mobile-checkout` — that
|
|||
|
|
convoy owns scanner chrome/cart and explicitly leaves identify out of
|
|||
|
|
scope.
|
|||
|
|
|
|||
|
|
## Why
|
|||
|
|
|
|||
|
|
`add-real-ocr-layer` (PR #38) targeted ≥70% of scans resolving at
|
|||
|
|
Layer-1 (Tesseract + `pg_trgm`) with zero Gemini calls. Live
|
|||
|
|
`scan_attempts` from the local Neon (2026-05-27 → 2026-08-12, n=224,
|
|||
|
|
1 user) shows that target was missed by a wide margin:
|
|||
|
|
|
|||
|
|
| Signal | Value | Target / note |
|
|||
|
|
| --- | --- | --- |
|
|||
|
|
| L1 auto-match (`result_kind=matched`) | **5.8% of L1** / 2.7% of all | ≥70% L1 resolve |
|
|||
|
|
| L1 escalate | **76.7% of L1** (79/103) | Should be the minority path |
|
|||
|
|
| L1 disambiguation | 17.5% of L1 (18/103) | Name-only matcher cannot pick printings |
|
|||
|
|
| End-to-end auto-match | **10.3%** (23/224) | User still picks or retries most cards |
|
|||
|
|
| L2 `not_a_card` | **44.6% of L2** (54/121) | Detector fires on non-cards |
|
|||
|
|
| L2 p50 / p90 latency | **1655 / 2349 ms** | Plus 3.5s hold-still *before* identify |
|
|||
|
|
| L1 p50 latency | **158 ms** | Fast, but almost never uniquely matches |
|
|||
|
|
| `card_submissions` | 27 pending, 0 reviewed | Catalog-gap queue is unread |
|
|||
|
|
|
|||
|
|
The six L1 "matches" include noisy Tesseract strips (`J ——`, `Rock
|
|||
|
|
Jockey ©`) at confidence 36–68. L1 is a cheap filter that rarely
|
|||
|
|
identifies a printing. Meanwhile every card waits
|
|||
|
|
`DETECTION_START_DELAY_MS` (1s) + `MIN_FIRST_SEEN_MS_FOR_VERIFY` (2.5s)
|
|||
|
|
before OCR starts, and L2 is capped at 5 Gemini calls / user / minute.
|
|||
|
|
|
|||
|
|
Users feel this as: hold still forever, then pick from a list, or get
|
|||
|
|
"not a card" / "saved for review."
|
|||
|
|
|
|||
|
|
## Scope
|
|||
|
|
|
|||
|
|
### In scope (this epic)
|
|||
|
|
|
|||
|
|
1. **`tighten-scan-identify-hot-path`** — Phase 1. Cut the hold-still
|
|||
|
|
gate, OCR collector number on L1, structured Gemini JSON, optional
|
|||
|
|
`SCAN_VISION_MODEL` bump, stop the automatic second Gemini refine.
|
|||
|
|
2. **`improve-scan-card-detection`** — Phase 2. Replace the 320×240
|
|||
|
|
Sobel brute-force detector with a real card crop + perspective warp.
|
|||
|
|
This was queued in `add-real-ocr-layer` and never opened.
|
|||
|
|
3. **`scan-visual-catalog-search`** — Phase 3. Embed catalog
|
|||
|
|
`image_url`s; nearest-neighbor the warped crop. Gemini becomes
|
|||
|
|
fallback. This is the accuracy leap.
|
|||
|
|
|
|||
|
|
### Out of scope (this epic)
|
|||
|
|
|
|||
|
|
- Scanner chrome, cart, checkout (`scanner-mobile-checkout` /
|
|||
|
|
`scanner-rebuild`). Identify libs only.
|
|||
|
|
- Training a custom card CNN or standing up a Python/CUDA OCR service.
|
|||
|
|
- Swapping Tesseract for EasyOCR / PaddleOCR as the *primary* identifier.
|
|||
|
|
- Raising the L2 rate limit until Phase 1 hit-rate is re-measured.
|
|||
|
|
- Admin review of the 27 pending `card_submissions` (ops, not this epic).
|
|||
|
|
|
|||
|
|
## Baseline (do not re-query to "start" Phase 1)
|
|||
|
|
|
|||
|
|
Pulled 2026-08-14 from local `.env.local` → Neon `scan_attempts`.
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
layer | result_kind | n | pct
|
|||
|
|
1 | escalate | 79 | 35.3
|
|||
|
|
1 | disambiguation | 18 | 8.0
|
|||
|
|
1 | matched | 6 | 2.7
|
|||
|
|
2 | not_a_card | 54 | 24.1
|
|||
|
|
2 | submitted | 26 | 11.6
|
|||
|
|
2 | disambiguation | 22 | 9.8
|
|||
|
|
2 | matched | 17 | 7.6
|
|||
|
|
2 | needs_input | 2 | 0.9
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
L1 escalate text-length buckets (all had ≥3 chars — Tesseract is
|
|||
|
|
emitting text that `pg_trgm` cannot match): 3–7 chars n=28; 8–19 n=30;
|
|||
|
|
20+ n=21.
|
|||
|
|
|
|||
|
|
Re-measure with the same grouping after each sub-convoy ships.
|
|||
|
|
|
|||
|
|
## Dependency graph
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
[baseline pulled 2026-08-14]
|
|||
|
|
│
|
|||
|
|
▼
|
|||
|
|
┌───────────────────────────────────┐
|
|||
|
|
│ 1. tighten-scan-identify-hot-path │
|
|||
|
|
│ gates, collector #, schema JSON│
|
|||
|
|
└───────────────┬───────────────────┘
|
|||
|
|
│
|
|||
|
|
▼
|
|||
|
|
┌───────────────────────────────────┐
|
|||
|
|
│ 2. improve-scan-card-detection │
|
|||
|
|
│ detect + homography crop │
|
|||
|
|
└───────────────┬───────────────────┘
|
|||
|
|
│
|
|||
|
|
▼
|
|||
|
|
┌───────────────────────────────────┐
|
|||
|
|
│ 3. scan-visual-catalog-search │
|
|||
|
|
│ embeddings + pgvector kNN │
|
|||
|
|
└───────────────────────────────────┘
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**Strict-blockers:** #1 before #2 only if #2 would retune the same
|
|||
|
|
stability constants — otherwise #1 (identify files) and #2 (detection
|
|||
|
|
files) are file-disjoint and may run in parallel after Architect
|
|||
|
|
confirms. #3 needs a stable crop (#2) to be worth the embedding job;
|
|||
|
|
do not start #3 until #2 has a warped JPEG.
|
|||
|
|
|
|||
|
|
**Sibling:** `scanner-mobile-checkout` (other worktree / branch) must
|
|||
|
|
not edit `lib/ocr-worker.js`, `lib/scan-vision.js`,
|
|||
|
|
`lib/card-text-match.js`, `lib/scanner-card-identify.js`,
|
|||
|
|
`lib/scanner-card-detection.js`, or `/api/scan/identify`.
|
|||
|
|
|
|||
|
|
## Roles invoked
|
|||
|
|
|
|||
|
|
Umbrella is planning-only. Each sub-convoy lists its own roles.
|
|||
|
|
Default for #1: Architect → Implementer → audit fan-out (reviewer +
|
|||
|
|
security-auditor). UX reviewer on #1 and #2. Skip IA and UI Designer
|
|||
|
|
on all three (no new routes or visual language).
|
|||
|
|
|
|||
|
|
## Todos
|
|||
|
|
|
|||
|
|
- [x] Pull `scan_attempts` baseline (2026-08-14)
|
|||
|
|
- [x] Open worktree `scanner-identify-upgrade` from `main`
|
|||
|
|
- [x] Seed sub-convoys #1–#3
|
|||
|
|
- [ ] Architect: pick up `tighten-scan-identify-hot-path` first
|
|||
|
|
- [ ] Re-measure `scan_attempts` after #1 ships
|
|||
|
|
- [ ] Architect: `improve-scan-card-detection` (or parallel if files stay disjoint)
|
|||
|
|
- [ ] Architect: `scan-visual-catalog-search` after warped crops exist
|
|||
|
|
|
|||
|
|
## Worktree
|
|||
|
|
|
|||
|
|
| Checkout | Branch | Purpose |
|
|||
|
|
| --- | --- | --- |
|
|||
|
|
| `tcg-vault-worktrees/scanner-identify-upgrade` | `convoy/scanner-identify-upgrade` | These convoy docs |
|
|||
|
|
| (later) Agents Window worktree per brief | `brief/scanner-identify-upgrade/<n>-<title>` | Implementer slices |
|
|||
|
|
|
|||
|
|
`scripts/wt.sh` is a deprecation stub. Create implementer worktrees from
|
|||
|
|
the Agents Window after Architect writes `slice_dependencies:`.
|
|||
|
|
|
|||
|
|
## Multitask dispatch
|
|||
|
|
|
|||
|
|
No implementer fan-out from this umbrella. After #1 Architect marks
|
|||
|
|
parallel-safe briefs (`depends_on: []` + disjoint `files:`):
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
/multitask role-implementer briefs <ids>
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
After each PR draft:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
/multitask role-reviewer + role-security-auditor
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Add design-system + a11y auditors only if the PR touches
|
|||
|
|
`components/` or `pages/scanner.js` (not expected in #1).
|
|||
|
|
|
|||
|
|
Group ids: `audit-tighten-scan-identify-hot-path-<pr>`,
|
|||
|
|
`audit-improve-scan-card-detection-<pr>`,
|
|||
|
|
`audit-scan-visual-catalog-search-<pr>`.
|