deckhearth/.convoys/add-real-ocr-layer.md

name	classification	success_metric	skip	status	created	depends_on
add-real-ocr-layer	feature	≥70% of legitimate scans resolve at Layer-1 (Tesseract + pg_trgm) with zero Gemini calls; scan_attempts.layer distribution proves it.	ia	open	2026-05-27	server-side-scan-pipeline

Convoy: add-real-ocr-layer

Add a cheap local OCR + fuzzy DB match layer so most scans never hit Gemini.

Why

Every scan today (post convoy #2) calls Gemini Flash server-side. That is slow, costs money, and hits rate limits under bulk scanning. The audit proposed a two-layer pipeline: Layer-1 runs Tesseract on the card name strip

• pg_trgm similarity against cards.name; only escalates to /api/scan/identify (Gemini) when confidence is low. Target: ≥70% Layer-1 hit rate measured via scan_attempts.layer.

Scope

In scope

• Migration — enable pg_trgm extension; GIN index on cards (name gin_trgm_ops).
• pages/api/cards/identify-by-text.js (new):
  • similarity > 0.85 → single match
  • 0.6–0.85 → disambiguation list
  • < 0.6 → { escalate: true } (client falls back to /api/scan/identify)
• tesseract.js dependency + lib/ocr-worker.js (new) — browser Worker wrapping Tesseract; OCR name strip before API calls.
• components/CameraScanner.js — integrate Worker: try Layer-1 path first, escalate on low confidence.
• docs/SCHEMA_MAP.md — index + extension documented.
• .github/workflows/ci.yml — piggyback fix: extend schema-map-fresh job if: condition to include migrations/** paths (currently only watches scripts/add-*, scripts/fix-*, setup-neon-db.js, and docs/SCHEMA_MAP.md — misses post-migration-tool migrations).

Out of scope

• OpenCV perspective transform / card boundary detection — queue improve-scan-card-detection if Layer-1 hit rate stays below 70% after this lands.
• Retraining or custom ML models — Tesseract + trigram is sufficient for v1.
• Scanner UX redesign — convoy redesign-scanner-flow (#4).

Roles invoked

1. role-ux-reviewer — Layer-1 vs escalation feedback (scanning status).
2. role-architect — similarity thresholds, Worker bundling, 2 briefs.
3. role-implementer — 2 briefs (Brief 2 depends on Brief 1).
4. role-reviewer + role-design-system-auditor + role-a11y-auditor.

Todos

• Architect: ratify similarity thresholds + Worker load strategy
• Brief 1 — migration (pg_trgm) + identify-by-text route + SCHEMA_MAP + CI schema-map-fresh fix
• Brief 2 — Tesseract Worker + CameraScanner integration
• Post-ship: measure scan_attempts.layer distribution for 70% target

Operator action required

None. No new secrets. Tesseract runs client-side; pg_trgm is a Postgres extension enabled via migration.

Multitask dispatch

Slice dependencies

slice_dependencies:
  - brief: 1
    depends_on: []
    files:
      - migrations/*
      - pages/api/cards/identify-by-text.js
      - docs/SCHEMA_MAP.md
      - .github/workflows/ci.yml
  - brief: 2
    depends_on: [1]
    files:
      - components/CameraScanner.js
      - lib/ocr-worker.js
      - package.json

Serial dispatch: Brief 2 after Brief 1 (Worker calls identify-by-text route).

Post-PR audit:

/multitask role-reviewer + role-design-system-auditor + role-a11y-auditor

Group id: audit-add-real-ocr-layer-<pr>.

CI impact

Workflow / job	Behavior
schema-map-fresh	Modified — if: paths include migrations/**.
forbidden-client-side-llm-keys	Unchanged (no new client LLM URLs).
preview-smoke.yml	Fires.
visual-diff.yml	Fires — components/CameraScanner.js in paths.

No new grep gate. Total added CI time: ~0 beyond existing workflows.

Decisions to ratify (architect)

1. Similarity thresholds — 0.85 / 0.6 defaults from audit; tune with sample set.
2. Tesseract language data — bundled vs CDN fetch; impact on first-load latency.
3. scan_attempts.layer values — recommend 1 | 2 (trgm vs gemini).

Known constraints

• pg_trgm on Neon — verify extension availability on prod tier.
• Worker + Turbopack — confirm tesseract.js Worker path works under Next.js 16 default bundler; fallback --webpack only if architect documents regression.
• Layer-1 must not block camera — Worker runs off main thread.

Acceptance criteria

1. Migration applies cleanly; GIN index exists on cards.name.
2. identify-by-text returns match / disambig / escalate per thresholds.
3. CameraScanner tries Layer-1 before /api/scan/identify.
4. scan_attempts.layer populated for analytics.
5. schema-map-fresh fires when only migrations/ changes.
6. Lint + vitest baseline preserved.

Out of scope follow-ups

• improve-scan-card-detection — if Layer-1 hit rate < 70%.
• god-component-split — CameraScanner remains large; split is P2.