deckhearth/.convoys/add-real-ocr-layer.md

---
name: add-real-ocr-layer
classification: feature
success_metric: |
  ≥70% of legitimate scans resolve at Layer-1 (Tesseract + pg_trgm) with
  zero Gemini calls; scan_attempts.layer distribution proves it.
skip:
  - ia
status: shipped
created: 2026-05-27
depends_on:
  - server-side-scan-pipeline
---

# Convoy: add-real-ocr-layer

**As-shipped:** PR #38 (+ follow-up polish in PR #39). Layer-1 Tesseract + `pg_trgm` before Gemini escalation.

Add a cheap local OCR + fuzzy DB match layer so most scans never hit Gemini.

## Why

Every scan today (post convoy #2) calls Gemini Flash server-side. That is
slow, costs money, and hits rate limits under bulk scanning. The audit
proposed a two-layer pipeline: Layer-1 runs Tesseract on the card name strip
+ `pg_trgm` similarity against `cards.name`; only escalates to
`/api/scan/identify` (Gemini) when confidence is low. Target: **≥70%
Layer-1 hit rate** measured via `scan_attempts.layer`.

## Scope

### In scope

- **Migration** — enable `pg_trgm` extension; GIN index on
  `cards (name gin_trgm_ops)`.
- **`pages/api/cards/identify-by-text.js`** (new):
  - similarity > 0.85 → single match
  - 0.6–0.85 → disambiguation list
  - < 0.6 → `{ escalate: true }` (client falls back to `/api/scan/identify`)
- **`tesseract.js`** dependency + **`lib/ocr-worker.js`** (new) — browser
  Worker wrapping Tesseract; OCR name strip before API calls.
- **`components/CameraScanner.js`** — integrate Worker: try Layer-1 path
  first, escalate on low confidence.
- **`docs/SCHEMA_MAP.md`** — index + extension documented.
- **`.github/workflows/ci.yml`** — piggyback fix: extend `schema-map-fresh`
  job `if:` condition to include `migrations/**` paths (currently only
  watches `scripts/add-*`, `scripts/fix-*`, `setup-neon-db.js`, and
  `docs/SCHEMA_MAP.md` — misses post-`migration-tool` migrations).

### Out of scope

- **OpenCV perspective transform / card boundary detection** — queue
  `improve-scan-card-detection` if Layer-1 hit rate stays below 70% after
  this lands.
- **Retraining or custom ML models** — Tesseract + trigram is sufficient
  for v1.
- **Scanner UX redesign** — convoy `redesign-scanner-flow` (#4).

## Roles invoked

1. `role-ux-reviewer` — Layer-1 vs escalation feedback (scanning status).
2. `role-architect` — similarity thresholds, Worker bundling, 2 briefs.
3. `role-implementer` — 2 briefs (Brief 2 depends on Brief 1).
4. `role-reviewer` + `role-design-system-auditor` + `role-a11y-auditor`.

## Todos

- [ ] Architect: ratify similarity thresholds + Worker load strategy
- [ ] Brief 1 — migration (pg_trgm) + identify-by-text route + SCHEMA_MAP + CI schema-map-fresh fix
- [ ] Brief 2 — Tesseract Worker + CameraScanner integration
- [ ] Post-ship: measure `scan_attempts.layer` distribution for 70% target

## Operator action required

**None.** No new secrets. Tesseract runs client-side; pg_trgm is a Postgres
extension enabled via migration.

## Multitask dispatch

### Slice dependencies

```yaml
slice_dependencies:
  - brief: 1
    depends_on: []
    files:
      - migrations/*
      - pages/api/cards/identify-by-text.js
      - docs/SCHEMA_MAP.md
      - .github/workflows/ci.yml
  - brief: 2
    depends_on: [1]
    files:
      - components/CameraScanner.js
      - lib/ocr-worker.js
      - package.json
```

Serial dispatch: Brief 2 after Brief 1 (Worker calls identify-by-text route).

Post-PR audit:

```
/multitask role-reviewer + role-design-system-auditor + role-a11y-auditor
```

Group id: `audit-add-real-ocr-layer-<pr>`.

## CI impact

| Workflow / job | Behavior |
| --- | --- |
| `schema-map-fresh` | **Modified** — `if:` paths include `migrations/**`. |
| `forbidden-client-side-llm-keys` | Unchanged (no new client LLM URLs). |
| `preview-smoke.yml` | Fires. |
| `visual-diff.yml` | **Fires** — `components/CameraScanner.js` in paths. |

No new grep gate. Total added CI time: ~0 beyond existing workflows.

## Decisions to ratify (architect)

1. **Similarity thresholds** — 0.85 / 0.6 defaults from audit; tune with
   sample set.
2. **Tesseract language data** — bundled vs CDN fetch; impact on first-load
   latency.
3. **`scan_attempts.layer` values** — recommend `1 | 2` (trgm vs gemini).

## Known constraints

- **`pg_trgm` on Neon** — verify extension availability on prod tier.
- **Worker + Turbopack** — confirm `tesseract.js` Worker path works under
  Next.js 16 default bundler; fallback `--webpack` only if architect
  documents regression.
- **Layer-1 must not block camera** — Worker runs off main thread.

## Acceptance criteria

1. Migration applies cleanly; GIN index exists on `cards.name`.
2. `identify-by-text` returns match / disambig / escalate per thresholds.
3. CameraScanner tries Layer-1 before `/api/scan/identify`.
4. `scan_attempts.layer` populated for analytics.
5. `schema-map-fresh` fires when only `migrations/` changes.
6. Lint + vitest baseline preserved.

## Out of scope follow-ups

- **`improve-scan-card-detection`** — if Layer-1 hit rate < 70%.
- **`god-component-split`** — CameraScanner remains large; split is P2.