8c58990fd9
Delete the public /api/config/gemini endpoint and remove client auto-load paths so GEMINI_AI_API_KEY stays server-side only. Add a scan rate-limit class for the upcoming server-side identify route and a CI gate that blocks reintroducing config key leaks or new browser LLM URLs. Co-authored-by: Cursor <cursoragent@cursor.com>
146 lines
4.9 KiB
Markdown
146 lines
4.9 KiB
Markdown
---
|
||
name: add-real-ocr-layer
|
||
classification: feature
|
||
success_metric: |
|
||
≥70% of legitimate scans resolve at Layer-1 (Tesseract + pg_trgm) with
|
||
zero Gemini calls; scan_attempts.layer distribution proves it.
|
||
skip:
|
||
- ia
|
||
status: open
|
||
created: 2026-05-27
|
||
depends_on:
|
||
- server-side-scan-pipeline
|
||
---
|
||
|
||
# Convoy: add-real-ocr-layer
|
||
|
||
Add a cheap local OCR + fuzzy DB match layer so most scans never hit Gemini.
|
||
|
||
## Why
|
||
|
||
Every scan today (post convoy #2) calls Gemini Flash server-side. That is
|
||
slow, costs money, and hits rate limits under bulk scanning. The audit
|
||
proposed a two-layer pipeline: Layer-1 runs Tesseract on the card name strip
|
||
+ `pg_trgm` similarity against `cards.name`; only escalates to
|
||
`/api/scan/identify` (Gemini) when confidence is low. Target: **≥70%
|
||
Layer-1 hit rate** measured via `scan_attempts.layer`.
|
||
|
||
## Scope
|
||
|
||
### In scope
|
||
|
||
- **Migration** — enable `pg_trgm` extension; GIN index on
|
||
`cards (name gin_trgm_ops)`.
|
||
- **`pages/api/cards/identify-by-text.js`** (new):
|
||
- similarity > 0.85 → single match
|
||
- 0.6–0.85 → disambiguation list
|
||
- < 0.6 → `{ escalate: true }` (client falls back to `/api/scan/identify`)
|
||
- **`tesseract.js`** dependency + **`lib/ocr-worker.js`** (new) — browser
|
||
Worker wrapping Tesseract; OCR name strip before API calls.
|
||
- **`components/CameraScanner.js`** — integrate Worker: try Layer-1 path
|
||
first, escalate on low confidence.
|
||
- **`docs/SCHEMA_MAP.md`** — index + extension documented.
|
||
- **`.github/workflows/ci.yml`** — piggyback fix: extend `schema-map-fresh`
|
||
job `if:` condition to include `migrations/**` paths (currently only
|
||
watches `scripts/add-*`, `scripts/fix-*`, `setup-neon-db.js`, and
|
||
`docs/SCHEMA_MAP.md` — misses post-`migration-tool` migrations).
|
||
|
||
### Out of scope
|
||
|
||
- **OpenCV perspective transform / card boundary detection** — queue
|
||
`improve-scan-card-detection` if Layer-1 hit rate stays below 70% after
|
||
this lands.
|
||
- **Retraining or custom ML models** — Tesseract + trigram is sufficient
|
||
for v1.
|
||
- **Scanner UX redesign** — convoy `redesign-scanner-flow` (#4).
|
||
|
||
## Roles invoked
|
||
|
||
1. `role-ux-reviewer` — Layer-1 vs escalation feedback (scanning status).
|
||
2. `role-architect` — similarity thresholds, Worker bundling, 2 briefs.
|
||
3. `role-implementer` — 2 briefs (Brief 2 depends on Brief 1).
|
||
4. `role-reviewer` + `role-design-system-auditor` + `role-a11y-auditor`.
|
||
|
||
## Todos
|
||
|
||
- [ ] Architect: ratify similarity thresholds + Worker load strategy
|
||
- [ ] Brief 1 — migration (pg_trgm) + identify-by-text route + SCHEMA_MAP + CI schema-map-fresh fix
|
||
- [ ] Brief 2 — Tesseract Worker + CameraScanner integration
|
||
- [ ] Post-ship: measure `scan_attempts.layer` distribution for 70% target
|
||
|
||
## Operator action required
|
||
|
||
**None.** No new secrets. Tesseract runs client-side; pg_trgm is a Postgres
|
||
extension enabled via migration.
|
||
|
||
## Multitask dispatch
|
||
|
||
### Slice dependencies
|
||
|
||
```yaml
|
||
slice_dependencies:
|
||
- brief: 1
|
||
depends_on: []
|
||
files:
|
||
- migrations/*
|
||
- pages/api/cards/identify-by-text.js
|
||
- docs/SCHEMA_MAP.md
|
||
- .github/workflows/ci.yml
|
||
- brief: 2
|
||
depends_on: [1]
|
||
files:
|
||
- components/CameraScanner.js
|
||
- lib/ocr-worker.js
|
||
- package.json
|
||
```
|
||
|
||
Serial dispatch: Brief 2 after Brief 1 (Worker calls identify-by-text route).
|
||
|
||
Post-PR audit:
|
||
|
||
```
|
||
/multitask role-reviewer + role-design-system-auditor + role-a11y-auditor
|
||
```
|
||
|
||
Group id: `audit-add-real-ocr-layer-<pr>`.
|
||
|
||
## CI impact
|
||
|
||
| Workflow / job | Behavior |
|
||
| --- | --- |
|
||
| `schema-map-fresh` | **Modified** — `if:` paths include `migrations/**`. |
|
||
| `forbidden-client-side-llm-keys` | Unchanged (no new client LLM URLs). |
|
||
| `preview-smoke.yml` | Fires. |
|
||
| `visual-diff.yml` | **Fires** — `components/CameraScanner.js` in paths. |
|
||
|
||
No new grep gate. Total added CI time: ~0 beyond existing workflows.
|
||
|
||
## Decisions to ratify (architect)
|
||
|
||
1. **Similarity thresholds** — 0.85 / 0.6 defaults from audit; tune with
|
||
sample set.
|
||
2. **Tesseract language data** — bundled vs CDN fetch; impact on first-load
|
||
latency.
|
||
3. **`scan_attempts.layer` values** — recommend `1 | 2` (trgm vs gemini).
|
||
|
||
## Known constraints
|
||
|
||
- **`pg_trgm` on Neon** — verify extension availability on prod tier.
|
||
- **Worker + Turbopack** — confirm `tesseract.js` Worker path works under
|
||
Next.js 16 default bundler; fallback `--webpack` only if architect
|
||
documents regression.
|
||
- **Layer-1 must not block camera** — Worker runs off main thread.
|
||
|
||
## Acceptance criteria
|
||
|
||
1. Migration applies cleanly; GIN index exists on `cards.name`.
|
||
2. `identify-by-text` returns match / disambig / escalate per thresholds.
|
||
3. CameraScanner tries Layer-1 before `/api/scan/identify`.
|
||
4. `scan_attempts.layer` populated for analytics.
|
||
5. `schema-map-fresh` fires when only `migrations/` changes.
|
||
6. Lint + vitest baseline preserved.
|
||
|
||
## Out of scope follow-ups
|
||
|
||
- **`improve-scan-card-detection`** — if Layer-1 hit rate < 70%.
|
||
- **`god-component-split`** — CameraScanner remains large; split is P2.
|