deckhearth/.convoys/add-rate-limiting.md
Randall Stillwell 60b842ee41 architect: add-rate-limiting (queued → in-progress; 1 brief, D1 routes back for operator gate-1)
Six decisions ratified — five architect-self-ratifiable, one
(D1, scope expansion to gate the three import routes with auth +
admin-role + import rate-limit, plus the matching Bearer-token fix in
pages/admin/card-import.js) routed back for operator approval before
implementer dispatch.

Architecture: single brief, ~180 LOC across 10 files. Lib refactor
preserves Brief 4's checkAuthRateLimit(req) contract; adds four named
exports (search/upload/generate/import) with a Map<className, Ratelimit>
cache and distinct Redis prefix per class. Per-class limits tuned
against real client behavior — search bumped from 30 to 60/min after
finding ShareModal handleSearch has no debounce; generate bumped from 3
to 5/hour after confirming the generator uses DiceBear (free) not a
paid AI service.

Brief includes verbatim new lib/rate-limit.js shape so the implementer
has zero design discretion, plus per-route verbatim post-edit shapes
for all seven gated handlers and the admin-UI fetch fix.

Decision 1 investigation finding: pages/admin/card-import.js currently
fetches without an Authorization header. Without the matching fix in
this brief, adding getUserFromRequest to the import APIs immediately
breaks the admin UI on first run — the brief is internally consistent
only with both edits shipping together.

Decision 6 correction: the convoy file's claim that vitest transitively
loads lib/rate-limit.js is stale (rg verified no test imports it). The
lib refactor is strictly safer than the convoy file implied.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-24 22:30:09 -05:00

676 lines
35 KiB
Markdown

---
name: add-rate-limiting
classification: convoy
success_metric: |
All endpoints listed in P0 #6 of `.convoys/ship-readiness.md` carry
a rate-limit gate appropriate to their abuse class (search /
upload / import / admin). The card-import endpoints are no longer
anonymous-callable. `lib/rate-limit.js` exposes named limiters
per route class with documented limits + windows + key-extraction
strategies. P0 #6 flips PARTIAL → RESOLVED, closing the last
open P0 ship-blocker.
skip:
- role-design-system-auditor
- role-a11y-auditor
- role-ux-reviewer
- role-ia-architect
status: in-progress
created: 2026-05-24
parent: ship-readiness
addresses: P0 #6 (PARTIAL → RESOLVED) — the LAST open P0
depends_on:
- fix-auth-bypass (Brief 4 shipped lib/rate-limit.js + auth-only limiter)
- cors-tighten (just closed P0 #5; not a hard dep but the API surface is now CORS-clean)
---
# Convoy: add-rate-limiting
Extend `lib/rate-limit.js` and wire the result into the remaining
abusable endpoints. This is the final P0 — when it lands, the
launch-readiness ship-blocker list is empty (all 8 of 8 RESOLVED).
## Why now
`fix-auth-bypass` Brief 4 shipped the rate-limit infrastructure
(`lib/rate-limit.js`, `@upstash/ratelimit@^2.0.8`,
`@upstash/redis@^1.38.0`, Vercel-Upstash Marketplace env vars
`KV_REST_API_URL` / `KV_REST_API_TOKEN`) and wired it into the
two auth endpoints (`login`, `register`) with a 5-attempts /
15-minute sliding window. The remaining abusable surface was
intentionally deferred to this convoy.
The "Queued convoys" entry in `.convoys/ship-readiness.md` lists
the surface as:
- `/api/users/search`
- `/api/cards/search`
- All `/api/cards/import-*`
- `/api/user/avatar*` (upload)
Parent's pre-architect audit at convoy creation surfaced a
**critical secondary finding** beyond "missing rate limit":
**`pages/api/cards/import-mtg.js`, `import-pokemon.js`,
`import-lorcana.js` have ZERO authentication checks.** They are
publicly callable, they hit Scryfall / Pokémon-TCG / Lorcana APIs
with no caller throttling AND no auth, and they perform UPSERTs
into the `cards` table. An attacker can:
1. Trigger expensive external API calls from your IP (Scryfall +
Pokémon-TCG have published rate limits; you'd hit them with no
recourse, getting your IP throttled at the source).
2. Cause unbounded DB writes (each import inserts hundreds-to-thousands
of rows; `card.id` lookups deduplicate but the INSERT path runs
in a tight per-card loop).
3. Even with a generous per-IP rate limit (5 req / hour), with
1000 IPs the math is 120,000 imports / day — enough to exhaust
Vercel function quotas and Neon row budgets.
The rate-limit alone is insufficient. **Auth-gate-then-rate-limit
is the correct shape.** Whether to add auth gates to the import
routes in this convoy (vs. spinning out a separate
`gate-import-routes` convoy) is **architect Decision 1** below.
Parent recommends in-scope; expansion is mid-convoy precedent
established by `drop-public-setup` (Brief 2's CJS/ESM expansion)
and other prior convoys.
## Scope
### Confirmed in-scope (regardless of architect Decision 1)
- **`lib/rate-limit.js`** — refactor to expose multiple named
limiters (one per route class), preserving the existing
`checkAuthRateLimit` export for `login` / `register` backwards
compatibility. New named limiters per Decision 2 below.
- **`pages/api/users/search.js`** — add rate-limit gate after the
existing JWT verification. Limit per Decision 2.
- **`pages/api/cards/search.js`** — add rate-limit gate at top of
handler (this endpoint is anonymous-by-design; rate-limit keyed
by IP).
- **`pages/api/user/avatar.js`** — add rate-limit gate after the
existing `getUserFromRequest` check. Limit keyed by `user.userId`
(authenticated; per-user makes more sense than per-IP for an
upload endpoint where a household might share an IP).
- **`pages/api/user/avatar/generate.js`** — same shape as
`avatar.js`. Per-user, stricter than avatar.js (AI generation
is more expensive than blob upload).
- **`.cursor/rules/api-routes.mdc`** § Rate limiting — update the
existing § with the new per-class pattern + each named limiter's
use case. The current § shows only the auth pattern.
### Scope-expansion-candidate (architect Decision 1)
- **`pages/api/cards/import-mtg.js`** — add `getUserFromRequest`
check at top of handler; return 403 if not admin (`user.role !==
'admin'`); then rate-limit gate (per-user, very strict — these
hit external APIs).
- **`pages/api/cards/import-pokemon.js`** — same shape.
- **`pages/api/cards/import-lorcana.js`** — same shape.
- (If Decision 1 = Option A "in scope": ship in this PR. If Option
B "spin out": queue `gate-import-routes` follow-up convoy and
mark P0 #6 RESOLVED-with-caveat at convoy close.)
### Out of scope
- **Global IP-based backstop limiter** (e.g., 1000 req / min per
IP across all routes via Next.js middleware). Useful but
separate scope; `add-global-rate-limit-middleware` would be its
own convoy.
- **Rate-limit headers on success responses**
(`X-RateLimit-Remaining`, `X-RateLimit-Reset`). The existing
`checkAuthRateLimit` returns `remaining` + `reset` in its
result object but `login.js` doesn't propagate them on
success; just on 429. Honoring the existing convention.
- **`@upstash/ratelimit` version bump** — pin stays at `^2.0.8`
from Brief 4. Bumping is its own convoy.
- **Per-route handler unit tests** (still deferred to the
queued `fill-vitest-handler-coverage` convoy — same reasoning
as `cors-tighten`).
- **Refactoring the existing `cards/search.js` SQL** (the file
has a known god-function shape with 7+ conditional SQL
branches; that's `god-function-split` scope, not here).
- **Verifying the avatar `parseMultipartFormData` body parser**
is rate-limit-safe (the body is read in `req.on('data')` before
any rate-limit gate could short-circuit; an attacker can still
exhaust the 5MB body even on a 429 path). That's a separate
body-streaming-defense concern (`harden-multipart-parser`) and
not in scope here.
## Operator action required
**None.** All Upstash env vars (`KV_REST_API_URL` /
`KV_REST_API_TOKEN`) are already auto-provisioned via the Vercel
Marketplace integration (seeded for `fix-auth-bypass` Brief 4).
No new secrets, no new dependencies (the `@upstash/ratelimit` +
`@upstash/redis` packages are already installed).
## Decisions to ratify with operator (architect routes)
1. **Scope expansion — auth gates on `pages/api/cards/import-*.js`
in this convoy?**
- **Option A — In scope (parent recommends).** Add
`getUserFromRequest` + admin-role check + rate-limit gate to
all 3 import routes in this PR. ~30 LOC across 3 files.
Closes the full P0 #6 attack surface in one shot. Precedent:
`drop-public-setup` Brief 2 (CJS/ESM expansion).
- **Option B — Spin out.** Stay narrow on the 4 originally-listed
routes; queue `gate-import-routes` as a separate convoy. Mark
P0 #6 RESOLVED-with-caveat noting the import-route auth gap.
- **Option C — Hybrid.** Add the rate-limit gate to import
routes now (with a clear `TODO: requires-auth` comment),
gate auth later. Worst of both worlds — leaves an
anonymous-callable abusive endpoint live with only IP-based
throttling.
- Architect investigates: confirm the import routes are
intended to be admin-only (per AGENTS.md / scripts/ folder
conventions), confirm Decision-A scope size is bounded, route
back recommendation.
2. **Named-limiter shape in `lib/rate-limit.js`.** The existing
`checkAuthRateLimit(req)` function has the limit + window
hardcoded. To support per-class limits cleanly, two main
patterns:
- **(a) Named-export functions.** Each route class gets its
own exported function: `checkSearchRateLimit`,
`checkUploadRateLimit`, `checkImportRateLimit`. Internal
`init()` builds one `Ratelimit` instance per class, cached
under different Redis prefixes. Verbose but very explicit at
each call site.
- **(b) Generic `checkRateLimit(req, options)`.** Single
exported function takes a config object. Calling code
becomes `await checkRateLimit(req, { class: 'search' })` or
similar. Less verbose at the lib but more at each call site,
and the per-route limits are documented in the lib module
rather than implicit in the function name.
- **(c) Hybrid.** Keep `checkAuthRateLimit` (used by login +
register; high-stakes contract; don't break). Add named
functions for the other 3 classes (search, upload, import).
Best of both — preserves Brief 4's contract, names new
classes explicitly.
- Parent recommends (c). Architect ratifies after reading
`lib/rate-limit.js` shape.
3. **Per-class limit values.** Reference table (architect's
to tune):
- **`auth`** (existing) — 5 / 15min, IP-keyed. (Don't change.)
- **`search`** (new) — recommend 30 / 1min, IP-keyed.
Legitimate users type-ahead-search; 30/min is generous but
stops a scraper in its tracks within 2-3s. Same IP-keying
as auth.
- **`upload`** (new — avatar.js, avatar/generate.js) —
recommend 10 / 1hour, user-keyed. Avatar uploads are rare;
10/hour catches accidental loops without blocking a user
who hits "save" three times by accident. User-keyed because
the user is authenticated and a household IP shouldn't
punish other users.
- **`generate`** (new — avatar/generate.js specifically) —
recommend 3 / 1hour, user-keyed. AI generation costs money;
stricter than plain upload.
- **`import`** (new — cards/import-*.js) — recommend 5 / 1hour,
user-keyed (if Decision 1 = Option A). Admin-triggered, hits
external APIs with their own rate limits; 5/hour is plenty.
4. **Identifier extraction.** `lib/rate-limit.js`'s
`extractIdentifier()` currently always extracts the first
`x-forwarded-for` IP. For per-user limiters (upload, generate,
import), we need a `user_id`-based extractor. Three options:
- **(a) Two extractors.** `extractIpIdentifier` (existing) +
`extractUserIdentifier(req, userId)` (new). Each limiter
calls the right one.
- **(b) Single extractor takes an `opts.user` flag.** Less
code but harder to read.
- **(c) Caller passes `key` directly.** Each handler calls
`checkUploadRateLimit(req, { key: user.userId })` and the lib
just trusts it.
- Parent recommends (a) — explicit, two small functions,
hard to misuse.
5. **429 response shape.** Match the existing
`login.js`/`register.js` pattern verbatim:
```js
res.setHeader('Retry-After', Math.ceil((reset - Date.now()) / 1000));
return res.status(429).json({ error: 'Too many attempts. Try again later.' });
```
Architect: should the error message vary per class (e.g., "Too
many search requests" vs "Too many uploads") or stay uniform?
Recommend uniform — minimizes attacker-fingerprinting of which
routes have which limits. Architect ratifies.
6. **Test coverage in this convoy.** No new per-route handler
tests (per Decision-4-equivalent from `cors-tighten`; same
deferred-to-`fill-vitest-handler-coverage` reasoning). The
Playwright smoke spec doesn't exercise any of these routes,
so no smoke coverage either. **The fail-loud-in-prod
semantics in `lib/rate-limit.js`'s `init()` function are the
integration test** — if the env vars are unset on a deployed
env, every limit-gated route fails closed on first call.
## Known constraints
- **`lib/rate-limit.js` already has the right architecture** for
multiple limiters (lazy `init()` returns a `cached` object;
refactor pattern is to make `cached` a Map<className,
Ratelimit> instead of a single instance). Don't rewrite from
scratch; extend.
- **Fail-closed semantics in prod must be preserved.** Brief 4's
`init()` throws when env vars are unset and `NODE_ENV ===
'production'`. New limiters must inherit this — a misconfigured
prod env should fail loud on every call, not silently disable
all rate limiting.
- **Card-search is anonymous-by-design** — do NOT add a
`getUserFromRequest` check. The IP-keyed limit is the correct
defense (search is a public catalogue feature).
- **User-keyed limits require an authenticated request.** The
rate-limit gate MUST sit AFTER the auth check in the handler
body. Wrong order = security regression (anonymous user bypasses
the gate because there's no `userId` to key on, the extractor
falls back to IP, and the per-user limit becomes per-IP — same
household IP could be locked out for a different user's
behavior).
- **Brief 4's `lib/rate-limit.js` is unit-tested transitively**
via the existing vitest suite (none of the 21 tests
specifically test rate-limit, but the auth-utils tests run
through the same module). Don't break those.
- **`@upstash/ratelimit` supports multiple shapes**
(`slidingWindow`, `fixedWindow`, `tokenBucket`); Brief 4 used
`slidingWindow`. New limiters should follow the same shape
unless the architect has a specific reason for a different
algorithm per class.
- **Per-route Pattern checks** — the 6 (or 9 if Decision 1 =
Option A) target files have different existing structures:
- `users/search.js`: auth check inside try/catch on JWT;
rate-limit goes after JWT verify, inside the existing try
- `cards/search.js`: no auth; rate-limit goes at top of
handler after the method check
- `user/avatar.js`: auth check inside the outer try (line
16-19); rate-limit goes after auth, before the method
branching at line 21
- `user/avatar/generate.js`: similar shape to avatar.js
- `cards/import-*.js`: NO auth currently. Decision 1 dictates
the shape — either add auth-then-rate-limit, or rate-limit-only
(Option C, not recommended)
## Acceptance criteria
The convoy is shippable when ALL of the following hold:
1. `lib/rate-limit.js` exposes the named limiters per Decision 2.
Backwards-compat: `checkAuthRateLimit` still works for
`login.js` / `register.js` (do NOT break Brief 4's contract).
2. All 4 originally-listed endpoints (`users/search`,
`cards/search`, `user/avatar`, `user/avatar/generate`) carry
the appropriate rate-limit gate at the correct ordering
(after auth if applicable; per Decision 4 keying).
3. If Decision 1 = Option A: all 3 import endpoints carry both
the auth gate (admin-role check) + the import rate-limit gate.
4. `npm run lint` exit code matches baseline (still 128 problems;
do NOT regress).
5. `npm run test:run` (vitest) still passes 21/21 (no regression
on the existing auth-utils tests that transitively load
`lib/rate-limit.js`).
6. `npm run test:smoke` (via CI on the PR) still passes 3/3
against the Vercel preview — proves the live login → verify
→ /api/health flow doesn't accidentally get rate-limited
(the smoke spec hits each endpoint once per run, well below
any limit).
7. `.cursor/rules/api-routes.mdc` § Rate limiting updated to
document each new limiter's use case + the per-class limits
+ the user-keyed vs IP-keyed convention.
8. Bypass-secret-leak check: zero matches of any
`KV_REST_API_*` or `UPSTASH_*` env-var values in any CI log.
(Auto-masked by GitHub Actions; verify post-merge.)
## Anything flagged but not acted on (in advance)
- **Global IP backstop limiter via Next.js middleware** —
out of scope; queue `add-global-rate-limit-middleware` if a
future audit shows non-listed routes being abused.
- **Body-streaming defense for avatar uploads** — the 5MB
multipart body is consumed before any rate-limit gate can
short-circuit. Real defense requires moving the parse into a
separate edge function or using `read-up-to` semantics; queue
`harden-multipart-parser` if it ever surfaces in a real abuse
incident.
- **Per-tier user limits** — premium users might get higher
search/upload limits. Queue `tiered-rate-limits` when a
tiering scheme exists (today there are only `admin` and
default roles).
- **Rate-limit metrics dashboard** — Upstash exposes per-key hit
counts; surfacing them in an admin dashboard would catch
abuse early. Queue `rate-limit-observability` post-launch.
- **Card-search SQL god-function** — known shape issue,
separate convoy (`god-function-split` or
`refactor-cards-search-sql`); do NOT touch in this convoy.
## Decisions (post-architect)
Six decisions ratified by `role-architect` on 2026-05-24. **Only Decision 1
requires operator gate-1 sign-off** (significant scope expansion to admin-only
import-route enforcement + admin UI source touch); the other five are
architect-self-ratifiable per the precedent established by `cors-tighten`
Decision D2-D5 + `fix-vercel-deployment-protection-in-ci` Decisions A/B/D.
### Decision 1 — Scope expansion: gate `pages/api/cards/import-*.js` in this convoy (**OPERATOR-RATIFIABLE**)
**Ratified: Option A — gate all three import routes with auth + admin-role + import rate-limit, AND fix the admin UI to send the Bearer token.**
Architect investigation results:
- **`pages/api/cards/import-mtg.js`** — LIVE admin tooling. Called by
`pages/admin/card-import.js` line 39-49 (the `<select>` defaults to
`'mtg'`). Currently fetched WITHOUT an `Authorization` header.
- **`pages/api/cards/import-pokemon.js`** — LIVE admin tooling. Same
admin UI fetch path; the `<select>` second option is `'pokemon'`.
- **`pages/api/cards/import-lorcana.js`** — DEAD in frontend. Architect
ran `rg 'import-lorcana' pages/ components/` and the file has zero
frontend callers; the admin UI's `<select>` only offers `'mtg'` and
`'pokemon'`. The standalone `scripts/import-lorcana.js` exists as an
independent CLI path. Gating defensively is the chosen path because:
(a) future Lorcana admin UI work inherits protection automatically,
(b) the diff is uniform (3 routes get identical treatment), (c) a
future cleanup convoy can delete the route if it stays unused — and
deletion is strictly easier than gating-then-deleting because Option A
preserves all the import infrastructure.
**Critical scope expansion required by Option A:**
`pages/admin/card-import.js` currently calls the import APIs without an
`Authorization` header (line 43-49). The moment the import APIs gain
`getUserFromRequest`, the admin UI begins returning 401 on every import
click — visible UX regression. The brief therefore includes a one-line
edit to `pages/admin/card-import.js` adding
`'Authorization': \`Bearer ${localStorage.getItem('auth_token')}\``
to the fetch's headers. This is the minimum touch; the 309-line
god-component is otherwise byte-identical.
**Rejected alternatives:**
- **Option B (spin out `gate-import-routes`).** Would leave a publicly-callable
abuse vector live for at least one more convoy cycle; closes P0 #6 only
partially. Rejected.
- **Option C (hybrid — rate-limit-only without auth).** Worst of both worlds:
anonymous abuse + only IP-based throttling. Rejected per the convoy spec.
- **Option D-full (delete all three import routes).** Would break the admin
UI immediately and require either deleting `pages/admin/card-import.js` too
or restructuring it to call something else. Wider scope than gating; defeats
the convoy's "close P0 #6 in one shot" success metric.
- **Option D-partial (delete only `import-lorcana.js`, gate mtg+pokemon).**
Considered. Slightly smaller attack surface, but introduces a non-uniform
pattern (two gated routes + one deleted route) that complicates the
reviewer's mental model. Lorcana support is a documented game in
`AGENTS.md` § 1; deleting the API forecloses the trivial path to a future
Lorcana admin UI. Rejected as marginal; queue `delete-dead-lorcana-import`
as a follow-up convoy if Lorcana never gets wired into the admin UI.
**Operator action**: ratify Option A or counter-propose. Architect proceeds
to brief generation assuming Option A; if operator counters before
implementer dispatch, the brief is revised in place.
### Decision 2 — Named-limiter shape in `lib/rate-limit.js` (architect-self-ratifiable)
**Ratified: Option (c) hybrid — preserve `checkAuthRateLimit(req)`, add four named functions.**
The verbatim new module shape is specified in
`.convoys/add-rate-limiting/brief-1-extend-rate-limit-and-wire-routes.md`
§ `lib/rate-limit.js` (modified). Key design choices:
- `Map<className, Ratelimit>` instance cache (one Redis client, five
Ratelimit instances, distinct prefix per class). Each `Ratelimit`
construction is a cheap object wrap around the shared Redis client; no
separate REST connection per class.
- Five exported functions: `checkAuthRateLimit(req)`,
`checkSearchRateLimit(req)`, `checkUploadRateLimit(req, userId)`,
`checkGenerateRateLimit(req, userId)`, `checkImportRateLimit(req, userId)`.
Internal `check(className, identifier)` shared helper.
- `extractIpIdentifier(req)` is module-private (renamed from the
pre-refactor `extractIdentifier` — that legacy name is gone but the
behavior is identical).
- `LIMITER_CONFIG` is a top-level const map of `{ limit, window, prefix }`
per class; adding a sixth class is a one-line addition + one new
exported function (no init() restructuring needed).
- Brief 4's `checkAuthRateLimit(req)` return shape is **byte-identical**
post-refactor (`{ allowed, remaining, reset }`) — Brief 4 contract
preserved.
### Decision 3 — Per-class limit values (architect-self-ratifiable)
**Ratified with tuning evidence:**
| Class | Limit | Window | Key | Rationale |
| --- | --- | --- | --- | --- |
| `auth` | 5 | 15 min | IP | Brief 4 unchanged. |
| `search` | **60** | 1 min | IP | **Raised from parent's 30.** `components/ShareModal.js::handleSearch` (lines 56-77) fires on every keystroke with no debounce; typing a 17-char email = 16 requests in <5s, which would 429 at 30/1min on a single legitimate user entry. 60/1min covers a realistic burst and still stops a scraper. |
| `upload` | 10 | 1 hour | user | Avatar uploads are rare; 10/hour catches accidental loops. User-keyed because a household IP shouldn't punish other users. |
| `generate` | **5** | 1 hour | user | **Raised from parent's 3.** `pages/api/user/avatar/generate.js` calls DiceBear (free public API), not OpenAI/Replicate; cost is Vercel blob + DiceBear-side throttling, not per-call $. 5/hour still catches loops without blocking a user trying 3-4 seeds. |
| `import` | 5 | 1 hour | user | Admin-only via Decision 1; hits Scryfall/Pokémon-TCG/Lorcana APIs with their own rate limits. 5/hour is plenty for the actual import workflow (one set per click; admin won't import 5 sets per hour in normal operation). |
If the operator wants different numbers, the change is a single-line edit
to `LIMITER_CONFIG` in `lib/rate-limit.js` call out in PR review.
### Decision 4 — Identifier extraction shape (architect-self-ratifiable)
**Ratified: Option (a) — two extractors, `extractUserIdentifier` THROWS on missing userId.**
Verbatim shape in the brief. Key safety property:
`extractUserIdentifier(userId)` THROWS the message
`'[rate-limit] extractUserIdentifier called without an authenticated userId. Place the rate-limit gate AFTER the auth check, never before.'`
when `userId` is `null`, `undefined`, `''`, or `NaN`. This surfaces
gate-ordering bugs **at dev time** rather than silently falling back to
IP and converting a per-user limit into a per-IP limit which would
lock other household members out for one user's behavior, the exact
regression flagged in `.convoys/add-rate-limiting.md` § Known constraints.
Numeric `0` is intentionally NOT in the throw conditional if a future
schema introduces user ID 0 the limiter still keys correctly as
`'user:0'`. There is no current user with ID 0 in the `users` table; the
check is defensively forward-compatible.
### Decision 5 — 429 response shape (architect-self-ratifiable)
**Ratified: uniform message across all five classes** `'Too many attempts. Try again later.'`.
Matches `login.js` + `register.js` verbatim. Per-class variation (e.g.,
`'Too many search requests'` vs `'Too many upload requests'`) was
considered and rejected: per-class messages would fingerprint to an
attacker which routes have which limits + windows, making it easier to
craft a request pattern that avoids 429 on the more-permissive routes
while still abusing the less-permissive ones. Single uniform message =
attacker doesn't know what they hit.
`Retry-After` calculation is also identical:
`Math.ceil((reset - Date.now()) / 1000)`. Status code is `429`.
### Decision 6 — Test coverage in this convoy (architect-self-ratifiable)
**Ratified: no new vitest or playwright tests.**
Same reasoning as `cors-tighten` Decision D4. Per-route handler tests
are deferred to the queued `fill-vitest-handler-coverage` convoy. The
fail-loud-in-prod predicate in `lib/rate-limit.js::init()` is the
integration test if `KV_REST_API_*` is unset on a deployed env, every
gated route fails closed on first call.
**Correction to the convoy file's "Known constraints" claim:** the file
states *"Brief 4's `lib/rate-limit.js` is unit-tested transitively via
the existing vitest suite (none of the 21 tests specifically test
rate-limit, but the auth-utils tests run through the same module)."*
This is stale. Architect ran `rg 'rate-limit|@upstash' test/` zero
matches. `test/api/auth-utils.test.js` only imports
`pages/api/auth-utils.js` + `lib/auth-secret.js` (with a mock for
`lib/database.js`); it does NOT load the auth handlers or
`lib/rate-limit.js`. The lib refactor is therefore strictly safer than
the convoy file implies there is no transitive test path to break.
Decision 6 still holds.
## Architecture
### File plan
| File | Action | Purpose |
| --- | --- | --- |
| `lib/rate-limit.js` | modified | Refactor from single auth-only limiter into Map-of-named-limiters with five classes. Preserves Brief 4's `checkAuthRateLimit(req)` contract; adds four new named exports. |
| `pages/api/users/search.js` | modified | Add `checkSearchRateLimit(req)` gate after the inline JWT verify, before query-length validation. IP-keyed. |
| `pages/api/cards/search.js` | modified | Add `checkSearchRateLimit(req)` gate at top of handler, after method check, before the 240-line SQL god-function (which stays byte-identical). IP-keyed, anonymous-by-design. |
| `pages/api/user/avatar.js` | modified | Add `checkUploadRateLimit(req, user.userId)` after the existing `getUserFromRequest` check, before the POST/DELETE branching. User-keyed; gate fires before `parseMultipartFormData` body parse. |
| `pages/api/user/avatar/generate.js` | modified | Add `checkGenerateRateLimit(req, user.userId)` after the existing `getUserFromRequest` check, before the user-data SQL query. User-keyed; calls DiceBear (free), not a paid AI service. |
| `pages/api/cards/import-mtg.js` | modified | Add `getUserFromRequest` + `if (user.role !== 'admin') return 403` + `checkImportRateLimit(req, user.userId)` before the existing `try` block. Closes the publicly-callable anonymous-abuse vector. |
| `pages/api/cards/import-pokemon.js` | modified | Same shape as `import-mtg.js`. Live admin tooling per architect investigation. |
| `pages/api/cards/import-lorcana.js` | modified | Same shape as `import-mtg.js`. Gated defensively despite zero current frontend callers future Lorcana admin UI inherits protection. |
| `pages/admin/card-import.js` | modified | **Scope expansion for D1.** Add `'Authorization': \`Bearer ${localStorage.getItem('auth_token')}\`` to the import fetch's headers (line 45-47). One-line addition; 309-line god-component otherwise byte-identical. |
| `.cursor/rules/api-routes.mdc` | modified | Replace § Rate limiting (lines 104-135) with the new per-class table + verbatim call shape + gate-ordering rules + identifier-extraction documentation. Every other section byte-identical. |
### API surface
No new routes. No request/response shape changes on the 7 gated routes
(429 is added to the possible status codes for each). The only public
surface change is:
- `/api/cards/import-mtg` POST: now requires `Authorization: Bearer <admin-token>`. Returns 401 (no token), 403 (non-admin), 429 (rate-limited), or the existing 200/400/500 contract.
- `/api/cards/import-pokemon` POST: same.
- `/api/cards/import-lorcana` POST: same.
- `/api/users/search` GET: adds 429 to the possible status codes (existing 401/400/200/500 unchanged).
- `/api/cards/search` GET: adds 429 to the possible status codes (existing 200/500 unchanged; no auth either pre- or post-edit).
- `/api/user/avatar` POST/DELETE: adds 429 (existing 401/400/200/500 unchanged).
- `/api/user/avatar/generate` POST: adds 429 (existing 401/404/200/500 unchanged).
All 429 responses set `Retry-After: <seconds>` and return
`{ "error": "Too many attempts. Try again later." }`.
### Schema diff
**No schema changes.** No new Postgres tables, columns, or indexes. The
limiter state lives in Upstash Redis (managed); the existing
`KV_REST_API_URL` / `KV_REST_API_TOKEN` env vars (auto-provisioned by
Vercel's Upstash Marketplace integration) cover all five classes.
### Test plan
Per Decision 6, **no new vitest or playwright specs are added in this convoy**.
Existing test coverage that must still pass post-refactor:
- `test/lib/auth-secret.test.js` (3 tests) unrelated to rate-limit.
- `test/lib/permission-middleware.test.js` (8 tests) unrelated.
- `test/api/auth-utils.test.js` (5 tests) unrelated (does NOT import the auth handlers, verified at architect time).
- `test/components/Layout.test.js` (5 tests) unrelated.
Total: 21/21 must still pass. `npm run test:run` is BLOCKING in CI per
the `test:` job in `.github/workflows/ci.yml`.
Playwright smoke (3/3, runs against Vercel preview):
- `home redirects or renders without 5xx` anonymous GET on `/`; not in scope.
- `sign-in page renders` anonymous GET on `/login`; not in scope.
- `public health endpoint responds` anonymous GET on `/api/health`; not in scope.
None of the smoke tests exercise any of the 7 gated endpoints, so no
smoke regression risk. Smoke run must stay 3/3 green.
Manual verification (run locally pre-PR, paste output in PR description):
documented in the brief's § Manual verification.
### Risk list
1. **Brief 4 contract regression on `checkAuthRateLimit(req)`.** If the
refactor changes the return shape from `{ allowed, remaining, reset }`,
`pages/api/auth/login.js` + `pages/api/auth/register.js` break silently
(destructuring undefined). Mitigation: the brief locks the return shape
as byte-identical; manual verification exercises the 6th-attempt 429
path on `/api/auth/login`.
2. **Per-class Redis prefix collision.** If two `LIMITER_CONFIG` entries
accidentally share a prefix (typo, copy-paste), a search hit eats from
the auth budget for the same IP. Mitigation: brief acceptance criterion
explicitly checks `rg "tcgvault:" lib/rate-limit.js | sort -u` returns
five distinct lines.
3. **Gate-ordering reversal for user-keyed limiters.** If an implementer
places `checkUploadRateLimit(req, user.userId)` BEFORE the
`getUserFromRequest` check, every anonymous request throws (via
`extractUserIdentifier`'s null guard). Mitigation: the throw IS the
defensive signal it surfaces as a dev-time 500 immediately rather than
a silent security regression. The brief's `## Cross-file checks` calls
out the order-check explicitly.
4. **Admin UI broken on first run if `pages/admin/card-import.js` Bearer
fix doesn't land.** Adding `getUserFromRequest` to the import APIs
without the matching admin-UI fix produces an immediate 401 on the next
"Import Cards" click. Mitigation: brief includes both edits as a single
atomic change; PR review enforces atomicity (acceptance criterion calls
the dependency out explicitly).
5. **Search 60/1min still too low for rapid typists.** If 60/min ends up
too restrictive in production, surface as a `tune-search-rate-limit`
follow-up convoy; the fix is a single LIMITER_CONFIG edit (60 90 or
120). Not a release-blocker the failure mode is a 429 with a
`Retry-After: 60` header, which the front-end can display as
"searching too fast, try again in a moment".
6. **`extractUserIdentifier` throws on edge-case userIds.** Numeric `0` is
intentionally accepted (returns `'user:0'`). Empty-string userId throws.
If a future auth refactor changes the `userId` type to a UUID string,
the empty-string guard still works. If it changes to an opaque `object`,
the throw fires (correct we don't want to key off an object).
7. **Upstash outage fails-open.** Brief 4's design choice carries through:
on `ratelimit.limit(...)` failure, return `{ allowed: true, ... }`. A
hard Upstash outage during an attack would defeat the limiter for the
duration of the outage. Mitigation: defense-in-depth (Vercel firewall,
future fail2ban-style lockout) and Upstash's published SLA. Not
addressable inside this convoy.
8. **Body-stream bypass on avatar.js.** `parseMultipartFormData` consumes
the 5MB body via `req.on('data')` before the response is sent, so an
attacker can still exhaust the 5MB ceiling per 429. This is explicitly
out-of-scope (`harden-multipart-parser` convoy). The brief's gate
ordering places the limiter BEFORE the method branches (which call
`parseMultipartFormData`), so when that future hardening lands, the
gate ordering is already correct.
### Verbatim new `lib/rate-limit.js` shape
Specified in full in
`.convoys/add-rate-limiting/brief-1-extend-rate-limit-and-wire-routes.md`
§ `lib/rate-limit.js` (modified). Implementer has zero design discretion.
## Decomposition
| Brief # | Title | Files | Depends on | Estimated PR size |
| --- | --- | --- | --- | --- |
| 1 | Extend `lib/rate-limit.js` to named per-class limiters + wire into the remaining abusable endpoints + gate the import routes | `lib/rate-limit.js`, `pages/api/users/search.js`, `pages/api/cards/search.js`, `pages/api/user/avatar.js`, `pages/api/user/avatar/generate.js`, `pages/api/cards/import-mtg.js`, `pages/api/cards/import-pokemon.js`, `pages/api/cards/import-lorcana.js`, `pages/admin/card-import.js`, `.cursor/rules/api-routes.mdc` | | ~180 LOC across 10 files (lib refactor ~65 lines net add; 7 route edits ~6-16 lines each; admin UI +1 line; rules doc ~70 lines + / ~32 lines -) |
**Brief count: 1.** Splitting into two briefs (one for lib refactor, one
for per-route wiring) was considered and rejected:
- Per-route wiring **depends on** the lib refactor (the new exports
don't exist until Brief 1 lands), so parallelism gain via
`/multitask` is zero Brief 2 would have `depends_on: [1]` and run
serially anyway.
- Single brief presents the reviewer with one coherent change instead
of two related-but-fragmented PRs.
- Cross-brief commitment overhead (Brief 1 ships stub limiters, Brief 2
resolves the wiring) creates exactly the kind of forward-declaration
scaffolding the `role-architect` Boot-the-brief check exists to
prevent.
The single brief is well-bounded at ~180 LOC across 10 files (mostly
small additions). No single file gets more than ~70 lines of edit; the
average file edit is ~12 lines.
### Slice dependencies (multitask-ready)
```yaml
slice_dependencies:
- brief: 1
depends_on: []
files:
- lib/rate-limit.js
- pages/api/users/search.js
- pages/api/cards/search.js
- pages/api/user/avatar.js
- pages/api/user/avatar/generate.js
- pages/api/cards/import-mtg.js
- pages/api/cards/import-pokemon.js
- pages/api/cards/import-lorcana.js
- pages/admin/card-import.js
- .cursor/rules/api-routes.mdc
```
One brief, no `/multitask` fan-out the conductor dispatches a single
implementer.