deckhearth/.convoys/add-rate-limiting.md
Randall Stillwell 7832e03dec docs: post-convoy cleanup for add-rate-limiting — MILESTONE, last P0 closed
Reflects the merged add-rate-limiting convoy (PR #20, squash commit
708ef45) in repo documentation. **This is the milestone cleanup** —
add-rate-limiting closed P0 #6 (No rate limiting anywhere), the LAST
open P0 ship-blocker. `.convoys/ship-readiness.md`'s § Status summary
flips from "7 of 8 RESOLVED; 1 remains" to **"8 of 8 RESOLVED.
Launch-readiness P0 checklist is empty."** One brief in the convoy:
Brief 1 shipped as planned with no scope expansions and no implementer
deviations from the verbatim spec; all six architect decisions ratified
verbatim at gate 1 (D1 operator-ratified Option A; D2-D6
architect-self-ratified).

.convoys/add-rate-limiting.md:
  - frontmatter status: in-progress -> shipped (added shipped: 2026-05-24)
  - new ## As-shipped section. Opens with the milestone language
    pointing back at ship-readiness.md's flipped § Status summary.
    Decisions section captures all 6 ratifications (D1 operator-
    ratified Option A — Critical: WHY the atomic admin UI fix in
    pages/admin/card-import.js was Decision 1's hidden coupling
    requirement, since API gating alone would have broken every
    "Import Cards" click; D2 hybrid named-limiter shape with
    Map<className, Ratelimit> cache; D3 per-class table including
    the two D3 tuning-evidence raises — search 30 -> 60/min because
    ShareModal.handleSearch has no debounce so a 17-char email = 16
    requests in <5s, and generate kept at 5/hour because DiceBear is
    free not paid AI; D4 two-extractor shape with defensive THROW
    on null/empty userId; D5 uniform 429 message; D6 no new vitest
    or playwright specs deferred to fill-vitest-handler-coverage).
    As-shipped surface broken into 4 layers (1 lib refactor + 6 route
    gates + 1 atomic admin UI fix + 1 rule extension) mirroring the
    cors-tighten cleanup's pattern-split shape. Empirical CI metrics
    from post-merge run 26382185019 (Playwright smoke 59s 3/3 in
    3.8s, forbidden-cors-headers pass, vitest 21/21, lint 128 baseline,
    Screenshot diff continue-on-error swallow per Decision 4).
    Cross-validation finding: smoke test 2 still passes against the
    post-rate-limit preview — that's three convoys in a row (PR #15
    Layout default-user, PR #19 CORS-tighten, PR #20 rate-limiting)
    where the same 3-test smoke spec defended the auth surface
    through sweeping changes. Operator-action-required: none. What
    did NOT change audit trail.

.convoys/ship-readiness.md:
  - § Status summary at the top flipped from 7/8 to 8/8 RESOLVED.
    Header text updated to "Launch-readiness P0 checklist is empty."
    P0 #6 row in the table flips from PARTIAL to RESOLVED with the
    two-convoy lineage (fix-auth-bypass Brief 4 + add-rate-limiting).
    Trailing paragraph rewritten as a milestone note: security gate
    closed; remaining launch work is P1 quality bar + P2/P3 polish.
  - P0 #6 entry flipped from PARTIAL to RESOLVED with the full
    add-rate-limiting as-shipped block (8 sub-bullets covering the
    lib refactor shape, the per-class table, the defensive THROW,
    the three import routes' auth-gating, the atomic admin UI fix
    and WHY, the rule extension, the 6 decisions, and the diff
    breakdown). Brief 4's 2026-05-23 partial is preserved as the
    prior as-shipped layer to maintain the audit trail.
  - Launch sequence step 4 marked RESOLVED 2026-05-24 with the
    squash commit + smoke metrics inline.
  - Queued convoys: removed the add-rate-limiting entry (it shipped).
    Added a new delete-dead-lorcana-import entry (P3 polish; the
    Lorcana import route was gated defensively in PR #20 despite
    zero current frontend callers — pages/admin/card-import.js's
    <select> only offers mtg + pokemon — so if Lorcana stays
    permanently out of the admin UI, this is the cleanup PR).
    Added three "flagged but kept out of scope" follow-ups per the
    convoy's § What did NOT change: harden-multipart-parser (P2;
    5MB body still consumed before the 429 path on avatar.js),
    god-function-split / refactor-cards-search-sql (P2; 240-line
    7-branch SQL in cards/search.js), and withAdmin(handler) wrapper
    extraction (P3 DX; the three import routes are call sites #3-5
    in the codebase but uniform inline shape was preserved for
    convoy atomicity). Updated tighten-visual-diff-path-filter to
    note PR #20 also tripped the same false-positive.

AGENTS.md:
  - Gotcha #12 extended end-to-end. Was the single-class auth-only
    lib + the env-var contract; is now the 5-class reality with a
    full per-class table (helper / limit-window / key / routes),
    the defensive THROW pattern in extractUserIdentifier, the
    gate-ordering rule for per-user limiters, and the
    auth → admin-role → rate-limit ordering for the three import
    routes. Prominent milestone line opens the new content:
    "add-rate-limiting convoy (squash 708ef45, PR #20, 2026-05-24)
    closed P0 #6 — all 8 P0s now RESOLVED." Original env-var
    contract paragraph (KV_REST_API_URL / KV_REST_API_TOKEN,
    fail-closed-in-prod / warn-and-noop-in-dev) is preserved
    verbatim above the new content.
  - § 6 Testing: intentionally untouched (no test surface changed;
    vitest 21/21 and smoke 3/3 still apply).
  - § 7 Deployment: intentionally untouched (no deployment-shape
    changed; same KV_REST_API_* env vars from Brief 4).

.cursor/rules/api-routes.mdc:
  - The implementer extended § Rate limiting in PR #20 with the
    per-class table + verbatim call shape + gate-ordering rules +
    identifier-extraction + uniform 429 + fail-closed env-var
    contract + fail-open Upstash-outage behavior. Doc-writer pass
    verified completeness; added a one-sentence convoy-attribution
    line at the top of § Rate limiting citing the two-convoy
    lineage (fix-auth-bypass Brief 4 for the auth class +
    add-rate-limiting for the other four classes and 7 newly-gated
    routes), mirroring the post-cors-tighten § CORS attribution
    shape. No other touch-ups needed.

No changes to: package.json, package-lock.json, lib/rate-limit.js,
pages/**, components/**, scripts/**, test/**, tests/**,
.github/workflows/**, README.md, TESTING_GUIDE.md, playwright.config.js,
eslint.config.mjs.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-24 23:09:07 -05:00

1035 lines
55 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
name: add-rate-limiting
classification: convoy
success_metric: |
All endpoints listed in P0 #6 of `.convoys/ship-readiness.md` carry
a rate-limit gate appropriate to their abuse class (search /
upload / import / admin). The card-import endpoints are no longer
anonymous-callable. `lib/rate-limit.js` exposes named limiters
per route class with documented limits + windows + key-extraction
strategies. P0 #6 flips PARTIAL → RESOLVED, closing the last
open P0 ship-blocker.
skip:
- role-design-system-auditor
- role-a11y-auditor
- role-ux-reviewer
- role-ia-architect
status: shipped
created: 2026-05-24
shipped: 2026-05-24
parent: ship-readiness
addresses: P0 #6 (PARTIAL → RESOLVED) — the LAST open P0
depends_on:
- fix-auth-bypass (Brief 4 shipped lib/rate-limit.js + auth-only limiter)
- cors-tighten (just closed P0 #5; not a hard dep but the API surface is now CORS-clean)
---
# Convoy: add-rate-limiting
Extend `lib/rate-limit.js` and wire the result into the remaining
abusable endpoints. This is the final P0 — when it lands, the
launch-readiness ship-blocker list is empty (all 8 of 8 RESOLVED).
## Why now
`fix-auth-bypass` Brief 4 shipped the rate-limit infrastructure
(`lib/rate-limit.js`, `@upstash/ratelimit@^2.0.8`,
`@upstash/redis@^1.38.0`, Vercel-Upstash Marketplace env vars
`KV_REST_API_URL` / `KV_REST_API_TOKEN`) and wired it into the
two auth endpoints (`login`, `register`) with a 5-attempts /
15-minute sliding window. The remaining abusable surface was
intentionally deferred to this convoy.
The "Queued convoys" entry in `.convoys/ship-readiness.md` lists
the surface as:
- `/api/users/search`
- `/api/cards/search`
- All `/api/cards/import-*`
- `/api/user/avatar*` (upload)
Parent's pre-architect audit at convoy creation surfaced a
**critical secondary finding** beyond "missing rate limit":
**`pages/api/cards/import-mtg.js`, `import-pokemon.js`,
`import-lorcana.js` have ZERO authentication checks.** They are
publicly callable, they hit Scryfall / Pokémon-TCG / Lorcana APIs
with no caller throttling AND no auth, and they perform UPSERTs
into the `cards` table. An attacker can:
1. Trigger expensive external API calls from your IP (Scryfall +
Pokémon-TCG have published rate limits; you'd hit them with no
recourse, getting your IP throttled at the source).
2. Cause unbounded DB writes (each import inserts hundreds-to-thousands
of rows; `card.id` lookups deduplicate but the INSERT path runs
in a tight per-card loop).
3. Even with a generous per-IP rate limit (5 req / hour), with
1000 IPs the math is 120,000 imports / day — enough to exhaust
Vercel function quotas and Neon row budgets.
The rate-limit alone is insufficient. **Auth-gate-then-rate-limit
is the correct shape.** Whether to add auth gates to the import
routes in this convoy (vs. spinning out a separate
`gate-import-routes` convoy) is **architect Decision 1** below.
Parent recommends in-scope; expansion is mid-convoy precedent
established by `drop-public-setup` (Brief 2's CJS/ESM expansion)
and other prior convoys.
## Scope
### Confirmed in-scope (regardless of architect Decision 1)
- **`lib/rate-limit.js`** — refactor to expose multiple named
limiters (one per route class), preserving the existing
`checkAuthRateLimit` export for `login` / `register` backwards
compatibility. New named limiters per Decision 2 below.
- **`pages/api/users/search.js`** — add rate-limit gate after the
existing JWT verification. Limit per Decision 2.
- **`pages/api/cards/search.js`** — add rate-limit gate at top of
handler (this endpoint is anonymous-by-design; rate-limit keyed
by IP).
- **`pages/api/user/avatar.js`** — add rate-limit gate after the
existing `getUserFromRequest` check. Limit keyed by `user.userId`
(authenticated; per-user makes more sense than per-IP for an
upload endpoint where a household might share an IP).
- **`pages/api/user/avatar/generate.js`** — same shape as
`avatar.js`. Per-user, stricter than avatar.js (AI generation
is more expensive than blob upload).
- **`.cursor/rules/api-routes.mdc`** § Rate limiting — update the
existing § with the new per-class pattern + each named limiter's
use case. The current § shows only the auth pattern.
### Scope-expansion-candidate (architect Decision 1)
- **`pages/api/cards/import-mtg.js`** — add `getUserFromRequest`
check at top of handler; return 403 if not admin (`user.role !==
'admin'`); then rate-limit gate (per-user, very strict — these
hit external APIs).
- **`pages/api/cards/import-pokemon.js`** — same shape.
- **`pages/api/cards/import-lorcana.js`** — same shape.
- (If Decision 1 = Option A "in scope": ship in this PR. If Option
B "spin out": queue `gate-import-routes` follow-up convoy and
mark P0 #6 RESOLVED-with-caveat at convoy close.)
### Out of scope
- **Global IP-based backstop limiter** (e.g., 1000 req / min per
IP across all routes via Next.js middleware). Useful but
separate scope; `add-global-rate-limit-middleware` would be its
own convoy.
- **Rate-limit headers on success responses**
(`X-RateLimit-Remaining`, `X-RateLimit-Reset`). The existing
`checkAuthRateLimit` returns `remaining` + `reset` in its
result object but `login.js` doesn't propagate them on
success; just on 429. Honoring the existing convention.
- **`@upstash/ratelimit` version bump** — pin stays at `^2.0.8`
from Brief 4. Bumping is its own convoy.
- **Per-route handler unit tests** (still deferred to the
queued `fill-vitest-handler-coverage` convoy — same reasoning
as `cors-tighten`).
- **Refactoring the existing `cards/search.js` SQL** (the file
has a known god-function shape with 7+ conditional SQL
branches; that's `god-function-split` scope, not here).
- **Verifying the avatar `parseMultipartFormData` body parser**
is rate-limit-safe (the body is read in `req.on('data')` before
any rate-limit gate could short-circuit; an attacker can still
exhaust the 5MB body even on a 429 path). That's a separate
body-streaming-defense concern (`harden-multipart-parser`) and
not in scope here.
## Operator action required
**None.** All Upstash env vars (`KV_REST_API_URL` /
`KV_REST_API_TOKEN`) are already auto-provisioned via the Vercel
Marketplace integration (seeded for `fix-auth-bypass` Brief 4).
No new secrets, no new dependencies (the `@upstash/ratelimit` +
`@upstash/redis` packages are already installed).
## Decisions to ratify with operator (architect routes)
1. **Scope expansion — auth gates on `pages/api/cards/import-*.js`
in this convoy?**
- **Option A — In scope (parent recommends).** Add
`getUserFromRequest` + admin-role check + rate-limit gate to
all 3 import routes in this PR. ~30 LOC across 3 files.
Closes the full P0 #6 attack surface in one shot. Precedent:
`drop-public-setup` Brief 2 (CJS/ESM expansion).
- **Option B — Spin out.** Stay narrow on the 4 originally-listed
routes; queue `gate-import-routes` as a separate convoy. Mark
P0 #6 RESOLVED-with-caveat noting the import-route auth gap.
- **Option C — Hybrid.** Add the rate-limit gate to import
routes now (with a clear `TODO: requires-auth` comment),
gate auth later. Worst of both worlds — leaves an
anonymous-callable abusive endpoint live with only IP-based
throttling.
- Architect investigates: confirm the import routes are
intended to be admin-only (per AGENTS.md / scripts/ folder
conventions), confirm Decision-A scope size is bounded, route
back recommendation.
2. **Named-limiter shape in `lib/rate-limit.js`.** The existing
`checkAuthRateLimit(req)` function has the limit + window
hardcoded. To support per-class limits cleanly, two main
patterns:
- **(a) Named-export functions.** Each route class gets its
own exported function: `checkSearchRateLimit`,
`checkUploadRateLimit`, `checkImportRateLimit`. Internal
`init()` builds one `Ratelimit` instance per class, cached
under different Redis prefixes. Verbose but very explicit at
each call site.
- **(b) Generic `checkRateLimit(req, options)`.** Single
exported function takes a config object. Calling code
becomes `await checkRateLimit(req, { class: 'search' })` or
similar. Less verbose at the lib but more at each call site,
and the per-route limits are documented in the lib module
rather than implicit in the function name.
- **(c) Hybrid.** Keep `checkAuthRateLimit` (used by login +
register; high-stakes contract; don't break). Add named
functions for the other 3 classes (search, upload, import).
Best of both — preserves Brief 4's contract, names new
classes explicitly.
- Parent recommends (c). Architect ratifies after reading
`lib/rate-limit.js` shape.
3. **Per-class limit values.** Reference table (architect's
to tune):
- **`auth`** (existing) — 5 / 15min, IP-keyed. (Don't change.)
- **`search`** (new) — recommend 30 / 1min, IP-keyed.
Legitimate users type-ahead-search; 30/min is generous but
stops a scraper in its tracks within 2-3s. Same IP-keying
as auth.
- **`upload`** (new — avatar.js, avatar/generate.js) —
recommend 10 / 1hour, user-keyed. Avatar uploads are rare;
10/hour catches accidental loops without blocking a user
who hits "save" three times by accident. User-keyed because
the user is authenticated and a household IP shouldn't
punish other users.
- **`generate`** (new — avatar/generate.js specifically) —
recommend 3 / 1hour, user-keyed. AI generation costs money;
stricter than plain upload.
- **`import`** (new — cards/import-*.js) — recommend 5 / 1hour,
user-keyed (if Decision 1 = Option A). Admin-triggered, hits
external APIs with their own rate limits; 5/hour is plenty.
4. **Identifier extraction.** `lib/rate-limit.js`'s
`extractIdentifier()` currently always extracts the first
`x-forwarded-for` IP. For per-user limiters (upload, generate,
import), we need a `user_id`-based extractor. Three options:
- **(a) Two extractors.** `extractIpIdentifier` (existing) +
`extractUserIdentifier(req, userId)` (new). Each limiter
calls the right one.
- **(b) Single extractor takes an `opts.user` flag.** Less
code but harder to read.
- **(c) Caller passes `key` directly.** Each handler calls
`checkUploadRateLimit(req, { key: user.userId })` and the lib
just trusts it.
- Parent recommends (a) — explicit, two small functions,
hard to misuse.
5. **429 response shape.** Match the existing
`login.js`/`register.js` pattern verbatim:
```js
res.setHeader('Retry-After', Math.ceil((reset - Date.now()) / 1000));
return res.status(429).json({ error: 'Too many attempts. Try again later.' });
```
Architect: should the error message vary per class (e.g., "Too
many search requests" vs "Too many uploads") or stay uniform?
Recommend uniform — minimizes attacker-fingerprinting of which
routes have which limits. Architect ratifies.
6. **Test coverage in this convoy.** No new per-route handler
tests (per Decision-4-equivalent from `cors-tighten`; same
deferred-to-`fill-vitest-handler-coverage` reasoning). The
Playwright smoke spec doesn't exercise any of these routes,
so no smoke coverage either. **The fail-loud-in-prod
semantics in `lib/rate-limit.js`'s `init()` function are the
integration test** — if the env vars are unset on a deployed
env, every limit-gated route fails closed on first call.
## Known constraints
- **`lib/rate-limit.js` already has the right architecture** for
multiple limiters (lazy `init()` returns a `cached` object;
refactor pattern is to make `cached` a Map<className,
Ratelimit> instead of a single instance). Don't rewrite from
scratch; extend.
- **Fail-closed semantics in prod must be preserved.** Brief 4's
`init()` throws when env vars are unset and `NODE_ENV ===
'production'`. New limiters must inherit this — a misconfigured
prod env should fail loud on every call, not silently disable
all rate limiting.
- **Card-search is anonymous-by-design** — do NOT add a
`getUserFromRequest` check. The IP-keyed limit is the correct
defense (search is a public catalogue feature).
- **User-keyed limits require an authenticated request.** The
rate-limit gate MUST sit AFTER the auth check in the handler
body. Wrong order = security regression (anonymous user bypasses
the gate because there's no `userId` to key on, the extractor
falls back to IP, and the per-user limit becomes per-IP — same
household IP could be locked out for a different user's
behavior).
- **Brief 4's `lib/rate-limit.js` is unit-tested transitively**
via the existing vitest suite (none of the 21 tests
specifically test rate-limit, but the auth-utils tests run
through the same module). Don't break those.
- **`@upstash/ratelimit` supports multiple shapes**
(`slidingWindow`, `fixedWindow`, `tokenBucket`); Brief 4 used
`slidingWindow`. New limiters should follow the same shape
unless the architect has a specific reason for a different
algorithm per class.
- **Per-route Pattern checks** — the 6 (or 9 if Decision 1 =
Option A) target files have different existing structures:
- `users/search.js`: auth check inside try/catch on JWT;
rate-limit goes after JWT verify, inside the existing try
- `cards/search.js`: no auth; rate-limit goes at top of
handler after the method check
- `user/avatar.js`: auth check inside the outer try (line
16-19); rate-limit goes after auth, before the method
branching at line 21
- `user/avatar/generate.js`: similar shape to avatar.js
- `cards/import-*.js`: NO auth currently. Decision 1 dictates
the shape — either add auth-then-rate-limit, or rate-limit-only
(Option C, not recommended)
## Acceptance criteria
The convoy is shippable when ALL of the following hold:
1. `lib/rate-limit.js` exposes the named limiters per Decision 2.
Backwards-compat: `checkAuthRateLimit` still works for
`login.js` / `register.js` (do NOT break Brief 4's contract).
2. All 4 originally-listed endpoints (`users/search`,
`cards/search`, `user/avatar`, `user/avatar/generate`) carry
the appropriate rate-limit gate at the correct ordering
(after auth if applicable; per Decision 4 keying).
3. If Decision 1 = Option A: all 3 import endpoints carry both
the auth gate (admin-role check) + the import rate-limit gate.
4. `npm run lint` exit code matches baseline (still 128 problems;
do NOT regress).
5. `npm run test:run` (vitest) still passes 21/21 (no regression
on the existing auth-utils tests that transitively load
`lib/rate-limit.js`).
6. `npm run test:smoke` (via CI on the PR) still passes 3/3
against the Vercel preview — proves the live login → verify
→ /api/health flow doesn't accidentally get rate-limited
(the smoke spec hits each endpoint once per run, well below
any limit).
7. `.cursor/rules/api-routes.mdc` § Rate limiting updated to
document each new limiter's use case + the per-class limits
+ the user-keyed vs IP-keyed convention.
8. Bypass-secret-leak check: zero matches of any
`KV_REST_API_*` or `UPSTASH_*` env-var values in any CI log.
(Auto-masked by GitHub Actions; verify post-merge.)
## Anything flagged but not acted on (in advance)
- **Global IP backstop limiter via Next.js middleware** —
out of scope; queue `add-global-rate-limit-middleware` if a
future audit shows non-listed routes being abused.
- **Body-streaming defense for avatar uploads** — the 5MB
multipart body is consumed before any rate-limit gate can
short-circuit. Real defense requires moving the parse into a
separate edge function or using `read-up-to` semantics; queue
`harden-multipart-parser` if it ever surfaces in a real abuse
incident.
- **Per-tier user limits** — premium users might get higher
search/upload limits. Queue `tiered-rate-limits` when a
tiering scheme exists (today there are only `admin` and
default roles).
- **Rate-limit metrics dashboard** — Upstash exposes per-key hit
counts; surfacing them in an admin dashboard would catch
abuse early. Queue `rate-limit-observability` post-launch.
- **Card-search SQL god-function** — known shape issue,
separate convoy (`god-function-split` or
`refactor-cards-search-sql`); do NOT touch in this convoy.
## Decisions (post-architect)
Six decisions ratified by `role-architect` on 2026-05-24. **Only Decision 1
requires operator gate-1 sign-off** (significant scope expansion to admin-only
import-route enforcement + admin UI source touch); the other five are
architect-self-ratifiable per the precedent established by `cors-tighten`
Decision D2-D5 + `fix-vercel-deployment-protection-in-ci` Decisions A/B/D.
### Decision 1 — Scope expansion: gate `pages/api/cards/import-*.js` in this convoy (**OPERATOR-RATIFIABLE**)
**Ratified: Option A — gate all three import routes with auth + admin-role + import rate-limit, AND fix the admin UI to send the Bearer token.**
Architect investigation results:
- **`pages/api/cards/import-mtg.js`** — LIVE admin tooling. Called by
`pages/admin/card-import.js` line 39-49 (the `<select>` defaults to
`'mtg'`). Currently fetched WITHOUT an `Authorization` header.
- **`pages/api/cards/import-pokemon.js`** — LIVE admin tooling. Same
admin UI fetch path; the `<select>` second option is `'pokemon'`.
- **`pages/api/cards/import-lorcana.js`** — DEAD in frontend. Architect
ran `rg 'import-lorcana' pages/ components/` and the file has zero
frontend callers; the admin UI's `<select>` only offers `'mtg'` and
`'pokemon'`. The standalone `scripts/import-lorcana.js` exists as an
independent CLI path. Gating defensively is the chosen path because:
(a) future Lorcana admin UI work inherits protection automatically,
(b) the diff is uniform (3 routes get identical treatment), (c) a
future cleanup convoy can delete the route if it stays unused — and
deletion is strictly easier than gating-then-deleting because Option A
preserves all the import infrastructure.
**Critical scope expansion required by Option A:**
`pages/admin/card-import.js` currently calls the import APIs without an
`Authorization` header (line 43-49). The moment the import APIs gain
`getUserFromRequest`, the admin UI begins returning 401 on every import
click — visible UX regression. The brief therefore includes a one-line
edit to `pages/admin/card-import.js` adding
`'Authorization': \`Bearer ${localStorage.getItem('auth_token')}\``
to the fetch's headers. This is the minimum touch; the 309-line
god-component is otherwise byte-identical.
**Rejected alternatives:**
- **Option B (spin out `gate-import-routes`).** Would leave a publicly-callable
abuse vector live for at least one more convoy cycle; closes P0 #6 only
partially. Rejected.
- **Option C (hybrid — rate-limit-only without auth).** Worst of both worlds:
anonymous abuse + only IP-based throttling. Rejected per the convoy spec.
- **Option D-full (delete all three import routes).** Would break the admin
UI immediately and require either deleting `pages/admin/card-import.js` too
or restructuring it to call something else. Wider scope than gating; defeats
the convoy's "close P0 #6 in one shot" success metric.
- **Option D-partial (delete only `import-lorcana.js`, gate mtg+pokemon).**
Considered. Slightly smaller attack surface, but introduces a non-uniform
pattern (two gated routes + one deleted route) that complicates the
reviewer's mental model. Lorcana support is a documented game in
`AGENTS.md` § 1; deleting the API forecloses the trivial path to a future
Lorcana admin UI. Rejected as marginal; queue `delete-dead-lorcana-import`
as a follow-up convoy if Lorcana never gets wired into the admin UI.
**Operator action**: ratify Option A or counter-propose. Architect proceeds
to brief generation assuming Option A; if operator counters before
implementer dispatch, the brief is revised in place.
### Decision 2 — Named-limiter shape in `lib/rate-limit.js` (architect-self-ratifiable)
**Ratified: Option (c) hybrid — preserve `checkAuthRateLimit(req)`, add four named functions.**
The verbatim new module shape is specified in
`.convoys/add-rate-limiting/brief-1-extend-rate-limit-and-wire-routes.md`
§ `lib/rate-limit.js` (modified). Key design choices:
- `Map<className, Ratelimit>` instance cache (one Redis client, five
Ratelimit instances, distinct prefix per class). Each `Ratelimit`
construction is a cheap object wrap around the shared Redis client; no
separate REST connection per class.
- Five exported functions: `checkAuthRateLimit(req)`,
`checkSearchRateLimit(req)`, `checkUploadRateLimit(req, userId)`,
`checkGenerateRateLimit(req, userId)`, `checkImportRateLimit(req, userId)`.
Internal `check(className, identifier)` shared helper.
- `extractIpIdentifier(req)` is module-private (renamed from the
pre-refactor `extractIdentifier` — that legacy name is gone but the
behavior is identical).
- `LIMITER_CONFIG` is a top-level const map of `{ limit, window, prefix }`
per class; adding a sixth class is a one-line addition + one new
exported function (no init() restructuring needed).
- Brief 4's `checkAuthRateLimit(req)` return shape is **byte-identical**
post-refactor (`{ allowed, remaining, reset }`) — Brief 4 contract
preserved.
### Decision 3 — Per-class limit values (architect-self-ratifiable)
**Ratified with tuning evidence:**
| Class | Limit | Window | Key | Rationale |
| --- | --- | --- | --- | --- |
| `auth` | 5 | 15 min | IP | Brief 4 unchanged. |
| `search` | **60** | 1 min | IP | **Raised from parent's 30.** `components/ShareModal.js::handleSearch` (lines 56-77) fires on every keystroke with no debounce; typing a 17-char email = 16 requests in <5s, which would 429 at 30/1min on a single legitimate user entry. 60/1min covers a realistic burst and still stops a scraper. |
| `upload` | 10 | 1 hour | user | Avatar uploads are rare; 10/hour catches accidental loops. User-keyed because a household IP shouldn't punish other users. |
| `generate` | **5** | 1 hour | user | **Raised from parent's 3.** `pages/api/user/avatar/generate.js` calls DiceBear (free public API), not OpenAI/Replicate; cost is Vercel blob + DiceBear-side throttling, not per-call $. 5/hour still catches loops without blocking a user trying 3-4 seeds. |
| `import` | 5 | 1 hour | user | Admin-only via Decision 1; hits Scryfall/Pokémon-TCG/Lorcana APIs with their own rate limits. 5/hour is plenty for the actual import workflow (one set per click; admin won't import 5 sets per hour in normal operation). |
If the operator wants different numbers, the change is a single-line edit
to `LIMITER_CONFIG` in `lib/rate-limit.js` call out in PR review.
### Decision 4 — Identifier extraction shape (architect-self-ratifiable)
**Ratified: Option (a) — two extractors, `extractUserIdentifier` THROWS on missing userId.**
Verbatim shape in the brief. Key safety property:
`extractUserIdentifier(userId)` THROWS the message
`'[rate-limit] extractUserIdentifier called without an authenticated userId. Place the rate-limit gate AFTER the auth check, never before.'`
when `userId` is `null`, `undefined`, `''`, or `NaN`. This surfaces
gate-ordering bugs **at dev time** rather than silently falling back to
IP and converting a per-user limit into a per-IP limit which would
lock other household members out for one user's behavior, the exact
regression flagged in `.convoys/add-rate-limiting.md` § Known constraints.
Numeric `0` is intentionally NOT in the throw conditional if a future
schema introduces user ID 0 the limiter still keys correctly as
`'user:0'`. There is no current user with ID 0 in the `users` table; the
check is defensively forward-compatible.
### Decision 5 — 429 response shape (architect-self-ratifiable)
**Ratified: uniform message across all five classes** `'Too many attempts. Try again later.'`.
Matches `login.js` + `register.js` verbatim. Per-class variation (e.g.,
`'Too many search requests'` vs `'Too many upload requests'`) was
considered and rejected: per-class messages would fingerprint to an
attacker which routes have which limits + windows, making it easier to
craft a request pattern that avoids 429 on the more-permissive routes
while still abusing the less-permissive ones. Single uniform message =
attacker doesn't know what they hit.
`Retry-After` calculation is also identical:
`Math.ceil((reset - Date.now()) / 1000)`. Status code is `429`.
### Decision 6 — Test coverage in this convoy (architect-self-ratifiable)
**Ratified: no new vitest or playwright tests.**
Same reasoning as `cors-tighten` Decision D4. Per-route handler tests
are deferred to the queued `fill-vitest-handler-coverage` convoy. The
fail-loud-in-prod predicate in `lib/rate-limit.js::init()` is the
integration test if `KV_REST_API_*` is unset on a deployed env, every
gated route fails closed on first call.
**Correction to the convoy file's "Known constraints" claim:** the file
states *"Brief 4's `lib/rate-limit.js` is unit-tested transitively via
the existing vitest suite (none of the 21 tests specifically test
rate-limit, but the auth-utils tests run through the same module)."*
This is stale. Architect ran `rg 'rate-limit|@upstash' test/` zero
matches. `test/api/auth-utils.test.js` only imports
`pages/api/auth-utils.js` + `lib/auth-secret.js` (with a mock for
`lib/database.js`); it does NOT load the auth handlers or
`lib/rate-limit.js`. The lib refactor is therefore strictly safer than
the convoy file implies there is no transitive test path to break.
Decision 6 still holds.
## Architecture
### File plan
| File | Action | Purpose |
| --- | --- | --- |
| `lib/rate-limit.js` | modified | Refactor from single auth-only limiter into Map-of-named-limiters with five classes. Preserves Brief 4's `checkAuthRateLimit(req)` contract; adds four new named exports. |
| `pages/api/users/search.js` | modified | Add `checkSearchRateLimit(req)` gate after the inline JWT verify, before query-length validation. IP-keyed. |
| `pages/api/cards/search.js` | modified | Add `checkSearchRateLimit(req)` gate at top of handler, after method check, before the 240-line SQL god-function (which stays byte-identical). IP-keyed, anonymous-by-design. |
| `pages/api/user/avatar.js` | modified | Add `checkUploadRateLimit(req, user.userId)` after the existing `getUserFromRequest` check, before the POST/DELETE branching. User-keyed; gate fires before `parseMultipartFormData` body parse. |
| `pages/api/user/avatar/generate.js` | modified | Add `checkGenerateRateLimit(req, user.userId)` after the existing `getUserFromRequest` check, before the user-data SQL query. User-keyed; calls DiceBear (free), not a paid AI service. |
| `pages/api/cards/import-mtg.js` | modified | Add `getUserFromRequest` + `if (user.role !== 'admin') return 403` + `checkImportRateLimit(req, user.userId)` before the existing `try` block. Closes the publicly-callable anonymous-abuse vector. |
| `pages/api/cards/import-pokemon.js` | modified | Same shape as `import-mtg.js`. Live admin tooling per architect investigation. |
| `pages/api/cards/import-lorcana.js` | modified | Same shape as `import-mtg.js`. Gated defensively despite zero current frontend callers future Lorcana admin UI inherits protection. |
| `pages/admin/card-import.js` | modified | **Scope expansion for D1.** Add `'Authorization': \`Bearer ${localStorage.getItem('auth_token')}\`` to the import fetch's headers (line 45-47). One-line addition; 309-line god-component otherwise byte-identical. |
| `.cursor/rules/api-routes.mdc` | modified | Replace § Rate limiting (lines 104-135) with the new per-class table + verbatim call shape + gate-ordering rules + identifier-extraction documentation. Every other section byte-identical. |
### API surface
No new routes. No request/response shape changes on the 7 gated routes
(429 is added to the possible status codes for each). The only public
surface change is:
- `/api/cards/import-mtg` POST: now requires `Authorization: Bearer <admin-token>`. Returns 401 (no token), 403 (non-admin), 429 (rate-limited), or the existing 200/400/500 contract.
- `/api/cards/import-pokemon` POST: same.
- `/api/cards/import-lorcana` POST: same.
- `/api/users/search` GET: adds 429 to the possible status codes (existing 401/400/200/500 unchanged).
- `/api/cards/search` GET: adds 429 to the possible status codes (existing 200/500 unchanged; no auth either pre- or post-edit).
- `/api/user/avatar` POST/DELETE: adds 429 (existing 401/400/200/500 unchanged).
- `/api/user/avatar/generate` POST: adds 429 (existing 401/404/200/500 unchanged).
All 429 responses set `Retry-After: <seconds>` and return
`{ "error": "Too many attempts. Try again later." }`.
### Schema diff
**No schema changes.** No new Postgres tables, columns, or indexes. The
limiter state lives in Upstash Redis (managed); the existing
`KV_REST_API_URL` / `KV_REST_API_TOKEN` env vars (auto-provisioned by
Vercel's Upstash Marketplace integration) cover all five classes.
### Test plan
Per Decision 6, **no new vitest or playwright specs are added in this convoy**.
Existing test coverage that must still pass post-refactor:
- `test/lib/auth-secret.test.js` (3 tests) unrelated to rate-limit.
- `test/lib/permission-middleware.test.js` (8 tests) unrelated.
- `test/api/auth-utils.test.js` (5 tests) unrelated (does NOT import the auth handlers, verified at architect time).
- `test/components/Layout.test.js` (5 tests) unrelated.
Total: 21/21 must still pass. `npm run test:run` is BLOCKING in CI per
the `test:` job in `.github/workflows/ci.yml`.
Playwright smoke (3/3, runs against Vercel preview):
- `home redirects or renders without 5xx` anonymous GET on `/`; not in scope.
- `sign-in page renders` anonymous GET on `/login`; not in scope.
- `public health endpoint responds` anonymous GET on `/api/health`; not in scope.
None of the smoke tests exercise any of the 7 gated endpoints, so no
smoke regression risk. Smoke run must stay 3/3 green.
Manual verification (run locally pre-PR, paste output in PR description):
documented in the brief's § Manual verification.
### Risk list
1. **Brief 4 contract regression on `checkAuthRateLimit(req)`.** If the
refactor changes the return shape from `{ allowed, remaining, reset }`,
`pages/api/auth/login.js` + `pages/api/auth/register.js` break silently
(destructuring undefined). Mitigation: the brief locks the return shape
as byte-identical; manual verification exercises the 6th-attempt 429
path on `/api/auth/login`.
2. **Per-class Redis prefix collision.** If two `LIMITER_CONFIG` entries
accidentally share a prefix (typo, copy-paste), a search hit eats from
the auth budget for the same IP. Mitigation: brief acceptance criterion
explicitly checks `rg "tcgvault:" lib/rate-limit.js | sort -u` returns
five distinct lines.
3. **Gate-ordering reversal for user-keyed limiters.** If an implementer
places `checkUploadRateLimit(req, user.userId)` BEFORE the
`getUserFromRequest` check, every anonymous request throws (via
`extractUserIdentifier`'s null guard). Mitigation: the throw IS the
defensive signal it surfaces as a dev-time 500 immediately rather than
a silent security regression. The brief's `## Cross-file checks` calls
out the order-check explicitly.
4. **Admin UI broken on first run if `pages/admin/card-import.js` Bearer
fix doesn't land.** Adding `getUserFromRequest` to the import APIs
without the matching admin-UI fix produces an immediate 401 on the next
"Import Cards" click. Mitigation: brief includes both edits as a single
atomic change; PR review enforces atomicity (acceptance criterion calls
the dependency out explicitly).
5. **Search 60/1min still too low for rapid typists.** If 60/min ends up
too restrictive in production, surface as a `tune-search-rate-limit`
follow-up convoy; the fix is a single LIMITER_CONFIG edit (60 90 or
120). Not a release-blocker the failure mode is a 429 with a
`Retry-After: 60` header, which the front-end can display as
"searching too fast, try again in a moment".
6. **`extractUserIdentifier` throws on edge-case userIds.** Numeric `0` is
intentionally accepted (returns `'user:0'`). Empty-string userId throws.
If a future auth refactor changes the `userId` type to a UUID string,
the empty-string guard still works. If it changes to an opaque `object`,
the throw fires (correct we don't want to key off an object).
7. **Upstash outage fails-open.** Brief 4's design choice carries through:
on `ratelimit.limit(...)` failure, return `{ allowed: true, ... }`. A
hard Upstash outage during an attack would defeat the limiter for the
duration of the outage. Mitigation: defense-in-depth (Vercel firewall,
future fail2ban-style lockout) and Upstash's published SLA. Not
addressable inside this convoy.
8. **Body-stream bypass on avatar.js.** `parseMultipartFormData` consumes
the 5MB body via `req.on('data')` before the response is sent, so an
attacker can still exhaust the 5MB ceiling per 429. This is explicitly
out-of-scope (`harden-multipart-parser` convoy). The brief's gate
ordering places the limiter BEFORE the method branches (which call
`parseMultipartFormData`), so when that future hardening lands, the
gate ordering is already correct.
### Verbatim new `lib/rate-limit.js` shape
Specified in full in
`.convoys/add-rate-limiting/brief-1-extend-rate-limit-and-wire-routes.md`
§ `lib/rate-limit.js` (modified). Implementer has zero design discretion.
## Decomposition
| Brief # | Title | Files | Depends on | Estimated PR size |
| --- | --- | --- | --- | --- |
| 1 | Extend `lib/rate-limit.js` to named per-class limiters + wire into the remaining abusable endpoints + gate the import routes | `lib/rate-limit.js`, `pages/api/users/search.js`, `pages/api/cards/search.js`, `pages/api/user/avatar.js`, `pages/api/user/avatar/generate.js`, `pages/api/cards/import-mtg.js`, `pages/api/cards/import-pokemon.js`, `pages/api/cards/import-lorcana.js`, `pages/admin/card-import.js`, `.cursor/rules/api-routes.mdc` | | ~180 LOC across 10 files (lib refactor ~65 lines net add; 7 route edits ~6-16 lines each; admin UI +1 line; rules doc ~70 lines + / ~32 lines -) |
**Brief count: 1.** Splitting into two briefs (one for lib refactor, one
for per-route wiring) was considered and rejected:
- Per-route wiring **depends on** the lib refactor (the new exports
don't exist until Brief 1 lands), so parallelism gain via
`/multitask` is zero Brief 2 would have `depends_on: [1]` and run
serially anyway.
- Single brief presents the reviewer with one coherent change instead
of two related-but-fragmented PRs.
- Cross-brief commitment overhead (Brief 1 ships stub limiters, Brief 2
resolves the wiring) creates exactly the kind of forward-declaration
scaffolding the `role-architect` Boot-the-brief check exists to
prevent.
The single brief is well-bounded at ~180 LOC across 10 files (mostly
small additions). No single file gets more than ~70 lines of edit; the
average file edit is ~12 lines.
### Slice dependencies (multitask-ready)
```yaml
slice_dependencies:
- brief: 1
depends_on: []
files:
- lib/rate-limit.js
- pages/api/users/search.js
- pages/api/cards/search.js
- pages/api/user/avatar.js
- pages/api/user/avatar/generate.js
- pages/api/cards/import-mtg.js
- pages/api/cards/import-pokemon.js
- pages/api/cards/import-lorcana.js
- pages/admin/card-import.js
- .cursor/rules/api-routes.mdc
```
One brief, no `/multitask` fan-out the conductor dispatches a single
implementer.
## As-shipped
Shipped 2026-05-24 as squash commit `708ef45` (PR #20, architect-commit
`60b842e`, implementer-commit `51a3a97`). **This is the milestone
convoy it closes the LAST open P0 (P0 #6) and flips
`.convoys/ship-readiness.md`'s § Status summary from "7 of 8 RESOLVED;
1 remains" to "8 of 8 RESOLVED. Launch-readiness P0 checklist is
empty."** Brief 1 shipped as planned single lib refactor + six
route-gate additions + one admin UI atomic fix + one rule extension,
no scope expansions beyond the architect-ratified Decision 1, and the
empirical CI metrics + the cross-validation finding + the
"D3 tuning-evidence" rationale captured here so the next architect /
reviewer has the audit trail.
### Decisions ratified at gate 1
All six decisions landed verbatim. Only D1 required operator gate-1
sign-off (significant scope expansion to admin-only enforcement on the
import surface plus an atomic admin UI touch); D2-D6 are
architect-self-ratifiable per the precedent established by
`cors-tighten` Decisions D2-D5 and
`fix-vercel-deployment-protection-in-ci` Decisions A/B/D.
- **Decision 1 Option A (operator-ratified).** Gate all three
`pages/api/cards/import-*.js` routes in this convoy with
`getUserFromRequest` + `if (user.role !== 'admin') return 403` +
`checkImportRateLimit(req, user.userId)`. **And** atomically update
`pages/admin/card-import.js` to send
`'Authorization': \`Bearer ${localStorage.getItem('auth_token')}\``
on its import fetch — without this, the API gating would have
immediately broken every "Import Cards" click in the admin UI. The
architect's pre-brief investigation flagged this as the critical
hidden coupling: gating the import APIs without the matching client
fetch fix would have closed P0 #6 but introduced a visible UX
regression on the only live admin tooling that depends on it. Lorcana
was gated defensively despite having zero current frontend callers
(`pages/admin/card-import.js`'s `<select>` only offers `mtg` /
`pokemon`) uniform protection across the three import shapes is
strictly easier to maintain than a 2-gated-1-deleted asymmetry, and
if Lorcana stays unused, the queued `delete-dead-lorcana-import`
follow-up convoy (see `.convoys/ship-readiness.md` § Queued
convoys) is the cleanup path.
- **Decision 2 Hybrid named-limiter shape (architect-self-ratified).**
`lib/rate-limit.js` refactored from a single auth-only `Ratelimit`
instance into a `Map<className, Ratelimit>` cache with one shared
Redis client and five `Ratelimit` instances (one per class, distinct
Redis prefix). Five exported functions: `checkAuthRateLimit(req)`
(Brief 4 contract preserved byte-identical),
`checkSearchRateLimit(req)`, `checkUploadRateLimit(req, userId)`,
`checkGenerateRateLimit(req, userId)`, `checkImportRateLimit(req,
userId)`. Internal `check(className, identifier)` shared helper.
`LIMITER_CONFIG` is a top-level `const` map of
`{ limit, window, prefix }` per class; adding a sixth class is a
one-line addition + one new exported function (no `init()`
restructuring needed).
- **Decision 3 Per-class limit values (architect-self-ratified with
tuning evidence).** Final table:
| Class | Limit | Window | Key | Helper |
| --- | --- | --- | --- | --- |
| `auth` | 5 | 15 min | IP | `checkAuthRateLimit(req)` (unchanged from Brief 4) |
| `search` | **60** | 1 min | IP | `checkSearchRateLimit(req)` |
| `upload` | 10 | 1 hour | user | `checkUploadRateLimit(req, userId)` |
| `generate` | **5** | 1 hour | user | `checkGenerateRateLimit(req, userId)` |
| `import` | 5 | 1 hour | user (admin-only) | `checkImportRateLimit(req, userId)` |
Two architect raises from the parent's pre-investigation defaults
warrant a permanent record so future tuning convoys see the
evidence:
- **`search` raised 30 60/min.** `components/ShareModal.js`'s
`handleSearch` (lines 56-77) fires on every keystroke with NO
debounce; typing a 17-char email = 16 requests in <5s, which
would 429 a single legitimate user against the 30/1min default.
60/1min covers a realistic burst and still stops a scraper
inside 2-3 seconds. If real users still 429, the fix is a
one-line `LIMITER_CONFIG` edit (60 90 or 120); not a
release-blocker. Queued follow-up name if needed:
`tune-search-rate-limit`.
- **`generate` kept at 5/hour (parent suggested 3).**
`pages/api/user/avatar/generate.js` calls DiceBear (free public
API), not OpenAI / Replicate / Stability cost is Vercel blob
storage + DiceBear-side throttling, not per-call $. 5/hour is
generous enough that a user trying 3-4 seeds doesn't hit the
wall, strict enough that an accidental render loop still trips
inside the first minute. If we ever switch generators to a paid
provider, the same one-line `LIMITER_CONFIG` edit drops it back
to 3 or lower.
- **Decision 4 Two-extractor shape with defensive THROW
(architect-self-ratified).** `extractIpIdentifier(req)`
(module-private; renamed from the pre-refactor `extractIdentifier`)
and `extractUserIdentifier(userId)` (module-private; new). The
user extractor **THROWS** with
`'[rate-limit] extractUserIdentifier called without an
authenticated userId. Place the rate-limit gate AFTER the auth
check, never before.'` when `userId` is `null` / `undefined` /
`''` / `NaN`. Surfaces gate-ordering bugs **at dev time** rather
than silently falling back to IP and converting a per-user limit
into a per-IP limit (which would lock other household members out
for one user's behavior the exact regression flagged in the
convoy file's § Known constraints). Numeric `0` is intentionally
accepted (returns `'user:0'`) defensively forward-compatible if
a future schema introduces user ID 0.
- **Decision 5 Uniform 429 response (architect-self-ratified).**
Single `'Too many attempts. Try again later.'` message across all
five classes, matching `login.js` + `register.js` verbatim.
`Retry-After` calculation is `Math.ceil((reset - Date.now()) /
1000)`; status code is `429`. Per-class variation (e.g., *"Too
many search requests"*) was considered and rejected because it
would fingerprint to an attacker which routes have which limits +
windows, making it easier to craft a request pattern that avoids
429 on the more-permissive routes while still abusing the
less-permissive ones.
- **Decision 6 No new vitest or playwright tests
(architect-self-ratified).** Same reasoning as `cors-tighten`
Decision D4. Per-route handler tests are deferred to the queued
`fill-vitest-handler-coverage` convoy. The fail-loud-in-prod
predicate in `lib/rate-limit.js::init()` is the integration test
if `KV_REST_API_*` is unset on a deployed env, every gated route
fails closed on the first call. Architect-time investigation
corrected a stale claim in the convoy file's § Known constraints
(the line *"auth-utils tests run through the same module"*): `rg
'rate-limit|@upstash' test/` returns zero matches, so the lib
refactor is strictly safer than the convoy file implied there
is no transitive vitest path to break.
### As-shipped surface
The shape splits cleanly into four layers (mirrors `cors-tighten`'s
24-source-files / 1-CI-job pattern split, but with one extra layer
because of the atomic admin UI fix and the rule extension):
1. **One `lib/rate-limit.js` refactor.** Single-class auth-only
limiter 5-class `Map<className, Ratelimit>` with distinct Redis
prefixes (`tcgvault:auth`, `tcgvault:search`, `tcgvault:upload`,
`tcgvault:generate`, `tcgvault:import`). +90 / -23 net (90 lines
added, 23 lines reshaped the existing `init()`,
`extractIdentifier`, `checkAuthRateLimit` functions are
conceptually preserved but restructured to share infrastructure
across all five classes). Brief 4's `checkAuthRateLimit(req)`
return shape is **byte-identical** post-refactor (`{ allowed,
remaining, reset }`); `login.js` + `register.js` were not
touched and continue to work unchanged.
2. **Six new route-gate additions** (lib refactor delivers the
helpers; routes call them):
- `pages/api/users/search.js` `checkSearchRateLimit(req)` after
the existing inline JWT verify. IP-keyed. +7 lines.
- `pages/api/cards/search.js` `checkSearchRateLimit(req)` at top
of handler after the method check, before the 240-line SQL
god-function (which stays byte-identical per § Files explicitly
out of scope). IP-keyed, anonymous-by-design. +7 lines.
- `pages/api/user/avatar.js` `checkUploadRateLimit(req,
user.userId)` after `getUserFromRequest`, before the POST/DELETE
method branching (so the limiter fires before
`parseMultipartFormData` consumes the 5MB body). User-keyed. +7
lines.
- `pages/api/user/avatar/generate.js`
`checkGenerateRateLimit(req, user.userId)` after
`getUserFromRequest`, before the user-data SQL query.
User-keyed. +7 lines.
- `pages/api/cards/import-mtg.js`
`getUserFromRequest` + `if (user.role !== 'admin') return 403`
+ `checkImportRateLimit(req, user.userId)` before the existing
`try` block. Closes the publicly-callable anonymous-abuse
vector. +16 lines.
- `pages/api/cards/import-pokemon.js` same shape as
`import-mtg.js`. +16 lines.
- `pages/api/cards/import-lorcana.js` same shape as
`import-mtg.js`; gated defensively despite zero current
frontend callers. +16 lines.
3. **One atomic admin UI fix.** `pages/admin/card-import.js` adds
`'Authorization': \`Bearer ${localStorage.getItem('auth_token')}\``
to the import fetch's headers (one-line addition; the 309-line
god-component is otherwise byte-identical). **This was the
architect's critical pre-brief discovery and the reason Decision
1 routes back to the operator** — gating the import APIs without
this matching client fetch fix would have closed P0 #6 but
introduced an immediate 401 on every "Import Cards" click,
producing a visible UX regression on the only live admin tooling
that exercises the gated routes. Shipping the gate + the client
fix in the same PR keeps the convoy atomic; PR review enforced
the dependency explicitly.
4. **One `.cursor/rules/api-routes.mdc` § Rate limiting extension.**
Replaced the previous auth-only § with the per-class table + the
verbatim call shape + gate-ordering rules (method check first;
auth before any user-keyed limiter; IP-keyed gate placement is
flexible; admin-role check goes between auth and rate-limit for
the import routes) + identifier-extraction documentation +
uniform 429 response shape + fail-closed env-var contract +
fail-open Upstash-outage behavior. Implementer landed this in
the same PR; doc-writer pass verified the extension is complete
and made no further touch-ups (see § What did NOT change below).
### As-shipped metrics
Diff size (per `git show --stat 708ef45`):
- **12 files modified, +1612 / -23.** Note that the 1612-addition
figure is dominated by `.convoys/add-rate-limiting.md` (676 lines)
and `.convoys/add-rate-limiting/brief-1-extend-rate-limit-and-wire-routes.md`
(751 lines), which the squash includes because the architect commit
preceded the implementer commit on the same branch. The actual
source-file diff is much smaller:
- `lib/rate-limit.js`: +90 / -23 (the lib refactor).
- `.cursor/rules/api-routes.mdc`: +41 (the § Rate limiting extension).
- 6 route files under `pages/api/`: +69 lines total (3× +16 for
import routes, 4× +7 for search/avatar/generate/users-search).
- `pages/admin/card-import.js`: +1 (the Bearer-header addition).
Post-merge CI run 26382185019 + subsequent runs on `main`:
- **`Playwright smoke` PASS in 59s, 3/3 tests in 3.8s** against the
post-rate-limit Vercel preview. Same three checks (`home redirects
or renders without 5xx` 431ms / `sign-in page renders` 331ms /
`public health endpoint responds` 193ms) all green. **Critical
cross-validation:** smoke calls `/api/health` once per run (well
below the search limiter's 60/min ceiling), and the home + sign-in
routes don't touch any of the 6 newly-gated endpoints, so smoke
does NOT 429 against the new search class. The cross-validation
finding accumulates: smoke test 2 (`'sign-in page renders'`) still
passes against the post-CORS + post-rate-limit preview that's
three convoys in a row (PR #15 Layout default-user PR #19
CORS-tighten PR #20 rate-limiting) where the auth surface stayed
stable under sweeping changes, and the same 3-test smoke spec
defended it every time.
- **`forbidden-cors-headers` (from `cors-tighten`)** PASS. None of
the 6 route edits introduced an `Access-Control-Allow-*` header
(the convoy is purely additive of rate-limit gate code; CORS
surface was not touched). The grep stays clean.
- **`forbidden-endpoints` (from `fix-auth-bypass` Brief 3)** PASS.
No new `pages/api/test-*.js` or other deleted-endpoint shapes
reintroduced.
- **`Unit tests (vitest)`** PASS, 21/21 in 27s. No new tests, no
removed tests; the existing `lib/auth-secret.test.js` (3) +
`lib/permission-middleware.test.js` (8) +
`pages/api/auth-utils.test.js` (5) +
`components/Layout.test.js` (5) suites are all unaffected. Decision
6's architect-time verification (`rg 'rate-limit|@upstash' test/`
returns zero matches) confirmed at merge.
- **`Lint`** 128 problems (lint baseline preserved, no
regression). The lib refactor + the 6 route edits + the admin UI
one-liner introduced zero new lint problems; the `|| true`
wrapper in `.github/workflows/ci.yml` was a no-op for this convoy.
- **`Screenshot diff`** `continue-on-error: true` swallow per
`adopt-playwright-smoke` Decision 4 (no baseline committed yet);
the documented Decision-4 end state. Triggered on PR #20 because
the `paths:` filter `pages/**` matches the 6 route edits under
`pages/api/`; same minor false-positive as PR #19, tracked by the
queued `tighten-visual-diff-path-filter` follow-up.
- **All other gates (`Schema map up to date`, `Aggregate gate`)**
green.
### Cross-validation finding: smoke spec defends the rate-limit
surface (organically)
The 3-test smoke spec was authored by `adopt-playwright-smoke` (PR
#18) against an un-rate-limited preview, with no foresight about
this convoy's gating. Post-merge run 26382185019 confirms the spec
**still passes against the post-rate-limit preview** the home
route, the `/login` route, and `/api/health` don't touch any of the
six newly-gated endpoints, and `/api/health` is anonymous /
unrate-limited so the smoke flow doesn't bump up against the search
class's 60/min ceiling. The cross-validation lineage accumulates:
- PR #15 (`fix-layout-default-user`, `ca302a8`) introduced the
`<Link href="/login">Sign in</Link>` CTA that smoke test 2 asserts
on.
- PR #19 (`cors-tighten`, `da50d78`) removed wildcard CORS from 24
handlers; smoke test 2 still passed against the post-CORS
preview.
- PR #20 (`add-rate-limiting`, `708ef45`) this convoy wired 6 new
route gates; smoke test 2 still passes.
Smoke is doing real work: it has now defended the auth surface
against three sweeping changes without anyone having to write a
single dedicated test. P0 #7's resolved state, P0 #5's resolved
state, and now P0 #6's resolved state are all backed by a live CI
signal not just a vitest assertion.
### Operator action required going forward
**None.** No new env vars (Upstash `KV_REST_API_URL` /
`KV_REST_API_TOKEN` were already auto-provisioned via the Vercel
Marketplace integration for Brief 4). No new dependencies
(`@upstash/ratelimit@^2.0.8` + `@upstash/redis@^1.38.0` already
installed). No new secrets, no infra changes, no CI gates to enable
or disable. The fail-loud-in-prod predicate in
`lib/rate-limit.js::init()` is self-defending: if a future deploy
unsets either env var, every gated route fails closed on the first
call (`throw new Error('[rate-limit] Upstash not configured...')`),
which surfaces immediately as a 500 in the Vercel logs rather than
silently disabling brute-force protection.
If a follow-up tuning need surfaces (search 60/min too tight, generate
5/hour too tight, etc.), the fix is a single-line `LIMITER_CONFIG`
edit. See `tune-search-rate-limit` (not yet queued; surface only if
real users 429) and the broader `tiered-rate-limits` (P3 polish,
when a premium-tier scheme exists) for the longer view.
### What did NOT change
Audit trail of files explicitly NOT touched by this convoy, despite
sitting near the rate-limit surface:
- **`pages/api/auth/login.js`, `pages/api/auth/register.js`**
Brief 4 contract preserved. Both files continue to call
`checkAuthRateLimit(req)` against the refactored lib; byte-identical
post-merge. Manual verification at architect time: the 6th-attempt
429 path still fires correctly.
- **`lib/permission-middleware.js`, `lib/auth-secret.js`,
`pages/api/auth-utils.js`** auth surface untouched. No `null`-vs-
synthetic-admin regression risk; `test/lib/permission-middleware.test.js`'s
negative regression test still defends Gotcha #2.
- **`package.json`, `package-lock.json`** zero dep additions, zero
version bumps. `@upstash/ratelimit@^2.0.8` + `@upstash/redis@^1.38.0`
pinned from Brief 4.
- **`.github/workflows/*.yml`** no new CI gate. Per-class
rate-limit wiring isn't grep-checkable; the fail-loud-in-prod
predicate is the integration test (see Decision 6).
- **`test/**`** Decision 6, no new vitest or playwright specs.
21/21 still green; smoke 3/3 still green.
- **`pages/api/cards/search.js`'s SQL** the 240-line god-function
with 7+ conditional `SELECT * FROM cards WHERE …` branches was
explicitly out of scope per `.convoys/add-rate-limiting.md` §
Out of scope. Queued as `god-function-split` /
`refactor-cards-search-sql` (not yet queued in `ship-readiness.md`
surface when the convoy is sized).
- **`pages/api/user/avatar.js`'s `parseMultipartFormData` body
streaming** the 5MB multipart body is consumed via
`req.on('data')` before any rate-limit gate can short-circuit, so
an attacker can still exhaust the 5MB body per 429. Queued as
`harden-multipart-parser` (not yet in `ship-readiness.md` surface
if a real abuse incident occurs). The brief's gate-ordering places
the limiter BEFORE the method branches that call
`parseMultipartFormData`, so when that future hardening lands, the
gate ordering is already correct.
- **`scripts/import-*.js`** standalone CLI scripts independent of
the API routes; not in scope per § Files explicitly out of scope.
An operator running `node scripts/import-mtg.js` directly bypasses
the rate-limit + admin-role gate entirely (which is the intended
flow local admin work is unthrottled).
- **`AGENTS.md`'s Gotcha #5 (deleted `setup-database.js`) + Gotcha
#8 (Layout default-user)** both still RESOLVED, both unchanged.
This convoy's doc-writer pass updates Gotcha #12 (rate-limit
env-var contract) to reflect the 5-class reality, but Gotcha #5
and #8 are independent of this convoy.
- **Auth wrapper extraction (`withAdmin(handler)`)** flagged in
`.cursor/rules/auth-and-permissions.mdc` (*"check user.role ===
'admin' directly; consider extracting `withAdmin()` if a third call
site appears"*). The three import routes are the third+fourth+fifth
call sites in the codebase, but extracting the wrapper is its own
scope for this convoy, the inline `if (user.role !== 'admin')
return 403` shape was preserved across all three. Surface as a
follow-up convoy if a sixth call site appears or if a reviewer
flags the inline shape as a maintainability concern; for now, the
uniform inline check across the three import routes is consistent
with the rest of the codebase.