Reflects the merged add-rate-limiting convoy (PR #20, squash commit708ef45) in repo documentation. **This is the milestone cleanup** — add-rate-limiting closed P0 #6 (No rate limiting anywhere), the LAST open P0 ship-blocker. `.convoys/ship-readiness.md`'s § Status summary flips from "7 of 8 RESOLVED; 1 remains" to **"8 of 8 RESOLVED. Launch-readiness P0 checklist is empty."** One brief in the convoy: Brief 1 shipped as planned with no scope expansions and no implementer deviations from the verbatim spec; all six architect decisions ratified verbatim at gate 1 (D1 operator-ratified Option A; D2-D6 architect-self-ratified). .convoys/add-rate-limiting.md: - frontmatter status: in-progress -> shipped (added shipped: 2026-05-24) - new ## As-shipped section. Opens with the milestone language pointing back at ship-readiness.md's flipped § Status summary. Decisions section captures all 6 ratifications (D1 operator- ratified Option A — Critical: WHY the atomic admin UI fix in pages/admin/card-import.js was Decision 1's hidden coupling requirement, since API gating alone would have broken every "Import Cards" click; D2 hybrid named-limiter shape with Map<className, Ratelimit> cache; D3 per-class table including the two D3 tuning-evidence raises — search 30 -> 60/min because ShareModal.handleSearch has no debounce so a 17-char email = 16 requests in <5s, and generate kept at 5/hour because DiceBear is free not paid AI; D4 two-extractor shape with defensive THROW on null/empty userId; D5 uniform 429 message; D6 no new vitest or playwright specs deferred to fill-vitest-handler-coverage). As-shipped surface broken into 4 layers (1 lib refactor + 6 route gates + 1 atomic admin UI fix + 1 rule extension) mirroring the cors-tighten cleanup's pattern-split shape. Empirical CI metrics from post-merge run 26382185019 (Playwright smoke 59s 3/3 in 3.8s, forbidden-cors-headers pass, vitest 21/21, lint 128 baseline, Screenshot diff continue-on-error swallow per Decision 4). Cross-validation finding: smoke test 2 still passes against the post-rate-limit preview — that's three convoys in a row (PR #15 Layout default-user, PR #19 CORS-tighten, PR #20 rate-limiting) where the same 3-test smoke spec defended the auth surface through sweeping changes. Operator-action-required: none. What did NOT change audit trail. .convoys/ship-readiness.md: - § Status summary at the top flipped from 7/8 to 8/8 RESOLVED. Header text updated to "Launch-readiness P0 checklist is empty." P0 #6 row in the table flips from PARTIAL to RESOLVED with the two-convoy lineage (fix-auth-bypass Brief 4 + add-rate-limiting). Trailing paragraph rewritten as a milestone note: security gate closed; remaining launch work is P1 quality bar + P2/P3 polish. - P0 #6 entry flipped from PARTIAL to RESOLVED with the full add-rate-limiting as-shipped block (8 sub-bullets covering the lib refactor shape, the per-class table, the defensive THROW, the three import routes' auth-gating, the atomic admin UI fix and WHY, the rule extension, the 6 decisions, and the diff breakdown). Brief 4's 2026-05-23 partial is preserved as the prior as-shipped layer to maintain the audit trail. - Launch sequence step 4 marked RESOLVED 2026-05-24 with the squash commit + smoke metrics inline. - Queued convoys: removed the add-rate-limiting entry (it shipped). Added a new delete-dead-lorcana-import entry (P3 polish; the Lorcana import route was gated defensively in PR #20 despite zero current frontend callers — pages/admin/card-import.js's <select> only offers mtg + pokemon — so if Lorcana stays permanently out of the admin UI, this is the cleanup PR). Added three "flagged but kept out of scope" follow-ups per the convoy's § What did NOT change: harden-multipart-parser (P2; 5MB body still consumed before the 429 path on avatar.js), god-function-split / refactor-cards-search-sql (P2; 240-line 7-branch SQL in cards/search.js), and withAdmin(handler) wrapper extraction (P3 DX; the three import routes are call sites #3-5 in the codebase but uniform inline shape was preserved for convoy atomicity). Updated tighten-visual-diff-path-filter to note PR #20 also tripped the same false-positive. AGENTS.md: - Gotcha #12 extended end-to-end. Was the single-class auth-only lib + the env-var contract; is now the 5-class reality with a full per-class table (helper / limit-window / key / routes), the defensive THROW pattern in extractUserIdentifier, the gate-ordering rule for per-user limiters, and the auth → admin-role → rate-limit ordering for the three import routes. Prominent milestone line opens the new content: "add-rate-limiting convoy (squash708ef45, PR #20, 2026-05-24) closed P0 #6 — all 8 P0s now RESOLVED." Original env-var contract paragraph (KV_REST_API_URL / KV_REST_API_TOKEN, fail-closed-in-prod / warn-and-noop-in-dev) is preserved verbatim above the new content. - § 6 Testing: intentionally untouched (no test surface changed; vitest 21/21 and smoke 3/3 still apply). - § 7 Deployment: intentionally untouched (no deployment-shape changed; same KV_REST_API_* env vars from Brief 4). .cursor/rules/api-routes.mdc: - The implementer extended § Rate limiting in PR #20 with the per-class table + verbatim call shape + gate-ordering rules + identifier-extraction + uniform 429 + fail-closed env-var contract + fail-open Upstash-outage behavior. Doc-writer pass verified completeness; added a one-sentence convoy-attribution line at the top of § Rate limiting citing the two-convoy lineage (fix-auth-bypass Brief 4 for the auth class + add-rate-limiting for the other four classes and 7 newly-gated routes), mirroring the post-cors-tighten § CORS attribution shape. No other touch-ups needed. No changes to: package.json, package-lock.json, lib/rate-limit.js, pages/**, components/**, scripts/**, test/**, tests/**, .github/workflows/**, README.md, TESTING_GUIDE.md, playwright.config.js, eslint.config.mjs. Co-authored-by: Cursor <cursoragent@cursor.com>
1035 lines
55 KiB
Markdown
1035 lines
55 KiB
Markdown
---
|
||
name: add-rate-limiting
|
||
classification: convoy
|
||
success_metric: |
|
||
All endpoints listed in P0 #6 of `.convoys/ship-readiness.md` carry
|
||
a rate-limit gate appropriate to their abuse class (search /
|
||
upload / import / admin). The card-import endpoints are no longer
|
||
anonymous-callable. `lib/rate-limit.js` exposes named limiters
|
||
per route class with documented limits + windows + key-extraction
|
||
strategies. P0 #6 flips PARTIAL → RESOLVED, closing the last
|
||
open P0 ship-blocker.
|
||
skip:
|
||
- role-design-system-auditor
|
||
- role-a11y-auditor
|
||
- role-ux-reviewer
|
||
- role-ia-architect
|
||
status: shipped
|
||
created: 2026-05-24
|
||
shipped: 2026-05-24
|
||
parent: ship-readiness
|
||
addresses: P0 #6 (PARTIAL → RESOLVED) — the LAST open P0
|
||
depends_on:
|
||
- fix-auth-bypass (Brief 4 shipped lib/rate-limit.js + auth-only limiter)
|
||
- cors-tighten (just closed P0 #5; not a hard dep but the API surface is now CORS-clean)
|
||
---
|
||
|
||
# Convoy: add-rate-limiting
|
||
|
||
Extend `lib/rate-limit.js` and wire the result into the remaining
|
||
abusable endpoints. This is the final P0 — when it lands, the
|
||
launch-readiness ship-blocker list is empty (all 8 of 8 RESOLVED).
|
||
|
||
## Why now
|
||
|
||
`fix-auth-bypass` Brief 4 shipped the rate-limit infrastructure
|
||
(`lib/rate-limit.js`, `@upstash/ratelimit@^2.0.8`,
|
||
`@upstash/redis@^1.38.0`, Vercel-Upstash Marketplace env vars
|
||
`KV_REST_API_URL` / `KV_REST_API_TOKEN`) and wired it into the
|
||
two auth endpoints (`login`, `register`) with a 5-attempts /
|
||
15-minute sliding window. The remaining abusable surface was
|
||
intentionally deferred to this convoy.
|
||
|
||
The "Queued convoys" entry in `.convoys/ship-readiness.md` lists
|
||
the surface as:
|
||
|
||
- `/api/users/search`
|
||
- `/api/cards/search`
|
||
- All `/api/cards/import-*`
|
||
- `/api/user/avatar*` (upload)
|
||
|
||
Parent's pre-architect audit at convoy creation surfaced a
|
||
**critical secondary finding** beyond "missing rate limit":
|
||
|
||
**`pages/api/cards/import-mtg.js`, `import-pokemon.js`,
|
||
`import-lorcana.js` have ZERO authentication checks.** They are
|
||
publicly callable, they hit Scryfall / Pokémon-TCG / Lorcana APIs
|
||
with no caller throttling AND no auth, and they perform UPSERTs
|
||
into the `cards` table. An attacker can:
|
||
|
||
1. Trigger expensive external API calls from your IP (Scryfall +
|
||
Pokémon-TCG have published rate limits; you'd hit them with no
|
||
recourse, getting your IP throttled at the source).
|
||
2. Cause unbounded DB writes (each import inserts hundreds-to-thousands
|
||
of rows; `card.id` lookups deduplicate but the INSERT path runs
|
||
in a tight per-card loop).
|
||
3. Even with a generous per-IP rate limit (5 req / hour), with
|
||
1000 IPs the math is 120,000 imports / day — enough to exhaust
|
||
Vercel function quotas and Neon row budgets.
|
||
|
||
The rate-limit alone is insufficient. **Auth-gate-then-rate-limit
|
||
is the correct shape.** Whether to add auth gates to the import
|
||
routes in this convoy (vs. spinning out a separate
|
||
`gate-import-routes` convoy) is **architect Decision 1** below.
|
||
Parent recommends in-scope; expansion is mid-convoy precedent
|
||
established by `drop-public-setup` (Brief 2's CJS/ESM expansion)
|
||
and other prior convoys.
|
||
|
||
## Scope
|
||
|
||
### Confirmed in-scope (regardless of architect Decision 1)
|
||
|
||
- **`lib/rate-limit.js`** — refactor to expose multiple named
|
||
limiters (one per route class), preserving the existing
|
||
`checkAuthRateLimit` export for `login` / `register` backwards
|
||
compatibility. New named limiters per Decision 2 below.
|
||
- **`pages/api/users/search.js`** — add rate-limit gate after the
|
||
existing JWT verification. Limit per Decision 2.
|
||
- **`pages/api/cards/search.js`** — add rate-limit gate at top of
|
||
handler (this endpoint is anonymous-by-design; rate-limit keyed
|
||
by IP).
|
||
- **`pages/api/user/avatar.js`** — add rate-limit gate after the
|
||
existing `getUserFromRequest` check. Limit keyed by `user.userId`
|
||
(authenticated; per-user makes more sense than per-IP for an
|
||
upload endpoint where a household might share an IP).
|
||
- **`pages/api/user/avatar/generate.js`** — same shape as
|
||
`avatar.js`. Per-user, stricter than avatar.js (AI generation
|
||
is more expensive than blob upload).
|
||
- **`.cursor/rules/api-routes.mdc`** § Rate limiting — update the
|
||
existing § with the new per-class pattern + each named limiter's
|
||
use case. The current § shows only the auth pattern.
|
||
|
||
### Scope-expansion-candidate (architect Decision 1)
|
||
|
||
- **`pages/api/cards/import-mtg.js`** — add `getUserFromRequest`
|
||
check at top of handler; return 403 if not admin (`user.role !==
|
||
'admin'`); then rate-limit gate (per-user, very strict — these
|
||
hit external APIs).
|
||
- **`pages/api/cards/import-pokemon.js`** — same shape.
|
||
- **`pages/api/cards/import-lorcana.js`** — same shape.
|
||
- (If Decision 1 = Option A "in scope": ship in this PR. If Option
|
||
B "spin out": queue `gate-import-routes` follow-up convoy and
|
||
mark P0 #6 RESOLVED-with-caveat at convoy close.)
|
||
|
||
### Out of scope
|
||
|
||
- **Global IP-based backstop limiter** (e.g., 1000 req / min per
|
||
IP across all routes via Next.js middleware). Useful but
|
||
separate scope; `add-global-rate-limit-middleware` would be its
|
||
own convoy.
|
||
- **Rate-limit headers on success responses**
|
||
(`X-RateLimit-Remaining`, `X-RateLimit-Reset`). The existing
|
||
`checkAuthRateLimit` returns `remaining` + `reset` in its
|
||
result object but `login.js` doesn't propagate them on
|
||
success; just on 429. Honoring the existing convention.
|
||
- **`@upstash/ratelimit` version bump** — pin stays at `^2.0.8`
|
||
from Brief 4. Bumping is its own convoy.
|
||
- **Per-route handler unit tests** (still deferred to the
|
||
queued `fill-vitest-handler-coverage` convoy — same reasoning
|
||
as `cors-tighten`).
|
||
- **Refactoring the existing `cards/search.js` SQL** (the file
|
||
has a known god-function shape with 7+ conditional SQL
|
||
branches; that's `god-function-split` scope, not here).
|
||
- **Verifying the avatar `parseMultipartFormData` body parser**
|
||
is rate-limit-safe (the body is read in `req.on('data')` before
|
||
any rate-limit gate could short-circuit; an attacker can still
|
||
exhaust the 5MB body even on a 429 path). That's a separate
|
||
body-streaming-defense concern (`harden-multipart-parser`) and
|
||
not in scope here.
|
||
|
||
## Operator action required
|
||
|
||
**None.** All Upstash env vars (`KV_REST_API_URL` /
|
||
`KV_REST_API_TOKEN`) are already auto-provisioned via the Vercel
|
||
Marketplace integration (seeded for `fix-auth-bypass` Brief 4).
|
||
No new secrets, no new dependencies (the `@upstash/ratelimit` +
|
||
`@upstash/redis` packages are already installed).
|
||
|
||
## Decisions to ratify with operator (architect routes)
|
||
|
||
1. **Scope expansion — auth gates on `pages/api/cards/import-*.js`
|
||
in this convoy?**
|
||
- **Option A — In scope (parent recommends).** Add
|
||
`getUserFromRequest` + admin-role check + rate-limit gate to
|
||
all 3 import routes in this PR. ~30 LOC across 3 files.
|
||
Closes the full P0 #6 attack surface in one shot. Precedent:
|
||
`drop-public-setup` Brief 2 (CJS/ESM expansion).
|
||
- **Option B — Spin out.** Stay narrow on the 4 originally-listed
|
||
routes; queue `gate-import-routes` as a separate convoy. Mark
|
||
P0 #6 RESOLVED-with-caveat noting the import-route auth gap.
|
||
- **Option C — Hybrid.** Add the rate-limit gate to import
|
||
routes now (with a clear `TODO: requires-auth` comment),
|
||
gate auth later. Worst of both worlds — leaves an
|
||
anonymous-callable abusive endpoint live with only IP-based
|
||
throttling.
|
||
- Architect investigates: confirm the import routes are
|
||
intended to be admin-only (per AGENTS.md / scripts/ folder
|
||
conventions), confirm Decision-A scope size is bounded, route
|
||
back recommendation.
|
||
|
||
2. **Named-limiter shape in `lib/rate-limit.js`.** The existing
|
||
`checkAuthRateLimit(req)` function has the limit + window
|
||
hardcoded. To support per-class limits cleanly, two main
|
||
patterns:
|
||
- **(a) Named-export functions.** Each route class gets its
|
||
own exported function: `checkSearchRateLimit`,
|
||
`checkUploadRateLimit`, `checkImportRateLimit`. Internal
|
||
`init()` builds one `Ratelimit` instance per class, cached
|
||
under different Redis prefixes. Verbose but very explicit at
|
||
each call site.
|
||
- **(b) Generic `checkRateLimit(req, options)`.** Single
|
||
exported function takes a config object. Calling code
|
||
becomes `await checkRateLimit(req, { class: 'search' })` or
|
||
similar. Less verbose at the lib but more at each call site,
|
||
and the per-route limits are documented in the lib module
|
||
rather than implicit in the function name.
|
||
- **(c) Hybrid.** Keep `checkAuthRateLimit` (used by login +
|
||
register; high-stakes contract; don't break). Add named
|
||
functions for the other 3 classes (search, upload, import).
|
||
Best of both — preserves Brief 4's contract, names new
|
||
classes explicitly.
|
||
- Parent recommends (c). Architect ratifies after reading
|
||
`lib/rate-limit.js` shape.
|
||
|
||
3. **Per-class limit values.** Reference table (architect's
|
||
to tune):
|
||
- **`auth`** (existing) — 5 / 15min, IP-keyed. (Don't change.)
|
||
- **`search`** (new) — recommend 30 / 1min, IP-keyed.
|
||
Legitimate users type-ahead-search; 30/min is generous but
|
||
stops a scraper in its tracks within 2-3s. Same IP-keying
|
||
as auth.
|
||
- **`upload`** (new — avatar.js, avatar/generate.js) —
|
||
recommend 10 / 1hour, user-keyed. Avatar uploads are rare;
|
||
10/hour catches accidental loops without blocking a user
|
||
who hits "save" three times by accident. User-keyed because
|
||
the user is authenticated and a household IP shouldn't
|
||
punish other users.
|
||
- **`generate`** (new — avatar/generate.js specifically) —
|
||
recommend 3 / 1hour, user-keyed. AI generation costs money;
|
||
stricter than plain upload.
|
||
- **`import`** (new — cards/import-*.js) — recommend 5 / 1hour,
|
||
user-keyed (if Decision 1 = Option A). Admin-triggered, hits
|
||
external APIs with their own rate limits; 5/hour is plenty.
|
||
4. **Identifier extraction.** `lib/rate-limit.js`'s
|
||
`extractIdentifier()` currently always extracts the first
|
||
`x-forwarded-for` IP. For per-user limiters (upload, generate,
|
||
import), we need a `user_id`-based extractor. Three options:
|
||
- **(a) Two extractors.** `extractIpIdentifier` (existing) +
|
||
`extractUserIdentifier(req, userId)` (new). Each limiter
|
||
calls the right one.
|
||
- **(b) Single extractor takes an `opts.user` flag.** Less
|
||
code but harder to read.
|
||
- **(c) Caller passes `key` directly.** Each handler calls
|
||
`checkUploadRateLimit(req, { key: user.userId })` and the lib
|
||
just trusts it.
|
||
- Parent recommends (a) — explicit, two small functions,
|
||
hard to misuse.
|
||
|
||
5. **429 response shape.** Match the existing
|
||
`login.js`/`register.js` pattern verbatim:
|
||
```js
|
||
res.setHeader('Retry-After', Math.ceil((reset - Date.now()) / 1000));
|
||
return res.status(429).json({ error: 'Too many attempts. Try again later.' });
|
||
```
|
||
Architect: should the error message vary per class (e.g., "Too
|
||
many search requests" vs "Too many uploads") or stay uniform?
|
||
Recommend uniform — minimizes attacker-fingerprinting of which
|
||
routes have which limits. Architect ratifies.
|
||
|
||
6. **Test coverage in this convoy.** No new per-route handler
|
||
tests (per Decision-4-equivalent from `cors-tighten`; same
|
||
deferred-to-`fill-vitest-handler-coverage` reasoning). The
|
||
Playwright smoke spec doesn't exercise any of these routes,
|
||
so no smoke coverage either. **The fail-loud-in-prod
|
||
semantics in `lib/rate-limit.js`'s `init()` function are the
|
||
integration test** — if the env vars are unset on a deployed
|
||
env, every limit-gated route fails closed on first call.
|
||
|
||
## Known constraints
|
||
|
||
- **`lib/rate-limit.js` already has the right architecture** for
|
||
multiple limiters (lazy `init()` returns a `cached` object;
|
||
refactor pattern is to make `cached` a Map<className,
|
||
Ratelimit> instead of a single instance). Don't rewrite from
|
||
scratch; extend.
|
||
- **Fail-closed semantics in prod must be preserved.** Brief 4's
|
||
`init()` throws when env vars are unset and `NODE_ENV ===
|
||
'production'`. New limiters must inherit this — a misconfigured
|
||
prod env should fail loud on every call, not silently disable
|
||
all rate limiting.
|
||
- **Card-search is anonymous-by-design** — do NOT add a
|
||
`getUserFromRequest` check. The IP-keyed limit is the correct
|
||
defense (search is a public catalogue feature).
|
||
- **User-keyed limits require an authenticated request.** The
|
||
rate-limit gate MUST sit AFTER the auth check in the handler
|
||
body. Wrong order = security regression (anonymous user bypasses
|
||
the gate because there's no `userId` to key on, the extractor
|
||
falls back to IP, and the per-user limit becomes per-IP — same
|
||
household IP could be locked out for a different user's
|
||
behavior).
|
||
- **Brief 4's `lib/rate-limit.js` is unit-tested transitively**
|
||
via the existing vitest suite (none of the 21 tests
|
||
specifically test rate-limit, but the auth-utils tests run
|
||
through the same module). Don't break those.
|
||
- **`@upstash/ratelimit` supports multiple shapes**
|
||
(`slidingWindow`, `fixedWindow`, `tokenBucket`); Brief 4 used
|
||
`slidingWindow`. New limiters should follow the same shape
|
||
unless the architect has a specific reason for a different
|
||
algorithm per class.
|
||
- **Per-route Pattern checks** — the 6 (or 9 if Decision 1 =
|
||
Option A) target files have different existing structures:
|
||
- `users/search.js`: auth check inside try/catch on JWT;
|
||
rate-limit goes after JWT verify, inside the existing try
|
||
- `cards/search.js`: no auth; rate-limit goes at top of
|
||
handler after the method check
|
||
- `user/avatar.js`: auth check inside the outer try (line
|
||
16-19); rate-limit goes after auth, before the method
|
||
branching at line 21
|
||
- `user/avatar/generate.js`: similar shape to avatar.js
|
||
- `cards/import-*.js`: NO auth currently. Decision 1 dictates
|
||
the shape — either add auth-then-rate-limit, or rate-limit-only
|
||
(Option C, not recommended)
|
||
|
||
## Acceptance criteria
|
||
|
||
The convoy is shippable when ALL of the following hold:
|
||
|
||
1. `lib/rate-limit.js` exposes the named limiters per Decision 2.
|
||
Backwards-compat: `checkAuthRateLimit` still works for
|
||
`login.js` / `register.js` (do NOT break Brief 4's contract).
|
||
2. All 4 originally-listed endpoints (`users/search`,
|
||
`cards/search`, `user/avatar`, `user/avatar/generate`) carry
|
||
the appropriate rate-limit gate at the correct ordering
|
||
(after auth if applicable; per Decision 4 keying).
|
||
3. If Decision 1 = Option A: all 3 import endpoints carry both
|
||
the auth gate (admin-role check) + the import rate-limit gate.
|
||
4. `npm run lint` exit code matches baseline (still 128 problems;
|
||
do NOT regress).
|
||
5. `npm run test:run` (vitest) still passes 21/21 (no regression
|
||
on the existing auth-utils tests that transitively load
|
||
`lib/rate-limit.js`).
|
||
6. `npm run test:smoke` (via CI on the PR) still passes 3/3
|
||
against the Vercel preview — proves the live login → verify
|
||
→ /api/health flow doesn't accidentally get rate-limited
|
||
(the smoke spec hits each endpoint once per run, well below
|
||
any limit).
|
||
7. `.cursor/rules/api-routes.mdc` § Rate limiting updated to
|
||
document each new limiter's use case + the per-class limits
|
||
+ the user-keyed vs IP-keyed convention.
|
||
8. Bypass-secret-leak check: zero matches of any
|
||
`KV_REST_API_*` or `UPSTASH_*` env-var values in any CI log.
|
||
(Auto-masked by GitHub Actions; verify post-merge.)
|
||
|
||
## Anything flagged but not acted on (in advance)
|
||
|
||
- **Global IP backstop limiter via Next.js middleware** —
|
||
out of scope; queue `add-global-rate-limit-middleware` if a
|
||
future audit shows non-listed routes being abused.
|
||
- **Body-streaming defense for avatar uploads** — the 5MB
|
||
multipart body is consumed before any rate-limit gate can
|
||
short-circuit. Real defense requires moving the parse into a
|
||
separate edge function or using `read-up-to` semantics; queue
|
||
`harden-multipart-parser` if it ever surfaces in a real abuse
|
||
incident.
|
||
- **Per-tier user limits** — premium users might get higher
|
||
search/upload limits. Queue `tiered-rate-limits` when a
|
||
tiering scheme exists (today there are only `admin` and
|
||
default roles).
|
||
- **Rate-limit metrics dashboard** — Upstash exposes per-key hit
|
||
counts; surfacing them in an admin dashboard would catch
|
||
abuse early. Queue `rate-limit-observability` post-launch.
|
||
- **Card-search SQL god-function** — known shape issue,
|
||
separate convoy (`god-function-split` or
|
||
`refactor-cards-search-sql`); do NOT touch in this convoy.
|
||
|
||
## Decisions (post-architect)
|
||
|
||
Six decisions ratified by `role-architect` on 2026-05-24. **Only Decision 1
|
||
requires operator gate-1 sign-off** (significant scope expansion to admin-only
|
||
import-route enforcement + admin UI source touch); the other five are
|
||
architect-self-ratifiable per the precedent established by `cors-tighten`
|
||
Decision D2-D5 + `fix-vercel-deployment-protection-in-ci` Decisions A/B/D.
|
||
|
||
### Decision 1 — Scope expansion: gate `pages/api/cards/import-*.js` in this convoy (**OPERATOR-RATIFIABLE**)
|
||
|
||
**Ratified: Option A — gate all three import routes with auth + admin-role + import rate-limit, AND fix the admin UI to send the Bearer token.**
|
||
|
||
Architect investigation results:
|
||
|
||
- **`pages/api/cards/import-mtg.js`** — LIVE admin tooling. Called by
|
||
`pages/admin/card-import.js` line 39-49 (the `<select>` defaults to
|
||
`'mtg'`). Currently fetched WITHOUT an `Authorization` header.
|
||
- **`pages/api/cards/import-pokemon.js`** — LIVE admin tooling. Same
|
||
admin UI fetch path; the `<select>` second option is `'pokemon'`.
|
||
- **`pages/api/cards/import-lorcana.js`** — DEAD in frontend. Architect
|
||
ran `rg 'import-lorcana' pages/ components/` and the file has zero
|
||
frontend callers; the admin UI's `<select>` only offers `'mtg'` and
|
||
`'pokemon'`. The standalone `scripts/import-lorcana.js` exists as an
|
||
independent CLI path. Gating defensively is the chosen path because:
|
||
(a) future Lorcana admin UI work inherits protection automatically,
|
||
(b) the diff is uniform (3 routes get identical treatment), (c) a
|
||
future cleanup convoy can delete the route if it stays unused — and
|
||
deletion is strictly easier than gating-then-deleting because Option A
|
||
preserves all the import infrastructure.
|
||
|
||
**Critical scope expansion required by Option A:**
|
||
`pages/admin/card-import.js` currently calls the import APIs without an
|
||
`Authorization` header (line 43-49). The moment the import APIs gain
|
||
`getUserFromRequest`, the admin UI begins returning 401 on every import
|
||
click — visible UX regression. The brief therefore includes a one-line
|
||
edit to `pages/admin/card-import.js` adding
|
||
`'Authorization': \`Bearer ${localStorage.getItem('auth_token')}\``
|
||
to the fetch's headers. This is the minimum touch; the 309-line
|
||
god-component is otherwise byte-identical.
|
||
|
||
**Rejected alternatives:**
|
||
- **Option B (spin out `gate-import-routes`).** Would leave a publicly-callable
|
||
abuse vector live for at least one more convoy cycle; closes P0 #6 only
|
||
partially. Rejected.
|
||
- **Option C (hybrid — rate-limit-only without auth).** Worst of both worlds:
|
||
anonymous abuse + only IP-based throttling. Rejected per the convoy spec.
|
||
- **Option D-full (delete all three import routes).** Would break the admin
|
||
UI immediately and require either deleting `pages/admin/card-import.js` too
|
||
or restructuring it to call something else. Wider scope than gating; defeats
|
||
the convoy's "close P0 #6 in one shot" success metric.
|
||
- **Option D-partial (delete only `import-lorcana.js`, gate mtg+pokemon).**
|
||
Considered. Slightly smaller attack surface, but introduces a non-uniform
|
||
pattern (two gated routes + one deleted route) that complicates the
|
||
reviewer's mental model. Lorcana support is a documented game in
|
||
`AGENTS.md` § 1; deleting the API forecloses the trivial path to a future
|
||
Lorcana admin UI. Rejected as marginal; queue `delete-dead-lorcana-import`
|
||
as a follow-up convoy if Lorcana never gets wired into the admin UI.
|
||
|
||
**Operator action**: ratify Option A or counter-propose. Architect proceeds
|
||
to brief generation assuming Option A; if operator counters before
|
||
implementer dispatch, the brief is revised in place.
|
||
|
||
### Decision 2 — Named-limiter shape in `lib/rate-limit.js` (architect-self-ratifiable)
|
||
|
||
**Ratified: Option (c) hybrid — preserve `checkAuthRateLimit(req)`, add four named functions.**
|
||
|
||
The verbatim new module shape is specified in
|
||
`.convoys/add-rate-limiting/brief-1-extend-rate-limit-and-wire-routes.md`
|
||
§ `lib/rate-limit.js` (modified). Key design choices:
|
||
|
||
- `Map<className, Ratelimit>` instance cache (one Redis client, five
|
||
Ratelimit instances, distinct prefix per class). Each `Ratelimit`
|
||
construction is a cheap object wrap around the shared Redis client; no
|
||
separate REST connection per class.
|
||
- Five exported functions: `checkAuthRateLimit(req)`,
|
||
`checkSearchRateLimit(req)`, `checkUploadRateLimit(req, userId)`,
|
||
`checkGenerateRateLimit(req, userId)`, `checkImportRateLimit(req, userId)`.
|
||
Internal `check(className, identifier)` shared helper.
|
||
- `extractIpIdentifier(req)` is module-private (renamed from the
|
||
pre-refactor `extractIdentifier` — that legacy name is gone but the
|
||
behavior is identical).
|
||
- `LIMITER_CONFIG` is a top-level const map of `{ limit, window, prefix }`
|
||
per class; adding a sixth class is a one-line addition + one new
|
||
exported function (no init() restructuring needed).
|
||
- Brief 4's `checkAuthRateLimit(req)` return shape is **byte-identical**
|
||
post-refactor (`{ allowed, remaining, reset }`) — Brief 4 contract
|
||
preserved.
|
||
|
||
### Decision 3 — Per-class limit values (architect-self-ratifiable)
|
||
|
||
**Ratified with tuning evidence:**
|
||
|
||
| Class | Limit | Window | Key | Rationale |
|
||
| --- | --- | --- | --- | --- |
|
||
| `auth` | 5 | 15 min | IP | Brief 4 unchanged. |
|
||
| `search` | **60** | 1 min | IP | **Raised from parent's 30.** `components/ShareModal.js::handleSearch` (lines 56-77) fires on every keystroke with no debounce; typing a 17-char email = 16 requests in <5s, which would 429 at 30/1min on a single legitimate user entry. 60/1min covers a realistic burst and still stops a scraper. |
|
||
| `upload` | 10 | 1 hour | user | Avatar uploads are rare; 10/hour catches accidental loops. User-keyed because a household IP shouldn't punish other users. |
|
||
| `generate` | **5** | 1 hour | user | **Raised from parent's 3.** `pages/api/user/avatar/generate.js` calls DiceBear (free public API), not OpenAI/Replicate; cost is Vercel blob + DiceBear-side throttling, not per-call $. 5/hour still catches loops without blocking a user trying 3-4 seeds. |
|
||
| `import` | 5 | 1 hour | user | Admin-only via Decision 1; hits Scryfall/Pokémon-TCG/Lorcana APIs with their own rate limits. 5/hour is plenty for the actual import workflow (one set per click; admin won't import 5 sets per hour in normal operation). |
|
||
|
||
If the operator wants different numbers, the change is a single-line edit
|
||
to `LIMITER_CONFIG` in `lib/rate-limit.js` — call out in PR review.
|
||
|
||
### Decision 4 — Identifier extraction shape (architect-self-ratifiable)
|
||
|
||
**Ratified: Option (a) — two extractors, `extractUserIdentifier` THROWS on missing userId.**
|
||
|
||
Verbatim shape in the brief. Key safety property:
|
||
`extractUserIdentifier(userId)` THROWS the message
|
||
`'[rate-limit] extractUserIdentifier called without an authenticated userId. Place the rate-limit gate AFTER the auth check, never before.'`
|
||
when `userId` is `null`, `undefined`, `''`, or `NaN`. This surfaces
|
||
gate-ordering bugs **at dev time** rather than silently falling back to
|
||
IP and converting a per-user limit into a per-IP limit — which would
|
||
lock other household members out for one user's behavior, the exact
|
||
regression flagged in `.convoys/add-rate-limiting.md` § Known constraints.
|
||
|
||
Numeric `0` is intentionally NOT in the throw conditional — if a future
|
||
schema introduces user ID 0 the limiter still keys correctly as
|
||
`'user:0'`. There is no current user with ID 0 in the `users` table; the
|
||
check is defensively forward-compatible.
|
||
|
||
### Decision 5 — 429 response shape (architect-self-ratifiable)
|
||
|
||
**Ratified: uniform message across all five classes** — `'Too many attempts. Try again later.'`.
|
||
|
||
Matches `login.js` + `register.js` verbatim. Per-class variation (e.g.,
|
||
`'Too many search requests'` vs `'Too many upload requests'`) was
|
||
considered and rejected: per-class messages would fingerprint to an
|
||
attacker which routes have which limits + windows, making it easier to
|
||
craft a request pattern that avoids 429 on the more-permissive routes
|
||
while still abusing the less-permissive ones. Single uniform message =
|
||
attacker doesn't know what they hit.
|
||
|
||
`Retry-After` calculation is also identical:
|
||
`Math.ceil((reset - Date.now()) / 1000)`. Status code is `429`.
|
||
|
||
### Decision 6 — Test coverage in this convoy (architect-self-ratifiable)
|
||
|
||
**Ratified: no new vitest or playwright tests.**
|
||
|
||
Same reasoning as `cors-tighten` Decision D4. Per-route handler tests
|
||
are deferred to the queued `fill-vitest-handler-coverage` convoy. The
|
||
fail-loud-in-prod predicate in `lib/rate-limit.js::init()` is the
|
||
integration test — if `KV_REST_API_*` is unset on a deployed env, every
|
||
gated route fails closed on first call.
|
||
|
||
**Correction to the convoy file's "Known constraints" claim:** the file
|
||
states *"Brief 4's `lib/rate-limit.js` is unit-tested transitively via
|
||
the existing vitest suite (none of the 21 tests specifically test
|
||
rate-limit, but the auth-utils tests run through the same module)."*
|
||
This is stale. Architect ran `rg 'rate-limit|@upstash' test/` → zero
|
||
matches. `test/api/auth-utils.test.js` only imports
|
||
`pages/api/auth-utils.js` + `lib/auth-secret.js` (with a mock for
|
||
`lib/database.js`); it does NOT load the auth handlers or
|
||
`lib/rate-limit.js`. The lib refactor is therefore strictly safer than
|
||
the convoy file implies — there is no transitive test path to break.
|
||
Decision 6 still holds.
|
||
|
||
## Architecture
|
||
|
||
### File plan
|
||
|
||
| File | Action | Purpose |
|
||
| --- | --- | --- |
|
||
| `lib/rate-limit.js` | modified | Refactor from single auth-only limiter into Map-of-named-limiters with five classes. Preserves Brief 4's `checkAuthRateLimit(req)` contract; adds four new named exports. |
|
||
| `pages/api/users/search.js` | modified | Add `checkSearchRateLimit(req)` gate after the inline JWT verify, before query-length validation. IP-keyed. |
|
||
| `pages/api/cards/search.js` | modified | Add `checkSearchRateLimit(req)` gate at top of handler, after method check, before the 240-line SQL god-function (which stays byte-identical). IP-keyed, anonymous-by-design. |
|
||
| `pages/api/user/avatar.js` | modified | Add `checkUploadRateLimit(req, user.userId)` after the existing `getUserFromRequest` check, before the POST/DELETE branching. User-keyed; gate fires before `parseMultipartFormData` body parse. |
|
||
| `pages/api/user/avatar/generate.js` | modified | Add `checkGenerateRateLimit(req, user.userId)` after the existing `getUserFromRequest` check, before the user-data SQL query. User-keyed; calls DiceBear (free), not a paid AI service. |
|
||
| `pages/api/cards/import-mtg.js` | modified | Add `getUserFromRequest` + `if (user.role !== 'admin') return 403` + `checkImportRateLimit(req, user.userId)` before the existing `try` block. Closes the publicly-callable anonymous-abuse vector. |
|
||
| `pages/api/cards/import-pokemon.js` | modified | Same shape as `import-mtg.js`. Live admin tooling per architect investigation. |
|
||
| `pages/api/cards/import-lorcana.js` | modified | Same shape as `import-mtg.js`. Gated defensively despite zero current frontend callers — future Lorcana admin UI inherits protection. |
|
||
| `pages/admin/card-import.js` | modified | **Scope expansion for D1.** Add `'Authorization': \`Bearer ${localStorage.getItem('auth_token')}\`` to the import fetch's headers (line 45-47). One-line addition; 309-line god-component otherwise byte-identical. |
|
||
| `.cursor/rules/api-routes.mdc` | modified | Replace § Rate limiting (lines 104-135) with the new per-class table + verbatim call shape + gate-ordering rules + identifier-extraction documentation. Every other section byte-identical. |
|
||
|
||
### API surface
|
||
|
||
No new routes. No request/response shape changes on the 7 gated routes
|
||
(429 is added to the possible status codes for each). The only public
|
||
surface change is:
|
||
|
||
- `/api/cards/import-mtg` POST: now requires `Authorization: Bearer <admin-token>`. Returns 401 (no token), 403 (non-admin), 429 (rate-limited), or the existing 200/400/500 contract.
|
||
- `/api/cards/import-pokemon` POST: same.
|
||
- `/api/cards/import-lorcana` POST: same.
|
||
- `/api/users/search` GET: adds 429 to the possible status codes (existing 401/400/200/500 unchanged).
|
||
- `/api/cards/search` GET: adds 429 to the possible status codes (existing 200/500 unchanged; no auth either pre- or post-edit).
|
||
- `/api/user/avatar` POST/DELETE: adds 429 (existing 401/400/200/500 unchanged).
|
||
- `/api/user/avatar/generate` POST: adds 429 (existing 401/404/200/500 unchanged).
|
||
|
||
All 429 responses set `Retry-After: <seconds>` and return
|
||
`{ "error": "Too many attempts. Try again later." }`.
|
||
|
||
### Schema diff
|
||
|
||
**No schema changes.** No new Postgres tables, columns, or indexes. The
|
||
limiter state lives in Upstash Redis (managed); the existing
|
||
`KV_REST_API_URL` / `KV_REST_API_TOKEN` env vars (auto-provisioned by
|
||
Vercel's Upstash Marketplace integration) cover all five classes.
|
||
|
||
### Test plan
|
||
|
||
Per Decision 6, **no new vitest or playwright specs are added in this convoy**.
|
||
|
||
Existing test coverage that must still pass post-refactor:
|
||
|
||
- `test/lib/auth-secret.test.js` (3 tests) — unrelated to rate-limit.
|
||
- `test/lib/permission-middleware.test.js` (8 tests) — unrelated.
|
||
- `test/api/auth-utils.test.js` (5 tests) — unrelated (does NOT import the auth handlers, verified at architect time).
|
||
- `test/components/Layout.test.js` (5 tests) — unrelated.
|
||
|
||
Total: 21/21 must still pass. `npm run test:run` is BLOCKING in CI per
|
||
the `test:` job in `.github/workflows/ci.yml`.
|
||
|
||
Playwright smoke (3/3, runs against Vercel preview):
|
||
- `home redirects or renders without 5xx` — anonymous GET on `/`; not in scope.
|
||
- `sign-in page renders` — anonymous GET on `/login`; not in scope.
|
||
- `public health endpoint responds` — anonymous GET on `/api/health`; not in scope.
|
||
|
||
None of the smoke tests exercise any of the 7 gated endpoints, so no
|
||
smoke regression risk. Smoke run must stay 3/3 green.
|
||
|
||
Manual verification (run locally pre-PR, paste output in PR description):
|
||
documented in the brief's § Manual verification.
|
||
|
||
### Risk list
|
||
|
||
1. **Brief 4 contract regression on `checkAuthRateLimit(req)`.** If the
|
||
refactor changes the return shape from `{ allowed, remaining, reset }`,
|
||
`pages/api/auth/login.js` + `pages/api/auth/register.js` break silently
|
||
(destructuring undefined). Mitigation: the brief locks the return shape
|
||
as byte-identical; manual verification exercises the 6th-attempt 429
|
||
path on `/api/auth/login`.
|
||
|
||
2. **Per-class Redis prefix collision.** If two `LIMITER_CONFIG` entries
|
||
accidentally share a prefix (typo, copy-paste), a search hit eats from
|
||
the auth budget for the same IP. Mitigation: brief acceptance criterion
|
||
explicitly checks `rg "tcgvault:" lib/rate-limit.js | sort -u` returns
|
||
five distinct lines.
|
||
|
||
3. **Gate-ordering reversal for user-keyed limiters.** If an implementer
|
||
places `checkUploadRateLimit(req, user.userId)` BEFORE the
|
||
`getUserFromRequest` check, every anonymous request throws (via
|
||
`extractUserIdentifier`'s null guard). Mitigation: the throw IS the
|
||
defensive signal — it surfaces as a dev-time 500 immediately rather than
|
||
a silent security regression. The brief's `## Cross-file checks` calls
|
||
out the order-check explicitly.
|
||
|
||
4. **Admin UI broken on first run if `pages/admin/card-import.js` Bearer
|
||
fix doesn't land.** Adding `getUserFromRequest` to the import APIs
|
||
without the matching admin-UI fix produces an immediate 401 on the next
|
||
"Import Cards" click. Mitigation: brief includes both edits as a single
|
||
atomic change; PR review enforces atomicity (acceptance criterion calls
|
||
the dependency out explicitly).
|
||
|
||
5. **Search 60/1min still too low for rapid typists.** If 60/min ends up
|
||
too restrictive in production, surface as a `tune-search-rate-limit`
|
||
follow-up convoy; the fix is a single LIMITER_CONFIG edit (60 → 90 or
|
||
120). Not a release-blocker — the failure mode is a 429 with a
|
||
`Retry-After: 60` header, which the front-end can display as
|
||
"searching too fast, try again in a moment".
|
||
|
||
6. **`extractUserIdentifier` throws on edge-case userIds.** Numeric `0` is
|
||
intentionally accepted (returns `'user:0'`). Empty-string userId throws.
|
||
If a future auth refactor changes the `userId` type to a UUID string,
|
||
the empty-string guard still works. If it changes to an opaque `object`,
|
||
the throw fires (correct — we don't want to key off an object).
|
||
|
||
7. **Upstash outage fails-open.** Brief 4's design choice carries through:
|
||
on `ratelimit.limit(...)` failure, return `{ allowed: true, ... }`. A
|
||
hard Upstash outage during an attack would defeat the limiter for the
|
||
duration of the outage. Mitigation: defense-in-depth (Vercel firewall,
|
||
future fail2ban-style lockout) and Upstash's published SLA. Not
|
||
addressable inside this convoy.
|
||
|
||
8. **Body-stream bypass on avatar.js.** `parseMultipartFormData` consumes
|
||
the 5MB body via `req.on('data')` before the response is sent, so an
|
||
attacker can still exhaust the 5MB ceiling per 429. This is explicitly
|
||
out-of-scope (`harden-multipart-parser` convoy). The brief's gate
|
||
ordering places the limiter BEFORE the method branches (which call
|
||
`parseMultipartFormData`), so when that future hardening lands, the
|
||
gate ordering is already correct.
|
||
|
||
### Verbatim new `lib/rate-limit.js` shape
|
||
|
||
Specified in full in
|
||
`.convoys/add-rate-limiting/brief-1-extend-rate-limit-and-wire-routes.md`
|
||
§ `lib/rate-limit.js` (modified). Implementer has zero design discretion.
|
||
|
||
## Decomposition
|
||
|
||
| Brief # | Title | Files | Depends on | Estimated PR size |
|
||
| --- | --- | --- | --- | --- |
|
||
| 1 | Extend `lib/rate-limit.js` to named per-class limiters + wire into the remaining abusable endpoints + gate the import routes | `lib/rate-limit.js`, `pages/api/users/search.js`, `pages/api/cards/search.js`, `pages/api/user/avatar.js`, `pages/api/user/avatar/generate.js`, `pages/api/cards/import-mtg.js`, `pages/api/cards/import-pokemon.js`, `pages/api/cards/import-lorcana.js`, `pages/admin/card-import.js`, `.cursor/rules/api-routes.mdc` | — | ~180 LOC across 10 files (lib refactor ~65 lines net add; 7 route edits ~6-16 lines each; admin UI +1 line; rules doc ~70 lines + / ~32 lines -) |
|
||
|
||
**Brief count: 1.** Splitting into two briefs (one for lib refactor, one
|
||
for per-route wiring) was considered and rejected:
|
||
|
||
- Per-route wiring **depends on** the lib refactor (the new exports
|
||
don't exist until Brief 1 lands), so parallelism gain via
|
||
`/multitask` is zero — Brief 2 would have `depends_on: [1]` and run
|
||
serially anyway.
|
||
- Single brief presents the reviewer with one coherent change instead
|
||
of two related-but-fragmented PRs.
|
||
- Cross-brief commitment overhead (Brief 1 ships stub limiters, Brief 2
|
||
resolves the wiring) creates exactly the kind of forward-declaration
|
||
scaffolding the `role-architect` Boot-the-brief check exists to
|
||
prevent.
|
||
|
||
The single brief is well-bounded at ~180 LOC across 10 files (mostly
|
||
small additions). No single file gets more than ~70 lines of edit; the
|
||
average file edit is ~12 lines.
|
||
|
||
### Slice dependencies (multitask-ready)
|
||
|
||
```yaml
|
||
slice_dependencies:
|
||
- brief: 1
|
||
depends_on: []
|
||
files:
|
||
- lib/rate-limit.js
|
||
- pages/api/users/search.js
|
||
- pages/api/cards/search.js
|
||
- pages/api/user/avatar.js
|
||
- pages/api/user/avatar/generate.js
|
||
- pages/api/cards/import-mtg.js
|
||
- pages/api/cards/import-pokemon.js
|
||
- pages/api/cards/import-lorcana.js
|
||
- pages/admin/card-import.js
|
||
- .cursor/rules/api-routes.mdc
|
||
```
|
||
|
||
One brief, no `/multitask` fan-out — the conductor dispatches a single
|
||
implementer.
|
||
|
||
## As-shipped
|
||
|
||
Shipped 2026-05-24 as squash commit `708ef45` (PR #20, architect-commit
|
||
`60b842e`, implementer-commit `51a3a97`). **This is the milestone
|
||
convoy — it closes the LAST open P0 (P0 #6) and flips
|
||
`.convoys/ship-readiness.md`'s § Status summary from "7 of 8 RESOLVED;
|
||
1 remains" to "8 of 8 RESOLVED. Launch-readiness P0 checklist is
|
||
empty."** Brief 1 shipped as planned — single lib refactor + six
|
||
route-gate additions + one admin UI atomic fix + one rule extension,
|
||
no scope expansions beyond the architect-ratified Decision 1, and the
|
||
empirical CI metrics + the cross-validation finding + the
|
||
"D3 tuning-evidence" rationale captured here so the next architect /
|
||
reviewer has the audit trail.
|
||
|
||
### Decisions ratified at gate 1
|
||
|
||
All six decisions landed verbatim. Only D1 required operator gate-1
|
||
sign-off (significant scope expansion to admin-only enforcement on the
|
||
import surface plus an atomic admin UI touch); D2-D6 are
|
||
architect-self-ratifiable per the precedent established by
|
||
`cors-tighten` Decisions D2-D5 and
|
||
`fix-vercel-deployment-protection-in-ci` Decisions A/B/D.
|
||
|
||
- **Decision 1 — Option A (operator-ratified).** Gate all three
|
||
`pages/api/cards/import-*.js` routes in this convoy with
|
||
`getUserFromRequest` + `if (user.role !== 'admin') return 403` +
|
||
`checkImportRateLimit(req, user.userId)`. **And** atomically update
|
||
`pages/admin/card-import.js` to send
|
||
`'Authorization': \`Bearer ${localStorage.getItem('auth_token')}\``
|
||
on its import fetch — without this, the API gating would have
|
||
immediately broken every "Import Cards" click in the admin UI. The
|
||
architect's pre-brief investigation flagged this as the critical
|
||
hidden coupling: gating the import APIs without the matching client
|
||
fetch fix would have closed P0 #6 but introduced a visible UX
|
||
regression on the only live admin tooling that depends on it. Lorcana
|
||
was gated defensively despite having zero current frontend callers
|
||
(`pages/admin/card-import.js`'s `<select>` only offers `mtg` /
|
||
`pokemon`) — uniform protection across the three import shapes is
|
||
strictly easier to maintain than a 2-gated-1-deleted asymmetry, and
|
||
if Lorcana stays unused, the queued `delete-dead-lorcana-import`
|
||
follow-up convoy (see `.convoys/ship-readiness.md` § Queued
|
||
convoys) is the cleanup path.
|
||
- **Decision 2 — Hybrid named-limiter shape (architect-self-ratified).**
|
||
`lib/rate-limit.js` refactored from a single auth-only `Ratelimit`
|
||
instance into a `Map<className, Ratelimit>` cache with one shared
|
||
Redis client and five `Ratelimit` instances (one per class, distinct
|
||
Redis prefix). Five exported functions: `checkAuthRateLimit(req)`
|
||
(Brief 4 contract preserved byte-identical),
|
||
`checkSearchRateLimit(req)`, `checkUploadRateLimit(req, userId)`,
|
||
`checkGenerateRateLimit(req, userId)`, `checkImportRateLimit(req,
|
||
userId)`. Internal `check(className, identifier)` shared helper.
|
||
`LIMITER_CONFIG` is a top-level `const` map of
|
||
`{ limit, window, prefix }` per class; adding a sixth class is a
|
||
one-line addition + one new exported function (no `init()`
|
||
restructuring needed).
|
||
- **Decision 3 — Per-class limit values (architect-self-ratified with
|
||
tuning evidence).** Final table:
|
||
|
||
| Class | Limit | Window | Key | Helper |
|
||
| --- | --- | --- | --- | --- |
|
||
| `auth` | 5 | 15 min | IP | `checkAuthRateLimit(req)` (unchanged from Brief 4) |
|
||
| `search` | **60** | 1 min | IP | `checkSearchRateLimit(req)` |
|
||
| `upload` | 10 | 1 hour | user | `checkUploadRateLimit(req, userId)` |
|
||
| `generate` | **5** | 1 hour | user | `checkGenerateRateLimit(req, userId)` |
|
||
| `import` | 5 | 1 hour | user (admin-only) | `checkImportRateLimit(req, userId)` |
|
||
|
||
Two architect raises from the parent's pre-investigation defaults
|
||
warrant a permanent record so future tuning convoys see the
|
||
evidence:
|
||
|
||
- **`search` raised 30 → 60/min.** `components/ShareModal.js`'s
|
||
`handleSearch` (lines 56-77) fires on every keystroke with NO
|
||
debounce; typing a 17-char email = 16 requests in <5s, which
|
||
would 429 a single legitimate user against the 30/1min default.
|
||
60/1min covers a realistic burst and still stops a scraper
|
||
inside 2-3 seconds. If real users still 429, the fix is a
|
||
one-line `LIMITER_CONFIG` edit (60 → 90 or 120); not a
|
||
release-blocker. Queued follow-up name if needed:
|
||
`tune-search-rate-limit`.
|
||
- **`generate` kept at 5/hour (parent suggested 3).**
|
||
`pages/api/user/avatar/generate.js` calls DiceBear (free public
|
||
API), not OpenAI / Replicate / Stability — cost is Vercel blob
|
||
storage + DiceBear-side throttling, not per-call $. 5/hour is
|
||
generous enough that a user trying 3-4 seeds doesn't hit the
|
||
wall, strict enough that an accidental render loop still trips
|
||
inside the first minute. If we ever switch generators to a paid
|
||
provider, the same one-line `LIMITER_CONFIG` edit drops it back
|
||
to 3 or lower.
|
||
- **Decision 4 — Two-extractor shape with defensive THROW
|
||
(architect-self-ratified).** `extractIpIdentifier(req)`
|
||
(module-private; renamed from the pre-refactor `extractIdentifier`)
|
||
and `extractUserIdentifier(userId)` (module-private; new). The
|
||
user extractor **THROWS** with
|
||
`'[rate-limit] extractUserIdentifier called without an
|
||
authenticated userId. Place the rate-limit gate AFTER the auth
|
||
check, never before.'` when `userId` is `null` / `undefined` /
|
||
`''` / `NaN`. Surfaces gate-ordering bugs **at dev time** rather
|
||
than silently falling back to IP and converting a per-user limit
|
||
into a per-IP limit (which would lock other household members out
|
||
for one user's behavior — the exact regression flagged in the
|
||
convoy file's § Known constraints). Numeric `0` is intentionally
|
||
accepted (returns `'user:0'`) — defensively forward-compatible if
|
||
a future schema introduces user ID 0.
|
||
- **Decision 5 — Uniform 429 response (architect-self-ratified).**
|
||
Single `'Too many attempts. Try again later.'` message across all
|
||
five classes, matching `login.js` + `register.js` verbatim.
|
||
`Retry-After` calculation is `Math.ceil((reset - Date.now()) /
|
||
1000)`; status code is `429`. Per-class variation (e.g., *"Too
|
||
many search requests"*) was considered and rejected because it
|
||
would fingerprint to an attacker which routes have which limits +
|
||
windows, making it easier to craft a request pattern that avoids
|
||
429 on the more-permissive routes while still abusing the
|
||
less-permissive ones.
|
||
- **Decision 6 — No new vitest or playwright tests
|
||
(architect-self-ratified).** Same reasoning as `cors-tighten`
|
||
Decision D4. Per-route handler tests are deferred to the queued
|
||
`fill-vitest-handler-coverage` convoy. The fail-loud-in-prod
|
||
predicate in `lib/rate-limit.js::init()` is the integration test
|
||
— if `KV_REST_API_*` is unset on a deployed env, every gated route
|
||
fails closed on the first call. Architect-time investigation
|
||
corrected a stale claim in the convoy file's § Known constraints
|
||
(the line *"auth-utils tests run through the same module"*): `rg
|
||
'rate-limit|@upstash' test/` returns zero matches, so the lib
|
||
refactor is strictly safer than the convoy file implied — there
|
||
is no transitive vitest path to break.
|
||
|
||
### As-shipped surface
|
||
|
||
The shape splits cleanly into four layers (mirrors `cors-tighten`'s
|
||
24-source-files / 1-CI-job pattern split, but with one extra layer
|
||
because of the atomic admin UI fix and the rule extension):
|
||
|
||
1. **One `lib/rate-limit.js` refactor.** Single-class auth-only
|
||
limiter → 5-class `Map<className, Ratelimit>` with distinct Redis
|
||
prefixes (`tcgvault:auth`, `tcgvault:search`, `tcgvault:upload`,
|
||
`tcgvault:generate`, `tcgvault:import`). +90 / -23 net (90 lines
|
||
added, 23 lines reshaped — the existing `init()`,
|
||
`extractIdentifier`, `checkAuthRateLimit` functions are
|
||
conceptually preserved but restructured to share infrastructure
|
||
across all five classes). Brief 4's `checkAuthRateLimit(req)`
|
||
return shape is **byte-identical** post-refactor (`{ allowed,
|
||
remaining, reset }`); `login.js` + `register.js` were not
|
||
touched and continue to work unchanged.
|
||
2. **Six new route-gate additions** (lib refactor delivers the
|
||
helpers; routes call them):
|
||
- `pages/api/users/search.js` — `checkSearchRateLimit(req)` after
|
||
the existing inline JWT verify. IP-keyed. +7 lines.
|
||
- `pages/api/cards/search.js` — `checkSearchRateLimit(req)` at top
|
||
of handler after the method check, before the 240-line SQL
|
||
god-function (which stays byte-identical per § Files explicitly
|
||
out of scope). IP-keyed, anonymous-by-design. +7 lines.
|
||
- `pages/api/user/avatar.js` — `checkUploadRateLimit(req,
|
||
user.userId)` after `getUserFromRequest`, before the POST/DELETE
|
||
method branching (so the limiter fires before
|
||
`parseMultipartFormData` consumes the 5MB body). User-keyed. +7
|
||
lines.
|
||
- `pages/api/user/avatar/generate.js` —
|
||
`checkGenerateRateLimit(req, user.userId)` after
|
||
`getUserFromRequest`, before the user-data SQL query.
|
||
User-keyed. +7 lines.
|
||
- `pages/api/cards/import-mtg.js` —
|
||
`getUserFromRequest` + `if (user.role !== 'admin') return 403`
|
||
+ `checkImportRateLimit(req, user.userId)` before the existing
|
||
`try` block. Closes the publicly-callable anonymous-abuse
|
||
vector. +16 lines.
|
||
- `pages/api/cards/import-pokemon.js` — same shape as
|
||
`import-mtg.js`. +16 lines.
|
||
- `pages/api/cards/import-lorcana.js` — same shape as
|
||
`import-mtg.js`; gated defensively despite zero current
|
||
frontend callers. +16 lines.
|
||
3. **One atomic admin UI fix.** `pages/admin/card-import.js` adds
|
||
`'Authorization': \`Bearer ${localStorage.getItem('auth_token')}\``
|
||
to the import fetch's headers (one-line addition; the 309-line
|
||
god-component is otherwise byte-identical). **This was the
|
||
architect's critical pre-brief discovery and the reason Decision
|
||
1 routes back to the operator** — gating the import APIs without
|
||
this matching client fetch fix would have closed P0 #6 but
|
||
introduced an immediate 401 on every "Import Cards" click,
|
||
producing a visible UX regression on the only live admin tooling
|
||
that exercises the gated routes. Shipping the gate + the client
|
||
fix in the same PR keeps the convoy atomic; PR review enforced
|
||
the dependency explicitly.
|
||
4. **One `.cursor/rules/api-routes.mdc` § Rate limiting extension.**
|
||
Replaced the previous auth-only § with the per-class table + the
|
||
verbatim call shape + gate-ordering rules (method check first;
|
||
auth before any user-keyed limiter; IP-keyed gate placement is
|
||
flexible; admin-role check goes between auth and rate-limit for
|
||
the import routes) + identifier-extraction documentation +
|
||
uniform 429 response shape + fail-closed env-var contract +
|
||
fail-open Upstash-outage behavior. Implementer landed this in
|
||
the same PR; doc-writer pass verified the extension is complete
|
||
and made no further touch-ups (see § What did NOT change below).
|
||
|
||
### As-shipped metrics
|
||
|
||
Diff size (per `git show --stat 708ef45`):
|
||
|
||
- **12 files modified, +1612 / -23.** Note that the 1612-addition
|
||
figure is dominated by `.convoys/add-rate-limiting.md` (676 lines)
|
||
and `.convoys/add-rate-limiting/brief-1-extend-rate-limit-and-wire-routes.md`
|
||
(751 lines), which the squash includes because the architect commit
|
||
preceded the implementer commit on the same branch. The actual
|
||
source-file diff is much smaller:
|
||
- `lib/rate-limit.js`: +90 / -23 (the lib refactor).
|
||
- `.cursor/rules/api-routes.mdc`: +41 (the § Rate limiting extension).
|
||
- 6 route files under `pages/api/`: +69 lines total (3× +16 for
|
||
import routes, 4× +7 for search/avatar/generate/users-search).
|
||
- `pages/admin/card-import.js`: +1 (the Bearer-header addition).
|
||
|
||
Post-merge CI run 26382185019 + subsequent runs on `main`:
|
||
|
||
- **`Playwright smoke` — PASS in 59s, 3/3 tests in 3.8s** against the
|
||
post-rate-limit Vercel preview. Same three checks (`home redirects
|
||
or renders without 5xx` ✓ 431ms / `sign-in page renders` ✓ 331ms /
|
||
`public health endpoint responds` ✓ 193ms) — all green. **Critical
|
||
cross-validation:** smoke calls `/api/health` once per run (well
|
||
below the search limiter's 60/min ceiling), and the home + sign-in
|
||
routes don't touch any of the 6 newly-gated endpoints, so smoke
|
||
does NOT 429 against the new search class. The cross-validation
|
||
finding accumulates: smoke test 2 (`'sign-in page renders'`) still
|
||
passes against the post-CORS + post-rate-limit preview — that's
|
||
three convoys in a row (PR #15 Layout default-user → PR #19
|
||
CORS-tighten → PR #20 rate-limiting) where the auth surface stayed
|
||
stable under sweeping changes, and the same 3-test smoke spec
|
||
defended it every time.
|
||
- **`forbidden-cors-headers` (from `cors-tighten`)** — PASS. None of
|
||
the 6 route edits introduced an `Access-Control-Allow-*` header
|
||
(the convoy is purely additive of rate-limit gate code; CORS
|
||
surface was not touched). The grep stays clean.
|
||
- **`forbidden-endpoints` (from `fix-auth-bypass` Brief 3)** — PASS.
|
||
No new `pages/api/test-*.js` or other deleted-endpoint shapes
|
||
reintroduced.
|
||
- **`Unit tests (vitest)`** — PASS, 21/21 in 27s. No new tests, no
|
||
removed tests; the existing `lib/auth-secret.test.js` (3) +
|
||
`lib/permission-middleware.test.js` (8) +
|
||
`pages/api/auth-utils.test.js` (5) +
|
||
`components/Layout.test.js` (5) suites are all unaffected. Decision
|
||
6's architect-time verification (`rg 'rate-limit|@upstash' test/`
|
||
returns zero matches) confirmed at merge.
|
||
- **`Lint`** — 128 problems (lint baseline preserved, no
|
||
regression). The lib refactor + the 6 route edits + the admin UI
|
||
one-liner introduced zero new lint problems; the `|| true`
|
||
wrapper in `.github/workflows/ci.yml` was a no-op for this convoy.
|
||
- **`Screenshot diff`** — `continue-on-error: true` swallow per
|
||
`adopt-playwright-smoke` Decision 4 (no baseline committed yet);
|
||
the documented Decision-4 end state. Triggered on PR #20 because
|
||
the `paths:` filter `pages/**` matches the 6 route edits under
|
||
`pages/api/`; same minor false-positive as PR #19, tracked by the
|
||
queued `tighten-visual-diff-path-filter` follow-up.
|
||
- **All other gates (`Schema map up to date`, `Aggregate gate`)** —
|
||
green.
|
||
|
||
### Cross-validation finding: smoke spec defends the rate-limit
|
||
surface (organically)
|
||
|
||
The 3-test smoke spec was authored by `adopt-playwright-smoke` (PR
|
||
#18) against an un-rate-limited preview, with no foresight about
|
||
this convoy's gating. Post-merge run 26382185019 confirms the spec
|
||
**still passes against the post-rate-limit preview** — the home
|
||
route, the `/login` route, and `/api/health` don't touch any of the
|
||
six newly-gated endpoints, and `/api/health` is anonymous /
|
||
unrate-limited so the smoke flow doesn't bump up against the search
|
||
class's 60/min ceiling. The cross-validation lineage accumulates:
|
||
|
||
- PR #15 (`fix-layout-default-user`, `ca302a8`) introduced the
|
||
`<Link href="/login">Sign in</Link>` CTA that smoke test 2 asserts
|
||
on.
|
||
- PR #19 (`cors-tighten`, `da50d78`) removed wildcard CORS from 24
|
||
handlers; smoke test 2 still passed against the post-CORS
|
||
preview.
|
||
- PR #20 (`add-rate-limiting`, `708ef45`) — this convoy — wired 6 new
|
||
route gates; smoke test 2 still passes.
|
||
|
||
Smoke is doing real work: it has now defended the auth surface
|
||
against three sweeping changes without anyone having to write a
|
||
single dedicated test. P0 #7's resolved state, P0 #5's resolved
|
||
state, and now P0 #6's resolved state are all backed by a live CI
|
||
signal — not just a vitest assertion.
|
||
|
||
### Operator action required going forward
|
||
|
||
**None.** No new env vars (Upstash `KV_REST_API_URL` /
|
||
`KV_REST_API_TOKEN` were already auto-provisioned via the Vercel
|
||
Marketplace integration for Brief 4). No new dependencies
|
||
(`@upstash/ratelimit@^2.0.8` + `@upstash/redis@^1.38.0` already
|
||
installed). No new secrets, no infra changes, no CI gates to enable
|
||
or disable. The fail-loud-in-prod predicate in
|
||
`lib/rate-limit.js::init()` is self-defending: if a future deploy
|
||
unsets either env var, every gated route fails closed on the first
|
||
call (`throw new Error('[rate-limit] Upstash not configured...')`),
|
||
which surfaces immediately as a 500 in the Vercel logs rather than
|
||
silently disabling brute-force protection.
|
||
|
||
If a follow-up tuning need surfaces (search 60/min too tight, generate
|
||
5/hour too tight, etc.), the fix is a single-line `LIMITER_CONFIG`
|
||
edit. See `tune-search-rate-limit` (not yet queued; surface only if
|
||
real users 429) and the broader `tiered-rate-limits` (P3 polish,
|
||
when a premium-tier scheme exists) for the longer view.
|
||
|
||
### What did NOT change
|
||
|
||
Audit trail of files explicitly NOT touched by this convoy, despite
|
||
sitting near the rate-limit surface:
|
||
|
||
- **`pages/api/auth/login.js`, `pages/api/auth/register.js`** —
|
||
Brief 4 contract preserved. Both files continue to call
|
||
`checkAuthRateLimit(req)` against the refactored lib; byte-identical
|
||
post-merge. Manual verification at architect time: the 6th-attempt
|
||
429 path still fires correctly.
|
||
- **`lib/permission-middleware.js`, `lib/auth-secret.js`,
|
||
`pages/api/auth-utils.js`** — auth surface untouched. No `null`-vs-
|
||
synthetic-admin regression risk; `test/lib/permission-middleware.test.js`'s
|
||
negative regression test still defends Gotcha #2.
|
||
- **`package.json`, `package-lock.json`** — zero dep additions, zero
|
||
version bumps. `@upstash/ratelimit@^2.0.8` + `@upstash/redis@^1.38.0`
|
||
pinned from Brief 4.
|
||
- **`.github/workflows/*.yml`** — no new CI gate. Per-class
|
||
rate-limit wiring isn't grep-checkable; the fail-loud-in-prod
|
||
predicate is the integration test (see Decision 6).
|
||
- **`test/**`** — Decision 6, no new vitest or playwright specs.
|
||
21/21 still green; smoke 3/3 still green.
|
||
- **`pages/api/cards/search.js`'s SQL** — the 240-line god-function
|
||
with 7+ conditional `SELECT * FROM cards WHERE …` branches was
|
||
explicitly out of scope per `.convoys/add-rate-limiting.md` §
|
||
Out of scope. Queued as `god-function-split` /
|
||
`refactor-cards-search-sql` (not yet queued in `ship-readiness.md`
|
||
— surface when the convoy is sized).
|
||
- **`pages/api/user/avatar.js`'s `parseMultipartFormData` body
|
||
streaming** — the 5MB multipart body is consumed via
|
||
`req.on('data')` before any rate-limit gate can short-circuit, so
|
||
an attacker can still exhaust the 5MB body per 429. Queued as
|
||
`harden-multipart-parser` (not yet in `ship-readiness.md` — surface
|
||
if a real abuse incident occurs). The brief's gate-ordering places
|
||
the limiter BEFORE the method branches that call
|
||
`parseMultipartFormData`, so when that future hardening lands, the
|
||
gate ordering is already correct.
|
||
- **`scripts/import-*.js`** — standalone CLI scripts independent of
|
||
the API routes; not in scope per § Files explicitly out of scope.
|
||
An operator running `node scripts/import-mtg.js` directly bypasses
|
||
the rate-limit + admin-role gate entirely (which is the intended
|
||
flow — local admin work is unthrottled).
|
||
- **`AGENTS.md`'s Gotcha #5 (deleted `setup-database.js`) + Gotcha
|
||
#8 (Layout default-user)** — both still RESOLVED, both unchanged.
|
||
This convoy's doc-writer pass updates Gotcha #12 (rate-limit
|
||
env-var contract) to reflect the 5-class reality, but Gotcha #5
|
||
and #8 are independent of this convoy.
|
||
- **Auth wrapper extraction (`withAdmin(handler)`)** — flagged in
|
||
`.cursor/rules/auth-and-permissions.mdc` (*"check user.role ===
|
||
'admin' directly; consider extracting `withAdmin()` if a third call
|
||
site appears"*). The three import routes are the third+fourth+fifth
|
||
call sites in the codebase, but extracting the wrapper is its own
|
||
scope — for this convoy, the inline `if (user.role !== 'admin')
|
||
return 403` shape was preserved across all three. Surface as a
|
||
follow-up convoy if a sixth call site appears or if a reviewer
|
||
flags the inline shape as a maintainability concern; for now, the
|
||
uniform inline check across the three import routes is consistent
|
||
with the rest of the codebase.
|