Closes out the migrate-ci-to-self-hosted convoy with the two defensive follow-ups Brief 1+2 (PR #132) intentionally deferred. Brief 3 — Check 8 of `forbidden-patterns` in ci.yml. Greps `.github/workflows/` for `runs-on: ubuntu-latest` and fails unless the match is in the documented allowlist (currently `agent-context-drift.yml` only, per Decision D4). Self-tested locally against the post-migration tree: 0 violations. Renames the job from "Forbidden patterns (7 checks)" → "(8 checks)" and normalizes the older "Check N/6" labels to "N/8" for consistency (the inherited mix of `/6` and `/7` was a known cosmetic from the unify-glass-panel-surfaces convoy). Brief 4 — AGENTS.md § 6 and § 7 updates: - § 6 "CI behavior": Playwright smoke runtime range updated to cover post-migration cold vs. warm cache (was a stale 59s figure from pre-migration ubuntu-latest). - § 6 new top-level bullet "Self-hosted runner pool" alongside "CI minute optimizations" — covers where runners live, where caches are bind-mounted on CT 111, the Postgres rewire on CT 102, and the agent-context-drift.yml exemption + how Check 8 enforces it. - § 7 new bullet for the operational story: PAT rotation cadence + the D5 one-line `sed` revert path for when axiom is offline mid-PR-storm. Cross-references the axiom-server CT 111 README and the Beszel down alert. Convoy doc — status flipped queued → shipped, shipped_in lists both PRs, and follow_ups makes the cleanup-stale-ci-runs-cron + seed-visual-baselines-on-linux items machine-greppable. Co-authored-by: Cursor <cursoragent@cursor.com>
205 lines
37 KiB
Markdown
205 lines
37 KiB
Markdown
# AGENTS.md — AI collaboration (tcg-vault)
|
||
|
||
Guidance for agents and humans working in this repo. Prefer existing patterns over new abstractions.
|
||
|
||
> Branding note: this product is **Deck Hearth** as of 2026-05-24
|
||
> (`pick-a-name` convoy, squash commit `9abbab6`, PR #21). The repo
|
||
> and Vercel project are still named `tcg-vault` — that rename is
|
||
> tracked in the queued `rename-repo-and-vercel-project` convoy
|
||
> (auto-redirects make it low-urgency). Admin email is
|
||
> `admin@deckhearth.com`; the prior `admin@tcgvault.com` literal is
|
||
> deliberately preserved in `test/lib/permission-middleware.test.js`
|
||
> as a historical regression-lock per Risk 4 of the pick-a-name
|
||
> convoy.
|
||
|
||
## Product vocabulary
|
||
|
||
User-facing copy distinguishes **ownership** (everything you own) from **curated lists** (binders/subsets). Import labels from `lib/collection-vocabulary.js` (`VOCAB`, `collectionDisplayName`) rather than hardcoding strings.
|
||
|
||
| Concept | UI label | Route / schema | Notes |
|
||
| --- | --- | --- | --- |
|
||
| Global ownership | **My Collection** | `/my-cards`, `user_cards` | Scanner default destination; replaces "Owned" / "Mark Owned" |
|
||
| Curated list / binder | **List** / **Lists** | `/collections`, `/collection/[id]`, `collections` table | URL slug unchanged in v1; nav says "Lists" |
|
||
| Auto-sync system list | **Synced binder** (display) | `collections` row with `is_system_collection = true` | DB name stays `'All My Cards'` — never render; use `collectionDisplayName()` |
|
||
| Add ownership | **Add to My Collection** | `POST /api/user-cards`, scanner bulk | Replaces "Mark Owned" / "Mark as Owned" |
|
||
| Add to curated list | **Add to List** | `POST /api/collections/:id/cards` | Replaces "Add to Collection" in scanner/card flows |
|
||
|
||
CI check `Forbidden patterns (6 checks)` → Check 6/6 blocks `"Mark Owned"`, `"Owned Cards"`, and `"All My Cards"` in `pages/` + `components/` (API literals exempt). [Formerly the standalone `forbidden-stale-strings` job; merged into `forbidden-patterns` by the `slash-ci-minutes` convoy on 2026-06-04.]
|
||
|
||
## Visual language
|
||
|
||
Deck Hearth's visual direction is **Liquid Glass** (in-progress as of
|
||
2026-06-03 — see `.convoys/liquid-glass-redesign.md` umbrella). Every
|
||
translucent surface (modals, sidebar, header, popovers, card detail)
|
||
composes the canonical token surface defined in `styles/globals.css`
|
||
and documented in [`docs/DESIGN_TOKENS.md`](docs/DESIGN_TOKENS.md).
|
||
**Do not** hardcode hex in `.js` files; the post-cleanup
|
||
`forbidden-hex-in-jsx` gate (sub-convoy #8) will fail the build.
|
||
|
||
Three rules of thumb:
|
||
|
||
- **Surfaces are glass.** Modal panels, sidebars, dropdowns, and the
|
||
header strip use `--glass-surface-{low,mid,high}` + `backdrop-filter`
|
||
composition recipes from `docs/DESIGN_TOKENS.md` § "Composite recipes".
|
||
- **Brand warmth is accent, not panel fill.** Ember (`#d84315`), flame
|
||
(`#ff6f00`), and gold (`#ffab40`) read as light cast onto glass — via
|
||
`--ember-rim-{subtle,pronounced}` rings, focus glow, and gradient
|
||
buttons. They are **NOT** the canonical panel-background color.
|
||
- **No `backdrop-filter` on card grid items.** GPU budget — glass goes
|
||
on grid containers and detail views, not per-card. See
|
||
`docs/DESIGN_TOKENS.md` § "Per-card grid performance budget".
|
||
|
||
## 1. Project overview
|
||
|
||
A web app for managing trading-card-game collections (Magic, Pokémon, Lorcana). Users authenticate, build collections + decks, scan physical cards via a camera+AI-OCR flow, and share publicly. Admin users curate the card database.
|
||
|
||
- **Framework:** Next.js 16 (Pages router) + React 18, JavaScript (not TypeScript — see Gotcha #9)
|
||
- **Data:** Neon Postgres. The runtime auth surface uses `@vercel/postgres` tagged-template SQL exclusively post-`single-sql-client` (PR #30, `c403ea4`; `lib/database.js` deleted). 11 `scripts/**` helpers (`setup-neon-db.js`, `reset-db.js`, `migrations/2026-05-24-rename-admin-email.js`, plus 8 historical add-*/fix-*/seed-* jobs) still use `@neondatabase/serverless`'s `neon()` directly — out-of-scope per the no-go-zones rule and tracked as the queued `purge-neondatabase-serverless-fully` follow-up. Schema changes ship as `node-pg-migrate` migrations under `migrations/` at the repo root post-`migration-tool` (PR #32, `de9f334`) — see § 3 Conventions § "Schema changes" and Gotcha #6.
|
||
- **Auth:** Custom JWT (jsonwebtoken + bcryptjs), token stored in `localStorage`, sent as `Authorization: Bearer …`. No NextAuth. The secret + canonical 24h TTL come from `lib/auth-secret.js` (single source of truth; throws at module load if `JWT_SECRET` is unset). `getUserFromRequest` returns `null` for unauthenticated requests — no synthetic admin fallback — and login + register are rate-limited (5 attempts / 15 min via `@upstash/ratelimit`). The seed admin row is created at `admin@deckhearth.com` with a password supplied via the required `ADMIN_INITIAL_PASSWORD` env var (`scripts/setup-neon-db.js` exits with code 1 before touching the DB if the var is unset); no credential ships in the source tree. Operators of envs that pre-date the `drop-public-setup` convoy still have the old `admin123` hash in their DB — rotate manually via the app (see Gotcha #4).
|
||
- **UI:** Tailwind CSS + custom CSS variables for theming (light/dark via `lib/theme-context.js`)
|
||
- **Hosting:** Vercel (`vercel.json`, `.vercel/` present)
|
||
|
||
## 2. Architecture quick reference
|
||
|
||
| Area | Path | Notes |
|
||
| --- | --- | --- |
|
||
| Pages router views | `pages/*.js` | Public + auth views; uses `components/Layout.js` |
|
||
| API routes | `pages/api/**/*.js` | Express-style `handler(req, res)`. **30+ handlers depend on `lib/permission-middleware.js::getUserFromRequest`** |
|
||
| Shared UI | `components/*.js` | `Layout`, `CardItem`, `CameraScanner`, modal family |
|
||
| Auth + DB libs | `lib/*.js` | `use-auth` (canonical client hook — sole surface post-`single-auth-provider`, PR #31, `0668b0c`), `auth-secret` (single JWT_SECRET + TTL source), `permission-middleware` (server-side `getUserFromRequest` + `withCollectionPermission`), `rate-limit` (5 named limiters — see Gotcha #12). The legacy `lib/database.js` was deleted by `single-sql-client` (PR #30, `c403ea4`); DB access now goes through `@vercel/postgres` tagged templates directly. |
|
||
| Migration scripts | `scripts/*.js` | 27+ one-off "add column" / "seed" scripts. No formal migration tool |
|
||
| Card-import jobs | `pages/api/cards/import-*.js`, `scripts/import-*.js` | Scryfall / Lorcana / Pokémon TCG APIs |
|
||
| Database schema | `scripts/setup-neon-db.js` | Bootstrap SQL DDL — the source of truth until a real migration tool lands |
|
||
| Schema map | `docs/SCHEMA_MAP.md` | Hand-curated; regenerate after schema changes |
|
||
|
||
Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 5602 edges). Ask: *"what calls `getUserFromRequest`?"* before refactoring auth.
|
||
|
||
## 3. Key conventions
|
||
|
||
- **Auth (server):** `import { getUserFromRequest } from '../../lib/permission-middleware'` → returns `{ userId, email, role }` or `null`. `null` means "send 401" — always early-return when the user is null before doing any work that depends on their identity.
|
||
- **Auth (client):** `import { useAuth } from '../lib/use-auth'` is the only client auth surface. Returns `{ user, loading, logout, refreshAuth }`; `user === null` means logged out, `loading === true` means token verification in flight. There is no client-side admin hook — compute `const isAdmin = user?.role === 'admin'` from the same `useAuth()` call. The legacy `lib/auth-context.js` + `lib/admin-auth.js` were deleted by `single-auth-provider` (PR #31, `0668b0c`); do not reintroduce a `<AuthProvider>` / `<AdminProvider>` wrapper in `pages/_app.js`. The login + signup flow uses direct `fetch('/api/auth/{login,register}')` from `pages/login.js` / `pages/signup.js` — there is no `useAuth().login(...)` / `useAuth().register(...)` method; do not add one.
|
||
- **Layout `user` prop:** pages should pass `user` from `useAuth()` to `<Layout>`. Layout's default is `null` and renders a logged-out "Sign in" CTA when no user is supplied — both paths are valid (some surfaces like `pages/invite/{accept,decline}.js` legitimately render Layout for anonymous visitors). Do not reintroduce a hardcoded user object as a default prop.
|
||
- **JWT secret + TTL:** `import { JWT_SECRET, JWT_TOKEN_TTL } from '../../lib/auth-secret.js'`. This is the only place either value is defined; do not reintroduce literal fallbacks. `JWT_TOKEN_TTL = '24h'` is canonical.
|
||
- **Auth helper (token mint / verify / password hash):** `import { ... } from '../../pages/api/auth-utils'` (`generateToken`, `verifyToken`, `hashPassword`, `verifyPassword`). Reads the secret + TTL from `lib/auth-secret.js` under the hood.
|
||
- **Rate limiting:** `import { checkAuthRateLimit } from '../../lib/rate-limit.js'` for any new auth-surface endpoint (`/api/auth/login` + `/api/auth/register` already wired). Returns `{ allowed, remaining, reset }`; on `!allowed` return 429 with a `Retry-After` header. See `.cursor/rules/api-routes.mdc` § "Rate limiting" for the verbatim shape.
|
||
- **Permission gate for collection routes:** wrap handlers with `withCollectionPermission('viewer' | 'editor' | 'owner')` from `lib/permission-middleware.js`.
|
||
- **DB access:** Use **tagged-template** style — `import { sql } from '@vercel/postgres'`. The legacy `lib/database.js` (`db.query(string, params)` wrapper around `@neondatabase/serverless`, which interpolated params into a string and called `sql.unsafe`) was deleted by `single-sql-client` (PR #30, `c403ea4`); do NOT reintroduce that shape. For `scripts/**` helpers that legitimately need the Neon HTTP driver (e.g. `setup-neon-db.js`, `reset-db.js`), import `{ neon } from '@neondatabase/serverless'` directly and use tagged-template SQL (`await sql\`...\``) — the safe shape, not the wrapper's unsafe shape.
|
||
- **Activity logging:** `logCollectionActivity(collectionId, userId, action, details)` — call it from any handler that mutates a collection.
|
||
- **File names:** `kebab-case.js` for libs/scripts; `PascalCase.js` for React components.
|
||
- **Imports:** No path aliases configured; use relative imports.
|
||
- **Slugs:** `lib/slug-utils.js::generateUniqueSlug` for any user-facing identifier (collections, decks).
|
||
- **CSS theme tokens:** Components read `var(--bg-primary)`, `var(--text-primary)`, `var(--accent-ember)`, etc. — defined in `styles/`. Don't hardcode hex colors.
|
||
- **Schema changes (post-`migration-tool`):** new column / constraint / table work ships as a `node-pg-migrate` migration under `migrations/` at the repo root. Generate via `npm run migrate create <name> -- -j js`, edit the `up()` (and `down()` when rollback is safe — for any migration that touches data, prefer a hard-stub `down()` that throws), and apply locally with `npm run migrate up`. `npm run setup-db` now chains `npm run migrate up` then seeds the admin user. The legacy `scripts/add-*.js` / `scripts/fix-*.js` / `scripts/seed-*.js` jobs are append-only history (no-go-zones rule); do NOT add new ones. See `.convoys/migration-tool.md` for the full architect-decision record and Gotcha #6 below for the historical context.
|
||
|
||
## 4. Common gotchas
|
||
|
||
- **#1 — Two SQL clients live in parallel. RESOLVED** by `single-sql-client` convoy (PR #30, squash commit `c403ea4`, 2026-05-26). `lib/database.js` is deleted; the 2 callers (`pages/api/auth-utils.js` source + `test/api/auth-utils.test.js` mock) migrated to `@vercel/postgres` tagged templates (byte-equivalent SQL semantics for the two single-parameter SELECT queries). The convoy's architect audit (D4) confirmed no current call site actually exercised the `sql.unsafe` injection vector — the `userId` callers passed a numeric SERIAL from a verified JWT — so this was foot-gun removal rather than a live security finding. **`@neondatabase/serverless` is still in `package.json` as a runtime dep** because 11 `scripts/*` helpers continue to use `neon()` directly (`setup-neon-db.js`, `migrations/2026-05-24-rename-admin-email.js`, `reset-db.js`, plus 8 historical `add-*` / `fix-*` / `seed-*` jobs). Those scripts use the safe tagged-template shape (`await sql\`...\``), not the deleted wrapper's unsafe `db.query(string, params)` shape. Full dep purge is tracked as the queued `purge-neondatabase-serverless-fully` follow-up (now unblocked by `migration-tool` PR #32 — the migration helpers all use `node-pg-migrate`'s `pg` client, not `@neondatabase/serverless`, so the only remaining direct `neon()` consumers are `setup-neon-db.js` (admin seed), `reset-db.js`, and the historical graveyard). Entry kept (not renumbered) to preserve cross-references.
|
||
- **#2 — `getUserFromRequest` synthetic-admin fallback. RESOLVED** by `fix-auth-bypass` Brief 2 (commit `258e479`). The helper now returns `null` for unauthenticated requests; `pages/api/auth/verify.js` returns 401 on the no-token branch. The 16 unit tests in `test/lib/permission-middleware.test.js` lock in the contract, including a negative regression against the old synthetic-admin shape. Entry kept (not renumbered) to preserve the audit trail and stable cross-references.
|
||
- **#3 — JWT_SECRET hardcoded across 7 files. RESOLVED** by `fix-auth-bypass` Brief 1 (commit `4a10dce`). `lib/auth-secret.js` is now the single source of truth and throws at module load when `JWT_SECRET` is unset. Canonical TTL is `JWT_TOKEN_TTL = '24h'`. The `'your-secret-key-change-in-production'` literal is gone from all 7 sites; CI lint passes against the post-fix tree. Entry kept (not renumbered) to preserve cross-references.
|
||
- **#4 — Default admin credentials in the seed. RESOLVED** by `drop-public-setup` Brief 1 (commit `ff80753`) + Brief 2 (commit `b63b509`). `scripts/setup-neon-db.js` no longer hardcodes `admin123`; it reads `ADMIN_INITIAL_PASSWORD` from the environment and exits with code 1 before opening a DB connection if the var is unset. README's "Default Admin Account" section is replaced with "First-time admin setup" copy that documents the env var, `openssl rand -base64 24` generation tip, and CI-secret alternative. Brief 2 converted the script from CJS to ESM so `npm run setup-db` actually runs on Node 22.x (the `bump-next-js` convoy's `"type": "module"` flag had silently broken it). **Operator caveat:** the seed is idempotent (`ON CONFLICT (email) DO NOTHING`); re-running setup-db on an env that already has the admin row does NOT rotate the password. Any deployed env that ran setup before this convoy still has the weak `admin123` hash — operators must rotate manually via the app, or wait for the queued `rotate-default-admin` follow-up convoy. Entry kept (not renumbered) to preserve cross-references.
|
||
|
||
Post-`pick-a-name` (2026-05-24, squash `9abbab6`), the seeded admin
|
||
email is `admin@deckhearth.com` (and alice/bob test users likewise
|
||
renamed). **If you are deploying past `9abbab6` and the prod Neon
|
||
DB still has `@tcgvault.com` rows, you MUST run
|
||
`node scripts/migrations/2026-05-24-rename-admin-email.js` BEFORE
|
||
the next admin login attempt** or it 401s. The migration is ESM,
|
||
idempotent, UNIQUE-collision-safe (fails loud if `setup-neon-db.js`
|
||
already ran post-rename — which would indicate an ordering error).
|
||
Order: migration FIRST, then any subsequent `npm run setup-db`.
|
||
- **#5 — `pages/api/setup-database.js` public endpoint. RESOLVED** by `fix-auth-bypass` Brief 3 (commit `fc0dd73`). The file is deleted along with the other three dev endpoints (`/api/simple`, `/api/test-auth`, `/api/test-db`), and `.github/workflows/ci.yml`'s `forbidden-patterns` job (formerly the standalone `forbidden-endpoints` job; consolidated by `slash-ci-minutes` convoy on 2026-06-04) fails the build if any of them are re-introduced (or if a new `pages/api/test-*.js` file appears). Entry kept (not renumbered) to preserve cross-references.
|
||
- **#6 — Migrations were bare scripts. RESOLVED** by `migration-tool` convoy (2026-05-26). `node-pg-migrate@^8` is the chosen tool (lightweight, raw-SQL-friendly, zero TS surface — matches the repo's JavaScript-only `@vercel/postgres` style). New migrations live under `migrations/` at the repo root and use the default `pgmigrations` tracking table. The initial backfill `migrations/1779853647564_initial-schema.js` reproduces `scripts/setup-neon-db.js`'s 7-table DDL verbatim using `CREATE TABLE IF NOT EXISTS`, so it's idempotent against fresh AND pre-existing envs — first-time `npm run migrate up` on an env that already ran `setup-neon-db.js` pre-convoy is a no-op DDL-wise (only records the `pgmigrations` row). The legacy 27 `scripts/add-*.js` / `scripts/fix-*.js` / `scripts/seed-*.js` jobs are append-only history per the no-go-zones rule — do NOT add new ones. New column / constraint work ships as a `node-pg-migrate` migration. See § 3 Conventions § "Schema changes" above + `.convoys/migration-tool.md`. Entry kept (not renumbered) to preserve cross-references.
|
||
- **#7 — Dual `is_public` semantics.** Collections and decks both have `is_public` columns; check which controls discovery vs. anonymous read in the relevant route.
|
||
- **#8 — Layout has hardcoded default user. RESOLVED** by `fix-layout-default-user` convoy (PR #15, squash commit `ca302a8`). `components/Layout.js`'s default prop is now `null`; `UserProfileDropdown` renders a `<Link href="/login">Sign in</Link>` CTA when `user === null`. Brief 2 also swept the 7 pages that needed page-level fixes (`scanner` / `decks` / `deck-builder` / `deck/[id]` now pass `user={user}` to Layout; `profile` / `settings` replaced leaky `useState({email:'me@…'})` with `useState(null)` + null-guards on every sync `user.*` read; `card/[id]` swapped a hardcoded `const user = {...}` for `useAuth()` from `lib/use-auth.js`). `test/components/Layout.test.js` adds 5 regression-lock assertions (no maintainer email when user is null/omitted; "Sign in" link present; supplied email renders; no "Guest" placeholder); vitest 21/21 green at merge. New devDeps: `jsdom@^29` + `@testing-library/react@^16`. See `.convoys/fix-layout-default-user.md` and `.convoys/ship-readiness.md` P0 #7. Entry kept (not renumbered) to preserve cross-references.
|
||
- **#9 — `typescript` is a devDep, but the source is still JavaScript-only.** `package.json` lists `typescript@^5.9.3` purely so `eslint-config-next@16`'s bundled `typescript-eslint` chain can satisfy its hard `require('typescript')` at module load (the `peerDependenciesMeta.typescript.optional: true` flag in `eslint-config-next` only suppresses npm's install-time warning, not the runtime require). There is no `tsconfig.json`, no `.ts`/`.tsx` files, and no `// @ts-check` directives. Do not rename `.js` files to `.ts` or add a `tsconfig.json` without an explicit convoy decision — TypeScript adoption is its own scope. See `.convoys/bump-next-js.md` § Decisions C.
|
||
- **#10 — ESLint pinned to v9 (maintenance), not v10 (latest).** `devDependencies.eslint` is `^9.39.4` even though `latest` is `10.4.0`. We tried v10 and `npm run lint` crashed with `TypeError: scopeManager.addGlobals is not a function` because `eslint-config-next@16`'s bundled `typescript-eslint@8.x` predates ESLint v10's redesigned global-ingestion path. Reverted to v9 under Decision D. **Do NOT bump ESLint independently** — wait for the queued `bump-eslint-10` follow-up convoy, which is upstream-blocked until `typescript-eslint` ships a v10-tested release that `eslint-config-next` bundles. See `.convoys/bump-next-js.md` § Decisions D + "Follow-up convoys queued".
|
||
- **#11 — Turbopack is now the default bundler.** `next dev` and `next build` use Turbopack by default in Next.js 16. The fallback per command is `--webpack` (e.g. `next build --webpack`). We have no custom `webpack:` block in `next.config.js`, no custom loaders/aliases, and no Sass tilde imports, so Turbopack should "just work" — but if a build/runtime regression appears, reproduce on both bundlers before deciding whether to revert or pin a script to webpack. Do not pre-emptively switch to `--webpack`.
|
||
- **#12 — Rate-limit env vars are `KV_REST_API_URL` / `KV_REST_API_TOKEN`, not `UPSTASH_REDIS_REST_*`.** `lib/rate-limit.js` reads the Vercel Upstash Marketplace integration's auto-provisioned names. Three other Upstash-shaped vars exist in the Vercel-managed env (`KV_URL`, `REDIS_URL`, `KV_REST_API_READ_ONLY_TOKEN`) but our `@upstash/redis` REST client does not use them — do not wire to them. In prod, the rate-limit module **fails closed** if either of the two REST vars is missing (a single failed login is a better outcome than silently disabling brute-force protection). In dev / test, it warn-and-continues as a no-op so local work is unaffected when Upstash isn't wired up.
|
||
|
||
**Milestone — `add-rate-limiting` convoy (squash `708ef45`, PR #20, 2026-05-24) closed P0 #6 — all 8 P0s now RESOLVED.** The lib refactored from a single auth-only limiter to **5 named limiters** with a `Map<className, Ratelimit>` cache (one shared Redis client, five `Ratelimit` instances, distinct Redis prefix per class). The five exports + their use cases:
|
||
|
||
| Helper | Class | Limit/window | Key | Redis prefix | Routes |
|
||
| --- | --- | --- | --- | --- | --- |
|
||
| `checkAuthRateLimit(req)` | `auth` | 5 / 15 min | IP | `deckhearth:auth` | `/api/auth/login`, `/api/auth/register` (Brief 4 contract; byte-identical return shape preserved) |
|
||
| `checkSearchRateLimit(req)` | `search` | 60 / 1 min | IP | `deckhearth:search` | `/api/users/search`, `/api/cards/search` |
|
||
| `checkUploadRateLimit(req, userId)` | `upload` | 10 / 1 hour | user | `deckhearth:upload` | `/api/user/avatar` |
|
||
| `checkGenerateRateLimit(req, userId)` | `generate` | 5 / 1 hour | user | `deckhearth:generate` | `/api/user/avatar/generate` |
|
||
| `checkImportRateLimit(req, userId)` | `import` | 5 / 1 hour | user (admin-only) | `deckhearth:import` | `/api/cards/import-mtg`, `/api/cards/import-pokemon` |
|
||
|
||
Prefixes renamed `tcgvault:*` → `deckhearth:*` in `pick-a-name` (squash `9abbab6`, 2026-05-24); accepted one-time per-15-min / per-1-hour counter reset; existing Upstash state at `tcgvault:*` keys is now stale and will TTL out naturally.
|
||
|
||
All five return the same `{ allowed, remaining, reset }` shape; on `!allowed`, set `Retry-After: Math.ceil((reset - Date.now()) / 1000)` and return 429 with the uniform message `'Too many attempts. Try again later.'` (per-class variation would fingerprint the limits to an attacker — explicitly rejected).
|
||
|
||
**Defensive THROW pattern.** `extractUserIdentifier(userId)` THROWS with a named error when `userId` is `null` / `undefined` / `''` / `NaN`. Surfaces gate-ordering bugs at dev time rather than silently falling back to IP and converting a per-user limit into a per-IP limit (which would lock household members out for one user's behavior). Numeric `0` is intentionally accepted (returns `'user:0'`) for forward-compat. **Gate-ordering rule: per-user rate-limit gates (`upload`, `generate`, `import`) MUST sit AFTER the auth check.** For the two `/api/cards/import-*` routes, the ordering is also `auth → admin-role check (403 if not admin) → rate-limit`; the admin-role check sits between auth and rate-limit. IP-keyed gates (`auth`, `search`) can sit anywhere after the method check.
|
||
|
||
Adding a sixth class is a one-line `LIMITER_CONFIG` addition + one new exported function (no `init()` restructuring needed). Tuning an existing class is a one-line `LIMITER_CONFIG` edit. The full verbatim call shape + gate-ordering rules + identifier-extraction documentation live in `.cursor/rules/api-routes.mdc` § Rate limiting.
|
||
|
||
## 5. Running locally
|
||
|
||
- **Runtime:** Node 20 (Vercel default).
|
||
- **Setup:** `npm install`, copy `.env.local` template (POSTGRES_URL + JWT_SECRET + RESEND_API_KEY + BLOB_READ_WRITE_TOKEN + ADMIN_INITIAL_PASSWORD — the last is required for `npm run setup-db` and the script exits with code 1 if it's unset; optionally KV_REST_API_URL + KV_REST_API_TOKEN to exercise the rate limiter locally — without them, `lib/rate-limit.js` warn-and-no-ops in dev), then `npm run setup-db` once.
|
||
- **Dev server:** `npm run dev` → http://localhost:3000.
|
||
|
||
## 6. Testing
|
||
|
||
- **Unit-test runner:** `vitest@^3.2.4` (installed via `fix-auth-bypass` Brief 5, commit `1629afb`). `npm test` for watch mode; `npm run test:run` for the CI / single-shot mode. Config in `vitest.config.js`, setup in `test/setup.js` (sets `JWT_SECRET` + `NODE_ENV=test` before any module loads). Specs live under `test/` mirroring source layout (`test/lib/*.test.js`, `test/api/*.test.js`, `test/components/*.test.js`). Last green: 21/21 tests pass.
|
||
- **Vitest coverage today:** 21 unit tests — `lib/auth-secret.js` (3), `lib/permission-middleware.js::getUserFromRequest` (8, incl. a negative regression against the old synthetic-admin shape — Gotcha #2), `pages/api/auth-utils.js` (5), and `components/Layout.js` (5 regression-lock assertions for the post-PR-#15 logged-out branch — Gotcha #8). These tests lock in the contracts established by `fix-auth-bypass` Briefs 1 + 2 and `fix-layout-default-user`; do not weaken them when refactoring auth or Layout.
|
||
- **E2E / smoke runner:** `@playwright/test@^1.60.0` (installed via `adopt-playwright-smoke`, PR #18 squash `7b6f751`). Config in `playwright.config.js` (root, ESM) declares two projects:
|
||
- `smoke` — `tests/smoke/**/*.spec.@(ts|js)`; invoked by `.github/workflows/preview-smoke.yml`. `npm run test:smoke` locally.
|
||
- `visual` — `tests/visual/**/*.spec.@(ts|js)`; invoked by `.github/workflows/visual-diff.yml`. `npm run test:visual` locally; `npm run test:visual:update` to (re-)seed baselines.
|
||
|
||
Local-run convention: boot `next dev` in one terminal, then in another run `BASE_URL=http://localhost:3000 npm run test:smoke` (or against a deployed preview, `BASE_URL=https://<preview>.vercel.app VERCEL_AUTOMATION_BYPASS_SECRET=<value> npm run test:smoke`). No `next dev` auto-boot in the test scripts (Decision 6 of `adopt-playwright-smoke`).
|
||
- **Browsers must be installed once locally:** `npx playwright install --with-deps chromium`. CI re-runs this on every workflow run (it's cached when possible).
|
||
- **Visual baselines:** none committed yet. `tests/visual/__screenshots__/` is intentionally absent and intentionally NOT in `.gitignore` (baselines, when they exist, must be committed). First-run baseline generation MUST happen in a Linux environment so the PNG matches what CI produces. Recommended path is the Playwright Docker image:
|
||
|
||
```
|
||
docker run --rm -v "$PWD":/work -w /work \
|
||
mcr.microsoft.com/playwright:v1.60.0-noble \
|
||
sh -c "npm ci && BASE_URL=<preview-url> \
|
||
VERCEL_AUTOMATION_BYPASS_SECRET=<value> \
|
||
npm run test:visual:update"
|
||
```
|
||
|
||
Mac-generated baselines will NOT match Linux CI — `playwright.config.js`'s custom `snapshotPathTemplate` has no `{platform}` token, so a Mac update silently overwrites the canonical Linux baseline. Tracked as the queued `seed-visual-baselines-on-linux` convoy (see `.convoys/ship-readiness.md` § Queued convoys).
|
||
- **CI behavior:**
|
||
- **Vitest:** the `test:` job in `.github/workflows/ci.yml` runs `npm run test:run` on every PR and push to `main` and is **blocking** (no `|| true`, no `continue-on-error`). A red test job blocks merge.
|
||
- **Playwright smoke:** runs on every PR via `preview-smoke.yml`. Gate skip via `pipeline: skip smoke` in the PR body (handled in the `gate:` job's Decide step via env-var routing — see § 7's shell-injection note). Pre-migration runtime: 59s end-to-end on `ubuntu-latest` (PR #18 post-merge run). Post-migration on the axiom pool: cold-cache first run ~6 min (Chromium download); warm cache thereafter ~1–2 min.
|
||
- **Screenshot diff:** runs only on PRs touching `pages/**` / `components/**` / `styles/**` / `tailwind.config.js` / `postcss.config.js` via `visual-diff.yml`. **First `Screenshot diff` run after `adopt-playwright-smoke` will fail at the test step** because no baseline exists yet; `continue-on-error: true` swallows the failure and the comment-on-PR step posts "Visual Diff — view run" with empty artifacts. That is the documented Decision-4 end state of `adopt-playwright-smoke`, not a regression — it stays that way until `seed-visual-baselines-on-linux` lands.
|
||
- **CI minute optimizations (slash-ci-minutes convoy, 2026-06-04):**
|
||
- **Doc-only PRs skip ALL of ci.yml + preview-smoke.yml.** Both workflows carry `paths-ignore` for `.convoys/**`, `**/*.md`, `docs/**`, `AGENTS.md`, `.cursor/**`, and `README.md`. A pure-docs PR triggers zero GitHub Actions jobs (Vercel still builds — it's not on the Actions billing). `visual-diff.yml` was already cost-conscious via a positive `paths:` allowlist and is unchanged.
|
||
- **The 6 grep-only forbidden-* jobs collapsed into one.** They previously ran as 6 independent jobs (each with its own `actions/checkout`); the consolidated `forbidden-patterns` job runs all 6 checks as labeled `::group::` sections in a single bash step, with a FAIL flag at the bottom so every violation across all 6 checks still surfaces in one run (same diagnostic behavior, ~5/6 of the per-PR checkout overhead removed). The 6 original job names (`forbidden-endpoints`, `forbidden-cors-headers`, `forbidden-client-side-llm-keys`, `forbidden-modal-shell-without-primitive`, `forbidden-deprecated-color-aliases`, `forbidden-stale-strings`) no longer appear in the checks list — references in this file (e.g. CI job `forbidden-stale-strings` blocks ...) are now informational, not check-name lookups. `pr-health-rollup.yml` was unaffected because it only looks up `Lint` and `Schema map up to date` by name.
|
||
- **`node_modules` cached between runs** in lint / test / migrate / preview-smoke / visual-diff. Keyed on `package-lock.json` hash so any dep change invalidates correctly. Cuts `npm ci` from ~30-45s to ~3-5s on cache hit. `actions/setup-node@v4`'s built-in `cache: npm` is layered above this (caches `~/.npm`) — both stay because the `setup-node` cache helps on cache-miss days too.
|
||
- **Playwright browsers cached** in `preview-smoke.yml` + `visual-diff.yml`. Keyed on the resolved `@playwright/test` version from `package-lock.json`. Cache invalidates automatically on any Playwright version bump. On cache hit, only the system deps install (`npx playwright install-deps chromium`) runs — saves ~15-25s/run.
|
||
- **Self-hosted runner pool (migrate-ci-to-self-hosted convoy, 2026-06-05):**
|
||
- **4 of 5 workflows run on the axiom homelab.** `ci.yml`, `preview-smoke.yml`, `visual-diff.yml`, `pr-health-rollup.yml` use `runs-on: [self-hosted, axiom]` and execute on CT 111 in the `axiom-server` Proxmox homelab (`axiom-runner-1..4`, registered org-scoped to `stwl-labs`, ephemeral one-job-per-container via `myoung34/github-runner`). Net effect: tcg-vault CI no longer consumes GitHub Actions minutes for those four workflows.
|
||
- **`agent-context-drift.yml` deliberately stays on `ubuntu-latest`** per Decision D4 of the convoy — it's a weekly cron, costs ~2 min/month, and must run even when axiom is down for maintenance. **Check 8 of `forbidden-patterns` enforces this** as a strict allowlist (anything else reintroducing `runs-on: ubuntu-latest` fails CI).
|
||
- **Cache mounts live on the CT 111 host** and are bind-mounted into every runner container, so they survive across the ephemeral-runner lifecycle and are shared across `axiom-runner-1..4`. Paths (on CT 111): `/opt/appdata/gha-runner/shared-cache/{npm,pnpm,yarn,pip,playwright,buildx}` and the per-runner workdirs under `/opt/appdata/gha-runner/runner-N/`. The `actions/cache@v4` keys above still apply on top — the bind mounts just keep the underlying tooling caches (`~/.npm`, `~/.cache/ms-playwright`) primed across jobs.
|
||
- **Migrate job uses CT 102 shared Postgres** instead of an in-runner `services.postgres` container. `HOMELAB_CI_POSTGRES_PASSWORD` repo secret (password only — `PGHOST`/`PGUSER`/`PGPORT` are hardcoded in `ci.yml`). Each run creates a per-run database named `ci_run_${run_id}_${run_attempt}` and drops it in an `if: always()` cleanup step so failed migrations don't leak DBs. The `deckhearth_ci` role has `CREATEDB` but no superuser; a compromised runner can't reach other apps' databases on CT 102.
|
||
- **Cross-references:** convoy decisions + risks in [`.convoys/migrate-ci-to-self-hosted.md`](.convoys/migrate-ci-to-self-hosted.md); homelab-side infra in `axiom-server/proxmox/ct111/README.md`; revert path in § 7 below.
|
||
- **Manual QA:** `TESTING_GUIDE.md` still applies for flows not yet covered by automated tests (scanner camera path, card-import jobs, multi-step UI wizards). The automated smoke + visual suite is steadily eclipsing it; `TESTING_GUIDE.md` will be renamed to `docs/MANUAL_QA.md` and trimmed to truly-manual-only flows in a future cleanup convoy (see `.convoys/ship-readiness.md` § Role-doc-writer findings).
|
||
|
||
## 7. Deployment
|
||
|
||
- **Vercel** auto-deploys `main` and creates Preview deployments for every PR. `vercel.json` and `.vercel/` are committed. CI in `.github/workflows/` runs lint + types (no duplicate build — Vercel handles it).
|
||
- **Preview protection bypass for automation.** The project has a Protection Bypass for Automation token exposed locally as `VERCEL_AUTOMATION_BYPASS_SECRET` in `.env.local` (not committed) and seeded into GitHub Actions as a repo secret (`gh secret set VERCEL_AUTOMATION_BYPASS_SECRET`, 2026-05-24). The secret is consumed in two shapes:
|
||
1. **Query parameter** on `wait-for-vercel-preview@v1.3.2`'s `path:` input in both `preview-smoke.yml` and `visual-diff.yml` — `path: '/?x-vercel-protection-bypass=…'`, bare form, **without** `&x-vercel-set-bypass-cookie=true` (the cookie variant returns 307 + Set-Cookie and axios in Node has no cookie jar, so it 401s on the redirect). Plumbed by PR #17 (`fix-vercel-deployment-protection-in-ci`, squash `9a3e077`).
|
||
2. **HTTP header** in `playwright.config.js`'s `use.extraHTTPHeaders` — `'x-vercel-protection-bypass': <secret>`. Playwright's browser context has a real cookie jar so this shape works there, and the testOptions surface forwards the header to the test-level `request` fixture's `APIRequestContext` as well, so both `page.goto(...)` calls and `request.get('/api/health')` calls hit the protected preview correctly without per-spec header injection. Plumbed by PR #18 (`adopt-playwright-smoke`, squash `7b6f751`) per Decision 2 of that convoy.
|
||
|
||
Decision 2 also wires a **fail-loud-in-CI / warn-in-dev** predicate: `if (process.env.CI === 'true' && !process.env.VERCEL_AUTOMATION_BYPASS_SECRET) throw ...` (with an error message that names the env var, the `gh secret set` rotation command, and points at this section); otherwise `console.warn` once and continue with `extraHTTPHeaders` undefined. Same fail-closed / warn-and-no-op shape as `lib/rate-limit.js`'s Upstash predicate — see Gotcha #12.
|
||
|
||
Do not log or echo the value. If the operator rotates the token in the Vercel dashboard, re-seed the GitHub secret via `gh secret set VERCEL_AUTOMATION_BYPASS_SECRET --body "<new value>"`. See `.convoys/fix-vercel-deployment-protection-in-ci.md` and `.convoys/adopt-playwright-smoke.md`.
|
||
- **Shell-injection hardening in workflow YAML.** Never inline `${{ github.event.* }}` directly into a `run:` block — route the value through the step's `env:` block and quote it (`"$VAR_NAME"`) in shell. PR #17's CI validation caught a real syntax error from a PR body containing `(` because the gate-job's Decide step inlined `${{ github.event.pull_request.body }}` straight into bash; commit `b6f8688` swept both `preview-smoke.yml` and `visual-diff.yml` to the `env:` + quoted-shell pattern. This is GitHub's official Security Hardening guidance ("Security hardening for GitHub Actions" → "Using a third-party action"). Apply to any new workflow that reads PR body / title / branch name / commit messages in shell.
|
||
- **CI runs on the axiom homelab (CT 111).** Four of the five workflows execute on `[self-hosted, axiom]` runners managed in the `axiom-server` repo (`proxmox/ct111/`). Day-to-day this is invisible — pushes still trigger jobs and Vercel still builds previews — but two operational notes matter:
|
||
1. **PAT rotation.** The runners authenticate to GitHub via an org-scoped PAT stored on CT 111 at `/opt/appdata/gha-runner/.env` (key `GH_PAT`, scopes `admin:org`, `repo`, `workflow`). Rotate every 90 days. After updating the value on CT 111, run `./proxmox/scripts/sync.sh restart 111` to re-register all 4 runners. If the PAT lapses silently, new jobs fail registration immediately; check `./proxmox/scripts/sync.sh logs 111 gha-runner-1` for `Http response code: NotFound` to confirm.
|
||
2. **1-line revert path (D5) — when axiom is offline mid-PR-storm.** If CT 111 is down for maintenance, hardware swap, or any reason, and a hot fix needs CI to land, swap every `[self-hosted, axiom]` back to `ubuntu-latest`:
|
||
|
||
```
|
||
sed -i '' 's/\[self-hosted, axiom\]/ubuntu-latest/g' .github/workflows/*.yml
|
||
# macOS sed needs the empty -i '' argument; on Linux it's `sed -i 's/...//g' ...`.
|
||
```
|
||
|
||
This re-bills GitHub Actions minutes for the duration of the outage. Commit the change directly to `main` (or to the affected PR's branch), let CI run, and revert the sed result once axiom is back. The `forbidden-patterns` Check 8 will block the next normal PR until the revert lands — that's intentional: the gate exists exactly to surface this drift, not to silently re-bill minutes for weeks. Beszel alerts on CT 111 down (axiom-server CT 101) so you usually know before a PR notices.
|
||
|
||
## 8. Code graph
|
||
|
||
A local code-knowledge-graph MCP server (`user-code-review-graph`) is set up for this repo. Ask "what calls X?" or "show me the flow from /api/auth/login" instead of grepping. See [`docs/agent-context/README.md`](docs/agent-context/README.md).
|