docs: sweep README + AGENTS.md to post-homelab reality
- README: Dokploy/CT102 Postgres stack, lib/sql.js, six rate-limit classes over REDIS_URL, Playwright smoke+visual, MinIO/S3 vars, inline .env.local contract (no .env.example exists), deckhearth.git clone URL, project-structure refresh. - AGENTS.md: infra banner for the Vercel/Neon -> homelab move, repo rename to stwl-labs/deckhearth, Data/Auth/Hosting overview bullets, DB-access convention re-pointed at lib/sql.js, Gotcha #12 rewritten for REDIS_URL + scan limiter (+ row in limiter table), test counts refreshed (231/231 across 43 files), §5 env contract, §7 marked legacy-pending-decommission with DOKPLOY_DEPLOY runbook pointer.
This commit is contained in:
parent
95c2f6003b
commit
f2ba333daf
2 changed files with 92 additions and 53 deletions
63
AGENTS.md
63
AGENTS.md
|
|
@ -3,15 +3,23 @@
|
|||
Guidance for agents and humans working in this repo. Prefer existing patterns over new abstractions.
|
||||
|
||||
> Branding note: this product is **Deck Hearth** as of 2026-05-24
|
||||
> (`pick-a-name` convoy, squash commit `9abbab6`, PR #21). The repo
|
||||
> and Vercel project are still named `tcg-vault` — that rename is
|
||||
> tracked in the queued `rename-repo-and-vercel-project` convoy
|
||||
> (auto-redirects make it low-urgency). Admin email is
|
||||
> (`pick-a-name` convoy, squash commit `9abbab6`, PR #21). The GitHub
|
||||
> repo is now `stwl-labs/deckhearth` (renamed from `stwl-labs/tcg-vault`;
|
||||
> local checkout folders named `tcg-vault` are fine). Admin email is
|
||||
> `admin@deckhearth.com`; the prior `admin@tcgvault.com` literal is
|
||||
> deliberately preserved in `test/lib/permission-middleware.test.js`
|
||||
> as a historical regression-lock per Risk 4 of the pick-a-name
|
||||
> convoy.
|
||||
|
||||
> Infra note (2026-08): Deck Hearth is moving off Vercel + Neon onto
|
||||
> the axiom homelab — Postgres/Redis/MinIO on CT 102, app on Dokploy
|
||||
> CT 112, public URL `deckhearth.stillwell.cloud`. Runtime DB access
|
||||
> goes through `lib/sql.js` (the `postgres` package), not
|
||||
> `@vercel/postgres`; rate limiting reads `REDIS_URL`. CI already gates
|
||||
> against the homelab deployment; Neon/Vercel decommission is pending
|
||||
> (`migrate-neon-to-homelab` convoy phases 6–8). See § 5–§ 7 and
|
||||
> `docs/DOKPLOY_DEPLOY.md` / `docs/HOMELAB_DATABASE.md`.
|
||||
|
||||
## Product vocabulary
|
||||
|
||||
User-facing copy distinguishes **ownership** (everything you own) from **curated lists** (binders/subsets). Import labels from `lib/collection-vocabulary.js` (`VOCAB`, `collectionDisplayName`) rather than hardcoding strings.
|
||||
|
|
@ -54,10 +62,10 @@ Three rules of thumb:
|
|||
A web app for managing trading-card-game collections (Magic, Pokémon, Lorcana). Users authenticate, build collections + decks, scan physical cards via a camera+AI-OCR flow, and share publicly. Admin users curate the card database.
|
||||
|
||||
- **Framework:** Next.js 16 (Pages router) + React 18, JavaScript (not TypeScript — see Gotcha #9)
|
||||
- **Data:** Neon Postgres. The runtime auth surface uses `@vercel/postgres` tagged-template SQL exclusively post-`single-sql-client` (PR #30, `c403ea4`; `lib/database.js` deleted). 11 `scripts/**` helpers (`setup-neon-db.js`, `reset-db.js`, `migrations/2026-05-24-rename-admin-email.js`, plus 8 historical add-*/fix-*/seed-* jobs) still use `@neondatabase/serverless`'s `neon()` directly — out-of-scope per the no-go-zones rule and tracked as the queued `purge-neondatabase-serverless-fully` follow-up. Schema changes ship as `node-pg-migrate` migrations under `migrations/` at the repo root post-`migration-tool` (PR #32, `de9f334`) — see § 3 Conventions § "Schema changes" and Gotcha #6.
|
||||
- **Auth:** Custom JWT (jsonwebtoken + bcryptjs), token stored in `localStorage`, sent as `Authorization: Bearer …`. No NextAuth. The secret + canonical 24h TTL come from `lib/auth-secret.js` (single source of truth; throws at module load if `JWT_SECRET` is unset). `getUserFromRequest` returns `null` for unauthenticated requests — no synthetic admin fallback — and login + register are rate-limited (5 attempts / 15 min via `@upstash/ratelimit`). The seed admin row is created at `admin@deckhearth.com` with a password supplied via the required `ADMIN_INITIAL_PASSWORD` env var (`scripts/setup-neon-db.js` exits with code 1 before touching the DB if the var is unset); no credential ships in the source tree. Operators of envs that pre-date the `drop-public-setup` convoy still have the old `admin123` hash in their DB — rotate manually via the app (see Gotcha #4).
|
||||
- **Data:** Postgres 17 + pgvector on the axiom homelab (CT 102, `192.168.68.102:5432`). Runtime DB access goes through `lib/sql.js` — a tagged-template `sql` helper over the `postgres` package returning `{ rows, rowCount }` (the former `@vercel/postgres` shape, so call sites only changed their import). Migrations read `POSTGRES_URL_DIRECT`. 11 `scripts/**` helpers (`setup-neon-db.js`, `reset-db.js`, `migrations/2026-05-24-rename-admin-email.js`, plus 8 historical add-*/fix-*/seed-* jobs) still use `@neondatabase/serverless`'s `neon()` directly — out-of-scope per the no-go-zones rule and tracked as the queued `purge-neondatabase-serverless-fully` follow-up. Schema changes ship as `node-pg-migrate` migrations under `migrations/` at the repo root post-`migration-tool` (PR #32, `de9f334`) — see § 3 Conventions § "Schema changes" and Gotcha #6.
|
||||
- **Auth:** Custom JWT (jsonwebtoken + bcryptjs), token stored in `localStorage`, sent as `Authorization: Bearer …`. No NextAuth. The secret + canonical 24h TTL come from `lib/auth-secret.js` (single source of truth; throws at module load if `JWT_SECRET` is unset). `getUserFromRequest` returns `null` for unauthenticated requests — no synthetic admin fallback — and login + register are rate-limited via `lib/rate-limit.js` (see Gotcha #12). The seed admin row is created at `admin@deckhearth.com` with a password supplied via the required `ADMIN_INITIAL_PASSWORD` env var (`scripts/setup-neon-db.js` exits with code 1 before touching the DB if the var is unset); no credential ships in the source tree. Operators of envs that pre-date the `drop-public-setup` convoy still have the old `admin123` hash in their DB — rotate manually via the app (see Gotcha #4).
|
||||
- **UI:** Tailwind CSS + custom CSS variables for theming (light/dark via `lib/theme-context.js`)
|
||||
- **Hosting:** Vercel (`vercel.json`, `.vercel/` present)
|
||||
- **Hosting:** Dokploy on CT 112 (`deckhearth.stillwell.cloud`, Traefik on CT 100). Vercel-era config (`vercel.json`, `.vercel/`) is still in the tree pending decommission (`migrate-neon-to-homelab` phases 6–8) — see `docs/DOKPLOY_DEPLOY.md`.
|
||||
|
||||
## 2. Architecture quick reference
|
||||
|
||||
|
|
@ -66,10 +74,10 @@ A web app for managing trading-card-game collections (Magic, Pokémon, Lorcana).
|
|||
| Pages router views | `pages/*.js` | Public + auth views; uses `components/Layout.js` |
|
||||
| API routes | `pages/api/**/*.js` | Express-style `handler(req, res)`. **30+ handlers depend on `lib/permission-middleware.js::getUserFromRequest`** |
|
||||
| Shared UI | `components/*.js` | `Layout`, `CardItem`, `CameraScanner`, modal family |
|
||||
| Auth + DB libs | `lib/*.js` | `use-auth` (canonical client hook — sole surface post-`single-auth-provider`, PR #31, `0668b0c`), `auth-secret` (single JWT_SECRET + TTL source), `permission-middleware` (server-side `getUserFromRequest` + `withCollectionPermission`), `rate-limit` (5 named limiters — see Gotcha #12). The legacy `lib/database.js` was deleted by `single-sql-client` (PR #30, `c403ea4`); DB access now goes through `@vercel/postgres` tagged templates directly. |
|
||||
| Auth + DB libs | `lib/*.js` | `use-auth` (canonical client hook — sole surface post-`single-auth-provider`, PR #31, `0668b0c`), `auth-secret` (single JWT_SECRET + TTL source), `permission-middleware` (server-side `getUserFromRequest` + `withCollectionPermission`), `rate-limit` (6 named limiters — see Gotcha #12), `sql.js` (canonical Postgres client — tagged-template helper over the `postgres` package), `object-storage.js` (MinIO/S3 scan-capture uploads). The legacy `lib/database.js` was deleted by `single-sql-client` (PR #30, `c403ea4`). |
|
||||
| Migration scripts | `scripts/*.js` | 27+ one-off "add column" / "seed" scripts. No formal migration tool |
|
||||
| Card-import jobs | `pages/api/cards/import-*.js`, `scripts/import-*.js` | Scryfall / Lorcana / Pokémon TCG APIs |
|
||||
| Database schema | `scripts/setup-neon-db.js` | Bootstrap SQL DDL — the source of truth until a real migration tool lands |
|
||||
| Database schema | `migrations/` + `scripts/setup-neon-db.js` | `node-pg-migrate` migrations are the source of truth post-`migration-tool`; `setup-neon-db.js` chains `migrate up` + admin seed |
|
||||
| Schema map | `docs/SCHEMA_MAP.md` | Hand-curated; regenerate after schema changes |
|
||||
|
||||
Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 5602 edges). Ask: *"what calls `getUserFromRequest`?"* before refactoring auth.
|
||||
|
|
@ -83,7 +91,7 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560
|
|||
- **Auth helper (token mint / verify / password hash):** `import { ... } from '../../pages/api/auth-utils'` (`generateToken`, `verifyToken`, `hashPassword`, `verifyPassword`). Reads the secret + TTL from `lib/auth-secret.js` under the hood.
|
||||
- **Rate limiting:** `import { checkAuthRateLimit } from '../../lib/rate-limit.js'` for any new auth-surface endpoint (`/api/auth/login` + `/api/auth/register` already wired). Returns `{ allowed, remaining, reset }`; on `!allowed` return 429 with a `Retry-After` header. See `.cursor/rules/api-routes.mdc` § "Rate limiting" for the verbatim shape.
|
||||
- **Permission gate for collection routes:** wrap handlers with `withCollectionPermission('viewer' | 'editor' | 'owner')` from `lib/permission-middleware.js`.
|
||||
- **DB access:** Use **tagged-template** style — `import { sql } from '@vercel/postgres'`. The legacy `lib/database.js` (`db.query(string, params)` wrapper around `@neondatabase/serverless`, which interpolated params into a string and called `sql.unsafe`) was deleted by `single-sql-client` (PR #30, `c403ea4`); do NOT reintroduce that shape. For `scripts/**` helpers that legitimately need the Neon HTTP driver (e.g. `setup-neon-db.js`, `reset-db.js`), import `{ neon } from '@neondatabase/serverless'` directly and use tagged-template SQL (`await sql\`...\``) — the safe shape, not the wrapper's unsafe shape.
|
||||
- **DB access:** Use **tagged-template** style — `import { sql } from '../lib/sql.js'` (path relative to the caller). The legacy `lib/database.js` (`db.query(string, params)` wrapper around `@neondatabase/serverless`, which interpolated params into a string and called `sql.unsafe`) was deleted by `single-sql-client` (PR #30, `c403ea4`); do NOT reintroduce that shape. `lib/sql.js` reads `POSTGRES_URL` (falls back to `DATABASE_URL`) and returns the former `@vercel/postgres` result shape `{ rows, rowCount }`. For `scripts/**` helpers that legitimately need the Neon HTTP driver during the transition (e.g. `reset-db.js`, the rename-email migration), import `{ neon } from '@neondatabase/serverless'` directly and use tagged-template SQL (`await sql\`...\``) — the safe shape, not a string-interpolating wrapper.
|
||||
- **Activity logging:** `logCollectionActivity(collectionId, userId, action, details)` — call it from any handler that mutates a collection.
|
||||
- **File names:** `kebab-case.js` for libs/scripts; `PascalCase.js` for React components.
|
||||
- **Imports:** No path aliases configured; use relative imports.
|
||||
|
|
@ -122,9 +130,9 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560
|
|||
- **#9 — `typescript` is a devDep, but the source is still JavaScript-only.** `package.json` lists `typescript@^5.9.3` purely so `eslint-config-next@16`'s bundled `typescript-eslint` chain can satisfy its hard `require('typescript')` at module load (the `peerDependenciesMeta.typescript.optional: true` flag in `eslint-config-next` only suppresses npm's install-time warning, not the runtime require). There is no `tsconfig.json`, no `.ts`/`.tsx` files, and no `// @ts-check` directives. Do not rename `.js` files to `.ts` or add a `tsconfig.json` without an explicit convoy decision — TypeScript adoption is its own scope. See `.convoys/bump-next-js.md` § Decisions C.
|
||||
- **#10 — ESLint pinned to v9 (maintenance), not v10 (latest).** `devDependencies.eslint` is `^9.39.4` even though `latest` is `10.4.0`. We tried v10 and `npm run lint` crashed with `TypeError: scopeManager.addGlobals is not a function` because `eslint-config-next@16`'s bundled `typescript-eslint@8.x` predates ESLint v10's redesigned global-ingestion path. Reverted to v9 under Decision D. **Do NOT bump ESLint independently** — wait for the queued `bump-eslint-10` follow-up convoy, which is upstream-blocked until `typescript-eslint` ships a v10-tested release that `eslint-config-next` bundles. See `.convoys/bump-next-js.md` § Decisions D + "Follow-up convoys queued".
|
||||
- **#11 — Turbopack is now the default bundler.** `next dev` and `next build` use Turbopack by default in Next.js 16. The fallback per command is `--webpack` (e.g. `next build --webpack`). We have no custom `webpack:` block in `next.config.js`, no custom loaders/aliases, and no Sass tilde imports, so Turbopack should "just work" — but if a build/runtime regression appears, reproduce on both bundlers before deciding whether to revert or pin a script to webpack. Do not pre-emptively switch to `--webpack`.
|
||||
- **#12 — Rate-limit env vars are `KV_REST_API_URL` / `KV_REST_API_TOKEN`, not `UPSTASH_REDIS_REST_*`.** `lib/rate-limit.js` reads the Vercel Upstash Marketplace integration's auto-provisioned names. Three other Upstash-shaped vars exist in the Vercel-managed env (`KV_URL`, `REDIS_URL`, `KV_REST_API_READ_ONLY_TOKEN`) but our `@upstash/redis` REST client does not use them — do not wire to them. In prod, the rate-limit module **fails closed** if either of the two REST vars is missing (a single failed login is a better outcome than silently disabling brute-force protection). In dev / test, it warn-and-continues as a no-op so local work is unaffected when Upstash isn't wired up.
|
||||
- **#12 — Rate limiting reads `REDIS_URL` (homelab Redis, CT 102).** `lib/rate-limit.js` is backed by `ioredis` + `rate-limiter-flexible` with **six named limiter classes** (`auth`, `search`, `upload`, `generate`, `import`, and the scanner-era `scan` — 15/min user-keyed), each with its own `deckhearth:*` Redis key prefix. The Vercel-Upstash era vars (`KV_REST_API_URL` / `KV_REST_API_TOKEN`) are obsolete — do not wire to them. In production the module **fails closed** if `REDIS_URL` is missing (a single failed login is a better outcome than silently disabling brute-force protection). In dev / test, it warn-and-continues as a no-op so local work is unaffected when Redis isn't reachable.
|
||||
|
||||
**Milestone — `add-rate-limiting` convoy (squash `708ef45`, PR #20, 2026-05-24) closed P0 #6 — all 8 P0s now RESOLVED.** The lib refactored from a single auth-only limiter to **5 named limiters** with a `Map<className, Ratelimit>` cache (one shared Redis client, five `Ratelimit` instances, distinct Redis prefix per class). The five exports + their use cases:
|
||||
**Milestone — `add-rate-limiting` convoy (squash `708ef45`, PR #20, 2026-05-24) closed P0 #6 — all 8 P0s now RESOLVED.** The lib refactored from a single auth-only limiter to **5 named limiters** with a `Map<className, Ratelimit>` cache (one shared Redis client, distinct Redis prefix per class). A sixth (`scan`, from the scanner-era hardening) joined later — current full set:
|
||||
|
||||
| Helper | Class | Limit/window | Key | Redis prefix | Routes |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
|
|
@ -133,30 +141,31 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560
|
|||
| `checkUploadRateLimit(req, userId)` | `upload` | 10 / 1 hour | user | `deckhearth:upload` | `/api/user/avatar` |
|
||||
| `checkGenerateRateLimit(req, userId)` | `generate` | 5 / 1 hour | user | `deckhearth:generate` | `/api/user/avatar/generate` |
|
||||
| `checkImportRateLimit(req, userId)` | `import` | 5 / 1 hour | user (admin-only) | `deckhearth:import` | `/api/cards/import-mtg`, `/api/cards/import-pokemon` |
|
||||
| `checkScanRateLimit(req, userId)` | `scan` | 15 / 1 min | user | `deckhearth:scan` | `/api/scan/identify` (one camera verify may escalate L0→L2; vision path is the expensive step) |
|
||||
|
||||
Prefixes renamed `tcgvault:*` → `deckhearth:*` in `pick-a-name` (squash `9abbab6`, 2026-05-24); accepted one-time per-15-min / per-1-hour counter reset; existing Upstash state at `tcgvault:*` keys is now stale and will TTL out naturally.
|
||||
|
||||
All five return the same `{ allowed, remaining, reset }` shape; on `!allowed`, set `Retry-After: Math.ceil((reset - Date.now()) / 1000)` and return 429 with the uniform message `'Too many attempts. Try again later.'` (per-class variation would fingerprint the limits to an attacker — explicitly rejected).
|
||||
All six return the same `{ allowed, remaining, reset }` shape; on `!allowed`, set `Retry-After: Math.ceil((reset - Date.now()) / 1000)` and return 429 with the uniform message `'Too many attempts. Try again later.'` (per-class variation would fingerprint the limits to an attacker — explicitly rejected).
|
||||
|
||||
**Defensive THROW pattern.** `extractUserIdentifier(userId)` THROWS with a named error when `userId` is `null` / `undefined` / `''` / `NaN`. Surfaces gate-ordering bugs at dev time rather than silently falling back to IP and converting a per-user limit into a per-IP limit (which would lock household members out for one user's behavior). Numeric `0` is intentionally accepted (returns `'user:0'`) for forward-compat. **Gate-ordering rule: per-user rate-limit gates (`upload`, `generate`, `import`) MUST sit AFTER the auth check.** For the two `/api/cards/import-*` routes, the ordering is also `auth → admin-role check (403 if not admin) → rate-limit`; the admin-role check sits between auth and rate-limit. IP-keyed gates (`auth`, `search`) can sit anywhere after the method check.
|
||||
|
||||
Adding a sixth class is a one-line `LIMITER_CONFIG` addition + one new exported function (no `init()` restructuring needed). Tuning an existing class is a one-line `LIMITER_CONFIG` edit. The full verbatim call shape + gate-ordering rules + identifier-extraction documentation live in `.cursor/rules/api-routes.mdc` § Rate limiting.
|
||||
Adding another class is a one-line `LIMITER_CONFIG` addition + one new exported function (no `init()` restructuring needed). Tuning an existing class is a one-line `LIMITER_CONFIG` edit. The full verbatim call shape + gate-ordering rules + identifier-extraction documentation live in `.cursor/rules/api-routes.mdc` § Rate limiting.
|
||||
|
||||
## 5. Running locally
|
||||
|
||||
- **Runtime:** Node 20 (Vercel default).
|
||||
- **Setup:** `npm install`, copy `.env.local` template (POSTGRES_URL + JWT_SECRET + RESEND_API_KEY + BLOB_READ_WRITE_TOKEN + ADMIN_INITIAL_PASSWORD — the last is required for `npm run setup-db` and the script exits with code 1 if it's unset; optionally KV_REST_API_URL + KV_REST_API_TOKEN to exercise the rate limiter locally — without them, `lib/rate-limit.js` warn-and-no-ops in dev), then `npm run setup-db` once.
|
||||
- **Runtime:** Node 20+ (`"type": "module"` — ESM everywhere).
|
||||
- **Setup:** `npm install`, create `.env.local` with `POSTGRES_URL` (+ `POSTGRES_URL_DIRECT` for migrations) pointing at CT 102 or any Postgres 17, plus `JWT_SECRET` + `ADMIN_INITIAL_PASSWORD` (required for `npm run setup-db`, which exits with code 1 if unset). Optionally `REDIS_URL` to exercise the rate limiter locally — without it, `lib/rate-limit.js` warn-and-no-ops in dev (production fails closed) — and the `S3_*` MinIO vars for scan-capture uploads. Then `npm run setup-db` once. Full env contract in README § Installation and `docs/DOKPLOY_DEPLOY.md`.
|
||||
- **Dev server:** `npm run dev` → http://localhost:3000.
|
||||
|
||||
## 6. Testing
|
||||
|
||||
- **Unit-test runner:** `vitest@^3.2.4` (installed via `fix-auth-bypass` Brief 5, commit `1629afb`). `npm test` for watch mode; `npm run test:run` for the CI / single-shot mode. Config in `vitest.config.js`, setup in `test/setup.js` (sets `JWT_SECRET` + `NODE_ENV=test` before any module loads). Specs live under `test/` mirroring source layout (`test/lib/*.test.js`, `test/api/*.test.js`, `test/components/*.test.js`). Last green: 21/21 tests pass.
|
||||
- **Vitest coverage today:** 21 unit tests — `lib/auth-secret.js` (3), `lib/permission-middleware.js::getUserFromRequest` (8, incl. a negative regression against the old synthetic-admin shape — Gotcha #2), `pages/api/auth-utils.js` (5), and `components/Layout.js` (5 regression-lock assertions for the post-PR-#15 logged-out branch — Gotcha #8). These tests lock in the contracts established by `fix-auth-bypass` Briefs 1 + 2 and `fix-layout-default-user`; do not weaken them when refactoring auth or Layout.
|
||||
- **Unit-test runner:** `vitest@^3.2.4` (installed via `fix-auth-bypass` Brief 5, commit `1629afb`). `npm test` for watch mode; `npm run test:run` for the CI / single-shot mode. Config in `vitest.config.js`, setup in `test/setup.js` (sets `JWT_SECRET` + `NODE_ENV=test` before any module loads, and stubs `ResizeObserver` for jsdom). Specs live under `test/` mirroring source layout (`test/lib/*.test.js`, `test/api/*.test.js`, `test/components/*.test.js`). Last green: 231/231 tests across 43 files (2026-08-23).
|
||||
- **Vitest coverage today:** 231 unit tests spanning auth (`lib/auth-secret.js`, `lib/permission-middleware.js::getUserFromRequest` incl. the negative regression against the old synthetic-admin shape — Gotcha #2), `pages/api/auth-utils.js`, Layout logged-out regressions (Gotcha #8), scanner libs/hooks/components (`use-scanner-identification`, `use-camera-scanner`, `ScannerCamera`, scanner page), card import + reconcile helpers, and catalog sync. The original fix-auth-bypass / fix-layout-default-user contract tests are still present — do not weaken them when refactoring auth or Layout.
|
||||
- **E2E / smoke runner:** `@playwright/test@^1.60.0` (installed via `adopt-playwright-smoke`, PR #18 squash `7b6f751`). Config in `playwright.config.js` (root, ESM) declares two projects:
|
||||
- `smoke` — `tests/smoke/**/*.spec.@(ts|js)`; invoked by `.github/workflows/preview-smoke.yml`. `npm run test:smoke` locally.
|
||||
- `visual` — `tests/visual/**/*.spec.@(ts|js)`; invoked by `.github/workflows/visual-diff.yml`. `npm run test:visual` locally; `npm run test:visual:update` to (re-)seed baselines.
|
||||
|
||||
Local-run convention: boot `next dev` in one terminal, then in another run `BASE_URL=http://localhost:3000 npm run test:smoke` (or against a deployed preview, `BASE_URL=https://<preview>.vercel.app VERCEL_AUTOMATION_BYPASS_SECRET=<value> npm run test:smoke`). No `next dev` auto-boot in the test scripts (Decision 6 of `adopt-playwright-smoke`).
|
||||
Local-run convention: boot `next dev` in one terminal, then in another run `BASE_URL=http://localhost:3000 npm run test:smoke`. CI defaults `BASE_URL` to the homelab deployment (`https://deckhearth.stillwell.cloud`) via `vars.SMOKE_BASE_URL`; a legacy Vercel-preview target still works with `BASE_URL=https://<preview>.vercel.app VERCEL_AUTOMATION_BYPASS_SECRET=<value>` but is pending decommission (§ 7). No `next dev` auto-boot in the test scripts (Decision 6 of `adopt-playwright-smoke`).
|
||||
- **Browsers must be installed once locally:** `npx playwright install --with-deps chromium`. CI re-runs this on every workflow run (it's cached when possible).
|
||||
- **Visual baselines:** committed under `tests/visual/__screenshots__/`. The initial Linux baseline (`home.png`) was seeded by PR #58 (`83a358b`, 2026-06-02). Baselines are committed to git — they are not gitignored — so a `Screenshot diff` failure is reviewable from PR comments + artifacts without bouncing through a regeneration step. **Re-seeding** (when the homepage changes intentionally) MUST happen in a Linux environment so the PNG matches what CI produces. Recommended paths:
|
||||
|
||||
|
|
@ -197,7 +206,19 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560
|
|||
|
||||
## 7. Deployment
|
||||
|
||||
- **Vercel** auto-deploys `main` and creates Preview deployments for every PR. `vercel.json` and `.vercel/` are committed. CI in `.github/workflows/` runs lint + types (no duplicate build — Vercel handles it).
|
||||
> **Status (2026-08): production is the Dokploy homelab deployment** —
|
||||
> app on CT 112, public URL `https://deckhearth.stillwell.cloud` via Traefik
|
||||
> on CT 100, data plane on CT 102 (Postgres/Redis/MinIO). CI smoke + visual
|
||||
> workflows already gate against it (`BASE_URL` defaults there). The
|
||||
> **Vercel bullets below are legacy**, retained until `migrate-neon-to-homelab`
|
||||
> phase 8 (decommission) lands — `vercel.json` / `.vercel/` are still in the
|
||||
> tree and `VERCEL_AUTOMATION_BYPASS_SECRET` is only needed for the legacy
|
||||
> `*.vercel.app` targets. Runbook: [`docs/DOKPLOY_DEPLOY.md`](docs/DOKPLOY_DEPLOY.md).
|
||||
|
||||
- **Vercel (legacy, pending decommission).** Historically, Vercel
|
||||
auto-deployed `main` and created Preview deployments for every PR.
|
||||
`vercel.json` and `.vercel/` are committed but scheduled for removal in
|
||||
phase 8 of `migrate-neon-to-homelab`.
|
||||
- **Preview protection bypass for automation.** The project has a Protection Bypass for Automation token exposed locally as `VERCEL_AUTOMATION_BYPASS_SECRET` in `.env.local` (not committed) and seeded into GitHub Actions as a repo secret (`gh secret set VERCEL_AUTOMATION_BYPASS_SECRET`, 2026-05-24). The secret is consumed in two shapes:
|
||||
1. **Query parameter** on `wait-for-vercel-preview@v1.3.2`'s `path:` input in both `preview-smoke.yml` and `visual-diff.yml` — `path: '/?x-vercel-protection-bypass=…'`, bare form, **without** `&x-vercel-set-bypass-cookie=true` (the cookie variant returns 307 + Set-Cookie and axios in Node has no cookie jar, so it 401s on the redirect). Plumbed by PR #17 (`fix-vercel-deployment-protection-in-ci`, squash `9a3e077`).
|
||||
2. **HTTP header** in `playwright.config.js`'s `use.extraHTTPHeaders` — `'x-vercel-protection-bypass': <secret>`. Playwright's browser context has a real cookie jar so this shape works there, and the testOptions surface forwards the header to the test-level `request` fixture's `APIRequestContext` as well, so both `page.goto(...)` calls and `request.get('/api/health')` calls hit the protected preview correctly without per-spec header injection. Plumbed by PR #18 (`adopt-playwright-smoke`, squash `7b6f751`) per Decision 2 of that convoy.
|
||||
|
|
@ -206,7 +227,7 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560
|
|||
|
||||
Do not log or echo the value. If the operator rotates the token in the Vercel dashboard, re-seed the GitHub secret via `gh secret set VERCEL_AUTOMATION_BYPASS_SECRET --body "<new value>"`. See `.convoys/fix-vercel-deployment-protection-in-ci.md` and `.convoys/adopt-playwright-smoke.md`.
|
||||
- **Shell-injection hardening in workflow YAML.** Never inline `${{ github.event.* }}` directly into a `run:` block — route the value through the step's `env:` block and quote it (`"$VAR_NAME"`) in shell. PR #17's CI validation caught a real syntax error from a PR body containing `(` because the gate-job's Decide step inlined `${{ github.event.pull_request.body }}` straight into bash; commit `b6f8688` swept both `preview-smoke.yml` and `visual-diff.yml` to the `env:` + quoted-shell pattern. This is GitHub's official Security Hardening guidance ("Security hardening for GitHub Actions" → "Using a third-party action"). Apply to any new workflow that reads PR body / title / branch name / commit messages in shell.
|
||||
- **CI runs on the axiom homelab (CT 111).** Four of the five workflows execute on `[self-hosted, axiom]` runners managed in the `axiom-server` repo (`proxmox/ct111/`). Day-to-day this is invisible — pushes still trigger jobs and Vercel still builds previews — but two operational notes matter:
|
||||
- **CI runs on the axiom homelab (CT 111).** Four of the five workflows execute on `[self-hosted, axiom]` runners managed in the `axiom-server` repo (`proxmox/ct111/`). Day-to-day this is invisible — pushes still trigger jobs, and Dokploy builds `main` on CT 112 (Vercel preview builds continue only until phase 8 decommission) — but two operational notes matter:
|
||||
1. **PAT rotation.** The runners authenticate to GitHub via an org-scoped PAT stored on CT 111 at `/opt/appdata/gha-runner/.env` (key `GH_PAT`, scopes `admin:org`, `repo`, `workflow`). Rotate every 90 days. After updating the value on CT 111, run `./proxmox/scripts/sync.sh restart 111` to re-register all 4 runners. If the PAT lapses silently, new jobs fail registration immediately; check `./proxmox/scripts/sync.sh logs 111 gha-runner-1` for `Http response code: NotFound` to confirm.
|
||||
2. **1-line revert path (D5) — when axiom is offline mid-PR-storm.** If CT 111 is down for maintenance, hardware swap, or any reason, and a hot fix needs CI to land, swap every `[self-hosted, axiom]` back to `ubuntu-latest`:
|
||||
|
||||
|
|
|
|||
82
README.md
82
README.md
|
|
@ -1,6 +1,10 @@
|
|||
# Deck Hearth
|
||||
|
||||
A modern trading card game collection manager built with Next.js and Neon Database.
|
||||
A modern trading card game collection manager built with Next.js and PostgreSQL.
|
||||
|
||||
> Repo history note: this project was formerly **TCG Vault** (`tcg-vault`);
|
||||
> the GitHub repo is now [`stwl-labs/deckhearth`](https://github.com/stwl-labs/deckhearth).
|
||||
> Local checkout folders named `tcg-vault` are fine.
|
||||
|
||||
## 🚀 Features
|
||||
|
||||
|
|
@ -15,19 +19,20 @@ A modern trading card game collection manager built with Next.js and Neon Databa
|
|||
|
||||
- **Frontend**: Next.js 16 (Pages router), React 18, JavaScript (TypeScript is a devDep only — see `AGENTS.md` Gotcha #9)
|
||||
- **Backend**: Next.js API Routes
|
||||
- **Database**: Neon PostgreSQL (serverless)
|
||||
- **Database**: PostgreSQL 17 + pgvector on the axiom homelab (CT 102, `192.168.68.102:5432`), accessed via the tagged-template helper in `lib/sql.js` — see `docs/HOMELAB_DATABASE.md`
|
||||
- **Authentication**: JWT with bcrypt (24-hour expiry; `lib/auth-secret.js` is the single source of truth for `JWT_SECRET`)
|
||||
- **Rate limiting**: `@upstash/ratelimit` on `/api/auth/login` + `/api/auth/register` (5 attempts / 15 min per IP)
|
||||
- **Testing**: Vitest (unit); Playwright queued
|
||||
- **Styling**: Tailwind CSS
|
||||
- **Deployment**: Vercel
|
||||
- **Rate limiting**: `lib/rate-limit.js` — six named limiters (`auth`, `search`, `upload`, `generate`, `import`, `scan`) backed by `ioredis` + `rate-limiter-flexible` against homelab Redis (CT 102). Fails closed in production if `REDIS_URL` is unset; warn-and-no-op in dev
|
||||
- **Testing**: Vitest (unit) + Playwright (`smoke` and `visual` projects)
|
||||
- **Object storage**: MinIO on CT 102 (scan captures; S3-compatible via `@aws-sdk/client-s3`)
|
||||
- **Styling**: Tailwind CSS + Liquid Glass design tokens (`docs/DESIGN_TOKENS.md`)
|
||||
- **Deployment**: Dokploy on CT 112, public URL `https://deckhearth.stillwell.cloud` via Traefik — see `docs/DOKPLOY_DEPLOY.md`
|
||||
|
||||
## 📦 Installation
|
||||
|
||||
1. **Clone the repository**
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd tcg-vault
|
||||
git clone https://github.com/stwl-labs/deckhearth.git
|
||||
cd deckhearth
|
||||
```
|
||||
|
||||
2. **Install dependencies**
|
||||
|
|
@ -35,27 +40,32 @@ A modern trading card game collection manager built with Next.js and Neon Databa
|
|||
npm install
|
||||
```
|
||||
|
||||
3. **Set up environment variables**
|
||||
```bash
|
||||
cp .env.example .env.local
|
||||
```
|
||||
|
||||
Update `.env.local` with your Neon database URL and a real JWT secret:
|
||||
3. **Create `.env.local`** (there is no committed template — use the shape below)
|
||||
```env
|
||||
POSTGRES_URL="postgresql://your-username:your-password@your-host/your-database"
|
||||
# Required — homelab Postgres (CT 102) or any Postgres 17 instance
|
||||
POSTGRES_URL="postgresql://deckhearth:<password>@192.168.68.102:5432/deckhearth"
|
||||
# Migrations (`npm run migrate`); on the homelab, same value as POSTGRES_URL
|
||||
POSTGRES_URL_DIRECT="postgresql://deckhearth:<password>@192.168.68.102:5432/deckhearth"
|
||||
JWT_SECRET="<generate with: openssl rand -hex 32>"
|
||||
# Required for `npm run setup-db` — used once to hash the initial admin password.
|
||||
# Set in .env.local for local dev, or as a CI secret if you run setup from CI.
|
||||
ADMIN_INITIAL_PASSWORD="<generate with: openssl rand -base64 24>"
|
||||
# Optional — exercise the rate limiter locally. Without them, `lib/rate-limit.js`
|
||||
# warn-and-no-ops in dev. In production these are auto-provisioned by the
|
||||
# Vercel Upstash Marketplace integration.
|
||||
KV_REST_API_URL="https://<your-upstash-host>.upstash.io"
|
||||
KV_REST_API_TOKEN="<your-upstash-rest-token>"
|
||||
# Rate limiting (homelab Redis, CT 102 — DB index 5 in prod). Optional in dev —
|
||||
# without it, `lib/rate-limit.js` warn-and-no-ops locally; production fails closed.
|
||||
REDIS_URL="redis://:password@192.168.68.102:6379/5"
|
||||
# Scan-capture uploads (MinIO on CT 102, S3-compatible — see lib/object-storage.js)
|
||||
S3_ENDPOINT="http://192.168.68.102:9000"
|
||||
S3_BUCKET="deckhearth"
|
||||
S3_ACCESS_KEY_ID="..."
|
||||
S3_SECRET_ACCESS_KEY="..."
|
||||
S3_PUBLIC_BASE_URL="https://cdn.stillwell.cloud/deckhearth"
|
||||
S3_REGION="us-east-1"
|
||||
```
|
||||
`JWT_SECRET` is **required** — `lib/auth-secret.js` throws at import time if it's unset.
|
||||
`ADMIN_INITIAL_PASSWORD` is **required** for `npm run setup-db` — the script exits with code 1 if it's unset.
|
||||
|
||||
> Migrating data off the old Neon instance? `NEON_DATABASE_URL` is read once by
|
||||
> `npm run migrate-neon-to-homelab`. See `docs/HOMELAB_DATABASE.md`.
|
||||
|
||||
4. **Set up the database**
|
||||
```bash
|
||||
npm run setup-db
|
||||
|
|
@ -127,27 +137,35 @@ The application uses the following tables:
|
|||
|
||||
## 🚀 Deployment
|
||||
|
||||
This app is configured for deployment on Vercel:
|
||||
Deck Hearth deploys to **Dokploy on CT 112** in the axiom homelab, fronted by
|
||||
Traefik on CT 100 at `https://deckhearth.stillwell.cloud`. The full runbook —
|
||||
Dokploy app settings, environment variables, Traefik route, and the n8n
|
||||
catalog-sync cron that replaced Vercel Cron — lives in
|
||||
[`docs/DOKPLOY_DEPLOY.md`](docs/DOKPLOY_DEPLOY.md).
|
||||
|
||||
1. **Connect your repository** to Vercel
|
||||
2. **Set environment variables** in Vercel dashboard
|
||||
3. **Deploy automatically** on push to main branch
|
||||
> **Transition note:** the Vercel + Neon era is being decommissioned
|
||||
> (`migrate-neon-to-homelab` convoy, phases 6–8). `vercel.json` and `.vercel/`
|
||||
> remain in the tree until that decommission lands; CI already gates against
|
||||
> the homelab deployment.
|
||||
|
||||
## 📁 Project Structure
|
||||
|
||||
```
|
||||
tcg-vault/
|
||||
deckhearth/ (formerly tcg-vault)
|
||||
├── pages/ # Next.js pages and API routes
|
||||
│ ├── api/ # API endpoints
|
||||
│ │ ├── auth/ # Authentication routes
|
||||
│ │ └── admin/ # Admin routes
|
||||
│ ├── _app.js # App wrapper
|
||||
│ └── index.js # Home page
|
||||
├── lib/ # Utility libraries
|
||||
│ └── database.js # Database adapter
|
||||
├── scripts/ # Database setup scripts
|
||||
├── public/ # Static assets
|
||||
└── .env.local # Environment variables
|
||||
├── components/ # React components (+ scanner/ subfamily)
|
||||
├── lib/ # Utility libraries
|
||||
│ └── sql.js # Canonical Postgres client (tagged templates)
|
||||
├── migrations/ # node-pg-migrate migrations (source of truth for DDL)
|
||||
├── scripts/ # Setup, import, and historical one-off jobs
|
||||
├── docs/ # Deploy / DB / design-token runbooks
|
||||
├── test/ + tests/ # Vitest unit specs; Playwright smoke + visual specs
|
||||
└── .env.local # Environment variables (not committed)
|
||||
```
|
||||
|
||||
## 🔐 First-time admin setup
|
||||
|
|
@ -161,7 +179,7 @@ variable is unset or empty.
|
|||
- **Local dev:** set `ADMIN_INITIAL_PASSWORD` in `.env.local` before running
|
||||
`npm run setup-db`. Use `openssl rand -base64 24` (or any other strong
|
||||
source) to generate the value.
|
||||
- **CI / Vercel:** set `ADMIN_INITIAL_PASSWORD` as a project secret if setup
|
||||
- **CI:** set `ADMIN_INITIAL_PASSWORD` as a repo secret if setup
|
||||
ever runs from CI. The env var is **only** read by the seed script; runtime
|
||||
auth uses the per-user password stored in the database.
|
||||
- **Admin email:** the seed creates `admin@deckhearth.com`. Change the password
|
||||
|
|
|
|||
Loading…
Reference in a new issue