From f2ba333daf195bb9cfc35c8297b3f78d6a9eb569 Mon Sep 17 00:00:00 2001 From: Randall Stillwell Date: Sun, 23 Aug 2026 22:34:54 -0500 Subject: [PATCH] docs: sweep README + AGENTS.md to post-homelab reality MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - README: Dokploy/CT102 Postgres stack, lib/sql.js, six rate-limit classes over REDIS_URL, Playwright smoke+visual, MinIO/S3 vars, inline .env.local contract (no .env.example exists), deckhearth.git clone URL, project-structure refresh. - AGENTS.md: infra banner for the Vercel/Neon -> homelab move, repo rename to stwl-labs/deckhearth, Data/Auth/Hosting overview bullets, DB-access convention re-pointed at lib/sql.js, Gotcha #12 rewritten for REDIS_URL + scan limiter (+ row in limiter table), test counts refreshed (231/231 across 43 files), §5 env contract, §7 marked legacy-pending-decommission with DOKPLOY_DEPLOY runbook pointer. --- AGENTS.md | 63 ++++++++++++++++++++++++++++-------------- README.md | 82 +++++++++++++++++++++++++++++++++---------------------- 2 files changed, 92 insertions(+), 53 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a4cd851..4d759af 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -3,15 +3,23 @@ Guidance for agents and humans working in this repo. Prefer existing patterns over new abstractions. > Branding note: this product is **Deck Hearth** as of 2026-05-24 -> (`pick-a-name` convoy, squash commit `9abbab6`, PR #21). The repo -> and Vercel project are still named `tcg-vault` — that rename is -> tracked in the queued `rename-repo-and-vercel-project` convoy -> (auto-redirects make it low-urgency). Admin email is +> (`pick-a-name` convoy, squash commit `9abbab6`, PR #21). The GitHub +> repo is now `stwl-labs/deckhearth` (renamed from `stwl-labs/tcg-vault`; +> local checkout folders named `tcg-vault` are fine). Admin email is > `admin@deckhearth.com`; the prior `admin@tcgvault.com` literal is > deliberately preserved in `test/lib/permission-middleware.test.js` > as a historical regression-lock per Risk 4 of the pick-a-name > convoy. +> Infra note (2026-08): Deck Hearth is moving off Vercel + Neon onto +> the axiom homelab — Postgres/Redis/MinIO on CT 102, app on Dokploy +> CT 112, public URL `deckhearth.stillwell.cloud`. Runtime DB access +> goes through `lib/sql.js` (the `postgres` package), not +> `@vercel/postgres`; rate limiting reads `REDIS_URL`. CI already gates +> against the homelab deployment; Neon/Vercel decommission is pending +> (`migrate-neon-to-homelab` convoy phases 6–8). See § 5–§ 7 and +> `docs/DOKPLOY_DEPLOY.md` / `docs/HOMELAB_DATABASE.md`. + ## Product vocabulary User-facing copy distinguishes **ownership** (everything you own) from **curated lists** (binders/subsets). Import labels from `lib/collection-vocabulary.js` (`VOCAB`, `collectionDisplayName`) rather than hardcoding strings. @@ -54,10 +62,10 @@ Three rules of thumb: A web app for managing trading-card-game collections (Magic, Pokémon, Lorcana). Users authenticate, build collections + decks, scan physical cards via a camera+AI-OCR flow, and share publicly. Admin users curate the card database. - **Framework:** Next.js 16 (Pages router) + React 18, JavaScript (not TypeScript — see Gotcha #9) -- **Data:** Neon Postgres. The runtime auth surface uses `@vercel/postgres` tagged-template SQL exclusively post-`single-sql-client` (PR #30, `c403ea4`; `lib/database.js` deleted). 11 `scripts/**` helpers (`setup-neon-db.js`, `reset-db.js`, `migrations/2026-05-24-rename-admin-email.js`, plus 8 historical add-*/fix-*/seed-* jobs) still use `@neondatabase/serverless`'s `neon()` directly — out-of-scope per the no-go-zones rule and tracked as the queued `purge-neondatabase-serverless-fully` follow-up. Schema changes ship as `node-pg-migrate` migrations under `migrations/` at the repo root post-`migration-tool` (PR #32, `de9f334`) — see § 3 Conventions § "Schema changes" and Gotcha #6. -- **Auth:** Custom JWT (jsonwebtoken + bcryptjs), token stored in `localStorage`, sent as `Authorization: Bearer …`. No NextAuth. The secret + canonical 24h TTL come from `lib/auth-secret.js` (single source of truth; throws at module load if `JWT_SECRET` is unset). `getUserFromRequest` returns `null` for unauthenticated requests — no synthetic admin fallback — and login + register are rate-limited (5 attempts / 15 min via `@upstash/ratelimit`). The seed admin row is created at `admin@deckhearth.com` with a password supplied via the required `ADMIN_INITIAL_PASSWORD` env var (`scripts/setup-neon-db.js` exits with code 1 before touching the DB if the var is unset); no credential ships in the source tree. Operators of envs that pre-date the `drop-public-setup` convoy still have the old `admin123` hash in their DB — rotate manually via the app (see Gotcha #4). +- **Data:** Postgres 17 + pgvector on the axiom homelab (CT 102, `192.168.68.102:5432`). Runtime DB access goes through `lib/sql.js` — a tagged-template `sql` helper over the `postgres` package returning `{ rows, rowCount }` (the former `@vercel/postgres` shape, so call sites only changed their import). Migrations read `POSTGRES_URL_DIRECT`. 11 `scripts/**` helpers (`setup-neon-db.js`, `reset-db.js`, `migrations/2026-05-24-rename-admin-email.js`, plus 8 historical add-*/fix-*/seed-* jobs) still use `@neondatabase/serverless`'s `neon()` directly — out-of-scope per the no-go-zones rule and tracked as the queued `purge-neondatabase-serverless-fully` follow-up. Schema changes ship as `node-pg-migrate` migrations under `migrations/` at the repo root post-`migration-tool` (PR #32, `de9f334`) — see § 3 Conventions § "Schema changes" and Gotcha #6. +- **Auth:** Custom JWT (jsonwebtoken + bcryptjs), token stored in `localStorage`, sent as `Authorization: Bearer …`. No NextAuth. The secret + canonical 24h TTL come from `lib/auth-secret.js` (single source of truth; throws at module load if `JWT_SECRET` is unset). `getUserFromRequest` returns `null` for unauthenticated requests — no synthetic admin fallback — and login + register are rate-limited via `lib/rate-limit.js` (see Gotcha #12). The seed admin row is created at `admin@deckhearth.com` with a password supplied via the required `ADMIN_INITIAL_PASSWORD` env var (`scripts/setup-neon-db.js` exits with code 1 before touching the DB if the var is unset); no credential ships in the source tree. Operators of envs that pre-date the `drop-public-setup` convoy still have the old `admin123` hash in their DB — rotate manually via the app (see Gotcha #4). - **UI:** Tailwind CSS + custom CSS variables for theming (light/dark via `lib/theme-context.js`) -- **Hosting:** Vercel (`vercel.json`, `.vercel/` present) +- **Hosting:** Dokploy on CT 112 (`deckhearth.stillwell.cloud`, Traefik on CT 100). Vercel-era config (`vercel.json`, `.vercel/`) is still in the tree pending decommission (`migrate-neon-to-homelab` phases 6–8) — see `docs/DOKPLOY_DEPLOY.md`. ## 2. Architecture quick reference @@ -66,10 +74,10 @@ A web app for managing trading-card-game collections (Magic, Pokémon, Lorcana). | Pages router views | `pages/*.js` | Public + auth views; uses `components/Layout.js` | | API routes | `pages/api/**/*.js` | Express-style `handler(req, res)`. **30+ handlers depend on `lib/permission-middleware.js::getUserFromRequest`** | | Shared UI | `components/*.js` | `Layout`, `CardItem`, `CameraScanner`, modal family | -| Auth + DB libs | `lib/*.js` | `use-auth` (canonical client hook — sole surface post-`single-auth-provider`, PR #31, `0668b0c`), `auth-secret` (single JWT_SECRET + TTL source), `permission-middleware` (server-side `getUserFromRequest` + `withCollectionPermission`), `rate-limit` (5 named limiters — see Gotcha #12). The legacy `lib/database.js` was deleted by `single-sql-client` (PR #30, `c403ea4`); DB access now goes through `@vercel/postgres` tagged templates directly. | +| Auth + DB libs | `lib/*.js` | `use-auth` (canonical client hook — sole surface post-`single-auth-provider`, PR #31, `0668b0c`), `auth-secret` (single JWT_SECRET + TTL source), `permission-middleware` (server-side `getUserFromRequest` + `withCollectionPermission`), `rate-limit` (6 named limiters — see Gotcha #12), `sql.js` (canonical Postgres client — tagged-template helper over the `postgres` package), `object-storage.js` (MinIO/S3 scan-capture uploads). The legacy `lib/database.js` was deleted by `single-sql-client` (PR #30, `c403ea4`). | | Migration scripts | `scripts/*.js` | 27+ one-off "add column" / "seed" scripts. No formal migration tool | | Card-import jobs | `pages/api/cards/import-*.js`, `scripts/import-*.js` | Scryfall / Lorcana / Pokémon TCG APIs | -| Database schema | `scripts/setup-neon-db.js` | Bootstrap SQL DDL — the source of truth until a real migration tool lands | +| Database schema | `migrations/` + `scripts/setup-neon-db.js` | `node-pg-migrate` migrations are the source of truth post-`migration-tool`; `setup-neon-db.js` chains `migrate up` + admin seed | | Schema map | `docs/SCHEMA_MAP.md` | Hand-curated; regenerate after schema changes | Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 5602 edges). Ask: *"what calls `getUserFromRequest`?"* before refactoring auth. @@ -83,7 +91,7 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560 - **Auth helper (token mint / verify / password hash):** `import { ... } from '../../pages/api/auth-utils'` (`generateToken`, `verifyToken`, `hashPassword`, `verifyPassword`). Reads the secret + TTL from `lib/auth-secret.js` under the hood. - **Rate limiting:** `import { checkAuthRateLimit } from '../../lib/rate-limit.js'` for any new auth-surface endpoint (`/api/auth/login` + `/api/auth/register` already wired). Returns `{ allowed, remaining, reset }`; on `!allowed` return 429 with a `Retry-After` header. See `.cursor/rules/api-routes.mdc` § "Rate limiting" for the verbatim shape. - **Permission gate for collection routes:** wrap handlers with `withCollectionPermission('viewer' | 'editor' | 'owner')` from `lib/permission-middleware.js`. -- **DB access:** Use **tagged-template** style — `import { sql } from '@vercel/postgres'`. The legacy `lib/database.js` (`db.query(string, params)` wrapper around `@neondatabase/serverless`, which interpolated params into a string and called `sql.unsafe`) was deleted by `single-sql-client` (PR #30, `c403ea4`); do NOT reintroduce that shape. For `scripts/**` helpers that legitimately need the Neon HTTP driver (e.g. `setup-neon-db.js`, `reset-db.js`), import `{ neon } from '@neondatabase/serverless'` directly and use tagged-template SQL (`await sql\`...\``) — the safe shape, not the wrapper's unsafe shape. +- **DB access:** Use **tagged-template** style — `import { sql } from '../lib/sql.js'` (path relative to the caller). The legacy `lib/database.js` (`db.query(string, params)` wrapper around `@neondatabase/serverless`, which interpolated params into a string and called `sql.unsafe`) was deleted by `single-sql-client` (PR #30, `c403ea4`); do NOT reintroduce that shape. `lib/sql.js` reads `POSTGRES_URL` (falls back to `DATABASE_URL`) and returns the former `@vercel/postgres` result shape `{ rows, rowCount }`. For `scripts/**` helpers that legitimately need the Neon HTTP driver during the transition (e.g. `reset-db.js`, the rename-email migration), import `{ neon } from '@neondatabase/serverless'` directly and use tagged-template SQL (`await sql\`...\``) — the safe shape, not a string-interpolating wrapper. - **Activity logging:** `logCollectionActivity(collectionId, userId, action, details)` — call it from any handler that mutates a collection. - **File names:** `kebab-case.js` for libs/scripts; `PascalCase.js` for React components. - **Imports:** No path aliases configured; use relative imports. @@ -122,9 +130,9 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560 - **#9 — `typescript` is a devDep, but the source is still JavaScript-only.** `package.json` lists `typescript@^5.9.3` purely so `eslint-config-next@16`'s bundled `typescript-eslint` chain can satisfy its hard `require('typescript')` at module load (the `peerDependenciesMeta.typescript.optional: true` flag in `eslint-config-next` only suppresses npm's install-time warning, not the runtime require). There is no `tsconfig.json`, no `.ts`/`.tsx` files, and no `// @ts-check` directives. Do not rename `.js` files to `.ts` or add a `tsconfig.json` without an explicit convoy decision — TypeScript adoption is its own scope. See `.convoys/bump-next-js.md` § Decisions C. - **#10 — ESLint pinned to v9 (maintenance), not v10 (latest).** `devDependencies.eslint` is `^9.39.4` even though `latest` is `10.4.0`. We tried v10 and `npm run lint` crashed with `TypeError: scopeManager.addGlobals is not a function` because `eslint-config-next@16`'s bundled `typescript-eslint@8.x` predates ESLint v10's redesigned global-ingestion path. Reverted to v9 under Decision D. **Do NOT bump ESLint independently** — wait for the queued `bump-eslint-10` follow-up convoy, which is upstream-blocked until `typescript-eslint` ships a v10-tested release that `eslint-config-next` bundles. See `.convoys/bump-next-js.md` § Decisions D + "Follow-up convoys queued". - **#11 — Turbopack is now the default bundler.** `next dev` and `next build` use Turbopack by default in Next.js 16. The fallback per command is `--webpack` (e.g. `next build --webpack`). We have no custom `webpack:` block in `next.config.js`, no custom loaders/aliases, and no Sass tilde imports, so Turbopack should "just work" — but if a build/runtime regression appears, reproduce on both bundlers before deciding whether to revert or pin a script to webpack. Do not pre-emptively switch to `--webpack`. -- **#12 — Rate-limit env vars are `KV_REST_API_URL` / `KV_REST_API_TOKEN`, not `UPSTASH_REDIS_REST_*`.** `lib/rate-limit.js` reads the Vercel Upstash Marketplace integration's auto-provisioned names. Three other Upstash-shaped vars exist in the Vercel-managed env (`KV_URL`, `REDIS_URL`, `KV_REST_API_READ_ONLY_TOKEN`) but our `@upstash/redis` REST client does not use them — do not wire to them. In prod, the rate-limit module **fails closed** if either of the two REST vars is missing (a single failed login is a better outcome than silently disabling brute-force protection). In dev / test, it warn-and-continues as a no-op so local work is unaffected when Upstash isn't wired up. +- **#12 — Rate limiting reads `REDIS_URL` (homelab Redis, CT 102).** `lib/rate-limit.js` is backed by `ioredis` + `rate-limiter-flexible` with **six named limiter classes** (`auth`, `search`, `upload`, `generate`, `import`, and the scanner-era `scan` — 15/min user-keyed), each with its own `deckhearth:*` Redis key prefix. The Vercel-Upstash era vars (`KV_REST_API_URL` / `KV_REST_API_TOKEN`) are obsolete — do not wire to them. In production the module **fails closed** if `REDIS_URL` is missing (a single failed login is a better outcome than silently disabling brute-force protection). In dev / test, it warn-and-continues as a no-op so local work is unaffected when Redis isn't reachable. - **Milestone — `add-rate-limiting` convoy (squash `708ef45`, PR #20, 2026-05-24) closed P0 #6 — all 8 P0s now RESOLVED.** The lib refactored from a single auth-only limiter to **5 named limiters** with a `Map` cache (one shared Redis client, five `Ratelimit` instances, distinct Redis prefix per class). The five exports + their use cases: + **Milestone — `add-rate-limiting` convoy (squash `708ef45`, PR #20, 2026-05-24) closed P0 #6 — all 8 P0s now RESOLVED.** The lib refactored from a single auth-only limiter to **5 named limiters** with a `Map` cache (one shared Redis client, distinct Redis prefix per class). A sixth (`scan`, from the scanner-era hardening) joined later — current full set: | Helper | Class | Limit/window | Key | Redis prefix | Routes | | --- | --- | --- | --- | --- | --- | @@ -133,30 +141,31 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560 | `checkUploadRateLimit(req, userId)` | `upload` | 10 / 1 hour | user | `deckhearth:upload` | `/api/user/avatar` | | `checkGenerateRateLimit(req, userId)` | `generate` | 5 / 1 hour | user | `deckhearth:generate` | `/api/user/avatar/generate` | | `checkImportRateLimit(req, userId)` | `import` | 5 / 1 hour | user (admin-only) | `deckhearth:import` | `/api/cards/import-mtg`, `/api/cards/import-pokemon` | + | `checkScanRateLimit(req, userId)` | `scan` | 15 / 1 min | user | `deckhearth:scan` | `/api/scan/identify` (one camera verify may escalate L0→L2; vision path is the expensive step) | Prefixes renamed `tcgvault:*` → `deckhearth:*` in `pick-a-name` (squash `9abbab6`, 2026-05-24); accepted one-time per-15-min / per-1-hour counter reset; existing Upstash state at `tcgvault:*` keys is now stale and will TTL out naturally. - All five return the same `{ allowed, remaining, reset }` shape; on `!allowed`, set `Retry-After: Math.ceil((reset - Date.now()) / 1000)` and return 429 with the uniform message `'Too many attempts. Try again later.'` (per-class variation would fingerprint the limits to an attacker — explicitly rejected). + All six return the same `{ allowed, remaining, reset }` shape; on `!allowed`, set `Retry-After: Math.ceil((reset - Date.now()) / 1000)` and return 429 with the uniform message `'Too many attempts. Try again later.'` (per-class variation would fingerprint the limits to an attacker — explicitly rejected). **Defensive THROW pattern.** `extractUserIdentifier(userId)` THROWS with a named error when `userId` is `null` / `undefined` / `''` / `NaN`. Surfaces gate-ordering bugs at dev time rather than silently falling back to IP and converting a per-user limit into a per-IP limit (which would lock household members out for one user's behavior). Numeric `0` is intentionally accepted (returns `'user:0'`) for forward-compat. **Gate-ordering rule: per-user rate-limit gates (`upload`, `generate`, `import`) MUST sit AFTER the auth check.** For the two `/api/cards/import-*` routes, the ordering is also `auth → admin-role check (403 if not admin) → rate-limit`; the admin-role check sits between auth and rate-limit. IP-keyed gates (`auth`, `search`) can sit anywhere after the method check. - Adding a sixth class is a one-line `LIMITER_CONFIG` addition + one new exported function (no `init()` restructuring needed). Tuning an existing class is a one-line `LIMITER_CONFIG` edit. The full verbatim call shape + gate-ordering rules + identifier-extraction documentation live in `.cursor/rules/api-routes.mdc` § Rate limiting. + Adding another class is a one-line `LIMITER_CONFIG` addition + one new exported function (no `init()` restructuring needed). Tuning an existing class is a one-line `LIMITER_CONFIG` edit. The full verbatim call shape + gate-ordering rules + identifier-extraction documentation live in `.cursor/rules/api-routes.mdc` § Rate limiting. ## 5. Running locally -- **Runtime:** Node 20 (Vercel default). -- **Setup:** `npm install`, copy `.env.local` template (POSTGRES_URL + JWT_SECRET + RESEND_API_KEY + BLOB_READ_WRITE_TOKEN + ADMIN_INITIAL_PASSWORD — the last is required for `npm run setup-db` and the script exits with code 1 if it's unset; optionally KV_REST_API_URL + KV_REST_API_TOKEN to exercise the rate limiter locally — without them, `lib/rate-limit.js` warn-and-no-ops in dev), then `npm run setup-db` once. +- **Runtime:** Node 20+ (`"type": "module"` — ESM everywhere). +- **Setup:** `npm install`, create `.env.local` with `POSTGRES_URL` (+ `POSTGRES_URL_DIRECT` for migrations) pointing at CT 102 or any Postgres 17, plus `JWT_SECRET` + `ADMIN_INITIAL_PASSWORD` (required for `npm run setup-db`, which exits with code 1 if unset). Optionally `REDIS_URL` to exercise the rate limiter locally — without it, `lib/rate-limit.js` warn-and-no-ops in dev (production fails closed) — and the `S3_*` MinIO vars for scan-capture uploads. Then `npm run setup-db` once. Full env contract in README § Installation and `docs/DOKPLOY_DEPLOY.md`. - **Dev server:** `npm run dev` → http://localhost:3000. ## 6. Testing -- **Unit-test runner:** `vitest@^3.2.4` (installed via `fix-auth-bypass` Brief 5, commit `1629afb`). `npm test` for watch mode; `npm run test:run` for the CI / single-shot mode. Config in `vitest.config.js`, setup in `test/setup.js` (sets `JWT_SECRET` + `NODE_ENV=test` before any module loads). Specs live under `test/` mirroring source layout (`test/lib/*.test.js`, `test/api/*.test.js`, `test/components/*.test.js`). Last green: 21/21 tests pass. -- **Vitest coverage today:** 21 unit tests — `lib/auth-secret.js` (3), `lib/permission-middleware.js::getUserFromRequest` (8, incl. a negative regression against the old synthetic-admin shape — Gotcha #2), `pages/api/auth-utils.js` (5), and `components/Layout.js` (5 regression-lock assertions for the post-PR-#15 logged-out branch — Gotcha #8). These tests lock in the contracts established by `fix-auth-bypass` Briefs 1 + 2 and `fix-layout-default-user`; do not weaken them when refactoring auth or Layout. +- **Unit-test runner:** `vitest@^3.2.4` (installed via `fix-auth-bypass` Brief 5, commit `1629afb`). `npm test` for watch mode; `npm run test:run` for the CI / single-shot mode. Config in `vitest.config.js`, setup in `test/setup.js` (sets `JWT_SECRET` + `NODE_ENV=test` before any module loads, and stubs `ResizeObserver` for jsdom). Specs live under `test/` mirroring source layout (`test/lib/*.test.js`, `test/api/*.test.js`, `test/components/*.test.js`). Last green: 231/231 tests across 43 files (2026-08-23). +- **Vitest coverage today:** 231 unit tests spanning auth (`lib/auth-secret.js`, `lib/permission-middleware.js::getUserFromRequest` incl. the negative regression against the old synthetic-admin shape — Gotcha #2), `pages/api/auth-utils.js`, Layout logged-out regressions (Gotcha #8), scanner libs/hooks/components (`use-scanner-identification`, `use-camera-scanner`, `ScannerCamera`, scanner page), card import + reconcile helpers, and catalog sync. The original fix-auth-bypass / fix-layout-default-user contract tests are still present — do not weaken them when refactoring auth or Layout. - **E2E / smoke runner:** `@playwright/test@^1.60.0` (installed via `adopt-playwright-smoke`, PR #18 squash `7b6f751`). Config in `playwright.config.js` (root, ESM) declares two projects: - `smoke` — `tests/smoke/**/*.spec.@(ts|js)`; invoked by `.github/workflows/preview-smoke.yml`. `npm run test:smoke` locally. - `visual` — `tests/visual/**/*.spec.@(ts|js)`; invoked by `.github/workflows/visual-diff.yml`. `npm run test:visual` locally; `npm run test:visual:update` to (re-)seed baselines. - Local-run convention: boot `next dev` in one terminal, then in another run `BASE_URL=http://localhost:3000 npm run test:smoke` (or against a deployed preview, `BASE_URL=https://.vercel.app VERCEL_AUTOMATION_BYPASS_SECRET= npm run test:smoke`). No `next dev` auto-boot in the test scripts (Decision 6 of `adopt-playwright-smoke`). + Local-run convention: boot `next dev` in one terminal, then in another run `BASE_URL=http://localhost:3000 npm run test:smoke`. CI defaults `BASE_URL` to the homelab deployment (`https://deckhearth.stillwell.cloud`) via `vars.SMOKE_BASE_URL`; a legacy Vercel-preview target still works with `BASE_URL=https://.vercel.app VERCEL_AUTOMATION_BYPASS_SECRET=` but is pending decommission (§ 7). No `next dev` auto-boot in the test scripts (Decision 6 of `adopt-playwright-smoke`). - **Browsers must be installed once locally:** `npx playwright install --with-deps chromium`. CI re-runs this on every workflow run (it's cached when possible). - **Visual baselines:** committed under `tests/visual/__screenshots__/`. The initial Linux baseline (`home.png`) was seeded by PR #58 (`83a358b`, 2026-06-02). Baselines are committed to git — they are not gitignored — so a `Screenshot diff` failure is reviewable from PR comments + artifacts without bouncing through a regeneration step. **Re-seeding** (when the homepage changes intentionally) MUST happen in a Linux environment so the PNG matches what CI produces. Recommended paths: @@ -197,7 +206,19 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560 ## 7. Deployment -- **Vercel** auto-deploys `main` and creates Preview deployments for every PR. `vercel.json` and `.vercel/` are committed. CI in `.github/workflows/` runs lint + types (no duplicate build — Vercel handles it). +> **Status (2026-08): production is the Dokploy homelab deployment** — +> app on CT 112, public URL `https://deckhearth.stillwell.cloud` via Traefik +> on CT 100, data plane on CT 102 (Postgres/Redis/MinIO). CI smoke + visual +> workflows already gate against it (`BASE_URL` defaults there). The +> **Vercel bullets below are legacy**, retained until `migrate-neon-to-homelab` +> phase 8 (decommission) lands — `vercel.json` / `.vercel/` are still in the +> tree and `VERCEL_AUTOMATION_BYPASS_SECRET` is only needed for the legacy +> `*.vercel.app` targets. Runbook: [`docs/DOKPLOY_DEPLOY.md`](docs/DOKPLOY_DEPLOY.md). + +- **Vercel (legacy, pending decommission).** Historically, Vercel + auto-deployed `main` and created Preview deployments for every PR. + `vercel.json` and `.vercel/` are committed but scheduled for removal in + phase 8 of `migrate-neon-to-homelab`. - **Preview protection bypass for automation.** The project has a Protection Bypass for Automation token exposed locally as `VERCEL_AUTOMATION_BYPASS_SECRET` in `.env.local` (not committed) and seeded into GitHub Actions as a repo secret (`gh secret set VERCEL_AUTOMATION_BYPASS_SECRET`, 2026-05-24). The secret is consumed in two shapes: 1. **Query parameter** on `wait-for-vercel-preview@v1.3.2`'s `path:` input in both `preview-smoke.yml` and `visual-diff.yml` — `path: '/?x-vercel-protection-bypass=…'`, bare form, **without** `&x-vercel-set-bypass-cookie=true` (the cookie variant returns 307 + Set-Cookie and axios in Node has no cookie jar, so it 401s on the redirect). Plumbed by PR #17 (`fix-vercel-deployment-protection-in-ci`, squash `9a3e077`). 2. **HTTP header** in `playwright.config.js`'s `use.extraHTTPHeaders` — `'x-vercel-protection-bypass': `. Playwright's browser context has a real cookie jar so this shape works there, and the testOptions surface forwards the header to the test-level `request` fixture's `APIRequestContext` as well, so both `page.goto(...)` calls and `request.get('/api/health')` calls hit the protected preview correctly without per-spec header injection. Plumbed by PR #18 (`adopt-playwright-smoke`, squash `7b6f751`) per Decision 2 of that convoy. @@ -206,7 +227,7 @@ Code graph is indexed by `user-code-review-graph` MCP (122 files, 628 nodes, 560 Do not log or echo the value. If the operator rotates the token in the Vercel dashboard, re-seed the GitHub secret via `gh secret set VERCEL_AUTOMATION_BYPASS_SECRET --body ""`. See `.convoys/fix-vercel-deployment-protection-in-ci.md` and `.convoys/adopt-playwright-smoke.md`. - **Shell-injection hardening in workflow YAML.** Never inline `${{ github.event.* }}` directly into a `run:` block — route the value through the step's `env:` block and quote it (`"$VAR_NAME"`) in shell. PR #17's CI validation caught a real syntax error from a PR body containing `(` because the gate-job's Decide step inlined `${{ github.event.pull_request.body }}` straight into bash; commit `b6f8688` swept both `preview-smoke.yml` and `visual-diff.yml` to the `env:` + quoted-shell pattern. This is GitHub's official Security Hardening guidance ("Security hardening for GitHub Actions" → "Using a third-party action"). Apply to any new workflow that reads PR body / title / branch name / commit messages in shell. -- **CI runs on the axiom homelab (CT 111).** Four of the five workflows execute on `[self-hosted, axiom]` runners managed in the `axiom-server` repo (`proxmox/ct111/`). Day-to-day this is invisible — pushes still trigger jobs and Vercel still builds previews — but two operational notes matter: +- **CI runs on the axiom homelab (CT 111).** Four of the five workflows execute on `[self-hosted, axiom]` runners managed in the `axiom-server` repo (`proxmox/ct111/`). Day-to-day this is invisible — pushes still trigger jobs, and Dokploy builds `main` on CT 112 (Vercel preview builds continue only until phase 8 decommission) — but two operational notes matter: 1. **PAT rotation.** The runners authenticate to GitHub via an org-scoped PAT stored on CT 111 at `/opt/appdata/gha-runner/.env` (key `GH_PAT`, scopes `admin:org`, `repo`, `workflow`). Rotate every 90 days. After updating the value on CT 111, run `./proxmox/scripts/sync.sh restart 111` to re-register all 4 runners. If the PAT lapses silently, new jobs fail registration immediately; check `./proxmox/scripts/sync.sh logs 111 gha-runner-1` for `Http response code: NotFound` to confirm. 2. **1-line revert path (D5) — when axiom is offline mid-PR-storm.** If CT 111 is down for maintenance, hardware swap, or any reason, and a hot fix needs CI to land, swap every `[self-hosted, axiom]` back to `ubuntu-latest`: diff --git a/README.md b/README.md index 6bc5e70..c282a64 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,10 @@ # Deck Hearth -A modern trading card game collection manager built with Next.js and Neon Database. +A modern trading card game collection manager built with Next.js and PostgreSQL. + +> Repo history note: this project was formerly **TCG Vault** (`tcg-vault`); +> the GitHub repo is now [`stwl-labs/deckhearth`](https://github.com/stwl-labs/deckhearth). +> Local checkout folders named `tcg-vault` are fine. ## 🚀 Features @@ -15,19 +19,20 @@ A modern trading card game collection manager built with Next.js and Neon Databa - **Frontend**: Next.js 16 (Pages router), React 18, JavaScript (TypeScript is a devDep only — see `AGENTS.md` Gotcha #9) - **Backend**: Next.js API Routes -- **Database**: Neon PostgreSQL (serverless) +- **Database**: PostgreSQL 17 + pgvector on the axiom homelab (CT 102, `192.168.68.102:5432`), accessed via the tagged-template helper in `lib/sql.js` — see `docs/HOMELAB_DATABASE.md` - **Authentication**: JWT with bcrypt (24-hour expiry; `lib/auth-secret.js` is the single source of truth for `JWT_SECRET`) -- **Rate limiting**: `@upstash/ratelimit` on `/api/auth/login` + `/api/auth/register` (5 attempts / 15 min per IP) -- **Testing**: Vitest (unit); Playwright queued -- **Styling**: Tailwind CSS -- **Deployment**: Vercel +- **Rate limiting**: `lib/rate-limit.js` — six named limiters (`auth`, `search`, `upload`, `generate`, `import`, `scan`) backed by `ioredis` + `rate-limiter-flexible` against homelab Redis (CT 102). Fails closed in production if `REDIS_URL` is unset; warn-and-no-op in dev +- **Testing**: Vitest (unit) + Playwright (`smoke` and `visual` projects) +- **Object storage**: MinIO on CT 102 (scan captures; S3-compatible via `@aws-sdk/client-s3`) +- **Styling**: Tailwind CSS + Liquid Glass design tokens (`docs/DESIGN_TOKENS.md`) +- **Deployment**: Dokploy on CT 112, public URL `https://deckhearth.stillwell.cloud` via Traefik — see `docs/DOKPLOY_DEPLOY.md` ## 📦 Installation 1. **Clone the repository** ```bash - git clone - cd tcg-vault + git clone https://github.com/stwl-labs/deckhearth.git + cd deckhearth ``` 2. **Install dependencies** @@ -35,27 +40,32 @@ A modern trading card game collection manager built with Next.js and Neon Databa npm install ``` -3. **Set up environment variables** - ```bash - cp .env.example .env.local - ``` - - Update `.env.local` with your Neon database URL and a real JWT secret: +3. **Create `.env.local`** (there is no committed template — use the shape below) ```env - POSTGRES_URL="postgresql://your-username:your-password@your-host/your-database" + # Required — homelab Postgres (CT 102) or any Postgres 17 instance + POSTGRES_URL="postgresql://deckhearth:@192.168.68.102:5432/deckhearth" + # Migrations (`npm run migrate`); on the homelab, same value as POSTGRES_URL + POSTGRES_URL_DIRECT="postgresql://deckhearth:@192.168.68.102:5432/deckhearth" JWT_SECRET="" # Required for `npm run setup-db` — used once to hash the initial admin password. - # Set in .env.local for local dev, or as a CI secret if you run setup from CI. ADMIN_INITIAL_PASSWORD="" - # Optional — exercise the rate limiter locally. Without them, `lib/rate-limit.js` - # warn-and-no-ops in dev. In production these are auto-provisioned by the - # Vercel Upstash Marketplace integration. - KV_REST_API_URL="https://.upstash.io" - KV_REST_API_TOKEN="" + # Rate limiting (homelab Redis, CT 102 — DB index 5 in prod). Optional in dev — + # without it, `lib/rate-limit.js` warn-and-no-ops locally; production fails closed. + REDIS_URL="redis://:password@192.168.68.102:6379/5" + # Scan-capture uploads (MinIO on CT 102, S3-compatible — see lib/object-storage.js) + S3_ENDPOINT="http://192.168.68.102:9000" + S3_BUCKET="deckhearth" + S3_ACCESS_KEY_ID="..." + S3_SECRET_ACCESS_KEY="..." + S3_PUBLIC_BASE_URL="https://cdn.stillwell.cloud/deckhearth" + S3_REGION="us-east-1" ``` `JWT_SECRET` is **required** — `lib/auth-secret.js` throws at import time if it's unset. `ADMIN_INITIAL_PASSWORD` is **required** for `npm run setup-db` — the script exits with code 1 if it's unset. + > Migrating data off the old Neon instance? `NEON_DATABASE_URL` is read once by + > `npm run migrate-neon-to-homelab`. See `docs/HOMELAB_DATABASE.md`. + 4. **Set up the database** ```bash npm run setup-db @@ -127,27 +137,35 @@ The application uses the following tables: ## 🚀 Deployment -This app is configured for deployment on Vercel: +Deck Hearth deploys to **Dokploy on CT 112** in the axiom homelab, fronted by +Traefik on CT 100 at `https://deckhearth.stillwell.cloud`. The full runbook — +Dokploy app settings, environment variables, Traefik route, and the n8n +catalog-sync cron that replaced Vercel Cron — lives in +[`docs/DOKPLOY_DEPLOY.md`](docs/DOKPLOY_DEPLOY.md). -1. **Connect your repository** to Vercel -2. **Set environment variables** in Vercel dashboard -3. **Deploy automatically** on push to main branch +> **Transition note:** the Vercel + Neon era is being decommissioned +> (`migrate-neon-to-homelab` convoy, phases 6–8). `vercel.json` and `.vercel/` +> remain in the tree until that decommission lands; CI already gates against +> the homelab deployment. ## 📁 Project Structure ``` -tcg-vault/ +deckhearth/ (formerly tcg-vault) ├── pages/ # Next.js pages and API routes │ ├── api/ # API endpoints │ │ ├── auth/ # Authentication routes │ │ └── admin/ # Admin routes │ ├── _app.js # App wrapper │ └── index.js # Home page -├── lib/ # Utility libraries -│ └── database.js # Database adapter -├── scripts/ # Database setup scripts -├── public/ # Static assets -└── .env.local # Environment variables +├── components/ # React components (+ scanner/ subfamily) +├── lib/ # Utility libraries +│ └── sql.js # Canonical Postgres client (tagged templates) +├── migrations/ # node-pg-migrate migrations (source of truth for DDL) +├── scripts/ # Setup, import, and historical one-off jobs +├── docs/ # Deploy / DB / design-token runbooks +├── test/ + tests/ # Vitest unit specs; Playwright smoke + visual specs +└── .env.local # Environment variables (not committed) ``` ## 🔐 First-time admin setup @@ -161,7 +179,7 @@ variable is unset or empty. - **Local dev:** set `ADMIN_INITIAL_PASSWORD` in `.env.local` before running `npm run setup-db`. Use `openssl rand -base64 24` (or any other strong source) to generate the value. -- **CI / Vercel:** set `ADMIN_INITIAL_PASSWORD` as a project secret if setup +- **CI:** set `ADMIN_INITIAL_PASSWORD` as a repo secret if setup ever runs from CI. The env var is **only** read by the seed script; runtime auth uses the per-user password stored in the database. - **Admin email:** the seed creates `admin@deckhearth.com`. Change the password