9bde5960bf
- Read NEXT_PUBLIC_COLLAB_SERVER_URL in yjs provider; bake NEXT_PUBLIC_* at Docker build - Add prod compose, deploy scripts, CT 116 proxmox scaffolding - Forgejo Actions CI/deploy; migrate-origin and post-migration scripts
169 lines
6.1 KiB
Markdown
169 lines
6.1 KiB
Markdown
# Echodo / Submatter — Production Deployment
|
||
|
||
Production runs on **CT 116** (`submatter`, `192.168.68.116`) as plain Docker Compose — not Dokploy.
|
||
|
||
| Doc | Purpose |
|
||
|-----|---------|
|
||
| This file | Operator checklist, env vars, troubleshooting |
|
||
| [`docs/forgejo-migration.md`](forgejo-migration.md) | GitHub → Forgejo + CT 116 cutover |
|
||
| [`proxmox/ct116/README.md`](../proxmox/ct116/README.md) | CT creation, Traefik, auto-deploy, Dokploy removal |
|
||
| `AGENT-DEPLOY.md` (local, gitignored) | Homelab credentials + CT map — add CT 116 row after cutover |
|
||
|
||
---
|
||
|
||
## Why a dedicated CT (not Dokploy)?
|
||
|
||
Echodo builds **five Docker images** from one monorepo; the Next.js compile alone reserves a 5 GB V8 heap. On Dokploy (CT 112), that competes with Zest, Umami, Comms, etc. and OOMs with exit 255.
|
||
|
||
**CT 116** gets the full 8 GB for serialized builds. CT 107 is not an alternative — it still hosts Coolify, Echo Board, Resonance, and other dev apps.
|
||
|
||
| Resource | CT 116 spec |
|
||
|----------|-------------|
|
||
| RAM | 8 GB |
|
||
| CPU | 2 cores |
|
||
| Disk | 32 GB rootfs |
|
||
| Runtime (Phase A) | ~3–4 GB |
|
||
| Phase B (+ OpenHands) | +2 GB |
|
||
|
||
External: Postgres/Redis on **CT 102**, TLS on **CT 100**, OmniRoute on **CT 108**.
|
||
|
||
---
|
||
|
||
## Stack files
|
||
|
||
| File | Use |
|
||
|------|-----|
|
||
| `docker/docker-compose.prod.yml` | Core stack |
|
||
| `docker/docker-compose.prod.standalone.yml` | Publish `:3000` / `:1234` for CT 100 Traefik |
|
||
| `scripts/build-prod.sh` | Serialized image build |
|
||
| `scripts/deploy-prod.sh` | Build + up + health wait |
|
||
|
||
```bash
|
||
docker compose -f docker/docker-compose.prod.yml \
|
||
-f docker/docker-compose.prod.standalone.yml up -d
|
||
```
|
||
|
||
---
|
||
|
||
## Phase A — First deploy on CT 116
|
||
|
||
1. **Create CT 116** — see [`proxmox/ct116/README.md`](../proxmox/ct116/README.md) (`pct create` one-liner).
|
||
2. **Bootstrap** — `bootstrap.sh` installs Docker, clones to `/opt/submatter`.
|
||
3. **Configure `.env`** — copy from `.env.example`; use `echodo` database on CT 102, Redis `/11`.
|
||
4. **Deploy** — `bash scripts/deploy-prod.sh build`
|
||
5. **Traefik** — apply `proxmox/ct116/traefik-ct100.example.yml` on CT 100; remove old Dokploy/Coolify Echodo routes.
|
||
6. **Verify** — `https://echodo.stillwell.cloud`, `wss://collab.echodo.stillwell.cloud`
|
||
|
||
### Required env vars
|
||
|
||
| Variable | Production example | Notes |
|
||
|----------|-------------------|-------|
|
||
| `DATABASE_URL` | `postgresql://echodo:***@192.168.68.102:5432/echodo` | |
|
||
| `REDIS_URL` | `redis://:***@192.168.68.102:6379/11` | |
|
||
| `NEXTAUTH_URL` | `https://echodo.stillwell.cloud` | |
|
||
| `NEXTAUTH_SECRET` / `AUTH_SECRET` | `openssl rand -base64 32` | |
|
||
| `AUTH_AUTHENTIK_*` | | Authentik OIDC app `echodo` |
|
||
| `COLLAB_SERVER_URL` | `wss://collab.echodo.stillwell.cloud` | Server-side |
|
||
| `NEXT_PUBLIC_COLLAB_SERVER_URL` | `wss://collab.echodo.stillwell.cloud` | **Build arg** — requires image rebuild |
|
||
| `OMNIROUTE_BASE_URL` | `https://omniroute.stillwell.cloud/v1` | Web + worker |
|
||
| `OMNIROUTE_API_KEY` | (your key) | + register in Settings → AI → Providers |
|
||
| `FORGEJO_TOKEN` / `GITHUB_TOKEN` | PAT values | Referenced by name in Connected repos settings |
|
||
| `NEXT_PUBLIC_UMAMI_*` | | Analytics (CT 112 Umami) |
|
||
|
||
Changing any `NEXT_PUBLIC_*` value requires `bash scripts/deploy-prod.sh build`, not just `docker compose restart`.
|
||
|
||
---
|
||
|
||
## Auto-deploy on push
|
||
|
||
**CI** (every PR): `.github/workflows/ci.yml` — lint, type-check, test on GitHub-hosted or PR runner.
|
||
|
||
**CD** (merge to `main`): `.github/workflows/deploy-homelab.yml` and `.forgejo/workflows/deploy-homelab.yml` — SSH from CT 111 `axiom` runner → CT 116 → `deploy-prod.sh build`.
|
||
|
||
### GitHub / Forgejo repo secrets
|
||
|
||
| Secret | Value |
|
||
|--------|-------|
|
||
| `HOMELAB_DEPLOY_HOST` | `192.168.68.116` |
|
||
| `HOMELAB_DEPLOY_USER` | `root` |
|
||
| `HOMELAB_DEPLOY_SSH_KEY` | SSH private key (ed25519) |
|
||
| `HOMELAB_DEPLOY_PATH` | `/opt/submatter` (optional) |
|
||
|
||
Generate deploy key on CT 116:
|
||
|
||
```bash
|
||
ssh-keygen -t ed25519 -f /root/.ssh/github_deploy -N ""
|
||
cat /root/.ssh/github_deploy.pub >> /root/.ssh/authorized_keys
|
||
# Paste private key into HOMELAB_DEPLOY_SSH_KEY secret
|
||
```
|
||
|
||
Until secrets are set, the deploy workflow skips gracefully (CI still runs).
|
||
|
||
### Forgejo webhook (optional second trigger)
|
||
|
||
If Forgejo is canonical git and you want push deploy without Actions:
|
||
|
||
1. Install [webhook](https://github.com/adhocore/webhook) on CT 116.
|
||
2. Copy `proxmox/ct116/webhook.hooks.json.example` → `/etc/webhook.conf`.
|
||
3. Forgejo repo → Webhook → POST push events → `http://192.168.68.116:9000/hooks/submatter-deploy`.
|
||
|
||
---
|
||
|
||
## Remove from Dokploy
|
||
|
||
1. Dokploy UI → Submatter/Echodo compose app → Stop → Delete.
|
||
2. CT 100 — remove Traefik routes to Dokploy for `echodo.stillwell.cloud` / collab hostname.
|
||
3. Update local `AGENT-DEPLOY.md`: add CT 116 row, note Echodo removed from CT 112.
|
||
|
||
---
|
||
|
||
## Phase B — OpenHands (optional)
|
||
|
||
```bash
|
||
export COMPOSE_PROFILES=agent-server
|
||
bash scripts/deploy-prod.sh build
|
||
```
|
||
|
||
| Variable | Notes |
|
||
|----------|-------|
|
||
| `LOCAL_BACKEND_API_KEY` | `openssl rand -hex 32` |
|
||
| `SESSION_API_KEY` | Same as above (worker) |
|
||
| `WORKER_SESSION_TYPE` | `openhands` |
|
||
| `OMNIROUTE_BASE_URL` | `https://omniroute.stillwell.cloud/v1` |
|
||
| `OMNIROUTE_API_KEY` | OmniRoute bearer key |
|
||
|
||
Requires Docker socket on CT 116 (`nesting=1` on LXC).
|
||
|
||
---
|
||
|
||
## Troubleshooting
|
||
|
||
| Symptom | Fix |
|
||
|---------|-----|
|
||
| Build exit 255 ~60–90s | Ensure `deploy-prod.sh` uses `build-prod.sh` (serialized) |
|
||
| OAuth broken | DB must be `echodo`, not legacy `tasks` |
|
||
| Deploy workflow skipped | Set `HOMELAB_DEPLOY_*` secrets |
|
||
| Collab websocket fails | Traefik route to `:1234`, `wss://` in public env vars |
|
||
| OpenHands unavailable | `SESSION_API_KEY` must match `LOCAL_BACKEND_API_KEY` |
|
||
|
||
Local build verify: `pnpm docker:build:prod`
|
||
|
||
---
|
||
|
||
## Future: registry pull deploy (no build on CT 116)
|
||
|
||
Build images on CT 111 CI, push to Forgejo container registry, then on CT 116:
|
||
|
||
```bash
|
||
bash scripts/deploy-prod.sh pull
|
||
```
|
||
|
||
Requires compose `image:` tags instead of `build:` — not implemented yet; track as follow-on if CT 116 builds remain too slow.
|
||
|
||
---
|
||
|
||
## Related
|
||
|
||
- `docs/AGENT-AUTONOMOUS-CONVOYS.md` — convoy + OpenHands roadmap
|
||
- `docs/omniroute-setup.md` — OmniRoute operator guide
|
||
- `.env.example` — full variable list
|