fix(deploy): collab URL, CT 116 prod compose, Forgejo migration
Some checks failed
CI / Lint, type-check, test (push) Has been cancelled
Deploy homelab (CT 116) / Deploy to CT 116 (push) Has been cancelled

- Read NEXT_PUBLIC_COLLAB_SERVER_URL in yjs provider; bake NEXT_PUBLIC_* at Docker build
- Add prod compose, deploy scripts, CT 116 proxmox scaffolding
- Forgejo Actions CI/deploy; migrate-origin and post-migration scripts
This commit is contained in:
Randall Stillwell 2026-09-08 22:33:59 -05:00
parent 4efaf6c0c8
commit 9bde5960bf
31 changed files with 1404 additions and 317 deletions

View file

@ -2,11 +2,9 @@
# ECHODO — Environment Variables
# =====================================================================
# Local dev: copy this file to .env and adjust values.
# Coolify (CT 107): paste these into the resource's "Environment Variables"
# tab. Coolify also accepts a .env file uploaded via the UI.
#
# See AGENT-DEPLOY.md for shared CT 102 service credentials and the
# CT 100 Traefik routing pattern.
# Production (CT 116 standalone LXC): paste into /opt/submatter/.env on the host.
# See docs/deployment-production.md and proxmox/ct116/README.md.
# See AGENT-DEPLOY.md for shared CT 102 service credentials and Traefik routing.
# =====================================================================
# ----- Database (CT 102 shared Postgres) -----
@ -44,6 +42,16 @@ REDIS_URL="redis://:CHANGE_ME@192.168.68.102:6379/11"
# WORKER_CHECKPOINT_EVERY_N_STEPS=1
# WORKER_REPO_ROOT= # defaults to monorepo root (pnpm-workspace.yaml walk-up)
# WORKER_WORKTREE_BASE=/tmp/echodo-worker-worktrees
#
# Phase B — OpenHands agent-server (optional; requires --profile agent-server)
# WORKER_SESSION_TYPE=minimal # set to `openhands` when agent-server is running
# AGENT_SERVER_URL=http://agent-server:8001
# WORKER_OPENHANDS_WORKSPACE=/workspace/repo
# SESSION_API_KEY= # must match LOCAL_BACKEND_API_KEY on agent-server
# LOCAL_BACKEND_API_KEY= # generate: openssl rand -hex 32
# OMNIROUTE_BASE_URL=https://omniroute.stillwell.cloud/v1
# OMNIROUTE_API_KEY=
# OMNIROUTE_MODEL=auto/coding
# ----- Auth (NextAuth v5) -----
# Local: http://localhost:3000 • Prod: https://echodo.stillwell.cloud
@ -72,10 +80,11 @@ AUTH_AUTHENTIK_ISSUER="https://auth.stillwell.cloud/application/o/echodo/"
# ----- Realtime collab server (Hocuspocus / Yjs) -----
# Server-side reference (used by Next.js server actions, if any)
COLLAB_SERVER_URL="ws://localhost:1234"
# Browser-facing — must be wss:// in production because the page is https
# Browser-facing — must be wss:// in production because the page is https.
# Baked into the web image at BUILD time (docker build args) — change requires rebuild.
NEXT_PUBLIC_COLLAB_SERVER_URL="ws://localhost:1234"
# ----- Umami Analytics (CT 107 — analytics.stillwell.cloud) -----
# ----- Umami Analytics (CT 112 — analytics.stillwell.cloud) -----
# In Umami: Settings → Websites → Add website. Copy the website UUID.
NEXT_PUBLIC_UMAMI_SCRIPT="https://analytics.stillwell.cloud/script.js"
NEXT_PUBLIC_UMAMI_WEBSITE_ID=""
@ -84,7 +93,7 @@ NEXT_PUBLIC_UMAMI_WEBSITE_ID=""
NEXT_PUBLIC_LIBREDESK_URL="https://support.stillwell.cloud"
NEXT_PUBLIC_LIBREDESK_WIDGET_ID=""
# ----- Directus Feedback (CT 107 — directus.stillwell.cloud) -----
# ----- Directus Feedback (archived — was CT 107) -----
# Static token scoped write-only on the `feedback` collection
NEXT_PUBLIC_DIRECTUS_URL="https://directus.stillwell.cloud"
DIRECTUS_FEEDBACK_TOKEN=""
@ -110,20 +119,40 @@ ANTHROPIC_API_KEY=""
GOOGLE_GENERATIVE_AI_API_KEY=""
NVIDIA_API_KEY=""
# ----- OmniRoute (CT 108 — omniroute.stillwell.cloud) -----
# Used by worker + web (Settings → AI provider tests) + Phase B agent-server.
OMNIROUTE_BASE_URL="https://omniroute.stillwell.cloud/v1"
OMNIROUTE_API_KEY=""
OMNIROUTE_MODEL=auto/coding
# ----- Connected repos (Settings → Connected repos) -----
# Token values live here; Settings stores only the env var NAME (e.g. FORGEJO_TOKEN).
# Forgejo self-hosted:
# Remote URL: https://forgejo.stillwell.cloud/rstillwell/Submatter.git
# Token env var name in UI: FORGEJO_TOKEN
FORGEJO_TOKEN=""
# GitHub.com:
# Remote URL: https://github.com/org/repo.git
# Token env var name in UI: GITHUB_TOKEN
GITHUB_TOKEN=""
# ----- Forge webhooks (PR merged → convoy transition) -----
FORGE_WEBHOOK_SECRET=""
# ----- MCP Server -----
MCP_SERVER_PORT=3001
# =====================================================================
# Coolify Docker Compose build (8 GB hosts)
# Production Docker Compose build (8 GB hosts)
# =====================================================================
# Serialize image builds to avoid kernel OOM (exit 255) during export.
# Coolify (8 GB host): set Custom Build Command to `bash scripts/build-coolify.sh`
# so compose builds collab → mcp → orchestrator → worker → web sequentially.
# COMPOSE_PARALLEL_LIMIT is documented for reference but the script is required.
# Dokploy / standalone: set Custom Build Command to `bash scripts/build-prod.sh`
# Local verify: pnpm docker:build:prod
# Legacy alias: pnpm docker:build:coolify
# =====================================================================
# Local Docker Compose only (docker/docker-compose.yml)
# Not needed for Coolify (Coolify uses CT 102's shared Postgres/Redis).
# Not needed for production (uses CT 102 shared Postgres/Redis).
# =====================================================================
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres

41
.forgejo/workflows/ci.yml Normal file
View file

@ -0,0 +1,41 @@
name: CI
on:
pull_request:
push:
branches: [main]
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
jobs:
check:
name: Lint, type-check, test
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up pnpm
uses: pnpm/action-setup@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 20
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Lint
run: pnpm lint
- name: Type-check
run: pnpm type-check
- name: Test
run: pnpm test

View file

@ -0,0 +1,40 @@
name: Deploy homelab (CT 116)
on:
push:
branches: [main]
workflow_dispatch:
concurrency:
group: deploy-homelab-main
cancel-in-progress: true
jobs:
deploy:
name: Deploy to CT 116
runs-on: [self-hosted, linux, axiom]
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Sync and deploy over SSH
env:
DEPLOY_HOST: ${{ secrets.HOMELAB_DEPLOY_HOST }}
DEPLOY_USER: ${{ secrets.HOMELAB_DEPLOY_USER }}
DEPLOY_PATH: ${{ secrets.HOMELAB_DEPLOY_PATH }}
DEPLOY_SSH_KEY: ${{ secrets.HOMELAB_DEPLOY_SSH_KEY }}
run: |
set -euo pipefail
if [[ -z "${DEPLOY_HOST:-}" || -z "${DEPLOY_USER:-}" || -z "${DEPLOY_SSH_KEY:-}" ]]; then
echo "Skipping deploy: set HOMELAB_DEPLOY_HOST, HOMELAB_DEPLOY_USER, HOMELAB_DEPLOY_SSH_KEY repo secrets."
exit 0
fi
deploy_path="${DEPLOY_PATH:-/opt/submatter}"
mkdir -p ~/.ssh
printf '%s\n' "$DEPLOY_SSH_KEY" > ~/.ssh/deploy_key
chmod 600 ~/.ssh/deploy_key
ssh -i ~/.ssh/deploy_key -o StrictHostKeyChecking=accept-new \
"${DEPLOY_USER}@${DEPLOY_HOST}" \
"set -euo pipefail; cd '${deploy_path}' && bash scripts/git-sync-deploy.sh && bash scripts/deploy-prod.sh build"

41
.github/workflows/deploy-homelab.yml vendored Normal file
View file

@ -0,0 +1,41 @@
name: Deploy homelab (CT 116)
on:
push:
branches: [main]
workflow_dispatch:
# CI already ran on the PR; main deploy is serialized and can take 15–30 min on axiom.
concurrency:
group: deploy-homelab-main
cancel-in-progress: true
jobs:
deploy:
name: Deploy to CT 116
runs-on: [self-hosted, linux, axiom]
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Sync and deploy over SSH
env:
DEPLOY_HOST: ${{ secrets.HOMELAB_DEPLOY_HOST }}
DEPLOY_USER: ${{ secrets.HOMELAB_DEPLOY_USER }}
DEPLOY_PATH: ${{ secrets.HOMELAB_DEPLOY_PATH }}
DEPLOY_SSH_KEY: ${{ secrets.HOMELAB_DEPLOY_SSH_KEY }}
run: |
set -euo pipefail
if [[ -z "${DEPLOY_HOST:-}" || -z "${DEPLOY_USER:-}" || -z "${DEPLOY_SSH_KEY:-}" ]]; then
echo "Skipping deploy: set HOMELAB_DEPLOY_HOST, HOMELAB_DEPLOY_USER, HOMELAB_DEPLOY_SSH_KEY repo secrets."
exit 0
fi
deploy_path="${DEPLOY_PATH:-/opt/submatter}"
mkdir -p ~/.ssh
printf '%s\n' "$DEPLOY_SSH_KEY" > ~/.ssh/deploy_key
chmod 600 ~/.ssh/deploy_key
ssh -i ~/.ssh/deploy_key -o StrictHostKeyChecking=accept-new \
"${DEPLOY_USER}@${DEPLOY_HOST}" \
"set -euo pipefail; cd '${deploy_path}' && bash scripts/git-sync-deploy.sh && bash scripts/deploy-prod.sh build"

View file

@ -73,6 +73,7 @@ export default function ConnectedReposSettingsPage() {
setParsedPreview(null);
},
});
const [connectError, setConnectError] = React.useState<string | null>(null);
const disconnectMut = api.connectedRepos.disconnect.useMutation({
onSuccess: () => void utils.connectedRepos.list.invalidate(),
@ -98,6 +99,7 @@ export default function ConnectedReposSettingsPage() {
const openCreate = () => {
setForm(EMPTY_FORM);
setParsedPreview(null);
setConnectError(null);
setDialogOpen(true);
};
@ -112,12 +114,18 @@ export default function ConnectedReposSettingsPage() {
if (!workspaceSlug || !form.remoteUrl.trim()) {
return;
}
connectMut.mutate({
workspace: workspaceSlug,
remoteUrl: form.remoteUrl.trim(),
displayName: form.displayName.trim() || undefined,
tokenEnvVar: form.tokenEnvVar.trim() || null,
});
setConnectError(null);
connectMut.mutate(
{
workspace: workspaceSlug,
remoteUrl: form.remoteUrl.trim(),
displayName: form.displayName.trim() || undefined,
tokenEnvVar: form.tokenEnvVar.trim() || null,
},
{
onError: (err) => setConnectError(err.message),
},
);
};
return (
@ -126,9 +134,27 @@ export default function ConnectedReposSettingsPage() {
<div>
<h1 className="text-2xl font-semibold tracking-tight">Connected repos</h1>
<p className="mt-1 text-sm text-muted-foreground">
Paste a git remote URL to link Forgejo, GitHub, or other forges. API tokens are
referenced by env var name — never stored in the database.
Link a git remote for worker automation (draft PRs, forge webhooks). Paste the clone
URL — OAuth repo pickers are planned; today you set the API token on the server and
reference its env var name here.
</p>
<details className="mt-3 text-sm text-muted-foreground">
<summary className="cursor-pointer font-medium text-foreground">Examples</summary>
<ul className="mt-2 list-disc space-y-2 pl-5">
<li>
<span className="font-medium text-foreground">Forgejo (self-hosted):</span>{" "}
<code className="text-xs">https://forgejo.stillwell.cloud/rstillwell/Submatter.git</code>
{" — "}token env var <code className="text-xs">FORGEJO_TOKEN</code> in{" "}
<code className="text-xs">/opt/submatter/.env</code>, then click Test connection.
</li>
<li>
<span className="font-medium text-foreground">GitHub.com:</span>{" "}
<code className="text-xs">https://github.com/org/repo.git</code>
{" — "}token env var <code className="text-xs">GITHUB_TOKEN</code> (PAT with repo
scope).
</li>
</ul>
</details>
</div>
{canManage && (
<Button type="button" onClick={openCreate}>
@ -269,16 +295,24 @@ export default function ConnectedReposSettingsPage() {
<div>
<label className="mb-1 block text-sm font-medium" htmlFor="token-env">
API token env var (optional)
API token env var
</label>
<Input
id="token-env"
placeholder="GITHUB_TOKEN"
placeholder="FORGEJO_TOKEN or GITHUB_TOKEN"
value={form.tokenEnvVar}
onChange={(e) => setForm((f) => ({ ...f, tokenEnvVar: e.target.value }))}
/>
<p className="mt-1 text-xs text-muted-foreground">
Name of the variable in the server <code className="text-xs">.env</code> — not the
secret itself. Required for Test connection to succeed.
</p>
</div>
{connectError && (
<p className="text-sm text-destructive">{connectError}</p>
)}
<div className="flex justify-end gap-2 pt-2">
<DialogPrimitive.Close asChild>
<Button type="button" variant="outline">

View file

@ -83,10 +83,6 @@ function releaseEntry(documentName: string) {
}
}
/**
* WebSocket URL for the collaboration server.
* Uses `NEXT_PUBLIC_COLLAB_URL`, then `COLLAB_SERVER_URL`, then ws://localhost:1234.
*/
/**
* Ask the cached provider for this document to open the WebSocket again (e.g. user clicked Retry).
*/
@ -98,12 +94,19 @@ export function reconnectCollaboration(documentName: string) {
}
}
/**
* WebSocket URL for the collaboration server.
* Reads `NEXT_PUBLIC_COLLAB_SERVER_URL` (deploy standard), legacy `NEXT_PUBLIC_COLLAB_URL`,
* then falls back to ws://localhost:1234 for local dev.
*
* Must be baked at Docker build time — runtime-only env changes do not update the client bundle.
*/
export function getCollabUrl(): string {
if (typeof process !== "undefined" && process.env.NEXT_PUBLIC_COLLAB_URL) {
return process.env.NEXT_PUBLIC_COLLAB_URL;
}
if (typeof process !== "undefined" && process.env.COLLAB_SERVER_URL) {
return process.env.COLLAB_SERVER_URL;
const fromEnv =
(typeof process !== "undefined" && process.env.NEXT_PUBLIC_COLLAB_SERVER_URL) ||
(typeof process !== "undefined" && process.env.NEXT_PUBLIC_COLLAB_URL);
if (fromEnv) {
return fromEnv;
}
return "ws://localhost:1234";
}

View file

@ -43,7 +43,12 @@ const envSchema = z.object({
.string()
.optional()
.transform((value) => value?.trim() || "http://agent-server:8001"),
/** OpenHands session API key for agent-server auth. */
/** Shared workspace path visible to agent-server sandboxes (volume-mounted in prod). */
WORKER_OPENHANDS_WORKSPACE: z
.string()
.optional()
.transform((value) => value?.trim() || "/workspace/repo"),
/** OpenHands session API key for agent-server auth. Must match LOCAL_BACKEND_API_KEY. */
SESSION_API_KEY: z
.string()
.optional()

View file

@ -34,7 +34,7 @@ export function createRemoteAgUiExecutor(
return {
id: "remote-ag-ui",
harnessType: "remote-ag-ui" as HarnessType,
async run(ctx: WorkerJobContext, opts: WorkerExecutorOptions): Promise<{ summary: string; toolCalls: number; tokensTotal?: number }> {
async run(ctx: WorkerJobContext, opts: WorkerExecutorOptions): Promise<{ summary: string; toolCalls: number; tokensTotal?: number; outcome?: "succeeded" | "failed" }> {
const authToken = deps.getEnvVar(config.authEnvVar);
if (!authToken) {
return {

View file

@ -87,6 +87,10 @@ const env = {
WORKER_MAX_TOOL_STEPS: 12,
WORKER_CHECKPOINT_EVERY_N_STEPS: 1,
WORKER_WORKTREE_BASE: "/tmp/echodo-worker-worktrees",
WORKER_SESSION_TYPE: "minimal" as const,
AGENT_SERVER_URL: "http://agent-server:8001",
WORKER_OPENHANDS_WORKSPACE: "/workspace/repo",
SESSION_API_KEY: "",
REDIS_URL: "redis://localhost:6379",
};

View file

@ -262,7 +262,7 @@ export async function handleOrchestratorJob(
const ohExecutor = createOpenHandsExecutor(
env.AGENT_SERVER_URL,
env.SESSION_API_KEY,
monorepoRoot,
env.WORKER_OPENHANDS_WORKSPACE,
);
if (ohExecutor) {
executor = ohExecutor;

View file

@ -129,6 +129,21 @@ ENV NODE_OPTIONS=--max-old-space-size=5120
# Local dev uses a gitignored apps/web/.env symlink; drop it so Next.js does not
# follow a broken link inside the container.
RUN rm -f apps/web/.env apps/web/.env.local apps/web/.env.production apps/web/.env.development
# NEXT_PUBLIC_* must be present at build time — Next.js inlines them into the client bundle.
ARG NEXT_PUBLIC_COLLAB_SERVER_URL=ws://localhost:1234
ARG NEXT_PUBLIC_UMAMI_SCRIPT=
ARG NEXT_PUBLIC_UMAMI_WEBSITE_ID=
ARG NEXT_PUBLIC_LIBREDESK_URL=
ARG NEXT_PUBLIC_LIBREDESK_WIDGET_ID=
ARG NEXT_PUBLIC_DIRECTUS_URL=
ENV NEXT_PUBLIC_COLLAB_SERVER_URL=${NEXT_PUBLIC_COLLAB_SERVER_URL}
ENV NEXT_PUBLIC_UMAMI_SCRIPT=${NEXT_PUBLIC_UMAMI_SCRIPT}
ENV NEXT_PUBLIC_UMAMI_WEBSITE_ID=${NEXT_PUBLIC_UMAMI_WEBSITE_ID}
ENV NEXT_PUBLIC_LIBREDESK_URL=${NEXT_PUBLIC_LIBREDESK_URL}
ENV NEXT_PUBLIC_LIBREDESK_WIDGET_ID=${NEXT_PUBLIC_LIBREDESK_WIDGET_ID}
ENV NEXT_PUBLIC_DIRECTUS_URL=${NEXT_PUBLIC_DIRECTUS_URL}
RUN pnpm exec turbo build --filter=@tasks/web
@ -277,8 +292,11 @@ COPY --chown=worker:nodejs packages ./repo/packages
COPY --chown=worker:nodejs plans ./repo/plans
COPY --chown=worker:nodejs .cursor/mcp.json ./repo/.cursor/mcp.json
COPY --chown=worker:nodejs .cursor/rules ./repo/.cursor/rules
COPY --chown=worker:nodejs .agents/skills ./repo/.agents/skills
RUN mkdir -p /app/repo/.cursor/agents
COPY --chown=worker:nodejs pnpm-workspace.yaml package.json turbo.json tsconfig.json ./repo/
COPY --chown=worker:nodejs docker/worker-entrypoint.sh /app/worker-entrypoint.sh
RUN chmod +x /app/worker-entrypoint.sh
# Minimal git repo so worktrees + allowlisted git commands work on the deploy snapshot.
USER root
@ -295,4 +313,4 @@ HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
CMD pgrep -f "node.*dist/index\\.js" > /dev/null || exit 1
WORKDIR /app/apps/worker
CMD ["node", "dist/index.js"]
ENTRYPOINT ["/app/worker-entrypoint.sh"]

View file

@ -1,51 +1,12 @@
# OpenHands Agent Server for Submatter
# Deploys on Dokploy (CT 102) alongside the existing stack
# Worker connects via internal Docker DNS: http://agent-server:8001
version: '3.8'
services:
agent-server:
image: ghcr.io/open-hands/agent-server:0.19.0 # pin to stable version
container_name: echodo-agent-server
ports:
- "8001:8001" # agent-server API
environment:
# Server config
- AGENT_SERVER_HOST=0.0.0.0
- AGENT_SERVER_PORT=8001
- LOCAL_BACKEND_API_KEY=${LOCAL_BACKEND_API_KEY}
# LLM config for the agent-server's own operations
- LLM_MODEL=${OMNIROUTE_MODEL:-auto/coding}
- LLM_API_KEY=${OMNIROUTE_API_KEY}
- LLM_BASE_URL=${OMNIROUTE_BASE_URL}
# Sandbox config
- SANDBOX_TYPE=docker
- SANDBOX_DOCKER_IMAGE=ghcr.io/open-hands/sandbox:latest
- SANDBOX_TIMEOUT=300
# Workspace persistence
- WORKSPACE_BASE=/workspace
volumes:
- agent-server-workspace:/workspace
- /var/run/docker.sock:/var/run/docker.sock # for sandbox containers
networks:
- echodo-network
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8001/server_info"]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
deploy:
resources:
limits:
memory: 2G
cpus: '2'
volumes:
agent-server-workspace:
networks:
echodo-network:
external: true
# DEPRECATED — OpenHands agent-server now lives in docker/docker-compose.prod.yml
# under the `agent-server` profile (same `echodo` network as worker).
#
# Enable Phase B:
# docker compose -f docker/docker-compose.prod.yml --profile agent-server up -d
#
# Required env (same values on worker + agent-server):
# LOCAL_BACKEND_API_KEY=<openssl rand -hex 32>
# SESSION_API_KEY=<same as LOCAL_BACKEND_API_KEY>
# WORKER_SESSION_TYPE=openhands
# OMNIROUTE_BASE_URL=https://omniroute.stillwell.cloud/v1
# OMNIROUTE_API_KEY=<key>

View file

@ -1,206 +1,7 @@
# Docker Compose for Coolify deployment of ECHODO.
# DEPRECATED — use docker/docker-compose.prod.yml
#
# Usage in Coolify:
# - New Resource → Docker Compose → connect this repo
# - Build pack: Docker Compose
# - Compose file path: docker/docker-compose.coolify.yml
# - Set domains in Coolify on the `web` and `collab` services using http://
# (CT 100 Traefik handles TLS termination per AGENT-DEPLOY.md)
#
# Path conventions:
# Coolify invokes `docker compose` with --project-directory set to the
# repository root, NOT this file's directory. So all build contexts and
# dockerfile paths below are written relative to the repo root.
#
# Differences vs. docker/docker-compose.yml:
# - No bundled postgres / redis (uses shared CT 102 services)
# - No host port mappings (Coolify's Traefik routes by container labels)
# - Uses Coolify SERVICE_FQDN_* magic env vars so Coolify auto-wires Traefik labels
#
# Build strategy:
# All five services share docker/Dockerfile and select runtime via `target:`.
# The shared `deps` stage is hashed identically across targets, so BuildKit
# runs `pnpm install` ONCE.
#
# The heavy `next build` for `web` (peak ~3 GB heap with 5 GB headroom)
# must not race the two tsup builds for collab+mcp on the 8 GB Coolify
# host or the kernel OOM-kills it (exit 255 with no Docker error). Build
# ordering is enforced inside docker/Dockerfile: the `web-build` stage
# has explicit `COPY --from=collab-build` / `COPY --from=mcp-build` lines
# that force BuildKit's DAG to finish those two before starting the web
# compile. The `depends_on` below only orders runtime startup, NOT image
# builds — that was a misconception from an earlier iteration of this
# file.
#
# Runtime image export still builds five targets in parallel by default,
# which OOMs 8 GB Coolify hosts (exit 255, no Docker error). Required:
# Coolify → Advanced → Custom Build Command: bash scripts/build-coolify.sh
# (COMPOSE_PARALLEL_LIMIT alone is not reliable across Compose versions.)
# Local verify: pnpm docker:build:coolify
# Kept so existing Coolify resources that still point at this path keep working.
# New Dokploy / standalone deploys should use docker-compose.prod.yml.
services:
web:
build:
context: .
dockerfile: docker/Dockerfile
target: web
depends_on:
- collab
- mcp
restart: unless-stopped
environment:
# Coolify FQDN magic var — set the domain in the Coolify UI for this service.
# SERVICE_FQDN_WEB_3000 makes Coolify expose the container on port 3000 via Traefik.
- SERVICE_FQDN_WEB_3000
- DATABASE_URL=${DATABASE_URL}
- REDIS_URL=${REDIS_URL}
- NEXTAUTH_URL=${NEXTAUTH_URL}
- NEXTAUTH_SECRET=${NEXTAUTH_SECRET}
- AUTH_SECRET=${AUTH_SECRET}
- AUTH_DEV_PASSWORD=${AUTH_DEV_PASSWORD}
- AUTH_GITHUB_ID=${AUTH_GITHUB_ID}
- AUTH_GITHUB_SECRET=${AUTH_GITHUB_SECRET}
- AUTH_GOOGLE_ID=${AUTH_GOOGLE_ID}
- AUTH_GOOGLE_SECRET=${AUTH_GOOGLE_SECRET}
- AUTH_AUTHENTIK_ID=${AUTH_AUTHENTIK_ID}
- AUTH_AUTHENTIK_SECRET=${AUTH_AUTHENTIK_SECRET}
- AUTH_AUTHENTIK_ISSUER=${AUTH_AUTHENTIK_ISSUER}
- COLLAB_SERVER_URL=${COLLAB_SERVER_URL}
- NEXT_PUBLIC_COLLAB_SERVER_URL=${NEXT_PUBLIC_COLLAB_SERVER_URL}
- NEXT_PUBLIC_UMAMI_SCRIPT=${NEXT_PUBLIC_UMAMI_SCRIPT}
- NEXT_PUBLIC_UMAMI_WEBSITE_ID=${NEXT_PUBLIC_UMAMI_WEBSITE_ID}
- NEXT_PUBLIC_LIBREDESK_URL=${NEXT_PUBLIC_LIBREDESK_URL}
- NEXT_PUBLIC_LIBREDESK_WIDGET_ID=${NEXT_PUBLIC_LIBREDESK_WIDGET_ID}
- NEXT_PUBLIC_DIRECTUS_URL=${NEXT_PUBLIC_DIRECTUS_URL}
- DIRECTUS_FEEDBACK_TOKEN=${DIRECTUS_FEEDBACK_TOKEN}
- OPENAI_API_KEY=${OPENAI_API_KEY}
- OPENAI_BASE_URL=${OPENAI_BASE_URL}
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
- NODE_ENV=production
- NEXT_TELEMETRY_DISABLED=1
expose:
- "3000"
healthcheck:
test:
- CMD-SHELL
- "wget -qO- --timeout=3 http://127.0.0.1:3000/ >/dev/null || exit 1"
interval: 30s
timeout: 5s
retries: 3
start_period: 90s
networks:
- echodo
collab:
build:
context: .
dockerfile: docker/Dockerfile
target: collab
restart: unless-stopped
environment:
# Separate FQDN for the websocket service, e.g. collab.echodo.stillwell.cloud
- SERVICE_FQDN_COLLAB_1234
- DATABASE_URL=${DATABASE_URL}
- REDIS_URL=${REDIS_URL}
- PORT=1234
- NODE_ENV=production
expose:
- "1234"
healthcheck:
test:
- CMD-SHELL
- 'node -e "const p=+(process.env.PORT||1234);require(''net'').createConnection(p,''127.0.0.1'').on(''connect'',()=>process.exit(0)).on(''error'',()=>process.exit(1))"'
interval: 30s
timeout: 5s
retries: 3
start_period: 25s
networks:
- echodo
# MCP server is stdio-based today; deploy only if you want it reachable on the LAN
# for future HTTP/SSE transport. Comment the whole block out if not needed.
mcp:
build:
context: .
dockerfile: docker/Dockerfile
target: mcp
args:
MCP_SERVER_PORT: ${MCP_SERVER_PORT:-3001}
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- MCP_SERVER_PORT=${MCP_SERVER_PORT:-3001}
- NODE_ENV=production
stdin_open: true
expose:
- "${MCP_SERVER_PORT:-3001}"
healthcheck:
test:
- CMD-SHELL
- "pgrep -f 'node.*dist/index\\.js' > /dev/null || exit 1"
interval: 30s
timeout: 5s
retries: 3
start_period: 25s
networks:
- echodo
orchestrator:
build:
context: .
dockerfile: docker/Dockerfile
target: orchestrator
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- REDIS_URL=${REDIS_URL}
- ORCHESTRATOR_DRY_RUN=${ORCHESTRATOR_DRY_RUN:-0}
- ORCHESTRATOR_POLL_MS=${ORCHESTRATOR_POLL_MS:-5000}
- ORCHESTRATOR_STALL_MINUTES=${ORCHESTRATOR_STALL_MINUTES:-120}
- ORCHESTRATOR_MAX_IMPLEMENTERS=${ORCHESTRATOR_MAX_IMPLEMENTERS:-2}
- NODE_ENV=production
healthcheck:
test:
- CMD-SHELL
- "pgrep -f 'node.*dist/index\\.js' > /dev/null || exit 1"
interval: 30s
timeout: 5s
retries: 3
start_period: 25s
networks:
- echodo
worker:
build:
context: .
dockerfile: docker/Dockerfile
target: worker
depends_on:
- orchestrator
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- REDIS_URL=${REDIS_URL}
- WORKER_DRY_RUN=${WORKER_DRY_RUN:-0}
- WORKER_BLPOP_TIMEOUT_S=${WORKER_BLPOP_TIMEOUT_S:-5}
- WORKER_MAX_TOOL_STEPS=${WORKER_MAX_TOOL_STEPS:-12}
- WORKER_REPO_ROOT=/app/repo
- WORKER_WORKTREE_BASE=/tmp/echodo-worker-worktrees
- OPENAI_API_KEY=${OPENAI_API_KEY}
- OPENAI_BASE_URL=${OPENAI_BASE_URL}
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
- NODE_ENV=production
healthcheck:
test:
- CMD-SHELL
- "pgrep -f 'node.*dist/index\\.js' > /dev/null || exit 1"
interval: 30s
timeout: 5s
retries: 3
start_period: 25s
networks:
- echodo
networks:
echodo:
driver: bridge
include:
- docker-compose.prod.yml

View file

@ -0,0 +1,14 @@
# Standalone LXC override — publish ports for CT 100 Traefik (no Dokploy/Coolify proxy).
#
# Usage:
# docker compose -f docker/docker-compose.prod.yml \
# -f docker/docker-compose.prod.standalone.yml up -d
services:
web:
ports:
- "3000:3000"
collab:
ports:
- "1234:1234"

View file

@ -0,0 +1,244 @@
# Echodo production stack — shared CT 102 Postgres/Redis, multi-service build.
#
# Phase A (default): web, collab, mcp, orchestrator, worker
# Phase B (optional): OpenHands agent-server — enable with profile `agent-server`
#
# Deploy targets:
# - Dokploy (CT 112): compose file path `docker/docker-compose.prod.yml`, project dir = repo root
# - Standalone LXC: `docker compose -f docker/docker-compose.prod.yml up -d`
#
# Domains (Dokploy UI or CT 100 Traefik):
# web → echodo.stillwell.cloud (port 3000)
# collab → collab.echodo.stillwell.cloud (port 1234)
#
# Build (required on ≤8 GB hosts — parallel export OOMs with exit 255):
# bash scripts/build-prod.sh
# Dokploy → Advanced → Custom Build Command: bash scripts/build-prod.sh
#
# Phase B:
# docker compose -f docker/docker-compose.prod.yml --profile agent-server up -d
# Set WORKER_SESSION_TYPE=openhands and matching SESSION_API_KEY / LOCAL_BACKEND_API_KEY.
services:
web:
build:
context: .
dockerfile: docker/Dockerfile
target: web
args:
NEXT_PUBLIC_COLLAB_SERVER_URL: ${NEXT_PUBLIC_COLLAB_SERVER_URL:-ws://localhost:1234}
NEXT_PUBLIC_UMAMI_SCRIPT: ${NEXT_PUBLIC_UMAMI_SCRIPT:-}
NEXT_PUBLIC_UMAMI_WEBSITE_ID: ${NEXT_PUBLIC_UMAMI_WEBSITE_ID:-}
NEXT_PUBLIC_LIBREDESK_URL: ${NEXT_PUBLIC_LIBREDESK_URL:-}
NEXT_PUBLIC_LIBREDESK_WIDGET_ID: ${NEXT_PUBLIC_LIBREDESK_WIDGET_ID:-}
NEXT_PUBLIC_DIRECTUS_URL: ${NEXT_PUBLIC_DIRECTUS_URL:-}
depends_on:
collab:
condition: service_started
mcp:
condition: service_started
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- REDIS_URL=${REDIS_URL}
- NEXTAUTH_URL=${NEXTAUTH_URL}
- NEXTAUTH_SECRET=${NEXTAUTH_SECRET}
- AUTH_SECRET=${AUTH_SECRET}
- AUTH_DEV_PASSWORD=${AUTH_DEV_PASSWORD}
- AUTH_GITHUB_ID=${AUTH_GITHUB_ID}
- AUTH_GITHUB_SECRET=${AUTH_GITHUB_SECRET}
- AUTH_GOOGLE_ID=${AUTH_GOOGLE_ID}
- AUTH_GOOGLE_SECRET=${AUTH_GOOGLE_SECRET}
- AUTH_AUTHENTIK_ID=${AUTH_AUTHENTIK_ID}
- AUTH_AUTHENTIK_SECRET=${AUTH_AUTHENTIK_SECRET}
- AUTH_AUTHENTIK_ISSUER=${AUTH_AUTHENTIK_ISSUER}
- COLLAB_SERVER_URL=${COLLAB_SERVER_URL}
- NEXT_PUBLIC_COLLAB_SERVER_URL=${NEXT_PUBLIC_COLLAB_SERVER_URL}
- NEXT_PUBLIC_UMAMI_SCRIPT=${NEXT_PUBLIC_UMAMI_SCRIPT}
- NEXT_PUBLIC_UMAMI_WEBSITE_ID=${NEXT_PUBLIC_UMAMI_WEBSITE_ID}
- NEXT_PUBLIC_LIBREDESK_URL=${NEXT_PUBLIC_LIBREDESK_URL}
- NEXT_PUBLIC_LIBREDESK_WIDGET_ID=${NEXT_PUBLIC_LIBREDESK_WIDGET_ID}
- NEXT_PUBLIC_DIRECTUS_URL=${NEXT_PUBLIC_DIRECTUS_URL}
- DIRECTUS_FEEDBACK_TOKEN=${DIRECTUS_FEEDBACK_TOKEN}
- OPENAI_API_KEY=${OPENAI_API_KEY}
- OPENAI_BASE_URL=${OPENAI_BASE_URL}
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
- OMNIROUTE_BASE_URL=${OMNIROUTE_BASE_URL:-}
- OMNIROUTE_API_KEY=${OMNIROUTE_API_KEY:-}
- GITHUB_TOKEN=${GITHUB_TOKEN:-}
- FORGEJO_TOKEN=${FORGEJO_TOKEN:-}
- FORGE_WEBHOOK_SECRET=${FORGE_WEBHOOK_SECRET:-}
- NODE_ENV=production
- NEXT_TELEMETRY_DISABLED=1
expose:
- "3000"
healthcheck:
test:
- CMD-SHELL
- "wget -qO- --timeout=3 http://127.0.0.1:3000/ >/dev/null || exit 1"
interval: 30s
timeout: 5s
retries: 3
start_period: 90s
networks:
- echodo
collab:
build:
context: .
dockerfile: docker/Dockerfile
target: collab
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- REDIS_URL=${REDIS_URL}
- PORT=1234
- NODE_ENV=production
expose:
- "1234"
healthcheck:
test:
- CMD-SHELL
- 'node -e "const p=+(process.env.PORT||1234);require(''net'').createConnection(p,''127.0.0.1'').on(''connect'',()=>process.exit(0)).on(''error'',()=>process.exit(1))"'
interval: 30s
timeout: 5s
retries: 3
start_period: 25s
networks:
- echodo
mcp:
build:
context: .
dockerfile: docker/Dockerfile
target: mcp
args:
MCP_SERVER_PORT: ${MCP_SERVER_PORT:-3001}
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- MCP_SERVER_PORT=${MCP_SERVER_PORT:-3001}
- NODE_ENV=production
stdin_open: true
expose:
- "${MCP_SERVER_PORT:-3001}"
healthcheck:
test:
- CMD-SHELL
- "pgrep -f 'node.*dist/index\\.js' > /dev/null || exit 1"
interval: 30s
timeout: 5s
retries: 3
start_period: 25s
networks:
- echodo
orchestrator:
build:
context: .
dockerfile: docker/Dockerfile
target: orchestrator
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- REDIS_URL=${REDIS_URL}
- ORCHESTRATOR_DRY_RUN=${ORCHESTRATOR_DRY_RUN:-0}
- ORCHESTRATOR_POLL_MS=${ORCHESTRATOR_POLL_MS:-5000}
- ORCHESTRATOR_STALL_MINUTES=${ORCHESTRATOR_STALL_MINUTES:-120}
- ORCHESTRATOR_MAX_IMPLEMENTERS=${ORCHESTRATOR_MAX_IMPLEMENTERS:-2}
- NODE_ENV=production
healthcheck:
test:
- CMD-SHELL
- "pgrep -f 'node.*dist/index\\.js' > /dev/null || exit 1"
interval: 30s
timeout: 5s
retries: 3
start_period: 25s
networks:
- echodo
worker:
build:
context: .
dockerfile: docker/Dockerfile
target: worker
depends_on:
orchestrator:
condition: service_started
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- REDIS_URL=${REDIS_URL}
- WORKER_DRY_RUN=${WORKER_DRY_RUN:-0}
- WORKER_BLPOP_TIMEOUT_S=${WORKER_BLPOP_TIMEOUT_S:-5}
- WORKER_MAX_TOOL_STEPS=${WORKER_MAX_TOOL_STEPS:-12}
- WORKER_REPO_ROOT=/app/repo
- WORKER_WORKTREE_BASE=/tmp/echodo-worker-worktrees
- WORKER_SESSION_TYPE=${WORKER_SESSION_TYPE:-minimal}
- WORKER_OPENHANDS_WORKSPACE=${WORKER_OPENHANDS_WORKSPACE:-/workspace/repo}
- AGENT_SERVER_URL=${AGENT_SERVER_URL:-http://agent-server:8001}
- SESSION_API_KEY=${SESSION_API_KEY:-}
- OPENAI_API_KEY=${OPENAI_API_KEY}
- OPENAI_BASE_URL=${OPENAI_BASE_URL}
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
- OMNIROUTE_BASE_URL=${OMNIROUTE_BASE_URL:-}
- OMNIROUTE_API_KEY=${OMNIROUTE_API_KEY:-}
- GITHUB_TOKEN=${GITHUB_TOKEN:-}
- FORGEJO_TOKEN=${FORGEJO_TOKEN:-}
- NODE_ENV=production
volumes:
- echodo-workspace:/workspace
healthcheck:
test:
- CMD-SHELL
- "pgrep -f 'node.*dist/index\\.js' > /dev/null || exit 1"
interval: 30s
timeout: 5s
retries: 3
start_period: 25s
networks:
- echodo
agent-server:
profiles: ["agent-server"]
image: ghcr.io/open-hands/agent-server:0.19.0
restart: unless-stopped
environment:
- AGENT_SERVER_HOST=0.0.0.0
- AGENT_SERVER_PORT=8001
- LOCAL_BACKEND_API_KEY=${LOCAL_BACKEND_API_KEY}
- LLM_MODEL=${OMNIROUTE_MODEL:-auto/coding}
- LLM_API_KEY=${OMNIROUTE_API_KEY}
- LLM_BASE_URL=${OMNIROUTE_BASE_URL}
- SANDBOX_TYPE=docker
- SANDBOX_DOCKER_IMAGE=ghcr.io/open-hands/sandbox:latest
- SANDBOX_TIMEOUT=300
- WORKSPACE_BASE=/workspace
volumes:
- echodo-workspace:/workspace
- /var/run/docker.sock:/var/run/docker.sock
expose:
- "8001"
healthcheck:
test:
- CMD-SHELL
- "wget -qO- --timeout=3 http://127.0.0.1:8001/server_info >/dev/null || exit 1"
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
deploy:
resources:
limits:
memory: 2G
cpus: "2"
networks:
- echodo
volumes:
echodo-workspace:
networks:
echodo:
driver: bridge

17
docker/worker-entrypoint.sh Executable file
View file

@ -0,0 +1,17 @@
#!/bin/sh
set -eu
# When OpenHands is enabled, seed the shared workspace volume from the deploy
# snapshot baked into the image (/app/repo). Agent-server reads the same path.
if [ "${WORKER_SESSION_TYPE:-minimal}" = "openhands" ]; then
target="${WORKER_OPENHANDS_WORKSPACE:-/workspace/repo}"
mkdir -p "$(dirname "$target")"
if [ ! -f "$target/pnpm-workspace.yaml" ]; then
echo "[worker-entrypoint] seeding OpenHands workspace at ${target}"
mkdir -p "$target"
cp -a /app/repo/. "$target/"
fi
fi
cd /app/apps/worker
exec node dist/index.js

View file

@ -0,0 +1,169 @@
# Echodo / Submatter — Production Deployment
Production runs on **CT 116** (`submatter`, `192.168.68.116`) as plain Docker Compose — not Dokploy.
| Doc | Purpose |
|-----|---------|
| This file | Operator checklist, env vars, troubleshooting |
| [`docs/forgejo-migration.md`](forgejo-migration.md) | GitHub → Forgejo + CT 116 cutover |
| [`proxmox/ct116/README.md`](../proxmox/ct116/README.md) | CT creation, Traefik, auto-deploy, Dokploy removal |
| `AGENT-DEPLOY.md` (local, gitignored) | Homelab credentials + CT map — add CT 116 row after cutover |
---
## Why a dedicated CT (not Dokploy)?
Echodo builds **five Docker images** from one monorepo; the Next.js compile alone reserves a 5 GB V8 heap. On Dokploy (CT 112), that competes with Zest, Umami, Comms, etc. and OOMs with exit 255.
**CT 116** gets the full 8 GB for serialized builds. CT 107 is not an alternative — it still hosts Coolify, Echo Board, Resonance, and other dev apps.
| Resource | CT 116 spec |
|----------|-------------|
| RAM | 8 GB |
| CPU | 2 cores |
| Disk | 32 GB rootfs |
| Runtime (Phase A) | ~3–4 GB |
| Phase B (+ OpenHands) | +2 GB |
External: Postgres/Redis on **CT 102**, TLS on **CT 100**, OmniRoute on **CT 108**.
---
## Stack files
| File | Use |
|------|-----|
| `docker/docker-compose.prod.yml` | Core stack |
| `docker/docker-compose.prod.standalone.yml` | Publish `:3000` / `:1234` for CT 100 Traefik |
| `scripts/build-prod.sh` | Serialized image build |
| `scripts/deploy-prod.sh` | Build + up + health wait |
```bash
docker compose -f docker/docker-compose.prod.yml \
-f docker/docker-compose.prod.standalone.yml up -d
```
---
## Phase A — First deploy on CT 116
1. **Create CT 116** — see [`proxmox/ct116/README.md`](../proxmox/ct116/README.md) (`pct create` one-liner).
2. **Bootstrap** — `bootstrap.sh` installs Docker, clones to `/opt/submatter`.
3. **Configure `.env`** — copy from `.env.example`; use `echodo` database on CT 102, Redis `/11`.
4. **Deploy** — `bash scripts/deploy-prod.sh build`
5. **Traefik** — apply `proxmox/ct116/traefik-ct100.example.yml` on CT 100; remove old Dokploy/Coolify Echodo routes.
6. **Verify** — `https://echodo.stillwell.cloud`, `wss://collab.echodo.stillwell.cloud`
### Required env vars
| Variable | Production example | Notes |
|----------|-------------------|-------|
| `DATABASE_URL` | `postgresql://echodo:***@192.168.68.102:5432/echodo` | |
| `REDIS_URL` | `redis://:***@192.168.68.102:6379/11` | |
| `NEXTAUTH_URL` | `https://echodo.stillwell.cloud` | |
| `NEXTAUTH_SECRET` / `AUTH_SECRET` | `openssl rand -base64 32` | |
| `AUTH_AUTHENTIK_*` | | Authentik OIDC app `echodo` |
| `COLLAB_SERVER_URL` | `wss://collab.echodo.stillwell.cloud` | Server-side |
| `NEXT_PUBLIC_COLLAB_SERVER_URL` | `wss://collab.echodo.stillwell.cloud` | **Build arg** — requires image rebuild |
| `OMNIROUTE_BASE_URL` | `https://omniroute.stillwell.cloud/v1` | Web + worker |
| `OMNIROUTE_API_KEY` | (your key) | + register in Settings → AI → Providers |
| `FORGEJO_TOKEN` / `GITHUB_TOKEN` | PAT values | Referenced by name in Connected repos settings |
| `NEXT_PUBLIC_UMAMI_*` | | Analytics (CT 112 Umami) |
Changing any `NEXT_PUBLIC_*` value requires `bash scripts/deploy-prod.sh build`, not just `docker compose restart`.
---
## Auto-deploy on push
**CI** (every PR): `.github/workflows/ci.yml` — lint, type-check, test on GitHub-hosted or PR runner.
**CD** (merge to `main`): `.github/workflows/deploy-homelab.yml` and `.forgejo/workflows/deploy-homelab.yml` — SSH from CT 111 `axiom` runner → CT 116 → `deploy-prod.sh build`.
### GitHub / Forgejo repo secrets
| Secret | Value |
|--------|-------|
| `HOMELAB_DEPLOY_HOST` | `192.168.68.116` |
| `HOMELAB_DEPLOY_USER` | `root` |
| `HOMELAB_DEPLOY_SSH_KEY` | SSH private key (ed25519) |
| `HOMELAB_DEPLOY_PATH` | `/opt/submatter` (optional) |
Generate deploy key on CT 116:
```bash
ssh-keygen -t ed25519 -f /root/.ssh/github_deploy -N ""
cat /root/.ssh/github_deploy.pub >> /root/.ssh/authorized_keys
# Paste private key into HOMELAB_DEPLOY_SSH_KEY secret
```
Until secrets are set, the deploy workflow skips gracefully (CI still runs).
### Forgejo webhook (optional second trigger)
If Forgejo is canonical git and you want push deploy without Actions:
1. Install [webhook](https://github.com/adhocore/webhook) on CT 116.
2. Copy `proxmox/ct116/webhook.hooks.json.example` → `/etc/webhook.conf`.
3. Forgejo repo → Webhook → POST push events → `http://192.168.68.116:9000/hooks/submatter-deploy`.
---
## Remove from Dokploy
1. Dokploy UI → Submatter/Echodo compose app → Stop → Delete.
2. CT 100 — remove Traefik routes to Dokploy for `echodo.stillwell.cloud` / collab hostname.
3. Update local `AGENT-DEPLOY.md`: add CT 116 row, note Echodo removed from CT 112.
---
## Phase B — OpenHands (optional)
```bash
export COMPOSE_PROFILES=agent-server
bash scripts/deploy-prod.sh build
```
| Variable | Notes |
|----------|-------|
| `LOCAL_BACKEND_API_KEY` | `openssl rand -hex 32` |
| `SESSION_API_KEY` | Same as above (worker) |
| `WORKER_SESSION_TYPE` | `openhands` |
| `OMNIROUTE_BASE_URL` | `https://omniroute.stillwell.cloud/v1` |
| `OMNIROUTE_API_KEY` | OmniRoute bearer key |
Requires Docker socket on CT 116 (`nesting=1` on LXC).
---
## Troubleshooting
| Symptom | Fix |
|---------|-----|
| Build exit 255 ~60–90s | Ensure `deploy-prod.sh` uses `build-prod.sh` (serialized) |
| OAuth broken | DB must be `echodo`, not legacy `tasks` |
| Deploy workflow skipped | Set `HOMELAB_DEPLOY_*` secrets |
| Collab websocket fails | Traefik route to `:1234`, `wss://` in public env vars |
| OpenHands unavailable | `SESSION_API_KEY` must match `LOCAL_BACKEND_API_KEY` |
Local build verify: `pnpm docker:build:prod`
---
## Future: registry pull deploy (no build on CT 116)
Build images on CT 111 CI, push to Forgejo container registry, then on CT 116:
```bash
bash scripts/deploy-prod.sh pull
```
Requires compose `image:` tags instead of `build:` — not implemented yet; track as follow-on if CT 116 builds remain too slow.
---
## Related
- `docs/AGENT-AUTONOMOUS-CONVOYS.md` — convoy + OpenHands roadmap
- `docs/omniroute-setup.md` — OmniRoute operator guide
- `.env.example` — full variable list

168
docs/forgejo-migration.md Normal file
View file

@ -0,0 +1,168 @@
# Git remote migration — GitHub → Forgejo
Canonical repo: **https://forgejo.stillwell.cloud/rstillwell/Submatter.git**
Production host (CT 116) clones from Forgejo `main`. CI/CD runs via Forgejo Actions (`.forgejo/workflows/`) on the axiom self-hosted runner.
---
## 1. Local machine — point origin at Forgejo
From your dev checkout:
```bash
cd /path/to/submatter
bash scripts/migrate-origin-to-forgejo.sh
```
This renames the old GitHub remote to `github` and sets `origin` → Forgejo.
Push all branches you care about:
```bash
git push -u origin main
git push origin --tags
# feature branches:
git push origin feat/brief-3-harness-worker-dispatch
```
Use a Forgejo personal access token when prompted (Settings → Applications → Generate token, `repo` scope).
---
## 2. Merge deploy fixes to `main`
The CT 116 collab URL fix, Forgejo wiring, and prod compose live on your current branch. Before CT 116 can pull them:
```bash
git checkout main
git merge feat/brief-3-harness-worker-dispatch # or your branch name
git push origin main
```
Or open a PR on Forgejo and merge there.
---
## 3. CT 116 — switch remote + redeploy
SSH to the production host (from a machine on your LAN):
```bash
ssh root@192.168.68.116
cd /opt/submatter
```
### 3a. Update `.env` (critical for doc/whiteboard fix)
```bash
nano .env
```
Ensure these lines exist and use **wss://** (not ws://):
```env
NEXT_PUBLIC_COLLAB_SERVER_URL=wss://collab.echodo.stillwell.cloud
COLLAB_SERVER_URL=wss://collab.echodo.stillwell.cloud
```
Add OmniRoute + Forgejo token (for Connected repos + worker):
```env
OMNIROUTE_BASE_URL=https://omniroute.stillwell.cloud/v1
OMNIROUTE_API_KEY=<your-key>
FORGEJO_TOKEN=<forgejo-pat-with-repo-scope>
```
`NEXT_PUBLIC_*` changes require a **full rebuild**, not just restart.
### 3b. Migrate git remote + deploy
After `main` on Forgejo contains the deploy fixes:
```bash
bash scripts/migrate-origin-to-forgejo.sh
bash scripts/ct116-post-migration.sh
```
Or one-liner from your laptop:
```bash
ssh root@192.168.68.116 'cd /opt/submatter && bash scripts/migrate-origin-to-forgejo.sh && bash scripts/ct116-post-migration.sh'
```
Build takes **15–30 minutes** on the 8 GB CT (serialized image build).
---
## 4. Forgejo Actions secrets
In Forgejo → **Submatter** → Settings → Actions → Secrets, add:
| Secret | Value |
|--------|-------|
| `HOMELAB_DEPLOY_HOST` | `192.168.68.116` |
| `HOMELAB_DEPLOY_USER` | `root` |
| `HOMELAB_DEPLOY_SSH_KEY` | private key allowed to SSH to CT 116 |
| `HOMELAB_DEPLOY_PATH` | `/opt/submatter` |
Generate deploy key on CT 116:
```bash
ssh-keygen -t ed25519 -f /root/.ssh/forgejo_deploy -N ""
cat /root/.ssh/forgejo_deploy.pub >> /root/.ssh/authorized_keys
# paste private key into Forgejo secret HOMELAB_DEPLOY_SSH_KEY
```
Enable Actions on the repo. Workflows:
- `.forgejo/workflows/ci.yml` — lint, type-check, test on PR/push
- `.forgejo/workflows/deploy-homelab.yml` — deploy to CT 116 on push to `main`
Register the axiom runner in Forgejo (same CT 111 runner pool as GitHub, if configured for Forgejo).
---
## 5. Optional — Forgejo push webhook (instant deploy)
Alternative to Actions deploy:
1. Install [webhook](https://github.com/adhocore/webhook) on CT 116
2. Configure `proxmox/ct116/webhook.hooks.json.example`
3. Forgejo repo → Webhooks → POST push to `main`
Set `DEPLOY_WEBHOOK_SECRET` in webhook config and Forgejo webhook secret.
---
## 6. Connected repos (in-app)
Settings → Connected repos:
| Field | Value |
|-------|-------|
| Remote URL | `https://forgejo.stillwell.cloud/rstillwell/Submatter.git` |
| Token env var | `FORGEJO_TOKEN` |
Token value lives in CT 116 `.env` only — not in the database.
---
## 7. Decommission GitHub (when ready)
- Archive or delete `stwl-labs/ubiquitous-invention` on GitHub
- Remove GitHub deploy secrets if unused
- Update any external links to point at Forgejo
Keep `github` remote locally for reference: `git fetch github`
---
## Verify after deploy
1. Create a document and whiteboard — no client-side exception
2. Browser devtools → Network → WS connects to `wss://collab.echodo.stillwell.cloud`
3. Settings → AI → Providers → Test on OmniRoute alias
4. Settings → Connected repos → Test connection on Forgejo repo
See also: [`docs/deployment-production.md`](deployment-production.md), [`proxmox/ct116/README.md`](../proxmox/ct116/README.md)

15
logs/app.log Normal file
View file

@ -0,0 +1,15 @@
2026-07-07 13:44:42 | INFO | Starting server on 0.0.0.0:8000
2026-07-07 13:44:42 | INFO | Multi-handler server configuration complete.
2026-07-07 13:44:42 | INFO | Starting Uvicorn server...
2026-07-07 13:44:42 | INFO | Model registry initialized
2026-07-07 13:44:42 | INFO | Spawning handler process for model 'mlx-community/Qwen2.5-Coder-7B-Instruct-4bit' (type=lm, path=mlx-community/Qwen2.5-Coder-7B-Instruct-4bit)
2026-07-07 13:44:42 | INFO | Spawned handler process for 'mlx-community/Qwen2.5-Coder-7B-Instruct-4bit' (pid=5116)
2026-07-07 13:44:43 | INFO | 'mlx-community/Qwen2.5-Coder-7B-Instruct-4bit': Fetching 'mlx-community/Qwen2.5-Coder-7B-Instruct-4bit' from HuggingFace...
2026-07-07 13:45:26 | INFO | 'mlx-community/Qwen2.5-Coder-7B-Instruct-4bit': Fetched 'mlx-community/Qwen2.5-Coder-7B-Instruct-4bit'
2026-07-07 13:45:33 | INFO | Handler process for 'mlx-community/Qwen2.5-Coder-7B-Instruct-4bit' is ready
2026-07-07 13:45:33 | INFO | Registered model: mlx-community/Qwen2.5-Coder-7B-Instruct-4bit (type=lm, context_length=None)
2026-07-07 13:45:33 | INFO | Model 'mlx-community/Qwen2.5-Coder-7B-Instruct-4bit' spawned and registered successfully
2026-07-07 13:45:33 | INFO | Multi-handler initialization complete. 1 model(s) spawned.
2026-07-07 15:22:19 | INFO | Shutting down multi-handler application
2026-07-07 15:22:19 | INFO | Cleaned up handler for model: mlx-community/Qwen2.5-Coder-7B-Instruct-4bit
2026-07-07 15:22:19 | INFO | All models unregistered and cleaned up

View file

@ -11,7 +11,8 @@
"db:migrate": "turbo db:migrate --filter=@tasks/database",
"db:push": "turbo db:push --filter=@tasks/database",
"format": "prettier --write \"**/*.{ts,tsx,js,jsx,json,css,md}\"",
"docker:build:coolify": "bash scripts/build-coolify.sh"
"docker:build:prod": "bash scripts/build-prod.sh",
"docker:build:coolify": "bash scripts/build-prod.sh"
},
"devDependencies": {
"@vitest/coverage-v8": "^2",

133
proxmox/ct116/README.md Normal file
View file

@ -0,0 +1,133 @@
# CT 116 — Submatter / Echodo (production)
Dedicated LXC for the multitenant task app + convoy worker stack. Replaces Dokploy (CT 112) and eventually decommissions the Echodo slice on CT 107 Coolify.
## Spec (from AGENT-DEPLOY.md sizing)
| Setting | Value | Why |
|---------|-------|-----|
| **CT ID** | **116** | Next free slot (114 planned Nextcloud, 115 Forgejo) |
| **Hostname** | `submatter` | Product name; DNS stays `echodo.stillwell.cloud` |
| **IP** | `192.168.68.116/22` | Sequential homelab LAN |
| **RAM** | **8192 MB** | Web build needs ~5 GB V8 heap + BuildKit headroom |
| **CPU** | 2 cores | Same as former CT 107 Echodo deploy |
| **Disk** | **32 GB** | Five images + layer cache (AGENT-DEPLOY: 30–32 GB for compose-heavy CTs) |
| **Features** | `nesting=1` | Required for Docker-in-Docker (Phase B OpenHands sandboxes) |
**Do not reuse CT 107** for this — it still runs Coolify, Echo Board, Resonance, pipeline dashboard, Amazon scraper, etc. Competing for RAM during builds was the original OOM problem.
## External services (unchanged)
| Service | Host |
|---------|------|
| Postgres `echodo` DB | CT 102 `:5432` |
| Redis DB `/11` | CT 102 `:6379` |
| TLS / public routing | CT 100 Traefik |
| OmniRoute (Phase B) | CT 108 |
| Umami analytics | CT 112 (script URL only) |
## Clone URL
Production checkout: **https://forgejo.stillwell.cloud/rstillwell/Submatter.git** (`main`).
Migration from GitHub: [`docs/forgejo-migration.md`](../../docs/forgejo-migration.md).
## Bootstrap
On Proxmox host:
```bash
pct create 116 local:vztmpl/debian-13-standard_13.0-1_amd64.tar.zst \
--hostname submatter --cores 2 --memory 8192 --swap 512 \
--rootfs local-lvm:32 \
--net0 name=eth0,bridge=vmbr0,ip=192.168.68.116/22,gw=192.168.68.1 \
--unprivileged 1 --features nesting=1,keyctl=1
pct start 116
./proxmox/scripts/sync.sh push 116 # when homelab sync script includes this tree
pct exec 116 -- bash /root/bootstrap.sh
```
Edit `/opt/submatter/.env`, then:
```bash
pct exec 116 -- bash -lc 'cd /opt/submatter && bash scripts/deploy-prod.sh build'
```
## Routing
1. Merge `traefik-ct100.example.yml` into CT 100 dynamic config.
2. Remove old Echodo routes pointing at CT 107 Coolify or CT 112 Dokploy once cutover is verified.
3. AdGuard wildcard `*.stillwell.cloud` → CT 100 (unchanged).
## Delete from Dokploy (CT 112)
In Dokploy UI:
1. Open the Submatter/Echodo compose application.
2. Stop containers → Delete application (keep CT 112 shared services like Umami).
3. Remove any CT 100 Traefik routes that pointed at Dokploy for `echodo.stillwell.cloud`.
## Auto-deploy on push
Three options (can combine GitHub + Forgejo):
### A. Self-hosted runner (recommended)
Uses CT 111 `axiom` runners (already org-scoped to `stwl-labs`).
Repo secrets:
| Secret | Value |
|--------|-------|
| `HOMELAB_DEPLOY_HOST` | `192.168.68.116` |
| `HOMELAB_DEPLOY_USER` | `root` (or dedicated deploy user) |
| `HOMELAB_DEPLOY_SSH_KEY` | private key allowed to SSH to CT 116 |
| `HOMELAB_DEPLOY_PATH` | `/opt/submatter` (optional) |
Workflows:
- `.github/workflows/deploy-homelab.yml` — GitHub pushes to `main`
- `.forgejo/workflows/deploy-homelab.yml` — Forgejo Actions (same file)
CI (lint/type-check/test) still runs on every PR via `.github/workflows/ci.yml`. Deploy runs **after** merge to `main`.
### B. Forgejo webhook (no Actions)
On Forgejo → repo → Webhooks:
- URL: `http://192.168.68.116:9000/hooks/deploy` (front with Caddy/`webhook` tool)
- Secret: set `DEPLOY_WEBHOOK_SECRET` on CT 116
- Events: Push
Handler: `deploy-webhook.sh`
### C. Manual
```bash
ssh root@192.168.68.116 'cd /opt/submatter && git pull && bash scripts/deploy-prod.sh build'
```
## Phase B (OpenHands)
```bash
export COMPOSE_PROFILES=agent-server
bash scripts/deploy-prod.sh build
```
Set `WORKER_SESSION_TYPE=openhands`, `SESSION_API_KEY`, `LOCAL_BACKEND_API_KEY`, OmniRoute vars in `.env`.
## AGENT-DEPLOY.md handoff
After CT is live, add to your local `AGENT-DEPLOY.md` (gitignored):
```markdown
| 116 | LXC | submatter | 192.168.68.116 | 8 GB | 2 | 32 GB | ACTIVE | Submatter/Echodo production (docker compose) |
```
Move Echodo row off CT 107 / CT 112 sections.
## Related
- [`docs/deployment-production.md`](../../docs/deployment-production.md) — env vars, troubleshooting
- [`docker/docker-compose.prod.yml`](../../docker/docker-compose.prod.yml) — stack definition

56
proxmox/ct116/bootstrap.sh Executable file
View file

@ -0,0 +1,56 @@
#!/usr/bin/env bash
# First-time bootstrap for CT 116 (Submatter / Echodo production host).
# Run as root on a fresh Debian 13 LXC after `pct create`.
#
# Example (on Proxmox host — adjust storage/template to your install):
# pct create 116 local:vztmpl/debian-13-standard_13.0-1_amd64.tar.zst \
# --hostname submatter --cores 2 --memory 8192 --swap 512 \
# --rootfs local-lvm:32 --net0 name=eth0,bridge=vmbr0,ip=192.168.68.116/22,gw=192.168.68.1 \
# --unprivileged 1 --features nesting=1,keyctl=1
# pct start 116
# pct push 116 proxmox/ct116/bootstrap.sh /root/bootstrap.sh
# pct exec 116 -- bash /root/bootstrap.sh
set -euo pipefail
APP_DIR="${APP_DIR:-/opt/submatter}"
REPO_URL="${REPO_URL:-https://forgejo.stillwell.cloud/rstillwell/Submatter.git}"
DEPLOY_BRANCH="${DEPLOY_BRANCH:-main}"
echo "[bootstrap] installing docker..."
apt-get update
apt-get install -y ca-certificates curl git rsync
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian $(. /etc/os-release && echo "$VERSION_CODENAME") stable" \
> /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
echo "[bootstrap] cloning repo to ${APP_DIR}..."
mkdir -p "$(dirname "$APP_DIR")"
if [[ ! -d "${APP_DIR}/.git" ]]; then
git clone --branch "$DEPLOY_BRANCH" "$REPO_URL" "$APP_DIR"
fi
cd "$APP_DIR"
if [[ ! -f .env ]]; then
cp .env.example .env
echo "[bootstrap] Wrote ${APP_DIR}/.env from .env.example — edit before first deploy."
fi
chmod +x scripts/build-prod.sh scripts/deploy-prod.sh docker/worker-entrypoint.sh 2>/dev/null || true
mkdir -p /opt/appdata/submatter
cat <<'EOF'
[bootstrap] Next steps (manual):
1. Edit /opt/submatter/.env with CT 102 DATABASE_URL, REDIS_URL, auth secrets, public URLs.
2. bash /opt/submatter/scripts/deploy-prod.sh build
3. Add CT 100 Traefik routes (see proxmox/ct116/traefik-ct100.example.yml).
4. Install Dozzle + Beszel agents per CT 101 runbook.
5. Configure deploy automation:
- GitHub/Forgejo repo secrets for HOMELAB_DEPLOY_* (see docs/deployment-production.md)
- OR Forgejo push webhook → proxmox/ct116/deploy-webhook.sh
EOF

43
proxmox/ct116/deploy-webhook.sh Executable file
View file

@ -0,0 +1,43 @@
#!/usr/bin/env bash
# Forgejo/GitHub push webhook target — validates HMAC and redeploys main.
#
# Install on CT 116 with almquist/webhook or Caddy reverse_proxy to this script.
# Forgejo → Repository → Webhooks → POST → http://192.168.68.116:9000/hooks/deploy
#
# Env:
# DEPLOY_WEBHOOK_SECRET — shared secret (same as Forgejo webhook secret)
# APP_DIR=/opt/submatter
set -euo pipefail
APP_DIR="${APP_DIR:-/opt/submatter}"
SECRET="${DEPLOY_WEBHOOK_SECRET:-}"
read -r payload || payload=""
if [[ -z "$payload" ]]; then
payload="$(cat)"
fi
signature="${HTTP_X_HUB_SIGNATURE_256:-${HTTP_X_FORGEJO_SIGNATURE:-}}"
if [[ -n "$SECRET" ]]; then
if [[ -z "$signature" ]]; then
echo "missing signature" >&2
exit 403
fi
sig="${signature#sha256=}"
expected="$(printf '%s' "$payload" | openssl dgst -sha256 -hmac "$SECRET" | awk '{print $2}')"
if [[ "$sig" != "$expected" ]]; then
echo "invalid signature" >&2
exit 403
fi
fi
ref="$(printf '%s' "$payload" | sed -n 's/.*"ref":"\([^"]*\)".*/\1/p')"
if [[ "$ref" != "refs/heads/main" ]]; then
echo "ignored ref=${ref}"
exit 0
fi
echo "[deploy-webhook] push to main — deploying"
cd "$APP_DIR"
bash scripts/git-sync-deploy.sh
exec bash scripts/deploy-prod.sh build

View file

@ -0,0 +1,35 @@
# CT 100 Traefik dynamic config snippet — add to `/opt/appdata/traefik/config/dynamic/`
# as `ct116-submatter.yml` (or merge into your homelab Traefik dynamic dir).
#
# CT 116 publishes web:3000 and collab:1234 directly (standalone compose override).
# TLS terminates on CT 100; backends are plain HTTP.
#
# Remove stale routes after cutover (edit on CT 100, then push-traefik):
# - ct107-dev.yml: echodo / collab routers pointing at 192.168.68.107:80 (Coolify)
# - Any Dokploy CT 112 echodo routes in dynamic config
# Keep only the CT 116 backends below for echodo.stillwell.cloud hostnames.
http:
routers:
echodo:
rule: Host(`echodo.stillwell.cloud`)
entryPoints: [websecure]
service: echodo-web
tls:
certResolver: letsencrypt
echodo-collab:
rule: Host(`collab.echodo.stillwell.cloud`)
entryPoints: [websecure]
service: echodo-collab
tls:
certResolver: letsencrypt
services:
echodo-web:
loadBalancer:
servers:
- url: "http://192.168.68.116:3000"
echodo-collab:
loadBalancer:
servers:
- url: "http://192.168.68.116:1234"

View file

@ -0,0 +1,25 @@
[
{
"id": "submatter-deploy",
"execute-command": "/opt/submatter/proxmox/ct116/deploy-webhook.sh",
"command-working-directory": "/opt/submatter",
"pass-arguments-to-command": [],
"pass-environment-to-command": [
{ "source": "string", "name": "DEPLOY_WEBHOOK_SECRET", "envname": "DEPLOY_WEBHOOK_SECRET" },
{ "source": "string", "name": "APP_DIR", "envname": "APP_DIR" }
],
"trigger-http-method": "POST",
"include-request-headers-in-environment": true,
"incoming-payload-content-type": "application/json",
"trigger-rule": {
"match": {
"type": "payload-hash-sha256",
"secret": "REPLACE_WITH_FORGEJO_WEBHOOK_SECRET",
"parameter": {
"source": "header",
"name": "X-Hub-Signature-256"
}
}
}
}
]

View file

@ -1,26 +1,3 @@
#!/usr/bin/env bash
# Coolify custom build command for docker-compose.coolify.yml.
#
# Set in Coolify → Resource → Advanced → Custom Build Command:
# bash scripts/build-coolify.sh
#
# Coolify auto-injects -f and --env-file; this script only serializes builds so
# an 8 GB host does not OOM (exit 255) while exporting five runtime images.
set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$repo_root"
export DOCKER_BUILDKIT=1
compose() {
docker compose --project-directory "$repo_root" \
-f docker/docker-compose.coolify.yml \
"$@"
}
# Build one service at a time so an 8 GB host does not OOM during export
# (Coolify default runs all five targets in parallel → exit 255).
for service in collab mcp orchestrator worker web; do
compose build --pull "$service"
done
# Backward-compatible alias — use scripts/build-prod.sh for new deploys.
exec "$(dirname "$0")/build-prod.sh" "$@"

29
scripts/build-prod.sh Executable file
View file

@ -0,0 +1,29 @@
#!/usr/bin/env bash
# Production custom build for docker/docker-compose.prod.yml.
#
# Dokploy → Compose resource → Advanced → Custom Build Command:
# bash scripts/build-prod.sh
#
# Serializes image builds so an 8 GB host does not OOM during export
# (parallel export of five targets → kernel exit 255, no Docker error).
set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$repo_root"
export DOCKER_BUILDKIT=1
compose_file="${COMPOSE_FILE:-docker/docker-compose.prod.yml}"
compose() {
docker compose --project-directory "$repo_root" \
-f "$compose_file" \
"$@"
}
for service in collab mcp orchestrator worker web; do
echo "[build-prod] building ${service}..."
compose build --pull "$service"
done
echo "[build-prod] done."

46
scripts/ct116-post-migration.sh Executable file
View file

@ -0,0 +1,46 @@
#!/usr/bin/env bash
# Operator runbook — run ON CT 116 after Forgejo migration + env updates.
# Copy-paste from your laptop:
# ssh root@192.168.68.116 'bash -s' < scripts/ct116-post-migration.sh
#
# Or on CT 116 directly:
# cd /opt/submatter && bash scripts/ct116-post-migration.sh
set -euo pipefail
APP_DIR="${APP_DIR:-/opt/submatter}"
cd "$APP_DIR"
echo "=== CT 116 post-migration ==="
if [[ -f scripts/migrate-origin-to-forgejo.sh ]]; then
bash scripts/migrate-origin-to-forgejo.sh
else
echo "WARN: migrate-origin-to-forgejo.sh missing — pull latest main first"
git fetch origin main && git checkout main && git reset --hard origin/main
fi
if grep -q '^NEXT_PUBLIC_COLLAB_SERVER_URL=ws://' .env 2>/dev/null; then
echo ""
echo "ACTION REQUIRED: update .env collab URLs to wss:// before deploy:"
echo " NEXT_PUBLIC_COLLAB_SERVER_URL=wss://collab.echodo.stillwell.cloud"
echo " COLLAB_SERVER_URL=wss://collab.echodo.stillwell.cloud"
echo ""
fi
missing=()
for var in OMNIROUTE_BASE_URL OMNIROUTE_API_KEY FORGEJO_TOKEN; do
if ! grep -q "^${var}=" .env 2>/dev/null; then
missing+=("$var")
fi
done
if ((${#missing[@]} > 0)); then
echo "Optional .env vars not set (add when ready): ${missing[*]}"
fi
echo ""
echo "=== deploying (serialized build — may take 15–30 min) ==="
bash scripts/deploy-prod.sh build
echo ""
echo "=== health ==="
docker compose -f docker/docker-compose.prod.yml -f docker/docker-compose.prod.standalone.yml ps

66
scripts/deploy-prod.sh Executable file
View file

@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Deploy Submatter / Echodo production stack on a dedicated host (CT 116).
#
# Run on the CT after git sync, or via SSH from CI:
# ssh root@192.168.68.116 'cd /opt/submatter && bash scripts/deploy-prod.sh'
#
# Modes:
# build — serialized image build + compose up (default; needs ~8 GB RAM)
# up — compose up only (images already built)
# pull — docker compose pull + up (when using a registry; future)
set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$repo_root"
mode="${1:-build}"
compose_files=(-f docker/docker-compose.prod.yml -f docker/docker-compose.prod.standalone.yml)
# Optional Phase B: export COMPOSE_PROFILES=agent-server before deploy
if [[ -n "${COMPOSE_PROFILES:-}" ]]; then
echo "[deploy-prod] profiles=${COMPOSE_PROFILES}"
fi
if [[ ! -f .env ]]; then
echo "[deploy-prod] ERROR: missing ${repo_root}/.env — copy from .env.example and fill production values." >&2
exit 1
fi
export DOCKER_BUILDKIT=1
compose() {
docker compose --project-directory "$repo_root" "${compose_files[@]}" "$@"
}
case "$mode" in
build)
bash scripts/build-prod.sh
compose up -d --remove-orphans
;;
up)
compose up -d --remove-orphans
;;
pull)
compose pull
compose up -d --remove-orphans
;;
*)
echo "Usage: $0 [build|up|pull]" >&2
exit 1
;;
esac
echo "[deploy-prod] waiting for web health..."
deadline=$((SECONDS + 120))
while (( SECONDS < deadline )); do
if compose ps web 2>/dev/null | grep -q "(healthy)"; then
echo "[deploy-prod] web is healthy"
compose ps
exit 0
fi
sleep 5
done
echo "[deploy-prod] WARN: web not healthy within 120s — check logs:" >&2
compose logs web --tail 40
exit 1

20
scripts/git-sync-deploy.sh Executable file
View file

@ -0,0 +1,20 @@
#!/usr/bin/env bash
# Sync production checkout to the deploy branch tip (used by deploy webhook + CI over SSH).
set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$repo_root"
branch="${DEPLOY_BRANCH:-main}"
remote="${DEPLOY_GIT_REMOTE:-origin}"
if [[ ! -d .git ]]; then
echo "[git-sync-deploy] ERROR: ${repo_root} is not a git repository" >&2
exit 1
fi
echo "[git-sync-deploy] fetching ${remote}/${branch}..."
git fetch "$remote" "$branch"
git checkout "$branch"
git reset --hard "${remote}/${branch}"
echo "[git-sync-deploy] at $(git rev-parse --short HEAD) ($(git log -1 --format='%s'))"

View file

@ -0,0 +1,52 @@
#!/usr/bin/env bash
# Point git origin at Forgejo Submatter (local dev machine or CT 116 production host).
#
# Usage:
# bash scripts/migrate-origin-to-forgejo.sh
# FORGEJO_REPO_URL=https://forgejo.stillwell.cloud/rstillwell/Submatter.git bash scripts/migrate-origin-to-forgejo.sh
#
# On CT 116:
# ssh root@192.168.68.116 'cd /opt/submatter && bash scripts/migrate-origin-to-forgejo.sh'
set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$repo_root"
forgejo_url="${FORGEJO_REPO_URL:-https://forgejo.stillwell.cloud/rstillwell/Submatter.git}"
branch="${DEPLOY_BRANCH:-main}"
if [[ ! -d .git ]]; then
echo "[migrate-origin] ERROR: not a git repo: ${repo_root}" >&2
exit 1
fi
old_url="$(git remote get-url origin 2>/dev/null || true)"
if [[ -n "$old_url" && "$old_url" != "$forgejo_url" ]]; then
echo "[migrate-origin] renaming old origin → github (was ${old_url})"
git remote rename origin github 2>/dev/null || true
if ! git remote get-url forgejo &>/dev/null; then
git remote add forgejo "$old_url"
fi
fi
if git remote get-url origin &>/dev/null; then
git remote set-url origin "$forgejo_url"
else
git remote add origin "$forgejo_url"
fi
echo "[migrate-origin] origin → ${forgejo_url}"
git remote -v
echo "[migrate-origin] fetching ${branch}..."
git fetch origin "$branch" || {
echo "[migrate-origin] WARN: fetch failed — check Forgejo credentials (git config credential.helper)" >&2
exit 1
}
current_branch="$(git branch --show-current 2>/dev/null || true)"
if [[ "$current_branch" != "$branch" ]]; then
git checkout "$branch" 2>/dev/null || git checkout -b "$branch" "origin/${branch}"
fi
echo "[migrate-origin] done. HEAD=$(git rev-parse --short HEAD)"