Commit graph

16 commits

Author SHA1 Message Date
Randall Stillwell
028840a887 Add SFTP support: replace FTP-only TLS toggle with protocol selector
Dreamhost (and most modern hosts) use SFTP (port 22), not FTP (port 21).
The previous implementation only supported FTP/FTPS via basic-ftp, causing
timeouts when connecting to SFTP servers.

Changes:
- Add ssh2-sftp-client for SFTP connections
- Replace ftpTls boolean with ftpProtocol ("sftp" | "ftp" | "ftps") in schema
- Rewrite ftp-watcher.ts to support both SFTP and FTP/FTPS protocols
- Update UI with protocol dropdown that auto-switches the default port
- Add ssh2/ssh2-sftp-client to serverExternalPackages in next.config
- Default to SFTP on port 22

Made-with: Cursor
2026-04-17 15:12:54 -05:00
Randall Stillwell
f53b08f99f Add dynamic fields, people directory, analytics, security hardening, and UX polish
Phase 1 - Security & Bug Fixes:
- Add requireApiAuth helper and protect all 25 unprotected API routes
- Add org-tenant scoping to all card, job, stats, and notification queries
- Fix SSRF in ai-test, mask secrets in settings API, fix middleware bypass
- Fix cards pagination routing, stat filter sync, drag-drop file passing
- Add PUT /api/auth/me for profile persistence, stuck job recovery
- Fix email watcher MIME type detection

Phase 2 - Dynamic Fields & Digital Survey:
- Add FormTemplate, FormField, Person, PasswordResetToken models to schema
- Add fieldData, formTemplateId, firstName, lastName, personId to ResponseCard
- Build FormTemplate CRUD API with field management and org scoping
- Build Form Builder UI with field ordering, type config, and section management
- Refactor card detail page to render fields dynamically from templates
- Add dynamic OCR prompt/schema generation from template fields
- Build public survey page at /s/[orgSlug]/[formSlug] with branding
- Add QR code generation API and share section component

Phase 3 - People & Analytics:
- Build People CRUD API with merge and batch auto-link endpoints
- Build People list and detail pages with search, merge dialog
- Add auto-link logic in OCR completion to match/create Person records
- Add /api/stats/trends endpoint with time series and team activity
- Build Reports page with Recharts (area charts, bar charts, pipeline)
- Upgrade dashboard with sparklines and People stat card

Phase 4 - UX Polish:
- Replace silent error handling with toast notifications across all pages
- Add loading skeletons, differentiated empty states
- Add ARIA labels, skip-to-content link, accessible column toggle
- Add forgot password flow, Cmd+K command palette, Collection Days pages
- Unify Echo branding and theme toggle consistency

Made-with: Cursor
2026-04-16 23:29:26 -05:00
Randall Stillwell
bf47fd262c Replace nodemailer with Brevo HTTP API for transactional email
- Swap nodemailer SMTP transport for @getbrevo/brevo SDK, which uses
  HTTP with built-in retries (better for Vercel serverless)
- Add sendInvitationEmail function and wire it into the org invitations
  POST route so new invites trigger an email automatically
- Update .env.example with BREVO_API_KEY, EMAIL_FROM_NAME,
  EMAIL_FROM_ADDRESS replacing the old SMTP_* vars

Made-with: Cursor
2026-04-16 12:51:17 -05:00
Randall Stillwell
21c863da01 Migrate to Vercel + Supabase + Upstash production stack
- Replace pdf2pic/GraphicsMagick with pdfjs-dist + @napi-rs/canvas for
  Vercel-compatible PDF rasterization
- Replace MinIO with Supabase Storage (S3-compatible); rename minio.ts
  to storage.ts and update all imports
- Replace in-memory job queue with Upstash QStash; upload route now
  persists files to storage before enqueuing, /api/jobs/process handles
  the QStash callback
- Convert email watcher from persistent IMAP connection to stateless
  scanInbox() polled by Vercel Cron every 2 minutes
- Add FTP watcher (basic-ftp) with cron polling for scanner integration
  via Dreamhost FTP drop directory
- Add FTP config fields to AppSettings schema
- Remove folder watcher (chokidar), standalone output, Docker-only code
- Update next.config.ts, middleware, instrumentation for serverless
- Add vercel.json with cron schedules for email and FTP polling
- Add migration scripts for database (pg_dump/restore) and storage
  (S3-to-S3 copy) with verification

Made-with: Cursor
2026-04-15 23:51:44 -05:00
Randall Stillwell
a975043670 Add onboarding wizard, email verification, integration architecture, and settings restructure
- Auto-sign-in after registration instead of redirect to login
- Email verification system with token generation, send/confirm API routes, and persistent banner
- 7-step onboarding wizard (org, location, services, upload source, AI, integrations, complete)
- Middleware redirects owners with incomplete onboarding to /onboarding
- Integration provider plugin architecture with registry and 6 providers (Planning Center, Monday.com, Airtable, Google Sheets, Webhook, CSV Export)
- Full integration CRUD API with test, sync, fields, and OAuth authorize/callback routes
- Refactored fireIntegrationEvent to use Integration model with legacy AppSettings fallback
- Migration script for existing Monday.com/webhook config to Integration rows
- Settings page restructured from monolithic 1290-line file into focused sub-routes with section navigation
- Integration hub UI with provider tiles, connect flow, and individual config pages
- Post-onboarding contextual guidance cards on dashboard with dismissible hints
- Schema: Integration model, onboardingComplete/onboardingStep on Organization, dismissedHints on OrgMember

Made-with: Cursor
2026-04-15 01:29:13 -05:00
Randall Stillwell
d3e7374439 Add SaaS foundation: Auth.js, dashboard shell, org model, auto-assignment
Major architectural upgrade preparing Echo OCR for self-hosted SaaS deployment:

- Auth: Built-in Auth.js v5 with credentials + Authentik OIDC SSO, JWT sessions,
  middleware route protection, login/signup/setup pages, registration API
- UI: Dashboard layout with collapsible sidebar nav, AppShell wrapper, route
  groups for (dashboard) and (auth), new pages for events/people/reports
- Schema: Auth.js tables (Account, Session, VerificationToken), Organization,
  OrgMember, Location, CollectionDay, Invitation, SystemConfig, ApiKey models;
  proper User relations to ResponseCard/ActivityLog/Notification
- Permissions: Role hierarchy (owner/admin/editor/reviewer/viewer) with
  action-based permission map and requirePermission/requireAuth helpers
- Onboarding: Multi-step setup wizard for first-user bootstrap (account, org,
  location) with SystemConfig tracking
- Events: CollectionDay model with rrule support for recurring church services
- Auto-assign: Event-aware card assignment engine replacing getPreviousSunday()
- Migration: seed-migration.ts script for upgrading existing deployments

Made-with: Cursor
2026-04-14 23:59:38 -05:00
Randall Stillwell
393114e9be Fix PDF page count detection and add processing diagnostics
The old getPdfPageCount used a fragile regex on raw PDF binary that
could undercount pages, causing survey sides to be skipped entirely.
Replaced with pdf-lib's PDFDocument.load() for reliable counting,
with the old regex as a fallback.

Added logging for page counts and pairing to diagnose missing backs.

Made-with: Cursor
2026-04-07 22:12:21 -05:00
Randall Stillwell
41000337e3 Add IMAP email monitoring for scanned card attachments
Watches an email inbox via IMAP IDLE for incoming scanned cards,
extracts PDF/image attachments, and feeds them through the existing
OCR pipeline. Configurable via the Settings page with test connection
support.

Made-with: Cursor
2026-04-07 11:41:52 -05:00
Randall Stillwell
81184040bb Fix Docker build: add explicit gateway dep and reduce memory pressure
- Add @ai-sdk/gateway as direct dependency (was only a transitive dep
  of ai, fragile in Docker builds with --ignore-scripts)
- Avoid double prisma generate in Dockerfile by calling next build
  directly instead of npm run build
- Set NODE_OPTIONS max-old-space-size=1536 for the builder stage to
  prevent OOM on the 3GB Coolify build server

Made-with: Cursor
2026-03-12 11:07:59 -05:00
Randall Stillwell
2db14b7336 Switch to Vercel AI Gateway for multi-provider OCR
Replace individual provider SDKs (@ai-sdk/openai, @ai-sdk/google,
@ai-sdk/anthropic) with the Vercel AI Gateway. Cloud models are now
accessed via a single AI_GATEWAY_API_KEY with provider/model strings
(e.g. openai/gpt-4o-mini, google/gemini-2.5-flash). Ollama remains
available as a local fallback via @ai-sdk/openai-compatible.

Made-with: Cursor
2026-03-11 11:05:38 -05:00
Randall Stillwell
1ec50d9033 Integrate Vercel AI SDK for multi-provider OCR
Replace the raw Ollama fetch-based OCR with the Vercel AI SDK,
adding support for OpenAI, Google Gemini, Anthropic, and Ollama
as selectable providers from the Settings page. Uses generateText
with Output.object() and Zod schemas for type-safe structured
data extraction.

Made-with: Cursor
2026-03-11 09:24:15 -05:00
Randall Stillwell
c879328775 Update npm packages and remove unused minio dependency
- Remove unused minio package (using @aws-sdk/client-s3 instead)
- Update @aws-sdk/client-s3, s3-request-presigner to latest
- Update react, react-dom to 19.2.4
- Update @types/node to v25
- Update shadcn to 4.0.5
- Run npm audit fix for non-breaking fixes
- Remaining 9 vulns are in Prisma's transitive deps (hono, lodash)
  and cannot be fixed without downgrading Prisma

Tested: local Docker build passes
Made-with: Cursor
2026-03-11 07:55:34 -05:00
Randall Stillwell
3b8d580ded Add database auto-migration, storage cleanup, and startup fixes
- Add start.sh that runs prisma db push before starting the server,
  creating tables automatically on first deploy
- Copy prisma CLI + engines into production image for runtime migrations
- Add configurable storage cleanup: source PDF retention (default 30d)
  and image retention (default 180d) with manual trigger in Settings
- Add /api/cleanup endpoint (GET for status, POST to run)
- Add retention settings to AppSettings schema
- Add ListObjectsV2 helper to minio.ts for bulk cleanup

Tested: local Docker build passes
Made-with: Cursor
2026-03-10 13:21:05 -05:00
Randall Stillwell
7e423a3e50 Build Echo OCR app: full-stack response card scanner
- Next.js 15 + Tailwind v4 + shadcn/ui + TanStack Table
- Prisma + PostgreSQL schema for ResponseCard, ProcessingJob, AppSettings
- Ollama vision integration with structured extraction prompts
- PDF-to-image pipeline with pdf2pic + sharp
- MinIO S3 storage for uploaded files and extracted images
- Chokidar-based folder monitoring for automatic processing
- REST API (9 endpoints) ready for Monday.com integration
- Dashboard with filterable data table, chip filters, bulk actions, CSV export
- Card detail page with editable fields and side-by-side scanned images
- Upload page with drag-and-drop and real-time processing queue
- Settings page for Ollama, folder watch, and Monday.com config
- Dockerfile (multi-stage) + docker-compose for local dev
- Configured for Coolify deployment at echo.stillwell.cloud

Made-with: Cursor
2026-03-10 07:17:45 -05:00
Randall Stillwell
6a6a140139 feat: initial commit 2026-03-09 15:31:45 -05:00
Randall Stillwell
95d05ac92e Initial commit from Create Next App 2026-03-09 15:31:21 -05:00