Commit graph

15 commits

Author SHA1 Message Date
Randall Stillwell
028840a887 Add SFTP support: replace FTP-only TLS toggle with protocol selector
Dreamhost (and most modern hosts) use SFTP (port 22), not FTP (port 21).
The previous implementation only supported FTP/FTPS via basic-ftp, causing
timeouts when connecting to SFTP servers.

Changes:
- Add ssh2-sftp-client for SFTP connections
- Replace ftpTls boolean with ftpProtocol ("sftp" | "ftp" | "ftps") in schema
- Rewrite ftp-watcher.ts to support both SFTP and FTP/FTPS protocols
- Update UI with protocol dropdown that auto-switches the default port
- Add ssh2/ssh2-sftp-client to serverExternalPackages in next.config
- Default to SFTP on port 22

Made-with: Cursor
2026-04-17 15:12:54 -05:00
Randall Stillwell
f53b08f99f Add dynamic fields, people directory, analytics, security hardening, and UX polish
Phase 1 - Security & Bug Fixes:
- Add requireApiAuth helper and protect all 25 unprotected API routes
- Add org-tenant scoping to all card, job, stats, and notification queries
- Fix SSRF in ai-test, mask secrets in settings API, fix middleware bypass
- Fix cards pagination routing, stat filter sync, drag-drop file passing
- Add PUT /api/auth/me for profile persistence, stuck job recovery
- Fix email watcher MIME type detection

Phase 2 - Dynamic Fields & Digital Survey:
- Add FormTemplate, FormField, Person, PasswordResetToken models to schema
- Add fieldData, formTemplateId, firstName, lastName, personId to ResponseCard
- Build FormTemplate CRUD API with field management and org scoping
- Build Form Builder UI with field ordering, type config, and section management
- Refactor card detail page to render fields dynamically from templates
- Add dynamic OCR prompt/schema generation from template fields
- Build public survey page at /s/[orgSlug]/[formSlug] with branding
- Add QR code generation API and share section component

Phase 3 - People & Analytics:
- Build People CRUD API with merge and batch auto-link endpoints
- Build People list and detail pages with search, merge dialog
- Add auto-link logic in OCR completion to match/create Person records
- Add /api/stats/trends endpoint with time series and team activity
- Build Reports page with Recharts (area charts, bar charts, pipeline)
- Upgrade dashboard with sparklines and People stat card

Phase 4 - UX Polish:
- Replace silent error handling with toast notifications across all pages
- Add loading skeletons, differentiated empty states
- Add ARIA labels, skip-to-content link, accessible column toggle
- Add forgot password flow, Cmd+K command palette, Collection Days pages
- Unify Echo branding and theme toggle consistency

Made-with: Cursor
2026-04-16 23:29:26 -05:00
Randall Stillwell
ab873f7f08 Add multi-workspace support with org switcher and open registration
- Open registration to all users (remove invite-only gate)
- Add domain auto-join: new users matching org allowedDomains get auto-added
- Track active workspace via activeOrgId on User model
- Refactor JWT callback to resolve active org from all memberships
- Add org switch, list, and create-personal API endpoints
- Add workspace-setup page for users without an org
- Build OrgSwitcher dropdown in sidebar header
- Add allowed email domains management to org settings

Made-with: Cursor
2026-04-16 18:08:08 -05:00
Randall Stillwell
21c863da01 Migrate to Vercel + Supabase + Upstash production stack
- Replace pdf2pic/GraphicsMagick with pdfjs-dist + @napi-rs/canvas for
  Vercel-compatible PDF rasterization
- Replace MinIO with Supabase Storage (S3-compatible); rename minio.ts
  to storage.ts and update all imports
- Replace in-memory job queue with Upstash QStash; upload route now
  persists files to storage before enqueuing, /api/jobs/process handles
  the QStash callback
- Convert email watcher from persistent IMAP connection to stateless
  scanInbox() polled by Vercel Cron every 2 minutes
- Add FTP watcher (basic-ftp) with cron polling for scanner integration
  via Dreamhost FTP drop directory
- Add FTP config fields to AppSettings schema
- Remove folder watcher (chokidar), standalone output, Docker-only code
- Update next.config.ts, middleware, instrumentation for serverless
- Add vercel.json with cron schedules for email and FTP polling
- Add migration scripts for database (pg_dump/restore) and storage
  (S3-to-S3 copy) with verification

Made-with: Cursor
2026-04-15 23:51:44 -05:00
Randall Stillwell
a975043670 Add onboarding wizard, email verification, integration architecture, and settings restructure
- Auto-sign-in after registration instead of redirect to login
- Email verification system with token generation, send/confirm API routes, and persistent banner
- 7-step onboarding wizard (org, location, services, upload source, AI, integrations, complete)
- Middleware redirects owners with incomplete onboarding to /onboarding
- Integration provider plugin architecture with registry and 6 providers (Planning Center, Monday.com, Airtable, Google Sheets, Webhook, CSV Export)
- Full integration CRUD API with test, sync, fields, and OAuth authorize/callback routes
- Refactored fireIntegrationEvent to use Integration model with legacy AppSettings fallback
- Migration script for existing Monday.com/webhook config to Integration rows
- Settings page restructured from monolithic 1290-line file into focused sub-routes with section navigation
- Integration hub UI with provider tiles, connect flow, and individual config pages
- Post-onboarding contextual guidance cards on dashboard with dismissible hints
- Schema: Integration model, onboardingComplete/onboardingStep on Organization, dismissedHints on OrgMember

Made-with: Cursor
2026-04-15 01:29:13 -05:00
Randall Stillwell
d3e7374439 Add SaaS foundation: Auth.js, dashboard shell, org model, auto-assignment
Major architectural upgrade preparing Echo OCR for self-hosted SaaS deployment:

- Auth: Built-in Auth.js v5 with credentials + Authentik OIDC SSO, JWT sessions,
  middleware route protection, login/signup/setup pages, registration API
- UI: Dashboard layout with collapsible sidebar nav, AppShell wrapper, route
  groups for (dashboard) and (auth), new pages for events/people/reports
- Schema: Auth.js tables (Account, Session, VerificationToken), Organization,
  OrgMember, Location, CollectionDay, Invitation, SystemConfig, ApiKey models;
  proper User relations to ResponseCard/ActivityLog/Notification
- Permissions: Role hierarchy (owner/admin/editor/reviewer/viewer) with
  action-based permission map and requirePermission/requireAuth helpers
- Onboarding: Multi-step setup wizard for first-user bootstrap (account, org,
  location) with SystemConfig tracking
- Events: CollectionDay model with rrule support for recurring church services
- Auto-assign: Event-aware card assignment engine replacing getPreviousSunday()
- Migration: seed-migration.ts script for upgrading existing deployments

Made-with: Cursor
2026-04-14 23:59:38 -05:00
Randall Stillwell
0ab9932599 Add multi-user card review workflow with role-based access
- Add User model synced from Authentik headers with admin/reviewer/viewer roles
- Add assignment fields (assignedToId, assignedById, reviewedById, etc.) to ResponseCard
- Add userId tracking to ActivityLog and Notification models
- Create auth.ts with getOrCreateUser() and role mapping from Authentik groups
- Create /api/users endpoint and /api/cards/assign batch assignment endpoint
- Gate card mutations behind role checks (viewers read-only, reviewers edit assigned only)
- Gate Monday.com push behind reviewStatus=reviewed instead of ocr_complete
- Add "My Cards" stat card, Assigned To filter, and assignment columns to table
- Add Assign button with user picker to batch selection toolbar (admin only)
- Update card detail: assignment banner, Mark Complete button, prev/next nav, reassign
- Make all field components accept readOnly prop for role-based editing
- Gate settings page behind admin role
- Add userId to stats API for per-user card counts
- Expose dbUser and role through UserProfileProvider context

Made-with: Cursor
2026-04-11 00:00:11 -05:00
Randall Stillwell
b2c29120cd Add First Time Guest Date and Salvation Date fields
- Add firstTimeGuestDate and salvationDate (DateTime?) to Prisma schema
- Auto-compute dates during OCR: find previous Sunday from card createdAt
  when visitType indicates first/second time guest or nextStep includes
  Baptism
- Add editable date inputs on card detail page in Workflow section
- Add to Monday.com mappable fields in settings for column mapping
- Add table columns (hidden by default) for both date fields
- Handle full ISO datetime strings in Monday.com date parser
- Apply same logic during reprocessing

Made-with: Cursor
2026-04-09 17:20:24 -05:00
Randall Stillwell
d58285785e Add Monday.com board fields and expand column mapper
- Add followUp, notes, serviceTime, planningCenter, iSaidYesBookSent,
  ftGuestLetterSent fields to ResponseCard to match Monday.com board
- Expand column mapper from 8 fields to all 32 ResponseCard fields with
  friendly labels matching the Easter Survey spreadsheet columns
- Add Workflow & Tracking section on card detail page for the new fields
- Fix pre-existing Buffer type error in images API route

Made-with: Cursor
2026-04-07 12:50:03 -05:00
Randall Stillwell
72375d2fae Add Integration Hub: Monday.com sync, webhooks, activity log, notifications
- Bidirectional Monday.com integration with column mapping and file uploads
- Generic webhook system with HMAC-SHA256 signing and configurable events
- Field-level activity log with diff tracking on card detail page
- Persistent notification center with unread badge in header
- Event dispatcher wired into OCR pipeline, card edits, exports, and deletes
- New Prisma models: ActivityLog, Notification; new fields on AppSettings and ResponseCard
- Settings UI with full Monday.com and webhook configuration panels

Made-with: Cursor
2026-04-07 12:23:29 -05:00
Randall Stillwell
41000337e3 Add IMAP email monitoring for scanned card attachments
Watches an email inbox via IMAP IDLE for incoming scanned cards,
extracts PDF/image attachments, and feeds them through the existing
OCR pipeline. Configurable via the Settings page with test connection
support.

Made-with: Cursor
2026-04-07 11:41:52 -05:00
Randall Stillwell
2db14b7336 Switch to Vercel AI Gateway for multi-provider OCR
Replace individual provider SDKs (@ai-sdk/openai, @ai-sdk/google,
@ai-sdk/anthropic) with the Vercel AI Gateway. Cloud models are now
accessed via a single AI_GATEWAY_API_KEY with provider/model strings
(e.g. openai/gpt-4o-mini, google/gemini-2.5-flash). Ollama remains
available as a local fallback via @ai-sdk/openai-compatible.

Made-with: Cursor
2026-03-11 11:05:38 -05:00
Randall Stillwell
1ec50d9033 Integrate Vercel AI SDK for multi-provider OCR
Replace the raw Ollama fetch-based OCR with the Vercel AI SDK,
adding support for OpenAI, Google Gemini, Anthropic, and Ollama
as selectable providers from the Settings page. Uses generateText
with Output.object() and Zod schemas for type-safe structured
data extraction.

Made-with: Cursor
2026-03-11 09:24:15 -05:00
Randall Stillwell
3b8d580ded Add database auto-migration, storage cleanup, and startup fixes
- Add start.sh that runs prisma db push before starting the server,
  creating tables automatically on first deploy
- Copy prisma CLI + engines into production image for runtime migrations
- Add configurable storage cleanup: source PDF retention (default 30d)
  and image retention (default 180d) with manual trigger in Settings
- Add /api/cleanup endpoint (GET for status, POST to run)
- Add retention settings to AppSettings schema
- Add ListObjectsV2 helper to minio.ts for bulk cleanup

Tested: local Docker build passes
Made-with: Cursor
2026-03-10 13:21:05 -05:00
Randall Stillwell
7e423a3e50 Build Echo OCR app: full-stack response card scanner
- Next.js 15 + Tailwind v4 + shadcn/ui + TanStack Table
- Prisma + PostgreSQL schema for ResponseCard, ProcessingJob, AppSettings
- Ollama vision integration with structured extraction prompts
- PDF-to-image pipeline with pdf2pic + sharp
- MinIO S3 storage for uploaded files and extracted images
- Chokidar-based folder monitoring for automatic processing
- REST API (9 endpoints) ready for Monday.com integration
- Dashboard with filterable data table, chip filters, bulk actions, CSV export
- Card detail page with editable fields and side-by-side scanned images
- Upload page with drag-and-drop and real-time processing queue
- Settings page for Ollama, folder watch, and Monday.com config
- Dockerfile (multi-stage) + docker-compose for local dev
- Configured for Coolify deployment at echo.stillwell.cloud

Made-with: Cursor
2026-03-10 07:17:45 -05:00