ff80753fe4
Closes P0 #3 from .convoys/ship-readiness.md. scripts/setup-neon-db.js: - Read ADMIN_INITIAL_PASSWORD env var at the top of setupNeonDatabase() before any DB connection. Fail loudly (process.exit(1)) with an actionable message if unset or empty. - Replace bcrypt.hash('admin123', 12) with bcrypt.hash(adminPassword, 12). - Delete the two console.log lines that echoed admin user + password to stdout (R3 - stdout leak into CI logs). - Keep ON CONFLICT (email) DO NOTHING unchanged. Re-running setup-db on an env with the admin row already present is a no-op for the password (R4 - silent rotation prevention). Rotation of existing weak-hash admin rows is out of scope (Decision A - queued for the rotate-default-admin follow-up convoy). README.md: - Add ADMIN_INITIAL_PASSWORD to the install-step env-example block with a CI-secret note (and add KV_REST_API_URL/KV_REST_API_TOKEN for completeness; they're optional for local dev). - Replace the "Default Admin Account" section with "First-time admin setup", documenting the env var, openssl rand suggestion, and the operator rotation note for envs that predate this change. - Zero occurrences of 'admin123' remain in README.md (the operator rotation note refers to "the prior weak default" instead of naming the literal string, so grep verification A2 holds). Decisions A1 (going-forward only), B (operational change allowed), C1 (no vitest coverage - manual smoke in PR description) per .convoys/drop-public-setup.md section Decisions. Smoke output: see PR description. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|---|---|---|
| .. | ||
| add-card-columns.js | ||
| add-collaboration-features.js | ||
| add-collection-slugs.js | ||
| add-favorites-system.js | ||
| add-image-column.js | ||
| add-system-collection-column.js | ||
| add-updated-at-column.js | ||
| add-user-profile-columns.js | ||
| add-user-profile-fields.js | ||
| bulk-import-all.js | ||
| create-sample-cards.js | ||
| create-sample-collections.js | ||
| create-test-users.js | ||
| demote-admin-to-user.js | ||
| fix-lorcana-images.js | ||
| fix-user-cards-constraints.js | ||
| import-lorcana-simple.js | ||
| import-lorcana.js | ||
| import-popular-sets.js | ||
| list-users.js | ||
| log-convoy-event.sh | ||
| promote-user-to-admin.js | ||
| README.md | ||
| reset-db.js | ||
| seed-collections-alice-bob.js | ||
| seed-collections-with-cards.js | ||
| setup-neon-db.js | ||
| wt.sh | ||
TCG Vault Bulk Import Scripts
This directory contains scripts for bulk importing TCG card data into the database.
Available Scripts
Card Import Scripts
1. import-popular-sets.js - Popular Sets Import
Imports the most popular and recent sets from all three TCGs (Magic, Pokemon, Lorcana).
Usage:
npm run import-popular
What it imports:
- Magic: The Gathering: ~100+ sets from Alpha to recent releases
- Pokemon: ~100+ sets from Base Set to current Scarlet & Violet
- Lorcana: All 3 available sets
Estimated time: 2-4 hours (depending on API response times)
2. bulk-import-all.js - Complete Import
Imports ALL available sets from all TCGs (comprehensive import).
Usage:
npm run import-all
What it imports:
- Magic: The Gathering: 100+ sets (Alpha to current)
- Pokemon: 100+ sets (Base Set to current)
- Lorcana: All available sets
Estimated time: 4-8 hours (depending on API response times)
User Management Scripts
3. list-users.js - List All Users
Lists all users in the database with their roles and details.
Usage:
node scripts/list-users.js
Output: Shows user ID, email, role (admin/user), and creation date.
4. promote-user-to-admin.js - Promote User to Admin
Promotes a regular user to admin role.
Usage:
node scripts/promote-user-to-admin.js user@example.com
Requirements: User must be registered first.
5. demote-admin-to-user.js - Demote Admin to User
Demotes an admin back to regular user role.
Usage:
node scripts/demote-admin-to-user.js admin@example.com
Safety: Ensures at least one admin always remains in the system.
Database Management Scripts
6. setup-neon-db.js - Database Setup
Sets up the Neon PostgreSQL database with all required tables and creates the default admin user.
Usage:
node scripts/setup-neon-db.js
7. reset-db.js - Database Reset
Resets the database by dropping and recreating all tables.
Usage:
node scripts/reset-db.js
How It Works
- Sequential Import: Scripts import sets one by one to avoid overwhelming the APIs
- Error Handling: Failed imports are logged but don't stop the process
- Progress Tracking: Real-time console output shows progress
- Results Logging: Detailed results are saved to JSON files
- Rate Limiting: 1-second delays between imports to be respectful to APIs
Output Files
After running, you'll get timestamped JSON files with detailed results:
popular-sets-import-results-[timestamp].jsonbulk-import-results-[timestamp].json
These files contain:
- Success/failure status for each set
- Number of cards imported per set
- Error messages for failed imports
- Summary statistics
Prerequisites
- Server Running: Make sure your Next.js dev server is running (
npm run dev) - Database Setup: Ensure the database is initialized (
npm run setup-db) - Dependencies: All required packages are installed
Recommendations
For First-Time Setup
Start with the popular sets import:
npm run import-popular
This will give you a solid foundation with the most relevant cards.
For Complete Database
If you want everything, use the full import:
npm run import-all
For Ongoing Management
After the initial bulk import, use the admin interface at /admin/card-import for:
- Importing new sets as they release
- Selective imports of specific sets
- Monitoring import progress
Troubleshooting
Common Issues
- API Rate Limits: If you get rate limit errors, the scripts will continue but log failures
- Network Issues: Scripts will retry and continue with the next set
- Server Restart: If the server restarts, just restart the import script
Monitoring Progress
Watch the console output for:
- ✅ Successful imports with card counts
- ❌ Failed imports with error messages
- 📊 Summary statistics at the end
Stopping and Resuming
You can stop the script with Ctrl+C and restart it later. The scripts will start from the beginning, but the database will only store unique cards (no duplicates).
API Endpoints Used
POST /api/cards/import-mtg- Magic: The Gathering importsPOST /api/cards/import-pokemon- Pokemon importsPOST /api/cards/import-lorcana- Lorcana imports
Data Sources
- Magic: The Gathering: Scryfall API
- Pokemon: Pokemon TCG API
- Lorcana: Lorcana API (limited availability)
Performance Notes
- Each set typically contains 100-400 cards
- Total database size after full import: ~50,000-100,000 cards
- Import speed: ~1 set per minute (with delays)
- Database storage: ~100-200MB after full import