Compare commits

...
Author SHA1 Message Date
Andrew BeveridgeandClaude Opus 4.5 8e5e81d6b8 docs: Add gen_users migration documentation
- README.md: Add entry in Recent Changes about user database separation
- ARCHITECTURE.md: Add Firestore Collections section documenting gen_users
- LESSONS-LEARNED.md: Add lesson about separate collections for multi-app projects

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 16:59:24 -05:00
Andrew Beveridge 73faacaebe Merge branch 'main' of github.com:nomadkaraoke/karaoke-gen 2026-01-03 16:59:11 -05:00
Andrew BeveridgeandClaude Opus 4.5 499c386ad7 fix: Remove double slashes in GCS upload paths
The prefix already ends with "/" from the gcs_uri, adding another "/"
creates paths like "encoded//file.mp4" instead of "encoded/file.mp4"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 16:49:33 -05:00
Andrew Beveridge 3f5a423f98 Merge branch 'main' of github.com:nomadkaraoke/karaoke-gen 2026-01-03 16:26:55 -05:00
Andrew BeveridgeandClaude Opus 4.5 b87e29d622 Fix GCE encoding worker output paths and format handling
- Convert local file paths to blob paths after upload (backend expects
  paths like "jobs/id/encoded/file.mp4", not full gs:// URIs)
- Add mp4_4k_lossless and mp4_4k_lossy format handlers to match what
  video worker sends

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 16:07:31 -05:00
4 changed files with 38 additions and 3 deletions

No files matched your search

+14
View File
@@ -155,6 +155,20 @@ LyricsTranscriber LyricsTranscriber
| Genius | Reference lyrics | Yes |
| YouTube API | Video upload | Optional |
## Firestore Collections
karaoke-gen shares a GCP project (`nomadkaraoke`) with karaoke-decide, but uses separate Firestore collections:
| Collection | Purpose | Key Fields |
|------------|---------|------------|
| `gen_users` | karaoke-gen user accounts | email, credits, role, is_active |
| `jobs` | Karaoke generation jobs | job_id, user_email, status, state_data |
| `sessions` | Magic link auth sessions | user_email, token, expires_at |
| `magic_links` | Passwordless auth tokens | email, token, expires_at, used |
| `beta_feedback` | Beta program feedback | user_email, ratings, comments |
**Note**: The `users` collection in the same Firestore instance belongs to karaoke-decide (different schema: user_id, is_guest, quiz_* fields). Don't use it for karaoke-gen.
## Tech Stack
- **Backend**: FastAPI, Python 3.12, Cloud Run
+19
View File
@@ -4,6 +4,25 @@ Key insights for future AI agents working on this codebase.
## Architecture Decisions
### Separate Collections for Multi-App Projects
**Problem**: karaoke-gen and karaoke-decide shared the same `users` Firestore collection with incompatible schemas. karaoke-gen users have `credits`, `role`, `is_active`; karaoke-decide users have `user_id`, `is_guest`, `quiz_*` fields.
**Symptoms**:
- Admin dashboard showed users from both apps mixed together
- Potential for data corruption if wrong app writes to wrong user
- Hard to run collection-wide queries (indexes, aggregations) for one app
**Solution**: Migrate karaoke-gen to dedicated `gen_users` collection. Created migration script that:
1. Identified karaoke-gen users by schema (has `credits` + `role` + `is_active`, lacks `user_id` + `is_guest`)
2. Copied 42 users to `gen_users` with same document IDs (email addresses)
3. Updated Pulumi indexes from `users` to `gen_users`
4. Left karaoke-decide's `users` collection untouched
**Key insight**: When multiple apps share a GCP project, use distinct collection names from day one. Prefixing collections with app name (e.g., `gen_users`, `decide_users`) avoids schema conflicts and makes ownership clear.
**Migration script**: `scripts/migrate_users_to_gen_users.py` - supports `--dry-run` and `--delete` flags.
### LyricsTranscriber: Library, Not Server
**Problem**: LyricsTranscriber's `ReviewServer` blocks waiting for human input - incompatible with async cloud architecture.
+2
View File
@@ -36,6 +36,8 @@
## Recent Changes
- **User Database Separation** (2026-01-03): Migrated karaoke-gen users from shared `users` collection to dedicated `gen_users` collection. This separates karaoke-gen user data from karaoke-decide user data (both apps share the same GCP project/Firestore instance). Migration script in `scripts/migrate_users_to_gen_users.py`. See [ARCHITECTURE.md](ARCHITECTURE.md#firestore-collections) for collection details.
- **Admin Dashboard** (2026-01-03): Added comprehensive admin dashboard at `/admin` with: stats overview (users, jobs, credits), user management (search, sort, add credits, toggle role, enable/disable), job browser (filter by status/user, view details, delete), and beta program monitoring. Query-param based routing for static export compatibility. See [API.md](API.md#admin-endpoints) for backend endpoints.
- **GCE Encoding Worker** (2026-01-03): Added optional high-performance encoding on dedicated GCE VM (C4-standard-8 with Intel Granite Rapids 3.9 GHz). Uses `USE_GCE_ENCODING=true` env var. Falls back to Cloud Run if unavailable. See [ARCHITECTURE.md](ARCHITECTURE.md).
+3 -3
View File
@@ -1268,12 +1268,12 @@ def upload_to_gcs(local_path: Path, gcs_uri: str):
# Upload a file or folder to GCS
parts = gcs_uri.replace("gs://", "").split("/", 1)
bucket_name = parts[0]
prefix = parts[1] if len(parts) > 1 else ""
prefix = parts[1].rstrip("/") if len(parts) > 1 else ""
bucket = storage_client.bucket(bucket_name)
if local_path.is_file():
blob_name = f"{prefix}/{local_path.name}".strip("/")
blob_name = f"{prefix}/{local_path.name}" if prefix else local_path.name
blob = bucket.blob(blob_name)
blob.upload_from_filename(str(local_path))
logger.info(f"Uploaded: {local_path} -> gs://{bucket_name}/{blob_name}")
@@ -1281,7 +1281,7 @@ def upload_to_gcs(local_path: Path, gcs_uri: str):
for file in local_path.rglob("*"):
if file.is_file():
rel_path = file.relative_to(local_path)
blob_name = f"{prefix}/{rel_path}".strip("/")
blob_name = f"{prefix}/{rel_path}" if prefix else str(rel_path)
blob = bucket.blob(blob_name)
blob.upload_from_filename(str(file))
logger.info(f"Uploaded: {file} -> gs://{bucket_name}/{blob_name}")