Compare commits
No files matched your search
@@ -0,0 +1,218 @@
|
||||
# Plan: Job Email Notifications & Admin Message Tools
|
||||
|
||||
**Created:** 2026-01-06
|
||||
**Branch:** feat/sess-20260106-0116-add-email-button
|
||||
**Status:** Draft
|
||||
|
||||
## Overview
|
||||
|
||||
Add comprehensive email notification system for karaoke-gen that:
|
||||
1. **Auto-sends emails** when jobs complete or need user input (after 5-minute idle threshold)
|
||||
2. **Uses cloud-stored templates** for the job completion message
|
||||
3. **Provides admin tools** to copy/send completion messages manually (for Fiverr, etc.)
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional Requirements
|
||||
- [ ] Auto-send job completion email to user (CC: gen@nomadkaraoke.com)
|
||||
- [ ] Auto-send "action needed" reminder emails when user is idle for 5+ minutes at blocking states (lyrics review, instrumental selection)
|
||||
- [ ] Store email templates in GCS (editable without code deploy)
|
||||
- [ ] Admin button: "Copy Message" - copies templated completion message to clipboard
|
||||
- [ ] Admin button: "Send Email" - prompts for email address, then sends completion message
|
||||
|
||||
### Non-Functional Requirements
|
||||
- [ ] Email sending must not block job completion (async/background)
|
||||
- [ ] Idle detection via Cloud Tasks scheduled task (not polling)
|
||||
- [ ] Template changes should take effect without restart (no long-term caching)
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### 1. Template Storage (GCS)
|
||||
Store templates in `gs://karaoke-gen-storage/templates/`:
|
||||
- `job-completion.txt` - Plain text template for job completion
|
||||
- `job-completion.html` - HTML version (optional, for rich emails)
|
||||
- `action-needed-lyrics.txt` - Reminder for lyrics review
|
||||
- `action-needed-instrumental.txt` - Reminder for instrumental selection
|
||||
|
||||
**Template Variables:**
|
||||
- `{name}` - User's display name or "there" if unknown
|
||||
- `{youtube_url}` - YouTube video URL
|
||||
- `{dropbox_url}` - Dropbox folder URL
|
||||
- `{artist}` - Artist name
|
||||
- `{title}` - Song title
|
||||
- `{job_id}` - Job ID
|
||||
- `{review_url}` - Lyrics review URL (for action-needed emails)
|
||||
- `{instrumental_url}` - Instrumental selection URL
|
||||
- `{feedback_url}` - Feedback form URL (for completion emails)
|
||||
|
||||
### 2. Email Service Extensions
|
||||
|
||||
Extend `backend/services/email_service.py`:
|
||||
- Add `send_job_completion()` method with CC support
|
||||
- Add `send_action_reminder()` method
|
||||
- Add `render_template()` helper to fetch & render GCS templates
|
||||
|
||||
### 3. Auto-Email Triggers
|
||||
|
||||
**Job Completion:**
|
||||
- Trigger point: When job enters `COMPLETE` status in `job_manager.transition_to_state()`
|
||||
- Implementation: Call email service async in background task
|
||||
|
||||
**Action Reminders (5-min idle detection):**
|
||||
- When job enters `AWAITING_REVIEW` or `AWAITING_INSTRUMENTAL_SELECTION`:
|
||||
- Record `blocking_state_entered_at` timestamp in `state_data`
|
||||
- Schedule a Cloud Tasks task for 5 minutes later
|
||||
- The scheduled task checks if job is still in blocking state and user hasn't started review
|
||||
- If idle, send reminder email and set `reminder_sent` flag to prevent duplicates
|
||||
- **Only one reminder per job** - if `reminder_sent` is true, skip sending
|
||||
|
||||
### 4. Admin UI (Frontend)
|
||||
|
||||
Add admin-only buttons to completed jobs in `OutputLinks.tsx`:
|
||||
- "Copy Message" button - fetches rendered template, copies to clipboard
|
||||
- "Send Email" button - opens dialog for email input, calls API to send
|
||||
|
||||
### 5. Backend API Endpoints
|
||||
|
||||
New endpoints in `backend/api/routes/admin.py`:
|
||||
- `GET /api/admin/jobs/{job_id}/completion-message` - Returns rendered completion message
|
||||
- `POST /api/admin/jobs/{job_id}/send-completion-email` - Sends email to specified address
|
||||
|
||||
New internal endpoint for scheduled reminder:
|
||||
- `POST /api/internal/jobs/{job_id}/check-idle-reminder` - Called by Cloud Tasks
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Template Infrastructure
|
||||
1. [ ] Upload initial template to GCS from user's existing template file
|
||||
2. [ ] Create `template_service.py` to fetch/render templates from GCS
|
||||
3. [ ] Add unit tests for template rendering
|
||||
|
||||
### Phase 2: Email Extensions
|
||||
4. [ ] Extend `email_service.py` with CC support in `send_email()`
|
||||
5. [ ] Add `send_job_completion()` method
|
||||
6. [ ] Add `send_action_reminder()` method
|
||||
7. [ ] Add unit tests for new email methods
|
||||
|
||||
### Phase 3: Auto-Email on Completion
|
||||
8. [ ] Modify `job_manager.py` to trigger completion email when entering COMPLETE
|
||||
9. [ ] Add integration test for completion email trigger
|
||||
|
||||
### Phase 4: Idle Reminder System
|
||||
10. [ ] Add `blocking_state_entered_at` field tracking to job state transitions
|
||||
11. [ ] Create Cloud Tasks queue for idle reminders (`idle-reminder-queue`)
|
||||
12. [ ] Schedule 5-minute delayed task when entering blocking states
|
||||
13. [ ] Implement `/api/internal/jobs/{job_id}/check-idle-reminder` endpoint
|
||||
14. [ ] Add tests for idle reminder flow
|
||||
|
||||
### Phase 5: Admin API
|
||||
15. [ ] Add `GET /api/admin/jobs/{job_id}/completion-message` endpoint
|
||||
16. [ ] Add `POST /api/admin/jobs/{job_id}/send-completion-email` endpoint
|
||||
17. [ ] Add tests for admin endpoints
|
||||
|
||||
### Phase 6: Admin UI
|
||||
18. [ ] Add `isAdmin` check to `OutputLinks.tsx` component
|
||||
19. [ ] Add "Copy Message" button (admin-only) with loading state
|
||||
20. [ ] Add "Send Email" button (admin-only) with email input dialog
|
||||
21. [ ] Add E2E test for admin buttons
|
||||
|
||||
### Phase 7: Documentation
|
||||
22. [ ] Document template management in docs/DEVELOPMENT.md
|
||||
23. [ ] Update docs/API.md with new endpoints
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `backend/services/template_service.py` | Create | GCS template fetching and rendering |
|
||||
| `backend/services/email_service.py` | Modify | Add CC support, completion/reminder methods |
|
||||
| `backend/services/job_manager.py` | Modify | Trigger emails on state transitions |
|
||||
| `backend/services/worker_service.py` | Modify | Add idle reminder task scheduling |
|
||||
| `backend/api/routes/admin.py` | Modify | Add completion message/email endpoints |
|
||||
| `backend/api/routes/internal.py` | Modify | Add idle reminder check endpoint |
|
||||
| `backend/models/job.py` | Modify | Add `blocking_state_entered_at`, `reminder_sent_at` to state_data docs |
|
||||
| `infrastructure/__main__.py` | Modify | Add idle-reminder-queue Cloud Tasks queue |
|
||||
| `frontend/components/job/OutputLinks.tsx` | Modify | Add admin-only Copy/Send buttons |
|
||||
| `frontend/lib/api.ts` | Modify | Add API methods for admin endpoints |
|
||||
| `tests/unit/test_template_service.py` | Create | Template rendering tests |
|
||||
| `tests/unit/test_email_service_completion.py` | Create | Completion email tests |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
**Unit Tests:**
|
||||
- Template rendering with all variable combinations
|
||||
- Email service CC support
|
||||
- Completion/reminder email generation
|
||||
- Idle detection logic
|
||||
|
||||
**Integration Tests:**
|
||||
- Full flow: job completion → email sent
|
||||
- Idle reminder: job enters blocking state → 5 min → reminder sent
|
||||
- Admin endpoint authentication
|
||||
|
||||
**Manual Testing:**
|
||||
- Verify emails render correctly in Gmail/Outlook
|
||||
- Test admin buttons on completed job
|
||||
- Test clipboard copy works across browsers
|
||||
|
||||
## Open Questions
|
||||
|
||||
- [x] Where to store templates? → **GCS bucket: `gs://karaoke-gen-storage/templates/`**
|
||||
- [x] Should we rate-limit reminder emails? → **Max 1 reminder per job** (send once after 5 min idle, then stop)
|
||||
- [x] Should completion emails also include a feedback request link? → **Yes, for all users**
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
1. **Email sending failures** - All email sends are async/fire-and-forget; job completion unaffected
|
||||
2. **Template fetch failures** - Fall back to hardcoded template in code
|
||||
3. **Idle reminder failures** - Jobs continue normally; users can still access review UI manually
|
||||
4. **Feature flag** - Add `ENABLE_AUTO_EMAILS=true/false` env var for quick disable
|
||||
|
||||
## Template Content
|
||||
|
||||
Initial template (from user's file, with feedback link added):
|
||||
|
||||
```
|
||||
Hi {name},
|
||||
|
||||
Thanks for your order!
|
||||
|
||||
Here's the link for the karaoke video published to YouTube:
|
||||
{youtube_url}
|
||||
|
||||
Here's the dropbox folder with all the finished files and source files, including:
|
||||
- "(Final Karaoke Lossless).mkv": combined karaoke video in 4k H264 with lossless FLAC audio
|
||||
- "(Final Karaoke).mp4": combined karaoke video with title/end screen in 4k H264/AAC
|
||||
- "(Final Karaoke 720p).mp4": combined karaoke video in 720p H264/AAC (smaller file for older systems)
|
||||
- "(With Vocals).mp4": sing along video in 4k H264/AAC with original vocals
|
||||
- "(Karaoke).mov": karaoke video output from MidiCo (no title/end screen)
|
||||
- "(Title).mov"/"(End).mov": title card and end screen videos
|
||||
- "(Final Karaoke CDG).zip": CDG+MP3 format for older/commercial karaoke systems
|
||||
- "(Final Karaoke TXT).zip": TXT+MP3 format for Power Karaoke
|
||||
- stems/*.flac: various separated instrumental and vocal audio stems in lossless format
|
||||
- lyrics/*.txt song lyrics from various sources in plain text format
|
||||
|
||||
{dropbox_url}
|
||||
|
||||
Let me know if anything isn't perfect and I'll happily tweak / fix, or if you need it in any other format I can probably convert it for you!
|
||||
|
||||
If you have a moment, I'd really appreciate your feedback (takes 2 minutes):
|
||||
{feedback_url}
|
||||
|
||||
Thanks again and have a great day!
|
||||
-Andrew
|
||||
```
|
||||
|
||||
**Note:** The `{feedback_url}` placeholder will be populated with the feedback form URL. If no feedback URL is configured, this section can be omitted from the rendered output.
|
||||
|
||||
## Updating Templates
|
||||
|
||||
To update email templates after deployment:
|
||||
|
||||
```bash
|
||||
# Upload new template
|
||||
gsutil cp my-template.txt gs://karaoke-gen-storage/templates/job-completion.txt
|
||||
|
||||
# Templates are fetched fresh on each use (no caching in code)
|
||||
# Changes take effect immediately
|
||||
```
|
||||
@@ -0,0 +1,115 @@
|
||||
# Build Encoding Worker GCE Image
|
||||
#
|
||||
# Rebuilds the custom Packer image when provisioning changes.
|
||||
# The image includes Python 3.13, FFmpeg, fonts, and dependencies
|
||||
# pre-installed to reduce VM startup time from ~10 min to ~30 sec.
|
||||
#
|
||||
# Triggers:
|
||||
# - Push to main with changes to infrastructure/packer/**
|
||||
# - Manual dispatch (workflow_dispatch)
|
||||
#
|
||||
# After image build, the encoding-worker VM will use the new image
|
||||
# on next recreation (e.g., via pulumi up or manual restart).
|
||||
|
||||
name: Build Encoding Worker Image
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'infrastructure/packer/**'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
python_version:
|
||||
description: 'Python version to install'
|
||||
required: false
|
||||
default: '3.13.1'
|
||||
|
||||
concurrency:
|
||||
group: build-encoding-worker-image
|
||||
cancel-in-progress: false # Don't cancel in-progress builds
|
||||
|
||||
jobs:
|
||||
build-image:
|
||||
name: Build Packer Image
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # Required for Workload Identity Federation
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Authenticate to Google Cloud (Workload Identity)
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Install Packer
|
||||
run: |
|
||||
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
|
||||
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
|
||||
sudo apt-get update && sudo apt-get install -y packer
|
||||
|
||||
- name: Initialize Packer
|
||||
working-directory: infrastructure/packer
|
||||
run: packer init encoding-worker.pkr.hcl
|
||||
|
||||
- name: Validate Packer Template
|
||||
working-directory: infrastructure/packer
|
||||
run: packer validate encoding-worker.pkr.hcl
|
||||
|
||||
- name: Build Image
|
||||
working-directory: infrastructure/packer
|
||||
env:
|
||||
PYTHON_VERSION: ${{ github.event.inputs.python_version || '3.13.1' }}
|
||||
run: |
|
||||
echo "Building encoding worker image with Python ${PYTHON_VERSION}..."
|
||||
packer build \
|
||||
-var "python_version=${PYTHON_VERSION}" \
|
||||
encoding-worker.pkr.hcl
|
||||
|
||||
- name: Get Latest Image Info
|
||||
id: image-info
|
||||
run: |
|
||||
IMAGE_NAME=$(gcloud compute images list \
|
||||
--project=nomadkaraoke \
|
||||
--filter="family:encoding-worker" \
|
||||
--sort-by="~creationTimestamp" \
|
||||
--limit=1 \
|
||||
--format="value(name)")
|
||||
echo "image_name=${IMAGE_NAME}" >> $GITHUB_OUTPUT
|
||||
echo "Built image: ${IMAGE_NAME}"
|
||||
|
||||
- name: Summary
|
||||
run: |
|
||||
echo "## Encoding Worker Image Built" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
echo "**Image:** \`${{ steps.image-info.outputs.image_name }}\`" >> $GITHUB_STEP_SUMMARY
|
||||
echo "**Family:** \`encoding-worker\`" >> $GITHUB_STEP_SUMMARY
|
||||
echo "**Project:** \`nomadkaraoke\`" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
echo "The encoding-worker VM will use this image on next recreation." >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
echo "To force VM recreation now:" >> $GITHUB_STEP_SUMMARY
|
||||
echo "\`\`\`bash" >> $GITHUB_STEP_SUMMARY
|
||||
echo "gcloud compute instances delete encoding-worker --zone=us-central1-a --project=nomadkaraoke" >> $GITHUB_STEP_SUMMARY
|
||||
echo "# Then run pulumi up or wait for next infrastructure deploy" >> $GITHUB_STEP_SUMMARY
|
||||
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
notify-on-failure:
|
||||
name: Notify on Failure
|
||||
runs-on: ubuntu-latest
|
||||
needs: build-image
|
||||
if: failure()
|
||||
steps:
|
||||
- name: Log Failure
|
||||
run: |
|
||||
echo "::error::Encoding worker image build failed. Check the build-image job logs."
|
||||
@@ -903,11 +903,14 @@ jobs:
|
||||
|
||||
deploy-backend:
|
||||
name: "Deploy - Backend (Cloud Run)"
|
||||
needs: [package-build]
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
outputs:
|
||||
base_rebuilt: ${{ steps.check-base.outputs.needs_rebuild }}
|
||||
env:
|
||||
REGISTRY: us-central1-docker.pkg.dev
|
||||
PROJECT_ID: nomadkaraoke
|
||||
@@ -965,6 +968,32 @@ jobs:
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
# ============================================
|
||||
# Upload wheel to GCS for GCE encoding worker
|
||||
# The GCE worker installs this wheel at job start for hot code updates
|
||||
# ============================================
|
||||
|
||||
- name: Download wheel artifact
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: package-dist
|
||||
path: dist/
|
||||
|
||||
- name: Upload wheel to GCS for GCE worker
|
||||
run: |
|
||||
echo "📦 Uploading wheel to GCS for GCE encoding worker..."
|
||||
WHEEL_FILE=$(ls dist/*.whl | head -1)
|
||||
if [ -z "$WHEEL_FILE" ]; then
|
||||
echo "::error::No wheel file found in dist/"
|
||||
exit 1
|
||||
fi
|
||||
echo "Found wheel: $WHEEL_FILE"
|
||||
|
||||
# Upload to GCS (GCE worker downloads latest at job start)
|
||||
gsutil cp "$WHEEL_FILE" gs://karaoke-gen-storage-${{ env.PROJECT_ID }}/wheels/
|
||||
|
||||
echo "✅ Wheel uploaded - GCE worker will pick up changes on next job"
|
||||
|
||||
- name: Configure Docker for Artifact Registry
|
||||
run: gcloud auth configure-docker ${{ env.REGISTRY }} --quiet
|
||||
|
||||
@@ -990,32 +1019,32 @@ jobs:
|
||||
run: |
|
||||
BASE_IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest"
|
||||
|
||||
# Try to get the existing base image's hash label
|
||||
# First check if image exists locally (fast, no network call)
|
||||
# Only pull from registry if not cached locally
|
||||
# Fetch hash label directly from registry (10KB config blob, not full image)
|
||||
# This avoids issues with stale local Docker cache on GitHub runners
|
||||
EXISTING_HASH=""
|
||||
if docker image inspect $BASE_IMAGE >/dev/null 2>&1; then
|
||||
echo "✅ Base image found in local cache (skipping slow registry pull)"
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
|
||||
# Fallback to old label name for backwards compatibility
|
||||
if [ -z "$EXISTING_HASH" ]; then
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "poetry.lock.hash"}}' 2>/dev/null || echo "")
|
||||
fi
|
||||
echo "Existing base image hash: $EXISTING_HASH"
|
||||
elif docker pull $BASE_IMAGE 2>/dev/null; then
|
||||
echo "📥 Base image pulled from registry"
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
|
||||
# Fallback to old label name for backwards compatibility
|
||||
if [ -z "$EXISTING_HASH" ]; then
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "poetry.lock.hash"}}' 2>/dev/null || echo "")
|
||||
fi
|
||||
echo "Existing base image hash: $EXISTING_HASH"
|
||||
|
||||
# Get the manifest (small JSON, ~1KB) - includes config digest
|
||||
MANIFEST=$(docker manifest inspect $BASE_IMAGE 2>/dev/null || echo "")
|
||||
CONFIG_DIGEST=$(echo "$MANIFEST" | jq -r '.config.digest // ""' 2>/dev/null || echo "")
|
||||
|
||||
if [ -n "$CONFIG_DIGEST" ] && [ "$CONFIG_DIGEST" != "null" ] && [ "$CONFIG_DIGEST" != "" ]; then
|
||||
echo "📋 Found manifest, config digest: $CONFIG_DIGEST"
|
||||
|
||||
# Fetch just the config blob (~10KB) to get labels - much faster than pulling full image
|
||||
TOKEN=$(gcloud auth print-access-token)
|
||||
CONFIG_JSON=$(curl -sL -H "Authorization: Bearer $TOKEN" \
|
||||
"https://${{ env.REGISTRY }}/v2/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}/blobs/$CONFIG_DIGEST" 2>/dev/null)
|
||||
|
||||
EXISTING_HASH=$(echo "$CONFIG_JSON" | jq -r '.config.Labels["base.content.hash"] // .config.Labels["poetry.lock.hash"] // ""' 2>/dev/null || echo "")
|
||||
echo "📋 Registry base image hash: $EXISTING_HASH"
|
||||
else
|
||||
echo "⚠️ No existing base image found - will build"
|
||||
echo "⚠️ No existing base image found in registry - will build"
|
||||
fi
|
||||
|
||||
if [ "${{ steps.base-hash.outputs.hash }}" != "$EXISTING_HASH" ]; then
|
||||
echo "🔄 Base image needs rebuild (poetry.lock or Dockerfile.base changed)"
|
||||
echo " Current: ${{ steps.base-hash.outputs.hash }}"
|
||||
echo " Registry: $EXISTING_HASH"
|
||||
echo "needs_rebuild=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "✅ Base image is up to date"
|
||||
@@ -1043,6 +1072,31 @@ jobs:
|
||||
# Provenance disabled for smaller image and faster push
|
||||
provenance: false
|
||||
|
||||
- name: Ensure local base image is current (if not rebuilt)
|
||||
if: steps.check-base.outputs.needs_rebuild == 'false'
|
||||
run: |
|
||||
BASE_IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest"
|
||||
|
||||
# Get the config digest from registry manifest (identifies the image content)
|
||||
# This is the same value as Docker's Image ID for the same image
|
||||
REGISTRY_CONFIG_DIGEST=$(docker manifest inspect $BASE_IMAGE 2>/dev/null | jq -r '.config.digest' || echo "")
|
||||
|
||||
# Get local image's ID (which is the config digest)
|
||||
LOCAL_IMAGE_ID=$(docker image inspect $BASE_IMAGE --format='{{.Id}}' 2>/dev/null || echo "")
|
||||
|
||||
echo "Registry config digest: $REGISTRY_CONFIG_DIGEST"
|
||||
echo "Local image ID: $LOCAL_IMAGE_ID"
|
||||
|
||||
if [ -z "$LOCAL_IMAGE_ID" ]; then
|
||||
echo "📥 Base image not in local cache - pulling from registry"
|
||||
docker pull $BASE_IMAGE
|
||||
elif [ "$LOCAL_IMAGE_ID" != "$REGISTRY_CONFIG_DIGEST" ]; then
|
||||
echo "🔄 Local cache has different image - pulling latest"
|
||||
docker pull $BASE_IMAGE
|
||||
else
|
||||
echo "✅ Local cache matches registry - skipping pull"
|
||||
fi
|
||||
|
||||
- name: Build and push app image
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
@@ -1085,7 +1139,7 @@ jobs:
|
||||
--concurrency 1 \
|
||||
--max-instances 50 \
|
||||
--min-instances 4 \
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=${{ env.PROJECT_ID }},GCS_BUCKET_NAME=karaoke-gen-storage-${{ env.PROJECT_ID }},FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD,USE_GCE_ENCODING=true" \
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=${{ env.PROJECT_ID }},GCS_BUCKET_NAME=karaoke-gen-storage-${{ env.PROJECT_ID }},FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD,USE_GCE_ENCODING=true,USE_GCE_PREVIEW_ENCODING=true" \
|
||||
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest,RED_API_KEY=red-api-key:latest,RED_API_URL=red-api-url:latest,OPS_API_KEY=ops-api-key:latest,OPS_API_URL=ops-api-url:latest,FLACFETCH_API_URL=flacfetch-api-url:latest,FLACFETCH_API_KEY=flacfetch-api-key:latest,SENDGRID_API_KEY=sendgrid-api-key:latest,STRIPE_SECRET_KEY=stripe-secret-key:latest,STRIPE_WEBHOOK_SECRET=stripe-webhook-secret:latest,LANGFUSE_PUBLIC_KEY=langfuse-public-key:latest,LANGFUSE_SECRET_KEY=langfuse-secret-key:latest,LANGFUSE_HOST=langfuse-host:latest,ENCODING_WORKER_URL=encoding-worker-url:latest,ENCODING_WORKER_API_KEY=encoding-worker-api-key:latest"
|
||||
|
||||
- name: Verify deployment and version
|
||||
@@ -1139,3 +1193,125 @@ jobs:
|
||||
echo ""
|
||||
echo "📊 Built on self-hosted GCP runner (200GB SSD, no disk space issues)"
|
||||
echo " Runner: gcp-runner with 4 vCPU, 16GB RAM"
|
||||
|
||||
# ============================================
|
||||
# PROPAGATE BASE IMAGE TO ALL RUNNERS
|
||||
# When base image is rebuilt, pull it to all 20 runners
|
||||
# so subsequent app builds don't wait 8 min for the pull
|
||||
#
|
||||
# NOTE: This is best-effort distribution. GitHub Actions doesn't
|
||||
# support targeting specific runners by name, so we spawn 20 jobs
|
||||
# and rely on the scheduler to distribute them across idle runners.
|
||||
# In practice, with all runners idle after deploy-backend, distribution
|
||||
# is good. Runners that miss this pull will get the image on their
|
||||
# next app build (with the 8 min delay).
|
||||
# ============================================
|
||||
|
||||
propagate-base-image:
|
||||
name: "Propagate Base Image (${{ matrix.runner_index }})"
|
||||
needs: deploy-backend
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push' && needs.deploy-backend.outputs.base_rebuilt == 'true'
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
strategy:
|
||||
matrix:
|
||||
# 20 parallel jobs - GitHub distributes across available runners
|
||||
runner_index: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]
|
||||
max-parallel: 20
|
||||
fail-fast: false # Don't cancel other runners if one fails
|
||||
env:
|
||||
REGISTRY: us-central1-docker.pkg.dev
|
||||
PROJECT_ID: nomadkaraoke
|
||||
REPOSITORY: karaoke-repo
|
||||
BASE_IMAGE_NAME: karaoke-backend-base
|
||||
steps:
|
||||
- name: Authenticate to Google Cloud (Workload Identity)
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Configure Docker for Artifact Registry
|
||||
run: gcloud auth configure-docker ${{ env.REGISTRY }} --quiet
|
||||
|
||||
- name: Pull latest base image
|
||||
run: |
|
||||
BASE_IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest"
|
||||
RUNNER_NAME=$(hostname)
|
||||
echo "📥 Pulling latest base image on runner $RUNNER_NAME..."
|
||||
docker pull $BASE_IMAGE
|
||||
echo "✅ Base image cached on $RUNNER_NAME"
|
||||
|
||||
deploy-infrastructure:
|
||||
name: "Deploy - Infrastructure (Pulumi)"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Authenticate to Google Cloud (Workload Identity)
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.13'
|
||||
|
||||
- name: Install Pulumi
|
||||
run: |
|
||||
curl -fsSL https://get.pulumi.com | sh
|
||||
echo "$HOME/.pulumi/bin" >> $GITHUB_PATH
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: infrastructure
|
||||
run: |
|
||||
pip install pulumi pulumi-gcp pulumi-docker
|
||||
|
||||
- name: Configure Pulumi
|
||||
env:
|
||||
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
|
||||
run: |
|
||||
pulumi login
|
||||
|
||||
- name: Deploy Infrastructure
|
||||
working-directory: infrastructure
|
||||
env:
|
||||
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
|
||||
run: |
|
||||
echo "🚀 Deploying infrastructure changes..."
|
||||
# Skip preview to avoid needing broad read permissions on all GCP resources
|
||||
# Pulumi state is authoritative since infrastructure is only managed via Pulumi
|
||||
pulumi up --yes --skip-preview --stack nomadkaraoke/karaoke-gen-infrastructure/prod
|
||||
|
||||
- name: Restart GCE Encoding Worker
|
||||
run: |
|
||||
echo "🔄 Restarting encoding worker to pick up new code..."
|
||||
gcloud compute ssh encoding-worker --zone=us-central1-a --project=nomadkaraoke \
|
||||
--command="sudo systemctl restart encoding-worker" || true
|
||||
echo "✅ Encoding worker restart triggered"
|
||||
|
||||
- name: Summary
|
||||
if: success()
|
||||
run: |
|
||||
echo "🎉 Infrastructure deployment successful!"
|
||||
echo "Commit: ${{ github.sha }}"
|
||||
echo "Branch: ${{ github.ref_name }}"
|
||||
@@ -18,6 +18,8 @@
|
||||
| Dev setup & testing | `docs/DEVELOPMENT.md` |
|
||||
| API reference | `docs/API.md` |
|
||||
| Past learnings | `docs/LESSONS-LEARNED.md` |
|
||||
| **Product communication** | `docs/PRODUCT-COMMUNICATION-GUIDE.md` |
|
||||
| Brand style guide | `docs/BRAND-STYLE-GUIDE.md` |
|
||||
|
||||
## Tech Stack
|
||||
|
||||
@@ -48,8 +50,10 @@ cd frontend && npm run test:all 2>&1 | tail -n 200
|
||||
- Skip for docs-only changes
|
||||
|
||||
### Infrastructure
|
||||
- All GCP changes via Pulumi in `infrastructure/`
|
||||
- `gcloud` CLI for reading/debugging only
|
||||
- **All GCP changes via Pulumi PRs** - changes in `infrastructure/` deploy automatically on merge to main
|
||||
- Never modify GCP resources directly via console or `gcloud` CLI
|
||||
- `gcloud` CLI for reading/debugging only (e.g., checking logs, SSH to VMs)
|
||||
- Local `pulumi preview` works but `pulumi up` requires broader permissions than CI
|
||||
- Stop and notify user on auth issues
|
||||
|
||||
## API Authentication (for Agents)
|
||||
|
||||
@@ -20,6 +20,7 @@ from backend.services.user_service import get_user_service, UserService, USERS_C
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.flacfetch_client import get_flacfetch_client, FlacfetchServiceError
|
||||
from backend.models.job import JobStatus
|
||||
from karaoke_gen.utils import sanitize_filename
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -579,4 +580,163 @@ async def get_flacfetch_cache_stats(
|
||||
status_code=502,
|
||||
detail=f"Failed to get cache stats: {e}"
|
||||
)
|
||||
# Trigger deployment
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Job Completion Message Endpoints (for admin copy/send functionality)
|
||||
# =============================================================================
|
||||
|
||||
class CompletionMessageResponse(BaseModel):
|
||||
"""Response containing the rendered completion message."""
|
||||
job_id: str
|
||||
message: str
|
||||
subject: str
|
||||
youtube_url: Optional[str] = None
|
||||
dropbox_url: Optional[str] = None
|
||||
|
||||
|
||||
class SendCompletionEmailRequest(BaseModel):
|
||||
"""Request to send a completion email."""
|
||||
to_email: str
|
||||
cc_admin: bool = True
|
||||
|
||||
|
||||
class SendCompletionEmailResponse(BaseModel):
|
||||
"""Response from sending a completion email."""
|
||||
success: bool
|
||||
job_id: str
|
||||
to_email: str
|
||||
message: str
|
||||
|
||||
|
||||
@router.get("/jobs/{job_id}/completion-message", response_model=CompletionMessageResponse)
|
||||
async def get_job_completion_message(
|
||||
job_id: str,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
):
|
||||
"""
|
||||
Get the rendered completion message for a job.
|
||||
|
||||
Returns the plain text message that would be sent to the user,
|
||||
rendered using the job completion template with the job's details.
|
||||
|
||||
This is useful for:
|
||||
- Copying the message to clipboard (e.g., for Fiverr)
|
||||
- Previewing the email before sending
|
||||
|
||||
Requires admin authentication.
|
||||
"""
|
||||
from backend.services.job_notification_service import get_job_notification_service
|
||||
|
||||
job_manager = JobManager()
|
||||
job = job_manager.get_job(job_id)
|
||||
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
# Get youtube, dropbox URLs, and brand_code from state_data (may be None)
|
||||
state_data = job.state_data or {}
|
||||
youtube_url = state_data.get('youtube_url')
|
||||
dropbox_url = state_data.get('dropbox_link')
|
||||
brand_code = state_data.get('brand_code')
|
||||
|
||||
# Render the completion message
|
||||
notification_service = get_job_notification_service()
|
||||
message = notification_service.get_completion_message(
|
||||
job_id=job.job_id,
|
||||
user_name=None, # Use default "there"
|
||||
artist=job.artist,
|
||||
title=job.title,
|
||||
youtube_url=youtube_url,
|
||||
dropbox_url=dropbox_url,
|
||||
)
|
||||
|
||||
# Build subject: "NOMAD-1178: Artist - Title (Your karaoke video is ready!)"
|
||||
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
# that cause email header encoding issues (MIME headers use latin-1)
|
||||
safe_artist = sanitize_filename(job.artist) if job.artist else None
|
||||
safe_title = sanitize_filename(job.title) if job.title else None
|
||||
if brand_code and safe_artist and safe_title:
|
||||
subject = f"{brand_code}: {safe_artist} - {safe_title} (Your karaoke video is ready!)"
|
||||
elif safe_artist and safe_title:
|
||||
subject = f"{safe_artist} - {safe_title} (Your karaoke video is ready!)"
|
||||
else:
|
||||
subject = "Your karaoke video is ready!"
|
||||
|
||||
return CompletionMessageResponse(
|
||||
job_id=job_id,
|
||||
message=message,
|
||||
subject=subject,
|
||||
youtube_url=youtube_url,
|
||||
dropbox_url=dropbox_url,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/jobs/{job_id}/send-completion-email", response_model=SendCompletionEmailResponse)
|
||||
async def send_job_completion_email(
|
||||
job_id: str,
|
||||
request: SendCompletionEmailRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
):
|
||||
"""
|
||||
Send a completion email for a job to a specified email address.
|
||||
|
||||
This allows admins to manually send (or re-send) completion emails,
|
||||
useful for:
|
||||
- Sending to customers who didn't have an email on file
|
||||
- Re-sending if the original email was lost
|
||||
- Sending to alternate email addresses
|
||||
|
||||
Requires admin authentication.
|
||||
"""
|
||||
from backend.services.job_notification_service import get_job_notification_service
|
||||
from backend.services.email_service import get_email_service
|
||||
|
||||
job_manager = JobManager()
|
||||
job = job_manager.get_job(job_id)
|
||||
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
# Get youtube, dropbox URLs, and brand_code from state_data (may be None)
|
||||
state_data = job.state_data or {}
|
||||
youtube_url = state_data.get('youtube_url')
|
||||
dropbox_url = state_data.get('dropbox_link')
|
||||
brand_code = state_data.get('brand_code')
|
||||
|
||||
# Render the completion message
|
||||
notification_service = get_job_notification_service()
|
||||
message = notification_service.get_completion_message(
|
||||
job_id=job.job_id,
|
||||
user_name=None, # Use default "there"
|
||||
artist=job.artist,
|
||||
title=job.title,
|
||||
youtube_url=youtube_url,
|
||||
dropbox_url=dropbox_url,
|
||||
)
|
||||
|
||||
# Send the email
|
||||
email_service = get_email_service()
|
||||
success = email_service.send_job_completion(
|
||||
to_email=request.to_email,
|
||||
message_content=message,
|
||||
artist=job.artist,
|
||||
title=job.title,
|
||||
brand_code=brand_code,
|
||||
cc_admin=request.cc_admin,
|
||||
)
|
||||
|
||||
if success:
|
||||
logger.info(f"Admin sent completion email for job {job_id} to {request.to_email}")
|
||||
return SendCompletionEmailResponse(
|
||||
success=True,
|
||||
job_id=job_id,
|
||||
to_email=request.to_email,
|
||||
message=f"Completion email sent to {request.to_email}",
|
||||
)
|
||||
else:
|
||||
logger.error(f"Failed to send completion email for job {job_id}")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail="Failed to send email. Check email service configuration."
|
||||
)
|
||||
@@ -18,9 +18,10 @@ import tempfile
|
||||
from typing import Optional, List, Dict, Any, Tuple
|
||||
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Request, Depends
|
||||
from pydantic import BaseModel, Field
|
||||
from pydantic import BaseModel, Field, validator
|
||||
|
||||
from backend.models.job import JobCreate, JobStatus
|
||||
from karaoke_gen.utils import normalize_text
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.services.worker_service import get_worker_service
|
||||
@@ -95,7 +96,7 @@ class AudioSearchRequest(BaseModel):
|
||||
|
||||
# Finalisation options
|
||||
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
|
||||
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
|
||||
enable_youtube_upload: Optional[bool] = Field(None, description="Upload to YouTube. None = use server default")
|
||||
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
|
||||
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
|
||||
|
||||
@@ -107,6 +108,10 @@ class AudioSearchRequest(BaseModel):
|
||||
lyrics_artist: Optional[str] = Field(None, description="Override artist name for lyrics search")
|
||||
lyrics_title: Optional[str] = Field(None, description="Override title for lyrics search")
|
||||
subtitle_offset_ms: int = Field(0, description="Subtitle timing offset in milliseconds")
|
||||
|
||||
# Display overrides (optional - if empty, search values are used for display)
|
||||
display_artist: Optional[str] = Field(None, description="Artist name for title screens/filenames. If empty, uses search artist.")
|
||||
display_title: Optional[str] = Field(None, description="Title for title screens/filenames. If empty, uses search title.")
|
||||
|
||||
# Audio separation model configuration
|
||||
clean_instrumental_model: Optional[str] = Field(None, description="Model for clean instrumental separation")
|
||||
@@ -120,6 +125,13 @@ class AudioSearchRequest(BaseModel):
|
||||
# Non-interactive mode
|
||||
non_interactive: bool = Field(False, description="Skip interactive steps (lyrics review, instrumental selection)")
|
||||
|
||||
@validator('artist', 'title', 'lyrics_artist', 'lyrics_title', 'display_artist', 'display_title')
|
||||
def normalize_text_fields(cls, v):
|
||||
"""Normalize text fields to standardize Unicode characters."""
|
||||
if v is not None and isinstance(v, str):
|
||||
return normalize_text(v)
|
||||
return v
|
||||
|
||||
|
||||
class AudioSearchResultResponse(BaseModel):
|
||||
"""A single audio search result.
|
||||
@@ -485,7 +497,8 @@ async def search_audio(
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
# Apply defaults for YouTube/Dropbox distribution (for web service)
|
||||
effective_enable_youtube_upload = body.enable_youtube_upload or settings.default_enable_youtube_upload
|
||||
# Use explicit value if provided, otherwise fall back to server default
|
||||
effective_enable_youtube_upload = body.enable_youtube_upload if body.enable_youtube_upload is not None else settings.default_enable_youtube_upload
|
||||
effective_brand_prefix = body.brand_prefix or settings.default_brand_prefix
|
||||
effective_youtube_description = body.youtube_description or settings.default_youtube_description
|
||||
|
||||
@@ -529,10 +542,16 @@ async def search_audio(
|
||||
# Use authenticated user's email
|
||||
effective_user_email = auth_result.user_email
|
||||
|
||||
# Determine display values - use display_* if provided, otherwise fall back to search values
|
||||
# Display values are used for title screens, filenames, YouTube, etc.
|
||||
# Search values (body.artist, body.title) are used for audio search
|
||||
effective_display_artist = body.display_artist.strip() if body.display_artist else body.artist
|
||||
effective_display_title = body.display_title.strip() if body.display_title else body.title
|
||||
|
||||
# Create job
|
||||
job_create = JobCreate(
|
||||
artist=body.artist,
|
||||
title=body.title,
|
||||
artist=effective_display_artist, # Display value for title screens, filenames
|
||||
title=effective_display_title, # Display value for title screens, filenames
|
||||
theme_id=body.theme_id,
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
|
||||
@@ -69,7 +69,7 @@ class CreateJobFromUrlRequest(BaseModel):
|
||||
|
||||
# Finalisation options
|
||||
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
|
||||
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
|
||||
enable_youtube_upload: Optional[bool] = Field(None, description="Upload to YouTube. None = use server default")
|
||||
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
|
||||
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
|
||||
webhook_url: Optional[str] = Field(None, description="Generic webhook URL")
|
||||
@@ -123,7 +123,7 @@ class CreateJobWithUploadUrlsRequest(BaseModel):
|
||||
|
||||
# Finalisation options
|
||||
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
|
||||
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
|
||||
enable_youtube_upload: Optional[bool] = Field(None, description="Upload to YouTube. None = use server default")
|
||||
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
|
||||
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
|
||||
webhook_url: Optional[str] = Field(None, description="Generic webhook URL")
|
||||
@@ -406,7 +406,7 @@ async def upload_and_create_job(
|
||||
enable_txt: Optional[bool] = Form(None, description="Generate TXT+MP3 package. Default: True if theme_id set, False otherwise"),
|
||||
# Finalisation options
|
||||
brand_prefix: Optional[str] = Form(None, description="Brand code prefix (e.g., NOMAD)"),
|
||||
enable_youtube_upload: bool = Form(False, description="Upload to YouTube"),
|
||||
enable_youtube_upload: Optional[bool] = Form(None, description="Upload to YouTube. None = use server default"),
|
||||
youtube_description: Optional[str] = Form(None, description="YouTube video description text"),
|
||||
discord_webhook_url: Optional[str] = Form(None, description="Discord webhook URL for notifications"),
|
||||
webhook_url: Optional[str] = Form(None, description="Generic webhook URL"),
|
||||
@@ -510,12 +510,19 @@ async def upload_and_create_job(
|
||||
if dist.brand_prefix and not brand_prefix:
|
||||
logger.info(f"Using default brand_prefix: {dist.brand_prefix}")
|
||||
|
||||
# Apply YouTube upload default from settings
|
||||
# Use explicit value if provided, otherwise fall back to server default
|
||||
settings = get_settings()
|
||||
effective_enable_youtube_upload = enable_youtube_upload if enable_youtube_upload is not None else settings.default_enable_youtube_upload
|
||||
if effective_enable_youtube_upload and enable_youtube_upload is None:
|
||||
logger.info("Using default enable_youtube_upload: True (from env)")
|
||||
|
||||
# Validate credentials for requested distribution services (including defaults)
|
||||
# This prevents accepting jobs that will fail later due to missing credentials
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
if enable_youtube_upload:
|
||||
if effective_enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
@@ -594,7 +601,7 @@ async def upload_and_create_job(
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=enable_youtube_upload,
|
||||
enable_youtube_upload=effective_enable_youtube_upload,
|
||||
youtube_description=youtube_description,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
webhook_url=webhook_url,
|
||||
@@ -815,11 +822,12 @@ async def upload_and_create_job(
|
||||
"using_default": gdrive_folder_id is None,
|
||||
}
|
||||
|
||||
if enable_youtube_upload:
|
||||
if effective_enable_youtube_upload:
|
||||
youtube_result = credential_manager.check_youtube_credentials()
|
||||
distribution_services["youtube"] = {
|
||||
"enabled": True,
|
||||
"credentials_valid": youtube_result.status == CredentialStatus.VALID,
|
||||
"using_default": enable_youtube_upload is None,
|
||||
}
|
||||
|
||||
if dist.discord_webhook_url:
|
||||
@@ -908,7 +916,7 @@ class CreateFinaliseOnlyJobRequest(BaseModel):
|
||||
# Finalisation options
|
||||
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
|
||||
keep_brand_code: Optional[str] = Field(None, description="Preserve existing brand code from folder name")
|
||||
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
|
||||
enable_youtube_upload: Optional[bool] = Field(None, description="Upload to YouTube. None = use server default")
|
||||
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
|
||||
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
|
||||
|
||||
@@ -1048,11 +1056,16 @@ async def create_job_with_upload_urls(
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
# Apply YouTube upload default from settings
|
||||
# Use explicit value if provided, otherwise fall back to server default
|
||||
settings = get_settings()
|
||||
effective_enable_youtube_upload = body.enable_youtube_upload if body.enable_youtube_upload is not None else settings.default_enable_youtube_upload
|
||||
|
||||
# Validate credentials for requested distribution services
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
if effective_enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
@@ -1105,7 +1118,7 @@ async def create_job_with_upload_urls(
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
enable_youtube_upload=effective_enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
youtube_description_template=body.youtube_description, # video_worker reads this field
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
@@ -1466,11 +1479,16 @@ async def create_job_from_url(
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
# Apply YouTube upload default from settings
|
||||
# Use explicit value if provided, otherwise fall back to server default
|
||||
settings = get_settings()
|
||||
effective_enable_youtube_upload = body.enable_youtube_upload if body.enable_youtube_upload is not None else settings.default_enable_youtube_upload
|
||||
|
||||
# Validate credentials for requested distribution services
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
if effective_enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
@@ -1522,7 +1540,7 @@ async def create_job_from_url(
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
enable_youtube_upload=effective_enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
webhook_url=body.webhook_url,
|
||||
@@ -1712,11 +1730,16 @@ async def create_finalise_only_job(
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
# Apply YouTube upload default from settings
|
||||
# Use explicit value if provided, otherwise fall back to server default
|
||||
settings = get_settings()
|
||||
effective_enable_youtube_upload = body.enable_youtube_upload if body.enable_youtube_upload is not None else settings.default_enable_youtube_upload
|
||||
|
||||
# Validate distribution credentials if services are requested
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
if effective_enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
@@ -1766,7 +1789,7 @@ async def create_finalise_only_job(
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
enable_youtube_upload=effective_enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
|
||||
@@ -227,6 +227,7 @@ async def check_encoding_worker_status() -> Dict[str, Any]:
|
||||
"active_jobs": health.get("active_jobs", 0),
|
||||
"queue_length": health.get("queue_length", 0),
|
||||
"ffmpeg_version": health.get("ffmpeg_version"),
|
||||
"wheel_version": health.get("wheel_version"),
|
||||
}
|
||||
except Exception as e:
|
||||
return {
|
||||
@@ -237,6 +238,39 @@ async def check_encoding_worker_status() -> Dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
@router.get("/health/encoding-worker")
|
||||
async def encoding_worker_health() -> Dict[str, Any]:
|
||||
"""
|
||||
Lightweight endpoint to check encoding worker status.
|
||||
|
||||
Returns minimal info for frontend footer display.
|
||||
No authentication required.
|
||||
"""
|
||||
status = await check_encoding_worker_status()
|
||||
|
||||
# Return simplified response for frontend
|
||||
if not status.get("configured"):
|
||||
return {
|
||||
"available": False,
|
||||
"status": "not_configured",
|
||||
}
|
||||
|
||||
if not status.get("available"):
|
||||
return {
|
||||
"available": False,
|
||||
"status": "offline",
|
||||
"error": status.get("error"),
|
||||
}
|
||||
|
||||
return {
|
||||
"available": True,
|
||||
"status": "ok",
|
||||
"version": status.get("wheel_version"),
|
||||
"active_jobs": status.get("active_jobs", 0),
|
||||
"queue_length": status.get("queue_length", 0),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/health/detailed")
|
||||
async def detailed_health_check() -> Dict[str, Any]:
|
||||
"""
|
||||
|
||||
@@ -316,13 +316,118 @@ async def trigger_render_video_worker(
|
||||
)
|
||||
|
||||
|
||||
@router.post("/jobs/{job_id}/check-idle-reminder")
|
||||
async def check_idle_reminder(
|
||||
job_id: str,
|
||||
http_request: Request,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
|
||||
):
|
||||
"""
|
||||
Check if a job needs an idle reminder email.
|
||||
|
||||
This endpoint is called by a Cloud Tasks scheduled task 5 minutes after
|
||||
a job enters a blocking state (AWAITING_REVIEW or AWAITING_INSTRUMENTAL_SELECTION).
|
||||
|
||||
If the job is still in the blocking state and no reminder has been sent yet,
|
||||
sends a reminder email to the user.
|
||||
|
||||
Idempotency: Only one reminder per job (tracked via reminder_sent flag).
|
||||
"""
|
||||
from backend.models.job import JobStatus
|
||||
from backend.services.job_notification_service import get_job_notification_service
|
||||
|
||||
# Extract trace context from incoming request
|
||||
trace_context = extract_trace_context(dict(http_request.headers))
|
||||
|
||||
logger.info(f"[job:{job_id}] IDLE_REMINDER_CHECK starting")
|
||||
add_span_attribute("job_id", job_id)
|
||||
add_span_attribute("operation", "idle_reminder_check")
|
||||
|
||||
job_manager = JobManager()
|
||||
job = job_manager.get_job(job_id)
|
||||
|
||||
if not job:
|
||||
logger.warning(f"[job:{job_id}] Job not found for idle reminder check")
|
||||
add_span_event("job_not_found")
|
||||
return {"status": "not_found", "job_id": job_id, "message": "Job not found"}
|
||||
|
||||
# Check if job is still in a blocking state
|
||||
blocking_states = [JobStatus.AWAITING_REVIEW, JobStatus.AWAITING_INSTRUMENTAL_SELECTION]
|
||||
if job.status not in [s.value for s in blocking_states]:
|
||||
logger.info(f"[job:{job_id}] Job no longer in blocking state ({job.status}), skipping reminder")
|
||||
add_span_event("not_blocking", {"current_status": job.status})
|
||||
return {
|
||||
"status": "skipped",
|
||||
"job_id": job_id,
|
||||
"message": f"Job not in blocking state (current: {job.status})"
|
||||
}
|
||||
|
||||
# Normalize state_data to prevent None errors
|
||||
state_data = job.state_data or {}
|
||||
|
||||
# Check if reminder was already sent (idempotency)
|
||||
if state_data.get('reminder_sent'):
|
||||
logger.info(f"[job:{job_id}] Reminder already sent, skipping")
|
||||
add_span_event("already_sent")
|
||||
return {"status": "already_sent", "job_id": job_id, "message": "Reminder already sent"}
|
||||
|
||||
# Check if user has an email
|
||||
if not job.user_email:
|
||||
logger.warning(f"[job:{job_id}] No user email, cannot send reminder")
|
||||
add_span_event("no_email")
|
||||
return {"status": "no_email", "job_id": job_id, "message": "No user email configured"}
|
||||
|
||||
# Determine action type
|
||||
action_type = state_data.get('blocking_action_type')
|
||||
if not action_type:
|
||||
action_type = "lyrics" if job.status == JobStatus.AWAITING_REVIEW.value else "instrumental"
|
||||
|
||||
# Send the reminder email
|
||||
try:
|
||||
notification_service = get_job_notification_service()
|
||||
|
||||
success = await notification_service.send_action_reminder_email(
|
||||
job_id=job.job_id,
|
||||
user_email=job.user_email,
|
||||
action_type=action_type,
|
||||
user_name=None, # Could fetch from user service if needed
|
||||
artist=job.artist,
|
||||
title=job.title,
|
||||
audio_hash=job.audio_hash,
|
||||
review_token=job.review_token,
|
||||
instrumental_token=job.instrumental_token,
|
||||
)
|
||||
|
||||
if success:
|
||||
# Mark reminder as sent (prevents duplicate sends)
|
||||
job_manager.firestore.update_job(job_id, {
|
||||
'state_data': {**state_data, 'reminder_sent': True}
|
||||
})
|
||||
logger.info(f"[job:{job_id}] Sent {action_type} reminder email to {job.user_email}")
|
||||
add_span_event("reminder_sent", {"action_type": action_type})
|
||||
return {
|
||||
"status": "sent",
|
||||
"job_id": job_id,
|
||||
"message": f"Sent {action_type} reminder to {job.user_email}"
|
||||
}
|
||||
else:
|
||||
logger.error(f"[job:{job_id}] Failed to send reminder email")
|
||||
add_span_event("send_failed")
|
||||
return {"status": "failed", "job_id": job_id, "message": "Failed to send reminder"}
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"[job:{job_id}] Error sending reminder: {e}")
|
||||
add_span_event("error", {"error": str(e)})
|
||||
return {"status": "error", "job_id": job_id, "message": str(e)}
|
||||
|
||||
|
||||
@router.get("/health")
|
||||
async def internal_health(
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
|
||||
):
|
||||
"""
|
||||
Internal health check endpoint.
|
||||
|
||||
|
||||
Used to verify the internal API is responsive.
|
||||
Requires admin authentication.
|
||||
"""
|
||||
|
||||
@@ -87,6 +87,11 @@ async def create_job(
|
||||
user_email = request.user_email
|
||||
logger.info(f"Admin {auth_result.user_email} creating job on behalf of {user_email}")
|
||||
|
||||
# Apply YouTube upload default from settings
|
||||
# Use explicit value if provided, otherwise fall back to server default
|
||||
settings = get_settings()
|
||||
effective_enable_youtube_upload = request.enable_youtube_upload if request.enable_youtube_upload is not None else settings.default_enable_youtube_upload
|
||||
|
||||
# Create job with all preferences
|
||||
job_create = JobCreate(
|
||||
url=str(request.url),
|
||||
@@ -94,7 +99,7 @@ async def create_job(
|
||||
title=request.title,
|
||||
enable_cdg=request.enable_cdg,
|
||||
enable_txt=request.enable_txt,
|
||||
enable_youtube_upload=request.enable_youtube_upload,
|
||||
enable_youtube_upload=effective_enable_youtube_upload,
|
||||
youtube_description=request.youtube_description,
|
||||
webhook_url=request.webhook_url,
|
||||
user_email=user_email
|
||||
@@ -1159,21 +1164,11 @@ async def download_file(
|
||||
content_type = content_types.get(ext, 'application/octet-stream')
|
||||
|
||||
# Build proper filename: "Artist - Title (Final Karaoke Lossy 4k).mp4"
|
||||
# Sanitize artist/title to remove filesystem-unsafe characters
|
||||
def sanitize_filename_part(s: str) -> str:
|
||||
if not s:
|
||||
return ""
|
||||
# Remove/replace characters unsafe for filesystems: / \ : * ? " < > |
|
||||
unsafe_chars = r'[/\\:*?"<>|]'
|
||||
import re
|
||||
sanitized = re.sub(unsafe_chars, '', s)
|
||||
# Collapse multiple spaces and trim
|
||||
sanitized = ' '.join(sanitized.split())
|
||||
# Limit length to prevent overly long filenames
|
||||
return sanitized[:100].strip()
|
||||
|
||||
artist_clean = sanitize_filename_part(job.artist) if job.artist else None
|
||||
title_clean = sanitize_filename_part(job.title) if job.title else None
|
||||
# Use sanitize_filename to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
# that cause HTTP header encoding issues (Content-Disposition uses latin-1)
|
||||
from karaoke_gen.utils import sanitize_filename
|
||||
artist_clean = sanitize_filename(job.artist) if job.artist else None
|
||||
title_clean = sanitize_filename(job.title) if job.title else None
|
||||
base_name = f"{artist_clean} - {title_clean}" if artist_clean and title_clean else None
|
||||
|
||||
if base_name and file_key in DOWNLOAD_FILENAME_SUFFIXES:
|
||||
|
||||
@@ -27,8 +27,10 @@ from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.services.job_logging import job_log_context, JobLogger
|
||||
from backend.services.tracing import create_span, add_span_attribute, add_span_event
|
||||
from backend.services.encoding_service import get_encoding_service
|
||||
from backend.api.dependencies import require_auth, require_review_auth
|
||||
from backend.services.auth_service import UserType
|
||||
from backend.config import get_settings
|
||||
|
||||
# LyricsTranscriber imports for preview generation
|
||||
from lyrics_transcriber.types import CorrectionResult
|
||||
@@ -385,28 +387,35 @@ async def generate_preview_video(
|
||||
):
|
||||
"""
|
||||
Generate a preview video with the current corrections.
|
||||
|
||||
|
||||
Uses the LyricsTranscriber's CorrectionOperations to generate a 360p preview
|
||||
video with the user's current corrections applied.
|
||||
|
||||
When USE_GCE_PREVIEW_ENCODING is enabled, video encoding is offloaded to
|
||||
the high-performance GCE worker for faster generation (15-20s vs 60+s).
|
||||
"""
|
||||
job_manager = JobManager()
|
||||
storage = StorageService()
|
||||
|
||||
settings = get_settings()
|
||||
encoding_service = get_encoding_service()
|
||||
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
|
||||
# Job must be in review state to generate preview
|
||||
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Job not in review state (current status: {job.status})"
|
||||
)
|
||||
|
||||
logger.info(f"Job {job_id}: Generating preview video")
|
||||
|
||||
|
||||
# Check if GCE preview encoding is enabled
|
||||
use_gce_preview = encoding_service.is_preview_enabled
|
||||
logger.info(f"Job {job_id}: Generating preview video (GCE preview: {use_gce_preview})")
|
||||
|
||||
# Use tracing and job_log_context for full observability
|
||||
with create_span("generate-preview-video", {"job_id": job_id}) as span:
|
||||
with create_span("generate-preview-video", {"job_id": job_id, "use_gce": use_gce_preview}) as span:
|
||||
with job_log_context(job_id, worker="preview"):
|
||||
try:
|
||||
# Create temp directory for this preview operation
|
||||
@@ -416,19 +425,19 @@ async def generate_preview_video(
|
||||
corrections_gcs = f"jobs/{job_id}/lyrics/corrections.json"
|
||||
corrections_path = os.path.join(temp_dir, "corrections.json")
|
||||
storage.download_file(corrections_gcs, corrections_path)
|
||||
|
||||
|
||||
with open(corrections_path, 'r', encoding='utf-8') as f:
|
||||
original_data = json.load(f)
|
||||
|
||||
|
||||
# 2. Download input audio
|
||||
audio_path = os.path.join(temp_dir, "audio.flac")
|
||||
storage.download_file(job.input_media_gcs_path, audio_path)
|
||||
download_span.set_attribute("audio_gcs_path", job.input_media_gcs_path)
|
||||
|
||||
|
||||
# 3. Load original as CorrectionResult
|
||||
correction_result = CorrectionResult.from_dict(original_data)
|
||||
add_span_event("corrections_loaded")
|
||||
|
||||
|
||||
# 4. Get or create styles file for preview using unified style loader
|
||||
with create_span("load-styles") as styles_span:
|
||||
styles_path, _ = load_styles_from_gcs(
|
||||
@@ -439,54 +448,127 @@ async def generate_preview_video(
|
||||
logger=logger,
|
||||
)
|
||||
styles_span.set_attribute("styles_path", styles_path)
|
||||
|
||||
|
||||
# 5. Set up output config for preview
|
||||
output_dir = os.path.join(temp_dir, "output")
|
||||
cache_dir = os.path.join(temp_dir, "cache")
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
os.makedirs(cache_dir, exist_ok=True)
|
||||
|
||||
|
||||
output_config = OutputConfig(
|
||||
output_styles_json=styles_path,
|
||||
output_dir=output_dir,
|
||||
cache_dir=cache_dir,
|
||||
video_resolution="360p",
|
||||
)
|
||||
|
||||
# 6. Generate preview video using CorrectionOperations (heavy operation)
|
||||
with create_span("render-preview-video") as render_span:
|
||||
render_span.set_attribute("resolution", "360p")
|
||||
result = CorrectionOperations.generate_preview_video(
|
||||
correction_result=correction_result,
|
||||
updated_data=updated_data,
|
||||
output_config=output_config,
|
||||
audio_filepath=audio_path,
|
||||
artist=job.artist,
|
||||
title=job.title,
|
||||
logger=logger
|
||||
)
|
||||
add_span_event("render_complete")
|
||||
|
||||
# 7. Upload preview video to GCS for serving
|
||||
preview_hash = result["preview_hash"]
|
||||
video_path = result["video_path"]
|
||||
|
||||
with create_span("upload-preview-video") as upload_span:
|
||||
preview_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.mp4"
|
||||
storage.upload_file(video_path, preview_gcs_path)
|
||||
upload_span.set_attribute("gcs_path", preview_gcs_path)
|
||||
|
||||
|
||||
# 6. Generate preview (ASS-only if using GCE, or full video if local)
|
||||
preview_gcs_path = None
|
||||
|
||||
if use_gce_preview:
|
||||
# GCE path: Generate ASS only, then offload encoding to GCE
|
||||
try:
|
||||
with create_span("generate-ass-subtitles") as ass_span:
|
||||
result = CorrectionOperations.generate_preview_video(
|
||||
correction_result=correction_result,
|
||||
updated_data=updated_data,
|
||||
output_config=output_config,
|
||||
audio_filepath=audio_path,
|
||||
artist=job.artist,
|
||||
title=job.title,
|
||||
logger=logger,
|
||||
ass_only=True, # Only generate ASS, skip video encoding
|
||||
)
|
||||
preview_hash = result["preview_hash"]
|
||||
ass_path = result["ass_path"]
|
||||
ass_span.set_attribute("ass_path", ass_path)
|
||||
add_span_event("ass_generated")
|
||||
|
||||
# Upload ASS to GCS
|
||||
with create_span("upload-ass-to-gcs") as upload_ass_span:
|
||||
ass_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.ass"
|
||||
storage.upload_file(ass_path, ass_gcs_path)
|
||||
upload_ass_span.set_attribute("ass_gcs_path", ass_gcs_path)
|
||||
|
||||
# Call GCE encoding service
|
||||
with create_span("gce-preview-encoding") as gce_span:
|
||||
bucket_name = settings.gcs_bucket_name
|
||||
preview_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.mp4"
|
||||
|
||||
# Get background image and font from style assets if available
|
||||
style_assets = job.style_assets or {}
|
||||
|
||||
background_image_gcs_path = None
|
||||
for key in ["karaoke_background", "style_karaoke_background"]:
|
||||
if key in style_assets:
|
||||
background_image_gcs_path = f"gs://{bucket_name}/{style_assets[key]}"
|
||||
gce_span.set_attribute("background_image", background_image_gcs_path)
|
||||
break
|
||||
|
||||
font_gcs_path = None
|
||||
for key in ["font", "style_font"]:
|
||||
if key in style_assets:
|
||||
font_gcs_path = f"gs://{bucket_name}/{style_assets[key]}"
|
||||
gce_span.set_attribute("font", font_gcs_path)
|
||||
break
|
||||
|
||||
gce_result = await encoding_service.encode_preview_video(
|
||||
job_id=f"preview_{job_id}_{preview_hash}",
|
||||
ass_gcs_path=f"gs://{bucket_name}/{ass_gcs_path}",
|
||||
audio_gcs_path=f"gs://{bucket_name}/{job.input_media_gcs_path}",
|
||||
output_gcs_path=f"gs://{bucket_name}/{preview_gcs_path}",
|
||||
background_color="black",
|
||||
background_image_gcs_path=background_image_gcs_path,
|
||||
font_gcs_path=font_gcs_path,
|
||||
)
|
||||
gce_span.set_attribute("gce_status", gce_result.get("status"))
|
||||
add_span_event("gce_encoding_complete")
|
||||
|
||||
logger.info(f"Job {job_id}: Preview generated via GCE: {preview_hash}")
|
||||
|
||||
except Exception as gce_error:
|
||||
# Fall back to local encoding if GCE fails
|
||||
logger.warning(
|
||||
f"Job {job_id}: GCE preview encoding failed, falling back to local: {gce_error}"
|
||||
)
|
||||
span.set_attribute("gce_fallback", True)
|
||||
use_gce_preview = False # Fall through to local encoding below
|
||||
|
||||
if not use_gce_preview:
|
||||
# Local path: Generate full preview video locally
|
||||
with create_span("render-preview-video-local") as render_span:
|
||||
render_span.set_attribute("resolution", "360p")
|
||||
result = CorrectionOperations.generate_preview_video(
|
||||
correction_result=correction_result,
|
||||
updated_data=updated_data,
|
||||
output_config=output_config,
|
||||
audio_filepath=audio_path,
|
||||
artist=job.artist,
|
||||
title=job.title,
|
||||
logger=logger,
|
||||
ass_only=False, # Generate full video locally
|
||||
)
|
||||
preview_hash = result["preview_hash"]
|
||||
video_path = result["video_path"]
|
||||
add_span_event("render_complete")
|
||||
|
||||
# Upload preview video to GCS
|
||||
with create_span("upload-preview-video") as upload_span:
|
||||
preview_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.mp4"
|
||||
storage.upload_file(video_path, preview_gcs_path)
|
||||
upload_span.set_attribute("gcs_path", preview_gcs_path)
|
||||
|
||||
# Store the GCS path for serving
|
||||
if job_id not in _preview_videos:
|
||||
_preview_videos[job_id] = {}
|
||||
_preview_videos[job_id][preview_hash] = preview_gcs_path
|
||||
|
||||
|
||||
logger.info(f"Job {job_id}: Preview video generated: {preview_hash}")
|
||||
span.set_attribute("preview_hash", preview_hash)
|
||||
span.set_attribute("success", True)
|
||||
|
||||
|
||||
return {"status": "success", "preview_hash": preview_hash}
|
||||
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Job {job_id}: Failed to generate preview video: {e}", exc_info=True)
|
||||
span.set_attribute("error", str(e))
|
||||
|
||||
@@ -68,6 +68,12 @@ class Settings(BaseSettings):
|
||||
use_gce_encoding: bool = os.getenv("USE_GCE_ENCODING", "false").lower() in ("true", "1", "yes")
|
||||
encoding_worker_url: Optional[str] = os.getenv("ENCODING_WORKER_URL") # e.g., http://136.119.50.148:8080
|
||||
encoding_worker_api_key: Optional[str] = os.getenv("ENCODING_WORKER_API_KEY")
|
||||
|
||||
# GCE Preview Encoding (for faster preview video generation)
|
||||
# When enabled, preview video encoding during lyrics review is offloaded to the GCE worker.
|
||||
# This reduces preview generation time from 60+ seconds to ~15-20 seconds.
|
||||
# Requires use_gce_encoding to be enabled and the GCE worker to support /encode-preview endpoint.
|
||||
use_gce_preview_encoding: bool = os.getenv("USE_GCE_PREVIEW_ENCODING", "false").lower() in ("true", "1", "yes")
|
||||
|
||||
# Storage paths
|
||||
temp_dir: str = os.getenv("TEMP_DIR", "/tmp/karaoke-gen")
|
||||
|
||||
@@ -10,6 +10,7 @@ from backend.config import settings
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes, users, admin
|
||||
from backend.services.tracing import setup_tracing, instrument_app, get_current_trace_id
|
||||
from backend.services.structured_logging import setup_structured_logging
|
||||
from backend.services.spacy_preloader import preload_spacy_model
|
||||
from backend.middleware.audit_logging import AuditLoggingMiddleware
|
||||
|
||||
|
||||
@@ -67,7 +68,14 @@ async def lifespan(app: FastAPI):
|
||||
logger.info(f"Environment: {settings.environment}")
|
||||
logger.info(f"GCS Bucket: {settings.gcs_bucket_name}")
|
||||
logger.info(f"Tracing enabled: {tracing_enabled}")
|
||||
|
||||
|
||||
# Preload SpaCy model to avoid 60+ second delay on first request
|
||||
# See docs/archive/2026-01-08-spacy-preload-plan.md for background
|
||||
try:
|
||||
preload_spacy_model("en_core_web_sm")
|
||||
except Exception as e:
|
||||
logger.warning(f"SpaCy preload failed (will load lazily): {e}")
|
||||
|
||||
# Validate OAuth credentials (non-blocking)
|
||||
try:
|
||||
await validate_credentials_on_startup()
|
||||
|
||||
@@ -9,6 +9,8 @@ from datetime import datetime
|
||||
from enum import Enum
|
||||
from pydantic import BaseModel, Field, validator
|
||||
|
||||
from karaoke_gen.utils import normalize_text
|
||||
|
||||
|
||||
class JobStatus(str, Enum):
|
||||
"""
|
||||
@@ -485,13 +487,29 @@ class JobCreate(BaseModel):
|
||||
- custom_headers: All X-* headers
|
||||
"""
|
||||
|
||||
@validator('url', 'artist', 'title')
|
||||
def validate_inputs(cls, v):
|
||||
"""Validate string inputs are not empty."""
|
||||
@validator('url')
|
||||
def validate_url(cls, v):
|
||||
"""Validate URL is not empty."""
|
||||
if v is not None and isinstance(v, str) and not v.strip():
|
||||
raise ValueError("Field cannot be empty string")
|
||||
return v.strip() if isinstance(v, str) else v
|
||||
|
||||
@validator('artist', 'title')
|
||||
def normalize_artist_title(cls, v):
|
||||
"""Normalize artist/title text to standardize Unicode characters.
|
||||
|
||||
This ensures consistent data storage by converting:
|
||||
- Curly quotes -> straight quotes
|
||||
- Various dashes -> hyphen
|
||||
- Unusual whitespace -> regular space
|
||||
"""
|
||||
if v is not None and isinstance(v, str):
|
||||
if not v.strip():
|
||||
raise ValueError("Field cannot be empty string")
|
||||
# normalize_text handles stripping and Unicode normalization
|
||||
return normalize_text(v)
|
||||
return v
|
||||
|
||||
|
||||
class JobResponse(BaseModel):
|
||||
"""Job response model."""
|
||||
|
||||
@@ -14,7 +14,7 @@ class URLSubmissionRequest(BaseModel):
|
||||
# Optional preferences
|
||||
enable_cdg: bool = False # Requires style config
|
||||
enable_txt: bool = False # Requires style config
|
||||
enable_youtube_upload: bool = False
|
||||
enable_youtube_upload: Optional[bool] = None # None = use server default
|
||||
youtube_description: Optional[str] = None
|
||||
webhook_url: Optional[str] = None
|
||||
user_email: Optional[str] = None
|
||||
@@ -28,7 +28,7 @@ class UploadSubmissionRequest(BaseModel):
|
||||
# Optional preferences
|
||||
enable_cdg: bool = False # Requires style config
|
||||
enable_txt: bool = False # Requires style config
|
||||
enable_youtube_upload: bool = False
|
||||
enable_youtube_upload: Optional[bool] = None # None = use server default
|
||||
youtube_description: Optional[str] = None
|
||||
webhook_url: Optional[str] = None
|
||||
user_email: Optional[str] = None
|
||||
|
||||
@@ -351,8 +351,8 @@ class GCEEncodingBackend(EncodingBackend):
|
||||
result = {}
|
||||
raw_output_files = result.get("output_files", {})
|
||||
|
||||
# Convert output_files from list of paths to dict if needed
|
||||
# GCE worker returns list like: ["path/output_4k_lossless.mp4", "path/output_720p.mp4"]
|
||||
# Convert output_files from list of paths to dict
|
||||
# GCE worker returns list like: ["path/Artist - Title (Final Karaoke Lossless 4k).mp4", ...]
|
||||
# We need dict like: {"mp4_4k_lossless": "path/...", "mp4_720p": "path/..."}
|
||||
if isinstance(raw_output_files, list):
|
||||
output_files = {}
|
||||
@@ -360,15 +360,18 @@ class GCEEncodingBackend(EncodingBackend):
|
||||
if not isinstance(path, str):
|
||||
continue
|
||||
filename = path.split("/")[-1] if "/" in path else path
|
||||
filename_lower = filename.lower()
|
||||
# Map filename patterns to output format keys
|
||||
if "4k_lossless" in filename:
|
||||
output_files["mp4_4k_lossless"] = path
|
||||
elif "4k_lossy" in filename:
|
||||
# Files are named like "Artist - Title (Final Karaoke Lossless 4k).mp4"
|
||||
if "lossless 4k" in filename_lower:
|
||||
if filename.endswith(".mkv"):
|
||||
output_files["mkv_4k"] = path
|
||||
else:
|
||||
output_files["mp4_4k_lossless"] = path
|
||||
elif "lossy 4k" in filename_lower:
|
||||
output_files["mp4_4k_lossy"] = path
|
||||
elif "720p" in filename:
|
||||
elif "720p" in filename_lower:
|
||||
output_files["mp4_720p"] = path
|
||||
elif filename.endswith(".mkv"):
|
||||
output_files["mkv_4k"] = path
|
||||
self.logger.info(f"Converted output_files list to dict: {output_files}")
|
||||
else:
|
||||
output_files = raw_output_files if isinstance(raw_output_files, dict) else {}
|
||||
|
||||
@@ -63,6 +63,11 @@ class EncodingService:
|
||||
"""Check if GCE encoding is enabled and configured."""
|
||||
return self.settings.use_gce_encoding and self.is_configured
|
||||
|
||||
@property
|
||||
def is_preview_enabled(self) -> bool:
|
||||
"""Check if GCE preview encoding is enabled and configured."""
|
||||
return self.settings.use_gce_preview_encoding and self.is_configured
|
||||
|
||||
async def submit_encoding_job(
|
||||
self,
|
||||
job_id: str,
|
||||
@@ -242,6 +247,127 @@ class EncodingService:
|
||||
job_id, progress_callback=progress_callback
|
||||
)
|
||||
|
||||
async def submit_preview_encoding_job(
|
||||
self,
|
||||
job_id: str,
|
||||
ass_gcs_path: str,
|
||||
audio_gcs_path: str,
|
||||
output_gcs_path: str,
|
||||
background_color: str = "black",
|
||||
background_image_gcs_path: Optional[str] = None,
|
||||
font_gcs_path: Optional[str] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Submit a preview video encoding job to the GCE worker.
|
||||
|
||||
Args:
|
||||
job_id: Unique job identifier
|
||||
ass_gcs_path: GCS path to ASS subtitles file (gs://bucket/path/file.ass)
|
||||
audio_gcs_path: GCS path to audio file
|
||||
output_gcs_path: GCS path for output video
|
||||
background_color: Background color (default: black)
|
||||
background_image_gcs_path: Optional GCS path to background image
|
||||
font_gcs_path: Optional GCS path to custom font file
|
||||
|
||||
Returns:
|
||||
Response from the encoding worker
|
||||
|
||||
Raises:
|
||||
Exception: If submission fails
|
||||
"""
|
||||
self._load_credentials()
|
||||
|
||||
if not self.is_configured:
|
||||
raise RuntimeError("Encoding service not configured")
|
||||
|
||||
url = f"{self._url}/encode-preview"
|
||||
headers = {"X-API-Key": self._api_key, "Content-Type": "application/json"}
|
||||
payload = {
|
||||
"job_id": job_id,
|
||||
"ass_gcs_path": ass_gcs_path,
|
||||
"audio_gcs_path": audio_gcs_path,
|
||||
"output_gcs_path": output_gcs_path,
|
||||
"background_color": background_color,
|
||||
}
|
||||
if background_image_gcs_path:
|
||||
payload["background_image_gcs_path"] = background_image_gcs_path
|
||||
if font_gcs_path:
|
||||
payload["font_gcs_path"] = font_gcs_path
|
||||
|
||||
logger.info(f"[job:{job_id}] Submitting preview encoding job to GCE worker: {url}")
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.post(url, json=payload, headers=headers, timeout=30) as resp:
|
||||
if resp.status == 401:
|
||||
raise RuntimeError("Invalid API key for encoding worker")
|
||||
if resp.status == 409:
|
||||
raise RuntimeError(f"Preview encoding job {job_id} already exists")
|
||||
if resp.status != 200:
|
||||
text = await resp.text()
|
||||
raise RuntimeError(f"Failed to submit preview encoding job: {resp.status} - {text}")
|
||||
|
||||
return await resp.json()
|
||||
|
||||
async def encode_preview_video(
|
||||
self,
|
||||
job_id: str,
|
||||
ass_gcs_path: str,
|
||||
audio_gcs_path: str,
|
||||
output_gcs_path: str,
|
||||
background_color: str = "black",
|
||||
background_image_gcs_path: Optional[str] = None,
|
||||
font_gcs_path: Optional[str] = None,
|
||||
timeout: float = 90.0,
|
||||
poll_interval: float = 2.0,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Submit preview encoding job and wait for completion.
|
||||
|
||||
This is a convenience method that combines submit + wait with shorter
|
||||
timeouts suitable for preview videos.
|
||||
|
||||
Args:
|
||||
job_id: Unique job identifier
|
||||
ass_gcs_path: GCS path to ASS subtitles file
|
||||
audio_gcs_path: GCS path to audio file
|
||||
output_gcs_path: GCS path for output video
|
||||
background_color: Background color (default: black)
|
||||
background_image_gcs_path: Optional GCS path to background image
|
||||
font_gcs_path: Optional GCS path to custom font file
|
||||
timeout: Maximum time to wait (default 90s for preview)
|
||||
poll_interval: Seconds between status checks (default 2s)
|
||||
|
||||
Returns:
|
||||
Final job status with output files
|
||||
"""
|
||||
# Submit the job
|
||||
submit_result = await self.submit_preview_encoding_job(
|
||||
job_id=job_id,
|
||||
ass_gcs_path=ass_gcs_path,
|
||||
audio_gcs_path=audio_gcs_path,
|
||||
output_gcs_path=output_gcs_path,
|
||||
background_color=background_color,
|
||||
background_image_gcs_path=background_image_gcs_path,
|
||||
font_gcs_path=font_gcs_path,
|
||||
)
|
||||
|
||||
# If cached, return immediately - video already exists in GCS
|
||||
submit_status = submit_result.get("status")
|
||||
if submit_status == "cached":
|
||||
logger.info(f"[job:{job_id}] Preview already cached, returning immediately")
|
||||
return {"status": "complete", "output_path": submit_result.get("output_path")}
|
||||
|
||||
# If in_progress, another request is encoding it - just wait for that
|
||||
if submit_status == "in_progress":
|
||||
logger.info(f"[job:{job_id}] Preview encoding already in progress, waiting")
|
||||
|
||||
# Wait for completion with shorter timeout
|
||||
return await self.wait_for_completion(
|
||||
job_id=job_id,
|
||||
poll_interval=poll_interval,
|
||||
timeout=timeout,
|
||||
)
|
||||
|
||||
async def health_check(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Check the health of the encoding worker.
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
# GCE Encoding Worker
|
||||
|
||||
HTTP API service that runs on the GCE encoding worker VM for video encoding jobs.
|
||||
|
||||
## Overview
|
||||
|
||||
This service provides HTTP endpoints for submitting and monitoring video encoding jobs. It runs as a systemd service on the `encoding-worker` GCE VM instance.
|
||||
|
||||
## Endpoints
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/encode` | POST | Submit a full encoding job |
|
||||
| `/encode-preview` | POST | Submit a preview encoding job (480x270, fast) |
|
||||
| `/status/{job_id}` | GET | Get job status and progress |
|
||||
| `/health` | GET | Health check |
|
||||
|
||||
## Authentication
|
||||
|
||||
All endpoints (except `/health`) require an API key via the `X-API-Key` header. The key is stored in Secret Manager (`encoding-worker-api-key`).
|
||||
|
||||
## Encoding Process
|
||||
|
||||
### Full Encoding (`/encode`)
|
||||
1. Downloads input files from GCS (title screen, karaoke video, end screen, instrumental audio)
|
||||
2. Uses `LocalEncodingService` from the karaoke-gen wheel to produce:
|
||||
- Lossless 4K MP4
|
||||
- Lossy 4K MP4
|
||||
- Lossless MKV
|
||||
- 720p MP4
|
||||
3. Uploads results to GCS
|
||||
4. Reports progress via status endpoint
|
||||
|
||||
### Preview Encoding (`/encode-preview`)
|
||||
1. Downloads ASS subtitle and audio files from GCS
|
||||
2. Runs FFmpeg to create a quick 480x270 preview video
|
||||
3. Uploads result to GCS
|
||||
4. Used for real-time preview in the web UI
|
||||
|
||||
## Dependencies
|
||||
|
||||
- **FFmpeg**: Static build from John Van Sickle
|
||||
- **Python 3.13**: Built from source (required for karaoke-gen)
|
||||
- **karaoke-gen wheel**: Downloaded from GCS at job start (enables hot updates)
|
||||
|
||||
## Deployment
|
||||
|
||||
This service is deployed via the VM's startup script. The script:
|
||||
1. Installs system dependencies (FFmpeg, fonts, Python build deps)
|
||||
2. Builds Python 3.13 from source
|
||||
3. Creates venv and installs dependencies
|
||||
4. Writes `main.py` to `/opt/encoding-worker/`
|
||||
5. Creates systemd service
|
||||
6. Starts the service on port 8080
|
||||
|
||||
## Local Development
|
||||
|
||||
This code is extracted from the infrastructure for maintainability. To test locally:
|
||||
|
||||
```bash
|
||||
cd backend/services/gce_encoding
|
||||
pip install -r requirements.txt
|
||||
pip install ../../../ # Install karaoke-gen wheel
|
||||
ENCODING_API_KEY=test python main.py
|
||||
```
|
||||
|
||||
## Architecture Notes
|
||||
|
||||
- **Hot code updates**: The karaoke-gen wheel is re-downloaded at the start of each job, allowing code updates without VM restart
|
||||
- **In-memory job tracking**: Job state is stored in memory. Restart clears queue.
|
||||
- **Parallel encoding**: ThreadPoolExecutor with 4 workers
|
||||
- **GCS integration**: Direct download/upload via google-cloud-storage client
|
||||
@@ -0,0 +1,22 @@
|
||||
"""
|
||||
GCE Encoding Worker Service.
|
||||
|
||||
This package contains the FastAPI application that runs on the GCE encoding worker VM.
|
||||
It provides HTTP endpoints for video encoding jobs.
|
||||
|
||||
The service:
|
||||
- Downloads input files from GCS
|
||||
- Runs FFmpeg encoding via LocalEncodingService
|
||||
- Uploads results back to GCS
|
||||
- Provides job status tracking
|
||||
|
||||
Endpoints:
|
||||
- POST /encode - Submit a full encoding job
|
||||
- POST /encode-preview - Submit a preview encoding job (fast, low-res)
|
||||
- GET /status/{job_id} - Get job status
|
||||
- GET /health - Health check
|
||||
"""
|
||||
|
||||
from .main import app
|
||||
|
||||
__all__ = ["app"]
|
||||
@@ -0,0 +1,589 @@
|
||||
import asyncio
|
||||
import glob as glob_module
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import uuid
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import FastAPI, BackgroundTasks, HTTPException, Header, Depends
|
||||
from google.cloud import storage
|
||||
from pydantic import BaseModel
|
||||
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
app = FastAPI(title="Encoding Worker", version="1.0.0")
|
||||
|
||||
# API key authentication
|
||||
API_KEY = os.environ.get("ENCODING_API_KEY", "")
|
||||
|
||||
|
||||
async def verify_api_key(x_api_key: str = Header(None)):
|
||||
# Verify API key for authentication
|
||||
if not API_KEY:
|
||||
logger.warning("No API key configured - authentication disabled")
|
||||
return True
|
||||
if x_api_key != API_KEY:
|
||||
raise HTTPException(status_code=401, detail="Invalid API key")
|
||||
return True
|
||||
|
||||
# Job tracking
|
||||
jobs: dict[str, dict] = {}
|
||||
executor = ThreadPoolExecutor(max_workers=4) # 4 parallel encoding jobs
|
||||
|
||||
# GCS client
|
||||
storage_client = storage.Client()
|
||||
|
||||
class EncodeRequest(BaseModel):
|
||||
job_id: str
|
||||
input_gcs_path: str # gs://bucket/path/to/inputs/
|
||||
output_gcs_path: str # gs://bucket/path/to/outputs/
|
||||
encoding_config: dict # Video formats to generate
|
||||
|
||||
|
||||
class EncodePreviewRequest(BaseModel):
|
||||
job_id: str
|
||||
ass_gcs_path: str # gs://bucket/path/to/subtitles.ass
|
||||
audio_gcs_path: str # gs://bucket/path/to/audio.flac
|
||||
output_gcs_path: str # gs://bucket/path/to/output.mp4
|
||||
background_color: str = "black"
|
||||
background_image_gcs_path: Optional[str] = None
|
||||
font_gcs_path: Optional[str] = None # gs://bucket/path/to/custom-font.ttf
|
||||
|
||||
|
||||
class JobStatus(BaseModel):
|
||||
job_id: str
|
||||
status: str # pending, running, complete, failed
|
||||
progress: int # 0-100
|
||||
error: Optional[str] = None
|
||||
output_files: Optional[list[str]] = None
|
||||
|
||||
|
||||
def download_from_gcs(gcs_uri: str, local_path: Path):
|
||||
# Download a file or folder from GCS
|
||||
# Parse gs://bucket/path
|
||||
parts = gcs_uri.replace("gs://", "").split("/", 1)
|
||||
bucket_name = parts[0]
|
||||
prefix = parts[1] if len(parts) > 1 else ""
|
||||
|
||||
bucket = storage_client.bucket(bucket_name)
|
||||
blobs = list(bucket.list_blobs(prefix=prefix))
|
||||
|
||||
for blob in blobs:
|
||||
# Get relative path from prefix
|
||||
rel_path = blob.name[len(prefix):].lstrip("/")
|
||||
if not rel_path:
|
||||
continue
|
||||
dest = local_path / rel_path
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
blob.download_to_filename(str(dest))
|
||||
logger.info(f"Downloaded: {blob.name} -> {dest}")
|
||||
|
||||
|
||||
def upload_to_gcs(local_path: Path, gcs_uri: str):
|
||||
# Upload a file or folder to GCS
|
||||
parts = gcs_uri.replace("gs://", "").split("/", 1)
|
||||
bucket_name = parts[0]
|
||||
prefix = parts[1].rstrip("/") if len(parts) > 1 else ""
|
||||
|
||||
bucket = storage_client.bucket(bucket_name)
|
||||
|
||||
if local_path.is_file():
|
||||
blob_name = f"{prefix}/{local_path.name}" if prefix else local_path.name
|
||||
blob = bucket.blob(blob_name)
|
||||
blob.upload_from_filename(str(local_path))
|
||||
logger.info(f"Uploaded: {local_path} -> gs://{bucket_name}/{blob_name}")
|
||||
else:
|
||||
for file in local_path.rglob("*"):
|
||||
if file.is_file():
|
||||
rel_path = file.relative_to(local_path)
|
||||
blob_name = f"{prefix}/{rel_path}" if prefix else str(rel_path)
|
||||
blob = bucket.blob(blob_name)
|
||||
blob.upload_from_filename(str(file))
|
||||
logger.info(f"Uploaded: {file} -> gs://{bucket_name}/{blob_name}")
|
||||
|
||||
|
||||
def download_single_file_from_gcs(gcs_uri: str, local_path: Path):
|
||||
# Download a single file from GCS
|
||||
parts = gcs_uri.replace("gs://", "").split("/", 1)
|
||||
bucket_name = parts[0]
|
||||
blob_name = parts[1] if len(parts) > 1 else ""
|
||||
|
||||
bucket = storage_client.bucket(bucket_name)
|
||||
blob = bucket.blob(blob_name)
|
||||
local_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
blob.download_to_filename(str(local_path))
|
||||
logger.info(f"Downloaded: {gcs_uri} -> {local_path}")
|
||||
|
||||
|
||||
def upload_single_file_to_gcs(local_path: Path, gcs_uri: str):
|
||||
# Upload a single file to a specific GCS path
|
||||
parts = gcs_uri.replace("gs://", "").split("/", 1)
|
||||
bucket_name = parts[0]
|
||||
blob_name = parts[1] if len(parts) > 1 else ""
|
||||
|
||||
bucket = storage_client.bucket(bucket_name)
|
||||
blob = bucket.blob(blob_name)
|
||||
blob.upload_from_filename(str(local_path))
|
||||
logger.info(f"Uploaded: {local_path} -> {gcs_uri}")
|
||||
|
||||
|
||||
def run_preview_encoding(job_id: str, work_dir: Path, request: "EncodePreviewRequest"):
|
||||
# Run FFmpeg encoding for preview video (480x270, fast settings)
|
||||
jobs[job_id]["status"] = "running"
|
||||
jobs[job_id]["progress"] = 10
|
||||
|
||||
try:
|
||||
# Download input files
|
||||
ass_path = work_dir / "subtitles.ass"
|
||||
audio_path = work_dir / "audio.flac"
|
||||
|
||||
download_single_file_from_gcs(request.ass_gcs_path, ass_path)
|
||||
jobs[job_id]["progress"] = 20
|
||||
|
||||
download_single_file_from_gcs(request.audio_gcs_path, audio_path)
|
||||
jobs[job_id]["progress"] = 30
|
||||
|
||||
# Download background image if provided
|
||||
bg_image_path = None
|
||||
if request.background_image_gcs_path:
|
||||
bg_image_path = work_dir / "background.png"
|
||||
download_single_file_from_gcs(request.background_image_gcs_path, bg_image_path)
|
||||
|
||||
# Download custom font if provided and register with fontconfig
|
||||
if request.font_gcs_path:
|
||||
# Use standard fontconfig location that's already in the search path
|
||||
fonts_dir = Path("/usr/local/share/fonts/custom")
|
||||
fonts_dir.mkdir(parents=True, exist_ok=True)
|
||||
font_filename = request.font_gcs_path.split("/")[-1]
|
||||
font_path = fonts_dir / font_filename
|
||||
download_single_file_from_gcs(request.font_gcs_path, font_path)
|
||||
logger.info(f"Downloaded custom font: {font_path}")
|
||||
# Update fontconfig cache so libass can find the font
|
||||
subprocess.run(["fc-cache", "-fv"], capture_output=True)
|
||||
logger.info(f"Updated fontconfig cache with custom font: {font_filename}")
|
||||
|
||||
# Build FFmpeg command
|
||||
output_path = work_dir / "preview.mp4"
|
||||
|
||||
# Escape special characters in path for FFmpeg filter syntax
|
||||
# FFmpeg filter parsing requires escaping: \ : , [ ] ;
|
||||
def escape_ffmpeg_filter_path(path: str) -> str:
|
||||
# Note: Extra escaping needed since this is inside a triple-quoted string in Pulumi
|
||||
return path.replace("\\", "\\\\").replace(":", "\\:").replace(",", "\\,").replace("[", "\\[").replace("]", "\\]").replace(";", "\\;")
|
||||
|
||||
escaped_ass_path = escape_ffmpeg_filter_path(str(ass_path))
|
||||
|
||||
# Base command with frame rate
|
||||
cmd = ["ffmpeg", "-y", "-r", "24"]
|
||||
|
||||
# Video input: background image or solid color
|
||||
if bg_image_path and bg_image_path.exists():
|
||||
cmd.extend(["-loop", "1", "-i", str(bg_image_path)])
|
||||
# Scale and pad background to 480x270
|
||||
vf = f"scale=480:270:force_original_aspect_ratio=decrease,pad=480:270:(ow-iw)/2:(oh-ih)/2,ass={escaped_ass_path}"
|
||||
else:
|
||||
# Solid color background
|
||||
color = request.background_color or "black"
|
||||
cmd.extend(["-f", "lavfi", "-i", f"color=c={color}:s=480x270:r=24"])
|
||||
vf = f"ass={escaped_ass_path}"
|
||||
|
||||
# Audio input
|
||||
cmd.extend(["-i", str(audio_path)])
|
||||
|
||||
# Video filter and encoding settings
|
||||
cmd.extend([
|
||||
"-vf", vf,
|
||||
"-c:a", "aac", "-b:a", "96k",
|
||||
"-c:v", "libx264",
|
||||
"-preset", "superfast",
|
||||
"-crf", "28",
|
||||
"-pix_fmt", "yuv420p",
|
||||
"-movflags", "+faststart",
|
||||
"-threads", "8",
|
||||
"-shortest",
|
||||
str(output_path)
|
||||
])
|
||||
|
||||
jobs[job_id]["progress"] = 40
|
||||
logger.info(f"Running preview encoding: {' '.join(cmd)}")
|
||||
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
|
||||
if result.returncode != 0:
|
||||
logger.error(f"FFmpeg failed: {result.stderr}")
|
||||
raise RuntimeError(f"FFmpeg preview encoding failed: {result.stderr[-500:]}")
|
||||
|
||||
jobs[job_id]["progress"] = 80
|
||||
logger.info(f"Preview encoded: {output_path}")
|
||||
|
||||
# Upload output to GCS
|
||||
upload_single_file_to_gcs(output_path, request.output_gcs_path)
|
||||
jobs[job_id]["progress"] = 95
|
||||
|
||||
jobs[job_id]["output_files"] = [request.output_gcs_path]
|
||||
return output_path
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Preview encoding failed: {e}")
|
||||
jobs[job_id]["status"] = "failed"
|
||||
jobs[job_id]["error"] = str(e)
|
||||
raise
|
||||
|
||||
|
||||
def ensure_latest_wheel():
|
||||
'''Download and install latest karaoke-gen wheel from GCS.
|
||||
|
||||
Called at the start of each job to enable hot code updates without restart.
|
||||
In-progress jobs continue with their version, new jobs get latest code.
|
||||
'''
|
||||
try:
|
||||
logger.info("Checking for latest karaoke-gen wheel in GCS...")
|
||||
|
||||
# Download latest wheel
|
||||
result = subprocess.run(
|
||||
["gsutil", "cp", "gs://karaoke-gen-storage-nomadkaraoke/wheels/karaoke_gen-*.whl", "/tmp/"],
|
||||
capture_output=True, text=True, timeout=60
|
||||
)
|
||||
|
||||
# Find the downloaded wheel (get the latest by version sorting)
|
||||
wheels = glob_module.glob("/tmp/karaoke_gen-*.whl")
|
||||
if not wheels:
|
||||
logger.warning("No wheel found in GCS, using fallback encoding logic")
|
||||
return False
|
||||
|
||||
# Sort to get latest version
|
||||
wheel_path = sorted(wheels)[-1]
|
||||
logger.info(f"Installing wheel: {wheel_path}")
|
||||
|
||||
# Install (or upgrade) the wheel
|
||||
# Use 5-minute timeout - first install at job start may need to resolve dependencies
|
||||
# Subsequent installs are faster since dependencies are cached
|
||||
install_result = subprocess.run(
|
||||
[sys.executable, "-m", "pip", "install", "--upgrade", "--quiet", wheel_path],
|
||||
capture_output=True, text=True, timeout=300
|
||||
)
|
||||
|
||||
if install_result.returncode != 0:
|
||||
logger.warning(f"Wheel installation failed: {install_result.stderr}")
|
||||
return False
|
||||
|
||||
logger.info(f"Successfully installed wheel: {wheel_path}")
|
||||
return True
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("Wheel download/install timed out, using fallback")
|
||||
return False
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to ensure latest wheel: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def find_file(work_dir: Path, *patterns):
|
||||
'''Find a file matching any of the given glob patterns.'''
|
||||
for pattern in patterns:
|
||||
matches = list(work_dir.glob(f"**/{pattern}"))
|
||||
if matches:
|
||||
return matches[0]
|
||||
return None
|
||||
|
||||
|
||||
def run_encoding(job_id: str, work_dir: Path, config: dict):
|
||||
'''Run encoding using LocalEncodingService (single source of truth).
|
||||
|
||||
Uses LocalEncodingService from the installed karaoke-gen wheel to ensure
|
||||
output files match local CLI exactly:
|
||||
- Proper names like "Artist - Title (Final Karaoke Lossless 4k).mp4"
|
||||
- Concatenated title + karaoke + end screens
|
||||
- All formats: lossless 4K MP4, lossy 4K MP4, lossless MKV, 720p MP4
|
||||
|
||||
Requires the karaoke-gen wheel to be installed (done by ensure_latest_wheel).
|
||||
'''
|
||||
jobs[job_id]["status"] = "running"
|
||||
jobs[job_id]["progress"] = 10
|
||||
|
||||
try:
|
||||
# Import LocalEncodingService from installed wheel (required, no fallback)
|
||||
from backend.services.local_encoding_service import LocalEncodingService, EncodingConfig
|
||||
logger.info("Using LocalEncodingService from installed wheel")
|
||||
|
||||
# Get artist/title from config for proper naming
|
||||
artist = config.get("artist", "Unknown Artist")
|
||||
title = config.get("title", "Unknown Title")
|
||||
base_name = f"{artist} - {title}"
|
||||
logger.info(f"Encoding for: {base_name}")
|
||||
|
||||
# Find input files in work_dir
|
||||
# Title/end screens are in screens/ subdirectory
|
||||
title_video = find_file(work_dir, "screens/title.mov", "*Title*.mov", "*title*.mov")
|
||||
end_video = find_file(work_dir, "screens/end.mov", "*End*.mov", "*end*.mov")
|
||||
|
||||
# Karaoke video - search for With Vocals or main karaoke video
|
||||
karaoke_video = find_file(
|
||||
work_dir,
|
||||
"*With Vocals*.mov", "*With Vocals*.mkv",
|
||||
"*Vocals*.mov", "*Vocals*.mkv",
|
||||
"*.mkv", "*.mov"
|
||||
)
|
||||
# Exclude title/end/output videos
|
||||
if karaoke_video:
|
||||
name_lower = karaoke_video.name.lower()
|
||||
if "title" in name_lower or "end" in name_lower or "outputs" in str(karaoke_video):
|
||||
# Search more specifically for karaoke video
|
||||
karaoke_video = find_file(work_dir, "*Karaoke*.mkv", "*Karaoke*.mov", "*vocals*.mkv")
|
||||
|
||||
# Instrumental audio
|
||||
instrumental = find_file(
|
||||
work_dir,
|
||||
"*instrumental_clean*.flac", "*Instrumental Clean*.flac",
|
||||
"*instrumental*.flac", "*Instrumental*.flac",
|
||||
"*instrumental*.wav"
|
||||
)
|
||||
|
||||
logger.info(f"Found files:")
|
||||
logger.info(f" Title video: {title_video}")
|
||||
logger.info(f" Karaoke video: {karaoke_video}")
|
||||
logger.info(f" End video: {end_video}")
|
||||
logger.info(f" Instrumental: {instrumental}")
|
||||
|
||||
# Validate required files
|
||||
if not title_video:
|
||||
raise ValueError(f"No title video found in {work_dir}. Check screens/ subdirectory.")
|
||||
if not karaoke_video:
|
||||
raise ValueError(f"No karaoke video found in {work_dir}")
|
||||
if not instrumental:
|
||||
raise ValueError(f"No instrumental audio found in {work_dir}")
|
||||
|
||||
output_dir = work_dir / "outputs"
|
||||
output_dir.mkdir(exist_ok=True)
|
||||
|
||||
jobs[job_id]["progress"] = 20
|
||||
|
||||
# Build encoding config with proper file names
|
||||
encoding_config = EncodingConfig(
|
||||
title_video=str(title_video),
|
||||
karaoke_video=str(karaoke_video),
|
||||
instrumental_audio=str(instrumental),
|
||||
end_video=str(end_video) if end_video else None,
|
||||
output_karaoke_mp4=str(output_dir / f"{base_name} (Karaoke).mp4"),
|
||||
output_with_vocals_mp4=str(output_dir / f"{base_name} (With Vocals).mp4"),
|
||||
output_lossless_4k_mp4=str(output_dir / f"{base_name} (Final Karaoke Lossless 4k).mp4"),
|
||||
output_lossy_4k_mp4=str(output_dir / f"{base_name} (Final Karaoke Lossy 4k).mp4"),
|
||||
output_lossless_mkv=str(output_dir / f"{base_name} (Final Karaoke Lossless 4k).mkv"),
|
||||
output_720p_mp4=str(output_dir / f"{base_name} (Final Karaoke Lossy 720p).mp4"),
|
||||
)
|
||||
|
||||
# Create service and run encoding
|
||||
service = LocalEncodingService(logger=logger)
|
||||
|
||||
jobs[job_id]["progress"] = 30
|
||||
logger.info("Starting LocalEncodingService.encode_all_formats()")
|
||||
|
||||
result = service.encode_all_formats(encoding_config)
|
||||
|
||||
if not result.success:
|
||||
raise RuntimeError(f"Encoding failed: {result.error}")
|
||||
|
||||
jobs[job_id]["progress"] = 90
|
||||
|
||||
# Collect output files
|
||||
output_files = [str(f) for f in output_dir.glob("*") if f.is_file()]
|
||||
jobs[job_id]["output_files"] = output_files
|
||||
|
||||
logger.info(f"Encoding complete. Output files: {output_files}")
|
||||
return output_dir
|
||||
|
||||
except ImportError as e:
|
||||
# No fallback - wheel must be installed
|
||||
error_msg = (
|
||||
f"LocalEncodingService not available: {e}. "
|
||||
"The karaoke-gen wheel must be installed. "
|
||||
"Check that ensure_latest_wheel() succeeded and wheel exists in GCS."
|
||||
)
|
||||
logger.error(error_msg)
|
||||
jobs[job_id]["status"] = "failed"
|
||||
jobs[job_id]["error"] = error_msg
|
||||
raise RuntimeError(error_msg) from e
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Encoding failed: {e}")
|
||||
jobs[job_id]["status"] = "failed"
|
||||
jobs[job_id]["error"] = str(e)
|
||||
raise
|
||||
|
||||
|
||||
async def process_job(job_id: str, request: EncodeRequest):
|
||||
# Process an encoding job asynchronously
|
||||
try:
|
||||
# Download and install latest wheel at job start (allows hot updates without restart)
|
||||
# This means in-progress jobs continue with their version, new jobs get latest code
|
||||
ensure_latest_wheel()
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
work_dir = Path(temp_dir) / "work"
|
||||
work_dir.mkdir()
|
||||
|
||||
# Download input files
|
||||
jobs[job_id]["progress"] = 5
|
||||
logger.info(f"Downloading from {request.input_gcs_path}")
|
||||
download_from_gcs(request.input_gcs_path, work_dir)
|
||||
|
||||
# Run encoding in thread pool (CPU-bound)
|
||||
loop = asyncio.get_event_loop()
|
||||
output_dir = await loop.run_in_executor(
|
||||
executor,
|
||||
run_encoding,
|
||||
job_id,
|
||||
work_dir,
|
||||
request.encoding_config
|
||||
)
|
||||
|
||||
# Upload outputs
|
||||
jobs[job_id]["progress"] = 95
|
||||
logger.info(f"Uploading to {request.output_gcs_path}")
|
||||
upload_to_gcs(output_dir, request.output_gcs_path)
|
||||
|
||||
# Convert local paths to blob paths (backend expects blob paths, not full gs:// URIs)
|
||||
# output_gcs_path is like "gs://bucket/jobs/id/encoded/"
|
||||
# We need paths like "jobs/id/encoded/Artist - Title (Final Karaoke Lossless 4k).mp4"
|
||||
gcs_path = request.output_gcs_path.replace("gs://", "")
|
||||
parts = gcs_path.split("/", 1)
|
||||
prefix = parts[1].rstrip("/") if len(parts) > 1 else ""
|
||||
local_output_files = jobs[job_id].get("output_files", [])
|
||||
blob_paths = []
|
||||
for local_path in local_output_files:
|
||||
filename = Path(local_path).name
|
||||
blob_path = f"{prefix}/{filename}" if prefix else filename
|
||||
blob_paths.append(blob_path)
|
||||
jobs[job_id]["output_files"] = blob_paths
|
||||
logger.info(f"Output files (blob paths): {blob_paths}")
|
||||
|
||||
jobs[job_id]["status"] = "complete"
|
||||
jobs[job_id]["progress"] = 100
|
||||
logger.info(f"Job {job_id} complete")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Job {job_id} failed: {e}")
|
||||
jobs[job_id]["status"] = "failed"
|
||||
jobs[job_id]["error"] = str(e)
|
||||
|
||||
|
||||
async def process_preview_job(job_id: str, request: EncodePreviewRequest):
|
||||
# Process a preview encoding job asynchronously
|
||||
try:
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
work_dir = Path(temp_dir) / "work"
|
||||
work_dir.mkdir()
|
||||
|
||||
# Run preview encoding in thread pool (CPU-bound)
|
||||
# Note: run_preview_encoding handles download/upload internally
|
||||
loop = asyncio.get_event_loop()
|
||||
await loop.run_in_executor(
|
||||
executor,
|
||||
run_preview_encoding,
|
||||
job_id,
|
||||
work_dir,
|
||||
request
|
||||
)
|
||||
|
||||
jobs[job_id]["status"] = "complete"
|
||||
jobs[job_id]["progress"] = 100
|
||||
logger.info(f"Preview job {job_id} complete")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Preview job {job_id} failed: {e}")
|
||||
jobs[job_id]["status"] = "failed"
|
||||
jobs[job_id]["error"] = str(e)
|
||||
|
||||
|
||||
@app.post("/encode-preview")
|
||||
async def submit_preview_encode_job(request: EncodePreviewRequest, background_tasks: BackgroundTasks, _auth: bool = Depends(verify_api_key)):
|
||||
# Submit a preview encoding job
|
||||
job_id = request.job_id
|
||||
|
||||
# If job already exists, return cached result or current status
|
||||
if job_id in jobs:
|
||||
existing_job = jobs[job_id]
|
||||
if existing_job["status"] == "complete":
|
||||
# Return cached result - preview already encoded
|
||||
return {"status": "cached", "job_id": job_id, "output_path": existing_job.get("output_path")}
|
||||
elif existing_job["status"] == "failed":
|
||||
# Previous attempt failed, allow retry by replacing the job
|
||||
pass
|
||||
else:
|
||||
# Job is still in progress
|
||||
return {"status": "in_progress", "job_id": job_id}
|
||||
|
||||
jobs[job_id] = {
|
||||
"job_id": job_id,
|
||||
"status": "pending",
|
||||
"progress": 0,
|
||||
"error": None,
|
||||
"output_files": None,
|
||||
}
|
||||
|
||||
background_tasks.add_task(process_preview_job, job_id, request)
|
||||
|
||||
return {"status": "accepted", "job_id": job_id}
|
||||
|
||||
|
||||
@app.post("/encode")
|
||||
async def submit_encode_job(request: EncodeRequest, background_tasks: BackgroundTasks, _auth: bool = Depends(verify_api_key)):
|
||||
# Submit an encoding job
|
||||
job_id = request.job_id
|
||||
|
||||
if job_id in jobs:
|
||||
raise HTTPException(status_code=409, detail=f"Job {job_id} already exists")
|
||||
|
||||
jobs[job_id] = {
|
||||
"job_id": job_id,
|
||||
"status": "pending",
|
||||
"progress": 0,
|
||||
"error": None,
|
||||
"output_files": None,
|
||||
}
|
||||
|
||||
background_tasks.add_task(process_job, job_id, request)
|
||||
|
||||
return {"status": "accepted", "job_id": job_id}
|
||||
|
||||
|
||||
@app.get("/status/{job_id}")
|
||||
async def get_job_status(job_id: str, _auth: bool = Depends(verify_api_key)) -> JobStatus:
|
||||
# Get the status of an encoding job
|
||||
if job_id not in jobs:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
return JobStatus(**jobs[job_id])
|
||||
|
||||
|
||||
@app.get("/health")
|
||||
async def health_check():
|
||||
# Health check endpoint
|
||||
active_jobs = sum(1 for j in jobs.values() if j["status"] == "running")
|
||||
|
||||
# Get karaoke-gen wheel version if installed
|
||||
wheel_version = None
|
||||
try:
|
||||
from importlib.metadata import version as get_version
|
||||
wheel_version = get_version("karaoke-gen")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return {
|
||||
"status": "ok",
|
||||
"active_jobs": active_jobs,
|
||||
"queue_length": sum(1 for j in jobs.values() if j["status"] == "pending"),
|
||||
"ffmpeg_version": subprocess.run(["ffmpeg", "-version"], capture_output=True, text=True).stdout.split("\n")[0],
|
||||
"wheel_version": wheel_version,
|
||||
}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
uvicorn.run(app, host="0.0.0.0", port=8080)
|
||||
@@ -0,0 +1,16 @@
|
||||
# GCE Encoding Worker dependencies
|
||||
# These are installed in the VM's Python virtual environment
|
||||
|
||||
# Web framework
|
||||
fastapi>=0.109.0
|
||||
uvicorn>=0.27.0
|
||||
|
||||
# GCS client
|
||||
google-cloud-storage>=2.14.0
|
||||
|
||||
# Async utilities (optional, for future improvements)
|
||||
aiofiles>=23.2.0
|
||||
aiohttp>=3.9.0
|
||||
|
||||
# Note: The karaoke-gen wheel is installed separately from GCS.
|
||||
# It provides LocalEncodingService which does the actual encoding.
|
||||
@@ -15,6 +15,7 @@ import os
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from backend.config import get_settings
|
||||
from karaoke_gen.utils import sanitize_filename
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -253,7 +254,10 @@ class GoogleDriveService:
|
||||
Returns:
|
||||
Dictionary mapping category to uploaded file ID
|
||||
"""
|
||||
filename_base = f"{brand_code} - {base_name}"
|
||||
# Sanitize base_name to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
# that could cause issues with Google Drive API queries and file naming
|
||||
safe_base_name = sanitize_filename(base_name) if base_name else base_name
|
||||
filename_base = f"{brand_code} - {safe_base_name}"
|
||||
uploaded_files = {}
|
||||
|
||||
logger.info(
|
||||
|
||||
@@ -359,9 +359,147 @@ class JobManager:
|
||||
'instrumental_token': updates['instrumental_token'],
|
||||
'instrumental_token_expires_at': updates['instrumental_token_expires_at']
|
||||
})
|
||||
|
||||
|
||||
logger.info(f"Job {job_id} transitioned to {new_status}")
|
||||
|
||||
# Trigger notifications asynchronously (fire-and-forget)
|
||||
self._trigger_state_notifications(job_id, new_status)
|
||||
|
||||
return True
|
||||
|
||||
def _trigger_state_notifications(self, job_id: str, new_status: JobStatus) -> None:
|
||||
"""
|
||||
Trigger email notifications based on state transitions.
|
||||
|
||||
This is fire-and-forget - notification failures don't affect job processing.
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
new_status: New job status
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
try:
|
||||
# Get the job to access user info
|
||||
job = self.get_job(job_id)
|
||||
if not job or not job.user_email:
|
||||
logger.debug(f"No user email for job {job_id}, skipping notifications")
|
||||
return
|
||||
|
||||
# Job completion notification
|
||||
if new_status == JobStatus.COMPLETE:
|
||||
self._schedule_completion_email(job)
|
||||
|
||||
# Idle reminder scheduling for blocking states
|
||||
elif new_status in [JobStatus.AWAITING_REVIEW, JobStatus.AWAITING_INSTRUMENTAL_SELECTION]:
|
||||
self._schedule_idle_reminder(job, new_status)
|
||||
|
||||
except Exception as e:
|
||||
# Never let notification failures affect job processing
|
||||
logger.error(f"Error triggering notifications for job {job_id}: {e}")
|
||||
|
||||
def _schedule_completion_email(self, job: Job) -> None:
|
||||
"""
|
||||
Schedule sending a job completion email.
|
||||
|
||||
Uses asyncio to fire-and-forget the email sending.
|
||||
"""
|
||||
import asyncio
|
||||
import threading
|
||||
|
||||
try:
|
||||
from backend.services.job_notification_service import get_job_notification_service
|
||||
|
||||
notification_service = get_job_notification_service()
|
||||
|
||||
# Get youtube, dropbox URLs, and brand_code from state_data (may be None)
|
||||
state_data = job.state_data or {}
|
||||
youtube_url = state_data.get('youtube_url')
|
||||
dropbox_url = state_data.get('dropbox_link')
|
||||
brand_code = state_data.get('brand_code')
|
||||
|
||||
# Create async task (fire-and-forget)
|
||||
async def send_email():
|
||||
await notification_service.send_job_completion_email(
|
||||
job_id=job.job_id,
|
||||
user_email=job.user_email,
|
||||
user_name=None, # Could fetch from user service if needed
|
||||
artist=job.artist,
|
||||
title=job.title,
|
||||
youtube_url=youtube_url,
|
||||
dropbox_url=dropbox_url,
|
||||
brand_code=brand_code,
|
||||
)
|
||||
|
||||
# Try to get existing event loop, create new one if none exists
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
# If we're in an async context, create a task
|
||||
loop.create_task(send_email())
|
||||
except RuntimeError:
|
||||
# No event loop - we're likely in a sync context
|
||||
# Use daemon thread to avoid blocking job completion
|
||||
def run_in_thread():
|
||||
asyncio.run(send_email())
|
||||
thread = threading.Thread(target=run_in_thread, daemon=True)
|
||||
thread.start()
|
||||
|
||||
logger.info(f"Scheduled completion email for job {job.job_id}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to schedule completion email for job {job.job_id}: {e}")
|
||||
|
||||
def _schedule_idle_reminder(self, job: Job, new_status: JobStatus) -> None:
|
||||
"""
|
||||
Schedule an idle reminder for a blocking state.
|
||||
|
||||
Records the timestamp when the blocking state was entered and
|
||||
schedules a Cloud Tasks task for 5 minutes later.
|
||||
"""
|
||||
import asyncio
|
||||
import threading
|
||||
|
||||
try:
|
||||
# Record when we entered blocking state (for idle detection)
|
||||
blocking_entered_at = datetime.utcnow().isoformat()
|
||||
|
||||
action_type = "lyrics" if new_status == JobStatus.AWAITING_REVIEW else "instrumental"
|
||||
|
||||
# Update state_data with blocking state info (handle None state_data)
|
||||
existing_state_data = job.state_data or {}
|
||||
state_data_update = {
|
||||
'blocking_state_entered_at': blocking_entered_at,
|
||||
'blocking_action_type': action_type,
|
||||
'reminder_sent': False, # Will be set to True after reminder is sent
|
||||
}
|
||||
|
||||
self.firestore.update_job(job.job_id, {
|
||||
'state_data': {**existing_state_data, **state_data_update}
|
||||
})
|
||||
|
||||
# Schedule the idle reminder check via worker service (5 min delay)
|
||||
from backend.services.worker_service import get_worker_service
|
||||
|
||||
async def schedule_reminder():
|
||||
worker_service = get_worker_service()
|
||||
await worker_service.schedule_idle_reminder(job.job_id)
|
||||
|
||||
# Try to get existing event loop, create new one if none exists
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
loop.create_task(schedule_reminder())
|
||||
except RuntimeError:
|
||||
# No event loop - we're in a sync context
|
||||
# Use daemon thread to avoid blocking job processing
|
||||
def run_in_thread():
|
||||
asyncio.run(schedule_reminder())
|
||||
thread = threading.Thread(target=run_in_thread, daemon=True)
|
||||
thread.start()
|
||||
|
||||
logger.info(f"Scheduled idle reminder for job {job.job_id} ({action_type})")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to schedule idle reminder for job {job.job_id}: {e}")
|
||||
|
||||
def update_state_data(self, job_id: str, key: str, value: Any) -> None:
|
||||
"""
|
||||
|
||||
@@ -0,0 +1,271 @@
|
||||
"""
|
||||
Job notification service for sending emails on job state changes.
|
||||
|
||||
Handles:
|
||||
- Job completion emails (when job enters COMPLETE status)
|
||||
- Action reminder emails (when user is idle at blocking states)
|
||||
|
||||
This service orchestrates between the template service and email service,
|
||||
building the complete notification flow for jobs.
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
from typing import Optional, Dict, Any
|
||||
|
||||
from backend.services.email_service import get_email_service
|
||||
from backend.services.template_service import get_template_service
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Environment variable to enable/disable auto emails
|
||||
ENABLE_AUTO_EMAILS = os.getenv("ENABLE_AUTO_EMAILS", "true").lower() == "true"
|
||||
|
||||
# Feedback form URL (configured per environment, empty by default to avoid placeholder in emails)
|
||||
FEEDBACK_FORM_URL = os.getenv("FEEDBACK_FORM_URL", "")
|
||||
|
||||
|
||||
def _mask_email(email: str) -> str:
|
||||
"""Mask email for logging to protect PII. Shows first char + domain."""
|
||||
if not email or "@" not in email:
|
||||
return "***"
|
||||
local, domain = email.split("@", 1)
|
||||
if len(local) <= 1:
|
||||
return f"*@{domain}"
|
||||
return f"{local[0]}***@{domain}"
|
||||
|
||||
|
||||
class JobNotificationService:
|
||||
"""
|
||||
Service for sending job-related email notifications.
|
||||
|
||||
Coordinates between template service (for message rendering) and
|
||||
email service (for sending).
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize notification service."""
|
||||
self.email_service = get_email_service()
|
||||
self.template_service = get_template_service()
|
||||
self.frontend_url = os.getenv("FRONTEND_URL", "https://gen.nomadkaraoke.com")
|
||||
self.backend_url = os.getenv("BACKEND_URL", "https://api.nomadkaraoke.com")
|
||||
|
||||
def _build_review_url(self, job_id: str, audio_hash: Optional[str] = None, review_token: Optional[str] = None) -> str:
|
||||
"""Build the lyrics review URL for a job."""
|
||||
import urllib.parse
|
||||
|
||||
review_ui_url = os.getenv("REVIEW_UI_URL", f"{self.frontend_url}/lyrics/")
|
||||
base_api_url = f"{self.backend_url}/api/review/{job_id}"
|
||||
encoded_api_url = urllib.parse.quote(base_api_url, safe='')
|
||||
|
||||
url = f"{review_ui_url}?baseApiUrl={encoded_api_url}"
|
||||
if audio_hash:
|
||||
url += f"&audioHash={urllib.parse.quote(audio_hash, safe='')}"
|
||||
if review_token:
|
||||
url += f"&reviewToken={urllib.parse.quote(review_token, safe='')}"
|
||||
return url
|
||||
|
||||
def _build_instrumental_url(self, job_id: str, instrumental_token: Optional[str] = None) -> str:
|
||||
"""Build the instrumental selection URL for a job."""
|
||||
base_api_url = f"{self.backend_url}/api/jobs/{job_id}"
|
||||
|
||||
import urllib.parse
|
||||
encoded_api_url = urllib.parse.quote(base_api_url, safe='')
|
||||
|
||||
url = f"{self.frontend_url}/instrumental/?baseApiUrl={encoded_api_url}"
|
||||
if instrumental_token:
|
||||
url += f"&instrumentalToken={urllib.parse.quote(instrumental_token, safe='')}"
|
||||
return url
|
||||
|
||||
async def send_job_completion_email(
|
||||
self,
|
||||
job_id: str,
|
||||
user_email: str,
|
||||
user_name: Optional[str] = None,
|
||||
artist: Optional[str] = None,
|
||||
title: Optional[str] = None,
|
||||
youtube_url: Optional[str] = None,
|
||||
dropbox_url: Optional[str] = None,
|
||||
brand_code: Optional[str] = None,
|
||||
) -> bool:
|
||||
"""
|
||||
Send job completion email to user.
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
user_email: User's email address
|
||||
user_name: User's display name (optional)
|
||||
artist: Artist name
|
||||
title: Song title
|
||||
youtube_url: YouTube video URL
|
||||
dropbox_url: Dropbox folder URL
|
||||
brand_code: Release ID (e.g., "NOMAD-1178")
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
if not ENABLE_AUTO_EMAILS:
|
||||
logger.info(f"Auto emails disabled, skipping completion email for job {job_id}")
|
||||
return False
|
||||
|
||||
if not user_email:
|
||||
logger.warning(f"No user email for job {job_id}, skipping completion email")
|
||||
return False
|
||||
|
||||
try:
|
||||
# Render the completion message using template service
|
||||
message_content = self.template_service.render_job_completion(
|
||||
name=user_name,
|
||||
youtube_url=youtube_url,
|
||||
dropbox_url=dropbox_url,
|
||||
artist=artist,
|
||||
title=title,
|
||||
job_id=job_id,
|
||||
feedback_url=FEEDBACK_FORM_URL,
|
||||
)
|
||||
|
||||
# Send the email with CC to admin
|
||||
success = self.email_service.send_job_completion(
|
||||
to_email=user_email,
|
||||
message_content=message_content,
|
||||
artist=artist,
|
||||
title=title,
|
||||
brand_code=brand_code,
|
||||
cc_admin=True,
|
||||
)
|
||||
|
||||
if success:
|
||||
logger.info(f"Sent completion email for job {job_id} to {_mask_email(user_email)}")
|
||||
else:
|
||||
logger.error(f"Failed to send completion email for job {job_id}")
|
||||
|
||||
return success
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error sending completion email for job {job_id}: {e}")
|
||||
return False
|
||||
|
||||
async def send_action_reminder_email(
|
||||
self,
|
||||
job_id: str,
|
||||
user_email: str,
|
||||
action_type: str,
|
||||
user_name: Optional[str] = None,
|
||||
artist: Optional[str] = None,
|
||||
title: Optional[str] = None,
|
||||
audio_hash: Optional[str] = None,
|
||||
review_token: Optional[str] = None,
|
||||
instrumental_token: Optional[str] = None,
|
||||
) -> bool:
|
||||
"""
|
||||
Send action-needed reminder email to user.
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
user_email: User's email address
|
||||
action_type: Type of action needed ("lyrics" or "instrumental")
|
||||
user_name: User's display name
|
||||
artist: Artist name
|
||||
title: Song title
|
||||
audio_hash: Audio hash for review URL
|
||||
review_token: Review token for URL
|
||||
instrumental_token: Instrumental token for URL
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
if not ENABLE_AUTO_EMAILS:
|
||||
logger.info(f"Auto emails disabled, skipping reminder for job {job_id}")
|
||||
return False
|
||||
|
||||
if not user_email:
|
||||
logger.warning(f"No user email for job {job_id}, skipping reminder")
|
||||
return False
|
||||
|
||||
try:
|
||||
# Render the appropriate template
|
||||
if action_type == "lyrics":
|
||||
review_url = self._build_review_url(job_id, audio_hash, review_token)
|
||||
message_content = self.template_service.render_action_needed_lyrics(
|
||||
name=user_name,
|
||||
artist=artist,
|
||||
title=title,
|
||||
review_url=review_url,
|
||||
)
|
||||
elif action_type == "instrumental":
|
||||
instrumental_url = self._build_instrumental_url(job_id, instrumental_token)
|
||||
message_content = self.template_service.render_action_needed_instrumental(
|
||||
name=user_name,
|
||||
artist=artist,
|
||||
title=title,
|
||||
instrumental_url=instrumental_url,
|
||||
)
|
||||
else:
|
||||
logger.error(f"Unknown action type: {action_type}")
|
||||
return False
|
||||
|
||||
# Send the reminder email (no CC)
|
||||
success = self.email_service.send_action_reminder(
|
||||
to_email=user_email,
|
||||
message_content=message_content,
|
||||
action_type=action_type,
|
||||
artist=artist,
|
||||
title=title,
|
||||
)
|
||||
|
||||
if success:
|
||||
logger.info(f"Sent {action_type} reminder for job {job_id} to {_mask_email(user_email)}")
|
||||
else:
|
||||
logger.error(f"Failed to send {action_type} reminder for job {job_id}")
|
||||
|
||||
return success
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error sending {action_type} reminder for job {job_id}: {e}")
|
||||
return False
|
||||
|
||||
def get_completion_message(
|
||||
self,
|
||||
job_id: str,
|
||||
user_name: Optional[str] = None,
|
||||
artist: Optional[str] = None,
|
||||
title: Optional[str] = None,
|
||||
youtube_url: Optional[str] = None,
|
||||
dropbox_url: Optional[str] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Get the rendered completion message for a job (for admin copy functionality).
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
user_name: User's display name
|
||||
artist: Artist name
|
||||
title: Song title
|
||||
youtube_url: YouTube video URL
|
||||
dropbox_url: Dropbox folder URL
|
||||
|
||||
Returns:
|
||||
Rendered message content as plain text
|
||||
"""
|
||||
return self.template_service.render_job_completion(
|
||||
name=user_name,
|
||||
youtube_url=youtube_url,
|
||||
dropbox_url=dropbox_url,
|
||||
artist=artist,
|
||||
title=title,
|
||||
job_id=job_id,
|
||||
feedback_url=FEEDBACK_FORM_URL,
|
||||
)
|
||||
|
||||
|
||||
# Global instance
|
||||
_job_notification_service: Optional[JobNotificationService] = None
|
||||
|
||||
|
||||
def get_job_notification_service() -> JobNotificationService:
|
||||
"""Get the global job notification service instance."""
|
||||
global _job_notification_service
|
||||
if _job_notification_service is None:
|
||||
_job_notification_service = JobNotificationService()
|
||||
return _job_notification_service
|
||||
@@ -0,0 +1,407 @@
|
||||
"""
|
||||
Local Preview Encoding Service.
|
||||
|
||||
Provides local FFmpeg-based preview video encoding functionality, extracted from
|
||||
VideoGenerator for use by the GCE worker. This ensures the same encoding logic
|
||||
is used across local CLI, Cloud Run, and GCE worker environments.
|
||||
|
||||
This service handles:
|
||||
- Generating preview videos with ASS subtitle overlay
|
||||
- Hardware acceleration detection (NVENC) with fallback to software encoding
|
||||
- Background image/color support
|
||||
- Custom font support
|
||||
- Optimized settings for fast preview generation
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Optional, List
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class PreviewEncodingConfig:
|
||||
"""Configuration for preview video encoding."""
|
||||
ass_path: str # Path to ASS subtitles file
|
||||
audio_path: str # Path to audio file
|
||||
output_path: str # Path for output video file
|
||||
|
||||
# Optional background settings
|
||||
background_image_path: Optional[str] = None # Path to background image
|
||||
background_color: str = "black" # Fallback background color
|
||||
|
||||
# Optional font settings
|
||||
font_path: Optional[str] = None # Path to custom font file
|
||||
|
||||
|
||||
@dataclass
|
||||
class PreviewEncodingResult:
|
||||
"""Result of preview encoding operation."""
|
||||
success: bool
|
||||
output_path: Optional[str] = None
|
||||
error: Optional[str] = None
|
||||
|
||||
|
||||
class LocalPreviewEncodingService:
|
||||
"""
|
||||
Service for local FFmpeg-based preview video encoding.
|
||||
|
||||
This is the single source of truth for preview encoding logic, used by:
|
||||
- Local CLI (via VideoGenerator which delegates here)
|
||||
- Cloud Run (when GCE is unavailable)
|
||||
- GCE worker (via installed wheel)
|
||||
|
||||
Supports hardware acceleration (NVENC) with automatic fallback
|
||||
to software encoding (libx264) when unavailable.
|
||||
"""
|
||||
|
||||
# Preview video settings - these are the canonical values
|
||||
PREVIEW_WIDTH = 480
|
||||
PREVIEW_HEIGHT = 270
|
||||
PREVIEW_FPS = 24
|
||||
PREVIEW_AUDIO_BITRATE = "96k"
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
logger: Optional[logging.Logger] = None,
|
||||
):
|
||||
"""
|
||||
Initialize the local preview encoding service.
|
||||
|
||||
Args:
|
||||
logger: Optional logger instance
|
||||
"""
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
|
||||
# Hardware acceleration settings (detected on first use)
|
||||
self._nvenc_available: Optional[bool] = None
|
||||
self._video_encoder: Optional[str] = None
|
||||
self._hwaccel_flags: Optional[List[str]] = None
|
||||
|
||||
def _detect_nvenc_support(self) -> bool:
|
||||
"""
|
||||
Detect if NVENC hardware encoding is available.
|
||||
|
||||
Returns:
|
||||
True if NVENC is available, False otherwise
|
||||
"""
|
||||
try:
|
||||
self.logger.info("Detecting NVENC hardware acceleration...")
|
||||
|
||||
# Test h264_nvenc encoder directly
|
||||
test_cmd = [
|
||||
"ffmpeg", "-hide_banner", "-loglevel", "error",
|
||||
"-f", "lavfi", "-i", "testsrc=duration=1:size=320x240:rate=1",
|
||||
"-c:v", "h264_nvenc", "-f", "null", "-"
|
||||
]
|
||||
|
||||
result = subprocess.run(
|
||||
test_cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30
|
||||
)
|
||||
|
||||
if result.returncode == 0:
|
||||
self.logger.info("NVENC hardware acceleration available")
|
||||
return True
|
||||
|
||||
# Try alternative test with different source
|
||||
alt_test_cmd = [
|
||||
"ffmpeg", "-hide_banner", "-loglevel", "error",
|
||||
"-f", "lavfi", "-i", "color=red:size=320x240:duration=0.1",
|
||||
"-c:v", "h264_nvenc", "-preset", "fast", "-f", "null", "-"
|
||||
]
|
||||
|
||||
alt_result = subprocess.run(
|
||||
alt_test_cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30
|
||||
)
|
||||
|
||||
if alt_result.returncode == 0:
|
||||
self.logger.info("NVENC hardware acceleration available")
|
||||
return True
|
||||
|
||||
self.logger.info("NVENC not available, using software encoding")
|
||||
return False
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
self.logger.debug("NVENC detection timed out")
|
||||
return False
|
||||
except Exception as e:
|
||||
self.logger.debug(f"NVENC detection failed: {e}")
|
||||
return False
|
||||
|
||||
def _configure_hardware_acceleration(self) -> None:
|
||||
"""Configure hardware acceleration settings based on detected capabilities."""
|
||||
if self._nvenc_available is None:
|
||||
self._nvenc_available = self._detect_nvenc_support()
|
||||
|
||||
if self._nvenc_available:
|
||||
self._video_encoder = "h264_nvenc"
|
||||
self._hwaccel_flags = ["-hwaccel", "cuda", "-hwaccel_output_format", "cuda"]
|
||||
self.logger.info("Using NVENC hardware acceleration for preview encoding")
|
||||
else:
|
||||
self._video_encoder = "libx264"
|
||||
self._hwaccel_flags = []
|
||||
self.logger.info("Using software encoding (libx264) for preview")
|
||||
|
||||
@property
|
||||
def nvenc_available(self) -> bool:
|
||||
"""Check if NVENC hardware acceleration is available."""
|
||||
if self._nvenc_available is None:
|
||||
self._configure_hardware_acceleration()
|
||||
return self._nvenc_available
|
||||
|
||||
@property
|
||||
def video_encoder(self) -> str:
|
||||
"""Get the video encoder to use."""
|
||||
if self._video_encoder is None:
|
||||
self._configure_hardware_acceleration()
|
||||
return self._video_encoder
|
||||
|
||||
@property
|
||||
def hwaccel_flags(self) -> List[str]:
|
||||
"""Get hardware acceleration flags."""
|
||||
if self._hwaccel_flags is None:
|
||||
self._configure_hardware_acceleration()
|
||||
return self._hwaccel_flags
|
||||
|
||||
def _escape_ffmpeg_filter_path(self, path: str) -> str:
|
||||
"""
|
||||
Escape a path for FFmpeg filter expressions (for subprocess without shell).
|
||||
|
||||
When using subprocess with a command list (no shell), FFmpeg receives the
|
||||
filter string directly. FFmpeg's filter parser requires escaping:
|
||||
- Backslashes: double them (\\ -> \\\\)
|
||||
- Single quotes/apostrophes: escape with three backslashes (' -> \\\\')
|
||||
- Spaces: escape with backslash ( -> \\ )
|
||||
- Special characters: :,[];
|
||||
|
||||
Example: "I'm With You" becomes "I\\\\'m\\ With\\ You"
|
||||
"""
|
||||
# First escape existing backslashes
|
||||
escaped = path.replace("\\", "\\\\")
|
||||
# Escape single quotes
|
||||
escaped = escaped.replace("'", "\\\\\\'")
|
||||
# Escape spaces
|
||||
escaped = escaped.replace(" ", "\\ ")
|
||||
# Escape FFmpeg filter special characters
|
||||
escaped = escaped.replace(":", "\\:")
|
||||
escaped = escaped.replace(",", "\\,")
|
||||
escaped = escaped.replace("[", "\\[")
|
||||
escaped = escaped.replace("]", "\\]")
|
||||
escaped = escaped.replace(";", "\\;")
|
||||
return escaped
|
||||
|
||||
def _build_ass_filter(self, ass_path: str, font_path: Optional[str] = None) -> str:
|
||||
"""
|
||||
Build ASS filter with optional font directory support.
|
||||
|
||||
Args:
|
||||
ass_path: Path to ASS subtitles file
|
||||
font_path: Optional path to custom font file
|
||||
|
||||
Returns:
|
||||
FFmpeg ASS filter string
|
||||
"""
|
||||
escaped_ass_path = self._escape_ffmpeg_filter_path(ass_path)
|
||||
ass_filter = f"ass={escaped_ass_path}"
|
||||
|
||||
if font_path and os.path.isfile(font_path):
|
||||
font_dir = os.path.dirname(font_path)
|
||||
escaped_font_dir = self._escape_ffmpeg_filter_path(font_dir)
|
||||
ass_filter += f":fontsdir={escaped_font_dir}"
|
||||
self.logger.debug(f"Using font directory: {font_dir}")
|
||||
|
||||
return ass_filter
|
||||
|
||||
def _build_preview_ffmpeg_command(self, config: PreviewEncodingConfig) -> List[str]:
|
||||
"""
|
||||
Build FFmpeg command for preview video generation.
|
||||
|
||||
This is the canonical preview encoding command, ensuring consistency
|
||||
across all environments (local CLI, Cloud Run, GCE worker).
|
||||
|
||||
Args:
|
||||
config: Preview encoding configuration
|
||||
|
||||
Returns:
|
||||
FFmpeg command as a list of arguments
|
||||
"""
|
||||
width, height = self.PREVIEW_WIDTH, self.PREVIEW_HEIGHT
|
||||
|
||||
cmd = [
|
||||
"ffmpeg",
|
||||
"-hide_banner",
|
||||
"-loglevel", "error",
|
||||
"-r", str(self.PREVIEW_FPS),
|
||||
]
|
||||
|
||||
# Add hardware acceleration flags if available
|
||||
cmd.extend(self.hwaccel_flags)
|
||||
|
||||
# Input source (background image or solid color)
|
||||
if config.background_image_path and os.path.isfile(config.background_image_path):
|
||||
self.logger.debug(f"Using background image: {config.background_image_path}")
|
||||
cmd.extend([
|
||||
"-loop", "1",
|
||||
"-i", config.background_image_path,
|
||||
])
|
||||
# Build video filter with scaling and ASS subtitles
|
||||
video_filter = (
|
||||
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
|
||||
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2,"
|
||||
f"{self._build_ass_filter(config.ass_path, config.font_path)}"
|
||||
)
|
||||
else:
|
||||
self.logger.debug(f"Using solid {config.background_color} background")
|
||||
cmd.extend([
|
||||
"-f", "lavfi",
|
||||
"-i", f"color=c={config.background_color}:s={width}x{height}:r={self.PREVIEW_FPS}",
|
||||
])
|
||||
# Just ASS subtitles, no scaling needed
|
||||
video_filter = self._build_ass_filter(config.ass_path, config.font_path)
|
||||
|
||||
cmd.extend([
|
||||
"-i", config.audio_path,
|
||||
"-vf", video_filter,
|
||||
"-c:a", "aac",
|
||||
"-b:a", self.PREVIEW_AUDIO_BITRATE,
|
||||
"-c:v", self.video_encoder,
|
||||
])
|
||||
|
||||
# Add encoder-specific settings optimized for speed
|
||||
if self.nvenc_available:
|
||||
cmd.extend([
|
||||
"-preset", "p1", # Fastest NVENC preset
|
||||
"-tune", "ll", # Low latency
|
||||
"-rc", "cbr", # Constant bitrate for speed
|
||||
"-b:v", "800k", # Lower bitrate for speed
|
||||
"-profile:v", "baseline", # Most compatible profile
|
||||
"-level", "3.1", # Lower level for speed
|
||||
])
|
||||
self.logger.debug("Using NVENC with maximum speed settings")
|
||||
else:
|
||||
cmd.extend([
|
||||
"-profile:v", "baseline",
|
||||
"-level", "3.0",
|
||||
"-preset", "superfast",
|
||||
"-tune", "fastdecode",
|
||||
"-b:v", "600k",
|
||||
"-maxrate", "800k",
|
||||
"-bufsize", "1200k",
|
||||
"-crf", "28",
|
||||
])
|
||||
self.logger.debug("Using software encoding with maximum speed settings")
|
||||
|
||||
cmd.extend([
|
||||
"-pix_fmt", "yuv420p", # Required for browser compatibility
|
||||
"-movflags", "+faststart+frag_keyframe+empty_moov+dash",
|
||||
"-g", "48", # Keyframe every 48 frames (2 seconds at 24fps)
|
||||
"-keyint_min", "48",
|
||||
"-sc_threshold", "0", # Disable scene change detection for speed
|
||||
"-threads", "0", # Use all available CPU threads
|
||||
"-shortest",
|
||||
"-y",
|
||||
config.output_path,
|
||||
])
|
||||
|
||||
return cmd
|
||||
|
||||
def encode_preview(self, config: PreviewEncodingConfig) -> PreviewEncodingResult:
|
||||
"""
|
||||
Encode a preview video.
|
||||
|
||||
Args:
|
||||
config: Preview encoding configuration
|
||||
|
||||
Returns:
|
||||
PreviewEncodingResult with success status and output path
|
||||
"""
|
||||
self.logger.info(f"Encoding preview video: {config.output_path}")
|
||||
|
||||
# Validate input files
|
||||
if not os.path.isfile(config.ass_path):
|
||||
return PreviewEncodingResult(
|
||||
success=False,
|
||||
error=f"ASS subtitles file not found: {config.ass_path}"
|
||||
)
|
||||
|
||||
if not os.path.isfile(config.audio_path):
|
||||
return PreviewEncodingResult(
|
||||
success=False,
|
||||
error=f"Audio file not found: {config.audio_path}"
|
||||
)
|
||||
|
||||
# Ensure output directory exists
|
||||
output_dir = os.path.dirname(config.output_path)
|
||||
if output_dir:
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
try:
|
||||
# Build and execute FFmpeg command
|
||||
cmd = self._build_preview_ffmpeg_command(config)
|
||||
self.logger.debug(f"FFmpeg command: {' '.join(cmd)}")
|
||||
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=300 # 5 minute timeout for preview encoding
|
||||
)
|
||||
|
||||
if result.returncode != 0:
|
||||
error_msg = result.stderr[-500:] if result.stderr else "Unknown error"
|
||||
self.logger.error(f"FFmpeg failed: {error_msg}")
|
||||
return PreviewEncodingResult(
|
||||
success=False,
|
||||
error=f"FFmpeg preview encoding failed: {error_msg}"
|
||||
)
|
||||
|
||||
self.logger.info(f"Preview encoded successfully: {config.output_path}")
|
||||
return PreviewEncodingResult(
|
||||
success=True,
|
||||
output_path=config.output_path
|
||||
)
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
self.logger.error("Preview encoding timed out")
|
||||
return PreviewEncodingResult(
|
||||
success=False,
|
||||
error="Preview encoding timed out after 5 minutes"
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Preview encoding failed: {e}")
|
||||
return PreviewEncodingResult(
|
||||
success=False,
|
||||
error=str(e)
|
||||
)
|
||||
|
||||
|
||||
# Singleton instance and factory function
|
||||
_local_preview_encoding_service: Optional[LocalPreviewEncodingService] = None
|
||||
|
||||
|
||||
def get_local_preview_encoding_service(**kwargs) -> LocalPreviewEncodingService:
|
||||
"""
|
||||
Get a local preview encoding service instance.
|
||||
|
||||
Args:
|
||||
**kwargs: Arguments passed to LocalPreviewEncodingService
|
||||
|
||||
Returns:
|
||||
LocalPreviewEncodingService instance
|
||||
"""
|
||||
global _local_preview_encoding_service
|
||||
|
||||
if _local_preview_encoding_service is None:
|
||||
_local_preview_encoding_service = LocalPreviewEncodingService(**kwargs)
|
||||
|
||||
return _local_preview_encoding_service
|
||||
@@ -0,0 +1,65 @@
|
||||
"""SpaCy model preloader for container startup.
|
||||
|
||||
Loads SpaCy models at container startup to avoid slow loading during request processing.
|
||||
Cloud Run filesystem I/O can cause 60+ second delays when loading SpaCy models lazily.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Singleton storage for preloaded models
|
||||
_preloaded_models: dict = {}
|
||||
|
||||
|
||||
def preload_spacy_model(model_name: str = "en_core_web_sm") -> None:
|
||||
"""Preload a SpaCy model at startup.
|
||||
|
||||
Args:
|
||||
model_name: The SpaCy model to load (default: en_core_web_sm)
|
||||
"""
|
||||
global _preloaded_models
|
||||
|
||||
if model_name in _preloaded_models:
|
||||
logger.info(f"SpaCy model '{model_name}' already preloaded")
|
||||
return
|
||||
|
||||
logger.info(f"Preloading SpaCy model '{model_name}'...")
|
||||
start_time = time.time()
|
||||
|
||||
try:
|
||||
import spacy
|
||||
|
||||
nlp = spacy.load(model_name)
|
||||
_preloaded_models[model_name] = nlp
|
||||
|
||||
elapsed = time.time() - start_time
|
||||
logger.info(f"SpaCy model '{model_name}' preloaded in {elapsed:.2f}s")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to preload SpaCy model '{model_name}': {e}")
|
||||
raise
|
||||
|
||||
|
||||
def get_preloaded_model(model_name: str = "en_core_web_sm") -> Optional[object]:
|
||||
"""Get a preloaded SpaCy model if available.
|
||||
|
||||
Args:
|
||||
model_name: The SpaCy model name
|
||||
|
||||
Returns:
|
||||
The preloaded SpaCy Language object, or None if not preloaded
|
||||
"""
|
||||
return _preloaded_models.get(model_name)
|
||||
|
||||
|
||||
def is_model_preloaded(model_name: str = "en_core_web_sm") -> bool:
|
||||
"""Check if a SpaCy model has been preloaded."""
|
||||
return model_name in _preloaded_models
|
||||
|
||||
|
||||
def clear_preloaded_models() -> None:
|
||||
"""Clear all preloaded models. Useful for testing."""
|
||||
global _preloaded_models
|
||||
_preloaded_models.clear()
|
||||
@@ -0,0 +1,330 @@
|
||||
"""
|
||||
Template service for managing email templates stored in GCS.
|
||||
|
||||
Templates are stored in GCS bucket and rendered with job-specific variables.
|
||||
This allows updating email content without code deployment.
|
||||
|
||||
Template locations:
|
||||
- gs://{bucket}/templates/job-completion.txt - Job completion email (plain text)
|
||||
- gs://{bucket}/templates/action-needed-lyrics.txt - Lyrics review reminder
|
||||
- gs://{bucket}/templates/action-needed-instrumental.txt - Instrumental selection reminder
|
||||
"""
|
||||
import logging
|
||||
import re
|
||||
from typing import Optional, Dict, Any
|
||||
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Default templates (fallback if GCS fetch fails)
|
||||
DEFAULT_JOB_COMPLETION_TEMPLATE = """Hi {name},
|
||||
|
||||
Thanks for your order!
|
||||
|
||||
Here's the link for the karaoke video published to YouTube:
|
||||
{youtube_url}
|
||||
|
||||
Here's the dropbox folder with all the finished files and source files, including:
|
||||
- "(Final Karaoke Lossless).mkv": combined karaoke video in 4k H264 with lossless FLAC audio
|
||||
- "(Final Karaoke).mp4": combined karaoke video with title/end screen in 4k H264/AAC
|
||||
- "(Final Karaoke 720p).mp4": combined karaoke video in 720p H264/AAC (smaller file for older systems)
|
||||
- "(With Vocals).mp4": sing along video in 4k H264/AAC with original vocals
|
||||
- "(Karaoke).mov": karaoke video output from MidiCo (no title/end screen)
|
||||
- "(Title).mov"/"(End).mov": title card and end screen videos
|
||||
- "(Final Karaoke CDG).zip": CDG+MP3 format for older/commercial karaoke systems
|
||||
- "(Final Karaoke TXT).zip": TXT+MP3 format for Power Karaoke
|
||||
- stems/*.flac: various separated instrumental and vocal audio stems in lossless format
|
||||
- lyrics/*.txt song lyrics from various sources in plain text format
|
||||
|
||||
{dropbox_url}
|
||||
|
||||
Let me know if anything isn't perfect and I'll happily tweak / fix, or if you need it in any other format I can probably convert it for you!
|
||||
|
||||
If you have a moment, I'd really appreciate your feedback (takes 2 minutes):
|
||||
{feedback_url}
|
||||
|
||||
Thanks again and have a great day!
|
||||
-Andrew
|
||||
"""
|
||||
|
||||
DEFAULT_ACTION_NEEDED_LYRICS_TEMPLATE = """Hi {name},
|
||||
|
||||
Your karaoke video for "{artist} - {title}" is ready for lyrics review!
|
||||
|
||||
Our system has transcribed and synchronized the lyrics, but they may need some corrections. Please review and make any corrections necessary so we can finish generating your video.
|
||||
|
||||
Review your lyrics here:
|
||||
{review_url}
|
||||
|
||||
This usually takes just a few minutes.
|
||||
|
||||
Thanks!
|
||||
-Andrew
|
||||
"""
|
||||
|
||||
DEFAULT_ACTION_NEEDED_INSTRUMENTAL_TEMPLATE = """Hi {name},
|
||||
|
||||
Your karaoke video for "{artist} - {title}" is almost done!
|
||||
|
||||
We've separated the audio into different versions. Please select which instrumental track you'd like to use for the final video.
|
||||
|
||||
Select your instrumental here:
|
||||
{instrumental_url}
|
||||
|
||||
Thanks!
|
||||
-Andrew
|
||||
"""
|
||||
|
||||
|
||||
class TemplateService:
|
||||
"""
|
||||
Service for fetching and rendering email templates from GCS.
|
||||
|
||||
Templates support the following variables:
|
||||
- {name} - User's display name or "there" if unknown
|
||||
- {youtube_url} - YouTube video URL
|
||||
- {dropbox_url} - Dropbox folder URL
|
||||
- {artist} - Artist name
|
||||
- {title} - Song title
|
||||
- {job_id} - Job ID
|
||||
- {review_url} - Lyrics review URL
|
||||
- {instrumental_url} - Instrumental selection URL
|
||||
- {feedback_url} - Feedback form URL
|
||||
"""
|
||||
|
||||
TEMPLATE_PREFIX = "templates/"
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize template service."""
|
||||
self.settings = get_settings()
|
||||
self._storage_client = None
|
||||
self._bucket = None
|
||||
|
||||
@property
|
||||
def storage_client(self):
|
||||
"""Lazy-initialize storage client."""
|
||||
if self._storage_client is None:
|
||||
from google.cloud import storage
|
||||
self._storage_client = storage.Client(project=self.settings.google_cloud_project)
|
||||
return self._storage_client
|
||||
|
||||
@property
|
||||
def bucket(self):
|
||||
"""Get the GCS bucket for templates."""
|
||||
if self._bucket is None:
|
||||
self._bucket = self.storage_client.bucket(self.settings.gcs_bucket_name)
|
||||
return self._bucket
|
||||
|
||||
def _fetch_template_from_gcs(self, template_name: str) -> Optional[str]:
|
||||
"""
|
||||
Fetch a template from GCS.
|
||||
|
||||
Args:
|
||||
template_name: Name of template file (e.g., "job-completion.txt")
|
||||
|
||||
Returns:
|
||||
Template content as string, or None if not found
|
||||
"""
|
||||
blob_path = f"{self.TEMPLATE_PREFIX}{template_name}"
|
||||
try:
|
||||
blob = self.bucket.blob(blob_path)
|
||||
if blob.exists():
|
||||
content = blob.download_as_text()
|
||||
logger.debug(f"Fetched template from GCS: {blob_path}")
|
||||
return content
|
||||
else:
|
||||
logger.warning(f"Template not found in GCS: {blob_path}")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to fetch template {blob_path}: {e}")
|
||||
return None
|
||||
|
||||
def get_job_completion_template(self) -> str:
|
||||
"""Get the job completion email template."""
|
||||
template = self._fetch_template_from_gcs("job-completion.txt")
|
||||
if template is None:
|
||||
logger.info("Using default job completion template")
|
||||
return DEFAULT_JOB_COMPLETION_TEMPLATE
|
||||
return template
|
||||
|
||||
def get_action_needed_lyrics_template(self) -> str:
|
||||
"""Get the lyrics review reminder template."""
|
||||
template = self._fetch_template_from_gcs("action-needed-lyrics.txt")
|
||||
if template is None:
|
||||
logger.info("Using default lyrics reminder template")
|
||||
return DEFAULT_ACTION_NEEDED_LYRICS_TEMPLATE
|
||||
return template
|
||||
|
||||
def get_action_needed_instrumental_template(self) -> str:
|
||||
"""Get the instrumental selection reminder template."""
|
||||
template = self._fetch_template_from_gcs("action-needed-instrumental.txt")
|
||||
if template is None:
|
||||
logger.info("Using default instrumental reminder template")
|
||||
return DEFAULT_ACTION_NEEDED_INSTRUMENTAL_TEMPLATE
|
||||
return template
|
||||
|
||||
def render_template(self, template: str, variables: Dict[str, Any]) -> str:
|
||||
"""
|
||||
Render a template with the given variables.
|
||||
|
||||
Missing variables are replaced with empty strings.
|
||||
Handles conditional sections like feedback URL.
|
||||
|
||||
Args:
|
||||
template: Template string with {variable} placeholders
|
||||
variables: Dictionary of variable values
|
||||
|
||||
Returns:
|
||||
Rendered template string
|
||||
"""
|
||||
result = template
|
||||
|
||||
# Handle feedback URL section - remove if not provided
|
||||
if not variables.get("feedback_url"):
|
||||
# Remove feedback section (lines containing feedback_url placeholder and surrounding text)
|
||||
result = re.sub(
|
||||
r'\n*If you have a moment.*?\{feedback_url\}\n*',
|
||||
'\n',
|
||||
result,
|
||||
flags=re.DOTALL
|
||||
)
|
||||
variables["feedback_url"] = ""
|
||||
|
||||
# Replace all variables
|
||||
for key, value in variables.items():
|
||||
placeholder = "{" + key + "}"
|
||||
result = result.replace(placeholder, str(value) if value else "")
|
||||
|
||||
# Clean up any remaining unreplaced placeholders
|
||||
result = re.sub(r'\{[a-z_]+\}', '', result)
|
||||
|
||||
return result.strip()
|
||||
|
||||
def render_job_completion(
|
||||
self,
|
||||
name: Optional[str] = None,
|
||||
youtube_url: Optional[str] = None,
|
||||
dropbox_url: Optional[str] = None,
|
||||
artist: Optional[str] = None,
|
||||
title: Optional[str] = None,
|
||||
job_id: Optional[str] = None,
|
||||
feedback_url: Optional[str] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Render the job completion email template.
|
||||
|
||||
Args:
|
||||
name: User's display name (defaults to "there")
|
||||
youtube_url: YouTube video URL
|
||||
dropbox_url: Dropbox folder URL
|
||||
artist: Artist name
|
||||
title: Song title
|
||||
job_id: Job ID
|
||||
feedback_url: Feedback form URL (optional)
|
||||
|
||||
Returns:
|
||||
Rendered email content
|
||||
"""
|
||||
template = self.get_job_completion_template()
|
||||
variables = {
|
||||
"name": name or "there",
|
||||
"youtube_url": youtube_url or "[YouTube URL not available]",
|
||||
"dropbox_url": dropbox_url or "[Dropbox URL not available]",
|
||||
"artist": artist or "Unknown Artist",
|
||||
"title": title or "Unknown Title",
|
||||
"job_id": job_id or "",
|
||||
"feedback_url": feedback_url,
|
||||
}
|
||||
return self.render_template(template, variables)
|
||||
|
||||
def render_action_needed_lyrics(
|
||||
self,
|
||||
name: Optional[str] = None,
|
||||
artist: Optional[str] = None,
|
||||
title: Optional[str] = None,
|
||||
review_url: str = "",
|
||||
) -> str:
|
||||
"""
|
||||
Render the lyrics review reminder template.
|
||||
|
||||
Args:
|
||||
name: User's display name
|
||||
artist: Artist name
|
||||
title: Song title
|
||||
review_url: Lyrics review URL
|
||||
|
||||
Returns:
|
||||
Rendered email content
|
||||
"""
|
||||
template = self.get_action_needed_lyrics_template()
|
||||
variables = {
|
||||
"name": name or "there",
|
||||
"artist": artist or "Unknown Artist",
|
||||
"title": title or "Unknown Title",
|
||||
"review_url": review_url,
|
||||
}
|
||||
return self.render_template(template, variables)
|
||||
|
||||
def render_action_needed_instrumental(
|
||||
self,
|
||||
name: Optional[str] = None,
|
||||
artist: Optional[str] = None,
|
||||
title: Optional[str] = None,
|
||||
instrumental_url: str = "",
|
||||
) -> str:
|
||||
"""
|
||||
Render the instrumental selection reminder template.
|
||||
|
||||
Args:
|
||||
name: User's display name
|
||||
artist: Artist name
|
||||
title: Song title
|
||||
instrumental_url: Instrumental selection URL
|
||||
|
||||
Returns:
|
||||
Rendered email content
|
||||
"""
|
||||
template = self.get_action_needed_instrumental_template()
|
||||
variables = {
|
||||
"name": name or "there",
|
||||
"artist": artist or "Unknown Artist",
|
||||
"title": title or "Unknown Title",
|
||||
"instrumental_url": instrumental_url,
|
||||
}
|
||||
return self.render_template(template, variables)
|
||||
|
||||
def upload_template(self, template_name: str, content: str) -> bool:
|
||||
"""
|
||||
Upload a template to GCS.
|
||||
|
||||
Args:
|
||||
template_name: Name of template file (e.g., "job-completion.txt")
|
||||
content: Template content
|
||||
|
||||
Returns:
|
||||
True if upload successful
|
||||
"""
|
||||
blob_path = f"{self.TEMPLATE_PREFIX}{template_name}"
|
||||
try:
|
||||
blob = self.bucket.blob(blob_path)
|
||||
blob.upload_from_string(content, content_type="text/plain")
|
||||
logger.info(f"Uploaded template to GCS: {blob_path}")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to upload template {blob_path}: {e}")
|
||||
return False
|
||||
|
||||
|
||||
# Global instance
|
||||
_template_service: Optional[TemplateService] = None
|
||||
|
||||
|
||||
def get_template_service() -> TemplateService:
|
||||
"""Get the global template service instance."""
|
||||
global _template_service
|
||||
if _template_service is None:
|
||||
_template_service = TemplateService()
|
||||
return _template_service
|
||||
@@ -37,6 +37,7 @@ WORKER_QUEUES = {
|
||||
"screens": "screens-worker-queue",
|
||||
"render-video": "render-worker-queue",
|
||||
"video": "video-worker-queue",
|
||||
"idle-reminder": "idle-reminder-queue", # For delayed idle reminder checks
|
||||
}
|
||||
|
||||
# Dispatch deadlines for each worker type (in seconds)
|
||||
@@ -49,8 +50,12 @@ WORKER_DISPATCH_DEADLINES = {
|
||||
"screens": 600, # 10 min - Screen generation is fast
|
||||
"render-video": 1800, # 30 min - Video encoding can be slow
|
||||
"video": 1800, # 30 min - Video encoding can be slow
|
||||
"idle-reminder": 60, # 1 min - Quick check and potential email send
|
||||
}
|
||||
|
||||
# Default delay for idle reminders (seconds)
|
||||
IDLE_REMINDER_DELAY_SECONDS = 5 * 60 # 5 minutes
|
||||
|
||||
|
||||
class WorkerService:
|
||||
"""
|
||||
@@ -430,6 +435,99 @@ class WorkerService:
|
||||
"""Trigger render video worker (post-review)."""
|
||||
return await self.trigger_worker("render-video", job_id)
|
||||
|
||||
async def schedule_idle_reminder(
|
||||
self,
|
||||
job_id: str,
|
||||
delay_seconds: int = IDLE_REMINDER_DELAY_SECONDS
|
||||
) -> bool:
|
||||
"""
|
||||
Schedule an idle reminder check for a job.
|
||||
|
||||
The reminder task will be delivered after the specified delay.
|
||||
When the task executes, it checks if the job is still in a blocking
|
||||
state and sends a reminder email if the user hasn't taken action.
|
||||
|
||||
Args:
|
||||
job_id: Job ID to check
|
||||
delay_seconds: Delay before executing the check (default: 5 minutes)
|
||||
|
||||
Returns:
|
||||
True if task was scheduled successfully, False otherwise
|
||||
"""
|
||||
if not self._use_cloud_tasks:
|
||||
# In development mode, log and skip (no delayed execution support)
|
||||
logger.info(
|
||||
f"[job:{job_id}] Idle reminder not scheduled (Cloud Tasks disabled). "
|
||||
f"Would have fired in {delay_seconds}s."
|
||||
)
|
||||
return True
|
||||
|
||||
try:
|
||||
from google.cloud import tasks_v2
|
||||
from google.protobuf import timestamp_pb2
|
||||
import time
|
||||
|
||||
queue_name = WORKER_QUEUES.get("idle-reminder")
|
||||
if not queue_name:
|
||||
logger.error("Idle reminder queue not configured")
|
||||
return False
|
||||
|
||||
project = self.settings.google_cloud_project
|
||||
if not project:
|
||||
logger.error("GOOGLE_CLOUD_PROJECT not set")
|
||||
return False
|
||||
|
||||
location = self.settings.gcp_region
|
||||
|
||||
# Build queue path
|
||||
parent = self.tasks_client.queue_path(project, location, queue_name)
|
||||
|
||||
# Build headers
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
if self._admin_token:
|
||||
headers["X-Admin-Token"] = self._admin_token
|
||||
|
||||
# Inject trace context
|
||||
headers = inject_trace_context(headers)
|
||||
|
||||
# Calculate schedule time
|
||||
schedule_time = timestamp_pb2.Timestamp()
|
||||
schedule_time.FromSeconds(int(time.time()) + delay_seconds)
|
||||
|
||||
dispatch_deadline_seconds = WORKER_DISPATCH_DEADLINES.get("idle-reminder", 60)
|
||||
|
||||
# Build task payload
|
||||
task = {
|
||||
"http_request": {
|
||||
"http_method": tasks_v2.HttpMethod.POST,
|
||||
"url": f"{self._base_url}/api/internal/jobs/{job_id}/check-idle-reminder",
|
||||
"headers": headers,
|
||||
"body": json.dumps({"job_id": job_id}).encode(),
|
||||
"oidc_token": {
|
||||
"service_account_email": f"karaoke-backend@{project}.iam.gserviceaccount.com",
|
||||
},
|
||||
},
|
||||
"schedule_time": schedule_time,
|
||||
"dispatch_deadline": duration_pb2.Duration(seconds=dispatch_deadline_seconds),
|
||||
}
|
||||
|
||||
# Create task
|
||||
response = self.tasks_client.create_task(parent=parent, task=task)
|
||||
logger.info(
|
||||
f"[job:{job_id}] Scheduled idle reminder check in {delay_seconds}s: {response.name}"
|
||||
)
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"[job:{job_id}] Failed to schedule idle reminder: {e}",
|
||||
exc_info=True
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
# Global worker service instance
|
||||
_worker_service: Optional[WorkerService] = None
|
||||
|
||||
@@ -0,0 +1,411 @@
|
||||
"""
|
||||
Unit tests for admin email endpoints.
|
||||
|
||||
Tests the completion message and send email API endpoints.
|
||||
"""
|
||||
import pytest
|
||||
from unittest.mock import Mock, patch, MagicMock, AsyncMock
|
||||
from fastapi.testclient import TestClient
|
||||
from fastapi import FastAPI
|
||||
|
||||
from backend.api.routes.admin import router
|
||||
from backend.api.dependencies import require_admin
|
||||
from backend.models.job import Job, JobStatus
|
||||
|
||||
|
||||
# Create a test app with the admin router
|
||||
# The router already has prefix="/admin", so we add /api prefix
|
||||
app = FastAPI()
|
||||
app.include_router(router, prefix="/api")
|
||||
|
||||
|
||||
def get_mock_admin():
|
||||
"""Override for require_admin dependency."""
|
||||
return ("admin@example.com", "admin", 1)
|
||||
|
||||
|
||||
# Override the require_admin dependency
|
||||
app.dependency_overrides[require_admin] = get_mock_admin
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client():
|
||||
"""Create a test client."""
|
||||
return TestClient(app)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_job():
|
||||
"""Create a mock job."""
|
||||
job = Mock(spec=Job)
|
||||
job.job_id = "test-job-123"
|
||||
job.user_email = "user@example.com"
|
||||
job.artist = "Test Artist"
|
||||
job.title = "Test Song"
|
||||
job.status = JobStatus.COMPLETE
|
||||
job.audio_hash = "hash123"
|
||||
job.review_token = "review123"
|
||||
job.instrumental_token = "inst123"
|
||||
job.state_data = {
|
||||
"youtube_url": "https://youtube.com/watch?v=test123",
|
||||
"dropbox_link": "https://dropbox.com/folder/test",
|
||||
"brand_code": "NOMAD-1234",
|
||||
}
|
||||
return job
|
||||
|
||||
|
||||
class TestGetCompletionMessage:
|
||||
"""Tests for GET /api/admin/jobs/{job_id}/completion-message endpoint."""
|
||||
|
||||
def test_returns_completion_message(self, client, mock_job):
|
||||
"""Test successful completion message retrieval."""
|
||||
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
|
||||
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
|
||||
|
||||
# Setup mocks
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
mock_ns = Mock()
|
||||
mock_ns.get_completion_message.return_value = "Your video is ready!"
|
||||
mock_get_ns.return_value = mock_ns
|
||||
|
||||
response = client.get(
|
||||
"/api/admin/jobs/test-job-123/completion-message",
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["job_id"] == "test-job-123"
|
||||
assert data["message"] == "Your video is ready!"
|
||||
# Subject format: "NOMAD-1234: Test Artist - Test Song (Your karaoke video is ready!)"
|
||||
assert data["subject"] == "NOMAD-1234: Test Artist - Test Song (Your karaoke video is ready!)"
|
||||
assert data["youtube_url"] == "https://youtube.com/watch?v=test123"
|
||||
assert data["dropbox_url"] == "https://dropbox.com/folder/test"
|
||||
|
||||
def test_returns_404_when_job_not_found(self, client):
|
||||
"""Test 404 when job doesn't exist."""
|
||||
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = None
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
response = client.get(
|
||||
"/api/admin/jobs/nonexistent-job/completion-message",
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
assert response.status_code == 404
|
||||
|
||||
def test_handles_none_state_data(self, client, mock_job):
|
||||
"""Test handling of job with None state_data."""
|
||||
mock_job.state_data = None
|
||||
|
||||
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
|
||||
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
|
||||
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
mock_ns = Mock()
|
||||
mock_ns.get_completion_message.return_value = "Your video is ready!"
|
||||
mock_get_ns.return_value = mock_ns
|
||||
|
||||
response = client.get(
|
||||
"/api/admin/jobs/test-job-123/completion-message",
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["youtube_url"] is None
|
||||
assert data["dropbox_url"] is None
|
||||
|
||||
def test_default_subject_without_song_info(self, client, mock_job):
|
||||
"""Test default subject when no artist/title."""
|
||||
mock_job.artist = None
|
||||
mock_job.title = None
|
||||
|
||||
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
|
||||
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
|
||||
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
mock_ns = Mock()
|
||||
mock_ns.get_completion_message.return_value = "Message"
|
||||
mock_get_ns.return_value = mock_ns
|
||||
|
||||
response = client.get(
|
||||
"/api/admin/jobs/test-job-123/completion-message",
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["subject"] == "Your karaoke video is ready!"
|
||||
|
||||
|
||||
class TestSendCompletionEmail:
|
||||
"""Tests for POST /api/admin/jobs/{job_id}/send-completion-email endpoint."""
|
||||
|
||||
def test_sends_email_successfully(self, client, mock_job):
|
||||
"""Test successful email sending."""
|
||||
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
|
||||
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
|
||||
patch('backend.services.email_service.get_email_service') as mock_get_es:
|
||||
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
mock_ns = Mock()
|
||||
mock_ns.get_completion_message.return_value = "Your video is ready!"
|
||||
mock_get_ns.return_value = mock_ns
|
||||
|
||||
mock_es = Mock()
|
||||
mock_es.send_job_completion.return_value = True
|
||||
mock_get_es.return_value = mock_es
|
||||
|
||||
response = client.post(
|
||||
"/api/admin/jobs/test-job-123/send-completion-email",
|
||||
json={"to_email": "customer@example.com", "cc_admin": True},
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["success"] is True
|
||||
assert data["job_id"] == "test-job-123"
|
||||
assert data["to_email"] == "customer@example.com"
|
||||
|
||||
# Verify email was sent with correct params
|
||||
mock_es.send_job_completion.assert_called_once_with(
|
||||
to_email="customer@example.com",
|
||||
message_content="Your video is ready!",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
brand_code="NOMAD-1234",
|
||||
cc_admin=True,
|
||||
)
|
||||
|
||||
def test_sends_email_without_cc(self, client, mock_job):
|
||||
"""Test email sending without CC."""
|
||||
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
|
||||
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
|
||||
patch('backend.services.email_service.get_email_service') as mock_get_es:
|
||||
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
mock_ns = Mock()
|
||||
mock_ns.get_completion_message.return_value = "Message"
|
||||
mock_get_ns.return_value = mock_ns
|
||||
|
||||
mock_es = Mock()
|
||||
mock_es.send_job_completion.return_value = True
|
||||
mock_get_es.return_value = mock_es
|
||||
|
||||
response = client.post(
|
||||
"/api/admin/jobs/test-job-123/send-completion-email",
|
||||
json={"to_email": "customer@example.com", "cc_admin": False},
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
|
||||
# Verify CC was not included
|
||||
call_kwargs = mock_es.send_job_completion.call_args.kwargs
|
||||
assert call_kwargs["cc_admin"] is False
|
||||
|
||||
def test_returns_404_when_job_not_found(self, client):
|
||||
"""Test 404 when job doesn't exist."""
|
||||
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = None
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
response = client.post(
|
||||
"/api/admin/jobs/nonexistent-job/send-completion-email",
|
||||
json={"to_email": "customer@example.com"},
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
assert response.status_code == 404
|
||||
|
||||
def test_returns_500_when_email_fails(self, client, mock_job):
|
||||
"""Test 500 when email sending fails."""
|
||||
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
|
||||
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
|
||||
patch('backend.services.email_service.get_email_service') as mock_get_es:
|
||||
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
mock_ns = Mock()
|
||||
mock_ns.get_completion_message.return_value = "Message"
|
||||
mock_get_ns.return_value = mock_ns
|
||||
|
||||
mock_es = Mock()
|
||||
mock_es.send_job_completion.return_value = False # Email failed
|
||||
mock_get_es.return_value = mock_es
|
||||
|
||||
response = client.post(
|
||||
"/api/admin/jobs/test-job-123/send-completion-email",
|
||||
json={"to_email": "customer@example.com"},
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
assert response.status_code == 500
|
||||
assert "Failed to send email" in response.json()["detail"]
|
||||
|
||||
def test_handles_none_state_data(self, client, mock_job):
|
||||
"""Test handling of job with None state_data."""
|
||||
mock_job.state_data = None
|
||||
|
||||
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
|
||||
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
|
||||
patch('backend.services.email_service.get_email_service') as mock_get_es:
|
||||
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
mock_ns = Mock()
|
||||
mock_ns.get_completion_message.return_value = "Message"
|
||||
mock_get_ns.return_value = mock_ns
|
||||
|
||||
mock_es = Mock()
|
||||
mock_es.send_job_completion.return_value = True
|
||||
mock_get_es.return_value = mock_es
|
||||
|
||||
response = client.post(
|
||||
"/api/admin/jobs/test-job-123/send-completion-email",
|
||||
json={"to_email": "customer@example.com"},
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
|
||||
def test_accepts_any_email_string(self, client, mock_job):
|
||||
"""Test that any string is currently accepted as email (no validation).
|
||||
|
||||
Documents current behavior: FastAPI/Pydantic doesn't validate email
|
||||
format by default. If validation is added later, this test should
|
||||
be updated to expect 422 for invalid emails.
|
||||
"""
|
||||
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
|
||||
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
|
||||
patch('backend.services.email_service.get_email_service') as mock_get_es:
|
||||
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
mock_ns = Mock()
|
||||
mock_ns.get_completion_message.return_value = "Message"
|
||||
mock_get_ns.return_value = mock_ns
|
||||
|
||||
mock_es = Mock()
|
||||
mock_es.send_job_completion.return_value = True
|
||||
mock_get_es.return_value = mock_es
|
||||
|
||||
response = client.post(
|
||||
"/api/admin/jobs/test-job-123/send-completion-email",
|
||||
json={"to_email": "invalid-email"}, # Not a valid email format
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
# Currently accepts any string - endpoint succeeds
|
||||
assert response.status_code == 200
|
||||
# Verify the invalid email was passed through
|
||||
mock_es.send_job_completion.assert_called_once()
|
||||
call_kwargs = mock_es.send_job_completion.call_args.kwargs
|
||||
assert call_kwargs["to_email"] == "invalid-email"
|
||||
|
||||
|
||||
class TestIdleReminderEndpoint:
|
||||
"""Tests for the internal idle reminder endpoint."""
|
||||
|
||||
def test_sends_reminder_when_still_idle(self, mock_job):
|
||||
"""Test that reminder is sent when user is still idle."""
|
||||
from backend.api.routes.internal import router as internal_router
|
||||
from backend.api.dependencies import require_admin
|
||||
|
||||
# Create test app for internal router
|
||||
# The router already has prefix="/internal", so we add /api prefix
|
||||
test_app = FastAPI()
|
||||
test_app.include_router(internal_router, prefix="/api")
|
||||
test_app.dependency_overrides[require_admin] = get_mock_admin
|
||||
|
||||
mock_job.status = JobStatus.AWAITING_REVIEW.value # Use .value for string comparison
|
||||
mock_job.state_data = {
|
||||
"blocking_state_entered_at": "2024-01-01T00:00:00",
|
||||
"blocking_action_type": "lyrics",
|
||||
"reminder_sent": False,
|
||||
}
|
||||
|
||||
with patch('backend.api.routes.internal.JobManager') as mock_jm_class, \
|
||||
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
|
||||
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_jm.firestore = Mock() # Mock firestore for update_job call
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
mock_ns = Mock()
|
||||
mock_ns.send_action_reminder_email = AsyncMock(return_value=True)
|
||||
mock_get_ns.return_value = mock_ns
|
||||
|
||||
test_client = TestClient(test_app)
|
||||
response = test_client.post(
|
||||
"/api/internal/jobs/test-job-123/check-idle-reminder",
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
# Should succeed and return "sent" status
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["status"] == "sent"
|
||||
assert data["job_id"] == "test-job-123"
|
||||
|
||||
def test_skips_reminder_when_already_sent(self, mock_job):
|
||||
"""Test that reminder is skipped if already sent."""
|
||||
from backend.api.routes.internal import router as internal_router
|
||||
from backend.api.dependencies import require_admin
|
||||
|
||||
# Create test app for internal router
|
||||
# The router already has prefix="/internal", so we add /api prefix
|
||||
test_app = FastAPI()
|
||||
test_app.include_router(internal_router, prefix="/api")
|
||||
test_app.dependency_overrides[require_admin] = get_mock_admin
|
||||
|
||||
mock_job.status = JobStatus.AWAITING_REVIEW.value # Use .value for string comparison
|
||||
mock_job.state_data = {
|
||||
"blocking_state_entered_at": "2024-01-01T00:00:00",
|
||||
"blocking_action_type": "lyrics",
|
||||
"reminder_sent": True, # Already sent
|
||||
}
|
||||
|
||||
with patch('backend.api.routes.internal.JobManager') as mock_jm_class:
|
||||
mock_jm = Mock()
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_jm_class.return_value = mock_jm
|
||||
|
||||
test_client = TestClient(test_app)
|
||||
response = test_client.post(
|
||||
"/api/internal/jobs/test-job-123/check-idle-reminder",
|
||||
headers={"Authorization": "Bearer admin-token"}
|
||||
)
|
||||
|
||||
# Should succeed with "already_sent" status
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["status"] == "already_sent"
|
||||
assert data["job_id"] == "test-job-123"
|
||||
@@ -0,0 +1,492 @@
|
||||
"""
|
||||
Unit tests for email service.
|
||||
|
||||
Tests the new job completion and action reminder email methods,
|
||||
as well as CC support.
|
||||
"""
|
||||
import pytest
|
||||
from unittest.mock import Mock, patch, MagicMock
|
||||
|
||||
from backend.services.email_service import (
|
||||
EmailService,
|
||||
EmailProvider,
|
||||
ConsoleEmailProvider,
|
||||
SendGridEmailProvider,
|
||||
get_email_service,
|
||||
)
|
||||
|
||||
|
||||
class TestConsoleEmailProvider:
|
||||
"""Tests for console email provider."""
|
||||
|
||||
def test_send_email_basic(self):
|
||||
"""Test basic email logging."""
|
||||
provider = ConsoleEmailProvider()
|
||||
|
||||
result = provider.send_email(
|
||||
to_email="user@example.com",
|
||||
subject="Test Subject",
|
||||
html_content="<p>Test content</p>",
|
||||
)
|
||||
|
||||
assert result is True
|
||||
|
||||
def test_send_email_with_text_content(self):
|
||||
"""Test email with text content."""
|
||||
provider = ConsoleEmailProvider()
|
||||
|
||||
result = provider.send_email(
|
||||
to_email="user@example.com",
|
||||
subject="Test Subject",
|
||||
html_content="<p>Test</p>",
|
||||
text_content="Test plain text",
|
||||
)
|
||||
|
||||
assert result is True
|
||||
|
||||
def test_send_email_with_cc(self):
|
||||
"""Test email with CC recipients."""
|
||||
provider = ConsoleEmailProvider()
|
||||
|
||||
result = provider.send_email(
|
||||
to_email="user@example.com",
|
||||
subject="Test Subject",
|
||||
html_content="<p>Test</p>",
|
||||
cc_emails=["cc1@example.com", "cc2@example.com"],
|
||||
)
|
||||
|
||||
assert result is True
|
||||
|
||||
|
||||
class TestSendGridEmailProvider:
|
||||
"""Tests for SendGrid email provider."""
|
||||
|
||||
def test_send_email_success(self):
|
||||
"""Test successful email sending via SendGrid."""
|
||||
provider = SendGridEmailProvider(
|
||||
api_key="test-api-key",
|
||||
from_email="from@example.com",
|
||||
from_name="Test Sender",
|
||||
)
|
||||
|
||||
mock_response = Mock()
|
||||
mock_response.status_code = 202
|
||||
|
||||
# Patch at the sendgrid module level since it's imported inline
|
||||
with patch('sendgrid.SendGridAPIClient') as mock_sg:
|
||||
mock_client = Mock()
|
||||
mock_client.send.return_value = mock_response
|
||||
mock_sg.return_value = mock_client
|
||||
|
||||
result = provider.send_email(
|
||||
to_email="user@example.com",
|
||||
subject="Test Subject",
|
||||
html_content="<p>Test</p>",
|
||||
)
|
||||
|
||||
assert result is True
|
||||
mock_client.send.assert_called_once()
|
||||
|
||||
def test_send_email_with_cc(self):
|
||||
"""Test email sending with CC via SendGrid."""
|
||||
provider = SendGridEmailProvider(
|
||||
api_key="test-api-key",
|
||||
from_email="from@example.com",
|
||||
)
|
||||
|
||||
mock_response = Mock()
|
||||
mock_response.status_code = 202
|
||||
|
||||
with patch('sendgrid.SendGridAPIClient') as mock_sg:
|
||||
mock_client = Mock()
|
||||
mock_client.send.return_value = mock_response
|
||||
mock_sg.return_value = mock_client
|
||||
|
||||
result = provider.send_email(
|
||||
to_email="user@example.com",
|
||||
subject="Test Subject",
|
||||
html_content="<p>Test</p>",
|
||||
cc_emails=["cc@example.com"],
|
||||
)
|
||||
|
||||
assert result is True
|
||||
|
||||
def test_send_email_failure_status(self):
|
||||
"""Test email sending failure due to bad status."""
|
||||
provider = SendGridEmailProvider(
|
||||
api_key="test-api-key",
|
||||
from_email="from@example.com",
|
||||
)
|
||||
|
||||
mock_response = Mock()
|
||||
mock_response.status_code = 400
|
||||
|
||||
with patch('sendgrid.SendGridAPIClient') as mock_sg:
|
||||
mock_client = Mock()
|
||||
mock_client.send.return_value = mock_response
|
||||
mock_sg.return_value = mock_client
|
||||
|
||||
result = provider.send_email(
|
||||
to_email="user@example.com",
|
||||
subject="Test Subject",
|
||||
html_content="<p>Test</p>",
|
||||
)
|
||||
|
||||
assert result is False
|
||||
|
||||
def test_send_email_exception(self):
|
||||
"""Test email sending exception handling."""
|
||||
provider = SendGridEmailProvider(
|
||||
api_key="test-api-key",
|
||||
from_email="from@example.com",
|
||||
)
|
||||
|
||||
with patch('sendgrid.SendGridAPIClient') as mock_sg:
|
||||
mock_sg.return_value.send.side_effect = Exception("API error")
|
||||
|
||||
result = provider.send_email(
|
||||
to_email="user@example.com",
|
||||
subject="Test Subject",
|
||||
html_content="<p>Test</p>",
|
||||
)
|
||||
|
||||
assert result is False
|
||||
|
||||
|
||||
class TestEmailServiceJobCompletion:
|
||||
"""Tests for job completion email method."""
|
||||
|
||||
def test_send_job_completion_basic(self):
|
||||
"""Test basic job completion email."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
result = service.send_job_completion(
|
||||
to_email="user@example.com",
|
||||
message_content="Your video is ready!",
|
||||
)
|
||||
|
||||
assert result is True
|
||||
service.provider.send_email.assert_called_once()
|
||||
|
||||
def test_send_job_completion_with_song_info(self):
|
||||
"""Test job completion email includes song in subject."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
service.send_job_completion(
|
||||
to_email="user@example.com",
|
||||
message_content="Your video is ready!",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
)
|
||||
|
||||
call_args = service.provider.send_email.call_args
|
||||
subject = call_args.kwargs.get('subject') or call_args[0][1]
|
||||
assert "Test Artist" in subject
|
||||
assert "Test Song" in subject
|
||||
|
||||
def test_send_job_completion_with_brand_code(self):
|
||||
"""Test job completion email includes brand code in subject."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
service.send_job_completion(
|
||||
to_email="user@example.com",
|
||||
message_content="Your video is ready!",
|
||||
artist="Seether",
|
||||
title="Tonight",
|
||||
brand_code="NOMAD-1178",
|
||||
)
|
||||
|
||||
call_args = service.provider.send_email.call_args
|
||||
subject = call_args.kwargs.get('subject') or call_args[0][1]
|
||||
# Subject format: "NOMAD-1178: Seether - Tonight (Your karaoke video is ready!)"
|
||||
assert subject == "NOMAD-1178: Seether - Tonight (Your karaoke video is ready!)"
|
||||
|
||||
def test_send_job_completion_default_subject(self):
|
||||
"""Test job completion email default subject without song info."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
service.send_job_completion(
|
||||
to_email="user@example.com",
|
||||
message_content="Your video is ready!",
|
||||
)
|
||||
|
||||
call_args = service.provider.send_email.call_args
|
||||
subject = call_args.kwargs.get('subject') or call_args[0][1]
|
||||
assert "karaoke video is ready" in subject.lower()
|
||||
|
||||
def test_send_job_completion_with_cc(self):
|
||||
"""Test job completion email with CC to admin."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
service.send_job_completion(
|
||||
to_email="user@example.com",
|
||||
message_content="Your video is ready!",
|
||||
cc_admin=True,
|
||||
)
|
||||
|
||||
call_kwargs = service.provider.send_email.call_args.kwargs
|
||||
assert "gen@nomadkaraoke.com" in call_kwargs.get('cc_emails', [])
|
||||
|
||||
def test_send_job_completion_without_cc(self):
|
||||
"""Test job completion email without CC."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
service.send_job_completion(
|
||||
to_email="user@example.com",
|
||||
message_content="Your video is ready!",
|
||||
cc_admin=False,
|
||||
)
|
||||
|
||||
call_kwargs = service.provider.send_email.call_args.kwargs
|
||||
assert call_kwargs.get('cc_emails') is None
|
||||
|
||||
def test_send_job_completion_escapes_html(self):
|
||||
"""Test that message content is HTML-escaped."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
service.send_job_completion(
|
||||
to_email="user@example.com",
|
||||
message_content="Test <script>alert('xss')</script>",
|
||||
)
|
||||
|
||||
call_kwargs = service.provider.send_email.call_args.kwargs
|
||||
html_content = call_kwargs.get('html_content')
|
||||
assert "<script>" not in html_content
|
||||
assert "<script>" in html_content
|
||||
|
||||
def test_send_job_completion_includes_plain_text(self):
|
||||
"""Test that plain text content is included."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
message = "Your video is ready!"
|
||||
service.send_job_completion(
|
||||
to_email="user@example.com",
|
||||
message_content=message,
|
||||
)
|
||||
|
||||
call_kwargs = service.provider.send_email.call_args.kwargs
|
||||
assert call_kwargs.get('text_content') == message
|
||||
|
||||
|
||||
class TestEmailServiceActionReminder:
|
||||
"""Tests for action reminder email method."""
|
||||
|
||||
def test_send_action_reminder_lyrics(self):
|
||||
"""Test lyrics action reminder email."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
result = service.send_action_reminder(
|
||||
to_email="user@example.com",
|
||||
message_content="Please review your lyrics",
|
||||
action_type="lyrics",
|
||||
)
|
||||
|
||||
assert result is True
|
||||
call_kwargs = service.provider.send_email.call_args.kwargs
|
||||
subject = call_kwargs.get('subject')
|
||||
assert "lyrics" in subject.lower()
|
||||
|
||||
def test_send_action_reminder_instrumental(self):
|
||||
"""Test instrumental action reminder email."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
result = service.send_action_reminder(
|
||||
to_email="user@example.com",
|
||||
message_content="Please select instrumental",
|
||||
action_type="instrumental",
|
||||
)
|
||||
|
||||
assert result is True
|
||||
call_kwargs = service.provider.send_email.call_args.kwargs
|
||||
subject = call_kwargs.get('subject')
|
||||
assert "instrumental" in subject.lower()
|
||||
|
||||
def test_send_action_reminder_with_song_info(self):
|
||||
"""Test action reminder includes song in subject."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
service.send_action_reminder(
|
||||
to_email="user@example.com",
|
||||
message_content="Please review",
|
||||
action_type="lyrics",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
)
|
||||
|
||||
call_kwargs = service.provider.send_email.call_args.kwargs
|
||||
subject = call_kwargs.get('subject')
|
||||
assert "Test Artist" in subject
|
||||
assert "Test Song" in subject
|
||||
|
||||
def test_send_action_reminder_unknown_type(self):
|
||||
"""Test action reminder with unknown type still sends."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
result = service.send_action_reminder(
|
||||
to_email="user@example.com",
|
||||
message_content="Please take action",
|
||||
action_type="unknown",
|
||||
)
|
||||
|
||||
assert result is True
|
||||
call_kwargs = service.provider.send_email.call_args.kwargs
|
||||
subject = call_kwargs.get('subject')
|
||||
assert "Action needed" in subject
|
||||
|
||||
def test_send_action_reminder_escapes_html(self):
|
||||
"""Test that message content is HTML-escaped."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
service.send_action_reminder(
|
||||
to_email="user@example.com",
|
||||
message_content="Test <script>alert('xss')</script>",
|
||||
action_type="lyrics",
|
||||
)
|
||||
|
||||
call_kwargs = service.provider.send_email.call_args.kwargs
|
||||
html_content = call_kwargs.get('html_content')
|
||||
assert "<script>" not in html_content
|
||||
assert "<script>" in html_content
|
||||
|
||||
def test_send_action_reminder_no_cc(self):
|
||||
"""Test that action reminders don't have CC."""
|
||||
service = EmailService()
|
||||
service.provider = Mock()
|
||||
service.provider.send_email.return_value = True
|
||||
|
||||
service.send_action_reminder(
|
||||
to_email="user@example.com",
|
||||
message_content="Please review",
|
||||
action_type="lyrics",
|
||||
)
|
||||
|
||||
call_kwargs = service.provider.send_email.call_args.kwargs
|
||||
# Action reminders should not have CC
|
||||
assert call_kwargs.get('cc_emails') is None
|
||||
|
||||
|
||||
class TestEmailServiceConfiguration:
|
||||
"""Tests for email service configuration."""
|
||||
|
||||
def test_uses_sendgrid_when_configured(self):
|
||||
"""Test that SendGrid is used when API key is set."""
|
||||
with patch.dict('os.environ', {'SENDGRID_API_KEY': 'test-key'}):
|
||||
service = EmailService()
|
||||
assert isinstance(service.provider, SendGridEmailProvider)
|
||||
|
||||
def test_uses_console_when_not_configured(self):
|
||||
"""Test that console is used when no API key."""
|
||||
with patch.dict('os.environ', {}, clear=True):
|
||||
# Remove SENDGRID_API_KEY if it exists
|
||||
import os
|
||||
original = os.environ.pop('SENDGRID_API_KEY', None)
|
||||
try:
|
||||
service = EmailService()
|
||||
assert isinstance(service.provider, ConsoleEmailProvider)
|
||||
finally:
|
||||
if original:
|
||||
os.environ['SENDGRID_API_KEY'] = original
|
||||
|
||||
def test_is_configured_true_for_sendgrid(self):
|
||||
"""Test is_configured returns True for SendGrid."""
|
||||
service = EmailService()
|
||||
service.provider = SendGridEmailProvider("key", "from@example.com")
|
||||
assert service.is_configured() is True
|
||||
|
||||
def test_is_configured_false_for_console(self):
|
||||
"""Test is_configured returns False for console."""
|
||||
service = EmailService()
|
||||
service.provider = ConsoleEmailProvider()
|
||||
assert service.is_configured() is False
|
||||
|
||||
|
||||
class TestGlobalInstance:
|
||||
"""Tests for global instance management."""
|
||||
|
||||
def test_get_email_service_returns_same_instance(self):
|
||||
"""Test that get_email_service returns singleton."""
|
||||
# Reset global
|
||||
import backend.services.email_service as es
|
||||
es._email_service = None
|
||||
|
||||
service1 = get_email_service()
|
||||
service2 = get_email_service()
|
||||
|
||||
assert service1 is service2
|
||||
|
||||
|
||||
class TestCCFunctionality:
|
||||
"""Integration tests for CC functionality across providers."""
|
||||
|
||||
def test_console_provider_logs_cc(self, caplog):
|
||||
"""Test that console provider logs CC recipients."""
|
||||
import logging
|
||||
caplog.set_level(logging.INFO)
|
||||
|
||||
provider = ConsoleEmailProvider()
|
||||
provider.send_email(
|
||||
to_email="user@example.com",
|
||||
subject="Test",
|
||||
html_content="<p>Test</p>",
|
||||
cc_emails=["cc1@example.com", "cc2@example.com"],
|
||||
)
|
||||
|
||||
# Check that CC was logged
|
||||
assert "cc1@example.com" in caplog.text or "CC:" in caplog.text
|
||||
|
||||
def test_sendgrid_provider_adds_cc_recipients(self):
|
||||
"""Test that SendGrid provider adds CC recipients to message."""
|
||||
provider = SendGridEmailProvider(
|
||||
api_key="test-api-key",
|
||||
from_email="from@example.com",
|
||||
)
|
||||
|
||||
mock_response = Mock()
|
||||
mock_response.status_code = 202
|
||||
|
||||
# Patch at sendgrid module level since imports are inline
|
||||
with patch('sendgrid.SendGridAPIClient') as mock_sg:
|
||||
mock_client = Mock()
|
||||
mock_client.send.return_value = mock_response
|
||||
mock_sg.return_value = mock_client
|
||||
|
||||
# We need to capture the Mail object
|
||||
with patch('sendgrid.helpers.mail.Mail') as mock_mail_class:
|
||||
mock_mail = MagicMock()
|
||||
mock_mail_class.return_value = mock_mail
|
||||
|
||||
provider.send_email(
|
||||
to_email="user@example.com",
|
||||
subject="Test",
|
||||
html_content="<p>Test</p>",
|
||||
cc_emails=["cc@example.com"],
|
||||
)
|
||||
|
||||
# Verify add_cc was called
|
||||
mock_mail.add_cc.assert_called()
|
||||
@@ -1128,7 +1128,8 @@ class TestFinaliseOnlyModels:
|
||||
assert request.enable_txt is None
|
||||
assert request.brand_prefix is None
|
||||
assert request.keep_brand_code is None
|
||||
assert request.enable_youtube_upload is False
|
||||
# YouTube upload default is None (server applies default_enable_youtube_upload)
|
||||
assert request.enable_youtube_upload is None
|
||||
assert request.dropbox_path is None
|
||||
assert request.gdrive_folder_id is None
|
||||
|
||||
|
||||
@@ -0,0 +1,329 @@
|
||||
"""
|
||||
Unit tests for job manager notification methods.
|
||||
|
||||
Tests the state transition triggers for email notifications.
|
||||
"""
|
||||
import pytest
|
||||
from unittest.mock import Mock, patch, MagicMock, AsyncMock
|
||||
from datetime import datetime
|
||||
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.models.job import Job, JobStatus
|
||||
|
||||
|
||||
class TestTriggerStateNotifications:
|
||||
"""Tests for _trigger_state_notifications method."""
|
||||
|
||||
def test_triggers_completion_email_on_complete(self):
|
||||
"""Test that completion email is triggered when job completes."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = {"youtube_url": "https://youtube.com/123"}
|
||||
manager.get_job = Mock(return_value=mock_job)
|
||||
|
||||
with patch.object(manager, '_schedule_completion_email') as mock_schedule:
|
||||
manager._trigger_state_notifications("job-123", JobStatus.COMPLETE)
|
||||
|
||||
mock_schedule.assert_called_once_with(mock_job)
|
||||
|
||||
def test_triggers_idle_reminder_on_awaiting_review(self):
|
||||
"""Test that idle reminder is triggered when job enters awaiting review."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = {}
|
||||
manager.get_job = Mock(return_value=mock_job)
|
||||
|
||||
with patch.object(manager, '_schedule_idle_reminder') as mock_schedule:
|
||||
manager._trigger_state_notifications("job-123", JobStatus.AWAITING_REVIEW)
|
||||
|
||||
mock_schedule.assert_called_once_with(mock_job, JobStatus.AWAITING_REVIEW)
|
||||
|
||||
def test_triggers_idle_reminder_on_awaiting_instrumental(self):
|
||||
"""Test that idle reminder is triggered when job enters awaiting instrumental."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = {}
|
||||
manager.get_job = Mock(return_value=mock_job)
|
||||
|
||||
with patch.object(manager, '_schedule_idle_reminder') as mock_schedule:
|
||||
manager._trigger_state_notifications(
|
||||
"job-123",
|
||||
JobStatus.AWAITING_INSTRUMENTAL_SELECTION
|
||||
)
|
||||
|
||||
mock_schedule.assert_called_once_with(
|
||||
mock_job,
|
||||
JobStatus.AWAITING_INSTRUMENTAL_SELECTION
|
||||
)
|
||||
|
||||
def test_skips_notification_when_no_job(self):
|
||||
"""Test that notification is skipped when job not found."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
manager.get_job = Mock(return_value=None)
|
||||
|
||||
with patch.object(manager, '_schedule_completion_email') as mock_schedule:
|
||||
manager._trigger_state_notifications("job-123", JobStatus.COMPLETE)
|
||||
|
||||
mock_schedule.assert_not_called()
|
||||
|
||||
def test_skips_notification_when_no_user_email(self):
|
||||
"""Test that notification is skipped when job has no user email."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = None
|
||||
manager.get_job = Mock(return_value=mock_job)
|
||||
|
||||
with patch.object(manager, '_schedule_completion_email') as mock_schedule:
|
||||
manager._trigger_state_notifications("job-123", JobStatus.COMPLETE)
|
||||
|
||||
mock_schedule.assert_not_called()
|
||||
|
||||
def test_does_not_trigger_on_other_states(self):
|
||||
"""Test that no notification is triggered for non-notification states."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = {}
|
||||
manager.get_job = Mock(return_value=mock_job)
|
||||
|
||||
with patch.object(manager, '_schedule_completion_email') as mock_complete, \
|
||||
patch.object(manager, '_schedule_idle_reminder') as mock_idle:
|
||||
|
||||
# Test various non-notification states
|
||||
for status in [JobStatus.PENDING, JobStatus.DOWNLOADING, JobStatus.TRANSCRIBING,
|
||||
JobStatus.RENDERING_VIDEO, JobStatus.FAILED]:
|
||||
manager._trigger_state_notifications("job-123", status)
|
||||
|
||||
mock_complete.assert_not_called()
|
||||
mock_idle.assert_not_called()
|
||||
|
||||
def test_handles_exception_gracefully(self):
|
||||
"""Test that exceptions don't propagate from notifications."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
manager.get_job = Mock(side_effect=Exception("Database error"))
|
||||
|
||||
# Should not raise
|
||||
manager._trigger_state_notifications("job-123", JobStatus.COMPLETE)
|
||||
|
||||
|
||||
class TestScheduleCompletionEmail:
|
||||
"""Tests for _schedule_completion_email method."""
|
||||
|
||||
def test_extracts_urls_from_state_data(self):
|
||||
"""Test that YouTube and Dropbox URLs are extracted from state_data."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.artist = "Test Artist"
|
||||
mock_job.title = "Test Song"
|
||||
mock_job.state_data = {
|
||||
"youtube_url": "https://youtube.com/watch?v=123",
|
||||
"dropbox_link": "https://dropbox.com/folder/abc",
|
||||
}
|
||||
|
||||
with patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_service:
|
||||
mock_service = Mock()
|
||||
mock_service.send_job_completion_email = AsyncMock(return_value=True)
|
||||
mock_get_service.return_value = mock_service
|
||||
|
||||
# Run in sync context - this will start a daemon thread
|
||||
manager._schedule_completion_email(mock_job)
|
||||
|
||||
# Give the thread a moment to execute
|
||||
import time
|
||||
time.sleep(0.1)
|
||||
|
||||
def test_handles_none_state_data(self):
|
||||
"""Test that None state_data is handled gracefully."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.artist = None
|
||||
mock_job.title = None
|
||||
mock_job.state_data = None # None state_data
|
||||
|
||||
with patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_service:
|
||||
mock_service = Mock()
|
||||
mock_service.send_job_completion_email = AsyncMock(return_value=True)
|
||||
mock_get_service.return_value = mock_service
|
||||
|
||||
# Should not raise
|
||||
manager._schedule_completion_email(mock_job)
|
||||
|
||||
def test_handles_exception_gracefully(self):
|
||||
"""Test that exceptions don't propagate."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = {}
|
||||
|
||||
with patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_service:
|
||||
mock_get_service.side_effect = Exception("Service error")
|
||||
|
||||
# Should not raise
|
||||
manager._schedule_completion_email(mock_job)
|
||||
|
||||
|
||||
class TestScheduleIdleReminder:
|
||||
"""Tests for _schedule_idle_reminder method."""
|
||||
|
||||
def test_updates_state_data_for_lyrics_review(self):
|
||||
"""Test that state_data is updated with blocking state info for lyrics."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = {"existing_key": "value"}
|
||||
|
||||
with patch('backend.services.worker_service.get_worker_service') as mock_get_service:
|
||||
mock_service = Mock()
|
||||
mock_service.schedule_idle_reminder = AsyncMock(return_value=True)
|
||||
mock_get_service.return_value = mock_service
|
||||
|
||||
manager._schedule_idle_reminder(mock_job, JobStatus.AWAITING_REVIEW)
|
||||
|
||||
# Verify state_data was updated
|
||||
update_call = manager.firestore.update_job.call_args
|
||||
assert update_call is not None
|
||||
updated_data = update_call[0][1] # Second positional arg
|
||||
state_data = updated_data.get('state_data', {})
|
||||
|
||||
assert state_data.get('blocking_action_type') == 'lyrics'
|
||||
assert state_data.get('reminder_sent') is False
|
||||
assert 'blocking_state_entered_at' in state_data
|
||||
assert state_data.get('existing_key') == 'value' # Preserved
|
||||
|
||||
def test_updates_state_data_for_instrumental_selection(self):
|
||||
"""Test that state_data is updated with blocking state info for instrumental."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = {}
|
||||
|
||||
with patch('backend.services.worker_service.get_worker_service') as mock_get_service:
|
||||
mock_service = Mock()
|
||||
mock_service.schedule_idle_reminder = AsyncMock(return_value=True)
|
||||
mock_get_service.return_value = mock_service
|
||||
|
||||
manager._schedule_idle_reminder(
|
||||
mock_job,
|
||||
JobStatus.AWAITING_INSTRUMENTAL_SELECTION
|
||||
)
|
||||
|
||||
# Verify state_data was updated
|
||||
update_call = manager.firestore.update_job.call_args
|
||||
updated_data = update_call[0][1]
|
||||
state_data = updated_data.get('state_data', {})
|
||||
|
||||
assert state_data.get('blocking_action_type') == 'instrumental'
|
||||
|
||||
def test_handles_none_state_data(self):
|
||||
"""Test that None state_data is handled gracefully."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = None
|
||||
|
||||
with patch('backend.services.worker_service.get_worker_service') as mock_get_service:
|
||||
mock_service = Mock()
|
||||
mock_service.schedule_idle_reminder = AsyncMock(return_value=True)
|
||||
mock_get_service.return_value = mock_service
|
||||
|
||||
# Should not raise
|
||||
manager._schedule_idle_reminder(mock_job, JobStatus.AWAITING_REVIEW)
|
||||
|
||||
# Verify update was still called
|
||||
manager.firestore.update_job.assert_called_once()
|
||||
|
||||
def test_handles_exception_gracefully(self):
|
||||
"""Test that exceptions don't propagate."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = {}
|
||||
|
||||
# Make firestore raise an exception
|
||||
manager.firestore.update_job.side_effect = Exception("Database error")
|
||||
|
||||
# Should not raise
|
||||
manager._schedule_idle_reminder(mock_job, JobStatus.AWAITING_REVIEW)
|
||||
|
||||
|
||||
class TestTransitionTriggersNotifications:
|
||||
"""Integration tests for transition_to_state triggering notifications."""
|
||||
|
||||
def test_transition_to_complete_triggers_notification(self):
|
||||
"""Test that transitioning to COMPLETE triggers notification."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
# Use ENCODING as the starting state since ENCODING -> COMPLETE is valid
|
||||
mock_job.status = JobStatus.ENCODING
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = {}
|
||||
manager.get_job = Mock(return_value=mock_job)
|
||||
|
||||
with patch.object(manager, '_trigger_state_notifications') as mock_trigger:
|
||||
manager.transition_to_state("job-123", JobStatus.COMPLETE)
|
||||
|
||||
mock_trigger.assert_called_once_with("job-123", JobStatus.COMPLETE)
|
||||
|
||||
def test_transition_to_awaiting_review_triggers_notification(self):
|
||||
"""Test that transitioning to AWAITING_REVIEW triggers notification."""
|
||||
manager = JobManager()
|
||||
manager.firestore = Mock()
|
||||
|
||||
mock_job = Mock(spec=Job)
|
||||
mock_job.job_id = "job-123"
|
||||
# Use APPLYING_PADDING as the starting state since APPLYING_PADDING -> AWAITING_REVIEW is valid
|
||||
mock_job.status = JobStatus.APPLYING_PADDING
|
||||
mock_job.user_email = "user@example.com"
|
||||
mock_job.state_data = {}
|
||||
manager.get_job = Mock(return_value=mock_job)
|
||||
|
||||
with patch.object(manager, '_trigger_state_notifications') as mock_trigger:
|
||||
manager.transition_to_state("job-123", JobStatus.AWAITING_REVIEW)
|
||||
|
||||
mock_trigger.assert_called_once_with("job-123", JobStatus.AWAITING_REVIEW)
|
||||
@@ -0,0 +1,443 @@
|
||||
"""
|
||||
Unit tests for job notification service.
|
||||
"""
|
||||
import pytest
|
||||
from unittest.mock import Mock, patch, AsyncMock
|
||||
import urllib.parse
|
||||
|
||||
from backend.services.job_notification_service import (
|
||||
JobNotificationService,
|
||||
get_job_notification_service,
|
||||
ENABLE_AUTO_EMAILS,
|
||||
FEEDBACK_FORM_URL,
|
||||
)
|
||||
|
||||
|
||||
class TestURLBuilding:
|
||||
"""Tests for URL building methods."""
|
||||
|
||||
def test_build_review_url_basic(self):
|
||||
"""Test basic review URL building."""
|
||||
service = JobNotificationService()
|
||||
service.frontend_url = "https://gen.nomadkaraoke.com"
|
||||
service.backend_url = "https://api.nomadkaraoke.com"
|
||||
|
||||
url = service._build_review_url("job-123")
|
||||
|
||||
assert "gen.nomadkaraoke.com/lyrics/" in url
|
||||
assert "baseApiUrl=" in url
|
||||
# The API URL should be URL-encoded
|
||||
assert urllib.parse.quote("https://api.nomadkaraoke.com/api/review/job-123", safe='') in url
|
||||
|
||||
def test_build_review_url_with_audio_hash(self):
|
||||
"""Test review URL with audio hash parameter."""
|
||||
service = JobNotificationService()
|
||||
service.frontend_url = "https://gen.nomadkaraoke.com"
|
||||
service.backend_url = "https://api.nomadkaraoke.com"
|
||||
|
||||
url = service._build_review_url("job-123", audio_hash="abc123")
|
||||
|
||||
assert "audioHash=abc123" in url
|
||||
|
||||
def test_build_review_url_with_review_token(self):
|
||||
"""Test review URL with review token parameter."""
|
||||
service = JobNotificationService()
|
||||
service.frontend_url = "https://gen.nomadkaraoke.com"
|
||||
service.backend_url = "https://api.nomadkaraoke.com"
|
||||
|
||||
url = service._build_review_url("job-123", review_token="token456")
|
||||
|
||||
assert "reviewToken=token456" in url
|
||||
|
||||
def test_build_review_url_with_all_params(self):
|
||||
"""Test review URL with all parameters."""
|
||||
service = JobNotificationService()
|
||||
service.frontend_url = "https://gen.nomadkaraoke.com"
|
||||
service.backend_url = "https://api.nomadkaraoke.com"
|
||||
|
||||
url = service._build_review_url(
|
||||
"job-123",
|
||||
audio_hash="hash789",
|
||||
review_token="token456"
|
||||
)
|
||||
|
||||
assert "baseApiUrl=" in url
|
||||
assert "audioHash=hash789" in url
|
||||
assert "reviewToken=token456" in url
|
||||
|
||||
def test_build_review_url_encodes_special_chars(self):
|
||||
"""Test that special characters in job ID are encoded."""
|
||||
service = JobNotificationService()
|
||||
service.frontend_url = "https://gen.nomadkaraoke.com"
|
||||
service.backend_url = "https://api.nomadkaraoke.com"
|
||||
|
||||
url = service._build_review_url("job/with/slashes")
|
||||
|
||||
# The baseApiUrl parameter should have encoded slashes
|
||||
assert "%2F" in url
|
||||
|
||||
def test_build_instrumental_url_basic(self):
|
||||
"""Test basic instrumental URL building."""
|
||||
service = JobNotificationService()
|
||||
service.frontend_url = "https://gen.nomadkaraoke.com"
|
||||
service.backend_url = "https://api.nomadkaraoke.com"
|
||||
|
||||
url = service._build_instrumental_url("job-123")
|
||||
|
||||
assert "gen.nomadkaraoke.com/instrumental/" in url
|
||||
assert "baseApiUrl=" in url
|
||||
|
||||
def test_build_instrumental_url_with_token(self):
|
||||
"""Test instrumental URL with token parameter."""
|
||||
service = JobNotificationService()
|
||||
service.frontend_url = "https://gen.nomadkaraoke.com"
|
||||
service.backend_url = "https://api.nomadkaraoke.com"
|
||||
|
||||
url = service._build_instrumental_url("job-123", instrumental_token="inst-token")
|
||||
|
||||
assert "instrumentalToken=inst-token" in url
|
||||
|
||||
|
||||
class TestCompletionEmail:
|
||||
"""Tests for job completion email sending."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_completion_email_success(self):
|
||||
"""Test successful completion email sending."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
service.email_service.send_job_completion.return_value = True
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_job_completion.return_value = "Test message"
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
result = await service.send_job_completion_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
user_name="Test User",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
youtube_url="https://youtube.com/watch?v=123",
|
||||
dropbox_url="https://dropbox.com/folder/abc",
|
||||
)
|
||||
|
||||
assert result is True
|
||||
service.template_service.render_job_completion.assert_called_once()
|
||||
service.email_service.send_job_completion.assert_called_once()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_completion_email_disabled(self):
|
||||
"""Test that completion email is skipped when auto emails are disabled."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
service.template_service = Mock()
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', False):
|
||||
result = await service.send_job_completion_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
)
|
||||
|
||||
assert result is False
|
||||
service.email_service.send_job_completion.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_completion_email_no_email(self):
|
||||
"""Test that completion email is skipped when no user email."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
service.template_service = Mock()
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
result = await service.send_job_completion_email(
|
||||
job_id="job-123",
|
||||
user_email=None,
|
||||
)
|
||||
|
||||
assert result is False
|
||||
service.email_service.send_job_completion.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_completion_email_empty_email(self):
|
||||
"""Test that completion email is skipped when user email is empty."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
service.template_service = Mock()
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
result = await service.send_job_completion_email(
|
||||
job_id="job-123",
|
||||
user_email="",
|
||||
)
|
||||
|
||||
assert result is False
|
||||
service.email_service.send_job_completion.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_completion_email_send_failure(self):
|
||||
"""Test handling of email send failure."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
service.email_service.send_job_completion.return_value = False
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_job_completion.return_value = "Test message"
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
result = await service.send_job_completion_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
)
|
||||
|
||||
assert result is False
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_completion_email_exception(self):
|
||||
"""Test handling of exceptions during email sending."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
service.email_service.send_job_completion.side_effect = Exception("Send error")
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_job_completion.return_value = "Test message"
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
result = await service.send_job_completion_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
)
|
||||
|
||||
assert result is False
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_completion_email_passes_cc_admin(self):
|
||||
"""Test that completion email is sent with CC to admin."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
service.email_service.send_job_completion.return_value = True
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_job_completion.return_value = "Test message"
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
await service.send_job_completion_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
)
|
||||
|
||||
# Verify cc_admin=True was passed
|
||||
call_kwargs = service.email_service.send_job_completion.call_args.kwargs
|
||||
assert call_kwargs.get('cc_admin') is True
|
||||
|
||||
|
||||
class TestActionReminderEmail:
|
||||
"""Tests for action reminder email sending."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_lyrics_reminder_success(self):
|
||||
"""Test successful lyrics reminder email sending."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
service.email_service.send_action_reminder.return_value = True
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_action_needed_lyrics.return_value = "Review your lyrics"
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
result = await service.send_action_reminder_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
action_type="lyrics",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
)
|
||||
|
||||
assert result is True
|
||||
service.template_service.render_action_needed_lyrics.assert_called_once()
|
||||
service.email_service.send_action_reminder.assert_called_once()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_instrumental_reminder_success(self):
|
||||
"""Test successful instrumental reminder email sending."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
service.email_service.send_action_reminder.return_value = True
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_action_needed_instrumental.return_value = "Select instrumental"
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
result = await service.send_action_reminder_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
action_type="instrumental",
|
||||
)
|
||||
|
||||
assert result is True
|
||||
service.template_service.render_action_needed_instrumental.assert_called_once()
|
||||
service.email_service.send_action_reminder.assert_called_once()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_reminder_disabled(self):
|
||||
"""Test that reminder email is skipped when auto emails are disabled."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', False):
|
||||
result = await service.send_action_reminder_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
action_type="lyrics",
|
||||
)
|
||||
|
||||
assert result is False
|
||||
service.email_service.send_action_reminder.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_reminder_no_email(self):
|
||||
"""Test that reminder email is skipped when no user email."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
result = await service.send_action_reminder_email(
|
||||
job_id="job-123",
|
||||
user_email=None,
|
||||
action_type="lyrics",
|
||||
)
|
||||
|
||||
assert result is False
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_reminder_unknown_action_type(self):
|
||||
"""Test handling of unknown action type."""
|
||||
service = JobNotificationService()
|
||||
service.email_service = Mock()
|
||||
service.template_service = Mock()
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
result = await service.send_action_reminder_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
action_type="unknown",
|
||||
)
|
||||
|
||||
assert result is False
|
||||
service.email_service.send_action_reminder.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_lyrics_reminder_includes_review_url(self):
|
||||
"""Test that lyrics reminder includes correct review URL."""
|
||||
service = JobNotificationService()
|
||||
service.frontend_url = "https://gen.nomadkaraoke.com"
|
||||
service.backend_url = "https://api.nomadkaraoke.com"
|
||||
service.email_service = Mock()
|
||||
service.email_service.send_action_reminder.return_value = True
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_action_needed_lyrics.return_value = "Review"
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
await service.send_action_reminder_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
action_type="lyrics",
|
||||
audio_hash="hash123",
|
||||
review_token="token456",
|
||||
)
|
||||
|
||||
# Verify the review URL was passed to template
|
||||
call_kwargs = service.template_service.render_action_needed_lyrics.call_args.kwargs
|
||||
review_url = call_kwargs.get('review_url')
|
||||
assert "audioHash=hash123" in review_url
|
||||
assert "reviewToken=token456" in review_url
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_instrumental_reminder_includes_url(self):
|
||||
"""Test that instrumental reminder includes correct URL."""
|
||||
service = JobNotificationService()
|
||||
service.frontend_url = "https://gen.nomadkaraoke.com"
|
||||
service.backend_url = "https://api.nomadkaraoke.com"
|
||||
service.email_service = Mock()
|
||||
service.email_service.send_action_reminder.return_value = True
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_action_needed_instrumental.return_value = "Select"
|
||||
|
||||
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
|
||||
await service.send_action_reminder_email(
|
||||
job_id="job-123",
|
||||
user_email="user@example.com",
|
||||
action_type="instrumental",
|
||||
instrumental_token="inst-token",
|
||||
)
|
||||
|
||||
# Verify the instrumental URL was passed to template
|
||||
call_kwargs = service.template_service.render_action_needed_instrumental.call_args.kwargs
|
||||
instrumental_url = call_kwargs.get('instrumental_url')
|
||||
assert "instrumentalToken=inst-token" in instrumental_url
|
||||
|
||||
|
||||
class TestGetCompletionMessage:
|
||||
"""Tests for get_completion_message method."""
|
||||
|
||||
def test_get_completion_message_basic(self):
|
||||
"""Test basic completion message retrieval."""
|
||||
service = JobNotificationService()
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_job_completion.return_value = "Your video is ready!"
|
||||
|
||||
result = service.get_completion_message(
|
||||
job_id="job-123",
|
||||
user_name="Test User",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
youtube_url="https://youtube.com/watch?v=123",
|
||||
dropbox_url="https://dropbox.com/folder/abc",
|
||||
)
|
||||
|
||||
assert result == "Your video is ready!"
|
||||
service.template_service.render_job_completion.assert_called_once()
|
||||
|
||||
def test_get_completion_message_includes_feedback_url(self):
|
||||
"""Test that completion message includes feedback URL."""
|
||||
service = JobNotificationService()
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_job_completion.return_value = "Message"
|
||||
|
||||
service.get_completion_message(job_id="job-123")
|
||||
|
||||
# Verify feedback URL was passed
|
||||
call_kwargs = service.template_service.render_job_completion.call_args.kwargs
|
||||
assert 'feedback_url' in call_kwargs
|
||||
|
||||
def test_get_completion_message_with_defaults(self):
|
||||
"""Test completion message with default values."""
|
||||
service = JobNotificationService()
|
||||
service.template_service = Mock()
|
||||
service.template_service.render_job_completion.return_value = "Message"
|
||||
|
||||
service.get_completion_message(job_id="job-123")
|
||||
|
||||
call_kwargs = service.template_service.render_job_completion.call_args.kwargs
|
||||
assert call_kwargs.get('job_id') == "job-123"
|
||||
# Other params should be None/default
|
||||
assert call_kwargs.get('name') is None
|
||||
assert call_kwargs.get('artist') is None
|
||||
|
||||
|
||||
class TestGlobalInstance:
|
||||
"""Tests for global instance management."""
|
||||
|
||||
def test_get_job_notification_service_returns_same_instance(self):
|
||||
"""Test that get_job_notification_service returns singleton."""
|
||||
# Reset global
|
||||
import backend.services.job_notification_service as jns
|
||||
jns._job_notification_service = None
|
||||
|
||||
service1 = get_job_notification_service()
|
||||
service2 = get_job_notification_service()
|
||||
|
||||
assert service1 is service2
|
||||
|
||||
def test_service_initializes_with_dependencies(self):
|
||||
"""Test that service initializes with email and template services."""
|
||||
service = JobNotificationService()
|
||||
|
||||
assert service.email_service is not None
|
||||
assert service.template_service is not None
|
||||
assert service.frontend_url is not None
|
||||
assert service.backend_url is not None
|
||||
@@ -0,0 +1,567 @@
|
||||
"""
|
||||
Tests for LocalPreviewEncodingService.
|
||||
|
||||
Tests cover:
|
||||
- Service initialization
|
||||
- Hardware acceleration detection (NVENC)
|
||||
- FFmpeg filter path escaping
|
||||
- ASS filter building
|
||||
- Preview FFmpeg command construction
|
||||
- Preview encoding execution
|
||||
- Error handling
|
||||
"""
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
import pytest
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from backend.services.local_preview_encoding_service import (
|
||||
LocalPreviewEncodingService,
|
||||
PreviewEncodingConfig,
|
||||
PreviewEncodingResult,
|
||||
get_local_preview_encoding_service,
|
||||
)
|
||||
|
||||
|
||||
class TestLocalPreviewEncodingServiceInit:
|
||||
"""Test service initialization."""
|
||||
|
||||
def test_init_default_values(self):
|
||||
"""Test default initialization."""
|
||||
service = LocalPreviewEncodingService()
|
||||
assert service._nvenc_available is None # Lazy detection
|
||||
assert service._video_encoder is None # Lazy detection
|
||||
assert service._hwaccel_flags is None # Lazy detection
|
||||
|
||||
def test_init_with_custom_logger(self):
|
||||
"""Test initialization with custom logger."""
|
||||
import logging
|
||||
custom_logger = logging.getLogger("test")
|
||||
service = LocalPreviewEncodingService(logger=custom_logger)
|
||||
assert service.logger is custom_logger
|
||||
|
||||
|
||||
class TestLocalPreviewEncodingServiceHWAccel:
|
||||
"""Test hardware acceleration detection."""
|
||||
|
||||
@patch("subprocess.run")
|
||||
def test_detect_nvenc_available(self, mock_run):
|
||||
"""Test NVENC detection when available."""
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._detect_nvenc_support()
|
||||
|
||||
assert result is True
|
||||
|
||||
@patch("subprocess.run")
|
||||
def test_detect_nvenc_not_available(self, mock_run):
|
||||
"""Test fallback when NVENC not available."""
|
||||
import subprocess
|
||||
mock_run.side_effect = subprocess.CalledProcessError(1, "test")
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._detect_nvenc_support()
|
||||
|
||||
assert result is False
|
||||
|
||||
@patch("subprocess.run")
|
||||
def test_detect_nvenc_timeout(self, mock_run):
|
||||
"""Test NVENC detection timeout handling."""
|
||||
import subprocess
|
||||
mock_run.side_effect = subprocess.TimeoutExpired("test", 30)
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._detect_nvenc_support()
|
||||
|
||||
assert result is False
|
||||
|
||||
@patch.object(LocalPreviewEncodingService, "_detect_nvenc_support")
|
||||
def test_configure_nvenc_available(self, mock_detect):
|
||||
"""Test configuration when NVENC is available."""
|
||||
mock_detect.return_value = True
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
service._configure_hardware_acceleration()
|
||||
|
||||
assert service._nvenc_available is True
|
||||
assert service._video_encoder == "h264_nvenc"
|
||||
assert "-hwaccel" in service._hwaccel_flags
|
||||
|
||||
@patch.object(LocalPreviewEncodingService, "_detect_nvenc_support")
|
||||
def test_configure_no_nvenc(self, mock_detect):
|
||||
"""Test configuration when NVENC not available."""
|
||||
mock_detect.return_value = False
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
service._configure_hardware_acceleration()
|
||||
|
||||
assert service._nvenc_available is False
|
||||
assert service._video_encoder == "libx264"
|
||||
assert service._hwaccel_flags == []
|
||||
|
||||
|
||||
class TestLocalPreviewEncodingServiceProperties:
|
||||
"""Test property accessors that trigger lazy detection."""
|
||||
|
||||
@patch.object(LocalPreviewEncodingService, "_configure_hardware_acceleration")
|
||||
def test_nvenc_available_triggers_detection(self, mock_config):
|
||||
"""Test that nvenc_available property triggers detection."""
|
||||
service = LocalPreviewEncodingService()
|
||||
service._nvenc_available = None
|
||||
|
||||
# Access property
|
||||
_ = service.nvenc_available
|
||||
|
||||
mock_config.assert_called_once()
|
||||
|
||||
@patch.object(LocalPreviewEncodingService, "_configure_hardware_acceleration")
|
||||
def test_video_encoder_triggers_detection(self, mock_config):
|
||||
"""Test that video_encoder property triggers detection."""
|
||||
service = LocalPreviewEncodingService()
|
||||
service._video_encoder = None
|
||||
|
||||
# Access property
|
||||
_ = service.video_encoder
|
||||
|
||||
mock_config.assert_called_once()
|
||||
|
||||
@patch.object(LocalPreviewEncodingService, "_configure_hardware_acceleration")
|
||||
def test_hwaccel_flags_triggers_detection(self, mock_config):
|
||||
"""Test that hwaccel_flags property triggers detection."""
|
||||
service = LocalPreviewEncodingService()
|
||||
service._hwaccel_flags = None
|
||||
|
||||
# Access property
|
||||
_ = service.hwaccel_flags
|
||||
|
||||
mock_config.assert_called_once()
|
||||
|
||||
|
||||
class TestLocalPreviewEncodingServicePathEscaping:
|
||||
"""Test FFmpeg filter path escaping."""
|
||||
|
||||
def test_escape_simple_path(self):
|
||||
"""Test escaping a simple path."""
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._escape_ffmpeg_filter_path("/simple/path.ass")
|
||||
assert result == "/simple/path.ass"
|
||||
|
||||
def test_escape_path_with_spaces(self):
|
||||
"""Test escaping path with spaces."""
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._escape_ffmpeg_filter_path("/path/with spaces/file.ass")
|
||||
assert "\\ " in result # Spaces should be escaped
|
||||
|
||||
def test_escape_path_with_apostrophe(self):
|
||||
"""Test escaping path with apostrophe (common in song titles)."""
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._escape_ffmpeg_filter_path("/path/I'm With You/file.ass")
|
||||
assert "\\\\\\'" in result # Apostrophe should be triple-backslash escaped
|
||||
|
||||
def test_escape_path_with_special_chars(self):
|
||||
"""Test escaping path with FFmpeg special characters."""
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._escape_ffmpeg_filter_path("/path:with[special];chars,here.ass")
|
||||
# Should escape :,[];
|
||||
assert "\\:" in result
|
||||
assert "\\[" in result
|
||||
assert "\\]" in result
|
||||
assert "\\;" in result
|
||||
assert "\\," in result
|
||||
|
||||
def test_escape_path_with_backslashes(self):
|
||||
"""Test escaping path with existing backslashes."""
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._escape_ffmpeg_filter_path("/path\\with\\backslashes.ass")
|
||||
assert "\\\\" in result
|
||||
|
||||
|
||||
class TestLocalPreviewEncodingServiceASSFilter:
|
||||
"""Test ASS filter building."""
|
||||
|
||||
def test_build_ass_filter_simple(self):
|
||||
"""Test building ASS filter without font."""
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._build_ass_filter("/path/to/file.ass")
|
||||
assert result.startswith("ass=")
|
||||
assert "fontsdir" not in result
|
||||
|
||||
def test_build_ass_filter_with_font(self):
|
||||
"""Test building ASS filter with custom font."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
font_path = os.path.join(tmpdir, "custom.ttf")
|
||||
with open(font_path, "w") as f:
|
||||
f.write("fake font")
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._build_ass_filter("/path/to/file.ass", font_path)
|
||||
|
||||
assert "fontsdir=" in result
|
||||
|
||||
def test_build_ass_filter_nonexistent_font(self):
|
||||
"""Test building ASS filter with nonexistent font file."""
|
||||
service = LocalPreviewEncodingService()
|
||||
result = service._build_ass_filter("/path/to/file.ass", "/nonexistent/font.ttf")
|
||||
# Should not include fontsdir for nonexistent font
|
||||
assert "fontsdir" not in result
|
||||
|
||||
|
||||
class TestLocalPreviewEncodingServiceFFmpegCommand:
|
||||
"""Test FFmpeg command building."""
|
||||
|
||||
def test_build_command_solid_background_libx264(self):
|
||||
"""Test command building with solid background and libx264."""
|
||||
service = LocalPreviewEncodingService()
|
||||
service._nvenc_available = False
|
||||
service._video_encoder = "libx264"
|
||||
service._hwaccel_flags = []
|
||||
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path="/path/to/subs.ass",
|
||||
audio_path="/path/to/audio.flac",
|
||||
output_path="/path/to/output.mp4",
|
||||
background_color="black"
|
||||
)
|
||||
|
||||
cmd = service._build_preview_ffmpeg_command(config)
|
||||
|
||||
assert "ffmpeg" in cmd
|
||||
assert "-r" in cmd and "24" in cmd # Frame rate
|
||||
assert "-c:v" in cmd
|
||||
assert "libx264" in cmd
|
||||
assert "-preset" in cmd and "superfast" in cmd
|
||||
assert "-crf" in cmd and "28" in cmd
|
||||
assert "color=c=black" in " ".join(cmd)
|
||||
|
||||
def test_build_command_with_nvenc(self):
|
||||
"""Test command building with NVENC hardware acceleration."""
|
||||
service = LocalPreviewEncodingService()
|
||||
service._nvenc_available = True
|
||||
service._video_encoder = "h264_nvenc"
|
||||
service._hwaccel_flags = ["-hwaccel", "cuda", "-hwaccel_output_format", "cuda"]
|
||||
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path="/path/to/subs.ass",
|
||||
audio_path="/path/to/audio.flac",
|
||||
output_path="/path/to/output.mp4"
|
||||
)
|
||||
|
||||
cmd = service._build_preview_ffmpeg_command(config)
|
||||
|
||||
assert "h264_nvenc" in cmd
|
||||
assert "-hwaccel" in cmd
|
||||
assert "-preset" in cmd and "p1" in cmd # Fastest NVENC preset
|
||||
|
||||
def test_build_command_with_background_image(self):
|
||||
"""Test command building with background image."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
bg_path = os.path.join(tmpdir, "background.png")
|
||||
with open(bg_path, "wb") as f:
|
||||
f.write(b"fake png")
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
service._nvenc_available = False
|
||||
service._video_encoder = "libx264"
|
||||
service._hwaccel_flags = []
|
||||
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path="/path/to/subs.ass",
|
||||
audio_path="/path/to/audio.flac",
|
||||
output_path="/path/to/output.mp4",
|
||||
background_image_path=bg_path
|
||||
)
|
||||
|
||||
cmd = service._build_preview_ffmpeg_command(config)
|
||||
|
||||
assert "-loop" in cmd
|
||||
assert "1" in cmd
|
||||
assert bg_path in cmd
|
||||
# Should have scale filter for background image
|
||||
vf_index = cmd.index("-vf")
|
||||
vf_value = cmd[vf_index + 1]
|
||||
assert "scale=" in vf_value
|
||||
assert "pad=" in vf_value
|
||||
|
||||
def test_build_command_resolution(self):
|
||||
"""Test that command uses correct preview resolution."""
|
||||
service = LocalPreviewEncodingService()
|
||||
service._nvenc_available = False
|
||||
service._video_encoder = "libx264"
|
||||
service._hwaccel_flags = []
|
||||
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path="/path/to/subs.ass",
|
||||
audio_path="/path/to/audio.flac",
|
||||
output_path="/path/to/output.mp4"
|
||||
)
|
||||
|
||||
cmd = service._build_preview_ffmpeg_command(config)
|
||||
cmd_str = " ".join(cmd)
|
||||
|
||||
# Resolution should be 480x270
|
||||
assert "480" in cmd_str
|
||||
assert "270" in cmd_str
|
||||
|
||||
def test_build_command_audio_settings(self):
|
||||
"""Test that command uses correct audio settings."""
|
||||
service = LocalPreviewEncodingService()
|
||||
service._nvenc_available = False
|
||||
service._video_encoder = "libx264"
|
||||
service._hwaccel_flags = []
|
||||
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path="/path/to/subs.ass",
|
||||
audio_path="/path/to/audio.flac",
|
||||
output_path="/path/to/output.mp4"
|
||||
)
|
||||
|
||||
cmd = service._build_preview_ffmpeg_command(config)
|
||||
|
||||
assert "-c:a" in cmd
|
||||
assert "aac" in cmd
|
||||
assert "-b:a" in cmd
|
||||
assert "96k" in cmd
|
||||
|
||||
|
||||
class TestLocalPreviewEncodingServiceEncode:
|
||||
"""Test preview encoding execution."""
|
||||
|
||||
@patch("subprocess.run")
|
||||
def test_encode_preview_success(self, mock_run):
|
||||
"""Test successful preview encoding."""
|
||||
mock_run.return_value = MagicMock(returncode=0, stderr="")
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
ass_path = os.path.join(tmpdir, "subs.ass")
|
||||
audio_path = os.path.join(tmpdir, "audio.flac")
|
||||
output_path = os.path.join(tmpdir, "output.mp4")
|
||||
|
||||
# Create input files
|
||||
with open(ass_path, "w") as f:
|
||||
f.write("[Script Info]\n")
|
||||
with open(audio_path, "wb") as f:
|
||||
f.write(b"fake audio")
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
service._nvenc_available = False
|
||||
service._video_encoder = "libx264"
|
||||
service._hwaccel_flags = []
|
||||
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path=ass_path,
|
||||
audio_path=audio_path,
|
||||
output_path=output_path
|
||||
)
|
||||
|
||||
result = service.encode_preview(config)
|
||||
|
||||
assert result.success is True
|
||||
assert result.output_path == output_path
|
||||
assert result.error is None
|
||||
mock_run.assert_called_once()
|
||||
|
||||
@patch("subprocess.run")
|
||||
def test_encode_preview_ffmpeg_failure(self, mock_run):
|
||||
"""Test preview encoding with FFmpeg failure."""
|
||||
mock_run.return_value = MagicMock(returncode=1, stderr="FFmpeg error message")
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
ass_path = os.path.join(tmpdir, "subs.ass")
|
||||
audio_path = os.path.join(tmpdir, "audio.flac")
|
||||
output_path = os.path.join(tmpdir, "output.mp4")
|
||||
|
||||
with open(ass_path, "w") as f:
|
||||
f.write("[Script Info]\n")
|
||||
with open(audio_path, "wb") as f:
|
||||
f.write(b"fake audio")
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
service._nvenc_available = False
|
||||
service._video_encoder = "libx264"
|
||||
service._hwaccel_flags = []
|
||||
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path=ass_path,
|
||||
audio_path=audio_path,
|
||||
output_path=output_path
|
||||
)
|
||||
|
||||
result = service.encode_preview(config)
|
||||
|
||||
assert result.success is False
|
||||
assert "FFmpeg preview encoding failed" in result.error
|
||||
|
||||
def test_encode_preview_missing_ass_file(self):
|
||||
"""Test preview encoding with missing ASS file."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
audio_path = os.path.join(tmpdir, "audio.flac")
|
||||
output_path = os.path.join(tmpdir, "output.mp4")
|
||||
|
||||
with open(audio_path, "wb") as f:
|
||||
f.write(b"fake audio")
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path="/nonexistent/subs.ass",
|
||||
audio_path=audio_path,
|
||||
output_path=output_path
|
||||
)
|
||||
|
||||
result = service.encode_preview(config)
|
||||
|
||||
assert result.success is False
|
||||
assert "not found" in result.error
|
||||
|
||||
def test_encode_preview_missing_audio_file(self):
|
||||
"""Test preview encoding with missing audio file."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
ass_path = os.path.join(tmpdir, "subs.ass")
|
||||
output_path = os.path.join(tmpdir, "output.mp4")
|
||||
|
||||
with open(ass_path, "w") as f:
|
||||
f.write("[Script Info]\n")
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path=ass_path,
|
||||
audio_path="/nonexistent/audio.flac",
|
||||
output_path=output_path
|
||||
)
|
||||
|
||||
result = service.encode_preview(config)
|
||||
|
||||
assert result.success is False
|
||||
assert "not found" in result.error
|
||||
|
||||
@patch("subprocess.run")
|
||||
def test_encode_preview_timeout(self, mock_run):
|
||||
"""Test preview encoding timeout."""
|
||||
import subprocess
|
||||
mock_run.side_effect = subprocess.TimeoutExpired("ffmpeg", 300)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
ass_path = os.path.join(tmpdir, "subs.ass")
|
||||
audio_path = os.path.join(tmpdir, "audio.flac")
|
||||
output_path = os.path.join(tmpdir, "output.mp4")
|
||||
|
||||
with open(ass_path, "w") as f:
|
||||
f.write("[Script Info]\n")
|
||||
with open(audio_path, "wb") as f:
|
||||
f.write(b"fake audio")
|
||||
|
||||
service = LocalPreviewEncodingService()
|
||||
service._nvenc_available = False
|
||||
service._video_encoder = "libx264"
|
||||
service._hwaccel_flags = []
|
||||
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path=ass_path,
|
||||
audio_path=audio_path,
|
||||
output_path=output_path
|
||||
)
|
||||
|
||||
result = service.encode_preview(config)
|
||||
|
||||
assert result.success is False
|
||||
assert "timed out" in result.error
|
||||
|
||||
|
||||
class TestPreviewEncodingConfig:
|
||||
"""Test PreviewEncodingConfig dataclass."""
|
||||
|
||||
def test_config_required_fields(self):
|
||||
"""Test that required fields must be provided."""
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path="/path/to/subs.ass",
|
||||
audio_path="/path/to/audio.flac",
|
||||
output_path="/path/to/output.mp4"
|
||||
)
|
||||
assert config.ass_path == "/path/to/subs.ass"
|
||||
assert config.background_color == "black" # Default
|
||||
assert config.background_image_path is None # Optional
|
||||
|
||||
def test_config_all_fields(self):
|
||||
"""Test config with all fields."""
|
||||
config = PreviewEncodingConfig(
|
||||
ass_path="/path/to/subs.ass",
|
||||
audio_path="/path/to/audio.flac",
|
||||
output_path="/path/to/output.mp4",
|
||||
background_image_path="/path/to/bg.png",
|
||||
background_color="red",
|
||||
font_path="/path/to/font.ttf"
|
||||
)
|
||||
assert config.background_image_path == "/path/to/bg.png"
|
||||
assert config.background_color == "red"
|
||||
assert config.font_path == "/path/to/font.ttf"
|
||||
|
||||
|
||||
class TestPreviewEncodingResult:
|
||||
"""Test PreviewEncodingResult dataclass."""
|
||||
|
||||
def test_result_success(self):
|
||||
"""Test successful result."""
|
||||
result = PreviewEncodingResult(
|
||||
success=True,
|
||||
output_path="/path/to/output.mp4"
|
||||
)
|
||||
assert result.success is True
|
||||
assert result.error is None
|
||||
assert result.output_path == "/path/to/output.mp4"
|
||||
|
||||
def test_result_failure(self):
|
||||
"""Test failure result."""
|
||||
result = PreviewEncodingResult(
|
||||
success=False,
|
||||
error="Something went wrong"
|
||||
)
|
||||
assert result.success is False
|
||||
assert result.error == "Something went wrong"
|
||||
assert result.output_path is None
|
||||
|
||||
|
||||
class TestGetLocalPreviewEncodingService:
|
||||
"""Test factory function."""
|
||||
|
||||
def test_get_service_creates_instance(self):
|
||||
"""Test that factory function creates a new instance."""
|
||||
import backend.services.local_preview_encoding_service as module
|
||||
module._local_preview_encoding_service = None
|
||||
|
||||
service = get_local_preview_encoding_service()
|
||||
|
||||
assert service is not None
|
||||
assert isinstance(service, LocalPreviewEncodingService)
|
||||
|
||||
def test_get_service_returns_singleton(self):
|
||||
"""Test that factory function returns the same instance."""
|
||||
import backend.services.local_preview_encoding_service as module
|
||||
module._local_preview_encoding_service = None
|
||||
|
||||
service1 = get_local_preview_encoding_service()
|
||||
service2 = get_local_preview_encoding_service()
|
||||
|
||||
assert service1 is service2
|
||||
|
||||
|
||||
class TestPreviewEncodingConstants:
|
||||
"""Test that encoding constants match expected values."""
|
||||
|
||||
def test_preview_resolution(self):
|
||||
"""Test preview resolution constants."""
|
||||
service = LocalPreviewEncodingService()
|
||||
assert service.PREVIEW_WIDTH == 480
|
||||
assert service.PREVIEW_HEIGHT == 270
|
||||
|
||||
def test_preview_fps(self):
|
||||
"""Test preview frame rate constant."""
|
||||
service = LocalPreviewEncodingService()
|
||||
assert service.PREVIEW_FPS == 24
|
||||
|
||||
def test_preview_audio_bitrate(self):
|
||||
"""Test preview audio bitrate constant."""
|
||||
service = LocalPreviewEncodingService()
|
||||
assert service.PREVIEW_AUDIO_BITRATE == "96k"
|
||||
@@ -149,7 +149,8 @@ class TestUploadSubmissionRequest:
|
||||
# CDG/TXT disabled by default (requires style config)
|
||||
assert request.enable_cdg is False
|
||||
assert request.enable_txt is False
|
||||
assert request.enable_youtube_upload is False
|
||||
# YouTube upload default is None (use server default)
|
||||
assert request.enable_youtube_upload is None
|
||||
|
||||
|
||||
class TestStartReviewRequest:
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
"""Tests for SpaCy preloader service."""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
from backend.services.spacy_preloader import (
|
||||
preload_spacy_model,
|
||||
get_preloaded_model,
|
||||
is_model_preloaded,
|
||||
clear_preloaded_models,
|
||||
)
|
||||
|
||||
|
||||
class TestSpacyPreloader:
|
||||
"""Tests for SpaCy preloading functionality."""
|
||||
|
||||
def setup_method(self):
|
||||
"""Clear preloaded models before each test."""
|
||||
clear_preloaded_models()
|
||||
|
||||
def teardown_method(self):
|
||||
"""Clear preloaded models after each test."""
|
||||
clear_preloaded_models()
|
||||
|
||||
def test_preload_spacy_model_loads_and_stores(self):
|
||||
"""GIVEN no preloaded models
|
||||
WHEN preload_spacy_model is called
|
||||
THEN model should be loaded and stored in singleton."""
|
||||
mock_nlp = MagicMock()
|
||||
|
||||
with patch("spacy.load", return_value=mock_nlp) as mock_load:
|
||||
preload_spacy_model("en_core_web_sm")
|
||||
|
||||
mock_load.assert_called_once_with("en_core_web_sm")
|
||||
assert is_model_preloaded("en_core_web_sm")
|
||||
assert get_preloaded_model("en_core_web_sm") is mock_nlp
|
||||
|
||||
def test_preload_is_idempotent(self):
|
||||
"""GIVEN a model already preloaded
|
||||
WHEN preload_spacy_model is called again
|
||||
THEN model should not be reloaded."""
|
||||
mock_nlp = MagicMock()
|
||||
|
||||
with patch("spacy.load", return_value=mock_nlp) as mock_load:
|
||||
preload_spacy_model("en_core_web_sm")
|
||||
preload_spacy_model("en_core_web_sm") # Second call
|
||||
|
||||
# Should only load once
|
||||
assert mock_load.call_count == 1
|
||||
|
||||
def test_get_preloaded_model_returns_none_if_not_loaded(self):
|
||||
"""GIVEN no preloaded models
|
||||
WHEN get_preloaded_model is called
|
||||
THEN should return None."""
|
||||
assert get_preloaded_model("en_core_web_sm") is None
|
||||
assert not is_model_preloaded("en_core_web_sm")
|
||||
|
||||
def test_preload_different_models(self):
|
||||
"""GIVEN no preloaded models
|
||||
WHEN preload_spacy_model is called with different model names
|
||||
THEN each model should be loaded and stored separately."""
|
||||
mock_nlp_sm = MagicMock(name="en_core_web_sm")
|
||||
mock_nlp_md = MagicMock(name="en_core_web_md")
|
||||
|
||||
def mock_load(model_name):
|
||||
if model_name == "en_core_web_sm":
|
||||
return mock_nlp_sm
|
||||
elif model_name == "en_core_web_md":
|
||||
return mock_nlp_md
|
||||
raise ValueError(f"Unknown model: {model_name}")
|
||||
|
||||
with patch("spacy.load", side_effect=mock_load):
|
||||
preload_spacy_model("en_core_web_sm")
|
||||
preload_spacy_model("en_core_web_md")
|
||||
|
||||
assert is_model_preloaded("en_core_web_sm")
|
||||
assert is_model_preloaded("en_core_web_md")
|
||||
assert get_preloaded_model("en_core_web_sm") is mock_nlp_sm
|
||||
assert get_preloaded_model("en_core_web_md") is mock_nlp_md
|
||||
|
||||
def test_clear_preloaded_models(self):
|
||||
"""GIVEN preloaded models
|
||||
WHEN clear_preloaded_models is called
|
||||
THEN all models should be removed."""
|
||||
mock_nlp = MagicMock()
|
||||
|
||||
with patch("spacy.load", return_value=mock_nlp):
|
||||
preload_spacy_model("en_core_web_sm")
|
||||
assert is_model_preloaded("en_core_web_sm")
|
||||
|
||||
clear_preloaded_models()
|
||||
assert not is_model_preloaded("en_core_web_sm")
|
||||
assert get_preloaded_model("en_core_web_sm") is None
|
||||
|
||||
def test_preload_failure_raises_exception(self):
|
||||
"""GIVEN spacy.load raises an exception
|
||||
WHEN preload_spacy_model is called
|
||||
THEN exception should be propagated."""
|
||||
with patch(
|
||||
"spacy.load",
|
||||
side_effect=OSError("Model not found"),
|
||||
):
|
||||
with pytest.raises(OSError, match="Model not found"):
|
||||
preload_spacy_model("nonexistent_model")
|
||||
|
||||
# Model should not be marked as preloaded
|
||||
assert not is_model_preloaded("nonexistent_model")
|
||||
|
||||
def test_preload_uses_default_model_name(self):
|
||||
"""GIVEN no model name specified
|
||||
WHEN preload_spacy_model is called without arguments
|
||||
THEN should use en_core_web_sm as default."""
|
||||
mock_nlp = MagicMock()
|
||||
|
||||
with patch("spacy.load", return_value=mock_nlp) as mock_load:
|
||||
preload_spacy_model()
|
||||
|
||||
mock_load.assert_called_once_with("en_core_web_sm")
|
||||
assert is_model_preloaded("en_core_web_sm")
|
||||
@@ -0,0 +1,295 @@
|
||||
"""
|
||||
Unit tests for template service.
|
||||
"""
|
||||
import pytest
|
||||
from unittest.mock import Mock, patch, MagicMock
|
||||
|
||||
from backend.services.template_service import (
|
||||
TemplateService,
|
||||
get_template_service,
|
||||
DEFAULT_JOB_COMPLETION_TEMPLATE,
|
||||
DEFAULT_ACTION_NEEDED_LYRICS_TEMPLATE,
|
||||
DEFAULT_ACTION_NEEDED_INSTRUMENTAL_TEMPLATE,
|
||||
)
|
||||
|
||||
|
||||
class TestTemplateRendering:
|
||||
"""Tests for template rendering functionality."""
|
||||
|
||||
def test_render_template_basic(self):
|
||||
"""Test basic template variable replacement."""
|
||||
service = TemplateService()
|
||||
template = "Hello {name}, your job {job_id} is ready!"
|
||||
variables = {"name": "Alice", "job_id": "123"}
|
||||
|
||||
result = service.render_template(template, variables)
|
||||
|
||||
assert result == "Hello Alice, your job 123 is ready!"
|
||||
|
||||
def test_render_template_missing_variables(self):
|
||||
"""Test that missing variables are replaced with empty strings."""
|
||||
service = TemplateService()
|
||||
template = "Hello {name}, your {thing} is {status}!"
|
||||
variables = {"name": "Bob"}
|
||||
|
||||
result = service.render_template(template, variables)
|
||||
|
||||
assert result == "Hello Bob, your is !"
|
||||
|
||||
def test_render_template_none_values(self):
|
||||
"""Test that None values are handled correctly."""
|
||||
service = TemplateService()
|
||||
template = "Hello {name}!"
|
||||
variables = {"name": None}
|
||||
|
||||
result = service.render_template(template, variables)
|
||||
|
||||
assert result == "Hello !"
|
||||
|
||||
def test_render_template_removes_feedback_section_when_no_url(self):
|
||||
"""Test that feedback section is removed when feedback_url is not provided."""
|
||||
service = TemplateService()
|
||||
template = """Thanks for your order!
|
||||
|
||||
If you have a moment, I'd really appreciate your feedback (takes 2 minutes):
|
||||
{feedback_url}
|
||||
|
||||
Have a great day!"""
|
||||
variables = {"feedback_url": None}
|
||||
|
||||
result = service.render_template(template, variables)
|
||||
|
||||
assert "feedback" not in result.lower()
|
||||
assert "Thanks for your order!" in result
|
||||
assert "Have a great day!" in result
|
||||
|
||||
def test_render_template_keeps_feedback_section_when_url_provided(self):
|
||||
"""Test that feedback section is kept when feedback_url is provided."""
|
||||
service = TemplateService()
|
||||
template = """Thanks!
|
||||
|
||||
If you have a moment, I'd really appreciate your feedback (takes 2 minutes):
|
||||
{feedback_url}
|
||||
|
||||
Bye!"""
|
||||
variables = {"feedback_url": "https://example.com/feedback"}
|
||||
|
||||
result = service.render_template(template, variables)
|
||||
|
||||
assert "https://example.com/feedback" in result
|
||||
assert "feedback" in result.lower()
|
||||
|
||||
|
||||
class TestJobCompletionTemplate:
|
||||
"""Tests for job completion template rendering."""
|
||||
|
||||
def test_render_job_completion_all_fields(self):
|
||||
"""Test rendering job completion with all fields."""
|
||||
service = TemplateService()
|
||||
|
||||
# Mock _fetch_template_from_gcs to return None (use default)
|
||||
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
|
||||
result = service.render_job_completion(
|
||||
name="Alice",
|
||||
youtube_url="https://youtube.com/watch?v=123",
|
||||
dropbox_url="https://dropbox.com/folder/abc",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
job_id="job-123",
|
||||
feedback_url="https://example.com/feedback",
|
||||
)
|
||||
|
||||
assert "Alice" in result
|
||||
assert "https://youtube.com/watch?v=123" in result
|
||||
assert "https://dropbox.com/folder/abc" in result
|
||||
assert "https://example.com/feedback" in result
|
||||
|
||||
def test_render_job_completion_defaults(self):
|
||||
"""Test rendering job completion with default values."""
|
||||
service = TemplateService()
|
||||
|
||||
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
|
||||
result = service.render_job_completion()
|
||||
|
||||
assert "there" in result # Default name
|
||||
assert "[YouTube URL not available]" in result
|
||||
assert "[Dropbox URL not available]" in result
|
||||
|
||||
def test_render_job_completion_no_feedback_url(self):
|
||||
"""Test rendering job completion without feedback URL removes section."""
|
||||
service = TemplateService()
|
||||
|
||||
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
|
||||
result = service.render_job_completion(
|
||||
name="Bob",
|
||||
youtube_url="https://youtube.com/123",
|
||||
dropbox_url="https://dropbox.com/abc",
|
||||
)
|
||||
|
||||
assert "Bob" in result
|
||||
# Feedback section should be removed
|
||||
assert "really appreciate your feedback" not in result
|
||||
|
||||
|
||||
class TestActionNeededTemplates:
|
||||
"""Tests for action-needed template rendering."""
|
||||
|
||||
def test_render_action_needed_lyrics(self):
|
||||
"""Test rendering lyrics review reminder."""
|
||||
service = TemplateService()
|
||||
|
||||
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
|
||||
result = service.render_action_needed_lyrics(
|
||||
name="Charlie",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
review_url="https://example.com/review",
|
||||
)
|
||||
|
||||
assert "Charlie" in result
|
||||
assert "Test Artist" in result
|
||||
assert "Test Song" in result
|
||||
assert "https://example.com/review" in result
|
||||
|
||||
def test_render_action_needed_instrumental(self):
|
||||
"""Test rendering instrumental selection reminder."""
|
||||
service = TemplateService()
|
||||
|
||||
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
|
||||
result = service.render_action_needed_instrumental(
|
||||
name="Diana",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
instrumental_url="https://example.com/instrumental",
|
||||
)
|
||||
|
||||
assert "Diana" in result
|
||||
assert "Test Artist" in result
|
||||
assert "Test Song" in result
|
||||
assert "https://example.com/instrumental" in result
|
||||
|
||||
|
||||
class TestGCSFetching:
|
||||
"""Tests for GCS template fetching."""
|
||||
|
||||
def test_fetch_template_from_gcs_success(self):
|
||||
"""Test successful template fetch from GCS."""
|
||||
service = TemplateService()
|
||||
|
||||
# Mock the bucket and blob
|
||||
mock_blob = Mock()
|
||||
mock_blob.exists.return_value = True
|
||||
mock_blob.download_as_text.return_value = "Custom template {name}"
|
||||
|
||||
mock_bucket = Mock()
|
||||
mock_bucket.blob.return_value = mock_blob
|
||||
|
||||
with patch.object(service, "_bucket", mock_bucket):
|
||||
result = service._fetch_template_from_gcs("test.txt")
|
||||
|
||||
assert result == "Custom template {name}"
|
||||
mock_bucket.blob.assert_called_once_with("templates/test.txt")
|
||||
|
||||
def test_fetch_template_from_gcs_not_found(self):
|
||||
"""Test template fetch when file doesn't exist in GCS."""
|
||||
service = TemplateService()
|
||||
|
||||
mock_blob = Mock()
|
||||
mock_blob.exists.return_value = False
|
||||
|
||||
mock_bucket = Mock()
|
||||
mock_bucket.blob.return_value = mock_blob
|
||||
|
||||
with patch.object(service, "_bucket", mock_bucket):
|
||||
result = service._fetch_template_from_gcs("nonexistent.txt")
|
||||
|
||||
assert result is None
|
||||
|
||||
def test_fetch_template_from_gcs_error(self):
|
||||
"""Test template fetch handles errors gracefully."""
|
||||
service = TemplateService()
|
||||
|
||||
mock_bucket = Mock()
|
||||
mock_bucket.blob.side_effect = Exception("GCS error")
|
||||
|
||||
with patch.object(service, "_bucket", mock_bucket):
|
||||
result = service._fetch_template_from_gcs("test.txt")
|
||||
|
||||
assert result is None
|
||||
|
||||
def test_get_job_completion_template_fallback(self):
|
||||
"""Test that default template is used when GCS fetch fails."""
|
||||
service = TemplateService()
|
||||
|
||||
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
|
||||
result = service.get_job_completion_template()
|
||||
|
||||
assert result == DEFAULT_JOB_COMPLETION_TEMPLATE
|
||||
|
||||
def test_get_action_needed_lyrics_template_fallback(self):
|
||||
"""Test that default lyrics template is used when GCS fetch fails."""
|
||||
service = TemplateService()
|
||||
|
||||
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
|
||||
result = service.get_action_needed_lyrics_template()
|
||||
|
||||
assert result == DEFAULT_ACTION_NEEDED_LYRICS_TEMPLATE
|
||||
|
||||
def test_get_action_needed_instrumental_template_fallback(self):
|
||||
"""Test that default instrumental template is used when GCS fetch fails."""
|
||||
service = TemplateService()
|
||||
|
||||
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
|
||||
result = service.get_action_needed_instrumental_template()
|
||||
|
||||
assert result == DEFAULT_ACTION_NEEDED_INSTRUMENTAL_TEMPLATE
|
||||
|
||||
|
||||
class TestTemplateUpload:
|
||||
"""Tests for template upload functionality."""
|
||||
|
||||
def test_upload_template_success(self):
|
||||
"""Test successful template upload."""
|
||||
service = TemplateService()
|
||||
|
||||
mock_blob = Mock()
|
||||
mock_bucket = Mock()
|
||||
mock_bucket.blob.return_value = mock_blob
|
||||
|
||||
with patch.object(service, "_bucket", mock_bucket):
|
||||
result = service.upload_template("test.txt", "Test content")
|
||||
|
||||
assert result is True
|
||||
mock_bucket.blob.assert_called_once_with("templates/test.txt")
|
||||
mock_blob.upload_from_string.assert_called_once_with(
|
||||
"Test content", content_type="text/plain"
|
||||
)
|
||||
|
||||
def test_upload_template_error(self):
|
||||
"""Test template upload handles errors."""
|
||||
service = TemplateService()
|
||||
|
||||
mock_blob = Mock()
|
||||
mock_blob.upload_from_string.side_effect = Exception("Upload failed")
|
||||
mock_bucket = Mock()
|
||||
mock_bucket.blob.return_value = mock_blob
|
||||
|
||||
with patch.object(service, "_bucket", mock_bucket):
|
||||
result = service.upload_template("test.txt", "Test content")
|
||||
|
||||
assert result is False
|
||||
|
||||
|
||||
class TestGlobalInstance:
|
||||
"""Tests for global instance management."""
|
||||
|
||||
def test_get_template_service_returns_same_instance(self):
|
||||
"""Test that get_template_service returns singleton."""
|
||||
# Reset global
|
||||
import backend.services.template_service as ts
|
||||
ts._template_service = None
|
||||
|
||||
service1 = get_template_service()
|
||||
service2 = get_template_service()
|
||||
|
||||
assert service1 is service2
|
||||
@@ -0,0 +1,522 @@
|
||||
"""
|
||||
Comprehensive tests for Unicode and special character handling.
|
||||
|
||||
This test file verifies that artist/title with Unicode characters (curly quotes,
|
||||
em dashes, non-ASCII characters) are properly sanitized throughout the codebase
|
||||
to prevent:
|
||||
1. HTTP header encoding failures (Content-Disposition, email subjects)
|
||||
2. API query failures (Google Drive)
|
||||
3. Filename encoding issues (Modal API, filesystem)
|
||||
|
||||
The root cause was job d49efab1 which had title "Mama Says (You Can't Back Down)"
|
||||
with a curly apostrophe (U+2019) that caused the audio separation to fail because
|
||||
the Modal API couldn't handle the non-latin-1 character in HTTP headers.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from karaoke_gen.utils import (
|
||||
sanitize_filename,
|
||||
normalize_text,
|
||||
UNICODE_REPLACEMENTS,
|
||||
TEXT_NORMALIZATIONS,
|
||||
APOSTROPHE_REPLACEMENTS,
|
||||
DOUBLE_QUOTE_REPLACEMENTS,
|
||||
DASH_REPLACEMENTS,
|
||||
WHITESPACE_REPLACEMENTS,
|
||||
)
|
||||
|
||||
|
||||
class TestSanitizeFilename:
|
||||
"""Test the sanitize_filename function handles all edge cases."""
|
||||
|
||||
def test_curly_single_quotes(self):
|
||||
"""Test that curly single quotes are converted to straight quotes."""
|
||||
# The exact character that caused job d49efab1 to fail
|
||||
assert sanitize_filename("Can't") == "Can't"
|
||||
assert sanitize_filename("It's") == "It's"
|
||||
# Left single quote
|
||||
assert sanitize_filename("'Hello'") == "'Hello'"
|
||||
|
||||
def test_curly_double_quotes(self):
|
||||
"""Test that curly double quotes are converted to underscores.
|
||||
|
||||
Note: Double quotes are filesystem-unsafe so they become underscores."""
|
||||
# Curly quotes first become straight, then straight quotes become underscore
|
||||
assert sanitize_filename("\u201cHello\u201d") == "_Hello_"
|
||||
|
||||
def test_em_dash(self):
|
||||
"""Test that em dashes are converted to regular hyphens."""
|
||||
assert sanitize_filename("Artist — Title") == "Artist - Title"
|
||||
assert sanitize_filename("Song—Name") == "Song-Name"
|
||||
|
||||
def test_en_dash(self):
|
||||
"""Test that en dashes are converted to regular hyphens."""
|
||||
assert sanitize_filename("1990–2000") == "1990-2000"
|
||||
|
||||
def test_ellipsis(self):
|
||||
"""Test that horizontal ellipsis is converted to three dots."""
|
||||
# Note: The ellipsis becomes "..." but trailing dots are stripped
|
||||
assert sanitize_filename("Wait\u2026") == "Wait"
|
||||
# But in the middle of a string they're preserved
|
||||
assert sanitize_filename("Wait\u2026Here") == "Wait...Here"
|
||||
|
||||
def test_non_breaking_space(self):
|
||||
"""Test that non-breaking spaces are converted to regular spaces."""
|
||||
# U+00A0 is non-breaking space
|
||||
assert sanitize_filename("Hello\u00A0World") == "Hello World"
|
||||
|
||||
def test_filesystem_unsafe_characters(self):
|
||||
"""Test that filesystem-unsafe characters are replaced."""
|
||||
assert sanitize_filename("file/name") == "file_name"
|
||||
assert sanitize_filename("file\\name") == "file_name"
|
||||
assert sanitize_filename("file:name") == "file_name"
|
||||
assert sanitize_filename("file*name") == "file_name"
|
||||
assert sanitize_filename("file?name") == "file_name"
|
||||
assert sanitize_filename('file"name') == "file_name"
|
||||
assert sanitize_filename("file<name") == "file_name"
|
||||
assert sanitize_filename("file>name") == "file_name"
|
||||
assert sanitize_filename("file|name") == "file_name"
|
||||
|
||||
def test_trailing_periods_and_spaces(self):
|
||||
"""Test that trailing periods and spaces are removed."""
|
||||
assert sanitize_filename("filename.") == "filename"
|
||||
assert sanitize_filename("filename...") == "filename"
|
||||
assert sanitize_filename("filename ") == "filename"
|
||||
assert sanitize_filename("filename . ") == "filename"
|
||||
|
||||
def test_leading_periods_and_spaces(self):
|
||||
"""Test that leading periods and spaces are removed."""
|
||||
assert sanitize_filename(".filename") == "filename"
|
||||
assert sanitize_filename("...filename") == "filename"
|
||||
assert sanitize_filename(" filename") == "filename"
|
||||
assert sanitize_filename(" . filename") == "filename"
|
||||
|
||||
def test_multiple_underscores_collapsed(self):
|
||||
"""Test that multiple consecutive underscores are collapsed to one."""
|
||||
assert sanitize_filename("file___name") == "file_name"
|
||||
# Multiple unsafe chars in a row
|
||||
assert sanitize_filename("file?*:name") == "file_name"
|
||||
|
||||
def test_multiple_spaces_collapsed(self):
|
||||
"""Test that multiple consecutive spaces are collapsed to one."""
|
||||
assert sanitize_filename("file name") == "file name"
|
||||
|
||||
def test_none_input(self):
|
||||
"""Test that None input returns None."""
|
||||
assert sanitize_filename(None) is None
|
||||
|
||||
def test_empty_string(self):
|
||||
"""Test that empty string returns empty string."""
|
||||
assert sanitize_filename("") == ""
|
||||
|
||||
def test_real_world_examples(self):
|
||||
"""Test real-world examples that have caused issues."""
|
||||
# The exact title from job d49efab1
|
||||
assert sanitize_filename("Mama Says (You Can't Back Down)") == "Mama Says (You Can't Back Down)"
|
||||
|
||||
# Broadway cast with smart quotes
|
||||
assert sanitize_filename("Footloose (Broadway Cast) \u2014 \u201cFinal Song\u201d") == "Footloose (Broadway Cast) - _Final Song_"
|
||||
|
||||
# Japanese artist with em dash
|
||||
assert sanitize_filename("宇多田ヒカル — First Love") == "宇多田ヒカル - First Love"
|
||||
|
||||
# Korean title (should pass through unchanged)
|
||||
assert sanitize_filename("아이유 - 좋은 날") == "아이유 - 좋은 날"
|
||||
|
||||
# Mixed content
|
||||
assert sanitize_filename("L'Arc~en~Ciel - Driver's High") == "L'Arc~en~Ciel - Driver's High"
|
||||
|
||||
def test_combination_of_issues(self):
|
||||
"""Test strings with multiple problematic characters."""
|
||||
result = sanitize_filename("It\u2019s \u201cMy\u201d Song \u2014 Volume\u20261")
|
||||
# Curly ' -> ', curly " -> _ (via filesystem check), em dash -> -, ellipsis -> ...
|
||||
assert result == "It's _My_ Song - Volume...1"
|
||||
|
||||
def test_unicode_replacements_dict_complete(self):
|
||||
"""Verify all expected Unicode characters are in the replacements dict."""
|
||||
# Curly quotes
|
||||
assert "\u2018" in UNICODE_REPLACEMENTS # LEFT SINGLE QUOTATION MARK
|
||||
assert "\u2019" in UNICODE_REPLACEMENTS # RIGHT SINGLE QUOTATION MARK
|
||||
assert "\u201A" in UNICODE_REPLACEMENTS # SINGLE LOW-9 QUOTATION MARK
|
||||
assert "\u201B" in UNICODE_REPLACEMENTS # SINGLE HIGH-REVERSED-9 QUOTATION MARK
|
||||
assert "\u201C" in UNICODE_REPLACEMENTS # LEFT DOUBLE QUOTATION MARK
|
||||
assert "\u201D" in UNICODE_REPLACEMENTS # RIGHT DOUBLE QUOTATION MARK
|
||||
assert "\u201E" in UNICODE_REPLACEMENTS # DOUBLE LOW-9 QUOTATION MARK
|
||||
assert "\u201F" in UNICODE_REPLACEMENTS # DOUBLE HIGH-REVERSED-9 QUOTATION MARK
|
||||
# Dashes
|
||||
assert "\u2013" in UNICODE_REPLACEMENTS # EN DASH
|
||||
assert "\u2014" in UNICODE_REPLACEMENTS # EM DASH
|
||||
# Other
|
||||
assert "\u2026" in UNICODE_REPLACEMENTS # HORIZONTAL ELLIPSIS
|
||||
assert "\u00A0" in UNICODE_REPLACEMENTS # NON-BREAKING SPACE
|
||||
|
||||
|
||||
class TestSanitizationInContext:
|
||||
"""Test that sanitization is applied correctly in various contexts.
|
||||
|
||||
These tests verify that the fixes we applied work in the actual
|
||||
code paths that use artist/title data.
|
||||
"""
|
||||
|
||||
def test_artist_title_format_for_audio_separation(self):
|
||||
"""Verify artist-title formatting is sanitized for Modal API."""
|
||||
# Simulate what audio_worker.py does
|
||||
artist = "Footloose (Broadway Cast)"
|
||||
title = "Mama Says (You Can\u2019t Back Down)" # Curly apostrophe U+2019
|
||||
|
||||
safe_artist = sanitize_filename(artist) if artist else "Unknown"
|
||||
safe_title = sanitize_filename(title) if title else "Unknown"
|
||||
artist_title = f"{safe_artist} - {safe_title}"
|
||||
|
||||
# The curly apostrophe should be converted to straight
|
||||
assert "'" in artist_title # Straight apostrophe
|
||||
assert "\u2019" not in artist_title # No curly apostrophe (U+2019)
|
||||
|
||||
def test_email_subject_safe_for_latin1(self):
|
||||
"""Verify email subjects can be encoded to latin-1 after sanitization."""
|
||||
artist = "Café del Mar" # Has accented e
|
||||
title = "It\u2019s a \u201cBeautiful\u201d Day \u2014 Remix"
|
||||
|
||||
safe_artist = sanitize_filename(artist) if artist else None
|
||||
safe_title = sanitize_filename(title) if title else None
|
||||
subject = f"{safe_artist} - {safe_title}"
|
||||
|
||||
# Accented characters should pass through (they're valid latin-1)
|
||||
assert "é" in subject
|
||||
# But curly quotes and em dash should be converted
|
||||
assert "'" in subject # Straight apostrophe
|
||||
assert "-" in subject # Regular hyphen
|
||||
# No curly apostrophe (U+2019)
|
||||
assert "\u2019" not in subject
|
||||
|
||||
def test_content_disposition_safe(self):
|
||||
"""Verify filenames are safe for HTTP Content-Disposition headers."""
|
||||
artist = "Artist\u2019s \u201cName\u201d"
|
||||
title = "Song\u2014Title"
|
||||
|
||||
safe_artist = sanitize_filename(artist) if artist else None
|
||||
safe_title = sanitize_filename(title) if title else None
|
||||
filename = f"{safe_artist} - {safe_title} (Final Karaoke).mp4"
|
||||
|
||||
# Should be able to encode to latin-1 for HTTP headers
|
||||
try:
|
||||
filename.encode('latin-1')
|
||||
can_encode = True
|
||||
except UnicodeEncodeError:
|
||||
can_encode = False
|
||||
|
||||
# Standard ASCII chars and latin-1 safe accented chars should work
|
||||
# Note: The test above with 'é' passes latin-1; smart quotes do not
|
||||
assert can_encode
|
||||
|
||||
def test_google_drive_query_safe(self):
|
||||
"""Verify filenames don't break Google Drive API queries."""
|
||||
base_name = "Artist\u2019s \u201cSong\u201d"
|
||||
|
||||
safe_base_name = sanitize_filename(base_name) if base_name else base_name
|
||||
filename = f"NOMAD-1234 - {safe_base_name}.mp4"
|
||||
|
||||
# Google Drive queries use single quotes - our sanitized string
|
||||
# should have straight single quotes, not curly
|
||||
# The query escaping handles straight quotes: '
|
||||
escaped = filename.replace("'", "\\'")
|
||||
query = f"name='{escaped}'"
|
||||
|
||||
# Should be a valid query string (no curly quotes to break syntax)
|
||||
assert "\u2019" not in query # No curly single quote (U+2019)
|
||||
assert "\u2018" not in query # No curly single quote (U+2018)
|
||||
|
||||
def test_dropbox_path_safe(self):
|
||||
"""Verify Dropbox paths don't have problematic characters."""
|
||||
artist = "Don\u2019t Stop"
|
||||
title = "Believin\u2019"
|
||||
|
||||
safe_artist = sanitize_filename(artist) if artist else "Unknown"
|
||||
safe_title = sanitize_filename(title) if title else "Unknown"
|
||||
folder_name = f"NOMAD-1234 - {safe_artist} - {safe_title}"
|
||||
remote_path = f"/Karaoke/{folder_name}"
|
||||
|
||||
# Path should not have curly quotes (U+2018 and U+2019)
|
||||
assert "\u2018" not in remote_path
|
||||
assert "\u2019" not in remote_path
|
||||
# But should have the converted straight apostrophe
|
||||
assert "'" in remote_path
|
||||
|
||||
|
||||
class TestInternationalCharacters:
|
||||
"""Test that international/non-ASCII characters are handled properly.
|
||||
|
||||
The goal is NOT to strip all non-ASCII, but to convert problematic
|
||||
Unicode to safe equivalents while preserving legitimate international text.
|
||||
"""
|
||||
|
||||
def test_japanese_preserved(self):
|
||||
"""Japanese characters should pass through unchanged."""
|
||||
assert sanitize_filename("宇多田ヒカル") == "宇多田ヒカル"
|
||||
assert sanitize_filename("君が代") == "君が代"
|
||||
|
||||
def test_korean_preserved(self):
|
||||
"""Korean characters should pass through unchanged."""
|
||||
assert sanitize_filename("방탄소년단") == "방탄소년단"
|
||||
assert sanitize_filename("좋은 날") == "좋은 날"
|
||||
|
||||
def test_chinese_preserved(self):
|
||||
"""Chinese characters should pass through unchanged."""
|
||||
assert sanitize_filename("周杰倫") == "周杰倫"
|
||||
assert sanitize_filename("青花瓷") == "青花瓷"
|
||||
|
||||
def test_cyrillic_preserved(self):
|
||||
"""Cyrillic characters should pass through unchanged."""
|
||||
assert sanitize_filename("Тату") == "Тату"
|
||||
assert sanitize_filename("Не верь, не бойся") == "Не верь, не бойся"
|
||||
|
||||
def test_arabic_preserved(self):
|
||||
"""Arabic characters should pass through unchanged."""
|
||||
assert sanitize_filename("فيروز") == "فيروز"
|
||||
|
||||
def test_accented_latin_preserved(self):
|
||||
"""Accented Latin characters should pass through unchanged."""
|
||||
assert sanitize_filename("Café") == "Café"
|
||||
assert sanitize_filename("Señorita") == "Señorita"
|
||||
assert sanitize_filename("Naïve") == "Naïve"
|
||||
assert sanitize_filename("Björk") == "Björk"
|
||||
|
||||
def test_mixed_script_preserved(self):
|
||||
"""Mixed script text should work correctly."""
|
||||
# Japanese artist, English title
|
||||
assert sanitize_filename("宇多田ヒカル - First Love") == "宇多田ヒカル - First Love"
|
||||
# K-pop with English
|
||||
assert sanitize_filename("BTS 방탄소년단 - Dynamite") == "BTS 방탄소년단 - Dynamite"
|
||||
|
||||
|
||||
class TestEdgeCasesAndRegression:
|
||||
"""Edge cases and regression tests for specific bugs."""
|
||||
|
||||
def test_job_d49efab1_exact_title(self):
|
||||
"""Exact reproduction of job d49efab1 failure case.
|
||||
|
||||
The job had:
|
||||
- artist: "Footloose (Broadway Cast)" (from display_artist override)
|
||||
- title: "Mama Says (You Can't Back Down)" (with curly apostrophe U+2019)
|
||||
|
||||
This caused Stage 2 audio separation to fail because the filename
|
||||
with the curly apostrophe couldn't be encoded in HTTP headers.
|
||||
"""
|
||||
artist = "Footloose (Broadway Cast)"
|
||||
title = "Mama Says (You Can\u2019t Back Down)" # Explicit U+2019
|
||||
|
||||
safe_artist = sanitize_filename(artist)
|
||||
safe_title = sanitize_filename(title)
|
||||
artist_title = f"{safe_artist} - {safe_title}"
|
||||
|
||||
# The result should use straight apostrophe
|
||||
assert "Can't" in artist_title
|
||||
assert "\u2019" not in artist_title # No curly apostrophe
|
||||
|
||||
# Should be HTTP-header safe
|
||||
try:
|
||||
artist_title.encode('latin-1')
|
||||
header_safe = True
|
||||
except UnicodeEncodeError:
|
||||
header_safe = False
|
||||
assert header_safe
|
||||
|
||||
def test_double_sanitization_idempotent(self):
|
||||
"""Sanitizing twice should give the same result as once."""
|
||||
original = "It\u2019s \u201cMy\u201d Song \u2014 Test\u2026"
|
||||
once = sanitize_filename(original)
|
||||
twice = sanitize_filename(once)
|
||||
assert once == twice
|
||||
|
||||
def test_only_problematic_chars_string(self):
|
||||
"""Test string made entirely of problematic characters."""
|
||||
result = sanitize_filename("\u2018\u2019\u201c\u201d\u2014\u2026")
|
||||
# Should become: ''""--... then quotes -> underscores, collapses
|
||||
# Actually: ' ' " " - ... -> underscores for filesystem chars
|
||||
assert result # Should not be empty
|
||||
|
||||
def test_very_long_filename(self):
|
||||
"""Test that very long filenames are handled."""
|
||||
long_name = "A" * 1000
|
||||
result = sanitize_filename(long_name)
|
||||
assert result == long_name # No truncation in sanitize_filename itself
|
||||
|
||||
def test_special_musical_characters(self):
|
||||
"""Test musical symbols and special characters."""
|
||||
# These should pass through as they're not in our replacement list
|
||||
assert "♪" in sanitize_filename("♪ Intro ♪")
|
||||
assert "♫" in sanitize_filename("♫ Music ♫")
|
||||
assert "♯" in sanitize_filename("C♯ Minor")
|
||||
assert "♭" in sanitize_filename("B♭ Major")
|
||||
|
||||
|
||||
class TestNormalizeText:
|
||||
"""Test the normalize_text function for data consistency normalization."""
|
||||
|
||||
def test_curly_single_quotes_normalized(self):
|
||||
"""Test that curly single quotes are converted to straight apostrophe."""
|
||||
assert normalize_text("Can\u2019t") == "Can't"
|
||||
assert normalize_text("It\u2018s") == "It's"
|
||||
assert normalize_text("\u201Aquote\u201B") == "'quote'"
|
||||
|
||||
def test_curly_double_quotes_normalized(self):
|
||||
"""Test that curly double quotes are converted to straight double quotes."""
|
||||
assert normalize_text("\u201CHello\u201D") == '"Hello"'
|
||||
assert normalize_text("\u201Equote\u201F") == '"quote"'
|
||||
|
||||
def test_backticks_normalized(self):
|
||||
"""Test that backticks and similar marks are converted to apostrophe."""
|
||||
assert normalize_text("code`here") == "code'here"
|
||||
assert normalize_text("acute\u00B4accent") == "acute'accent"
|
||||
assert normalize_text("prime\u2032mark") == "prime'mark"
|
||||
|
||||
def test_dashes_normalized(self):
|
||||
"""Test that various dashes are converted to hyphen-minus."""
|
||||
# EN DASH
|
||||
assert normalize_text("1990\u20132000") == "1990-2000"
|
||||
# EM DASH
|
||||
assert normalize_text("word\u2014word") == "word-word"
|
||||
# MINUS SIGN
|
||||
assert normalize_text("a\u2212b") == "a-b"
|
||||
# FIGURE DASH
|
||||
assert normalize_text("phone\u2012number") == "phone-number"
|
||||
|
||||
def test_whitespace_normalized(self):
|
||||
"""Test that various whitespace characters are normalized."""
|
||||
# NON-BREAKING SPACE
|
||||
assert normalize_text("hello\u00A0world") == "hello world"
|
||||
# EM SPACE
|
||||
assert normalize_text("hello\u2003world") == "hello world"
|
||||
# IDEOGRAPHIC SPACE (CJK full-width)
|
||||
assert normalize_text("hello\u3000world") == "hello world"
|
||||
# ZERO WIDTH SPACE (removed entirely)
|
||||
assert normalize_text("hello\u200Bworld") == "helloworld"
|
||||
|
||||
def test_ellipsis_normalized(self):
|
||||
"""Test that ellipsis character is converted to three dots."""
|
||||
assert normalize_text("Wait\u2026") == "Wait..."
|
||||
assert normalize_text("Loading\u2026please wait") == "Loading...please wait"
|
||||
|
||||
def test_multiple_spaces_collapsed(self):
|
||||
"""Test that multiple spaces are collapsed to one."""
|
||||
assert normalize_text("hello world") == "hello world"
|
||||
# After normalizing multiple whitespace chars
|
||||
assert normalize_text("hello\u00A0\u00A0\u00A0world") == "hello world"
|
||||
|
||||
def test_leading_trailing_whitespace_stripped(self):
|
||||
"""Test that leading/trailing whitespace is stripped."""
|
||||
assert normalize_text(" hello ") == "hello"
|
||||
assert normalize_text("\u00A0hello\u00A0") == "hello"
|
||||
|
||||
def test_none_input(self):
|
||||
"""Test that None input returns None."""
|
||||
assert normalize_text(None) is None
|
||||
|
||||
def test_non_string_input(self):
|
||||
"""Test that non-string input is returned unchanged."""
|
||||
assert normalize_text(123) == 123
|
||||
assert normalize_text(["list"]) == ["list"]
|
||||
|
||||
def test_international_characters_preserved(self):
|
||||
"""Test that international characters are NOT normalized away."""
|
||||
assert normalize_text("日本語") == "日本語"
|
||||
assert normalize_text("한국어") == "한국어"
|
||||
assert normalize_text("Café") == "Café"
|
||||
assert normalize_text("Björk") == "Björk"
|
||||
|
||||
def test_real_world_examples(self):
|
||||
"""Test real-world examples with mixed content."""
|
||||
# Job d49efab1's title
|
||||
assert normalize_text("Mama Says (You Can\u2019t Back Down)") == "Mama Says (You Can't Back Down)"
|
||||
# Broadway cast with smart quotes and em dash
|
||||
assert normalize_text("Footloose \u2014 \u201CFinal Song\u201D") == 'Footloose - "Final Song"'
|
||||
# Japanese with em dash
|
||||
assert normalize_text("宇多田ヒカル \u2014 First Love") == "宇多田ヒカル - First Love"
|
||||
|
||||
def test_idempotent(self):
|
||||
"""Test that normalizing twice gives the same result."""
|
||||
original = "It\u2019s \u201CMy\u201D Song \u2014 Test\u2026"
|
||||
once = normalize_text(original)
|
||||
twice = normalize_text(once)
|
||||
assert once == twice
|
||||
|
||||
def test_text_normalizations_dict_complete(self):
|
||||
"""Verify TEXT_NORMALIZATIONS includes all expected categories."""
|
||||
# Check apostrophe-like characters
|
||||
for char in APOSTROPHE_REPLACEMENTS:
|
||||
assert char in TEXT_NORMALIZATIONS
|
||||
|
||||
# Check double quote-like characters
|
||||
for char in DOUBLE_QUOTE_REPLACEMENTS:
|
||||
assert char in TEXT_NORMALIZATIONS
|
||||
|
||||
# Check dash-like characters
|
||||
for char in DASH_REPLACEMENTS:
|
||||
assert char in TEXT_NORMALIZATIONS
|
||||
|
||||
# Check whitespace characters
|
||||
for char in WHITESPACE_REPLACEMENTS:
|
||||
assert char in TEXT_NORMALIZATIONS
|
||||
|
||||
|
||||
class TestModelValidatorNormalization:
|
||||
"""Test that model validators correctly normalize input text."""
|
||||
|
||||
def test_job_create_normalizes_artist_title(self):
|
||||
"""Test that JobCreate model normalizes artist and title fields."""
|
||||
from backend.models.job import JobCreate
|
||||
|
||||
job = JobCreate(
|
||||
artist="Don\u2019t Stop",
|
||||
title="Believin\u2019 \u2014 Live",
|
||||
)
|
||||
assert job.artist == "Don't Stop"
|
||||
assert job.title == "Believin' - Live"
|
||||
|
||||
def test_job_create_preserves_international_chars(self):
|
||||
"""Test that JobCreate preserves international characters."""
|
||||
from backend.models.job import JobCreate
|
||||
|
||||
job = JobCreate(
|
||||
artist="宇多田ヒカル",
|
||||
title="First Love",
|
||||
)
|
||||
assert job.artist == "宇多田ヒカル"
|
||||
assert job.title == "First Love"
|
||||
|
||||
def test_job_create_handles_none(self):
|
||||
"""Test that JobCreate handles None values correctly."""
|
||||
from backend.models.job import JobCreate
|
||||
|
||||
job = JobCreate(
|
||||
artist=None,
|
||||
title="Some Title",
|
||||
)
|
||||
assert job.artist is None
|
||||
assert job.title == "Some Title"
|
||||
|
||||
def test_audio_search_normalizes_fields(self):
|
||||
"""Test that AudioSearchRequest normalizes text fields."""
|
||||
from backend.api.routes.audio_search import AudioSearchRequest
|
||||
|
||||
request = AudioSearchRequest(
|
||||
artist="Artist\u2019s Name",
|
||||
title="Song \u2014 Remix",
|
||||
display_artist="Display\u2019s Artist",
|
||||
display_title="Display\u2019s Title",
|
||||
)
|
||||
assert request.artist == "Artist's Name"
|
||||
assert request.title == "Song - Remix"
|
||||
assert request.display_artist == "Display's Artist"
|
||||
assert request.display_title == "Display's Title"
|
||||
|
||||
def test_audio_search_preserves_international_chars(self):
|
||||
"""Test that AudioSearchRequest preserves international characters."""
|
||||
from backend.api.routes.audio_search import AudioSearchRequest
|
||||
|
||||
request = AudioSearchRequest(
|
||||
artist="방탄소년단",
|
||||
title="Dynamite",
|
||||
)
|
||||
assert request.artist == "방탄소년단"
|
||||
assert request.title == "Dynamite"
|
||||
@@ -425,6 +425,111 @@ class TestVideoWorkerOrchestratorEncoding:
|
||||
|
||||
assert "Encoding failed" in str(exc_info.value)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_encoding_gce_downloads_files(self):
|
||||
"""Test that GCE encoding downloads files from GCS to local directory.
|
||||
|
||||
This test verifies the fix for YouTube upload failure when using GCE encoding.
|
||||
GCE encoding returns GCS blob paths, which need to be downloaded locally
|
||||
before YouTube upload can access them.
|
||||
"""
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
config = OrchestratorConfig(
|
||||
job_id="test-job",
|
||||
artist="Test Artist",
|
||||
title="Test Title",
|
||||
title_video_path=os.path.join(temp_dir, "title.mov"),
|
||||
karaoke_video_path=os.path.join(temp_dir, "karaoke.mov"),
|
||||
instrumental_audio_path=os.path.join(temp_dir, "audio.flac"),
|
||||
output_dir=temp_dir,
|
||||
)
|
||||
|
||||
# Mock storage service
|
||||
mock_storage = MagicMock()
|
||||
mock_storage.download_file = MagicMock()
|
||||
|
||||
orchestrator = VideoWorkerOrchestrator(
|
||||
config,
|
||||
storage=mock_storage,
|
||||
)
|
||||
|
||||
with patch.object(orchestrator, "_get_encoding_backend") as mock_get:
|
||||
from backend.services.encoding_interface import EncodingOutput
|
||||
|
||||
# GCE backend returns GCS blob paths (not local paths)
|
||||
mock_backend = MagicMock()
|
||||
mock_backend.name = "gce" # Important: must be "gce" to trigger download
|
||||
mock_backend.encode = AsyncMock(return_value=EncodingOutput(
|
||||
success=True,
|
||||
lossless_4k_mp4_path="jobs/test-job/finals/output_4k_lossless.mp4",
|
||||
lossy_4k_mp4_path="jobs/test-job/finals/output_4k_lossy.mp4",
|
||||
lossy_720p_mp4_path="jobs/test-job/finals/output_720p.mp4",
|
||||
lossless_mkv_path="jobs/test-job/finals/output_4k.mkv",
|
||||
encoding_time_seconds=60.0,
|
||||
encoding_backend="gce",
|
||||
))
|
||||
mock_get.return_value = mock_backend
|
||||
|
||||
await orchestrator._run_encoding()
|
||||
|
||||
# Verify download_file was called for each output file
|
||||
assert mock_storage.download_file.call_count == 4
|
||||
|
||||
# Verify the result paths were updated to local paths
|
||||
assert orchestrator.result.final_video.startswith(temp_dir)
|
||||
assert orchestrator.result.final_video_mkv.startswith(temp_dir)
|
||||
assert orchestrator.result.final_video_lossy.startswith(temp_dir)
|
||||
assert orchestrator.result.final_video_720p.startswith(temp_dir)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_encoding_local_does_not_download(self):
|
||||
"""Test that local encoding does NOT trigger GCS download.
|
||||
|
||||
Local encoding produces files directly in the output directory,
|
||||
so no download is needed.
|
||||
"""
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
config = OrchestratorConfig(
|
||||
job_id="test-job",
|
||||
artist="Test Artist",
|
||||
title="Test Title",
|
||||
title_video_path=os.path.join(temp_dir, "title.mov"),
|
||||
karaoke_video_path=os.path.join(temp_dir, "karaoke.mov"),
|
||||
instrumental_audio_path=os.path.join(temp_dir, "audio.flac"),
|
||||
output_dir=temp_dir,
|
||||
)
|
||||
|
||||
# Mock storage service
|
||||
mock_storage = MagicMock()
|
||||
mock_storage.download_file = MagicMock()
|
||||
|
||||
orchestrator = VideoWorkerOrchestrator(
|
||||
config,
|
||||
storage=mock_storage,
|
||||
)
|
||||
|
||||
with patch.object(orchestrator, "_get_encoding_backend") as mock_get:
|
||||
from backend.services.encoding_interface import EncodingOutput
|
||||
|
||||
# Local backend returns local paths directly
|
||||
mock_backend = MagicMock()
|
||||
mock_backend.name = "local" # Local backend
|
||||
mock_backend.encode = AsyncMock(return_value=EncodingOutput(
|
||||
success=True,
|
||||
lossless_4k_mp4_path=os.path.join(temp_dir, "lossless.mp4"),
|
||||
lossy_4k_mp4_path=os.path.join(temp_dir, "lossy.mp4"),
|
||||
lossy_720p_mp4_path=os.path.join(temp_dir, "720p.mp4"),
|
||||
lossless_mkv_path=os.path.join(temp_dir, "lossless.mkv"),
|
||||
encoding_time_seconds=120.0,
|
||||
encoding_backend="local",
|
||||
))
|
||||
mock_get.return_value = mock_backend
|
||||
|
||||
await orchestrator._run_encoding()
|
||||
|
||||
# Verify download_file was NOT called for local encoding
|
||||
mock_storage.download_file.assert_not_called()
|
||||
|
||||
|
||||
class TestVideoWorkerOrchestratorOrganization:
|
||||
"""Test organization stage."""
|
||||
|
||||
@@ -344,7 +344,12 @@ async def process_audio_separation(job_id: str) -> bool:
|
||||
job_log.info(f"Stored effective model names: clean={effective_model_names['clean_instrumental_model']}")
|
||||
|
||||
# Format artist-title for file naming (matches CLI behavior)
|
||||
artist_title = f"{job.artist} - {job.title}"
|
||||
# Sanitize to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
# that cause HTTP header encoding issues with the remote API
|
||||
from karaoke_gen.utils import sanitize_filename
|
||||
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
|
||||
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
|
||||
artist_title = f"{safe_artist} - {safe_title}"
|
||||
|
||||
# Run audio separation (calls Modal API internally)
|
||||
# This returns a dict with paths to all separated stems
|
||||
|
||||
@@ -639,40 +639,45 @@ async def upload_lyrics_results(
|
||||
logger.error(f"Job {job_id}: {error_msg}")
|
||||
raise Exception(error_msg)
|
||||
|
||||
# Upload reference lyrics if available
|
||||
# Upload ALL reference lyrics files (not just first) so they're available for distribution
|
||||
# Note: Source names use .title() so "lrclib" -> "Lrclib", "genius" -> "Genius"
|
||||
# Use sanitized artist/title to match LyricsProcessor's file naming
|
||||
reference_files = [
|
||||
(f"{safe_artist} - {safe_title} (Lyrics Genius).txt", "Genius"),
|
||||
(f"{safe_artist} - {safe_title} (Lyrics Spotify).txt", "Spotify"),
|
||||
(f"{safe_artist} - {safe_title} (Lyrics Musixmatch).txt", "Musixmatch"),
|
||||
(f"{safe_artist} - {safe_title} (Lyrics Lrclib).txt", "LRCLib"), # Note: source is 'lrclib', .title() = 'Lrclib'
|
||||
(f"{safe_artist} - {safe_title} (Lyrics Genius).txt", "genius"),
|
||||
(f"{safe_artist} - {safe_title} (Lyrics Spotify).txt", "spotify"),
|
||||
(f"{safe_artist} - {safe_title} (Lyrics Musixmatch).txt", "musixmatch"),
|
||||
(f"{safe_artist} - {safe_title} (Lyrics Lrclib).txt", "lrclib"),
|
||||
]
|
||||
|
||||
found_reference = False
|
||||
for ref_filename, source_name in reference_files:
|
||||
|
||||
found_references = []
|
||||
for ref_filename, source_key in reference_files:
|
||||
ref_path = os.path.join(lyrics_dir, ref_filename)
|
||||
if os.path.exists(ref_path):
|
||||
# Upload with original filename to preserve proper naming
|
||||
gcs_path = f"jobs/{job_id}/lyrics/{ref_filename}"
|
||||
url = storage.upload_file(ref_path, gcs_path)
|
||||
# Track in job.files so video_worker can download for distribution
|
||||
job_manager.update_file_url(job_id, 'lyrics', f'reference_{source_key}', url)
|
||||
if job_log:
|
||||
job_log.info(f"Found reference lyrics from {source_name}")
|
||||
job_log.info(f"Found reference lyrics from {source_key}")
|
||||
logger.info(f"Job {job_id}: Uploaded reference lyrics: {ref_filename}")
|
||||
found_reference = True
|
||||
break # Only upload first available reference
|
||||
|
||||
if not found_reference:
|
||||
found_references.append(source_key)
|
||||
|
||||
if not found_references:
|
||||
if job_log:
|
||||
job_log.warning("No reference lyrics found from any source (Genius, Spotify, Musixmatch, LRCLib)")
|
||||
logger.warning(f"Job {job_id}: No reference lyrics found from any source")
|
||||
else:
|
||||
logger.info(f"Job {job_id}: Found {len(found_references)} reference lyrics sources: {found_references}")
|
||||
|
||||
# Upload uncorrected transcription if available (use sanitized names)
|
||||
uncorrected_file = os.path.join(lyrics_dir, f"{safe_artist} - {safe_title} (Lyrics Uncorrected).txt")
|
||||
# Upload uncorrected transcription if available (preserve original filename for distribution)
|
||||
uncorrected_filename = f"{safe_artist} - {safe_title} (Lyrics Uncorrected).txt"
|
||||
uncorrected_file = os.path.join(lyrics_dir, uncorrected_filename)
|
||||
if os.path.exists(uncorrected_file):
|
||||
gcs_path = f"jobs/{job_id}/lyrics/uncorrected.txt"
|
||||
gcs_path = f"jobs/{job_id}/lyrics/{uncorrected_filename}"
|
||||
url = storage.upload_file(uncorrected_file, gcs_path)
|
||||
job_manager.update_file_url(job_id, 'lyrics', 'uncorrected', url)
|
||||
logger.info(f"Job {job_id}: Uploaded uncorrected transcription")
|
||||
logger.info(f"Job {job_id}: Uploaded uncorrected transcription: {uncorrected_filename}")
|
||||
|
||||
logger.info(f"Job {job_id}: All lyrics results uploaded successfully")
|
||||
|
||||
@@ -43,6 +43,7 @@ from lyrics_transcriber.core.config import OutputConfig
|
||||
|
||||
# Import from the unified style loader
|
||||
from karaoke_gen.style_loader import load_styles_from_gcs
|
||||
from karaoke_gen.utils import sanitize_filename
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -242,7 +243,10 @@ async def process_render_video(job_id: str) -> bool:
|
||||
output_generator = OutputGenerator(config, logger)
|
||||
|
||||
# 8. Generate outputs (video, LRC, ASS, etc.)
|
||||
output_prefix = f"{job.artist or 'Unknown'} - {job.title or 'Unknown'}"
|
||||
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
|
||||
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
|
||||
output_prefix = f"{safe_artist} - {safe_title}"
|
||||
job_log.info(f"Generating outputs with prefix '{output_prefix}'")
|
||||
logger.info(f"Job {job_id}: Generating outputs with prefix '{output_prefix}'")
|
||||
|
||||
|
||||
@@ -39,6 +39,7 @@ from backend.services.tracing import job_span, add_span_event, add_span_attribut
|
||||
|
||||
# Import from karaoke_gen package
|
||||
from karaoke_gen.video_generator import VideoGenerator
|
||||
from karaoke_gen.utils import sanitize_filename
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -294,9 +295,12 @@ async def _generate_title_screen(
|
||||
"""
|
||||
try:
|
||||
logger.info(f"Job {job_id}: Generating title screen")
|
||||
|
||||
|
||||
# Set up output paths
|
||||
artist_title = f"{job.artist} - {job.title}"
|
||||
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
|
||||
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
|
||||
artist_title = f"{safe_artist} - {safe_title}"
|
||||
output_image_filepath_noext = os.path.join(temp_dir, f"{artist_title} (Title)")
|
||||
output_video_filepath = os.path.join(temp_dir, f"{artist_title} (Title).mov")
|
||||
|
||||
@@ -372,9 +376,12 @@ async def _generate_end_screen(
|
||||
"""
|
||||
try:
|
||||
logger.info(f"Job {job_id}: Generating end screen")
|
||||
|
||||
|
||||
# Set up output paths
|
||||
artist_title = f"{job.artist} - {job.title}"
|
||||
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
|
||||
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
|
||||
artist_title = f"{safe_artist} - {safe_title}"
|
||||
output_image_filepath_noext = os.path.join(temp_dir, f"{artist_title} (End)")
|
||||
output_video_filepath = os.path.join(temp_dir, f"{artist_title} (End).mov")
|
||||
|
||||
|
||||
@@ -44,6 +44,7 @@ from backend.services.tracing import job_span, add_span_event, add_span_attribut
|
||||
|
||||
# Import from karaoke_gen package - reuse existing implementation
|
||||
from karaoke_gen.karaoke_finalise.karaoke_finalise import KaraokeFinalise
|
||||
from karaoke_gen.utils import sanitize_filename
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -160,14 +161,17 @@ async def _encode_via_gce(
|
||||
storage.download_file(gcs_path, local_path)
|
||||
|
||||
# Map to result keys expected by _upload_results
|
||||
if "4K Lossless" in filename or "4k_lossless" in filename.lower():
|
||||
local_files["final_video"] = local_path
|
||||
elif "4K Lossy" in filename or "4k_lossy" in filename.lower():
|
||||
# Files are named like "Artist - Title (Final Karaoke Lossless 4k).mp4"
|
||||
filename_lower = filename.lower()
|
||||
if "lossless 4k" in filename_lower:
|
||||
if filename.endswith(".mkv"):
|
||||
local_files["final_video_mkv"] = local_path
|
||||
else:
|
||||
local_files["final_video"] = local_path
|
||||
elif "lossy 4k" in filename_lower:
|
||||
local_files["final_video_lossy"] = local_path
|
||||
elif "720p" in filename:
|
||||
elif "720p" in filename_lower:
|
||||
local_files["final_video_720p"] = local_path
|
||||
elif filename.endswith(".mkv"):
|
||||
local_files["final_video_mkv"] = local_path
|
||||
|
||||
job_log.info(f"Downloaded {len(local_files)} encoded files")
|
||||
|
||||
@@ -258,7 +262,10 @@ async def generate_video_orchestrated(job_id: str) -> bool:
|
||||
)
|
||||
|
||||
# Download and set up files
|
||||
base_name = f"{job.artist} - {job.title}"
|
||||
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
|
||||
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
|
||||
base_name = f"{safe_artist} - {safe_title}"
|
||||
with job_span("download-files", job_id):
|
||||
job_log.info("Downloading files from GCS...")
|
||||
await _setup_working_directory(job_id, job, storage, temp_dir, base_name, job_log)
|
||||
@@ -332,16 +339,9 @@ async def generate_video_orchestrated(job_id: str) -> bool:
|
||||
'final_karaoke_txt_zip': result.final_karaoke_txt_zip,
|
||||
})
|
||||
|
||||
# Mark job as complete
|
||||
# Store result metadata in job BEFORE transitioning to COMPLETE
|
||||
# This ensures youtube_url is available when completion email is sent
|
||||
logger.info(f"[job:{job_id}] Video generation complete")
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.COMPLETE,
|
||||
progress=100,
|
||||
message="Karaoke generation complete!"
|
||||
)
|
||||
|
||||
# Store result metadata in job
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {
|
||||
**job.state_data,
|
||||
@@ -352,6 +352,14 @@ async def generate_video_orchestrated(job_id: str) -> bool:
|
||||
}
|
||||
})
|
||||
|
||||
# Mark job as complete (triggers completion email with youtube_url now available)
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.COMPLETE,
|
||||
progress=100,
|
||||
message="Karaoke generation complete!"
|
||||
)
|
||||
|
||||
duration = time.time() - start_time
|
||||
root_span.set_attribute("duration_seconds", duration)
|
||||
root_span.set_attribute("brand_code", result.brand_code or '')
|
||||
@@ -489,7 +497,10 @@ async def generate_video_legacy(job_id: str) -> bool:
|
||||
)
|
||||
|
||||
# Download and set up files in the format KaraokeFinalise expects
|
||||
base_name = f"{job.artist} - {job.title}"
|
||||
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
|
||||
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
|
||||
base_name = f"{safe_artist} - {safe_title}"
|
||||
with job_span("download-files", job_id):
|
||||
job_log.info("Downloading files from GCS (this may take a few minutes for large files)...")
|
||||
await _setup_working_directory(job_id, job, storage, temp_dir, base_name, job_log)
|
||||
@@ -662,19 +673,12 @@ async def generate_video_legacy(job_id: str) -> bool:
|
||||
with job_span("upload-results", job_id):
|
||||
await _upload_results(job_id, job_manager, storage, temp_dir, result)
|
||||
|
||||
# Mark job as complete
|
||||
logger.info(f"[job:{job_id}] Video generation complete")
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.COMPLETE,
|
||||
progress=100,
|
||||
message="Karaoke generation complete!"
|
||||
)
|
||||
|
||||
# Store result metadata in job
|
||||
# Store result metadata in job BEFORE transitioning to COMPLETE
|
||||
# This ensures youtube_url is available when completion email is sent
|
||||
# NOTE: Must include dropbox_link and gdrive_files from result, since
|
||||
# _handle_native_distribution already saved them but job.state_data
|
||||
# is stale (fetched before distribution ran)
|
||||
logger.info(f"[job:{job_id}] Video generation complete")
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {
|
||||
**job.state_data,
|
||||
@@ -684,7 +688,15 @@ async def generate_video_legacy(job_id: str) -> bool:
|
||||
'gdrive_files': result.get('gdrive_files'),
|
||||
}
|
||||
})
|
||||
|
||||
|
||||
# Mark job as complete (triggers completion email with youtube_url now available)
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.COMPLETE,
|
||||
progress=100,
|
||||
message="Karaoke generation complete!"
|
||||
)
|
||||
|
||||
duration = time.time() - start_time
|
||||
root_span.set_attribute("duration_seconds", duration)
|
||||
root_span.set_attribute("brand_code", result.get('brand_code', ''))
|
||||
@@ -750,8 +762,11 @@ async def _handle_native_distribution(
|
||||
storage: StorageService instance for downloading stems/lyrics
|
||||
"""
|
||||
brand_code = result.get('brand_code')
|
||||
base_name = f"{job.artist} - {job.title}"
|
||||
|
||||
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
|
||||
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
|
||||
base_name = f"{safe_artist} - {safe_title}"
|
||||
|
||||
# Check if we should preserve existing brand code (Batch 6: --keep-brand-code)
|
||||
keep_brand_code = getattr(job, 'keep_brand_code', None)
|
||||
if keep_brand_code:
|
||||
@@ -1108,29 +1123,63 @@ async def _prepare_distribution_directory(
|
||||
log_progress(f"Downloaded {stem_key} to stems/")
|
||||
|
||||
# --- Download lyrics files to lyrics/ subfolder ---
|
||||
# Use proper filenames matching local CLI output structure
|
||||
|
||||
# Common lyrics file types
|
||||
lyrics_file_keys = ['lrc', 'ass', 'srt', 'txt', 'json', 'original_json', 'corrected_json']
|
||||
for lyrics_key in lyrics_file_keys:
|
||||
# Map job.files.lyrics keys to proper local filenames
|
||||
# Format: (job_files_key, local_filename_pattern)
|
||||
lyrics_file_mappings = [
|
||||
# Karaoke output files
|
||||
('lrc', f"{base_name} (Karaoke).lrc"),
|
||||
('ass', f"{base_name} (Karaoke).ass"),
|
||||
# Corrections and transcription files
|
||||
('corrections', f"{base_name} (Lyrics Corrections).json"),
|
||||
('corrected_txt', f"{base_name} (Lyrics Corrected).txt"),
|
||||
('uncorrected', f"{base_name} (Lyrics Uncorrected).txt"),
|
||||
# Reference lyrics from various sources
|
||||
('reference_genius', f"{base_name} (Lyrics Genius).txt"),
|
||||
('reference_spotify', f"{base_name} (Lyrics Spotify).txt"),
|
||||
('reference_musixmatch', f"{base_name} (Lyrics Musixmatch).txt"),
|
||||
('reference_lrclib', f"{base_name} (Lyrics Lrclib).txt"),
|
||||
]
|
||||
|
||||
for lyrics_key, local_filename in lyrics_file_mappings:
|
||||
if lyrics.get(lyrics_key):
|
||||
# Determine proper extension
|
||||
ext_map = {
|
||||
'lrc': '.lrc',
|
||||
'ass': '.ass',
|
||||
'srt': '.srt',
|
||||
'txt': '.txt',
|
||||
'json': '.json',
|
||||
'original_json': '_original.json',
|
||||
'corrected_json': '_corrected.json',
|
||||
}
|
||||
ext = ext_map.get(lyrics_key, f'.{lyrics_key}')
|
||||
dest_path = os.path.join(lyrics_dir, f"{base_name} (Karaoke){ext}")
|
||||
dest_path = os.path.join(lyrics_dir, local_filename)
|
||||
try:
|
||||
storage.download_file(lyrics[lyrics_key], dest_path)
|
||||
log_progress(f"Downloaded {lyrics_key} to lyrics/")
|
||||
log_progress(f"Downloaded {lyrics_key} to lyrics/{local_filename}")
|
||||
except Exception as e:
|
||||
log_progress(f"Could not download {lyrics_key}: {e}")
|
||||
|
||||
# --- Download (With Vocals).mkv to lyrics/ subfolder ---
|
||||
# Local CLI places the karaoke video with vocals in the lyrics folder
|
||||
videos = job.file_urls.get('videos', {})
|
||||
if videos.get('with_vocals'):
|
||||
with_vocals_dest = os.path.join(lyrics_dir, f"{base_name} (With Vocals).mkv")
|
||||
try:
|
||||
storage.download_file(videos['with_vocals'], with_vocals_dest)
|
||||
log_progress(f"Downloaded with_vocals to lyrics/{base_name} (With Vocals).mkv")
|
||||
except Exception as e:
|
||||
log_progress(f"Could not download with_vocals to lyrics/: {e}")
|
||||
|
||||
# --- Download preview ASS files to lyrics/previews/ subfolder ---
|
||||
# Preview files are stored in jobs/{job_id}/previews/ in GCS
|
||||
previews_dir = os.path.join(lyrics_dir, "previews")
|
||||
os.makedirs(previews_dir, exist_ok=True)
|
||||
|
||||
# List all preview files from GCS and download them
|
||||
try:
|
||||
preview_prefix = f"jobs/{job_id}/previews/"
|
||||
preview_files = storage.list_files(preview_prefix)
|
||||
for blob_name in preview_files:
|
||||
if blob_name.endswith('.ass'):
|
||||
filename = os.path.basename(blob_name)
|
||||
dest_path = os.path.join(previews_dir, filename)
|
||||
storage.download_file(blob_name, dest_path)
|
||||
log_progress(f"Downloaded preview {filename} to lyrics/previews/")
|
||||
except Exception as e:
|
||||
log_progress(f"Could not download preview files: {e}")
|
||||
|
||||
log_progress("Distribution directory prepared with stems/ and lyrics/ subfolders")
|
||||
|
||||
|
||||
|
||||
@@ -27,6 +27,7 @@ from backend.models.job import JobStatus
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.services.tracing import job_span, add_span_event
|
||||
from karaoke_gen.utils import sanitize_filename
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -367,15 +368,70 @@ class VideoWorkerOrchestrator:
|
||||
if not output.success:
|
||||
raise Exception(f"Encoding failed: {output.error_message}")
|
||||
|
||||
# Store results
|
||||
# Store results - for GCE backend, these are GCS blob paths that need to be downloaded
|
||||
self.result.final_video = output.lossless_4k_mp4_path
|
||||
self.result.final_video_mkv = output.lossless_mkv_path
|
||||
self.result.final_video_lossy = output.lossy_4k_mp4_path
|
||||
self.result.final_video_720p = output.lossy_720p_mp4_path
|
||||
self.result.encoding_time_seconds = output.encoding_time_seconds
|
||||
|
||||
# For GCE encoding, download the encoded files from GCS to local directory
|
||||
# This is required for YouTube upload and other local file operations
|
||||
if encoding_backend.name == "gce" and self.storage:
|
||||
await self._download_gce_encoded_files(output)
|
||||
|
||||
self.job_log.info(f"Encoding complete ({encoding_backend.name}) in {output.encoding_time_seconds:.1f}s")
|
||||
|
||||
async def _download_gce_encoded_files(self, output):
|
||||
"""
|
||||
Download GCE-encoded files from GCS to the local output directory.
|
||||
|
||||
GCE encoding stores files in GCS and returns blob paths like:
|
||||
'jobs/{job_id}/finals/Artist - Title (Final Karaoke Lossless 4k).mp4'
|
||||
|
||||
This method downloads those files locally so that subsequent stages
|
||||
(YouTube upload, etc.) can access them as local files.
|
||||
|
||||
Args:
|
||||
output: EncodingOutput from the GCE backend with GCS blob paths
|
||||
"""
|
||||
self.job_log.info("Downloading GCE-encoded files from GCS")
|
||||
|
||||
# Map of result attributes to download
|
||||
file_mappings = [
|
||||
('lossless_4k_mp4_path', 'final_video'),
|
||||
('lossless_mkv_path', 'final_video_mkv'),
|
||||
('lossy_4k_mp4_path', 'final_video_lossy'),
|
||||
('lossy_720p_mp4_path', 'final_video_720p'),
|
||||
]
|
||||
|
||||
downloaded_count = 0
|
||||
for output_attr, result_attr in file_mappings:
|
||||
gcs_path = getattr(output, output_attr, None)
|
||||
if not gcs_path:
|
||||
continue
|
||||
|
||||
# Extract filename from GCS path
|
||||
filename = os.path.basename(gcs_path)
|
||||
local_path = os.path.join(self.config.output_dir, filename)
|
||||
|
||||
try:
|
||||
self.job_log.info(f"Downloading {filename} from GCS")
|
||||
self.storage.download_file(gcs_path, local_path)
|
||||
|
||||
# Update the result to point to local file
|
||||
setattr(self.result, result_attr, local_path)
|
||||
downloaded_count += 1
|
||||
self.job_log.info(f"Downloaded {filename} to {local_path}")
|
||||
|
||||
except Exception as e:
|
||||
self.job_log.error(f"Failed to download {filename}: {e}")
|
||||
# Clear the result attribute so downstream doesn't try to use invalid GCS path
|
||||
setattr(self.result, result_attr, None)
|
||||
# Don't fail - some formats might not be generated
|
||||
|
||||
self.job_log.info(f"Downloaded {downloaded_count} encoded files from GCS")
|
||||
|
||||
async def _run_organization(self):
|
||||
"""Run the organization stage (brand code generation)."""
|
||||
self.job_log.info("Starting organization stage")
|
||||
@@ -483,7 +539,10 @@ class VideoWorkerOrchestrator:
|
||||
self.job_log.warning("Dropbox not configured, skipping upload")
|
||||
return
|
||||
|
||||
base_name = f"{self.config.artist} - {self.config.title}"
|
||||
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
safe_artist = sanitize_filename(self.config.artist) if self.config.artist else "Unknown"
|
||||
safe_title = sanitize_filename(self.config.title) if self.config.title else "Unknown"
|
||||
base_name = f"{safe_artist} - {safe_title}"
|
||||
folder_name = f"{self.result.brand_code or 'TRACK-0000'} - {base_name}"
|
||||
remote_folder = f"{self.config.dropbox_path}/{folder_name}"
|
||||
|
||||
@@ -582,7 +641,10 @@ def create_orchestrator_config_from_job(
|
||||
Returns:
|
||||
OrchestratorConfig for the orchestrator
|
||||
"""
|
||||
base_name = f"{job.artist} - {job.title}"
|
||||
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
|
||||
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
|
||||
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
|
||||
base_name = f"{safe_artist} - {safe_title}"
|
||||
|
||||
# Determine instrumental file path
|
||||
instrumental_selection = job.state_data.get('instrumental_selection', 'clean')
|
||||
|
||||
@@ -20,6 +20,30 @@ GET /health
|
||||
|
||||
No auth required. Returns service status.
|
||||
|
||||
#### Encoding Worker Health
|
||||
|
||||
```http
|
||||
GET /health/encoding-worker
|
||||
```
|
||||
|
||||
No auth required. Returns GCE encoding worker status for frontend display.
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"available": true,
|
||||
"status": "ok",
|
||||
"version": "0.95.1",
|
||||
"active_jobs": 0,
|
||||
"queue_length": 0
|
||||
}
|
||||
```
|
||||
|
||||
Status values:
|
||||
- `ok` - Worker is healthy and available
|
||||
- `offline` - Worker is unavailable (includes `error` field)
|
||||
- `not_configured` - Encoding worker not configured on backend
|
||||
|
||||
### Jobs
|
||||
|
||||
#### Create Job with Upload
|
||||
@@ -114,11 +138,52 @@ Content-Type: application/json
|
||||
|
||||
### Audio Search
|
||||
|
||||
#### Search for Audio
|
||||
|
||||
```http
|
||||
GET /api/audio-search?q=song+name
|
||||
POST /api/audio-search/search
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"artist": "Artist Name",
|
||||
"title": "Song Title",
|
||||
"auto_download": false,
|
||||
"theme_id": "nomad",
|
||||
"display_artist": "Display Artist (optional)",
|
||||
"display_title": "Display Title (optional)"
|
||||
}
|
||||
```
|
||||
|
||||
Search for songs (flacfetch integration).
|
||||
Searches for audio sources (flacfetch integration). Creates a job and returns search results.
|
||||
|
||||
**Key fields:**
|
||||
- `artist`, `title` - Used to search for audio on trackers
|
||||
- `display_artist`, `display_title` - Optional overrides for how artist/title appear on title screens and filenames. If omitted, search values are used.
|
||||
- `auto_download` - If true, automatically selects best result and starts processing
|
||||
- `theme_id` - Video theme to use
|
||||
|
||||
**Use case:** Search for "Jeremy Kushnier" but display "Footloose (Broadway Cast)" on the karaoke video.
|
||||
|
||||
#### Get Search Results
|
||||
|
||||
```http
|
||||
GET /api/audio-search/{job_id}/results
|
||||
```
|
||||
|
||||
Returns cached search results for a job awaiting audio selection.
|
||||
|
||||
#### Select Audio Source
|
||||
|
||||
```http
|
||||
POST /api/audio-search/{job_id}/select
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"selection_index": 0
|
||||
}
|
||||
```
|
||||
|
||||
Selects an audio source from the search results and starts processing.
|
||||
|
||||
### Themes
|
||||
|
||||
@@ -487,6 +552,66 @@ Response:
|
||||
}
|
||||
```
|
||||
|
||||
### Email Notifications (Admin)
|
||||
|
||||
#### Get Completion Message
|
||||
|
||||
```http
|
||||
GET /api/admin/jobs/{job_id}/completion-message
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
```
|
||||
|
||||
Returns the rendered completion message for a job (for copying to clipboard).
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"job_id": "abc123",
|
||||
"message": "Hi there! Your karaoke video is ready...",
|
||||
"subject": "Your Karaoke Video is Ready! 🎤",
|
||||
"youtube_url": "https://youtube.com/watch?v=...",
|
||||
"dropbox_url": "https://dropbox.com/..."
|
||||
}
|
||||
```
|
||||
|
||||
#### Send Completion Email
|
||||
|
||||
```http
|
||||
POST /api/admin/jobs/{job_id}/send-completion-email
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"to_email": "customer@example.com",
|
||||
"cc_admin": true
|
||||
}
|
||||
```
|
||||
|
||||
Sends the job completion email to the specified address. When `cc_admin` is true (default), CCs gen@nomadkaraoke.com.
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"job_id": "abc123",
|
||||
"to_email": "customer@example.com",
|
||||
"message": "Completion email sent successfully"
|
||||
}
|
||||
```
|
||||
|
||||
### Internal Email Endpoints
|
||||
|
||||
These endpoints are used by Cloud Tasks for automated notifications.
|
||||
|
||||
#### Check Idle Reminder
|
||||
|
||||
```http
|
||||
POST /api/internal/jobs/{job_id}/check-idle-reminder
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
```
|
||||
|
||||
Called by Cloud Tasks 5 minutes after a job enters a blocking state (awaiting_review or awaiting_instrumental_selection). Sends a reminder email if the user is still idle and no reminder has been sent yet.
|
||||
|
||||
## Rate Limits
|
||||
|
||||
No rate limits currently implemented.
|
||||
|
||||
@@ -31,7 +31,8 @@
|
||||
└──────────┘ └──────────┘ └──────────────┘ └──────────────┘
|
||||
|
||||
(*) GCE Encoding Worker: C4-standard-8 VM with Intel Granite Rapids for
|
||||
high-performance FFmpeg encoding. Optional - falls back to Cloud Run.
|
||||
high-performance FFmpeg encoding. Used for both final video encoding and
|
||||
preview video generation. Uses LocalEncodingService via GCS wheel deployment.
|
||||
```
|
||||
|
||||
## Processing Pipeline
|
||||
@@ -57,7 +58,7 @@
|
||||
▼
|
||||
4. HUMAN REVIEW (AWAITING_REVIEW)
|
||||
• React UI for lyrics correction
|
||||
• Preview video generation
|
||||
• Preview video generation (optionally via GCE for speed)
|
||||
• Submit corrections
|
||||
▼
|
||||
5. VIDEO RENDERING
|
||||
@@ -154,6 +155,8 @@ LyricsTranscriber LyricsTranscriber
|
||||
| Vertex AI | Agentic AI correction (Gemini 3 Flash) | Default on |
|
||||
| Genius | Reference lyrics | Yes |
|
||||
| YouTube API | Video upload | Optional |
|
||||
| SendGrid | Email notifications | Optional |
|
||||
| Cloud Tasks | Delayed task scheduling (idle reminders) | Optional |
|
||||
|
||||
## Firestore Collections
|
||||
|
||||
@@ -190,7 +193,8 @@ The Video Worker uses an orchestrator pattern to ensure all features work regard
|
||||
│ └─ YouTube upload │
|
||||
│ │
|
||||
│ Stage 5: NOTIFICATIONS │
|
||||
│ └─ Discord webhooks │
|
||||
│ ├─ Discord webhooks │
|
||||
│ └─ Email notifications (completion, reminders) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
@@ -208,6 +212,9 @@ The Video Worker uses an orchestrator pattern to ensure all features work regard
|
||||
| `local_encoding_service.py` | FFmpeg encoding (Cloud Run) |
|
||||
| `encoding_interface.py` | Abstract interface + GCE backend |
|
||||
| `video_worker_orchestrator.py` | Stage coordination |
|
||||
| `email_service.py` | SendGrid email delivery with CC support |
|
||||
| `template_service.py` | GCS-backed email templates |
|
||||
| `job_notification_service.py` | Email orchestration (completion, reminders) |
|
||||
|
||||
## Tech Stack
|
||||
|
||||
@@ -216,5 +223,7 @@ The Video Worker uses an orchestrator pattern to ensure all features work regard
|
||||
- **Database**: Firestore
|
||||
- **Storage**: Google Cloud Storage
|
||||
- **Secrets**: Google Secret Manager
|
||||
- **Task Queues**: Cloud Tasks (idle reminders)
|
||||
- **Email**: SendGrid
|
||||
- **IaC**: Pulumi
|
||||
- **CI/CD**: GitHub Actions
|
||||
@@ -0,0 +1,701 @@
|
||||
# Nomad Karaoke Brand Style Guide
|
||||
|
||||
> **Version**: 1.1
|
||||
> **Last Updated**: 2026-01-07
|
||||
> **Purpose**: Unified brand identity for all Nomad Karaoke products and communications
|
||||
|
||||
This style guide is designed for both human designers and LLM agents implementing the Nomad Karaoke brand across web applications, emails, marketing materials, and other touchpoints.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Brand Overview](#brand-overview)
|
||||
2. [Color Palette](#color-palette)
|
||||
3. [Typography](#typography)
|
||||
4. [Logo Usage](#logo-usage)
|
||||
5. [Iconography](#iconography)
|
||||
6. [UI Components](#ui-components)
|
||||
7. [Layout & Spacing](#layout--spacing)
|
||||
8. [Email Design](#email-design)
|
||||
9. [Tone & Voice](#tone--voice)
|
||||
10. [Light/Dark Theme Requirements](#lightdark-theme-requirements)
|
||||
11. [Implementation Reference](#implementation-reference)
|
||||
|
||||
---
|
||||
|
||||
## Brand Overview
|
||||
|
||||
### Brand Personality
|
||||
Nomad Karaoke is **approachable**, **professional**, and **modern**. We make sophisticated technology feel accessible and fun. Our brand bridges the gap between serious audio engineering and the joy of karaoke.
|
||||
|
||||
### Core Values
|
||||
- **Approachable**: Technology that anyone can use
|
||||
- **Professional**: Studio-quality output
|
||||
- **Playful**: Karaoke should be fun
|
||||
- **Trustworthy**: Reliable service and results
|
||||
|
||||
### Brand Keywords
|
||||
`modern` `vibrant` `accessible` `professional` `musical` `AI-powered` `creative`
|
||||
|
||||
---
|
||||
|
||||
## Color Palette
|
||||
|
||||
### Primary Colors
|
||||
|
||||
| Name | Hex | RGB | Usage |
|
||||
|------|-----|-----|-------|
|
||||
| **Hot Pink** | `#ff7acc` | rgb(255, 122, 204) | Primary brand color, logo, primary buttons, key accents |
|
||||
| **Gold** | `#ffdf6b` | rgb(255, 223, 107) | Secondary accent, highlights, success states, premium feel |
|
||||
|
||||
### Supporting Colors
|
||||
|
||||
| Name | Hex | RGB | Usage |
|
||||
|------|-----|-----|-------|
|
||||
| **Deep Purple** | `#8b5cf6` | rgb(139, 92, 246) | Gradients, special features, beta badges |
|
||||
| **Electric Blue** | `#3b82f6` | rgb(59, 130, 246) | Links, interactive elements, info states |
|
||||
| **Soft Pink** | `#ec4899` | rgb(236, 72, 153) | Gradient end points, hover states |
|
||||
|
||||
### Semantic Colors
|
||||
|
||||
| Name | Hex | Usage |
|
||||
|------|-----|-------|
|
||||
| **Success** | `#22c55e` | Completed states, positive feedback |
|
||||
| **Warning** | `#f59e0b` | Attention needed, caution states |
|
||||
| **Error** | `#ef4444` | Errors, destructive actions |
|
||||
| **Info** | `#3b82f6` | Informational messages |
|
||||
|
||||
### Dark Theme (Default)
|
||||
|
||||
| Name | Hex | Usage |
|
||||
|------|-----|-------|
|
||||
| **Background** | `#0f0f0f` | Page background |
|
||||
| **Card Background** | `#1a1a1a` | Card surfaces |
|
||||
| **Card Border** | `#2a2a2a` | Subtle borders |
|
||||
| **Text Primary** | `#e5e5e5` | Main content text |
|
||||
| **Text Muted** | `#888888` | Secondary text, labels |
|
||||
| **Text Subtle** | `#666666` | Tertiary text, hints |
|
||||
|
||||
### Light Theme
|
||||
|
||||
| Name | Hex | Usage |
|
||||
|------|-----|-------|
|
||||
| **Background** | `#f8fafc` | Page background |
|
||||
| **Card Background** | `#ffffff` | Card surfaces |
|
||||
| **Card Border** | `#e2e8f0` | Subtle borders |
|
||||
| **Text Primary** | `#1e293b` | Main content text |
|
||||
| **Text Muted** | `#64748b` | Secondary text, labels |
|
||||
|
||||
### Gradients
|
||||
|
||||
#### Primary Gradient (Brand Gradient)
|
||||
```css
|
||||
background: linear-gradient(135deg, #3b82f6 0%, #8b5cf6 50%, #ff7acc 100%);
|
||||
```
|
||||
Use for: Hero text, feature highlights, premium elements. The gradient flows from blue through purple to the brand pink.
|
||||
|
||||
#### Radial Background Gradient (Landing Pages)
|
||||
```css
|
||||
/* Layered radial gradients using brand colors */
|
||||
body::after {
|
||||
content: '';
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
z-index: -1;
|
||||
background:
|
||||
radial-gradient(ellipse at 20% 0%, rgba(139, 92, 246, 0.12) 0%, transparent 50%), /* Purple glow top-left */
|
||||
radial-gradient(ellipse at 80% 100%, rgba(255, 122, 204, 0.10) 0%, transparent 50%), /* Pink glow bottom-right */
|
||||
radial-gradient(ellipse at 50% 50%, rgba(255, 223, 107, 0.03) 0%, transparent 70%); /* Gold subtle center */
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
/* Optional: Add subtle pulsing animation for ambient glow effect */
|
||||
@keyframes ambient-glow {
|
||||
0%, 100% { opacity: 1; }
|
||||
50% { opacity: 0.7; }
|
||||
}
|
||||
body::after {
|
||||
animation: ambient-glow 8s ease-in-out infinite;
|
||||
}
|
||||
```
|
||||
Use for: Landing page backgrounds, hero sections. Creates depth with brand colors without being distracting.
|
||||
|
||||
**Light theme variant**: Reduce opacity values (0.08, 0.06, 0.02) for subtler effect on light backgrounds.
|
||||
|
||||
#### Beta/Special Gradient
|
||||
```css
|
||||
background: linear-gradient(135deg, #8b5cf6, #ec4899);
|
||||
```
|
||||
Use for: Beta badges, special promotions, exclusive features.
|
||||
|
||||
---
|
||||
|
||||
## Typography
|
||||
|
||||
### Font Stack
|
||||
|
||||
#### Primary (Sans-serif)
|
||||
```css
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
|
||||
```
|
||||
Use system fonts for optimal performance and native feel.
|
||||
|
||||
#### Monospace (Code/Technical)
|
||||
```css
|
||||
font-family: "SF Mono", Monaco, "Cascadia Code", "Roboto Mono", monospace;
|
||||
```
|
||||
Use for: Code snippets, technical information, job IDs.
|
||||
|
||||
### Type Scale
|
||||
|
||||
| Name | Size | Weight | Line Height | Usage |
|
||||
|------|------|--------|-------------|-------|
|
||||
| **Display** | 48-60px | 800 (extrabold) | 1.1 | Hero headlines |
|
||||
| **H1** | 36-48px | 700 (bold) | 1.2 | Page titles |
|
||||
| **H2** | 28-32px | 700 (bold) | 1.3 | Section headers |
|
||||
| **H3** | 20-24px | 600 (semibold) | 1.4 | Card titles |
|
||||
| **Body** | 16px | 400 (regular) | 1.6 | Main content |
|
||||
| **Body Small** | 14px | 400 (regular) | 1.5 | Secondary content |
|
||||
| **Caption** | 12px | 400 (regular) | 1.4 | Labels, hints |
|
||||
|
||||
### Text Styling
|
||||
|
||||
- **Gradient Text**: Apply brand gradient to hero headlines
|
||||
- **Antialiasing**: Always use `-webkit-font-smoothing: antialiased`
|
||||
- **Letter Spacing**: Default, slightly increased for ALL CAPS text
|
||||
|
||||
---
|
||||
|
||||
## Logo Usage
|
||||
|
||||
**Important**: Always use the Nomad Karaoke logo rather than generic emojis (like 🎤) in headers, navigation, and brand touchpoints.
|
||||
|
||||
### Primary Logo
|
||||
The Nomad Karaoke logo consists of:
|
||||
1. **Wordmark**: "NOMAD" stylized letters (pink #ff7acc)
|
||||
2. **Tagline**: "KARAOKE" below (pink #ff7acc)
|
||||
3. **Subline**: "WHERE EVERY SONG'S YOURS, SINGING!" (gold #ffdf6b)
|
||||
|
||||
### Logo Format Priority
|
||||
|
||||
1. **SVG (Preferred)**: Use `nomad-karaoke-logo.svg` for all web applications
|
||||
- Vector format scales perfectly at any size
|
||||
- Smaller file size
|
||||
- Supports both light and dark themes
|
||||
- Located in: `frontend/public/` directory
|
||||
|
||||
2. **PNG**: Use for contexts that don't support SVG
|
||||
- `nomad-logo.png` - Standard PNG fallback
|
||||
|
||||
3. **GIF (Email Only)**: Use the hosted GIF **only** for email templates
|
||||
- URL: `https://beveradb.github.io/public-images/Nomad-Karaoke-Logo-small-indexed-websafe-rectangle.gif`
|
||||
- This hosted URL exists because emails cannot embed local assets
|
||||
- Do NOT use this URL in web applications - use the local SVG instead
|
||||
|
||||
### Logo Files
|
||||
- `nomad-karaoke-logo.svg` - Full vector logo (**primary choice**)
|
||||
- `nomad-logo.png` - PNG fallback
|
||||
- `favicon.ico`, `favicon-16x16.png`, `favicon-32x32.png` - Favicons
|
||||
- `apple-touch-icon.png` - iOS home screen icon
|
||||
|
||||
### Logo Placement
|
||||
- **Navigation**: Left-aligned, 32-40px height, use SVG
|
||||
- **Footer**: Smaller, 24-32px height, use SVG
|
||||
- **Email Header**: Centered, 130-180px width, use hosted GIF URL
|
||||
|
||||
### Logo Clear Space
|
||||
Maintain clear space equal to the "N" letter height around all sides of the logo.
|
||||
|
||||
### Logo Don'ts
|
||||
- Don't use emojis (🎤, 🎵) instead of the logo
|
||||
- Don't change the logo colors
|
||||
- Don't stretch or distort the logo
|
||||
- Don't add effects (shadows, gradients, etc.)
|
||||
- Don't place on busy backgrounds without contrast
|
||||
- Don't use the logo smaller than 24px height
|
||||
- Don't use the hosted GIF URL in web applications (use SVG)
|
||||
|
||||
---
|
||||
|
||||
## Iconography
|
||||
|
||||
### Icon Library
|
||||
Use [Lucide Icons](https://lucide.dev/) for UI icons. They are:
|
||||
- Open source
|
||||
- Consistent stroke width
|
||||
- Good visual balance
|
||||
|
||||
### Common Icons
|
||||
|
||||
| Icon | Name | Usage |
|
||||
|------|------|-------|
|
||||
| `MicVocal` | Microphone | Brand representation in nav (when logo not used) |
|
||||
| `Music` | Music note | Audio/song related |
|
||||
| `Music2` | Double notes | Empty states for songs |
|
||||
| `Sparkles` | Stars | AI/magic features |
|
||||
| `Video` | Video | Video output |
|
||||
| `Youtube` | YouTube logo | YouTube integration |
|
||||
| `Gift` | Gift box | Free credits, promotions |
|
||||
| `Check` | Checkmark | Success, confirmation |
|
||||
| `Loader2` | Spinner | Loading states |
|
||||
|
||||
### Icon Sizing
|
||||
- **Small**: 16px (inline with text)
|
||||
- **Medium**: 20-24px (buttons, lists)
|
||||
- **Large**: 32-48px (feature highlights)
|
||||
- **XL**: 64px+ (empty states, hero)
|
||||
|
||||
### Icon Colors
|
||||
- Match text color in context
|
||||
- Use brand pink (#ff7acc) for emphasis
|
||||
- Use muted color for decorative icons
|
||||
|
||||
---
|
||||
|
||||
## UI Components
|
||||
|
||||
### Buttons
|
||||
|
||||
#### Primary Button (CTA)
|
||||
```css
|
||||
background-color: #ff7acc;
|
||||
color: #ffffff;
|
||||
padding: 14px 28px;
|
||||
border-radius: 12px;
|
||||
font-weight: 600;
|
||||
box-shadow: 0 0 20px rgba(255, 122, 204, 0.4);
|
||||
transition: all 0.2s ease;
|
||||
|
||||
/* Hover */
|
||||
background-color: #ff5bb8;
|
||||
box-shadow: 0 0 30px rgba(255, 122, 204, 0.6);
|
||||
```
|
||||
|
||||
#### Secondary Button
|
||||
```css
|
||||
background-color: #252525;
|
||||
color: #e5e5e5;
|
||||
padding: 12px 24px;
|
||||
border-radius: 8px;
|
||||
border: 1px solid #2a2a2a;
|
||||
|
||||
/* Hover */
|
||||
background-color: #333333;
|
||||
```
|
||||
|
||||
#### Ghost Button
|
||||
```css
|
||||
background-color: transparent;
|
||||
color: #888888;
|
||||
padding: 8px 16px;
|
||||
|
||||
/* Hover */
|
||||
color: #e5e5e5;
|
||||
background-color: rgba(255, 255, 255, 0.05);
|
||||
```
|
||||
|
||||
### Cards
|
||||
|
||||
```css
|
||||
background-color: var(--card); /* #1a1a1a dark / #ffffff light */
|
||||
border: 1px solid var(--card-border); /* #2a2a2a dark / #e2e8f0 light */
|
||||
border-radius: 12px;
|
||||
padding: 24px;
|
||||
|
||||
/* Hover effect (optional) */
|
||||
transition: transform 0.2s ease, box-shadow 0.2s ease;
|
||||
transform: translateY(-4px);
|
||||
box-shadow: 0 20px 40px rgba(0, 0, 0, 0.3);
|
||||
```
|
||||
|
||||
### Form Inputs
|
||||
|
||||
```css
|
||||
background-color: #0f0f0f; /* Darker than card */
|
||||
border: 1px solid #2a2a2a;
|
||||
border-radius: 8px;
|
||||
padding: 12px 16px;
|
||||
color: #e5e5e5;
|
||||
|
||||
/* Focus */
|
||||
border-color: #ff7acc;
|
||||
box-shadow: 0 0 0 2px rgba(255, 122, 204, 0.2);
|
||||
outline: none;
|
||||
|
||||
/* Placeholder */
|
||||
color: #666666;
|
||||
```
|
||||
|
||||
### Badges
|
||||
|
||||
#### Beta Badge
|
||||
```css
|
||||
background: linear-gradient(135deg, #8b5cf6, #ec4899);
|
||||
color: #ffffff;
|
||||
padding: 4px 12px;
|
||||
border-radius: 20px;
|
||||
font-size: 12px;
|
||||
font-weight: 600;
|
||||
```
|
||||
|
||||
#### Status Badges
|
||||
Use semantic colors with 20% opacity backgrounds:
|
||||
- Success: `background: rgba(34, 197, 94, 0.2); color: #22c55e;`
|
||||
- Warning: `background: rgba(245, 158, 11, 0.2); color: #f59e0b;`
|
||||
- Error: `background: rgba(239, 68, 68, 0.2); color: #ef4444;`
|
||||
|
||||
---
|
||||
|
||||
## Layout & Spacing
|
||||
|
||||
### Spacing Scale
|
||||
Use a consistent 4px base unit:
|
||||
|
||||
| Name | Value | Usage |
|
||||
|------|-------|-------|
|
||||
| `xs` | 4px | Tight spacing |
|
||||
| `sm` | 8px | Small gaps |
|
||||
| `md` | 16px | Default gaps |
|
||||
| `lg` | 24px | Section padding |
|
||||
| `xl` | 32px | Large sections |
|
||||
| `2xl` | 48px | Major sections |
|
||||
| `3xl` | 64px | Page sections |
|
||||
|
||||
### Container Widths
|
||||
- **Max content width**: 1200px (6xl)
|
||||
- **Text content**: 672px (2xl) for readability
|
||||
- **Form width**: 448px (md) for focused input
|
||||
|
||||
### Grid
|
||||
Use CSS Grid or Flexbox with responsive breakpoints:
|
||||
- Mobile: 1 column
|
||||
- Tablet (md: 768px): 2 columns
|
||||
- Desktop (lg: 1024px): 2-4 columns
|
||||
|
||||
### Border Radius
|
||||
- **Small** (inputs, small buttons): 8px
|
||||
- **Medium** (cards, buttons): 12px
|
||||
- **Large** (feature cards, modals): 16px
|
||||
- **Pill** (badges, tags): 20px or 9999px
|
||||
|
||||
---
|
||||
|
||||
## Email Design
|
||||
|
||||
### Email Structure
|
||||
|
||||
```html
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
background-color: #ffffff;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<!-- Header with logo -->
|
||||
<div style="text-align: center; padding: 20px 0;">
|
||||
<img src="logo-url" alt="Nomad Karaoke" width="150" />
|
||||
</div>
|
||||
|
||||
<!-- Content -->
|
||||
<div>...</div>
|
||||
|
||||
<!-- Footer -->
|
||||
<div style="margin-top: 30px; padding-top: 20px; border-top: 1px solid #ff7acc; font-size: 12px; color: #666;">
|
||||
...
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
```
|
||||
|
||||
### Email Colors
|
||||
- **Background**: `#ffffff` (white for email client compatibility)
|
||||
- **Text**: `#333333`
|
||||
- **Primary Button**: `#ff7acc` with white text
|
||||
- **Links**: `#3b82f6`
|
||||
- **Dividers**: `#ff7acc` (brand pink) or `#eeeeee`
|
||||
- **Footer text**: `#666666`
|
||||
|
||||
### Email Button
|
||||
```html
|
||||
<a href="URL" style="display: inline-block; background-color: #ff7acc; color: #ffffff; padding: 14px 28px; text-decoration: none; border-radius: 8px; font-weight: 600;">
|
||||
Button Text
|
||||
</a>
|
||||
```
|
||||
|
||||
### Email Header
|
||||
Always use the Nomad Karaoke logo image in email headers, not an emoji.
|
||||
|
||||
```html
|
||||
<div style="text-align: center; padding: 20px 0;">
|
||||
<a href="https://nomadkaraoke.com">
|
||||
<img src="https://beveradb.github.io/public-images/Nomad-Karaoke-Logo-small-indexed-websafe-rectangle.gif"
|
||||
alt="Nomad Karaoke"
|
||||
style="max-width: 180px; height: auto; border-radius: 10px;" />
|
||||
</a>
|
||||
</div>
|
||||
```
|
||||
|
||||
### Email Signature
|
||||
Include consistent signature with:
|
||||
- Profile image
|
||||
- Name and title
|
||||
- Social media icons
|
||||
- Contact information
|
||||
- Pink (#ff7acc) accent lines
|
||||
|
||||
---
|
||||
|
||||
## Tone & Voice
|
||||
|
||||
### Writing Style
|
||||
- **Friendly but professional**: "Hi there!" not "Dear Customer"
|
||||
- **Clear and concise**: Get to the point
|
||||
- **Encouraging**: Celebrate user accomplishments
|
||||
- **Helpful**: Guide users through actions
|
||||
|
||||
### Example Phrases
|
||||
|
||||
| Instead of... | Use... |
|
||||
|---------------|--------|
|
||||
| "Your order has been processed" | "Your karaoke video is ready!" |
|
||||
| "Error occurred" | "Something went wrong - we're on it!" |
|
||||
| "Please wait" | "Just a moment..." |
|
||||
| "Submit" | "Create Karaoke" or "Get Started" |
|
||||
|
||||
### Emoji Usage
|
||||
- Use sparingly for warmth
|
||||
- Common: 🎤 (brand), 🎵 (music), ✨ (magic), 🎉 (celebration)
|
||||
- Avoid in error messages
|
||||
- Email subject lines: One emoji max
|
||||
|
||||
---
|
||||
|
||||
## Light/Dark Theme Requirements
|
||||
|
||||
### Overview
|
||||
All Nomad Karaoke web application screens must support both light and dark themes. Dark theme is the default, but users should have easy access to toggle between themes.
|
||||
|
||||
### Implementation Requirements
|
||||
|
||||
1. **Theme Toggle**: Every screen/page must include a visible theme toggle
|
||||
- Location: Typically in the header or settings area
|
||||
- Use `next-themes` library for Next.js applications
|
||||
- Toggle should be easily discoverable
|
||||
|
||||
2. **Default Theme**: Dark mode
|
||||
- Respects system preference on first visit
|
||||
- User preference is persisted
|
||||
|
||||
3. **Color Consistency**: All brand colors must work in both themes
|
||||
- Brand pink (#ff7acc) remains consistent
|
||||
- Background, card, and text colors adapt per theme tables above
|
||||
- Ensure sufficient contrast in both modes
|
||||
|
||||
### next-themes Setup (Next.js)
|
||||
|
||||
```tsx
|
||||
// app/layout.tsx or _app.tsx
|
||||
import { ThemeProvider } from 'next-themes'
|
||||
|
||||
export default function RootLayout({ children }) {
|
||||
return (
|
||||
<html suppressHydrationWarning>
|
||||
<body>
|
||||
<ThemeProvider attribute="class" defaultTheme="dark" enableSystem>
|
||||
{children}
|
||||
</ThemeProvider>
|
||||
</body>
|
||||
</html>
|
||||
)
|
||||
}
|
||||
|
||||
// Theme toggle component
|
||||
import { useTheme } from 'next-themes'
|
||||
import { Sun, Moon } from 'lucide-react'
|
||||
|
||||
export function ThemeToggle() {
|
||||
const { theme, setTheme } = useTheme()
|
||||
|
||||
return (
|
||||
<button
|
||||
onClick={() => setTheme(theme === 'dark' ? 'light' : 'dark')}
|
||||
className="p-2 rounded-lg hover:bg-white/10"
|
||||
aria-label="Toggle theme"
|
||||
>
|
||||
{theme === 'dark' ? <Sun size={20} /> : <Moon size={20} />}
|
||||
</button>
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
### CSS Variables for Theming
|
||||
|
||||
```css
|
||||
/* Dark theme (default) */
|
||||
:root {
|
||||
--background: #0f0f0f;
|
||||
--card: #1a1a1a;
|
||||
--card-border: #2a2a2a;
|
||||
--text: #e5e5e5;
|
||||
--text-muted: #888888;
|
||||
}
|
||||
|
||||
/* Light theme */
|
||||
.light {
|
||||
--background: #f8fafc;
|
||||
--card: #ffffff;
|
||||
--card-border: #e2e8f0;
|
||||
--text: #1e293b;
|
||||
--text-muted: #64748b;
|
||||
}
|
||||
|
||||
/* Brand colors stay consistent */
|
||||
:root {
|
||||
--brand-pink: #ff7acc;
|
||||
--brand-gold: #ffdf6b;
|
||||
--brand-purple: #8b5cf6;
|
||||
}
|
||||
```
|
||||
|
||||
### Testing Checklist
|
||||
- [ ] Theme toggle is visible on all pages
|
||||
- [ ] Toggle persists preference across sessions
|
||||
- [ ] All text is readable in both themes
|
||||
- [ ] Brand colors render correctly in both themes
|
||||
- [ ] Interactive elements have proper hover/focus states in both themes
|
||||
- [ ] Radial background gradient adjusts opacity for light theme
|
||||
|
||||
---
|
||||
|
||||
## Implementation Reference
|
||||
|
||||
### CSS Variables (Recommended)
|
||||
|
||||
```css
|
||||
:root {
|
||||
/* Brand Colors */
|
||||
--brand-pink: #ff7acc;
|
||||
--brand-pink-hover: #ff5bb8;
|
||||
--brand-gold: #ffdf6b;
|
||||
--brand-purple: #8b5cf6;
|
||||
--brand-blue: #3b82f6;
|
||||
|
||||
/* Semantic Colors */
|
||||
--success: #22c55e;
|
||||
--warning: #f59e0b;
|
||||
--danger: #ef4444;
|
||||
|
||||
/* Dark Theme */
|
||||
--bg: #0f0f0f;
|
||||
--card: #1a1a1a;
|
||||
--card-border: #2a2a2a;
|
||||
--text: #e5e5e5;
|
||||
--text-muted: #888888;
|
||||
|
||||
/* Spacing */
|
||||
--space-xs: 4px;
|
||||
--space-sm: 8px;
|
||||
--space-md: 16px;
|
||||
--space-lg: 24px;
|
||||
--space-xl: 32px;
|
||||
|
||||
/* Border Radius */
|
||||
--radius-sm: 8px;
|
||||
--radius-md: 12px;
|
||||
--radius-lg: 16px;
|
||||
--radius-pill: 9999px;
|
||||
}
|
||||
```
|
||||
|
||||
### Tailwind Configuration
|
||||
|
||||
```javascript
|
||||
// tailwind.config.js
|
||||
module.exports = {
|
||||
theme: {
|
||||
extend: {
|
||||
colors: {
|
||||
brand: {
|
||||
pink: '#ff7acc',
|
||||
'pink-hover': '#ff5bb8',
|
||||
gold: '#ffdf6b',
|
||||
purple: '#8b5cf6',
|
||||
blue: '#3b82f6',
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### React Component Example
|
||||
|
||||
```tsx
|
||||
// Primary Button Component
|
||||
export function PrimaryButton({ children, ...props }) {
|
||||
return (
|
||||
<button
|
||||
className="bg-brand-pink hover:bg-brand-pink-hover text-white font-semibold px-7 py-3.5 rounded-xl transition-all shadow-[0_0_20px_rgba(255,122,204,0.4)] hover:shadow-[0_0_30px_rgba(255,122,204,0.6)]"
|
||||
{...props}
|
||||
>
|
||||
{children}
|
||||
</button>
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Checklist for Brand Compliance
|
||||
|
||||
### Landing Pages
|
||||
- [ ] Uses brand pink (#ff7acc) for primary CTAs
|
||||
- [ ] Logo displayed correctly
|
||||
- [ ] Dark theme as default
|
||||
- [ ] Gradient text on hero headlines
|
||||
- [ ] Consistent typography scale
|
||||
- [ ] Proper spacing (multiples of 4px)
|
||||
|
||||
### Application UI
|
||||
- [ ] Logo in header (32-40px height)
|
||||
- [ ] Brand pink for primary actions
|
||||
- [ ] Consistent card styling
|
||||
- [ ] Proper focus states with brand pink
|
||||
- [ ] Dark/light theme toggle working
|
||||
|
||||
### Emails
|
||||
- [ ] Header with logo image (not emoji)
|
||||
- [ ] Brand pink buttons
|
||||
- [ ] Pink divider lines
|
||||
- [ ] Consistent footer with signature
|
||||
- [ ] White background for compatibility
|
||||
|
||||
### Marketing Materials
|
||||
- [ ] Logo with proper clear space
|
||||
- [ ] Brand colors only
|
||||
- [ ] Consistent typography
|
||||
- [ ] No logo modifications
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 1.1 | 2026-01-07 | Updated radial background gradient (now uses layered radial gradients with optional animation); Updated logo usage (SVG preferred, hosted GIF URL for emails only); Updated email header (use logo image, not emoji); Added Light/Dark Theme Requirements section; Fixed success color consistency (#22c55e) |
|
||||
| 1.0 | 2026-01-07 | Initial brand guide created |
|
||||
|
||||
---
|
||||
|
||||
*This guide is the source of truth for Nomad Karaoke brand identity. For questions or clarifications, contact the design team.*
|
||||
@@ -110,17 +110,41 @@ Push to `main` triggers GitHub Actions:
|
||||
3. Deploy to Cloud Run
|
||||
4. Deploy frontend to Cloudflare Pages
|
||||
|
||||
### Manual Backend Deploy
|
||||
### Infrastructure Changes (Pulumi)
|
||||
|
||||
**All infrastructure changes must go through PRs.** The CI pipeline automatically deploys Pulumi changes when PRs merge to main.
|
||||
|
||||
```bash
|
||||
# Preview infrastructure changes
|
||||
# Preview changes locally (requires GCP auth)
|
||||
cd infrastructure
|
||||
pulumi preview
|
||||
pulumi preview --stack nomadkaraoke/karaoke-gen-infrastructure/prod
|
||||
|
||||
# Apply changes
|
||||
pulumi up
|
||||
# DO NOT run `pulumi up` locally - let CI handle deployment
|
||||
# CI uses --skip-preview due to limited service account permissions
|
||||
```
|
||||
|
||||
**What's managed by Pulumi:**
|
||||
- Firestore database and indexes
|
||||
- GCS buckets
|
||||
- Cloud Run service configuration
|
||||
- Cloud Tasks queues
|
||||
- Secret Manager secrets
|
||||
- Service accounts and IAM bindings
|
||||
- Workload Identity Federation
|
||||
- GCE instances (GitHub runners, encoding worker)
|
||||
|
||||
**Workflow:**
|
||||
1. Make changes in the appropriate module under `infrastructure/`
|
||||
- `modules/` - Core GCP resources (database, storage, secrets, IAM, etc.)
|
||||
- `compute/` - VM definitions and startup scripts
|
||||
- `config.py` - Shared configuration constants
|
||||
- `__main__.py` - Entry point that wires modules together
|
||||
2. Run `pulumi preview` locally to verify
|
||||
3. Create PR and merge
|
||||
4. CI deploys automatically
|
||||
|
||||
See `docs/LESSONS-LEARNED.md` for Pulumi CI gotchas.
|
||||
|
||||
### Manual Frontend Deploy
|
||||
|
||||
```bash
|
||||
|
||||
@@ -78,6 +78,62 @@ Audio separation and lyrics transcription run in parallel:
|
||||
|
||||
## Common Gotchas
|
||||
|
||||
### Verify Active Worktree Before Making Changes
|
||||
|
||||
**Problem**: When multiple git worktrees exist (e.g., `karaoke-gen` for main, `karaoke-gen-brand-consistency` for a feature branch), it's easy to make changes in the wrong worktree, especially if not paying attention to which directory the user's dev server is running from.
|
||||
|
||||
**Symptoms**:
|
||||
- Changes made successfully but don't appear in the user's browser
|
||||
- User says "I'm running the dev server from X" and X is a different directory
|
||||
- Build passes but UI doesn't reflect changes
|
||||
|
||||
**Solution**: Before making any file changes in a multi-worktree setup:
|
||||
1. Ask or confirm which worktree/directory the user is actively working in
|
||||
2. Verify by checking where their dev server is running
|
||||
3. Make changes in the correct worktree path
|
||||
|
||||
**Lesson**: The working directory in your terminal may not match where the user is running their dev server. Always verify the active worktree before editing files.
|
||||
|
||||
### Frontend Theme-Aware Styling: Search All Files Proactively
|
||||
|
||||
**Problem**: When making frontend styling changes for theme support (light/dark mode), fixing issues one-by-one as they're reported is inefficient. Hardcoded colors are scattered across many files.
|
||||
|
||||
**Wrong approach**:
|
||||
1. User reports: "email placeholder is invisible in light mode"
|
||||
2. Fix that one file
|
||||
3. User reports: "sign-in dialog is dark"
|
||||
4. Fix that file
|
||||
5. Repeat for every component...
|
||||
|
||||
**Right approach**: Proactively search for ALL non-theme-aware color patterns and fix them in one pass:
|
||||
|
||||
```bash
|
||||
# Find all hardcoded slate/gray colors that should be theme-aware
|
||||
grep -r --include="*.tsx" --include="*.ts" -E \
|
||||
"(text-slate-|bg-slate-|border-slate-|text-gray-|bg-gray-|border-gray-)" \
|
||||
app/ components/ lib/
|
||||
```
|
||||
|
||||
**Theme-aware replacements**:
|
||||
| Hardcoded | Theme-aware |
|
||||
|-----------|-------------|
|
||||
| `text-slate-400`, `text-gray-400` | `text-muted-foreground` |
|
||||
| `text-slate-100`, `text-white` | `text-foreground` |
|
||||
| `bg-slate-800`, `bg-gray-800` | `bg-card` or `bg-secondary` |
|
||||
| `bg-slate-900`, `bg-gray-900` | `bg-background` |
|
||||
| `border-slate-700`, `border-gray-700` | `border-border` |
|
||||
|
||||
**CSS variables** (defined in `globals.css`):
|
||||
- `--background`, `--foreground` - Main page colors
|
||||
- `--card`, `--card-foreground` - Card/dialog colors
|
||||
- `--muted`, `--muted-foreground` - Subdued text
|
||||
- `--border` - Border colors
|
||||
- `--primary`, `--secondary`, `--destructive`, etc. - Semantic colors
|
||||
|
||||
**Also check tests**: After bulk-replacing colors in source files, update test expectations that assert on color classes.
|
||||
|
||||
**Lesson**: When adding theme support, treat it as a codebase-wide refactor. Search for all hardcoded color patterns upfront rather than playing whack-a-mole with user-reported issues.
|
||||
|
||||
### Cross-Domain localStorage Isolation
|
||||
|
||||
**Problem**: Auth tokens stored in localStorage on one subdomain are invisible to other subdomains.
|
||||
@@ -222,6 +278,52 @@ escaped = filename.replace("'", "\\'")
|
||||
query = f"name='{escaped}' and '{parent_id}' in parents"
|
||||
```
|
||||
|
||||
### FFmpeg Filter Path Escaping (subprocess without shell)
|
||||
|
||||
**Problem**: Song titles with apostrophes (e.g., "I'm With You") cause FFmpeg's ASS subtitle filter to fail because paths aren't escaped correctly.
|
||||
|
||||
```python
|
||||
# BAD - FFmpeg strips the apostrophe, looks for wrong path
|
||||
ass_filter = f"ass=./Avril Lavigne - I'm With You/cache/temp.ass"
|
||||
# FFmpeg error: Could not create a libass track when reading file './Avril Lavigne - Im With You/cache/temp.ass'
|
||||
```
|
||||
|
||||
**Wrong approach #1**: Simple backslash escaping like `\'` doesn't work because FFmpeg's filter parser requires more escaping.
|
||||
|
||||
**Wrong approach #2**: Shell-style escaping `'\''` (end quote, escaped quote, start quote) only works when passing through a shell. When using Python `subprocess` with a command list, there's no shell - arguments go directly to FFmpeg, so shell escaping patterns don't work.
|
||||
|
||||
**Solution**: Use FFmpeg filter escaping (for subprocess without shell):
|
||||
```python
|
||||
def _escape_ffmpeg_filter_path(path: str) -> str:
|
||||
"""Escape a path for FFmpeg filter expressions (for subprocess without shell).
|
||||
|
||||
FFmpeg filter escaping rules:
|
||||
- Backslashes: double them (\ -> \\)
|
||||
- Single quotes/apostrophes: escape with three backslashes (' -> \\\')
|
||||
- Spaces: escape with backslash ( -> \ )
|
||||
"""
|
||||
# First escape existing backslashes (\ -> \\)
|
||||
escaped = path.replace("\\", "\\\\")
|
||||
# Escape single quotes (' -> \\\')
|
||||
# In Python source: 6 backslashes + apostrophe = 3 backslashes + apostrophe in string
|
||||
escaped = escaped.replace("'", "\\\\\\'")
|
||||
# Escape spaces
|
||||
escaped = escaped.replace(" ", "\\ ")
|
||||
return escaped
|
||||
|
||||
# Result: "I'm With You" becomes "I\\\'m\ With\ You"
|
||||
# FFmpeg correctly interprets this as: I'm With You
|
||||
```
|
||||
|
||||
**Symptoms**:
|
||||
- Video generation fails with "Could not create a libass track"
|
||||
- Error path is subtly different from the path in your code (missing apostrophe)
|
||||
- Works fine for songs without special characters
|
||||
|
||||
**Key insight**: FFmpeg filter escaping is different from shell escaping. When using `subprocess.run(cmd_list)` (no `shell=True`), there's no shell to interpret the arguments. FFmpeg receives the strings directly and applies its own filter expression parsing rules. The `'\''` shell idiom does NOT work in this context.
|
||||
|
||||
**Lesson**: When debugging FFmpeg filter path issues, check whether you're using shell or subprocess. For subprocess without shell, escape spaces with `\ ` and apostrophes with `\\\'` (3 backslashes + quote in the actual string).
|
||||
|
||||
### Duplicate Code Leads to Inconsistent Behavior
|
||||
|
||||
**Problem**: Multiple endpoints computing the same defaults independently. `audio_search.py` had `effective_brand_prefix = body.brand_prefix or settings.default_brand_prefix` but `file_upload.py` was missing it for some endpoints.
|
||||
@@ -247,6 +349,43 @@ def _get_effective_distribution_settings(...) -> EffectiveDistributionSettings:
|
||||
|
||||
**Lesson**: When you see the same logic repeated in multiple places, refactor into a helper function immediately. Otherwise bugs will be fixed in one place but not others.
|
||||
|
||||
### Python `or` Operator Can't Override True Defaults with False
|
||||
|
||||
**Problem**: Using Python's `or` operator for default logic prevents explicit `false` from overriding a `true` server default.
|
||||
|
||||
```python
|
||||
# BAD - can't explicitly disable if server default is true
|
||||
effective_value = body.enable_feature or settings.default_enable_feature
|
||||
# When body.enable_feature=False and settings.default=True:
|
||||
# False or True = True (ignores explicit False!)
|
||||
```
|
||||
|
||||
**Symptoms**:
|
||||
- Feature can be enabled but never disabled via API
|
||||
- Works fine when server default is `false` but breaks when default is `true`
|
||||
- No errors - just ignores the explicit `false` value
|
||||
|
||||
**Solution**: Use `Optional[bool] = None` with explicit `is not None` check:
|
||||
|
||||
```python
|
||||
# Request model
|
||||
enable_feature: Optional[bool] = None # None = use server default
|
||||
|
||||
# Default logic
|
||||
effective_value = (
|
||||
body.enable_feature
|
||||
if body.enable_feature is not None
|
||||
else settings.default_enable_feature
|
||||
)
|
||||
```
|
||||
|
||||
This allows three states:
|
||||
- `None` (not specified) → use server default
|
||||
- `True` → explicitly enable
|
||||
- `False` → explicitly disable (overrides server default)
|
||||
|
||||
**Lesson**: When a boolean field needs to support "use server default" behavior, use `Optional[bool] = None` instead of `bool = False`. The `or` operator conflates "not specified" with "explicitly false".
|
||||
|
||||
### Field Name Mismatches Between Endpoints
|
||||
|
||||
**Problem**: Different endpoints setting different field names for the same logical value, where consumers check only one field name.
|
||||
@@ -588,6 +727,43 @@ GitHub-hosted runners have ~14GB disk, which caused failures during Docker build
|
||||
|
||||
See `docs/archive/2025-12-28-self-hosted-github-runner.md` for full details.
|
||||
|
||||
### Pulumi CI Deployments Need Skip-Preview for Limited Service Accounts
|
||||
|
||||
**Problem**: Running `pulumi up --refresh` in CI requires the service account to have read permissions on ALL managed GCP resources. Pulumi refreshes state by querying each resource, and permission errors cause the deployment to fail before any changes are applied.
|
||||
|
||||
**Symptoms**:
|
||||
```
|
||||
Error when reading or editing Resource "project" with IAM Member: Role "roles/...":
|
||||
Error retrieving IAM policy for project "...": googleapi: Error 403: The caller does not have permission
|
||||
```
|
||||
|
||||
**Root cause**: `--refresh` (default behavior) makes read API calls to verify current state matches Pulumi's state. This requires permissions like:
|
||||
- `resourcemanager.projects.getIamPolicy` (for IAM bindings)
|
||||
- `secretmanager.secrets.get` (for secrets)
|
||||
- `cloudtasks.queues.get` (for Cloud Tasks)
|
||||
- `iam.workloadIdentityPools.get` (for Workload Identity)
|
||||
- And many more for each resource type managed
|
||||
|
||||
**Solution**: Use `--skip-preview` for CI deployments where Pulumi state is authoritative:
|
||||
|
||||
```yaml
|
||||
# In CI workflow
|
||||
pulumi up --yes --skip-preview --stack org/project/stack
|
||||
```
|
||||
|
||||
**Why this is safe**:
|
||||
- Infrastructure is only modified through Pulumi (single source of truth)
|
||||
- Preview/refresh is still useful during local development with broader permissions
|
||||
- Reduces permissions surface for CI service account (principle of least privilege)
|
||||
|
||||
**Alternative** (if refresh is required): Grant the service account `roles/editor` or individual read roles for each resource type. This is less secure but allows drift detection.
|
||||
|
||||
**Lesson**: When setting up Pulumi in CI, either:
|
||||
1. Grant broad read permissions and keep `--refresh` (detects drift)
|
||||
2. Use minimal permissions and `--skip-preview` (trusts Pulumi state)
|
||||
|
||||
Choose based on your threat model. For most projects, trusting Pulumi state is acceptable since manual console changes are rare.
|
||||
|
||||
### Docker Disk Space Management on Self-Hosted Runners
|
||||
|
||||
**Problem**: Even with 200GB SSDs, self-hosted runners can fill up with Docker images. Each CI build creates new dangling images (~15GB each), and without aggressive cleanup, disk fills in days.
|
||||
@@ -633,6 +809,53 @@ See `docs/archive/2025-12-28-self-hosted-github-runner.md` for full details.
|
||||
- Pre-job checks provide fail-fast behavior with clear error messages
|
||||
- `docker builder prune -af` catches buildkit cache that `system prune` misses
|
||||
|
||||
### Unicode Characters in Artist/Title Break HTTP Headers
|
||||
|
||||
**Problem**: Non-ASCII "smart" characters (curly quotes, em dashes) in artist/title fields caused HTTP header encoding failures. Job d49efab1 failed because the title "Mama Says (You Can't Back Down)" contained a curly apostrophe (U+2019 `'`) instead of a straight apostrophe (U+0027 `'`).
|
||||
|
||||
**Symptoms**:
|
||||
- Audio separation fails with "no files were downloaded"
|
||||
- Modal API returns encoding errors
|
||||
- Email subjects cause UnicodeEncodeError
|
||||
- Google Drive API queries fail with "Invalid Value"
|
||||
- Content-Disposition headers fail
|
||||
|
||||
**Root cause**: HTTP headers must be latin-1 encodable. Many Unicode characters (curly quotes U+2018/U+2019, em dash U+2014, ellipsis U+2026, etc.) are NOT latin-1 compatible, but look identical to ASCII equivalents.
|
||||
|
||||
**Solution**: Two-tier sanitization approach:
|
||||
|
||||
1. **Input-time normalization** (`normalize_text()`): Standardize visually-similar Unicode characters to ASCII equivalents when data enters the system. Applied via Pydantic validators on `JobCreate` and `AudioSearchRequest` models.
|
||||
|
||||
2. **Output-time sanitization** (`sanitize_filename()`): Additional safety for filenames and HTTP headers. Removes filesystem-unsafe characters, strips periods/spaces, collapses underscores.
|
||||
|
||||
```python
|
||||
# Character categories normalized:
|
||||
APOSTROPHE_REPLACEMENTS = {
|
||||
"\u2018": "'", # LEFT SINGLE QUOTATION MARK
|
||||
"\u2019": "'", # RIGHT SINGLE QUOTATION MARK (caused the bug!)
|
||||
"\u0060": "'", # GRAVE ACCENT (backtick)
|
||||
# ... more
|
||||
}
|
||||
DASH_REPLACEMENTS = {
|
||||
"\u2013": "-", # EN DASH
|
||||
"\u2014": "-", # EM DASH
|
||||
# ... more
|
||||
}
|
||||
WHITESPACE_REPLACEMENTS = {
|
||||
"\u00A0": " ", # NON-BREAKING SPACE
|
||||
"\u3000": " ", # IDEOGRAPHIC SPACE (CJK full-width)
|
||||
# ... more
|
||||
}
|
||||
```
|
||||
|
||||
**Key insight**: These characters are often invisible problems - they look correct in logs and UIs but fail at encoding time. Users copy/paste from Word, macOS, or websites that use "smart" typography.
|
||||
|
||||
**Where to sanitize**:
|
||||
- `sanitize_filename()` call anywhere artist/title is used in: HTTP headers, filenames, API queries, email subjects
|
||||
- `normalize_text()` applied automatically via model validators at job creation and audio search
|
||||
|
||||
**Lesson**: Always normalize user-provided text at input time. Don't trust that "obviously ASCII" text is actually ASCII - smart quotes from copy/paste are invisible bugs waiting to happen.
|
||||
|
||||
### Fonts in Docker for Video Rendering
|
||||
|
||||
**Problem**: Video rendering showed replacement characters (squares with question marks) for special Unicode symbols like `♪` in intro/instrumental screens.
|
||||
@@ -667,6 +890,173 @@ RUN apt-get update && apt-get install -y \
|
||||
|
||||
**Lesson**: When building Docker images for video/subtitle rendering, install fonts explicitly. The default slim images have nothing, and font fallback behavior differs from local development machines.
|
||||
|
||||
### Reuse LLM Model Instances Across Operations
|
||||
|
||||
**Problem**: Creating a new LLM model instance for each operation in a loop causes massive overhead. Each instance creation triggers model initialization, connection establishment, and optional "warm-up" calls - adding 2+ seconds per operation.
|
||||
|
||||
**Example**: Agentic lyrics correction was creating a new `AgenticCorrector` for each gap (lyric segment needing correction):
|
||||
|
||||
```python
|
||||
# BAD - 31 gaps × 2s initialization = 62s overhead
|
||||
for gap in gaps_to_correct:
|
||||
agent = AgenticCorrector.from_model(model_id, ...) # New instance every time!
|
||||
agent.correct_gap(gap)
|
||||
```
|
||||
|
||||
**Symptoms**:
|
||||
- Operations that should take ~1 minute take 5+ minutes
|
||||
- Logs show "Initializing model..." or "Warming up..." repeatedly
|
||||
- Each loop iteration has a consistent ~2s overhead before actual work
|
||||
- Total time scales linearly with loop count, not with actual work
|
||||
|
||||
**Solution**: Create the model instance ONCE before the loop, then reuse:
|
||||
|
||||
```python
|
||||
# GOOD - 1 initialization, reused for all gaps
|
||||
agent = AgenticCorrector.from_model(model_id, ...) # Create once
|
||||
for gap in gaps_to_correct:
|
||||
agent.correct_gap(gap) # Reuse existing instance
|
||||
```
|
||||
|
||||
**Additional optimization**: If operations are independent, process them in parallel:
|
||||
|
||||
```python
|
||||
# BETTER - parallel processing with shared model
|
||||
agent = AgenticCorrector.from_model(model_id, ...)
|
||||
with ThreadPoolExecutor(max_workers=5) as executor:
|
||||
futures = {executor.submit(agent.correct_gap, gap): gap for gap in gaps}
|
||||
for future in as_completed(futures):
|
||||
result = future.result()
|
||||
```
|
||||
|
||||
**Results**:
|
||||
- Before: 31 gaps → ~5 minutes (2s init + 3s work per gap)
|
||||
- After: 20 gaps → ~55 seconds (1 init + parallel processing)
|
||||
- Speedup: ~5-6x
|
||||
|
||||
**Why warm-up is often unnecessary**: The warm-up pattern (sending a trivial request to "prime" the model) made sense for cold-start scenarios in older systems. Modern LLM APIs:
|
||||
- Don't have cold-start penalties per request
|
||||
- Handle connection pooling internally
|
||||
- May even rate-limit warm-up calls
|
||||
|
||||
If you're already making real requests, warm-up adds latency without benefit.
|
||||
|
||||
**Key insight**: LLM client instances are designed for reuse. They maintain connection pools, authentication state, and caching. Creating a new instance per request throws away these benefits.
|
||||
|
||||
**Configuration**: Added `AGENTIC_MAX_PARALLEL_GAPS` env var (default: 5) to control parallelism. Higher values increase throughput but may hit rate limits.
|
||||
|
||||
### Thread-Safe Lazy Initialization in Shared Components
|
||||
|
||||
**Problem**: Lazy-initialized singleton resources (like LLM model instances) that are shared across parallel threads can have race conditions if the initialization check isn't thread-safe.
|
||||
|
||||
**Example bug**: `LangChainBridge._chat_model` was lazily initialized with a simple `if not self._chat_model:` check. When 5 parallel gap-processing threads called `generate_correction_proposals()` simultaneously, ALL threads saw `_chat_model == None` and entered the initialization block, causing:
|
||||
- 5 concurrent model initializations instead of 1
|
||||
- ~6 minute delay from Vertex AI resource contention
|
||||
- All 5 threads timing out (30s limit each)
|
||||
- Circuit breaker triggered → 0/8 gaps corrected by AI
|
||||
|
||||
**Symptoms**:
|
||||
- Multiple "Initializing model X with timeout..." log entries appearing simultaneously
|
||||
- Long unexplained delays in parallel processing
|
||||
- Timeouts that don't make sense given individual operation times
|
||||
- Circuit breaker opening when single-threaded tests pass
|
||||
|
||||
**Solution**: Use double-checked locking pattern:
|
||||
|
||||
```python
|
||||
import threading
|
||||
|
||||
class LangChainBridge:
|
||||
def __init__(self):
|
||||
self._chat_model = None
|
||||
self._model_init_lock = threading.Lock() # Add lock
|
||||
|
||||
def generate_correction_proposals(self, ...):
|
||||
# Fast path: model already initialized
|
||||
if not self._chat_model:
|
||||
with self._model_init_lock:
|
||||
# Double-check after acquiring lock
|
||||
if not self._chat_model:
|
||||
self._chat_model = self._factory.create_chat_model(...)
|
||||
```
|
||||
|
||||
**Why double-checked locking**:
|
||||
1. First check (without lock): Fast path for normal case - avoids lock overhead when model is already initialized
|
||||
2. Lock acquisition: Only happens when model might need initialization
|
||||
3. Second check (with lock): Prevents race where another thread initialized while we waited for lock
|
||||
|
||||
**Key insight**: Any lazily-initialized shared resource accessed from parallel threads needs synchronization. The pattern of "check if None, then initialize" is inherently racy without locking. This applies to:
|
||||
- LLM model instances
|
||||
- Database connections
|
||||
- API client instances
|
||||
- Cached computation results
|
||||
|
||||
**Testing**: Write tests that use `threading.Barrier` to ensure multiple threads hit the initialization check simultaneously, then assert the initialization function was called exactly once.
|
||||
|
||||
### Preloading Heavy Resources at Container Startup
|
||||
|
||||
**Problem**: Lazy-loading heavy resources (like SpaCy NLP models) during request processing can cause 60+ second delays on Cloud Run due to slow filesystem I/O during cold starts. The first request after deployment/scaling pays the full load penalty.
|
||||
|
||||
**Example bug**: `PhraseAnalyzer` loaded SpaCy's `en_core_web_sm` model lazily on first use during agentic lyrics correction. On Cloud Run, this took 63 seconds due to container filesystem being slower than local SSD. Job `2ccbdf6b` showed 73-second gap between "Initializing PhraseAnalyzer" and "Initialized AnchorSequenceFinder" logs.
|
||||
|
||||
**Symptoms**:
|
||||
- First job after deployment is extremely slow
|
||||
- Large time gaps in logs between "loading X" and "using X"
|
||||
- Same code runs much faster locally (where filesystem is faster)
|
||||
- Subsequent requests (warm container) are fast
|
||||
|
||||
**Solution**: Preload heavy resources at container startup using FastAPI's lifespan handler:
|
||||
|
||||
```python
|
||||
# backend/services/spacy_preloader.py - singleton storage
|
||||
_preloaded_models: dict = {}
|
||||
|
||||
def preload_spacy_model(model_name: str = "en_core_web_sm") -> None:
|
||||
if model_name not in _preloaded_models:
|
||||
_preloaded_models[model_name] = spacy.load(model_name)
|
||||
|
||||
def get_preloaded_model(model_name: str) -> Optional[object]:
|
||||
return _preloaded_models.get(model_name)
|
||||
|
||||
# backend/main.py - load at startup
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
preload_spacy_model("en_core_web_sm") # Pay cost once at startup
|
||||
yield
|
||||
```
|
||||
|
||||
Then modify consumers to check for preloaded model:
|
||||
```python
|
||||
class PhraseAnalyzer:
|
||||
def __init__(self):
|
||||
preloaded = get_preloaded_model("en_core_web_sm")
|
||||
if preloaded:
|
||||
self.nlp = preloaded # Fast path
|
||||
else:
|
||||
self.nlp = spacy.load("en_core_web_sm") # Fallback
|
||||
```
|
||||
|
||||
**Key insight**: On Cloud Run, container startup time doesn't directly impact users (they don't see the startup delay), but request processing time does. Moving heavy initialization from request-time to startup-time improves user experience even if total work is the same.
|
||||
|
||||
**Good candidates for preloading**:
|
||||
- ML model files (SpaCy, transformers, etc.)
|
||||
- Large data files
|
||||
- Expensive-to-initialize clients
|
||||
- Anything that touches the filesystem heavily
|
||||
|
||||
**Not worth preloading**:
|
||||
- Fast in-memory operations
|
||||
- Resources that vary per-request
|
||||
- Rarely-used code paths
|
||||
|
||||
**Logging**: Add timing logs to both startup preload and usage paths so you can verify the optimization is working:
|
||||
```
|
||||
# At startup: "SpaCy model 'en_core_web_sm' preloaded in 2.34s"
|
||||
# During request: "Using preloaded SpaCy model: en_core_web_sm"
|
||||
```
|
||||
|
||||
See `docs/archive/2026-01-08-spacy-preload-plan.md` for full implementation details.
|
||||
|
||||
## Performance Observations
|
||||
|
||||
| Operation | Duration |
|
||||
@@ -755,6 +1145,48 @@ export function getToken() {
|
||||
|
||||
**Lesson**: For values that can change (auth tokens, user preferences), read fresh from storage on each access rather than caching in module scope.
|
||||
|
||||
### Zustand Module-Level Initialization Doesn't Re-run on Client Navigation
|
||||
|
||||
**Problem**: Module-level initialization code in zustand stores only runs once when the module is first imported. Client-side navigation (e.g., `router.push()`) doesn't reload modules, so initialization code doesn't re-run.
|
||||
|
||||
**Example bug**: Auth module had initialization that fetched user data if a token existed:
|
||||
```typescript
|
||||
// auth.ts - module level (runs once at import)
|
||||
if (typeof window !== 'undefined') {
|
||||
const token = getAccessToken()
|
||||
if (token) {
|
||||
useAuth.getState().fetchUser() // Only runs on initial page load!
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
After beta enrollment:
|
||||
1. User lands on `/` (landing page) - auth.ts imported, no token yet, `fetchUser()` not called
|
||||
2. User completes enrollment, token stored via `setAccessToken()`
|
||||
3. `router.push('/app')` - client-side navigation, modules already loaded
|
||||
4. Auth module initialization doesn't re-run - `user` stays `null`
|
||||
5. UI shows "Login" button instead of user credits
|
||||
|
||||
**Symptoms**:
|
||||
- Feature works after page reload but not after client-side navigation
|
||||
- Zustand store has correct persisted data (token) but not computed data (user profile)
|
||||
- E2E tests that use reload pass; tests that use `router.push` fail
|
||||
|
||||
**Solution**: Components that need fresh data should trigger fetches in useEffect:
|
||||
```typescript
|
||||
// In /app page component
|
||||
const { user, fetchUser } = useAuth()
|
||||
|
||||
useEffect(() => {
|
||||
const token = getAccessToken()
|
||||
if (token && !user) {
|
||||
fetchUser() // Ensure user data is loaded regardless of how we got here
|
||||
}
|
||||
}, [user, fetchUser])
|
||||
```
|
||||
|
||||
**Lesson**: Don't rely on module-level initialization for data that might need to be fetched after client-side navigation. Use useEffect in components to ensure data is loaded when needed.
|
||||
|
||||
### GCP Organization Policies Block Service Account Keys
|
||||
|
||||
**Problem**: GCP organization policies can block service account key creation even when you have Owner access to the project.
|
||||
@@ -1079,6 +1511,45 @@ logs_ref.document(log_id).set({
|
||||
|
||||
**Lesson**: Never use embedded arrays for unbounded data. If an array can grow indefinitely based on operation duration or user input, use a subcollection instead. The 1MB limit applies to the entire document, including all fields.
|
||||
|
||||
### Splitting Fast Generation from Slow Encoding
|
||||
|
||||
**Problem**: Preview video generation took 60+ seconds on Cloud Run, causing poor UX during lyrics review. The bottleneck was FFmpeg encoding (CPU-bound), not the subtitle generation (fast text processing).
|
||||
|
||||
**Solution**: Split the operation into two phases with optional offloading:
|
||||
|
||||
1. **ASS Generation (Cloud Run)**: Generate ASS subtitle file locally (~2s). This is pure text processing with no CPU-intensive encoding.
|
||||
|
||||
2. **Video Encoding (GCE)**: Upload ASS to GCS and offload FFmpeg encoding to the high-performance GCE worker (~15-20s on C4-standard-8).
|
||||
|
||||
```python
|
||||
# Generate ASS only (fast, local)
|
||||
result = CorrectionOperations.generate_preview_video(
|
||||
...,
|
||||
ass_only=True, # Skip video encoding
|
||||
)
|
||||
|
||||
# Upload ASS to GCS
|
||||
storage.upload_file(result["ass_path"], f"jobs/{job_id}/previews/{preview_hash}.ass")
|
||||
|
||||
# Offload encoding to GCE
|
||||
gce_result = await encoding_service.encode_preview_video(
|
||||
job_id=preview_hash,
|
||||
ass_gcs_path=f"gs://bucket/jobs/{job_id}/previews/{preview_hash}.ass",
|
||||
audio_gcs_path=audio_gcs_path,
|
||||
output_gcs_path=f"gs://bucket/jobs/{job_id}/previews/{preview_hash}.mp4",
|
||||
)
|
||||
```
|
||||
|
||||
**Key insight**: When a pipeline has both fast (text generation) and slow (media encoding) steps, design for separability:
|
||||
- Add `_only` parameters to skip slow steps
|
||||
- Return intermediate outputs (paths to generated files)
|
||||
- Allow external services to handle the slow part
|
||||
- Keep fallback to local processing for reliability
|
||||
|
||||
**Configuration**: Controlled by `USE_GCE_PREVIEW_ENCODING` env var. Falls back to local encoding if GCE fails or is disabled.
|
||||
|
||||
**Result**: Preview generation dropped from 60+ seconds to ~15-20 seconds.
|
||||
|
||||
### External Service Response Format Mismatches
|
||||
|
||||
**Problem**: The GCE encoding worker returned responses in unexpected formats, causing `'list' object has no attribute 'get'` errors that were hard to debug through the async pipeline.
|
||||
@@ -1127,6 +1598,214 @@ if isinstance(output_files, list):
|
||||
|
||||
**Debugging tip**: When hitting format errors in async pipelines, check the raw response from the external service directly (curl with API key) before assuming the bug is in your code.
|
||||
|
||||
### GCE Encoding Returns GCS Paths, Not Local Files
|
||||
|
||||
**Problem**: When using GCE encoding (remote high-performance encoder), the orchestrator received GCS blob paths like `jobs/{id}/finals/output_4k_lossless.mp4` instead of local file paths. Subsequent operations like YouTube upload checked `os.path.isfile()` which returned `False`, causing silent failures.
|
||||
|
||||
**Symptoms**:
|
||||
- YouTube upload logged "No video file available for YouTube upload"
|
||||
- Other file-based operations silently skipped
|
||||
- Dropbox/GDrive uploads succeeded (they use different code paths)
|
||||
- No errors in logs - just warnings that were easy to miss
|
||||
|
||||
**Root cause**: The `VideoWorkerOrchestrator._run_encoding()` method stored the paths returned by `GCEEncodingBackend` directly without downloading the files. For local encoding, the paths are already local. For GCE encoding, the paths are GCS blob paths that need to be downloaded first.
|
||||
|
||||
**Solution**: Added `_download_gce_encoded_files()` method that:
|
||||
1. Detects when GCE backend was used (`encoding_backend.name == "gce"`)
|
||||
2. Downloads each encoded file from GCS to the local temp directory
|
||||
3. Updates result paths to point to local files
|
||||
|
||||
```python
|
||||
# After encoding completes
|
||||
if encoding_backend.name == "gce" and self.storage:
|
||||
await self._download_gce_encoded_files(output)
|
||||
```
|
||||
|
||||
**Key insight**: When an operation can run on multiple backends (local vs remote), ensure the outputs are normalized to a common format before subsequent stages consume them. The orchestrator pattern should abstract away backend differences, not leak them to downstream consumers.
|
||||
|
||||
**Testing tip**: Test the full pipeline with each backend type, not just the encoding step in isolation. The bug only manifested when YouTube upload ran after GCE encoding - unit tests of each stage passed.
|
||||
|
||||
### Unify Encoding Logic with GCS Wheel Deployment
|
||||
|
||||
**Problem**: The GCE encoding worker had simplified, duplicated encoding logic that:
|
||||
- Didn't concatenate title + karaoke + end screens
|
||||
- Used generic filenames like `output_4k_lossless.mp4` instead of proper names
|
||||
- Lacked feature parity with the local CLI's `LocalEncodingService`
|
||||
|
||||
This caused cloud-generated videos to be incomplete (no title/end screens) and have unhelpful filenames that didn't match the local CLI output.
|
||||
|
||||
**Solution**: Deploy the karaoke-gen wheel to GCS, have the GCE worker install and use `LocalEncodingService`:
|
||||
|
||||
1. **CI uploads wheel to GCS**: `deploy-backend` job uploads wheel to `gs://bucket/wheels/`
|
||||
|
||||
2. **Dynamic wheel loading**: GCE worker downloads and installs latest wheel at job start:
|
||||
```python
|
||||
def ensure_latest_wheel():
|
||||
"""Download and install latest karaoke-gen wheel from GCS."""
|
||||
subprocess.run(["gsutil", "cp", "gs://bucket/wheels/karaoke_gen-*.whl", "/tmp/"])
|
||||
wheel = sorted(glob.glob("/tmp/karaoke_gen-*.whl"))[-1]
|
||||
subprocess.run([sys.executable, "-m", "pip", "install", "--upgrade", wheel])
|
||||
|
||||
async def process_job(job_id, request):
|
||||
ensure_latest_wheel() # Get latest code at job start
|
||||
# ... rest of processing
|
||||
```
|
||||
|
||||
3. **Single encoding path**: GCE worker imports and uses `LocalEncodingService`:
|
||||
```python
|
||||
from backend.services.local_encoding_service import LocalEncodingService, EncodingConfig
|
||||
|
||||
service = LocalEncodingService(logger=logger)
|
||||
config = EncodingConfig(
|
||||
title_video=str(title_video),
|
||||
karaoke_video=str(karaoke_video),
|
||||
output_lossless_4k_mp4=f"{base_name} (Final Karaoke Lossless 4k).mp4",
|
||||
# ... proper naming for all formats
|
||||
)
|
||||
result = service.encode_all_formats(config)
|
||||
```
|
||||
|
||||
**Key benefits**:
|
||||
1. **Single source of truth**: Same encoding logic used by CLI, Cloud Run, and GCE
|
||||
2. **Hot updates without restart**: New code deployed on next job, not VM restart
|
||||
3. **In-progress jobs unaffected**: Jobs use the code version they started with
|
||||
4. **Proper output**: Titles/end screens concatenated, files named correctly
|
||||
|
||||
**No fallback by design**: We removed all fallback logic to ensure there's only ONE code path. If the wheel fails to load, the job fails clearly rather than producing inconsistent output.
|
||||
|
||||
**Lesson**: When multiple deployment targets (CLI, Cloud Run, GCE) need the same complex logic, package it as a library and deploy via a shared mechanism (GCS wheel). Duplicating logic leads to feature drift and inconsistent behavior.
|
||||
|
||||
### Unify Preview Encoding with LocalPreviewEncodingService
|
||||
|
||||
**Problem**: Following the pattern of final encoding, the GCE preview encoding also had divergent code. The `run_preview_encoding` function in `infrastructure/__main__.py` was a 100-line "simplified" reimplementation that:
|
||||
- Hardcoded resolution (480x270) instead of using constants
|
||||
- Had its own FFmpeg filter path escaping function with different logic
|
||||
- Didn't support hardware acceleration (NVENC)
|
||||
- Lacked font directory support for custom fonts
|
||||
- Had different movflags and encoding settings
|
||||
|
||||
While the final encoding was unified earlier, preview encoding was overlooked, creating a potential source of inconsistent preview videos between local development and cloud jobs.
|
||||
|
||||
**Solution**: Created `LocalPreviewEncodingService` in `backend/services/local_preview_encoding_service.py`:
|
||||
|
||||
1. **Extracted preview logic from VideoGenerator**: The service encapsulates the same FFmpeg command building from `VideoGenerator._build_preview_ffmpeg_command`
|
||||
|
||||
2. **Single source of truth for preview settings**:
|
||||
```python
|
||||
class LocalPreviewEncodingService:
|
||||
PREVIEW_WIDTH = 480
|
||||
PREVIEW_HEIGHT = 270
|
||||
PREVIEW_FPS = 24
|
||||
PREVIEW_AUDIO_BITRATE = "96k"
|
||||
```
|
||||
|
||||
3. **GCE worker imports from installed wheel**:
|
||||
```python
|
||||
from backend.services.local_preview_encoding_service import (
|
||||
LocalPreviewEncodingService,
|
||||
PreviewEncodingConfig,
|
||||
)
|
||||
|
||||
service = LocalPreviewEncodingService(logger=logger)
|
||||
result = service.encode_preview(config)
|
||||
```
|
||||
|
||||
4. **Process preview jobs call ensure_latest_wheel()**:
|
||||
```python
|
||||
async def process_preview_job(job_id, request):
|
||||
ensure_latest_wheel() # Get latest code at job start
|
||||
# ... rest of processing
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- Consistent preview encoding across CLI, Cloud Run, and GCE
|
||||
- Hardware acceleration (NVENC) when available
|
||||
- Proper FFmpeg filter escaping for special characters in paths
|
||||
- Custom font support via fontsdir filter
|
||||
- Single place to modify preview settings
|
||||
|
||||
**Lesson**: After unifying a major component (final encoding), audit related features (preview encoding) that may have similar duplication. It's easy to overlook "simpler" variations when focusing on the main feature.
|
||||
|
||||
### Cloud Distribution Must Match Local CLI Output Structure
|
||||
|
||||
**Problem**: Cloud-generated karaoke outputs had different file organization than local CLI, causing confusion and missing files in Dropbox uploads:
|
||||
|
||||
1. **lyrics/ folder filename differences**:
|
||||
- Cloud used generic `(Karaoke).json` instead of `(Lyrics Corrections).json`
|
||||
- Cloud used `(Karaoke).txt` instead of `(Lyrics Corrected).txt`
|
||||
- Reference lyrics files (`(Lyrics Genius).txt`, etc.) weren't downloaded
|
||||
|
||||
2. **Missing files in lyrics/ folder**:
|
||||
- `(With Vocals).mkv` wasn't copied to lyrics/ subfolder
|
||||
- `previews/` subfolder with preview ASS files wasn't created
|
||||
- Reference lyrics from multiple sources weren't included
|
||||
|
||||
**Root causes**:
|
||||
- Lyrics worker uploaded reference files to GCS but didn't track them in `job.files`
|
||||
- Video worker used generic naming pattern for all lyrics files
|
||||
- Video worker didn't copy `(With Vocals).mkv` or previews to lyrics folder
|
||||
|
||||
**Solution**:
|
||||
|
||||
1. **Track all files in job.files** (lyrics_worker.py):
|
||||
```python
|
||||
# Upload ALL reference lyrics and track them for distribution
|
||||
for ref_filename, source_key in reference_files:
|
||||
if os.path.exists(ref_path):
|
||||
url = storage.upload_file(ref_path, gcs_path)
|
||||
# Track in job.files so video_worker can download
|
||||
job_manager.update_file_url(job_id, 'lyrics', f'reference_{source_key}', url)
|
||||
```
|
||||
|
||||
2. **Use proper filenames in distribution** (video_worker.py):
|
||||
```python
|
||||
lyrics_file_mappings = [
|
||||
('lrc', f"{base_name} (Karaoke).lrc"),
|
||||
('corrections', f"{base_name} (Lyrics Corrections).json"),
|
||||
('corrected_txt', f"{base_name} (Lyrics Corrected).txt"),
|
||||
('uncorrected', f"{base_name} (Lyrics Uncorrected).txt"),
|
||||
('reference_genius', f"{base_name} (Lyrics Genius).txt"),
|
||||
# ... etc
|
||||
]
|
||||
```
|
||||
|
||||
3. **Copy (With Vocals).mkv to lyrics folder**:
|
||||
```python
|
||||
if videos.get('with_vocals'):
|
||||
dest = os.path.join(lyrics_dir, f"{base_name} (With Vocals).mkv")
|
||||
storage.download_file(videos['with_vocals'], dest)
|
||||
```
|
||||
|
||||
4. **Create previews/ subfolder**:
|
||||
```python
|
||||
previews_dir = os.path.join(lyrics_dir, "previews")
|
||||
os.makedirs(previews_dir, exist_ok=True)
|
||||
preview_files = storage.list_files(f"jobs/{job_id}/previews/")
|
||||
for blob_name in preview_files:
|
||||
if blob_name.endswith('.ass'):
|
||||
storage.download_file(blob_name, dest_path)
|
||||
```
|
||||
|
||||
**Expected output structure** (matches local CLI):
|
||||
```
|
||||
NOMAD-XXXX - Artist - Title/
|
||||
├── stems/
|
||||
│ ├── Artist - Title (Vocals model_bs_roformer_...).flac
|
||||
│ └── ...
|
||||
└── lyrics/
|
||||
├── Artist - Title (Karaoke).lrc
|
||||
├── Artist - Title (Karaoke).ass
|
||||
├── Artist - Title (Lyrics Corrections).json
|
||||
├── Artist - Title (Lyrics Corrected).txt
|
||||
├── Artist - Title (Lyrics Uncorrected).txt
|
||||
├── Artist - Title (Lyrics Genius).txt
|
||||
├── Artist - Title (With Vocals).mkv
|
||||
└── previews/
|
||||
└── preview_abc123 (Karaoke).ass
|
||||
```
|
||||
|
||||
**Lesson**: When building distribution packages from cloud storage, the organization code must replicate the exact structure users expect from local execution. Track ALL intermediate files in `job.files` even if they're "optional" - users may rely on them.
|
||||
|
||||
## What We'd Do Differently
|
||||
|
||||
1. **Add Pydantic model fields test first** - Would have caught the silent field issue immediately
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
# Product Communication Guide
|
||||
|
||||
Guidelines for how to communicate about Nomad Karaoke to users, in marketing materials, UI copy, and support interactions.
|
||||
|
||||
## Core Principles
|
||||
|
||||
1. **Transparency over hype** - Set realistic expectations so users aren't surprised
|
||||
2. **Avoid AI terminology** - "AI" has become a loaded term; say "our system" or "the tool" instead
|
||||
3. **Acknowledge user effort** - The human review step is required, not optional
|
||||
4. **Highlight real differentiators** - Real instrumentals (not covers), precise sync, multiple formats
|
||||
|
||||
---
|
||||
|
||||
## Key Facts to Communicate Accurately
|
||||
|
||||
### Timing
|
||||
- **Accurate:** "Create a karaoke video in under 30 minutes"
|
||||
- **Avoid:** "in minutes" (implies much faster than reality)
|
||||
- Typical flow: ~20-30 minutes including human review
|
||||
|
||||
### Song Availability
|
||||
- **Accurate:** "Any song" - this IS true
|
||||
- Users can: search by artist/title, paste YouTube URL, or upload their own audio
|
||||
- If users can get the audio from somewhere, we can make karaoke from it
|
||||
|
||||
### Lyrics Sync Quality
|
||||
- **Accurate:** "Precise lyrics sync" - the timing/sync IS very accurate
|
||||
- The sync (when words appear) is reliable and handled automatically
|
||||
- The challenge is lyrics ACCURACY (correct words), not timing
|
||||
|
||||
### Human Review Requirement
|
||||
- **Be clear:** Users ALWAYS need to review the transcribed lyrics
|
||||
- For songs with clear vocals (country, ballads): few or no corrections needed
|
||||
- For complex songs: expect 5-10 minutes of corrections
|
||||
- Frame as "Review & Correct" not "Preview and fine-tune" (the latter sounds optional)
|
||||
|
||||
### Audio Separation
|
||||
- **Accurate:** "Real instrumentals from the original song"
|
||||
- **Avoid:** "Studio-quality separation" (oversells it)
|
||||
- Key differentiator: NOT a cover band or MIDI recreation - it's the actual song
|
||||
- Separation quality is very good but not perfect for all songs
|
||||
|
||||
### Backing Vocals Option
|
||||
- Users can choose between:
|
||||
- Clean instrumental (all vocals removed)
|
||||
- Instrumental with backing vocals preserved
|
||||
- This matters for songs with backing vocals; irrelevant for songs without
|
||||
|
||||
### Output Formats
|
||||
- 4K MP4 karaoke video (main output)
|
||||
- "With Vocals" version for sing-along practice
|
||||
- CDG+MP3 ZIP for older/commercial karaoke systems
|
||||
- All videos published to Nomad Karaoke YouTube channel (not user's channel)
|
||||
|
||||
### YouTube Integration
|
||||
- **Accurate:** Videos are published to the Nomad Karaoke YouTube channel
|
||||
- **Wrong:** "Upload to your YouTube channel" - we don't do this
|
||||
- This is for easy sharing, not user channel integration
|
||||
|
||||
---
|
||||
|
||||
## Beta Tester Program
|
||||
|
||||
- Beta testers get up to 5 free karaoke video generations
|
||||
- Mandatory feedback required after each video
|
||||
- Must acknowledge they'll review/correct lyrics transcription
|
||||
- Helps improve the system through real-world testing
|
||||
|
||||
---
|
||||
|
||||
## Terminology Preferences
|
||||
|
||||
| Avoid | Use Instead |
|
||||
|-------|-------------|
|
||||
| AI, AI-powered, AI-generated | "our system", "the tool", "automated" |
|
||||
| Perfect lyrics | "precise sync", "accurate timing" |
|
||||
| Studio-quality | "real instrumentals", "actual song" |
|
||||
| In minutes | "in under 30 minutes" |
|
||||
| Fine-tune (sounds optional) | "review and correct" (sounds required) |
|
||||
| Upload to YouTube | "published to our YouTube channel" |
|
||||
|
||||
---
|
||||
|
||||
## FAQ Guidance
|
||||
|
||||
When answering user questions:
|
||||
|
||||
**Q: Is it fully automatic?**
|
||||
A: The transcription and sync are automatic, but you'll need to review and correct any misheard words before the final video is generated. For clear songs, this might take just a minute. For complex songs, expect 5-10 minutes.
|
||||
|
||||
**Q: What songs work?**
|
||||
A: Any song you can get audio for. Search our database, paste a YouTube link, or upload your own audio file.
|
||||
|
||||
**Q: What do I get?**
|
||||
A: Multiple files - 4K karaoke video, a sing-along version with vocals, and CDG+MP3 for older systems. Videos are also published to our YouTube channel.
|
||||
|
||||
**Q: How long does it take?**
|
||||
A: About 20-30 minutes total, including your review time.
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 1.0 | 2026-01-07 | Initial guide created based on founder feedback |
|
||||
@@ -34,8 +34,34 @@
|
||||
|
||||
- CDG format generation requires additional style configuration
|
||||
|
||||
## Pending Work
|
||||
|
||||
(No pending work items)
|
||||
|
||||
## Recent Changes
|
||||
|
||||
- **SpaCy Preloading** (2026-01-08): Implemented SpaCy model preloading at container startup to eliminate 60+ second delay during agentic correction. The `en_core_web_sm` model is now loaded during FastAPI lifespan startup, and `PhraseAnalyzer`/`SyllablesMatchHandler` reuse the preloaded model. Added timing logs to verify performance. See [archive/2026-01-08-spacy-preload-plan.md](archive/2026-01-08-spacy-preload-plan.md).
|
||||
|
||||
- **Thread-Safe LangChain Model Initialization** (2026-01-08): Fixed race condition in `LangChainBridge` where parallel threads could all try to initialize the AI model simultaneously, causing 6+ minute delays. Added double-checked locking with `threading.Lock()`. PR #232. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#thread-safe-lazy-initialization-in-shared-components).
|
||||
|
||||
- **Brand Consistency & Style Guide** (2026-01-07): Implemented unified brand theming across all Nomad Karaoke frontends. Primary color changed from blue to brand pink (#ff7acc). Added comprehensive [BRAND-STYLE-GUIDE.md](BRAND-STYLE-GUIDE.md) for human and LLM reference. Includes: CSS variables for light/dark mode, MUI theme updates for lyrics transcriber, email template branding, and Playwright visual tests. See [BRAND-STYLE-GUIDE.md](BRAND-STYLE-GUIDE.md).
|
||||
|
||||
- **Audio Search Display Override** (2026-01-06): Added optional "Display As" fields to audio search, allowing users to search for audio by one artist/title (e.g., "Jeremy Kushnier") but display a different artist/title on title screens and filenames (e.g., "Footloose (Broadway Cast)"). Useful for Broadway casts, covers, remixes where tracker metadata differs from desired display. See [API.md](API.md#audio-search).
|
||||
|
||||
- **Infrastructure Modularization** (2026-01-06): Refactored `infrastructure/__main__.py` from 2,602 lines to 339 lines (87% reduction). Split into organized modules: `modules/` for core GCP resources, `compute/` for VMs and startup scripts, `config.py` for shared constants. Extracted embedded encoding worker Python app to `backend/services/gce_encoding/`. See `infrastructure/README.md` for new structure and `infrastructure/docs/PHASE5-PACKER-IMAGE-PLAN.md` for planned Packer image optimization.
|
||||
|
||||
- **Email Notification System** (2026-01-06): Added automated email notifications for job completion and user action reminders. Features: GCS-backed HTML email templates with fallback defaults, SendGrid with CC support, auto-completion emails on job finish, idle reminder emails via Cloud Tasks (5-min delay for blocking states), admin UI buttons to copy message or send email manually. Endpoints: `GET /api/admin/jobs/{id}/completion-message`, `POST /api/admin/jobs/{id}/send-completion-email`. Feature flag `ENABLE_AUTO_EMAILS` (default: false). See [API.md](API.md#email-notifications-admin) and [ARCHITECTURE.md](ARCHITECTURE.md#video-worker-orchestrator).
|
||||
|
||||
- **GCE Preview Encoding Unified with LocalPreviewEncodingService** (2026-01-06): GCE preview encoding now uses the same `LocalPreviewEncodingService` from the installed wheel, eliminating 100 lines of duplicated FFmpeg logic. Preview videos are now identical across local CLI, Cloud Run, and GCE worker environments. Benefits include: consistent encoding settings (480x270, 24fps, crf 28), proper FFmpeg filter path escaping for special characters, hardware acceleration (NVENC) when available, and custom font support. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#unify-preview-encoding-with-localpreviewencodingservice).
|
||||
|
||||
- **GCE Encoding Worker Python 3.13** (2026-01-06): Upgraded GCE encoding worker from Debian's Python 3.11 to Python 3.13 built from source. Uses dedicated virtual environment at `/opt/encoding-worker/venv`, removing the need for `--break-system-packages`. Aligns the encoding worker with CI/Cloud Run Python version.
|
||||
|
||||
- **GCE Encoding Unified with LocalEncodingService** (2026-01-06): GCE encoding worker now uses the same `LocalEncodingService` as the local CLI, eliminating duplicated encoding logic. Output files now have proper names (`Artist - Title (Final Karaoke Lossless 4k).mp4` instead of `output_4k_lossless.mp4`) and include title/end screen concatenation. Wheel deployed to GCS; worker installs at job start for hot updates without VM restart. Removed all fallback logic - single code path for consistent output across CLI, Cloud Run, and GCE. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#unify-encoding-logic-with-gcs-wheel-deployment).
|
||||
|
||||
- **Cloud Distribution Matches Local CLI** (2026-01-06): Fixed cloud distribution (Dropbox uploads) to match local CLI output structure. The `lyrics/` folder now includes: proper filenames (`(Lyrics Corrections).json` instead of `(Karaoke).json`), all reference lyrics files (`(Lyrics Genius).txt`, `(Lyrics Spotify).txt`, etc.), `(With Vocals).mkv`, and `previews/` subfolder with preview ASS files. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#cloud-distribution-must-match-local-cli-output-structure).
|
||||
|
||||
- **Agentic Correction Performance** (2026-01-05): Optimized agentic AI correction from ~5 minutes to ~55 seconds for 20 gaps (~5-6x speedup). Fixed anti-pattern of creating new model instance per gap (caused repeated 2s warm-up overhead). Now creates model once and processes gaps in parallel using ThreadPoolExecutor (default 5 workers). Configure via `AGENTIC_MAX_PARALLEL_GAPS` env var. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#reuse-llm-model-instances-across-operations).
|
||||
|
||||
- **Worker Logs Subcollection** (2026-01-04): Moved `worker_logs` from embedded array in job documents to Firestore subcollection (`jobs/{job_id}/logs`). Fixes job failures when logs exceed 1MB (job 501258e1 had 1.26MB of logs). New logs stored with 30-day TTL via Firestore TTL policy. Feature flag `USE_LOG_SUBCOLLECTION=true` (default). See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#firestore-document-1mb-limit-with-embedded-arrays).
|
||||
|
||||
- **GCE Encoding Response Fixes** (2026-01-04): Fixed multiple response format mismatches with GCE encoding worker. The worker returns `output_files` as a list of paths, not a dict with format keys - added conversion logic. Also added defensive type checking for status responses that could be lists. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#external-service-response-format-mismatches).
|
||||
@@ -81,6 +107,7 @@
|
||||
| [ARCHITECTURE.md](ARCHITECTURE.md) | System design, data flow, integrations |
|
||||
| [DEVELOPMENT.md](DEVELOPMENT.md) | Local setup, testing, deployment |
|
||||
| [API.md](API.md) | Backend endpoint reference |
|
||||
| [BRAND-STYLE-GUIDE.md](BRAND-STYLE-GUIDE.md) | Brand colors, typography, UI patterns |
|
||||
| [LESSONS-LEARNED.md](LESSONS-LEARNED.md) | Key insights for future agents |
|
||||
| [STRIPE-SETUP.md](STRIPE-SETUP.md) | Payment integration setup |
|
||||
| [MOBILE-UX-BEST-PRACTICES.md](MOBILE-UX-BEST-PRACTICES.md) | Mobile/responsive design guidelines |
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
# Brand Consistency Implementation - 2026-01-07
|
||||
|
||||
## Summary
|
||||
|
||||
Implemented unified brand theming across all Nomad Karaoke frontends and created a comprehensive brand style guide for human and LLM reference.
|
||||
|
||||
## Key Changes
|
||||
|
||||
### Frontend (Next.js)
|
||||
- Added `ThemeToggle` component using `next-themes`
|
||||
- Made all components theme-aware (light/dark mode support)
|
||||
- Changed primary color from blue to brand pink (#ff7acc)
|
||||
- Updated CSS variables for consistent theming
|
||||
- Added brand screenshots for visual verification
|
||||
|
||||
### Lyrics Transcriber Frontend (MUI)
|
||||
- Updated MUI theme with brand colors
|
||||
- Made timeline, modals, and controls theme-aware
|
||||
- Fixed mobile responsiveness issues
|
||||
- Added mobile TAP button for manual sync
|
||||
- Improved WordDivider for mobile/desktop parity
|
||||
|
||||
### Email Templates
|
||||
- Refactored to use reusable helper methods (`_build_email_html`, `_get_email_header`, `_get_email_footer`)
|
||||
- Fixed success color to match brand guide (#22c55e)
|
||||
- Updated preview script to use actual email service instead of duplicated content
|
||||
- Changed email header from emoji to logo image
|
||||
|
||||
### Brand Style Guide (docs/BRAND-STYLE-GUIDE.md)
|
||||
- Created comprehensive v1.1 guide covering:
|
||||
- Color palette (primary, supporting, semantic colors)
|
||||
- Typography (font stack, type scale)
|
||||
- Logo usage (SVG preferred, hosted GIF for emails)
|
||||
- UI components (buttons, cards, badges)
|
||||
- Email design standards
|
||||
- Light/dark theme requirements with `next-themes` examples
|
||||
- Implementation reference (CSS variables, Tailwind config)
|
||||
|
||||
## Decisions Made
|
||||
|
||||
1. **Dark theme as default** - More common in modern applications, easier on the eyes for karaoke use
|
||||
2. **SVG logo preferred** - Scalable, smaller file size; GIF only for emails where SVG not supported
|
||||
3. **No emojis in headers** - Always use the Nomad Karaoke logo for brand consistency
|
||||
4. **Hosted logo URL for emails only** - Local SVG for webapp, hosted GIF only when necessary
|
||||
|
||||
## Files Changed
|
||||
|
||||
### Core Brand Files
|
||||
- `docs/BRAND-STYLE-GUIDE.md` - New comprehensive guide
|
||||
- `frontend/components/ThemeToggle.tsx` - New theme toggle
|
||||
- `frontend/app/globals.css` - Updated CSS variables
|
||||
|
||||
### Email System
|
||||
- `backend/services/email_service.py` - Refactored with reusable helpers
|
||||
- `scripts/preview-emails.py` - Now uses real email service
|
||||
|
||||
### Lyrics Transcriber (submodule)
|
||||
- Multiple component files updated for theme awareness
|
||||
- `src/theme.ts` - Updated MUI theme with brand colors
|
||||
|
||||
## Future Considerations
|
||||
|
||||
- Consider adding ambient glow animation to landing page background
|
||||
- Monitor theme toggle usage to ensure discoverability
|
||||
- Keep brand guide updated as new components are added
|
||||
@@ -0,0 +1,270 @@
|
||||
# Implementation Plan: SpaCy Preloading at Container Startup
|
||||
|
||||
**Date**: 2026-01-08
|
||||
**Status**: Ready for implementation
|
||||
**Related Issue**: Job `2ccbdf6b` slow execution investigation
|
||||
**Related PR**: #232 (thread-safety fix - already merged)
|
||||
|
||||
## Background
|
||||
|
||||
Job `2ccbdf6b` (ABBA - Waterloo) took 17 minutes instead of ~5 minutes expected. Investigation revealed two issues:
|
||||
|
||||
1. **FIXED (PR #232)**: Race condition in `LangChainBridge` where 5 parallel threads all tried to initialize the AI model simultaneously, causing 6+ minute delays.
|
||||
|
||||
2. **PENDING (this plan)**: SpaCy model loading takes ~63 seconds even on warm Cloud Run instances. This happens when `PhraseAnalyzer` is first instantiated during request processing.
|
||||
|
||||
## Problem Details
|
||||
|
||||
From job `2ccbdf6b` worker logs:
|
||||
|
||||
- `04:10:21` - "Initializing PhraseAnalyzer with language model: en_core_web_sm"
|
||||
- `04:11:24` - "Initialized AnchorSequenceFinder" (63 seconds later)
|
||||
|
||||
Key observations:
|
||||
- **SpaCy model**: `en_core_web_sm` (~14MB) is already pre-downloaded in `backend/Dockerfile.base` line 64
|
||||
- **Cloud Run config**: `min-instances: 4` configured - instances are warm
|
||||
- **Root cause**: SpaCy loads lazily when first needed during request processing, not at container startup
|
||||
- **Cloud Run I/O**: Filesystem access on Cloud Run can be slow, causing extended load times
|
||||
|
||||
## Solution: Preload SpaCy at Container Startup
|
||||
|
||||
Load SpaCy during FastAPI lifespan handler (before accepting requests) and make it available as a singleton for reuse.
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
### 1. NEW: `backend/services/spacy_preloader.py`
|
||||
|
||||
Create a new module to handle SpaCy preloading:
|
||||
|
||||
```python
|
||||
"""SpaCy model preloader for container startup.
|
||||
|
||||
Loads SpaCy models at container startup to avoid slow loading during request processing.
|
||||
Cloud Run filesystem I/O can cause 60+ second delays when loading SpaCy models lazily.
|
||||
"""
|
||||
import logging
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Singleton storage for preloaded models
|
||||
_preloaded_models: dict = {}
|
||||
|
||||
|
||||
def preload_spacy_model(model_name: str = "en_core_web_sm") -> None:
|
||||
"""Preload a SpaCy model at startup.
|
||||
|
||||
Args:
|
||||
model_name: The SpaCy model to load (default: en_core_web_sm)
|
||||
"""
|
||||
global _preloaded_models
|
||||
|
||||
if model_name in _preloaded_models:
|
||||
logger.info(f"SpaCy model '{model_name}' already preloaded")
|
||||
return
|
||||
|
||||
logger.info(f"Preloading SpaCy model '{model_name}'...")
|
||||
start_time = time.time()
|
||||
|
||||
try:
|
||||
import spacy
|
||||
nlp = spacy.load(model_name)
|
||||
_preloaded_models[model_name] = nlp
|
||||
|
||||
elapsed = time.time() - start_time
|
||||
logger.info(f"SpaCy model '{model_name}' preloaded in {elapsed:.2f}s")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to preload SpaCy model '{model_name}': {e}")
|
||||
raise
|
||||
|
||||
|
||||
def get_preloaded_model(model_name: str = "en_core_web_sm") -> Optional[object]:
|
||||
"""Get a preloaded SpaCy model if available.
|
||||
|
||||
Args:
|
||||
model_name: The SpaCy model name
|
||||
|
||||
Returns:
|
||||
The preloaded SpaCy Language object, or None if not preloaded
|
||||
"""
|
||||
return _preloaded_models.get(model_name)
|
||||
|
||||
|
||||
def is_model_preloaded(model_name: str = "en_core_web_sm") -> bool:
|
||||
"""Check if a SpaCy model has been preloaded."""
|
||||
return model_name in _preloaded_models
|
||||
```
|
||||
|
||||
### 2. MODIFY: `backend/main.py`
|
||||
|
||||
Add SpaCy preloading to the lifespan handler. Current code is at lines 62-80.
|
||||
|
||||
**Add import at top of file:**
|
||||
```python
|
||||
from backend.services.spacy_preloader import preload_spacy_model
|
||||
```
|
||||
|
||||
**Modify lifespan function (around line 62):**
|
||||
```python
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Lifespan event handler for startup and shutdown."""
|
||||
# Startup
|
||||
logger.info("Starting karaoke generation backend")
|
||||
logger.info(f"Environment: {settings.environment}")
|
||||
logger.info(f"GCS Bucket: {settings.gcs_bucket_name}")
|
||||
logger.info(f"Tracing enabled: {tracing_enabled}")
|
||||
|
||||
# Preload SpaCy model to avoid 60+ second delay on first request
|
||||
try:
|
||||
preload_spacy_model("en_core_web_sm")
|
||||
except Exception as e:
|
||||
logger.warning(f"SpaCy preload failed (will load lazily): {e}")
|
||||
|
||||
# Validate OAuth credentials (non-blocking)
|
||||
try:
|
||||
await validate_credentials_on_startup()
|
||||
except Exception as e:
|
||||
logger.error(f"Credential validation failed: {e}")
|
||||
|
||||
yield
|
||||
|
||||
# Shutdown
|
||||
logger.info("Shutting down karaoke generation backend")
|
||||
```
|
||||
|
||||
### 3. MODIFY: `lyrics_transcriber_temp/lyrics_transcriber/correction/phrase_analyzer.py`
|
||||
|
||||
The `PhraseAnalyzer` class loads SpaCy in its `__init__` method. Modify to use preloaded model if available.
|
||||
|
||||
**Add imports at top:**
|
||||
```python
|
||||
# Try to import preloader (may not exist in standalone library usage)
|
||||
try:
|
||||
from backend.services.spacy_preloader import get_preloaded_model
|
||||
_HAS_PRELOADER = True
|
||||
except ImportError:
|
||||
_HAS_PRELOADER = False
|
||||
```
|
||||
|
||||
**Modify `__init__` method:**
|
||||
```python
|
||||
def __init__(self, logger: logging.Logger, language_code: str = "en_core_web_sm"):
|
||||
self.logger = logger
|
||||
|
||||
# Try to use preloaded model first (avoids 60+ second load on Cloud Run)
|
||||
if _HAS_PRELOADER:
|
||||
preloaded = get_preloaded_model(language_code)
|
||||
if preloaded is not None:
|
||||
self.logger.info(f"Using preloaded SpaCy model: {language_code}")
|
||||
self.nlp = preloaded
|
||||
return
|
||||
|
||||
# Fall back to loading model directly
|
||||
self.logger.info(f"Initializing PhraseAnalyzer with language model: {language_code}")
|
||||
try:
|
||||
self.nlp = spacy.load(language_code)
|
||||
except OSError:
|
||||
# ... keep existing download fallback code ...
|
||||
```
|
||||
|
||||
### 4. MODIFY: `lyrics_transcriber_temp/lyrics_transcriber/correction/handlers/syllables_match.py`
|
||||
|
||||
This file also loads SpaCy. Apply the same pattern.
|
||||
|
||||
**Add imports at top:**
|
||||
```python
|
||||
try:
|
||||
from backend.services.spacy_preloader import get_preloaded_model
|
||||
_HAS_PRELOADER = True
|
||||
except ImportError:
|
||||
_HAS_PRELOADER = False
|
||||
```
|
||||
|
||||
**Modify the SpaCy loading section in `__init__`:**
|
||||
```python
|
||||
# Try to use preloaded model first
|
||||
if _HAS_PRELOADER:
|
||||
preloaded = get_preloaded_model("en_core_web_sm")
|
||||
if preloaded is not None:
|
||||
self.logger.info("Using preloaded SpaCy model for syllable analysis")
|
||||
self.nlp = preloaded
|
||||
# Still need to add syllables component if not present
|
||||
if "syllables" not in self.nlp.pipe_names:
|
||||
self.nlp.add_pipe("syllables", after="tagger")
|
||||
return
|
||||
|
||||
# ... existing loading code as fallback ...
|
||||
```
|
||||
|
||||
## Testing Approach
|
||||
|
||||
### Unit Tests for Preloader
|
||||
|
||||
See `backend/tests/test_spacy_preloader.py` for the actual implementation. Key tests:
|
||||
|
||||
- `test_preload_spacy_model_loads_and_stores` - Verifies model is loaded and stored
|
||||
- `test_preload_is_idempotent` - Verifies repeated calls don't reload
|
||||
- `test_get_preloaded_model_returns_none_if_not_loaded` - Verifies None returned when not preloaded
|
||||
- `test_preload_different_models` - Verifies multiple models can be preloaded
|
||||
- `test_clear_preloaded_models` - Verifies clearing works
|
||||
- `test_preload_failure_raises_exception` - Verifies errors propagate
|
||||
- `test_preload_uses_default_model_name` - Verifies default is en_core_web_sm
|
||||
|
||||
All tests use `clear_preloaded_models()` in setup/teardown for isolation.
|
||||
|
||||
### Integration Verification
|
||||
|
||||
After deployment, check Cloud Run logs:
|
||||
|
||||
1. **At startup**: Look for "SpaCy model 'en_core_web_sm' preloaded in X.XXs"
|
||||
2. **During job processing**: Look for "Using preloaded SpaCy model: en_core_web_sm"
|
||||
3. **Timing**: Verify no 60+ second gaps in agentic correction phase
|
||||
|
||||
## Expected Outcomes
|
||||
|
||||
1. **Startup time**: SpaCy load time (~60s) moves from request processing to container startup
|
||||
2. **Request latency**: First agentic correction request no longer has 60+ second SpaCy delay
|
||||
3. **Visibility**: Clear logs showing preload time at startup
|
||||
4. **Total job time**: Agentic correction phase should complete in ~5 minutes instead of ~17 minutes
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Create new worktree:
|
||||
```bash
|
||||
cd /Users/andrew/Projects/karaoke-gen-multiagent/karaoke-gen
|
||||
git worktree add -b feat/spacy-preload ../karaoke-gen-spacy-preload origin/main
|
||||
cd ../karaoke-gen-spacy-preload
|
||||
```
|
||||
|
||||
2. Create `backend/services/spacy_preloader.py`
|
||||
|
||||
3. Modify `backend/main.py` to add preloading in lifespan
|
||||
|
||||
4. Modify `lyrics_transcriber_temp/lyrics_transcriber/correction/phrase_analyzer.py`
|
||||
|
||||
5. Modify `lyrics_transcriber_temp/lyrics_transcriber/correction/handlers/syllables_match.py`
|
||||
|
||||
6. Create unit tests for the preloader
|
||||
|
||||
7. Run tests:
|
||||
```bash
|
||||
make test 2>&1 | tail -n 500
|
||||
```
|
||||
|
||||
8. Bump version in `pyproject.toml`
|
||||
|
||||
9. Follow standard PR workflow: `/docs-review` -> `/coderabbit` -> `/pr`
|
||||
|
||||
## Risk Considerations
|
||||
|
||||
- **Startup time increase**: Container startup will take ~60s longer, but this happens once per container lifecycle, not per request
|
||||
- **Memory usage**: SpaCy model (~50MB in memory) will persist for container lifetime - acceptable tradeoff
|
||||
- **Import error handling**: Graceful fallback if preloader module unavailable (library standalone use case)
|
||||
|
||||
## Related Files Reference
|
||||
|
||||
- `backend/Dockerfile.base:64` - SpaCy model download at build time
|
||||
- `infrastructure/cloudrun.ts` - Cloud Run configuration (min-instances: 4)
|
||||
- `docs/LESSONS-LEARNED.md` - Contains thread-safety lesson from related fix
|
||||
@@ -26,7 +26,7 @@ describe('getJobStep', () => {
|
||||
expect(result.total).toBe(10);
|
||||
expect(result.label).toBe('Setting up');
|
||||
expect(result.isBlocking).toBe(false);
|
||||
expect(result.color).toBe('text-slate-400');
|
||||
expect(result.color).toBe('text-muted-foreground');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -328,7 +328,7 @@ describe('getJobStep', () => {
|
||||
const result = getJobStep(job);
|
||||
expect(result.step).toBe(0);
|
||||
expect(result.label).toBe('Cancelled');
|
||||
expect(result.color).toBe('text-slate-500');
|
||||
expect(result.color).toBe('text-muted-foreground');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -339,7 +339,7 @@ describe('getJobStep', () => {
|
||||
expect(result.step).toBe(0);
|
||||
expect(result.label).toBe('some unknown status');
|
||||
expect(result.isBlocking).toBe(false);
|
||||
expect(result.color).toBe('text-slate-400');
|
||||
expect(result.color).toBe('text-muted-foreground');
|
||||
});
|
||||
|
||||
it('handles null status gracefully', () => {
|
||||
|
||||
@@ -125,7 +125,7 @@ export default function AdminJobDetailPage() {
|
||||
case "INFO":
|
||||
return "text-blue-500"
|
||||
case "DEBUG":
|
||||
return "text-gray-500"
|
||||
return "text-muted-foreground"
|
||||
default:
|
||||
return "text-foreground"
|
||||
}
|
||||
|
||||
@@ -164,7 +164,7 @@ export default function AdminDashboardPage() {
|
||||
</div>
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<XCircle className="w-4 h-4 text-gray-500" />
|
||||
<XCircle className="w-4 h-4 text-muted-foreground" />
|
||||
<span className="text-sm">Cancelled</span>
|
||||
</div>
|
||||
<Badge variant="outline">{stats?.jobs_by_status?.cancelled ?? 0}</Badge>
|
||||
|
||||
@@ -190,7 +190,7 @@ export default function AdminSearchesPage() {
|
||||
case "ops":
|
||||
return "bg-blue-500/20 text-blue-400 border-blue-500/30"
|
||||
default:
|
||||
return "bg-slate-500/20 text-slate-400 border-slate-500/30"
|
||||
return "bg-muted text-muted-foreground border-border"
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
import { useState, useEffect, useCallback } from "react"
|
||||
import { useRouter } from "next/navigation"
|
||||
import { api, Job, getAccessToken } from "@/lib/api"
|
||||
import { useAuth } from "@/lib/auth"
|
||||
import { useJobNotifications, useVisibilityRefresh } from "@/hooks/use-notifications"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
|
||||
@@ -28,6 +29,7 @@ export default function AppPage() {
|
||||
const [isInitialLoad, setIsInitialLoad] = useState(true)
|
||||
const [isAuthenticated, setIsAuthenticated] = useState<boolean | null>(null)
|
||||
const { isDarkMode, toggleTheme, mounted } = useTheme()
|
||||
const { user, fetchUser } = useAuth()
|
||||
|
||||
// Memoize loadJobs for use with visibility refresh
|
||||
const loadJobs = useCallback(async () => {
|
||||
@@ -67,6 +69,16 @@ export default function AppPage() {
|
||||
setIsAuthenticated(true)
|
||||
}, [router])
|
||||
|
||||
// Ensure user data is loaded after authentication
|
||||
// This handles the case where user navigates via client-side navigation
|
||||
// (e.g., after beta enrollment) and the module-level fetchUser() didn't run
|
||||
useEffect(() => {
|
||||
const token = getAccessToken()
|
||||
if (token && !user) {
|
||||
fetchUser()
|
||||
}
|
||||
}, [user, fetchUser])
|
||||
|
||||
// Load jobs on mount (only if authenticated)
|
||||
useEffect(() => {
|
||||
if (isAuthenticated !== true) {
|
||||
|
||||
@@ -51,14 +51,14 @@ function VerifyMagicLinkContent() {
|
||||
}, [searchParams, verifyMagicLink, router])
|
||||
|
||||
return (
|
||||
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
|
||||
<div className="max-w-md w-full bg-card border border-border rounded-xl p-8 text-center">
|
||||
{state === "loading" && (
|
||||
<>
|
||||
<Loader2 className="w-12 h-12 text-blue-400 animate-spin mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-white mb-2">
|
||||
<Loader2 className="w-12 h-12 text-primary animate-spin mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-foreground mb-2">
|
||||
Verifying your sign-in link...
|
||||
</h1>
|
||||
<p className="text-slate-400">
|
||||
<p className="text-muted-foreground">
|
||||
Please wait while we sign you in.
|
||||
</p>
|
||||
</>
|
||||
@@ -66,14 +66,14 @@ function VerifyMagicLinkContent() {
|
||||
|
||||
{state === "success" && (
|
||||
<>
|
||||
<CheckCircle className="w-12 h-12 text-green-400 mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-white mb-2">
|
||||
<CheckCircle className="w-12 h-12 text-success mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-foreground mb-2">
|
||||
Successfully signed in!
|
||||
</h1>
|
||||
<p className="text-slate-400 mb-4">
|
||||
<p className="text-muted-foreground mb-4">
|
||||
Welcome back{user?.email ? `, ${user.email}` : ""}!
|
||||
</p>
|
||||
<p className="text-sm text-slate-500">
|
||||
<p className="text-sm text-muted-foreground">
|
||||
Redirecting you to the app...
|
||||
</p>
|
||||
</>
|
||||
@@ -81,21 +81,21 @@ function VerifyMagicLinkContent() {
|
||||
|
||||
{state === "error" && (
|
||||
<>
|
||||
<XCircle className="w-12 h-12 text-red-400 mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-white mb-2">
|
||||
<XCircle className="w-12 h-12 text-destructive mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-foreground mb-2">
|
||||
Sign-in failed
|
||||
</h1>
|
||||
<p className="text-slate-400 mb-6">
|
||||
<p className="text-muted-foreground mb-6">
|
||||
{errorMessage || "The link may have expired or already been used."}
|
||||
</p>
|
||||
<div className="space-y-2">
|
||||
<Button
|
||||
onClick={() => router.push("/app")}
|
||||
className="w-full bg-blue-600 hover:bg-blue-500"
|
||||
className="w-full bg-primary hover:bg-primary/90"
|
||||
>
|
||||
Go to App
|
||||
</Button>
|
||||
<p className="text-xs text-slate-500">
|
||||
<p className="text-xs text-muted-foreground">
|
||||
You can request a new sign-in link from the app.
|
||||
</p>
|
||||
</div>
|
||||
@@ -107,12 +107,12 @@ function VerifyMagicLinkContent() {
|
||||
|
||||
function VerifyLoadingFallback() {
|
||||
return (
|
||||
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
|
||||
<Loader2 className="w-12 h-12 text-blue-400 animate-spin mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-white mb-2">
|
||||
<div className="max-w-md w-full bg-card border border-border rounded-xl p-8 text-center">
|
||||
<Loader2 className="w-12 h-12 text-primary animate-spin mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-foreground mb-2">
|
||||
Loading...
|
||||
</h1>
|
||||
<p className="text-slate-400">
|
||||
<p className="text-muted-foreground">
|
||||
Please wait.
|
||||
</p>
|
||||
</div>
|
||||
@@ -121,7 +121,7 @@ function VerifyLoadingFallback() {
|
||||
|
||||
export default function VerifyMagicLinkPage() {
|
||||
return (
|
||||
<div className="min-h-screen bg-slate-950 flex items-center justify-center p-4">
|
||||
<div className="min-h-screen bg-background flex items-center justify-center p-4">
|
||||
<Suspense fallback={<VerifyLoadingFallback />}>
|
||||
<VerifyMagicLinkContent />
|
||||
</Suspense>
|
||||
|
||||
@@ -4,22 +4,40 @@
|
||||
@custom-variant dark (&:is(.dark *));
|
||||
|
||||
:root {
|
||||
/* Dark theme (default) - Matching instrumental review UI color scheme */
|
||||
/* ===========================================
|
||||
Nomad Karaoke Brand Colors
|
||||
See docs/BRAND-STYLE-GUIDE.md for details
|
||||
=========================================== */
|
||||
|
||||
/* Brand Primary Colors */
|
||||
--brand-pink: #ff7acc;
|
||||
--brand-pink-hover: #ff5bb8;
|
||||
--brand-pink-glow: rgba(255, 122, 204, 0.4);
|
||||
--brand-gold: #ffdf6b;
|
||||
--brand-purple: #8b5cf6;
|
||||
--brand-blue: #3b82f6;
|
||||
--brand-soft-pink: #ec4899;
|
||||
|
||||
/* Dark theme (default) */
|
||||
--bg: #0f0f0f;
|
||||
--card: #1a1a1a;
|
||||
--card-border: #2a2a2a;
|
||||
--waveform-bg: #0d1117;
|
||||
--text: #e5e5e5;
|
||||
--text-muted: #888;
|
||||
--primary: #3b82f6;
|
||||
--primary-hover: #2563eb;
|
||||
--text-muted: #888888;
|
||||
--text-subtle: #666666;
|
||||
|
||||
/* Primary now uses brand pink */
|
||||
--primary: var(--brand-pink);
|
||||
--primary-hover: var(--brand-pink-hover);
|
||||
|
||||
--secondary: #252525;
|
||||
--secondary-hover: #333;
|
||||
--secondary-hover: #333333;
|
||||
--success: #22c55e;
|
||||
--warning: #f59e0b;
|
||||
--danger: #ef4444;
|
||||
--pink: #ec4899;
|
||||
--blue: #60a5fa;
|
||||
--pink: var(--brand-pink);
|
||||
--blue: var(--brand-blue);
|
||||
|
||||
/* Tailwind compatibility mappings */
|
||||
--background: var(--bg);
|
||||
@@ -38,26 +56,38 @@
|
||||
--border: var(--card-border);
|
||||
--input: var(--secondary);
|
||||
--ring: var(--primary);
|
||||
--radius: 0.5rem;
|
||||
--radius: 0.75rem;
|
||||
}
|
||||
|
||||
:root.light {
|
||||
/* Light theme */
|
||||
.light {
|
||||
/* Light theme - applied via next-themes class on html element */
|
||||
--bg: #f8fafc;
|
||||
--card: #ffffff;
|
||||
--card-border: #e2e8f0;
|
||||
--waveform-bg: #f1f5f9;
|
||||
--text: #1e293b;
|
||||
--text-muted: #64748b;
|
||||
--primary: #3b82f6;
|
||||
--primary-hover: #2563eb;
|
||||
--text-subtle: #94a3b8;
|
||||
--primary: var(--brand-pink);
|
||||
--primary-hover: var(--brand-pink-hover);
|
||||
--secondary: #f1f5f9;
|
||||
--secondary-hover: #e2e8f0;
|
||||
--success: #22c55e;
|
||||
--warning: #f59e0b;
|
||||
--danger: #ef4444;
|
||||
--pink: #ec4899;
|
||||
--blue: #60a5fa;
|
||||
--pink: var(--brand-pink);
|
||||
--blue: var(--brand-blue);
|
||||
|
||||
/* Tailwind compatibility - light mode adjustments */
|
||||
--background: var(--bg);
|
||||
--foreground: var(--text);
|
||||
--card-foreground: var(--text);
|
||||
--popover: var(--card);
|
||||
--popover-foreground: var(--text);
|
||||
--muted: var(--secondary);
|
||||
--muted-foreground: var(--text-muted);
|
||||
--border: var(--card-border);
|
||||
--input: var(--secondary);
|
||||
}
|
||||
|
||||
@theme inline {
|
||||
@@ -95,21 +125,45 @@
|
||||
body {
|
||||
@apply bg-background text-foreground;
|
||||
}
|
||||
|
||||
/* Radial gradient background overlay (like decide.nomadkaraoke.com) */
|
||||
body::after {
|
||||
content: '';
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background:
|
||||
radial-gradient(ellipse at 20% 0%, rgba(139, 92, 246, 0.12) 0%, transparent 50%),
|
||||
radial-gradient(ellipse at 80% 100%, rgba(255, 122, 204, 0.1) 0%, transparent 50%),
|
||||
radial-gradient(ellipse at 50% 50%, rgba(255, 223, 107, 0.03) 0%, transparent 70%);
|
||||
pointer-events: none;
|
||||
z-index: -1;
|
||||
}
|
||||
|
||||
/* Light mode uses subtle pink tints */
|
||||
.light body::after {
|
||||
background:
|
||||
radial-gradient(ellipse at 20% 0%, rgba(139, 92, 246, 0.08) 0%, transparent 50%),
|
||||
radial-gradient(ellipse at 80% 100%, rgba(255, 122, 204, 0.06) 0%, transparent 50%),
|
||||
radial-gradient(ellipse at 50% 50%, rgba(255, 223, 107, 0.02) 0%, transparent 70%);
|
||||
}
|
||||
}
|
||||
|
||||
/* Landing page utilities */
|
||||
@layer utilities {
|
||||
/* Glow effect for primary buttons */
|
||||
/* Glow effect for primary buttons - uses brand pink */
|
||||
.btn-glow {
|
||||
box-shadow: 0 0 20px rgba(59, 130, 246, 0.5);
|
||||
box-shadow: 0 0 20px var(--brand-pink-glow);
|
||||
}
|
||||
.btn-glow:hover {
|
||||
box-shadow: 0 0 30px rgba(59, 130, 246, 0.7);
|
||||
box-shadow: 0 0 30px rgba(255, 122, 204, 0.6);
|
||||
}
|
||||
|
||||
/* Gradient text */
|
||||
/* Gradient text - flows to brand pink */
|
||||
.gradient-text {
|
||||
background: linear-gradient(135deg, #3b82f6 0%, #8b5cf6 50%, #ec4899 100%);
|
||||
background: linear-gradient(135deg, #3b82f6 0%, #8b5cf6 50%, #ff7acc 100%);
|
||||
-webkit-background-clip: text;
|
||||
-webkit-text-fill-color: transparent;
|
||||
background-clip: text;
|
||||
@@ -124,31 +178,35 @@
|
||||
box-shadow: 0 20px 40px rgba(0, 0, 0, 0.3);
|
||||
}
|
||||
|
||||
/* Animated background gradient */
|
||||
@keyframes gradient-shift {
|
||||
0% { background-position: 0% 50%; }
|
||||
50% { background-position: 100% 50%; }
|
||||
100% { background-position: 0% 50%; }
|
||||
}
|
||||
/* Animated background gradient - subtle brand integration */
|
||||
/* animated-gradient class - now just ensures min-height, gradient provided by body::after */
|
||||
.animated-gradient {
|
||||
background: linear-gradient(-45deg, #0f172a, #1e293b, #1e3a8a, #312e81);
|
||||
background-size: 400% 400%;
|
||||
animation: gradient-shift 15s ease infinite;
|
||||
/* Background is now provided by body::after radial gradient */
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
/* Dark theme color utilities for landing page */
|
||||
.text-dark-300 { color: #d1d5db; }
|
||||
.text-dark-400 { color: #9ca3af; }
|
||||
.text-dark-500 { color: #6b7280; }
|
||||
.bg-dark-700 { background-color: #374151; }
|
||||
.bg-dark-800 { background-color: #1f2937; }
|
||||
.bg-dark-900 { background-color: #111827; }
|
||||
.border-dark-600 { border-color: #4b5563; }
|
||||
.border-dark-700 { border-color: #374151; }
|
||||
.text-primary-400 { color: #60a5fa; }
|
||||
.text-primary-500 { color: #3b82f6; }
|
||||
.bg-primary-500 { background-color: #3b82f6; }
|
||||
.hover\:bg-primary-600:hover { background-color: #2563eb; }
|
||||
/* Theme-aware color utilities - use CSS variables for theme switching */
|
||||
.text-dark-300 { color: var(--text); }
|
||||
.text-dark-400 { color: var(--text-muted); }
|
||||
.text-dark-500 { color: var(--text-subtle); }
|
||||
.bg-dark-700 { background-color: var(--secondary); }
|
||||
.bg-dark-800 { background-color: var(--card); }
|
||||
.bg-dark-900 { background-color: var(--bg); }
|
||||
.border-dark-600 { border-color: var(--card-border); }
|
||||
.border-dark-700 { border-color: var(--card-border); }
|
||||
|
||||
/* Primary color utilities - now brand pink */
|
||||
.text-primary-400 { color: #ff8fd4; } /* Lighter pink */
|
||||
.text-primary-500 { color: var(--brand-pink); }
|
||||
.bg-primary-500 { background-color: var(--brand-pink); }
|
||||
.hover\:bg-primary-600:hover { background-color: var(--brand-pink-hover); }
|
||||
|
||||
/* Brand accent utilities */
|
||||
.text-brand-pink { color: var(--brand-pink); }
|
||||
.text-brand-gold { color: var(--brand-gold); }
|
||||
.bg-brand-pink { background-color: var(--brand-pink); }
|
||||
.bg-brand-gold { background-color: var(--brand-gold); }
|
||||
.border-brand-pink { border-color: var(--brand-pink); }
|
||||
|
||||
/* Fade in animation for cold start messages */
|
||||
@keyframes fade-in {
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import type React from "react"
|
||||
import type { Metadata } from "next"
|
||||
import { ThemeProvider } from "@/components/theme-provider"
|
||||
import "./globals.css"
|
||||
|
||||
export const metadata: Metadata = {
|
||||
@@ -21,9 +22,16 @@ export default function RootLayout({
|
||||
children: React.ReactNode
|
||||
}>) {
|
||||
return (
|
||||
<html lang="en" className="dark">
|
||||
<html lang="en" suppressHydrationWarning>
|
||||
<body className="font-sans antialiased" style={{ background: 'var(--bg)', color: 'var(--text)' }}>
|
||||
{children}
|
||||
<ThemeProvider
|
||||
attribute="class"
|
||||
defaultTheme="dark"
|
||||
enableSystem
|
||||
disableTransitionOnChange
|
||||
>
|
||||
{children}
|
||||
</ThemeProvider>
|
||||
</body>
|
||||
</html>
|
||||
)
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
|
||||
import { useState, useEffect, useRef } from 'react';
|
||||
import { useRouter } from 'next/navigation';
|
||||
import Image from 'next/image';
|
||||
import {
|
||||
MicVocal,
|
||||
Music,
|
||||
Sparkles,
|
||||
Video,
|
||||
@@ -18,28 +18,29 @@ import {
|
||||
import { api, setAccessToken, getAccessToken, CreditPackage, BetaEnrollResponse } from '@/lib/api';
|
||||
import { AuthDialog } from '@/components/auth/AuthDialog';
|
||||
import { WarmingUpLoader } from '@/components/WarmingUpLoader';
|
||||
import { ThemeToggle } from '@/components/ThemeToggle';
|
||||
|
||||
// FAQ data
|
||||
const faqs = [
|
||||
{
|
||||
question: 'How does it work?',
|
||||
answer:
|
||||
'Enter the artist and song title, and our AI finds the audio, removes the vocals, transcribes and syncs the lyrics, then generates a professional karaoke video. You can review and edit the lyrics before final export.',
|
||||
'Enter an artist and song title (or paste a YouTube link, or upload your own audio file). Our system finds the audio, separates the vocals from the instrumental, transcribes the lyrics, and syncs them to the music. You then review and correct any transcription errors before we generate your final karaoke video.',
|
||||
},
|
||||
{
|
||||
question: 'What songs can I use?',
|
||||
answer:
|
||||
'Any song! Our system searches multiple audio sources to find high-quality versions. If a song exists online, we can likely create a karaoke video for it.',
|
||||
'Any song! Search by artist and title, paste a YouTube URL, or upload your own audio file. If you can get the audio, we can make it into karaoke.',
|
||||
},
|
||||
{
|
||||
question: 'What format is the output?',
|
||||
answer:
|
||||
'You get a 4K MP4 video file with perfectly synced lyrics. We also offer direct YouTube upload and cloud storage integration.',
|
||||
'You get multiple files: a 4K MP4 karaoke video, a "With Vocals" version for sing-along practice, and a CDG+MP3 ZIP for older karaoke systems. All videos are also published to our YouTube channel for easy sharing.',
|
||||
},
|
||||
{
|
||||
question: 'What if something goes wrong?',
|
||||
question: 'Do I need to do anything, or is it fully automatic?',
|
||||
answer:
|
||||
"If the AI makes mistakes, you can edit the lyrics and timing before generating the final video. If there's a technical issue, contact us and we'll make it right.",
|
||||
"The lyrics transcription is very accurate, but you'll need to review it and correct any misheard words before the final video is generated. For songs with clear vocals, there may be few or no corrections needed. For complex songs, expect to spend 5-10 minutes fixing errors. The word timing/sync is handled automatically and is very precise.",
|
||||
},
|
||||
{
|
||||
question: 'Do credits expire?',
|
||||
@@ -69,8 +70,6 @@ export default function LandingPage() {
|
||||
const [betaSuccess, setBetaSuccess] = useState(false);
|
||||
const [betaWillRedirect, setBetaWillRedirect] = useState(false);
|
||||
|
||||
// FAQ expansion state
|
||||
const [expandedFaq, setExpandedFaq] = useState<number | null>(null);
|
||||
|
||||
// Auth dialog state
|
||||
const [showAuthDialog, setShowAuthDialog] = useState(false);
|
||||
@@ -212,16 +211,18 @@ export default function LandingPage() {
|
||||
{/* Navigation */}
|
||||
<nav className="fixed top-0 left-0 right-0 z-50 bg-dark-900/80 backdrop-blur-md border-b border-dark-700">
|
||||
<div className="max-w-6xl mx-auto px-4 py-4 flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<MicVocal className="w-8 h-8 text-primary-500" />
|
||||
<span className="text-xl font-bold">Nomad Karaoke</span>
|
||||
<div className="flex items-center gap-3">
|
||||
<img src="/nomad-karaoke-logo.svg" alt="Nomad Karaoke" className="h-10" />
|
||||
</div>
|
||||
<div className="flex items-center gap-4">
|
||||
<ThemeToggle />
|
||||
<button
|
||||
onClick={() => setShowAuthDialog(true)}
|
||||
className="text-sm text-dark-300 hover:text-white transition-colors"
|
||||
>
|
||||
Already have credits? Sign in
|
||||
</button>
|
||||
</div>
|
||||
<button
|
||||
onClick={() => setShowAuthDialog(true)}
|
||||
className="text-sm text-dark-300 hover:text-white transition-colors"
|
||||
>
|
||||
Already have credits? Sign in
|
||||
</button>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
@@ -229,33 +230,51 @@ export default function LandingPage() {
|
||||
<section className="pt-32 pb-16 px-4">
|
||||
<div className="max-w-4xl mx-auto text-center">
|
||||
<h1 className="text-4xl sm:text-5xl md:text-6xl font-extrabold mb-6 leading-tight">
|
||||
Turn Any Song Into a{' '}
|
||||
<span className="gradient-text">Karaoke Video</span>
|
||||
Create a <span className="gradient-text">Karaoke Video</span> for Any Song
|
||||
</h1>
|
||||
<p className="text-xl text-dark-300 mb-8 max-w-2xl mx-auto">
|
||||
Professional karaoke videos in minutes. AI-powered vocal removal,
|
||||
perfect lyrics sync, and stunning video output.
|
||||
Create professional karaoke videos in under 30 minutes. Real instrumentals
|
||||
from the original song, precise lyrics sync, and 4K video output.
|
||||
</p>
|
||||
<a
|
||||
href="#pricing"
|
||||
href="#beta"
|
||||
className="inline-flex items-center gap-2 bg-primary-500 hover:bg-primary-600 text-white font-semibold px-8 py-4 rounded-xl transition-all btn-glow"
|
||||
>
|
||||
Get Started
|
||||
Try It For Free
|
||||
<ChevronDown className="w-5 h-5" />
|
||||
</a>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Demo Video Section */}
|
||||
<section className="py-8 px-4">
|
||||
<div className="max-w-4xl mx-auto">
|
||||
<div className="bg-dark-800 border border-dark-700 rounded-2xl p-2 overflow-hidden">
|
||||
<div className="aspect-video">
|
||||
<iframe
|
||||
className="w-full h-full rounded-xl"
|
||||
src="https://www.youtube.com/embed/wHiMti3xLJE?si=dij5wsaHnedaZdFg"
|
||||
title="Nomad Karaoke Demo"
|
||||
frameBorder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
referrerPolicy="strict-origin-when-cross-origin"
|
||||
allowFullScreen
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* How It Works */}
|
||||
<section className="py-16 px-4">
|
||||
<div className="max-w-6xl mx-auto">
|
||||
<h2 className="text-3xl font-bold text-center mb-12">How It Works</h2>
|
||||
<div className="grid md:grid-cols-4 gap-8">
|
||||
{[
|
||||
{ icon: Music, title: 'Search', desc: 'Enter artist & song title' },
|
||||
{ icon: Sparkles, title: 'AI Magic', desc: 'We remove vocals & sync lyrics' },
|
||||
{ icon: Video, title: 'Review', desc: 'Preview and fine-tune' },
|
||||
{ icon: Youtube, title: 'Export', desc: 'Download or upload to YouTube' },
|
||||
{ icon: Music, title: 'Choose a Song', desc: 'Search, paste a YouTube link, or upload audio' },
|
||||
{ icon: Sparkles, title: 'We Do the Heavy Lifting', desc: 'Vocals removed, lyrics transcribed & synced' },
|
||||
{ icon: Video, title: 'Review & Correct', desc: 'Fix any transcription errors (usually 5-10 min)' },
|
||||
{ icon: Youtube, title: 'Get Your Video', desc: 'Download files or watch on our YouTube' },
|
||||
].map((step, i) => (
|
||||
<div key={i} className="text-center">
|
||||
<div className="w-16 h-16 bg-primary-500/20 rounded-2xl flex items-center justify-center mx-auto mb-4">
|
||||
@@ -275,12 +294,12 @@ export default function LandingPage() {
|
||||
<h2 className="text-3xl font-bold text-center mb-12">What You Get</h2>
|
||||
<div className="grid md:grid-cols-3 gap-6">
|
||||
{[
|
||||
{ title: 'Studio-Quality Separation', desc: 'State-of-the-art AI removes vocals cleanly, keeping the instrumental crisp.' },
|
||||
{ title: 'Perfect Lyrics Sync', desc: 'Word-by-word timing that highlights exactly when to sing.' },
|
||||
{ title: 'Multiple Instrumental Options', desc: 'Choose from different vocal removal levels to get the sound you want.' },
|
||||
{ title: '4K Video Output', desc: 'Stunning video quality that looks great on any screen.' },
|
||||
{ title: 'YouTube Integration', desc: 'Upload directly to your YouTube channel with one click.' },
|
||||
{ title: 'Edit Before Export', desc: 'Fix any lyrics issues before generating your final video.' },
|
||||
{ title: 'Real Instrumentals', desc: "Uses the actual instrumental from your song—not a cover band or MIDI recreation." },
|
||||
{ title: 'Precise Lyrics Sync', desc: 'Word-by-word timing that highlights exactly when to sing. You review the lyrics for accuracy.' },
|
||||
{ title: 'Keep or Remove Backing Vocals', desc: 'Choose a clean instrumental or one that preserves backing vocals for a fuller sound.' },
|
||||
{ title: '4K Video + Multiple Formats', desc: 'Get a 4K karaoke video, a sing-along version with vocals, and CDG+MP3 for older systems.' },
|
||||
{ title: 'Published to YouTube', desc: 'Every video is automatically published to our YouTube channel for easy sharing.' },
|
||||
{ title: 'Full Control Before Export', desc: 'Review and correct any transcription errors before the final video is generated.' },
|
||||
].map((feature, i) => (
|
||||
<div key={i} className="bg-dark-800 border border-dark-700 rounded-xl p-6 card-hover">
|
||||
<h3 className="font-semibold text-lg mb-2">{feature.title}</h3>
|
||||
@@ -291,25 +310,104 @@ export default function LandingPage() {
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Screenshots Section */}
|
||||
<section className="py-16 px-4">
|
||||
<div className="max-w-6xl mx-auto">
|
||||
<h2 className="text-3xl font-bold text-center mb-4">See It In Action</h2>
|
||||
<p className="text-dark-400 text-center mb-12 max-w-xl mx-auto">
|
||||
Here's what the process looks like at each step
|
||||
</p>
|
||||
<div className="grid md:grid-cols-2 gap-8">
|
||||
{/* Job Dashboard Screenshot */}
|
||||
<div className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
|
||||
<div className="aspect-video bg-dark-900 relative">
|
||||
<Image
|
||||
src="/screenshots/job-dashboard.avif"
|
||||
alt="Job Dashboard showing karaoke video projects with status indicators"
|
||||
fill
|
||||
className="object-cover"
|
||||
sizes="(max-width: 768px) 100vw, 50vw"
|
||||
/>
|
||||
</div>
|
||||
<div className="p-4">
|
||||
<h3 className="font-semibold mb-1">Job Dashboard</h3>
|
||||
<p className="text-dark-400 text-sm">Track progress of all your karaoke video projects</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Lyrics Review Screenshot */}
|
||||
<div className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
|
||||
<div className="aspect-video bg-dark-900 relative">
|
||||
<Image
|
||||
src="/screenshots/lyrics-review.avif"
|
||||
alt="Lyrics Review UI with word-by-word editing and timing controls"
|
||||
fill
|
||||
className="object-cover"
|
||||
sizes="(max-width: 768px) 100vw, 50vw"
|
||||
/>
|
||||
</div>
|
||||
<div className="p-4">
|
||||
<h3 className="font-semibold mb-1">Lyrics Review</h3>
|
||||
<p className="text-dark-400 text-sm">Correct any transcription errors before generating your video</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Instrumental Selection Screenshot */}
|
||||
<div className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
|
||||
<div className="aspect-video bg-dark-900 relative">
|
||||
<Image
|
||||
src="/screenshots/instrumental-review.avif"
|
||||
alt="Instrumental Selection UI with audio player and backing vocals options"
|
||||
fill
|
||||
className="object-cover"
|
||||
sizes="(max-width: 768px) 100vw, 50vw"
|
||||
/>
|
||||
</div>
|
||||
<div className="p-4">
|
||||
<h3 className="font-semibold mb-1">Instrumental Selection</h3>
|
||||
<p className="text-dark-400 text-sm">Pick the sound that works best for your karaoke</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Example Output Screenshot */}
|
||||
<div className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
|
||||
<div className="aspect-video bg-dark-900 relative">
|
||||
<Image
|
||||
src="/screenshots/example-output.avif"
|
||||
alt="Example karaoke video output with synchronized lyrics"
|
||||
fill
|
||||
className="object-cover"
|
||||
sizes="(max-width: 768px) 100vw, 50vw"
|
||||
/>
|
||||
</div>
|
||||
<div className="p-4">
|
||||
<h3 className="font-semibold mb-1">Final Video</h3>
|
||||
<p className="text-dark-400 text-sm">4K karaoke video ready for your next singing session</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Beta Tester Program */}
|
||||
<section className="py-12 px-4">
|
||||
<section id="beta" className="py-12 px-4">
|
||||
<div className="max-w-2xl mx-auto">
|
||||
<div className="bg-gradient-to-r from-purple-900/40 to-pink-900/40 border border-purple-500/30 rounded-2xl p-8 text-center">
|
||||
<div className="inline-flex items-center gap-2 bg-purple-500/20 text-purple-300 px-4 py-2 rounded-full text-sm font-medium mb-4">
|
||||
<div className="bg-gradient-to-r from-pink-500/10 to-purple-500/10 border border-primary/30 rounded-2xl p-8 text-center">
|
||||
<div className="inline-flex items-center gap-2 bg-primary-500/20 text-primary-400 px-4 py-2 rounded-full text-sm font-medium mb-4">
|
||||
<Gift className="w-4 h-4" />
|
||||
Beta Tester Program
|
||||
</div>
|
||||
<h3 className="text-2xl font-bold mb-3">Try It Free!</h3>
|
||||
<p className="text-dark-300 mb-6">
|
||||
Help us improve by testing the tool and sharing your feedback. Get a{' '}
|
||||
<span className="text-green-400 font-semibold">free credit</span> to
|
||||
create your first karaoke video!
|
||||
Help us improve by testing the tool and sharing your feedback. Beta testers get{' '}
|
||||
<span className="text-green-400 font-semibold">up to 5 free karaoke videos</span>{' '}
|
||||
in exchange for providing feedback on each one.
|
||||
</p>
|
||||
|
||||
{!showBetaForm && !betaSuccess && (
|
||||
<button
|
||||
onClick={() => setShowBetaForm(true)}
|
||||
className="inline-flex items-center gap-2 bg-purple-500 hover:bg-purple-600 text-white font-semibold px-6 py-3 rounded-xl transition-all"
|
||||
className="inline-flex items-center gap-2 bg-primary-500 hover:bg-primary-600 text-white font-semibold px-6 py-3 rounded-xl transition-all btn-glow"
|
||||
>
|
||||
Join Beta Program
|
||||
<MessageSquare className="w-5 h-5" />
|
||||
@@ -328,7 +426,7 @@ export default function LandingPage() {
|
||||
value={betaEmail}
|
||||
onChange={(e) => setBetaEmail(e.target.value)}
|
||||
placeholder="you@example.com"
|
||||
className="w-full px-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-purple-500 text-white placeholder-dark-500"
|
||||
className="w-full px-4 py-3 bg-secondary border border-border rounded-lg focus:outline-none focus:ring-2 focus:ring-primary text-foreground placeholder:text-muted-foreground"
|
||||
required
|
||||
/>
|
||||
</div>
|
||||
@@ -343,7 +441,7 @@ export default function LandingPage() {
|
||||
onChange={(e) => setBetaPromise(e.target.value)}
|
||||
placeholder="Tell us about a song you'd love to sing..."
|
||||
rows={3}
|
||||
className="w-full px-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-purple-500 text-white placeholder-dark-500 resize-none"
|
||||
className="w-full px-4 py-3 bg-secondary border border-border rounded-lg focus:outline-none focus:ring-2 focus:ring-primary text-foreground placeholder:text-muted-foreground resize-none"
|
||||
/>
|
||||
</div>
|
||||
|
||||
@@ -353,10 +451,10 @@ export default function LandingPage() {
|
||||
id="beta-accept"
|
||||
checked={betaAccept}
|
||||
onChange={(e) => setBetaAccept(e.target.checked)}
|
||||
className="mt-1 w-4 h-4 rounded border-dark-600 bg-dark-900 text-purple-500 focus:ring-purple-500"
|
||||
className="mt-1 w-4 h-4 rounded border-border bg-secondary text-primary focus:ring-primary"
|
||||
/>
|
||||
<label htmlFor="beta-accept" className="text-sm text-dark-300">
|
||||
I understand that beta testers help improve lyrics accuracy by reviewing and correcting AI-generated lyrics
|
||||
<label htmlFor="beta-accept" className="text-sm text-muted-foreground">
|
||||
I understand that I'll need to review and correct any lyrics transcription errors, and provide feedback after each video
|
||||
</label>
|
||||
</div>
|
||||
|
||||
@@ -367,7 +465,7 @@ export default function LandingPage() {
|
||||
<button
|
||||
type="submit"
|
||||
disabled={betaLoading}
|
||||
className="w-full bg-purple-500 hover:bg-purple-600 disabled:bg-purple-500/50 text-white font-semibold py-3 rounded-xl transition-all flex items-center justify-center gap-2"
|
||||
className="w-full bg-primary-500 hover:bg-primary-600 disabled:bg-primary-500/50 text-white font-semibold py-3 rounded-xl transition-all btn-glow flex items-center justify-center gap-2"
|
||||
>
|
||||
{betaLoading ? 'Enrolling...' : 'Get My Free Credit'}
|
||||
{!betaLoading && <Gift className="w-5 h-5" />}
|
||||
@@ -461,14 +559,14 @@ export default function LandingPage() {
|
||||
Email address
|
||||
</label>
|
||||
<div className="relative">
|
||||
<Mail className="absolute left-3 top-1/2 -translate-y-1/2 w-5 h-5 text-dark-500" />
|
||||
<Mail className="absolute left-3 top-1/2 -translate-y-1/2 w-5 h-5 text-muted-foreground" />
|
||||
<input
|
||||
type="email"
|
||||
id="email"
|
||||
value={email}
|
||||
onChange={(e) => setEmail(e.target.value)}
|
||||
placeholder="you@example.com"
|
||||
className="w-full pl-10 pr-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-primary-500 focus:border-transparent text-white placeholder-dark-500"
|
||||
className="w-full pl-10 pr-4 py-3 bg-secondary border border-border rounded-lg focus:outline-none focus:ring-2 focus:ring-primary focus:border-transparent text-foreground placeholder:text-muted-foreground"
|
||||
required
|
||||
/>
|
||||
</div>
|
||||
@@ -503,22 +601,9 @@ export default function LandingPage() {
|
||||
<h2 className="text-3xl font-bold text-center mb-12">Questions?</h2>
|
||||
<div className="space-y-4">
|
||||
{faqs.map((faq, i) => (
|
||||
<div key={i} className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
|
||||
<button
|
||||
onClick={() => setExpandedFaq(expandedFaq === i ? null : i)}
|
||||
aria-expanded={expandedFaq === i}
|
||||
className="w-full px-6 py-4 text-left flex items-center justify-between hover:bg-dark-700/50 transition-colors"
|
||||
>
|
||||
<span className="font-medium">{faq.question}</span>
|
||||
<ChevronDown
|
||||
className={`w-5 h-5 text-dark-400 transition-transform ${
|
||||
expandedFaq === i ? 'rotate-180' : ''
|
||||
}`}
|
||||
/>
|
||||
</button>
|
||||
{expandedFaq === i && (
|
||||
<div className="px-6 pb-4 text-dark-300 text-sm">{faq.answer}</div>
|
||||
)}
|
||||
<div key={i} className="bg-dark-800 border border-dark-700 rounded-xl p-6">
|
||||
<h3 className="font-medium mb-2">{faq.question}</h3>
|
||||
<p className="text-dark-400 text-sm">{faq.answer}</p>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
@@ -526,20 +611,19 @@ export default function LandingPage() {
|
||||
</section>
|
||||
|
||||
{/* Footer */}
|
||||
<footer className="py-12 px-4 border-t border-dark-700">
|
||||
<footer className="py-12 px-4 border-t border-primary-500/30">
|
||||
<div className="max-w-6xl mx-auto flex flex-col md:flex-row items-center justify-between gap-4">
|
||||
<div className="flex items-center gap-2">
|
||||
<MicVocal className="w-6 h-6 text-primary-500" />
|
||||
<span className="font-semibold">Nomad Karaoke</span>
|
||||
<img src="/nomad-karaoke-logo.svg" alt="Nomad Karaoke" className="h-8" />
|
||||
</div>
|
||||
<div className="text-sm text-dark-500">
|
||||
© {new Date().getFullYear()} Nomad Karaoke. All rights reserved.
|
||||
</div>
|
||||
<div className="flex gap-6 text-sm text-dark-400">
|
||||
<a href="mailto:support@nomadkaraoke.com" className="hover:text-white transition-colors">
|
||||
<a href="mailto:support@nomadkaraoke.com" className="hover:text-primary-400 transition-colors">
|
||||
Support
|
||||
</a>
|
||||
<a href="https://nomadkaraoke.com" className="hover:text-white transition-colors">
|
||||
<a href="https://nomadkaraoke.com" className="hover:text-primary-400 transition-colors">
|
||||
About
|
||||
</a>
|
||||
</div>
|
||||
|
||||
@@ -25,28 +25,28 @@ function PaymentSuccessContent() {
|
||||
const sessionId = searchParams.get("session_id")
|
||||
|
||||
return (
|
||||
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
|
||||
<CheckCircle className="w-16 h-16 text-green-400 mx-auto mb-4" />
|
||||
<div className="max-w-md w-full bg-card border border-border rounded-xl p-8 text-center">
|
||||
<CheckCircle className="w-16 h-16 text-success mx-auto mb-4" />
|
||||
|
||||
<h1 className="text-2xl font-bold text-white mb-2">
|
||||
<h1 className="text-2xl font-bold text-foreground mb-2">
|
||||
Payment Successful!
|
||||
</h1>
|
||||
|
||||
<p className="text-slate-400 mb-6">
|
||||
<p className="text-muted-foreground mb-6">
|
||||
Thank you for your purchase. Your credits have been added to your account.
|
||||
</p>
|
||||
|
||||
{isLoading ? (
|
||||
<div className="flex items-center justify-center gap-2 text-slate-400 mb-6">
|
||||
<div className="flex items-center justify-center gap-2 text-muted-foreground mb-6">
|
||||
<Loader2 className="w-4 h-4 animate-spin" />
|
||||
Loading your balance...
|
||||
</div>
|
||||
) : user ? (
|
||||
<div className="bg-slate-800 rounded-lg p-4 mb-6">
|
||||
<div className="flex items-center justify-center gap-2 text-amber-400">
|
||||
<div className="bg-secondary rounded-lg p-4 mb-6">
|
||||
<div className="flex items-center justify-center gap-2 text-warning">
|
||||
<Coins className="w-5 h-5" />
|
||||
<span className="text-2xl font-bold">{user.credits}</span>
|
||||
<span className="text-slate-400">credits available</span>
|
||||
<span className="text-muted-foreground">credits available</span>
|
||||
</div>
|
||||
</div>
|
||||
) : null}
|
||||
@@ -54,12 +54,12 @@ function PaymentSuccessContent() {
|
||||
<div className="space-y-3">
|
||||
<Button
|
||||
onClick={() => router.push("/app")}
|
||||
className="w-full bg-blue-600 hover:bg-blue-500"
|
||||
className="w-full bg-primary hover:bg-primary/90"
|
||||
>
|
||||
Start Creating Karaoke
|
||||
</Button>
|
||||
|
||||
<p className="text-xs text-slate-500">
|
||||
<p className="text-xs text-muted-foreground">
|
||||
A confirmation email has been sent to your email address.
|
||||
</p>
|
||||
</div>
|
||||
@@ -69,12 +69,12 @@ function PaymentSuccessContent() {
|
||||
|
||||
export default function PaymentSuccessPage() {
|
||||
return (
|
||||
<div className="min-h-screen bg-slate-950 flex items-center justify-center p-4">
|
||||
<div className="min-h-screen bg-background flex items-center justify-center p-4">
|
||||
<Suspense
|
||||
fallback={
|
||||
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
|
||||
<Loader2 className="w-16 h-16 text-blue-400 mx-auto mb-4 animate-spin" />
|
||||
<p className="text-slate-400">Loading...</p>
|
||||
<div className="max-w-md w-full bg-card border border-border rounded-xl p-8 text-center">
|
||||
<Loader2 className="w-16 h-16 text-primary mx-auto mb-4 animate-spin" />
|
||||
<p className="text-muted-foreground">Loading...</p>
|
||||
</div>
|
||||
}
|
||||
>
|
||||
|
||||
|
After Width: | Height: | Size: 920 KiB |
|
After Width: | Height: | Size: 270 KiB |
|
After Width: | Height: | Size: 220 KiB |
|
After Width: | Height: | Size: 166 KiB |
|
After Width: | Height: | Size: 733 KiB |
|
After Width: | Height: | Size: 222 KiB |
|
After Width: | Height: | Size: 171 KiB |
|
After Width: | Height: | Size: 137 KiB |
@@ -0,0 +1,35 @@
|
||||
"use client"
|
||||
|
||||
import { Moon, Sun } from "lucide-react"
|
||||
import { useTheme } from "@/lib/theme"
|
||||
import { Button } from "@/components/ui/button"
|
||||
|
||||
export function ThemeToggle() {
|
||||
const { theme, toggleTheme, mounted } = useTheme()
|
||||
|
||||
// Avoid hydration mismatch by not rendering until mounted
|
||||
if (!mounted) {
|
||||
return (
|
||||
<Button variant="ghost" size="icon" className="w-9 h-9">
|
||||
<span className="sr-only">Toggle theme</span>
|
||||
</Button>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={toggleTheme}
|
||||
className="w-9 h-9"
|
||||
title={`Switch to ${theme === 'dark' ? 'light' : 'dark'} mode`}
|
||||
>
|
||||
{theme === "dark" ? (
|
||||
<Sun className="h-5 w-5 text-muted-foreground hover:text-foreground transition-colors" />
|
||||
) : (
|
||||
<Moon className="h-5 w-5 text-muted-foreground hover:text-foreground transition-colors" />
|
||||
)}
|
||||
<span className="sr-only">Toggle theme</span>
|
||||
</Button>
|
||||
)
|
||||
}
|
||||
@@ -52,7 +52,7 @@ const CATEGORY_CONFIG: Record<string, CategoryConfig> = {
|
||||
'COMPILATIONS': { name: 'COMPILATIONS', maxDisplay: 2, color: 'text-teal-400', borderColor: 'border-teal-500/30' },
|
||||
'VINYL RIPS': { name: 'VINYL RIPS', maxDisplay: 2, color: 'text-orange-400', borderColor: 'border-orange-500/30' },
|
||||
'YOUTUBE/LOSSY': { name: 'YOUTUBE/LOSSY', maxDisplay: 3, color: 'text-red-400', borderColor: 'border-red-500/30' },
|
||||
'OTHER': { name: 'OTHER', maxDisplay: 3, color: 'text-slate-400', borderColor: 'border-slate-500/30' },
|
||||
'OTHER': { name: 'OTHER', maxDisplay: 3, color: 'text-muted-foreground', borderColor: 'border-border' },
|
||||
}
|
||||
|
||||
const CATEGORY_ORDER = [
|
||||
@@ -261,30 +261,30 @@ export function AudioSearchDialog({ jobId, open, onClose, onSelect }: AudioSearc
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={(open) => !open && onClose()}>
|
||||
<DialogContent className="max-w-7xl w-[95vw] max-h-[90vh] flex flex-col bg-slate-900 border-slate-700 !p-0 gap-0 overflow-hidden">
|
||||
<div className="px-4 py-2.5 border-b border-slate-700 shrink-0 bg-slate-900 flex items-center justify-between">
|
||||
<DialogTitle className="text-white text-sm font-semibold flex items-center gap-2">
|
||||
<DialogContent className="max-w-7xl w-[95vw] max-h-[90vh] flex flex-col bg-card border-border !p-0 gap-0 overflow-hidden">
|
||||
<div className="px-4 py-2.5 border-b border-border shrink-0 bg-card flex items-center justify-between">
|
||||
<DialogTitle className="text-foreground text-sm font-semibold flex items-center gap-2">
|
||||
Select Audio Source
|
||||
<span className="text-[10px] px-1.5 py-0.5 rounded bg-slate-700 text-slate-400 font-mono">v{APP_VERSION}</span>
|
||||
<span className="text-slate-400 font-normal text-xs">
|
||||
<span className="text-[10px] px-1.5 py-0.5 rounded bg-secondary text-muted-foreground font-mono">v{APP_VERSION}</span>
|
||||
<span className="text-muted-foreground font-normal text-xs">
|
||||
({results.length} results across {totalCategories} categories)
|
||||
</span>
|
||||
</DialogTitle>
|
||||
</div>
|
||||
|
||||
{error && (
|
||||
<div className="px-4 py-2 text-xs text-red-400 bg-red-500/10 border-b border-slate-800">
|
||||
<div className="px-4 py-2 text-xs text-red-400 bg-red-500/10 border-b border-border">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{isLoading ? (
|
||||
<div className="flex-1 flex items-center justify-center py-12">
|
||||
<Loader2 className="w-5 h-5 animate-spin text-slate-400" />
|
||||
<span className="ml-2 text-sm text-slate-400">Loading...</span>
|
||||
<Loader2 className="w-5 h-5 animate-spin text-muted-foreground" />
|
||||
<span className="ml-2 text-sm text-muted-foreground">Loading...</span>
|
||||
</div>
|
||||
) : results.length === 0 ? (
|
||||
<div className="flex-1 flex flex-col items-center justify-center py-12 text-slate-500">
|
||||
<div className="flex-1 flex flex-col items-center justify-center py-12 text-muted-foreground">
|
||||
<Music2 className="w-8 h-8 mb-2 opacity-50" />
|
||||
<p className="text-sm">No audio sources found</p>
|
||||
</div>
|
||||
@@ -299,17 +299,17 @@ export function AudioSearchDialog({ jobId, open, onClose, onSelect }: AudioSearc
|
||||
const hasMore = hiddenCount > 0
|
||||
|
||||
return (
|
||||
<div key={category} className="border border-slate-700 rounded-lg overflow-hidden bg-slate-800/20">
|
||||
<div key={category} className="border border-border rounded-lg overflow-hidden bg-secondary/20">
|
||||
{/* Category Header */}
|
||||
<div className={`px-3 py-2 bg-slate-800/70 flex items-center justify-between ${config.color}`}>
|
||||
<div className={`px-3 py-2 bg-secondary flex items-center justify-between ${config.color}`}>
|
||||
<div className="flex items-center gap-2 text-xs font-semibold">
|
||||
<span>{category}</span>
|
||||
<span className="text-slate-500 font-normal">({catResults.length})</span>
|
||||
<span className="text-muted-foreground font-normal">({catResults.length})</span>
|
||||
</div>
|
||||
{hasMore && (
|
||||
<button
|
||||
onClick={() => toggleCategory(category)}
|
||||
className="text-[10px] text-slate-400 hover:text-white flex items-center gap-1"
|
||||
className="text-[10px] text-muted-foreground hover:text-foreground flex items-center gap-1"
|
||||
>
|
||||
{isExpanded ? (
|
||||
<>Show less <ChevronUp className="w-3 h-3" /></>
|
||||
@@ -325,10 +325,10 @@ export function AudioSearchDialog({ jobId, open, onClose, onSelect }: AudioSearc
|
||||
{displayResults.map((result) => (
|
||||
<div
|
||||
key={result.index}
|
||||
className="px-3 py-1.5 hover:bg-slate-800/50 flex items-center gap-2 text-xs"
|
||||
className="px-3 py-1.5 hover:bg-secondary/50 flex items-center gap-2 text-xs"
|
||||
>
|
||||
{/* Index */}
|
||||
<span className="w-5 text-slate-500 font-mono shrink-0 text-[10px]">
|
||||
<span className="w-5 text-muted-foreground font-mono shrink-0 text-[10px]">
|
||||
{result.index + 1}.
|
||||
</span>
|
||||
|
||||
@@ -345,17 +345,17 @@ export function AudioSearchDialog({ jobId, open, onClose, onSelect }: AudioSearc
|
||||
result.provider === "YouTube" ? "bg-red-600/20 text-red-400" :
|
||||
result.provider === "RED" ? "bg-red-600/30 text-red-300" :
|
||||
result.provider === "OPS" ? "bg-blue-600/20 text-blue-400" :
|
||||
"bg-slate-600/20 text-slate-400"
|
||||
"bg-muted text-muted-foreground"
|
||||
}`}>
|
||||
{result.provider}
|
||||
</span>
|
||||
<span className="text-green-400 font-medium">{getDisplayName(result)}</span>
|
||||
<span className="text-slate-500">-</span>
|
||||
<span className="text-white">{result.title}</span>
|
||||
<span className={`text-[10px] ${result.is_lossless ? "text-green-400" : "text-slate-400"}`}>
|
||||
<span className="text-muted-foreground">-</span>
|
||||
<span className="text-foreground">{result.title}</span>
|
||||
<span className={`text-[10px] ${result.is_lossless ? "text-green-400" : "text-muted-foreground"}`}>
|
||||
({formatQuality(result)})
|
||||
</span>
|
||||
<span className="text-[10px] text-slate-400">{result.formatted_size || '-'}</span>
|
||||
<span className="text-[10px] text-muted-foreground">{result.formatted_size || '-'}</span>
|
||||
{result.seeders !== undefined && result.seeders !== null ? (
|
||||
<span className={`text-[10px] ${
|
||||
result.seeders >= 50 ? 'text-green-400' :
|
||||
@@ -368,7 +368,7 @@ export function AudioSearchDialog({ jobId, open, onClose, onSelect }: AudioSearc
|
||||
<span className={`text-[10px] ${
|
||||
result.view_count >= 1000000 ? 'text-green-400' :
|
||||
result.view_count >= 10000 ? 'text-yellow-400' :
|
||||
'text-slate-400'
|
||||
'text-muted-foreground'
|
||||
}`}>
|
||||
{formatCount(result.view_count)} views
|
||||
</span>
|
||||
@@ -377,7 +377,7 @@ export function AudioSearchDialog({ jobId, open, onClose, onSelect }: AudioSearc
|
||||
|
||||
{/* Line 2: metadata + filename */}
|
||||
{(formatMetadata(result) || result.target_file) && (
|
||||
<div className="flex items-center gap-1.5 text-[10px] text-slate-500">
|
||||
<div className="flex items-center gap-1.5 text-[10px] text-muted-foreground">
|
||||
{formatMetadata(result) && (
|
||||
<span className="truncate">{formatMetadata(result)}</span>
|
||||
)}
|
||||
@@ -393,7 +393,7 @@ export function AudioSearchDialog({ jobId, open, onClose, onSelect }: AudioSearc
|
||||
size="sm"
|
||||
onClick={() => handleSelect(result.index)}
|
||||
disabled={isSelecting !== null}
|
||||
className="w-14 h-6 text-[9px] bg-amber-600 hover:bg-amber-500 text-white px-2 shrink-0"
|
||||
className="w-14 h-6 text-[9px] bg-amber-600 hover:bg-amber-500 text-foreground px-2 shrink-0"
|
||||
>
|
||||
{isSelecting === result.index ? (
|
||||
<Loader2 className="w-3 h-3 animate-spin" />
|
||||
|
||||
@@ -79,15 +79,15 @@ export function AuthDialog({ open, onClose, onSuccess }: AuthDialogProps) {
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={(open) => !open && handleClose()}>
|
||||
<DialogContent className="sm:max-w-md bg-slate-900 border-slate-700">
|
||||
<DialogContent className="sm:max-w-md bg-card border-border">
|
||||
{step === "email" && (
|
||||
<>
|
||||
<DialogHeader>
|
||||
<DialogTitle className="text-white flex items-center gap-2">
|
||||
<Mail className="w-5 h-5 text-blue-400" />
|
||||
<DialogTitle className="text-foreground flex items-center gap-2">
|
||||
<Mail className="w-5 h-5 text-primary" />
|
||||
Sign In
|
||||
</DialogTitle>
|
||||
<DialogDescription className="text-slate-400">
|
||||
<DialogDescription className="text-muted-foreground">
|
||||
Enter your email to receive a sign-in link
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
@@ -102,19 +102,19 @@ export function AuthDialog({ open, onClose, onSuccess }: AuthDialogProps) {
|
||||
setLocalError("")
|
||||
clearError()
|
||||
}}
|
||||
className="bg-slate-800 border-slate-600 text-white"
|
||||
className="bg-secondary border-border text-foreground"
|
||||
autoFocus
|
||||
disabled={isLoading}
|
||||
/>
|
||||
{displayError && (
|
||||
<p className="text-xs text-red-400 mt-1">{displayError}</p>
|
||||
<p className="text-xs text-destructive mt-1">{displayError}</p>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex flex-col gap-2">
|
||||
<Button
|
||||
type="submit"
|
||||
disabled={!email.trim() || isLoading}
|
||||
className="w-full bg-blue-600 hover:bg-blue-500"
|
||||
className="w-full bg-primary hover:bg-primary/90"
|
||||
>
|
||||
{isLoading ? (
|
||||
<>
|
||||
@@ -129,13 +129,13 @@ export function AuthDialog({ open, onClose, onSuccess }: AuthDialogProps) {
|
||||
type="button"
|
||||
variant="ghost"
|
||||
onClick={() => setStep("token")}
|
||||
className="text-slate-400 hover:text-white"
|
||||
className="text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
<KeyRound className="w-4 h-4 mr-2" />
|
||||
Use Access Token Instead
|
||||
</Button>
|
||||
</div>
|
||||
<p className="text-xs text-slate-500 text-center">
|
||||
<p className="text-xs text-muted-foreground text-center">
|
||||
No account? Just enter your email to get started.
|
||||
</p>
|
||||
</form>
|
||||
@@ -145,26 +145,26 @@ export function AuthDialog({ open, onClose, onSuccess }: AuthDialogProps) {
|
||||
{step === "sent" && (
|
||||
<>
|
||||
<DialogHeader>
|
||||
<DialogTitle className="text-white flex items-center gap-2">
|
||||
<CheckCircle className="w-5 h-5 text-green-400" />
|
||||
<DialogTitle className="text-foreground flex items-center gap-2">
|
||||
<CheckCircle className="w-5 h-5 text-success" />
|
||||
Check Your Email
|
||||
</DialogTitle>
|
||||
</DialogHeader>
|
||||
<div className="space-y-4 py-4">
|
||||
<p className="text-slate-300">
|
||||
<p className="text-foreground">
|
||||
We sent a sign-in link to:
|
||||
</p>
|
||||
<p className="text-white font-medium bg-slate-800 px-4 py-2 rounded-md">
|
||||
<p className="text-foreground font-medium bg-secondary px-4 py-2 rounded-md">
|
||||
{email}
|
||||
</p>
|
||||
<p className="text-sm text-slate-400">
|
||||
<p className="text-sm text-muted-foreground">
|
||||
Click the link in the email to sign in. The link expires in 15 minutes.
|
||||
</p>
|
||||
<div className="pt-4 border-t border-slate-700 space-y-2">
|
||||
<div className="pt-4 border-t border-border space-y-2">
|
||||
<Button
|
||||
variant="outline"
|
||||
onClick={() => setStep("email")}
|
||||
className="w-full border-slate-600 text-slate-300 hover:text-white"
|
||||
className="w-full border-border text-foreground hover:text-foreground"
|
||||
>
|
||||
<ArrowLeft className="w-4 h-4 mr-2" />
|
||||
Try a Different Email
|
||||
@@ -172,7 +172,7 @@ export function AuthDialog({ open, onClose, onSuccess }: AuthDialogProps) {
|
||||
<Button
|
||||
variant="ghost"
|
||||
onClick={handleClose}
|
||||
className="w-full text-slate-400"
|
||||
className="w-full text-muted-foreground"
|
||||
>
|
||||
Close
|
||||
</Button>
|
||||
@@ -184,11 +184,11 @@ export function AuthDialog({ open, onClose, onSuccess }: AuthDialogProps) {
|
||||
{step === "token" && (
|
||||
<>
|
||||
<DialogHeader>
|
||||
<DialogTitle className="text-white flex items-center gap-2">
|
||||
<KeyRound className="w-5 h-5 text-amber-400" />
|
||||
<DialogTitle className="text-foreground flex items-center gap-2">
|
||||
<KeyRound className="w-5 h-5 text-warning" />
|
||||
Access Token
|
||||
</DialogTitle>
|
||||
<DialogDescription className="text-slate-400">
|
||||
<DialogDescription className="text-muted-foreground">
|
||||
Enter your access token to authenticate
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
@@ -203,19 +203,19 @@ export function AuthDialog({ open, onClose, onSuccess }: AuthDialogProps) {
|
||||
setLocalError("")
|
||||
clearError()
|
||||
}}
|
||||
className="bg-slate-800 border-slate-600 text-white"
|
||||
className="bg-secondary border-border text-foreground"
|
||||
autoFocus
|
||||
disabled={isLoading}
|
||||
/>
|
||||
{displayError && (
|
||||
<p className="text-xs text-red-400 mt-1">{displayError}</p>
|
||||
<p className="text-xs text-destructive mt-1">{displayError}</p>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex flex-col gap-2">
|
||||
<Button
|
||||
type="submit"
|
||||
disabled={!token.trim() || isLoading}
|
||||
className="w-full bg-amber-600 hover:bg-amber-500"
|
||||
className="w-full bg-warning hover:bg-warning/90 text-warning-foreground"
|
||||
>
|
||||
{isLoading ? (
|
||||
<>
|
||||
@@ -230,13 +230,13 @@ export function AuthDialog({ open, onClose, onSuccess }: AuthDialogProps) {
|
||||
type="button"
|
||||
variant="ghost"
|
||||
onClick={() => setStep("email")}
|
||||
className="text-slate-400 hover:text-white"
|
||||
className="text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
<ArrowLeft className="w-4 h-4 mr-2" />
|
||||
Back to Email Sign-In
|
||||
</Button>
|
||||
</div>
|
||||
<p className="text-xs text-slate-500 text-center">
|
||||
<p className="text-xs text-muted-foreground text-center">
|
||||
Contact an admin to get an access token
|
||||
</p>
|
||||
</form>
|
||||
|
||||
@@ -62,46 +62,46 @@ export function AuthStatus({ onAuthChange }: AuthStatusProps) {
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
className="text-slate-300 hover:text-white flex items-center gap-2 min-h-[40px] px-2 sm:px-3"
|
||||
className="text-muted-foreground hover:text-foreground flex items-center gap-2 min-h-[40px] px-2 sm:px-3"
|
||||
>
|
||||
<User className="w-4 h-4" />
|
||||
<span className="hidden sm:inline max-w-[150px] truncate">
|
||||
{user.display_name || user.email}
|
||||
</span>
|
||||
<span className="flex items-center gap-1 text-amber-400 font-medium">
|
||||
<span className="flex items-center gap-1 text-warning font-medium">
|
||||
<Coins className="w-3 h-3" />
|
||||
{user.credits} {user.credits === 1 ? 'credit' : 'credits'}
|
||||
</span>
|
||||
</Button>
|
||||
</DropdownMenuTrigger>
|
||||
<DropdownMenuContent align="end" className="w-56 bg-slate-900 border-slate-700">
|
||||
<DropdownMenuLabel className="text-slate-400 font-normal">
|
||||
<DropdownMenuContent align="end" className="w-56 bg-card border-border">
|
||||
<DropdownMenuLabel className="text-muted-foreground font-normal">
|
||||
<div className="flex flex-col space-y-1">
|
||||
<p className="text-sm font-medium text-white">
|
||||
<p className="text-sm font-medium text-foreground">
|
||||
{user.display_name || "Karaoke Fan"}
|
||||
</p>
|
||||
<p className="text-xs text-slate-500 truncate">{user.email}</p>
|
||||
<p className="text-xs text-muted-foreground truncate">{user.email}</p>
|
||||
</div>
|
||||
</DropdownMenuLabel>
|
||||
<DropdownMenuSeparator className="bg-slate-700" />
|
||||
<DropdownMenuSeparator className="bg-border" />
|
||||
<DropdownMenuItem
|
||||
className="text-slate-300 focus:text-white focus:bg-slate-800 cursor-default"
|
||||
className="text-muted-foreground focus:text-foreground focus:bg-secondary cursor-default"
|
||||
disabled
|
||||
>
|
||||
<Coins className="w-4 h-4 mr-2 text-amber-400" />
|
||||
<Coins className="w-4 h-4 mr-2 text-warning" />
|
||||
<span>{user.credits} credits available</span>
|
||||
</DropdownMenuItem>
|
||||
<DropdownMenuItem
|
||||
onClick={handleBuyCredits}
|
||||
className="text-slate-300 focus:text-white focus:bg-slate-800"
|
||||
className="text-muted-foreground focus:text-foreground focus:bg-secondary"
|
||||
>
|
||||
<CreditCard className="w-4 h-4 mr-2" />
|
||||
<span>Buy More Credits</span>
|
||||
</DropdownMenuItem>
|
||||
{(user.role === "admin" || user.email?.endsWith("@nomadkaraoke.com")) && (
|
||||
<>
|
||||
<DropdownMenuSeparator className="bg-slate-700" />
|
||||
<DropdownMenuItem asChild className="text-slate-300 focus:text-white focus:bg-slate-800">
|
||||
<DropdownMenuSeparator className="bg-border" />
|
||||
<DropdownMenuItem asChild className="text-muted-foreground focus:text-foreground focus:bg-secondary">
|
||||
<Link href="/admin">
|
||||
<Shield className="w-4 h-4 mr-2" />
|
||||
<span>Admin Dashboard</span>
|
||||
@@ -109,10 +109,10 @@ export function AuthStatus({ onAuthChange }: AuthStatusProps) {
|
||||
</DropdownMenuItem>
|
||||
</>
|
||||
)}
|
||||
<DropdownMenuSeparator className="bg-slate-700" />
|
||||
<DropdownMenuSeparator className="bg-border" />
|
||||
<DropdownMenuItem
|
||||
onClick={handleLogout}
|
||||
className="text-red-400 focus:text-red-300 focus:bg-slate-800"
|
||||
className="text-destructive focus:text-destructive focus:bg-secondary"
|
||||
>
|
||||
<LogOut className="w-4 h-4 mr-2" />
|
||||
<span>Sign Out</span>
|
||||
@@ -128,7 +128,7 @@ export function AuthStatus({ onAuthChange }: AuthStatusProps) {
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={() => setShowAuthDialog(true)}
|
||||
className="text-amber-400 hover:text-amber-300 min-h-[40px] px-2 sm:px-3"
|
||||
className="text-warning hover:text-warning/80 min-h-[40px] px-2 sm:px-3"
|
||||
>
|
||||
<KeyRound className="w-4 h-4 sm:mr-2" />
|
||||
<span className="hidden sm:inline">Login</span>
|
||||
|
||||
@@ -45,7 +45,7 @@ function ProgressBar({ job }: { job: Job }) {
|
||||
|
||||
// Map color class to background color for the bar
|
||||
const barColorMap: Record<string, string> = {
|
||||
"text-slate-400": "bg-slate-400",
|
||||
"text-muted-foreground": "bg-muted-foreground",
|
||||
"text-blue-400": "bg-blue-400",
|
||||
"text-purple-400": "bg-purple-400",
|
||||
"text-teal-400": "bg-teal-400",
|
||||
@@ -56,7 +56,7 @@ function ProgressBar({ job }: { job: Job }) {
|
||||
"text-violet-400": "bg-violet-400",
|
||||
"text-green-400": "bg-green-400",
|
||||
}
|
||||
const barColor = barColorMap[color] || "bg-blue-400"
|
||||
const barColor = barColorMap[color] || "bg-primary-500"
|
||||
|
||||
return (
|
||||
<div className="w-full h-1 rounded-full mt-2 overflow-hidden" style={{ backgroundColor: 'var(--secondary)' }}>
|
||||
@@ -133,7 +133,7 @@ export function JobCard({ job, onRefresh }: JobCardProps) {
|
||||
href={getReviewUrl()}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-blue-600 hover:bg-blue-500 text-white"
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-primary-500 hover:bg-primary-600 text-white"
|
||||
>
|
||||
<ExternalLink className="w-3 h-3" />
|
||||
Review Lyrics
|
||||
@@ -146,7 +146,7 @@ export function JobCard({ job, onRefresh }: JobCardProps) {
|
||||
<Button
|
||||
size="sm"
|
||||
onClick={() => setShowAudioSearch(true)}
|
||||
className="text-xs h-7 px-3 bg-blue-600 hover:bg-blue-500"
|
||||
className="text-xs h-7 px-3 bg-primary-500 hover:bg-primary-600"
|
||||
>
|
||||
Select Audio
|
||||
</Button>
|
||||
@@ -159,7 +159,7 @@ export function JobCard({ job, onRefresh }: JobCardProps) {
|
||||
href={getInstrumentalUrl()}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-blue-600 hover:bg-blue-500 text-white"
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-primary-500 hover:bg-primary-600 text-white"
|
||||
>
|
||||
<ExternalLink className="w-3 h-3" />
|
||||
Select Instrumental
|
||||
|
||||
@@ -34,6 +34,9 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
// Audio search form
|
||||
const [searchArtist, setSearchArtist] = useState("")
|
||||
const [searchTitle, setSearchTitle] = useState("")
|
||||
// Display As overrides (optional - if empty, search values used for display)
|
||||
const [displayArtist, setDisplayArtist] = useState("")
|
||||
const [displayTitle, setDisplayTitle] = useState("")
|
||||
|
||||
// Theme selection (shared across all tabs)
|
||||
const [selectedTheme, setSelectedTheme] = useState<string | undefined>()
|
||||
@@ -128,9 +131,14 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
theme_id: selectedTheme,
|
||||
color_overrides: cleanColorOverrides(colorOverrides),
|
||||
non_interactive: nonInteractive,
|
||||
// Display overrides (empty string means use search values)
|
||||
display_artist: displayArtist.trim() || undefined,
|
||||
display_title: displayTitle.trim() || undefined,
|
||||
})
|
||||
setSearchArtist("")
|
||||
setSearchTitle("")
|
||||
setDisplayArtist("")
|
||||
setDisplayTitle("")
|
||||
onJobCreated()
|
||||
} catch (err) {
|
||||
if (err instanceof ApiError) {
|
||||
@@ -265,7 +273,7 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
checked={nonInteractive}
|
||||
onChange={(e) => setNonInteractive(e.target.checked)}
|
||||
disabled={isSubmitting}
|
||||
className="mt-1 w-4 h-4 rounded border-gray-600 bg-gray-700 text-amber-500 focus:ring-amber-500 focus:ring-offset-gray-800"
|
||||
className="mt-1 w-4 h-4 rounded border-border bg-secondary text-amber-500 focus:ring-amber-500 focus:ring-offset-background"
|
||||
/>
|
||||
<div>
|
||||
<Label htmlFor="non-interactive-upload" className="cursor-pointer" style={{ color: 'var(--text)' }}>
|
||||
@@ -370,7 +378,7 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
checked={nonInteractive}
|
||||
onChange={(e) => setNonInteractive(e.target.checked)}
|
||||
disabled={isSubmitting}
|
||||
className="mt-1 w-4 h-4 rounded border-gray-600 bg-gray-700 text-amber-500 focus:ring-amber-500 focus:ring-offset-gray-800"
|
||||
className="mt-1 w-4 h-4 rounded border-border bg-secondary text-amber-500 focus:ring-amber-500 focus:ring-offset-background"
|
||||
/>
|
||||
<div>
|
||||
<Label htmlFor="non-interactive-url" className="cursor-pointer" style={{ color: 'var(--text)' }}>
|
||||
@@ -407,33 +415,69 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
|
||||
<TabsContent value="search" className="mt-4">
|
||||
<form onSubmit={handleSearchSubmit} className="space-y-4">
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 sm:gap-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="search-artist" style={{ color: 'var(--text)' }}>Artist</Label>
|
||||
<Input
|
||||
id="search-artist"
|
||||
placeholder="Artist name"
|
||||
value={searchArtist}
|
||||
onChange={(e) => setSearchArtist(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
{/* Search For section */}
|
||||
<div className="space-y-2">
|
||||
<Label style={{ color: 'var(--text)' }}>Search For</Label>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 sm:gap-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="search-artist" className="text-xs" style={{ color: 'var(--text-muted)' }}>Artist</Label>
|
||||
<Input
|
||||
id="search-artist"
|
||||
placeholder="Artist name on trackers"
|
||||
value={searchArtist}
|
||||
onChange={(e) => setSearchArtist(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="search-title" className="text-xs" style={{ color: 'var(--text-muted)' }}>Title</Label>
|
||||
<Input
|
||||
id="search-title"
|
||||
placeholder="Song title on trackers"
|
||||
value={searchTitle}
|
||||
onChange={(e) => setSearchTitle(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="search-title" style={{ color: 'var(--text)' }}>Title</Label>
|
||||
<Input
|
||||
id="search-title"
|
||||
placeholder="Song title"
|
||||
value={searchTitle}
|
||||
onChange={(e) => setSearchTitle(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
|
||||
{/* Display As section (optional) */}
|
||||
<div className="space-y-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<Label style={{ color: 'var(--text)' }}>Display As</Label>
|
||||
<span className="text-xs px-1.5 py-0.5 rounded" style={{ color: 'var(--text-muted)', backgroundColor: 'var(--secondary)' }}>optional</span>
|
||||
</div>
|
||||
<p className="text-xs" style={{ color: 'var(--text-muted)' }}>
|
||||
Override how artist/title appear on the title card and filename. Leave empty to use search values.
|
||||
</p>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 sm:gap-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="display-artist" className="text-xs" style={{ color: 'var(--text-muted)' }}>Display Artist</Label>
|
||||
<Input
|
||||
id="display-artist"
|
||||
placeholder="e.g., Footloose (Broadway Cast)"
|
||||
value={displayArtist}
|
||||
onChange={(e) => setDisplayArtist(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="display-title" className="text-xs" style={{ color: 'var(--text-muted)' }}>Display Title</Label>
|
||||
<Input
|
||||
id="display-title"
|
||||
placeholder="e.g., I Can't Stand Still"
|
||||
value={displayTitle}
|
||||
onChange={(e) => setDisplayTitle(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-xs" style={{ color: 'var(--text-muted)' }}>
|
||||
<span className="text-amber-500 font-medium">Note:</span> Format these exactly as you want them on the title card and video filename.
|
||||
</p>
|
||||
|
||||
{/* Theme Selection */}
|
||||
<div className="space-y-4 pt-4 border-t" style={{ borderColor: 'var(--card-border)' }}>
|
||||
@@ -458,7 +502,7 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
checked={nonInteractive}
|
||||
onChange={(e) => setNonInteractive(e.target.checked)}
|
||||
disabled={isSubmitting}
|
||||
className="mt-1 w-4 h-4 rounded border-gray-600 bg-gray-700 text-amber-500 focus:ring-amber-500 focus:ring-offset-gray-800"
|
||||
className="mt-1 w-4 h-4 rounded border-border bg-secondary text-amber-500 focus:ring-amber-500 focus:ring-offset-background"
|
||||
/>
|
||||
<div>
|
||||
<Label htmlFor="non-interactive-search" className="cursor-pointer" style={{ color: 'var(--text)' }}>
|
||||
|
||||
@@ -1,9 +1,20 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState, useRef, useCallback } from "react"
|
||||
import { api } from "@/lib/api"
|
||||
import { api, adminApi } from "@/lib/api"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Download, Loader2, ExternalLink, FolderOpen, Copy } from "lucide-react"
|
||||
import { Download, Loader2, ExternalLink, FolderOpen, Copy, Mail, MessageSquare } from "lucide-react"
|
||||
import { useAuth } from "@/lib/auth"
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from "@/components/ui/dialog"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { Label } from "@/components/ui/label"
|
||||
|
||||
interface OutputLinksProps {
|
||||
jobId: string
|
||||
@@ -15,18 +26,35 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
const [dropboxUrl, setDropboxUrl] = useState<string | null>(null)
|
||||
const [isLoading, setIsLoading] = useState(false)
|
||||
const [copiedUrl, setCopiedUrl] = useState<string | null>(null)
|
||||
const [copiedMessage, setCopiedMessage] = useState(false)
|
||||
const [isLoadingMessage, setIsLoadingMessage] = useState(false)
|
||||
const [showEmailDialog, setShowEmailDialog] = useState(false)
|
||||
const [emailInput, setEmailInput] = useState("")
|
||||
const [isSendingEmail, setIsSendingEmail] = useState(false)
|
||||
const [emailSent, setEmailSent] = useState(false)
|
||||
const copyTimeoutRef = useRef<NodeJS.Timeout | null>(null)
|
||||
const messageTimeoutRef = useRef<NodeJS.Timeout | null>(null)
|
||||
const emailTimeoutRef = useRef<NodeJS.Timeout | null>(null)
|
||||
|
||||
const { user } = useAuth()
|
||||
const isAdmin = user?.role === 'admin'
|
||||
|
||||
useEffect(() => {
|
||||
loadOutputLinks()
|
||||
}, [jobId])
|
||||
|
||||
// Cleanup timeout on unmount
|
||||
// Cleanup timeouts on unmount
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
if (copyTimeoutRef.current) {
|
||||
clearTimeout(copyTimeoutRef.current)
|
||||
}
|
||||
if (messageTimeoutRef.current) {
|
||||
clearTimeout(messageTimeoutRef.current)
|
||||
}
|
||||
if (emailTimeoutRef.current) {
|
||||
clearTimeout(emailTimeoutRef.current)
|
||||
}
|
||||
}
|
||||
}, [])
|
||||
|
||||
@@ -76,6 +104,60 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
}
|
||||
}, [])
|
||||
|
||||
const copyCompletionMessage = useCallback(async () => {
|
||||
if (!navigator?.clipboard?.writeText) {
|
||||
console.error("Clipboard API not available")
|
||||
return
|
||||
}
|
||||
|
||||
setIsLoadingMessage(true)
|
||||
try {
|
||||
const response = await adminApi.getCompletionMessage(jobId)
|
||||
await navigator.clipboard.writeText(response.message)
|
||||
setCopiedMessage(true)
|
||||
|
||||
// Clear any existing timeout
|
||||
if (messageTimeoutRef.current) {
|
||||
clearTimeout(messageTimeoutRef.current)
|
||||
}
|
||||
|
||||
// Set new timeout
|
||||
messageTimeoutRef.current = setTimeout(() => {
|
||||
setCopiedMessage(false)
|
||||
}, 2000)
|
||||
} catch (err) {
|
||||
console.error("Failed to copy completion message:", err)
|
||||
} finally {
|
||||
setIsLoadingMessage(false)
|
||||
}
|
||||
}, [jobId])
|
||||
|
||||
const handleSendEmail = useCallback(async () => {
|
||||
if (!emailInput.trim()) return
|
||||
|
||||
setIsSendingEmail(true)
|
||||
try {
|
||||
await adminApi.sendCompletionEmail(jobId, emailInput.trim(), true)
|
||||
setEmailSent(true)
|
||||
|
||||
// Clear any existing timeout
|
||||
if (emailTimeoutRef.current) {
|
||||
clearTimeout(emailTimeoutRef.current)
|
||||
}
|
||||
// Set new timeout to close dialog after success message
|
||||
emailTimeoutRef.current = setTimeout(() => {
|
||||
setShowEmailDialog(false)
|
||||
setEmailSent(false)
|
||||
setEmailInput("")
|
||||
}, 1500)
|
||||
} catch (err) {
|
||||
console.error("Failed to send email:", err)
|
||||
alert("Failed to send email. Please check the email address and try again.")
|
||||
} finally {
|
||||
setIsSendingEmail(false)
|
||||
}
|
||||
}, [jobId, emailInput])
|
||||
|
||||
if (isLoading) {
|
||||
return (
|
||||
<div className="flex items-center gap-2 text-xs" style={{ color: 'var(--text-muted)' }}>
|
||||
@@ -124,7 +206,7 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
href={dropboxUrl}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded-l bg-blue-600 hover:bg-blue-500 text-white"
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded-l bg-primary-500 hover:bg-primary-600 text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<FolderOpen className="w-3 h-3" />
|
||||
@@ -133,7 +215,7 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => { e.stopPropagation(); copyToClipboard(dropboxUrl) }}
|
||||
className="inline-flex items-center text-xs px-2 py-1.5 rounded-r bg-blue-700 hover:bg-blue-600 text-white"
|
||||
className="inline-flex items-center text-xs px-2 py-1.5 rounded-r bg-[#cc5fa3] hover:bg-primary-600 text-white"
|
||||
title="Copy Dropbox URL"
|
||||
aria-label={copiedUrl === dropboxUrl ? "Dropbox URL copied" : "Copy Dropbox URL"}
|
||||
>
|
||||
@@ -175,7 +257,7 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
{downloadUrls?.packages?.cdg_zip && (
|
||||
<a
|
||||
href={api.getDownloadUrl(jobId, "packages", "cdg_zip")}
|
||||
className="inline-flex items-center gap-1 text-xs px-2 py-1.5 rounded bg-gray-700 hover:bg-gray-600 text-white"
|
||||
className="inline-flex items-center gap-1 text-xs px-2 py-1.5 rounded bg-secondary hover:bg-muted text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<Download className="w-3 h-3" />
|
||||
@@ -185,7 +267,7 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
{downloadUrls?.packages?.txt_zip && (
|
||||
<a
|
||||
href={api.getDownloadUrl(jobId, "packages", "txt_zip")}
|
||||
className="inline-flex items-center gap-1 text-xs px-2 py-1.5 rounded bg-gray-700 hover:bg-gray-600 text-white"
|
||||
className="inline-flex items-center gap-1 text-xs px-2 py-1.5 rounded bg-secondary hover:bg-muted text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<Download className="w-3 h-3" />
|
||||
@@ -193,12 +275,99 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
</a>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Admin-only buttons */}
|
||||
{isAdmin && (
|
||||
<div className="flex flex-wrap gap-2 pt-1 border-t border-border/50 mt-2">
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => { e.stopPropagation(); copyCompletionMessage() }}
|
||||
disabled={isLoadingMessage}
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-amber-600 hover:bg-amber-500 text-white disabled:opacity-50"
|
||||
title="Copy completion message to clipboard"
|
||||
>
|
||||
{isLoadingMessage ? (
|
||||
<Loader2 className="w-3 h-3 animate-spin" />
|
||||
) : (
|
||||
<MessageSquare className="w-3 h-3" />
|
||||
)}
|
||||
{copiedMessage ? "Copied!" : "Copy Message"}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => { e.stopPropagation(); setShowEmailDialog(true) }}
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-teal-600 hover:bg-teal-500 text-white"
|
||||
title="Send completion email"
|
||||
>
|
||||
<Mail className="w-3 h-3" />
|
||||
Send Email
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{!hasOutputs && (
|
||||
<p className="text-xs" style={{ color: 'var(--text-muted)' }}>No outputs available yet</p>
|
||||
)}
|
||||
|
||||
{/* Send Email Dialog */}
|
||||
<Dialog open={showEmailDialog} onOpenChange={setShowEmailDialog}>
|
||||
<DialogContent className="sm:max-w-[425px]" onClick={(e) => e.stopPropagation()}>
|
||||
<DialogHeader>
|
||||
<DialogTitle>Send Completion Email</DialogTitle>
|
||||
<DialogDescription>
|
||||
Send the job completion message to an email address. The email will be CC'd to gen@nomadkaraoke.com.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<div className="grid gap-4 py-4">
|
||||
<div className="grid grid-cols-4 items-center gap-4">
|
||||
<Label htmlFor="email" className="text-right">
|
||||
Email
|
||||
</Label>
|
||||
<Input
|
||||
id="email"
|
||||
type="email"
|
||||
value={emailInput}
|
||||
onChange={(e) => setEmailInput(e.target.value)}
|
||||
placeholder="customer@example.com"
|
||||
className="col-span-3"
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter') {
|
||||
handleSendEmail()
|
||||
}
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<DialogFooter>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
onClick={() => setShowEmailDialog(false)}
|
||||
disabled={isSendingEmail}
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
onClick={handleSendEmail}
|
||||
disabled={isSendingEmail || !emailInput.trim() || emailSent}
|
||||
>
|
||||
{isSendingEmail ? (
|
||||
<>
|
||||
<Loader2 className="w-4 h-4 mr-2 animate-spin" />
|
||||
Sending...
|
||||
</>
|
||||
) : emailSent ? (
|
||||
"Sent!"
|
||||
) : (
|
||||
"Send Email"
|
||||
)}
|
||||
</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -1,10 +1,11 @@
|
||||
"use client";
|
||||
|
||||
import { useEffect, useState } from "react";
|
||||
import { api } from "@/lib/api";
|
||||
import { api, EncodingWorkerHealth } from "@/lib/api";
|
||||
|
||||
export function VersionFooter() {
|
||||
const [backendVersion, setBackendVersion] = useState<string | null>(null);
|
||||
const [encodingWorker, setEncodingWorker] = useState<EncodingWorkerHealth | null>(null);
|
||||
// Frontend version is injected at build time from pyproject.toml
|
||||
const frontendVersion = process.env.NEXT_PUBLIC_APP_VERSION || "dev";
|
||||
|
||||
@@ -19,9 +20,56 @@ export function VersionFooter() {
|
||||
}
|
||||
};
|
||||
|
||||
const fetchEncodingWorkerHealth = async () => {
|
||||
try {
|
||||
const health = await api.getEncodingWorkerHealth();
|
||||
setEncodingWorker(health);
|
||||
} catch (error) {
|
||||
console.error("Failed to fetch encoding worker health:", error);
|
||||
setEncodingWorker(null);
|
||||
}
|
||||
};
|
||||
|
||||
fetchBackendVersion();
|
||||
fetchEncodingWorkerHealth();
|
||||
}, []);
|
||||
|
||||
// Render encoding worker status indicator
|
||||
const renderEncodingWorkerStatus = () => {
|
||||
if (!encodingWorker) {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (encodingWorker.status === "not_configured") {
|
||||
return null; // Don't show if not configured
|
||||
}
|
||||
|
||||
const isHealthy = encodingWorker.available && encodingWorker.status === "ok";
|
||||
const statusColor = isHealthy ? "text-green-500" : "text-red-500";
|
||||
const statusDot = isHealthy ? "●" : "○";
|
||||
|
||||
// Build status text
|
||||
let statusText = isHealthy ? "Encoder" : "Encoder offline";
|
||||
if (isHealthy && encodingWorker.version) {
|
||||
statusText = `Encoder: v${encodingWorker.version}`;
|
||||
}
|
||||
|
||||
// Add active jobs indicator if there are any
|
||||
if (isHealthy && encodingWorker.active_jobs && encodingWorker.active_jobs > 0) {
|
||||
statusText += ` (${encodingWorker.active_jobs} active)`;
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
{" | "}
|
||||
<span className={statusColor} title={isHealthy ? "Encoding worker healthy" : encodingWorker.error || "Encoding worker offline"}>
|
||||
{statusDot}
|
||||
</span>{" "}
|
||||
{statusText}
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
return (
|
||||
<footer className="border-t mt-12 py-6" style={{ borderColor: 'var(--card-border)' }}>
|
||||
<div className="px-4 text-center text-sm" style={{ color: 'var(--text-muted)' }}>
|
||||
@@ -39,6 +87,7 @@ export function VersionFooter() {
|
||||
Backend: v{backendVersion}
|
||||
</>
|
||||
)}
|
||||
{renderEncodingWorkerStatus()}
|
||||
</p>
|
||||
</div>
|
||||
</footer>
|
||||
|
||||
@@ -43,8 +43,10 @@ export interface EmailHelper {
|
||||
isAvailable: boolean;
|
||||
createInbox: () => Promise<InboxDto>;
|
||||
waitForEmail: (inboxId: string, timeout?: number) => Promise<Email>;
|
||||
waitForCompletionEmail: (inboxId: string, artist?: string, title?: string, timeout?: number) => Promise<Email>;
|
||||
extractMagicLink: (email: Email) => string | null;
|
||||
extractVerificationCode: (email: Email) => string | null;
|
||||
isCompletionEmail: (email: Email) => boolean;
|
||||
deleteInbox: (inboxId: string) => Promise<void>;
|
||||
}
|
||||
|
||||
@@ -78,6 +80,37 @@ function generateUniqueTag(): string {
|
||||
return `test-${timestamp}-${random}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if an email is a job completion email.
|
||||
* Extracted to standalone function for TypeScript compatibility.
|
||||
*/
|
||||
function checkIsCompletionEmail(email: Email): boolean {
|
||||
const subject = (email.subject || '').toLowerCase();
|
||||
const body = (email.body || '').toLowerCase();
|
||||
|
||||
// Check subject for completion indicators
|
||||
const subjectIndicators = [
|
||||
'karaoke video is ready',
|
||||
'your video is ready',
|
||||
'video ready',
|
||||
'complete',
|
||||
];
|
||||
|
||||
// Check body for completion indicators
|
||||
const bodyIndicators = [
|
||||
'your karaoke video is ready',
|
||||
'video has been created',
|
||||
'youtube',
|
||||
'dropbox',
|
||||
'download your video',
|
||||
];
|
||||
|
||||
const hasSubjectMatch = subjectIndicators.some(indicator => subject.includes(indicator));
|
||||
const hasBodyMatch = bodyIndicators.some(indicator => body.includes(indicator));
|
||||
|
||||
return hasSubjectMatch || hasBodyMatch;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates an email testing helper.
|
||||
* Returns a helper with isAvailable=false if testmail.app env vars are not set.
|
||||
@@ -234,6 +267,110 @@ export async function createEmailHelper(): Promise<EmailHelper> {
|
||||
return null;
|
||||
},
|
||||
|
||||
/**
|
||||
* Check if an email is a job completion email
|
||||
*/
|
||||
isCompletionEmail(email: Email): boolean {
|
||||
return checkIsCompletionEmail(email);
|
||||
},
|
||||
|
||||
/**
|
||||
* Wait for a job completion email to arrive
|
||||
* @param inboxId - The tag to check (returned from createInbox as id)
|
||||
* @param artist - Optional artist name to verify in email
|
||||
* @param title - Optional song title to verify in email
|
||||
* @param timeout - Timeout in ms (default: 120000 = 2 minutes)
|
||||
*/
|
||||
async waitForCompletionEmail(
|
||||
inboxId: string,
|
||||
artist?: string,
|
||||
title?: string,
|
||||
timeout = 120000
|
||||
): Promise<Email> {
|
||||
console.log(`Waiting for completion email with tag ${inboxId}...`);
|
||||
|
||||
const pollInterval = Math.min(timeout, 30000);
|
||||
const startTime = Date.now();
|
||||
|
||||
// Track emails we've already seen (by timestamp)
|
||||
const seenTimestamps = new Set<number>();
|
||||
|
||||
while (Date.now() - startTime < timeout) {
|
||||
const remainingTime = timeout - (Date.now() - startTime);
|
||||
const waitTime = Math.min(pollInterval, remainingTime);
|
||||
const waitSeconds = Math.ceil(waitTime / 1000);
|
||||
|
||||
const url = new URL(TESTMAIL_API_URL);
|
||||
url.searchParams.set('apikey', TESTMAIL_API_KEY!);
|
||||
url.searchParams.set('namespace', TESTMAIL_NAMESPACE!);
|
||||
url.searchParams.set('tag', inboxId);
|
||||
url.searchParams.set('livequery', 'true');
|
||||
url.searchParams.set('wait', waitSeconds.toString());
|
||||
|
||||
try {
|
||||
const response = await fetch(url.toString());
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`testmail.app API error: ${response.status} ${response.statusText}`);
|
||||
}
|
||||
|
||||
const data: TestmailResponse = await response.json();
|
||||
|
||||
if (data.result !== 'success') {
|
||||
throw new Error(`testmail.app API error: ${data.message || 'Unknown error'}`);
|
||||
}
|
||||
|
||||
if (data.emails && data.emails.length > 0) {
|
||||
// Look for completion emails we haven't seen yet
|
||||
for (const testmailEmail of data.emails.sort((a, b) => b.timestamp - a.timestamp)) {
|
||||
// Skip if we've already processed this email
|
||||
if (seenTimestamps.has(testmailEmail.timestamp)) {
|
||||
continue;
|
||||
}
|
||||
seenTimestamps.add(testmailEmail.timestamp);
|
||||
|
||||
const email: Email = {
|
||||
subject: testmailEmail.subject,
|
||||
body: testmailEmail.html || testmailEmail.text,
|
||||
from: testmailEmail.from,
|
||||
timestamp: testmailEmail.timestamp,
|
||||
};
|
||||
|
||||
// Check if this is a completion email
|
||||
if (checkIsCompletionEmail(email)) {
|
||||
console.log(`Found completion email: ${email.subject}`);
|
||||
|
||||
// Optionally verify artist/title in subject or body
|
||||
if (artist && title) {
|
||||
const content = `${email.subject} ${email.body}`.toLowerCase();
|
||||
const hasArtist = content.includes(artist.toLowerCase());
|
||||
const hasTitle = content.includes(title.toLowerCase());
|
||||
|
||||
if (hasArtist && hasTitle) {
|
||||
console.log(` ✓ Verified artist "${artist}" and title "${title}" in email`);
|
||||
} else {
|
||||
console.log(` ⚠ Artist/title verification: artist=${hasArtist}, title=${hasTitle}`);
|
||||
}
|
||||
}
|
||||
|
||||
return email;
|
||||
} else {
|
||||
console.log(` Skipping non-completion email: ${testmailEmail.subject}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
if (Date.now() - startTime < timeout) {
|
||||
console.log(`Polling for completion email... (${Math.round((Date.now() - startTime) / 1000)}s elapsed)`);
|
||||
continue;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error(`Timed out waiting for completion email after ${timeout}ms`);
|
||||
},
|
||||
|
||||
/**
|
||||
* Delete an inbox after testing.
|
||||
* In testmail.app, emails auto-expire - this is a no-op.
|
||||
@@ -257,8 +394,10 @@ function createDisabledHelper(): EmailHelper {
|
||||
isAvailable: false,
|
||||
createInbox: notAvailable,
|
||||
waitForEmail: notAvailable,
|
||||
waitForCompletionEmail: notAvailable,
|
||||
extractMagicLink: () => null,
|
||||
extractVerificationCode: () => null,
|
||||
isCompletionEmail: () => false,
|
||||
deleteInbox: notAvailable,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('lyrics review UI loads test data correctly', async ({ page }) => {
|
||||
// Navigate to the UI with the API URL parameter
|
||||
const url = 'http://localhost:8767?baseApiUrl=http%3A%2F%2Flocalhost%3A8767%2Fapi';
|
||||
|
||||
await page.goto(url);
|
||||
await page.waitForLoadState('networkidle');
|
||||
|
||||
// Wait for content to render
|
||||
await page.waitForTimeout(2000);
|
||||
|
||||
// Check NOT in read-only mode
|
||||
const pageContent = await page.content();
|
||||
expect(pageContent.toLowerCase()).not.toContain('read-only mode');
|
||||
|
||||
// Check for lyrics content - look for actual words from test data
|
||||
await expect(page.locator('body')).toContainText('Hello', { timeout: 5000 });
|
||||
|
||||
// Take screenshot for verification
|
||||
await page.screenshot({ path: '/tmp/lyrics-review-test.png', fullPage: true });
|
||||
});
|
||||
@@ -154,6 +154,23 @@ test.describe('E2E Happy Path - Real User with Full UI Interactions', () => {
|
||||
// Only create email helper if we're doing the full signup flow
|
||||
const emailHelper = usePreConfiguredToken ? null : await createEmailHelper();
|
||||
|
||||
// Intercept API requests to disable YouTube upload for E2E tests
|
||||
// This prevents test jobs from uploading to YouTube when server default is true
|
||||
await page.route('**/api/audio-search/search', async (route) => {
|
||||
const request = route.request();
|
||||
if (request.method() === 'POST') {
|
||||
const postData = request.postDataJSON();
|
||||
// Explicitly disable YouTube upload for E2E test jobs
|
||||
postData.enable_youtube_upload = false;
|
||||
console.log(' [Request Intercept] Disabled YouTube upload for audio search');
|
||||
await route.continue({
|
||||
postData: JSON.stringify(postData),
|
||||
});
|
||||
} else {
|
||||
await route.continue();
|
||||
}
|
||||
});
|
||||
|
||||
try {
|
||||
// =========================================================================
|
||||
// STEP 1: Landing Page
|
||||
@@ -815,6 +832,62 @@ test.describe('E2E Happy Path - Real User with Full UI Interactions', () => {
|
||||
await page.screenshot({ path: 'test-results/09-job-complete.png' });
|
||||
console.log('STEP 9 COMPLETE: Job finished');
|
||||
|
||||
// =========================================================================
|
||||
// STEP 9.5: Verify Completion Email (if email testing available)
|
||||
// =========================================================================
|
||||
if (emailHelper && inboxId) {
|
||||
console.log('\n========================================');
|
||||
console.log('STEP 9.5: Verify Completion Email');
|
||||
console.log('========================================');
|
||||
|
||||
try {
|
||||
console.log(' Waiting for completion email to arrive...');
|
||||
|
||||
// Wait up to 3 minutes for the completion email
|
||||
// Email is sent asynchronously after job completion, may take a moment
|
||||
const completionEmail = await emailHelper.waitForCompletionEmail(
|
||||
inboxId,
|
||||
TEST_SONG.artist,
|
||||
TEST_SONG.title,
|
||||
180_000 // 3 minutes
|
||||
);
|
||||
|
||||
console.log(` ✓ Completion email received: "${completionEmail.subject}"`);
|
||||
|
||||
// Verify email content
|
||||
const emailBody = completionEmail.body || '';
|
||||
const emailSubject = completionEmail.subject || '';
|
||||
|
||||
// Check for expected content indicators
|
||||
const hasVideoReadyText = emailBody.toLowerCase().includes('ready') ||
|
||||
emailSubject.toLowerCase().includes('ready');
|
||||
const hasSongInfo = emailBody.toLowerCase().includes(TEST_SONG.artist.toLowerCase()) ||
|
||||
emailSubject.toLowerCase().includes(TEST_SONG.artist.toLowerCase());
|
||||
|
||||
console.log(` Email verification:`);
|
||||
console.log(` - "ready" indicator: ${hasVideoReadyText ? '✓' : '✗'}`);
|
||||
console.log(` - Artist name: ${hasSongInfo ? '✓' : '✗'}`);
|
||||
|
||||
// Log email details for debugging
|
||||
console.log(` Email from: ${completionEmail.from}`);
|
||||
console.log(` Subject: ${emailSubject}`);
|
||||
console.log(` Body preview: ${emailBody.substring(0, 200)}...`);
|
||||
|
||||
await page.screenshot({ path: 'test-results/09.5-completion-email-received.png' });
|
||||
console.log('STEP 9.5 COMPLETE: Completion email verified');
|
||||
} catch (e) {
|
||||
// Don't fail the test if email verification times out - it's an enhancement
|
||||
console.log(` ⚠ WARNING: Completion email verification failed: ${e}`);
|
||||
console.log(' This may indicate email sending is disabled or delayed.');
|
||||
console.log(' Continuing with remaining steps...');
|
||||
}
|
||||
} else {
|
||||
console.log('\n========================================');
|
||||
console.log('STEP 9.5: Skip Completion Email Verification');
|
||||
console.log('========================================');
|
||||
console.log(' Skipping - using pre-configured token (no testmail inbox)');
|
||||
}
|
||||
|
||||
// =========================================================================
|
||||
// STEP 10: Verify Downloads
|
||||
// =========================================================================
|
||||
|
||||
@@ -171,6 +171,71 @@ test.describe('Authentication - Beta Enrollment', () => {
|
||||
await page.waitForURL(/\/app/, { timeout: 10000 });
|
||||
expect(page.url()).toContain('/app');
|
||||
});
|
||||
|
||||
test('beta enrollment shows credits after redirect to app', async ({ page }) => {
|
||||
// This test verifies that after beta enrollment, the /app page
|
||||
// correctly fetches and displays user data including credits.
|
||||
// This catches the bug where client-side navigation didn't trigger fetchUser().
|
||||
await setupApiFixtures(page, {
|
||||
mocks: [
|
||||
{
|
||||
method: 'POST',
|
||||
path: '/api/users/beta/enroll',
|
||||
response: {
|
||||
status: 200,
|
||||
body: {
|
||||
status: 'success',
|
||||
message: 'Enrolled in beta',
|
||||
credits_granted: 1,
|
||||
session_token: 'mock-session-token-456',
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
method: 'GET',
|
||||
path: '/api/users/me',
|
||||
response: {
|
||||
status: 200,
|
||||
body: {
|
||||
user: {
|
||||
email: 'beta-tester@example.com',
|
||||
credits: 1,
|
||||
role: 'user',
|
||||
display_name: null,
|
||||
total_jobs_created: 0,
|
||||
total_jobs_completed: 0,
|
||||
},
|
||||
has_session: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
method: 'GET',
|
||||
path: '/api/jobs',
|
||||
response: { body: [] },
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
await page.goto('/');
|
||||
await page.waitForLoadState('networkidle');
|
||||
|
||||
// Complete beta enrollment
|
||||
await page.getByRole('button', { name: /join beta program/i }).click();
|
||||
await page.locator('#beta-email').fill('beta-tester@example.com');
|
||||
await page.locator('textarea').fill(
|
||||
'I want to create a karaoke video for my favorite song!'
|
||||
);
|
||||
await page.locator('input[type="checkbox"]').check();
|
||||
await page.getByRole('button', { name: /get my free credit/i }).click();
|
||||
|
||||
// Wait for redirect to /app
|
||||
await page.waitForURL(/\/app/, { timeout: 10000 });
|
||||
|
||||
// Verify credits are displayed in the UI
|
||||
// The AuthStatus component should show "1 credit" after fetchUser() completes
|
||||
await expect(page.getByText(/1 credit/i)).toBeVisible({ timeout: 10000 });
|
||||
});
|
||||
});
|
||||
|
||||
test.describe('Authentication - Token Persistence', () => {
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
import { test, expect, Page } from '@playwright/test';
|
||||
|
||||
/**
|
||||
* Mobile Responsiveness Tests for Lyrics Review UI
|
||||
*
|
||||
* Tests the lyrics review frontend at localhost:8767 for mobile compatibility.
|
||||
*/
|
||||
|
||||
// Common mobile viewport sizes
|
||||
const MOBILE_VIEWPORTS = {
|
||||
'iPhone SE': { width: 375, height: 667 },
|
||||
'iPhone 14': { width: 390, height: 844 },
|
||||
'Pixel 7': { width: 412, height: 915 },
|
||||
} as const;
|
||||
|
||||
// Helper to check no horizontal overflow
|
||||
async function hasNoHorizontalOverflow(page: Page): Promise<boolean> {
|
||||
return await page.evaluate(() => {
|
||||
const docWidth = document.documentElement.scrollWidth;
|
||||
const viewportWidth = window.innerWidth;
|
||||
return docWidth <= viewportWidth + 2; // Allow 2px tolerance
|
||||
});
|
||||
}
|
||||
|
||||
// Use dev server with API pointing to test server
|
||||
const BASE_URL = 'http://localhost:5173?baseApiUrl=http%3A%2F%2Flocalhost%3A8767%2Fapi';
|
||||
|
||||
test.describe('Lyrics Review UI - Mobile Responsiveness', () => {
|
||||
for (const [deviceName, viewport] of Object.entries(MOBILE_VIEWPORTS)) {
|
||||
test(`${deviceName}: No horizontal overflow`, async ({ page }) => {
|
||||
await page.setViewportSize(viewport);
|
||||
await page.goto(BASE_URL);
|
||||
await page.waitForLoadState('networkidle');
|
||||
|
||||
// Wait for content to render
|
||||
await page.waitForTimeout(1000);
|
||||
|
||||
const noOverflow = await hasNoHorizontalOverflow(page);
|
||||
expect(noOverflow).toBe(true);
|
||||
});
|
||||
|
||||
test(`${deviceName}: Header content is visible`, async ({ page }) => {
|
||||
await page.setViewportSize(viewport);
|
||||
await page.goto(BASE_URL);
|
||||
await page.waitForLoadState('networkidle');
|
||||
|
||||
// Wait for content
|
||||
await page.waitForTimeout(1000);
|
||||
|
||||
// Header title should be visible
|
||||
await expect(page.locator('text=Nomad Karaoke')).toBeVisible({ timeout: 5000 });
|
||||
});
|
||||
|
||||
test(`${deviceName}: Lyrics segments are visible`, async ({ page }) => {
|
||||
await page.setViewportSize(viewport);
|
||||
await page.goto(BASE_URL);
|
||||
await page.waitForLoadState('networkidle');
|
||||
|
||||
// Wait for content
|
||||
await page.waitForTimeout(1000);
|
||||
|
||||
// Should show lyrics text
|
||||
await expect(page.locator('body')).toContainText('Hello', { timeout: 5000 });
|
||||
});
|
||||
|
||||
test(`${deviceName}: Action buttons fit viewport`, async ({ page }) => {
|
||||
await page.setViewportSize(viewport);
|
||||
await page.goto(BASE_URL);
|
||||
await page.waitForLoadState('networkidle');
|
||||
|
||||
// Wait for content
|
||||
await page.waitForTimeout(1000);
|
||||
|
||||
// Look for action buttons (Find/Replace, Edit All, etc.)
|
||||
const findReplaceBtn = page.getByRole('button', { name: /find.*replace/i });
|
||||
|
||||
if (await findReplaceBtn.isVisible({ timeout: 3000 }).catch(() => false)) {
|
||||
const box = await findReplaceBtn.boundingBox();
|
||||
if (box) {
|
||||
// Button should be within viewport
|
||||
expect(box.x).toBeGreaterThanOrEqual(0);
|
||||
expect(box.x + box.width).toBeLessThanOrEqual(viewport.width + 10);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
test('Screenshot comparison - iPhone SE', async ({ page }) => {
|
||||
await page.setViewportSize(MOBILE_VIEWPORTS['iPhone SE']);
|
||||
await page.goto(BASE_URL);
|
||||
await page.waitForLoadState('networkidle');
|
||||
await page.waitForTimeout(2000);
|
||||
|
||||
// Take screenshot for visual verification
|
||||
await page.screenshot({
|
||||
path: '/tmp/lyrics-review-mobile-iphone-se.png',
|
||||
fullPage: true
|
||||
});
|
||||
|
||||
// Just verify content loaded
|
||||
await expect(page.locator('body')).toContainText('Hello');
|
||||
});
|
||||
|
||||
test('Screenshot comparison - Desktop', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 1280, height: 900 });
|
||||
await page.goto(BASE_URL);
|
||||
await page.waitForLoadState('networkidle');
|
||||
await page.waitForTimeout(2000);
|
||||
|
||||
// Take screenshot for visual verification
|
||||
await page.screenshot({
|
||||
path: '/tmp/lyrics-review-desktop.png',
|
||||
fullPage: true
|
||||
});
|
||||
|
||||
// Just verify content loaded
|
||||
await expect(page.locator('body')).toContainText('Hello');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,91 @@
|
||||
/**
|
||||
* Screenshot capture script for PR documentation.
|
||||
* Run with: npm run test:e2e -- tests/take-screenshots.spec.ts
|
||||
*/
|
||||
|
||||
import { test, expect } from '@playwright/test';
|
||||
import path from 'path';
|
||||
import fs from 'fs';
|
||||
|
||||
const SCREENSHOT_DIR = path.join(__dirname, '..', '..', 'public', 'screenshots');
|
||||
|
||||
test.describe('PR Screenshots', () => {
|
||||
test.beforeAll(async () => {
|
||||
// Ensure screenshot directory exists
|
||||
if (!fs.existsSync(SCREENSHOT_DIR)) {
|
||||
fs.mkdirSync(SCREENSHOT_DIR, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('capture homepage', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 1280, height: 800 });
|
||||
|
||||
// Use local dev server for latest code
|
||||
await page.goto('http://localhost:3000', { waitUntil: 'networkidle' });
|
||||
await page.waitForTimeout(1000);
|
||||
|
||||
// Hero section
|
||||
await page.screenshot({
|
||||
path: path.join(SCREENSHOT_DIR, 'homepage-hero.png'),
|
||||
});
|
||||
|
||||
// Full page
|
||||
await page.screenshot({
|
||||
path: path.join(SCREENSHOT_DIR, 'homepage-full.png'),
|
||||
fullPage: true,
|
||||
});
|
||||
});
|
||||
|
||||
test('capture lyrics review UI', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 1280, height: 800 });
|
||||
|
||||
const lyricsUrl = 'http://localhost:8766?baseApiUrl=http%3A%2F%2Flocalhost%3A8766%2Fapi';
|
||||
|
||||
try {
|
||||
await page.goto(lyricsUrl, { waitUntil: 'networkidle', timeout: 10000 });
|
||||
await page.waitForTimeout(2000);
|
||||
|
||||
await page.screenshot({
|
||||
path: path.join(SCREENSHOT_DIR, 'lyrics-review.png'),
|
||||
});
|
||||
} catch (e) {
|
||||
console.log('Warning: Could not capture lyrics review UI. Make sure test-lyrics-review.py is running.');
|
||||
}
|
||||
});
|
||||
|
||||
test('capture instrumental review UI', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 1280, height: 800 });
|
||||
|
||||
const instrumentalUrl = 'http://localhost:8765';
|
||||
|
||||
try {
|
||||
await page.goto(instrumentalUrl, { waitUntil: 'networkidle', timeout: 10000 });
|
||||
await page.waitForTimeout(2000);
|
||||
|
||||
await page.screenshot({
|
||||
path: path.join(SCREENSHOT_DIR, 'instrumental-review.png'),
|
||||
});
|
||||
} catch (e) {
|
||||
console.log('Warning: Could not capture instrumental review UI. Make sure test-instrumental-review.py is running.');
|
||||
}
|
||||
});
|
||||
|
||||
test('capture email templates', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 700, height: 900 });
|
||||
|
||||
const emailDir = '/tmp/email-previews';
|
||||
const emails = ['job_completion', 'action_reminder', 'beta_welcome'];
|
||||
|
||||
for (const emailName of emails) {
|
||||
const emailPath = path.join(emailDir, `${emailName}.html`);
|
||||
if (fs.existsSync(emailPath)) {
|
||||
await page.goto(`file://${emailPath}`, { waitUntil: 'load' });
|
||||
await page.waitForTimeout(300);
|
||||
|
||||
await page.screenshot({
|
||||
path: path.join(SCREENSHOT_DIR, `email-${emailName}.png`),
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -138,6 +138,15 @@ export interface JobLog {
|
||||
details?: Record<string, any>;
|
||||
}
|
||||
|
||||
export interface EncodingWorkerHealth {
|
||||
available: boolean;
|
||||
status: string; // 'ok' | 'offline' | 'not_configured'
|
||||
version?: string | null;
|
||||
active_jobs?: number;
|
||||
queue_length?: number;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
class ApiError extends Error {
|
||||
status: number;
|
||||
data?: any;
|
||||
@@ -418,6 +427,9 @@ export const api = {
|
||||
theme_id?: string;
|
||||
color_overrides?: ColorOverrides;
|
||||
non_interactive?: boolean;
|
||||
// Display overrides - if provided, these appear on title screens/filenames instead of search values
|
||||
display_artist?: string;
|
||||
display_title?: string;
|
||||
}
|
||||
): Promise<AudioSearchResponse> {
|
||||
const body: Record<string, any> = { artist, title, auto_download: autoDownload };
|
||||
@@ -428,6 +440,9 @@ export const api = {
|
||||
if (options?.theme_id) body.theme_id = options.theme_id;
|
||||
if (options?.color_overrides) body.color_overrides = options.color_overrides;
|
||||
if (options?.non_interactive !== undefined) body.non_interactive = options.non_interactive;
|
||||
// Display overrides
|
||||
if (options?.display_artist) body.display_artist = options.display_artist;
|
||||
if (options?.display_title) body.display_title = options.display_title;
|
||||
|
||||
const response = await fetch(`${API_BASE_URL}/api/audio-search/search`, {
|
||||
method: 'POST',
|
||||
@@ -583,7 +598,7 @@ export const api = {
|
||||
},
|
||||
|
||||
// ==========================================================================
|
||||
// Version API endpoint
|
||||
// Version & Health API endpoints
|
||||
// ==========================================================================
|
||||
|
||||
/**
|
||||
@@ -596,6 +611,14 @@ export const api = {
|
||||
return handleResponse(response);
|
||||
},
|
||||
|
||||
/**
|
||||
* Get encoding worker health status
|
||||
*/
|
||||
async getEncodingWorkerHealth(): Promise<EncodingWorkerHealth> {
|
||||
const response = await fetch(`${API_BASE_URL}/health/encoding-worker`);
|
||||
return handleResponse(response);
|
||||
},
|
||||
|
||||
// ==========================================================================
|
||||
// Credits/Payment API endpoints
|
||||
// ==========================================================================
|
||||
@@ -1038,7 +1061,52 @@ export const adminApi = {
|
||||
);
|
||||
return handleResponse(response);
|
||||
},
|
||||
|
||||
/**
|
||||
* Get rendered completion message for a job (for copy to clipboard)
|
||||
*/
|
||||
async getCompletionMessage(jobId: string): Promise<CompletionMessageResponse> {
|
||||
const response = await fetch(
|
||||
`${API_BASE_URL}/api/admin/jobs/${jobId}/completion-message`,
|
||||
{ headers: getAuthHeaders() }
|
||||
);
|
||||
return handleResponse(response);
|
||||
},
|
||||
|
||||
/**
|
||||
* Send completion email for a job
|
||||
*/
|
||||
async sendCompletionEmail(jobId: string, toEmail: string, ccAdmin: boolean = true): Promise<SendCompletionEmailResponse> {
|
||||
const response = await fetch(
|
||||
`${API_BASE_URL}/api/admin/jobs/${jobId}/send-completion-email`,
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
...getAuthHeaders(),
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ to_email: toEmail, cc_admin: ccAdmin }),
|
||||
}
|
||||
);
|
||||
return handleResponse(response);
|
||||
},
|
||||
};
|
||||
|
||||
// Types for admin completion message API
|
||||
export interface CompletionMessageResponse {
|
||||
job_id: string;
|
||||
message: string;
|
||||
subject: string;
|
||||
youtube_url?: string;
|
||||
dropbox_url?: string;
|
||||
}
|
||||
|
||||
export interface SendCompletionEmailResponse {
|
||||
success: boolean;
|
||||
job_id: string;
|
||||
to_email: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
export { ApiError };
|
||||
|
||||
@@ -35,7 +35,7 @@ const STATUS_CONFIG: Record<
|
||||
{ step: number; label: string; isBlocking: boolean; color: string }
|
||||
> = {
|
||||
// Step 1: Setup
|
||||
pending: { step: 1, label: "Setting up", isBlocking: false, color: "text-slate-400" },
|
||||
pending: { step: 1, label: "Setting up", isBlocking: false, color: "text-muted-foreground" },
|
||||
|
||||
// Step 2: Audio Search (optional path)
|
||||
searching_audio: { step: 2, label: "Searching for audio", isBlocking: false, color: "text-blue-400" },
|
||||
@@ -82,7 +82,7 @@ const STATUS_CONFIG: Record<
|
||||
|
||||
// Terminal states (no step progression)
|
||||
failed: { step: 0, label: "Failed", isBlocking: false, color: "text-red-400" },
|
||||
cancelled: { step: 0, label: "Cancelled", isBlocking: false, color: "text-slate-500" },
|
||||
cancelled: { step: 0, label: "Cancelled", isBlocking: false, color: "text-muted-foreground" },
|
||||
};
|
||||
|
||||
const TOTAL_STEPS = 10;
|
||||
@@ -118,7 +118,7 @@ export function getJobStep(job: Job): JobStep {
|
||||
total: TOTAL_STEPS,
|
||||
label: status.replace(/_/g, " "),
|
||||
isBlocking: false,
|
||||
color: "text-slate-400",
|
||||
color: "text-muted-foreground",
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "karaoke-gen-frontend",
|
||||
"version": "0.79.0",
|
||||
"version": "0.81.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "karaoke-gen-frontend",
|
||||
"version": "0.79.0",
|
||||
"version": "0.81.0",
|
||||
"dependencies": {
|
||||
"@hookform/resolvers": "^3.10.0",
|
||||
"@radix-ui/react-accordion": "1.2.2",
|
||||
@@ -75,6 +75,7 @@
|
||||
"concurrently": "^9.2.1",
|
||||
"jest": "^29.7.0",
|
||||
"jest-environment-jsdom": "^29.7.0",
|
||||
"playwright": "^1.57.0",
|
||||
"postcss": "^8.5",
|
||||
"tailwindcss": "^4.1.9",
|
||||
"ts-node": "^10.9.2",
|
||||
@@ -3302,6 +3303,66 @@
|
||||
"node": ">=14.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@emnapi/core": {
|
||||
"version": "1.6.0",
|
||||
"dev": true,
|
||||
"inBundle": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"@emnapi/wasi-threads": "1.1.0",
|
||||
"tslib": "^2.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@emnapi/runtime": {
|
||||
"version": "1.6.0",
|
||||
"dev": true,
|
||||
"inBundle": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"tslib": "^2.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@emnapi/wasi-threads": {
|
||||
"version": "1.1.0",
|
||||
"dev": true,
|
||||
"inBundle": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"tslib": "^2.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@napi-rs/wasm-runtime": {
|
||||
"version": "1.0.7",
|
||||
"dev": true,
|
||||
"inBundle": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"@emnapi/core": "^1.5.0",
|
||||
"@emnapi/runtime": "^1.5.0",
|
||||
"@tybys/wasm-util": "^0.10.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@tybys/wasm-util": {
|
||||
"version": "0.10.1",
|
||||
"dev": true,
|
||||
"inBundle": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"tslib": "^2.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/tslib": {
|
||||
"version": "2.8.1",
|
||||
"dev": true,
|
||||
"inBundle": true,
|
||||
"license": "0BSD",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/@tailwindcss/oxide-win32-arm64-msvc": {
|
||||
"version": "4.1.17",
|
||||
"resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-arm64-msvc/-/oxide-win32-arm64-msvc-4.1.17.tgz",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "karaoke-gen-frontend",
|
||||
"version": "0.80.0",
|
||||
"version": "0.81.0",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"build": "next build",
|
||||
@@ -82,6 +82,7 @@
|
||||
},
|
||||
"devDependencies": {
|
||||
"@playwright/test": "^1.57.0",
|
||||
"playwright": "^1.57.0",
|
||||
"@tailwindcss/postcss": "^4.1.9",
|
||||
"@testing-library/jest-dom": "^6.9.1",
|
||||
"@testing-library/react": "^16.3.1",
|
||||
|
||||
|
After Width: | Height: | Size: 101 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 112 KiB |
|
After Width: | Height: | Size: 38 KiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 602 KiB |
|
After Width: | Height: | Size: 23 KiB |
|
After Width: | Height: | Size: 14 KiB |
|
After Width: | Height: | Size: 19 KiB |