Compare commits

...
Author SHA1 Message Date
Andrew Beveridge ☄️andClaude Opus 4.5 dacd2b58ea fix(admin): Fix impersonation button and remove Make Admin from detail page (#251)
## Summary

- Fix impersonate button not working in admin dashboard - clicking did
nothing (no network request, no errors)
- Remove Make Admin/Remove Admin button from user detail page that
should have been removed in #249

## Changes

- **auth.ts**: Fixed broken import - `adminApi` is a separate export
from `api.ts`, not nested under the `api` object.
`api.adminApi.impersonateUser()` was calling
`undefined.impersonateUser()`
- **detail/page.tsx**: Removed Make Admin button, `handleToggleRole`
function, and unused `Shield`/`ShieldOff` imports

## Test plan

- [x] Unit tests pass (150 tests)
- [x] E2E impersonation tests pass (4 tests)
- [ ] Manual: Click impersonate button on admin users list → should
redirect to /app as that user
- [ ] Manual: Visit user detail page → Make Admin button should not
appear

🤖 Generated with [Claude Code](https://claude.ai/code)

@coderabbitai ignore

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 20:51:14 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 686b53a9d3 refactor(api): Rename 'done for you' to 'made for you' + enhance checkout UX (#250)
## Summary

- Rename all "done for you" / "done_for_you" / "DoneForYou" references
to "made for you" variants throughout the codebase
- Enhance the Stripe checkout experience with richer product details and
more payment options

## Changes

### Naming Refactor
- Rename `DONE_FOR_YOU_PACKAGE` → `MADE_FOR_YOU_PACKAGE`
- Rename endpoint `/done-for-you/checkout` → `/made-for-you/checkout`
- Rename `DoneForYouCheckoutRequest` → `MadeForYouCheckoutRequest`
- Update Stripe metadata `order_type` from `done_for_you` →
`made_for_you`
- Update all email subjects, log messages, and documentation

### Checkout UX Improvements
- Product name now shows song info: "Karaoke Video: Artist - Title"
- Description shows service details: "Professional 4K karaoke video with
perfectly synced lyrics, delivered to your email within 24 hours"
- Added product image (Nomad Karaoke logo)
- Enabled all payment methods (Apple Pay, Google Pay, Link, Cash App,
Klarna, etc.) by removing `payment_method_types=['card']` restriction

## Test plan

- [x] Tested checkout flow locally with production Stripe keys
- [x] Verified product image displays correctly
- [x] Verified additional payment methods appear (Cash App, Klarna
shown; Apple Pay/Google Pay show on compatible devices)
- [ ] Backend tests pass

<!-- @coderabbitai ignore -->

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 00:58:30 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 1110d02519 feat(admin): Add user impersonation feature (#249)
## Summary
- Add user impersonation feature for admins to view the app as any user
- Replace "Make Admin" button with "Impersonate" button in admin users
page
- Display amber impersonation banner when viewing as another user

## Changes
- **Backend**: Add `POST /api/admin/users/{email}/impersonate` endpoint
that creates a session token for the target user
- **Frontend Auth**: Add `startImpersonation` / `endImpersonation`
methods to auth store with token rollback on failure and nested
impersonation prevention
- **Admin UI**: Replace role toggle button with impersonate button
(disabled for self)
- **Banner Component**: New `ImpersonationBanner` component showing
"Viewing as {email}" with Stop button
- **Tests**: 7 backend unit tests + 4 E2E tests (2 skipped - require
stateful mocks)
- **Docs**: Updated API.md with new endpoint, added testing lesson to
LESSONS-LEARNED.md

## Testing
- [x] Backend unit tests pass (7/7 impersonation tests)
- [x] Frontend E2E tests pass (4 pass, 2 skipped)
- [x] Manual testing needed for full flow

## Review
- [x] CodeRabbit CLI review completed locally
- [x] Review feedback addressed (token rollback, nested impersonation
guard)

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 19:13:48 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 addd0e1ce5 fix(frontend): Admin dashboard crash + add test jobs toggle (#248)
## Summary

- Fixes React error #310 that was crashing the admin dashboard at
`/admin`
- Adds "Show Test Jobs" toggle in user menu for admins to view E2E test
data on `/app`
- Adds ESLint to CI to prevent similar bugs in the future

## Root Cause

PR #239 added `useAdminSettings()` hook **after** conditional returns in
`admin/layout.tsx`, violating React's Rules of Hooks. The hook was only
called on some renders, causing React to crash with "Rendered more hooks
than during the previous render".

## Changes

### Bug Fix
- `frontend/app/admin/layout.tsx` - Move `useAdminSettings()` before
conditional returns

### Feature: Test Jobs Toggle
- `frontend/components/auth/AuthStatus.tsx` - Add "Show Test Jobs"
toggle in admin section of user dropdown
- `frontend/app/app/page.tsx` - Pass `exclude_test` param based on admin
preference
- `frontend/lib/api.ts` - Add `exclude_test` param to `api.listJobs()`

### CI: ESLint for Hooks
- `frontend/eslint.config.mjs` - New ESLint config with
`react-hooks/rules-of-hooks: error`
- `frontend/package.json` - Add ESLint dependencies
- `.github/workflows/ci.yml` - Add `frontend-lint` job to CI gate

## Test plan

- [x] Build passes locally
- [x] ESLint passes (no hooks violations)
- [x] 68 unit tests pass
- [ ] Verify admin dashboard loads at
https://gen.nomadkaraoke.com/admin/
- [ ] Verify "Show Test Jobs" toggle appears in user menu for admins
- [ ] Verify toggling it shows/hides test jobs on /app page

<!-- @coderabbitai ignore -->

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 17:13:27 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 23e28489f6 fix(video-worker): Prevent duplicate Google Drive uploads causing cleanup failures (#247)
## Summary
- Fixed duplicate Google Drive uploads that were causing E2E test
cleanup to leave orphaned files
- The orchestrator path was uploading twice: once in
`VideoWorkerOrchestrator._run_gdrive_distribution()` and again in
`_handle_native_distribution()`
- This created duplicate files with different IDs, only one set got
saved for cleanup

## Changes
- Added check for existing `gdrive_files` in
`_handle_native_distribution()` to skip upload if orchestrator already
handled it
- Added logging when upload is skipped
- Documented the issue and solution in LESSONS-LEARNED.md

## Test plan
- [x] All backend tests pass (68 passed)
- [ ] E2E happy path test should no longer leave orphaned Google Drive
files

<!-- @coderabbitai ignore -->

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 17:06:44 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 42c304dcdf feat(multitenancy): Implement white-label B2B portal infrastructure (#241)
## Summary

Adds multi-tenant white-label portal infrastructure for B2B customers.
Vocal Star is the first tenant at `vocalstar.nomadkaraoke.com`.

## Changes

### Backend
- **Tenant Models** (`backend/models/tenant.py`): Pydantic models for
TenantConfig with branding, features, defaults, and auth settings
- **Tenant Service** (`backend/services/tenant_service.py`): GCS-backed
config loading with 5-min cache, signed URL generation for logos
- **Tenant API** (`backend/api/routes/tenant.py`): `GET
/api/tenant/config` and `GET /api/tenant/config/{tenant_id}` endpoints
- **Tenant Middleware** (`backend/middleware/tenant.py`): Extracts
tenant from X-Tenant-ID header, query param (dev only), or Host
subdomain
- **Auth Updates**: Email domain validation for tenant-restricted auth,
sender email override
- **Feature Enforcement**: Audio search endpoint checks
`features.audio_search`, returns 403 if disabled
- **Job Scoping**: Added `tenant_id` field to JobCreate

### Frontend
- **Tenant Store** (`frontend/lib/tenant.ts`): Zustand store with tenant
state, `useTenant()` hook, CSS variable application
- **TenantProvider** (`frontend/components/tenant-provider.tsx`):
Hydration-safe initialization via useEffect
- **TenantLogo** (`frontend/components/tenant-logo.tsx`): Dynamic logo
with fallback to Nomad Karaoke
- **Feature-Flagged UI**: Tabs conditionally rendered based on tenant
features, active tab sync
- **Locked Themes**: `defaults.locked_theme` hides theme selector and
enforces specific theme
- **Output Links**: Filtered based on distribution features

### Security
- Query param tenant detection disabled in production (prevents
spoofing)
- Strict subdomain patterns: only `{tenant}.nomadkaraoke.com` or
`{tenant}.gen.nomadkaraoke.com`
- PII masking in logs (`an***@vo***.com`)
- `is_active` check prevents inactive tenants from being used

### Documentation
- ARCHITECTURE.md: New Multitenancy section
- API.md: Tenant Config endpoints
- LESSONS-LEARNED.md: 3 new lessons on multitenancy patterns
- Archive: Comprehensive implementation document

### Setup
- `scripts/setup-vocalstar-tenant.py`: Uploads theme assets, creates
tenant/theme configs in GCS

## Vocal Star Configuration

- **Subdomain**: `vocalstar.nomadkaraoke.com`
- **Features**: File upload only (no audio search, no YouTube URL)
- **Distribution**: Download only (no YouTube/Dropbox/GDrive)
- **Theme**: Locked to "vocalstar" (yellow/blue)
- **Auth**: Restricted to `@vocal-star.com` and `@vocalstarmusic.com`
emails

## Testing

- [x] Frontend TypeScript build passes
- [x] CodeRabbit CLI review completed (3 cycles, all issues fixed)
- [x] Manual testing with `?tenant=vocalstar` query param

## Next Steps (post-merge)

1. Configure DNS: CNAME for `vocalstar.nomadkaraoke.com`
2. Update Cloudflare Pages for subdomain
3. Run setup script to upload Vocal Star assets to GCS
4. Test at `https://vocalstar.nomadkaraoke.com`

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 16:39:56 -05:00
Andrew Beveridge ☄️andClaude b39f37610c feat(frontend): Display brand code prefix on completed jobs (#243)
## Summary
- Display the brand code (e.g., "NOMAD-1194:") as a prefix on completed
job titles in the Recent Jobs UI
- Brand code is already stored in `state_data.brand_code` when jobs
complete with Dropbox distribution
- This change simply displays that existing data

## Changes
- `frontend/components/job/JobCard.tsx`: Add brand code prefix to job
title display

## Testing
- [x] Unit tests pass (128/128)
- [x] Manual review of code change

## Review
- [x] CodeRabbit CLI review completed locally
- [x] No issues found

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-08 15:31:53 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 6da758dc0c docs: Add zone availability notes for c4d instances (#246)
## Summary
- Documents zone availability lesson learned during c4d-highcpu-32
deployment
- Updates performance investigation status to Phase 2 Complete

## Changes
- `docs/LESSONS-LEARNED.md`: Added note about c4d instance zone
availability (us-central1-c works when us-central1-a/b have resource
exhaustion)
- `docs/archive/2026-01-08-performance-investigation.md`: Updated status
from "In Progress" to "Complete"

## Testing
- [x] Documentation only - no code changes

## Review
- [x] Docs-only change, no code review needed

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 15:30:53 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 98ea1519a1 fix(infra): Use us-central1-c for encoding worker (c4d availability) (#245)
## Summary

Quick fix for encoding worker deployment failure - c4d-highcpu-32 is
unavailable in us-central1-a.

## Changes

- Added `ENCODING_WORKER_ZONE = us-central1-c` in config.py
- Updated encoding_worker_vm.py to use the new zone constant
- Deleted the terminated instance in us-central1-a

## Context

PR #244 merged but Pulumi failed with:
> The zone 'projects/nomadkaraoke/zones/us-central1-a' does not have
enough resources available

Tested: us-central1-c has c4d-highcpu-32 capacity.

@coderabbitai ignore

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 19:27:27 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 2e71dc3158 feat(infra): Upgrade encoding worker to c4d-highcpu-32 (4.92x faster) (#244)
## Summary

Upgrades the GCE encoding worker from c4-standard-8 (Intel Xeon) to
c4d-highcpu-32 (AMD EPYC 9B45 Turin), achieving **4.92x faster encoding
performance** based on rigorous benchmarking.

## Changes

- **Infrastructure**: Updated `MachineTypes.ENCODING_WORKER` to
`c4d-highcpu-32` in `config.py`
- **Documentation**: Comprehensive benchmark results in
`docs/archive/2026-01-08-performance-investigation.md`
- **Lessons Learned**: Added "GCE Instance Type Selection for FFmpeg
Encoding" section
- **Benchmark Scripts**: Added reusable scripts for future
infrastructure testing:
- `scripts/benchmark_encoding.py` - Local Python benchmark using actual
code paths
  - `scripts/benchmark_encoding_gce.sh` - Shell benchmark for GCE VMs
- `scripts/benchmark_candidates.sh` - Multi-VM orchestration for
comparative testing

## Benchmark Results

| Instance | CPU | Total Time | vs Baseline |
|----------|-----|------------|-------------|
| c4-standard-8 (baseline) | Intel Xeon | 666s | 1.00x |
| c4-highcpu-16 | Intel Xeon | 309s | 2.16x |
| c4a-highcpu-16 | Google Axion ARM | 248s | 2.69x |
| c4d-highcpu-16 | AMD EPYC 9B45 | 220s | 3.03x |
| **c4d-highcpu-32** | **AMD EPYC 9B45** | **135s** | **4.92x** |

Key finding: AMD EPYC Turin significantly outperforms Intel Granite
Rapids for CPU-bound FFmpeg/libass workloads.

## Testing

- [x] Benchmarked on actual GCE instances with production data
- [x] Verified c4d-highcpu-32 uses hyperdisk-balanced (already
configured)
- [ ] Infrastructure deploys via CI after merge

## Deployment Impact

⚠️ Changing machine type will cause Pulumi to **replace** the VM (~1-2
min downtime). Jobs queued during restart will retry automatically.

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 14:17:02 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 03a7d573f5 fix: Theme enforcement and GCE encoding retry logic (#242)
## Summary
- Fix E2E test job encoding failure caused by GCE worker restart during
deployment
- Fix done-for-you orders missing theme (producing unstyled videos)
- Add defense-in-depth theme enforcement to prevent future theme-less
jobs

## Changes
- **Theme Enforcement**: Jobs now require `theme_id` at creation time
(JobManager rejects without it) with safety net validation in
screens_worker
- **Retry Logic**: Added exponential backoff (3 retries, 2s→4s→8s) to
`encoding_service.py` for transient connection failures
- **Done-for-you Webhook**: Apply default Nomad theme to orders created
via Stripe webhook
- **Tests**: Updated test fixtures to provide theme mocks for new
validation
- **Docs**: Added lessons learned for defense-in-depth and retry
patterns

## Root Causes Investigated
1. **Job b099c91b** (E2E test): Failed at encoding with "Cannot connect
to host" - GCE worker restarted at exact moment of connection attempt
2. **Job 88669f92**: Had `theme_id: None` - done-for-you webhook didn't
apply default theme

## Testing
- [x] Unit tests pass (`make test`)
- [x] Emulator integration tests pass
- [x] CodeRabbit CLI review completed and issues addressed

## Review
- [x] CodeRabbit CLI review completed locally (3 issues fixed)
- [x] Documentation updated

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 12:52:18 -05:00
Andrew Beveridge ☄️andClaude 80b8d9c5ae feat(admin): Add toggle to hide/show test data in admin dashboard (#239)
## Summary

- Filter out E2E test data from admin dashboard by default (emails
ending in `@inbox.testmail.app`)
- Add "Show test data" toggle in admin header that persists to
localStorage
- Add `exclude_test` query parameter to all admin list/stats endpoints
(defaults to `true`)
- Add streaming limits with warnings to prevent memory issues on large
datasets

## Changes

**Backend:**
- New `backend/utils/test_data.py` with `is_test_email()` utility
- Updated admin stats endpoint with test data filtering
- Updated users list, jobs list, beta stats, and audio searches
endpoints
- Added USERS_STREAM_LIMIT (2000) and JOBS_STREAM_LIMIT (10000) with
warning logs

**Frontend:**
- New `frontend/lib/admin-settings.ts` zustand store for admin
preferences
- Toggle switch in admin layout header
- All admin pages pass `exclude_test: !showTestData` to API calls

**Docs:**
- Updated API.md with exclude_test parameter documentation

## Test Plan

- [x] Backend tests pass (68 passed)
- [x] Frontend tests pass (128 passed)
- [x] Frontend build succeeds
- [x] CodeRabbit review passed
- [ ] Manual: Visit `/admin` - stats should show lower numbers by
default
- [ ] Manual: Toggle "Show test data" ON - counts should increase
- [ ] Manual: Toggle persists across page refreshes

<!-- @coderabbitai ignore -->

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-08 16:21:34 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 2337f70311 docs: Update performance investigation with empirical E2E test data (#240)
## Summary

Updates the performance investigation documentation with empirical data
from production E2E tests comparing before/after the optimization PR
#236.

## Key Updates

- **Empirical comparison**: Added actual timing data from E2E tests
showing 50% reduction in lyrics processing (12m53s → 6m28s)
- **Complete pipeline breakdown**: Documented all 4 stages (Audio,
Lyrics, Video Render, Encoding) with sub-stage timing
- **Future optimization ideas**: Listed 9 potential optimizations with
effort/impact estimates
- **Path to 10-minute goal**: Clear roadmap showing which optimizations
would get us there

## Test Plan

- [x] Documentation-only change, no code changes

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 11:20:28 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 e2298c2b29 fix: Use audio search flow for done-for-you orders without YouTube URL (#238)
## Summary
- Fix done-for-you orders failing immediately when submitted without a
YouTube URL
- Orders with only artist+title now properly use the audio search flow
to find and download audio
- Added documentation for this gotcha to prevent future occurrences

## Changes
- **backend/api/routes/users.py**: Updated `_handle_done_for_you_order`
to:
  - Use audio search flow when no YouTube URL provided
  - Search for audio sources using flacfetch
  - Auto-select best result and download to GCS
- Gracefully handle search/download failures by transitioning to
AWAITING_AUDIO_SELECTION
- Properly cleanup temp directories using try/finally with shutil.rmtree
- **docs/LESSONS-LEARNED.md**: Added "Jobs Without URLs Must Use Audio
Search Flow" gotcha

## Testing
- [x] Syntax validation passes
- [x] Manual testing: order submitted without URL now progresses through
search flow

## Review
- [x] CodeRabbit CLI review completed locally
- [x] Review feedback addressed (temp directory cleanup)

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 05:19:38 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 c898689fb6 WIP: Lyrics processing performance optimization (#236)
## Summary

Major performance optimization for lyrics processing pipeline. Analysis
of job 36c21ece revealed the lyrics transcription stage alone took **16
minutes 28 seconds** - the target is 10 minutes for the entire pipeline.

### Key Bottlenecks Identified & Fixed

| Bottleneck | Time Lost | Fix | Status |
|------------|-----------|-----|--------|
| Langfuse callback init | 201s | Preload at startup | **Done** |
| SyllablesMatchHandler x2 | 158s | Preload NLTK + singleton | **Done**
|
| Model init timeouts | 300s+ | Warmup + share instance | **Done** |
| Sequential anchor search | 38s | Parallelize | **Done** |

### Changes

**Phase 1 (Quick Fixes):**
- [x] Add NLTK preloader (`backend/services/nltk_preloader.py`)
- [x] Add Langfuse preloader (`backend/services/langfuse_preloader.py`)
- [x] Add `warmup()` method to LangChainBridge for eager model init
- [x] Update SyllablesMatchHandler to use preloaded cmudict
- [x] Update ModelFactory to use preloaded Langfuse handler

**Phase 2 (Architecture):**
- [x] Parallelize anchor sequence search (ThreadPoolExecutor, 4 workers)
- [x] Add `_process_ngram_length_no_state()` for thread-safe processing
- [x] Design document for future GCE lyrics worker

### Expected Impact

- **Before**: ~30 minutes total, 16+ minutes for lyrics stage
- **After**: ~18 minutes total, ~5 minutes for lyrics stage
- **Savings**: ~11 minutes per job

### Configuration

New environment variables:
- `ANCHOR_SEARCH_WORKERS`: Number of parallel workers for anchor search
(default: 4)
- `ANCHOR_SEARCH_SEQUENTIAL`: Set to "1" to disable parallel processing

## Test plan

- [x] Unit tests pass: `make test` (1020 passed)
- [ ] Deploy to staging and run E2E test with same song
- [ ] Compare job times before/after

See detailed analysis:
`docs/archive/2026-01-08-performance-investigation.md`

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 04:47:37 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 04467fe856 feat: Add done-for-you karaoke video order flow (#237)
## Summary
- Adds "Done For You" service at $15 for full-service karaoke video
creation
- Integrates with new nomadkaraoke.com homepage order form
- Creates jobs automatically when payment completes
- Sends email notifications to customer and admin

## Changes
- `stripe_service.py`: Add `DONE_FOR_YOU_PACKAGE` ($15) and
`create_done_for_you_checkout_session()` method
- `users.py`: Add `/users/done-for-you/checkout` endpoint and webhook
handler for done_for_you orders

## Flow
1. Customer fills order form on nomadkaraoke.com with artist, title,
source type
2. Frontend calls `POST /api/users/done-for-you/checkout` to create
Stripe checkout session
3. Customer completes payment via Stripe Checkout
4. Webhook receives `checkout.session.completed` with `order_type:
done_for_you` metadata
5. Job is created automatically with customer as owner
6. Confirmation email sent to customer, notification sent to Andrew
7. Andrew reviews job within 24 hours (lyrics, instrumental selection)
8. Customer receives video delivery email when complete

## Test plan
- [ ] Unit tests pass
- [ ] Create test checkout session via API
- [ ] Process test webhook for done_for_you order
- [ ] Verify job is created with correct metadata
- [ ] Verify email notifications are sent

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 04:26:31 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 22c0b582ca fix(theme): Apply default Nomad theme when no theme specified (#235)
## Summary
- Fix regression where new jobs weren't getting the Nomad theme applied
after the UI simplification removed theme selection from the frontend
- Backend now automatically applies the default theme ("nomad") when no
theme_id is provided in API requests

## Changes
- Added default theme application logic to all job creation endpoints:
  - `POST /api/audio-search/search`
  - `POST /api/jobs/upload`
  - `POST /api/jobs/create-with-upload-urls`
  - `POST /api/jobs/create-from-url`
  - `POST /api/jobs/{job_id}/finalise-uploads-complete`

## Root Cause
Commit a573152 simplified the UI by hiding theme options, but the
backend wasn't applying a default theme when `theme_id` was not
provided. Job 36c21ece confirmed `Theme: None` was being set.

## Test plan
- [x] Code imports successfully without syntax errors
- [x] Verified "nomad" theme is configured as `is_default: true` via
production API
- [ ] Deploy and create a test job to verify theme is applied

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 04:02:44 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 a573152606 feat(ui): Simplify Create Karaoke form and fix brand color contrast (#234)
## Summary
- Simplify the "Create Karaoke Video" form by hiding advanced options
and improving UX
- Fix brand pink color contrast for accessibility (WCAG compliance)
- Fix E2E tests that were failing due to ambiguous form selectors
- Reorganize job output links into logical sections

## Changes

### UI Simplification
- Hide theme settings from user-facing UI (now admin-only)
- Add collapsible "Display As" toggle for title screen overrides
- Make non-interactive mode checkbox admin-only
- Change placeholders to user-friendly language (removed "on trackers")

### Audio Search Dialog
- Hide technical tracker names (RED, OPS), only show YouTube badge
- Replace exact seeder count with "High/Medium/Low availability" badges
- Rename "TOP SEEDED" category to "BEST CHOICE"
- Add red VINYL badge for vinyl rips

### Brand Color Accessibility
- Swap brand pink colors for better white text contrast
- Default `--brand-pink`: #ff5bb8 (darker, better WCAG contrast)
- Hover `--brand-pink-hover`: #ff7acc (lighter, original brand pink)
- Apply brand pink to form tabs, buttons, and primary CTAs

### Output Links Reorganization
- Organize into "Downloads", "Links", "Admin Tools" sections
- 4K Video as primary CTA (brand pink), 720p as secondary
- Secondary downloads use dark gray style with border
- Dropbox uses Dropbox blue (#0061FE)
- YouTube keeps red (platform branding)

### E2E Test Fixes
- Add `data-testid` attributes to search form inputs
- Update all tests to use `getByTestId` for robust selectors
- Remove obsolete theme selection logic from production tests

## Test plan
- [x] Regression tests pass (9/9 karaoke-generation, 28/28 mobile)
- [x] Frontend builds successfully
- [ ] Manual verification of UI changes in browser
- [ ] Production E2E test after deployment

## Documentation
- Updated `docs/BRAND-STYLE-GUIDE.md` to v1.2 with accessibility notes
- Added testing lesson to `docs/LESSONS-LEARNED.md`
- Updated `docs/README.md` with recent changes entry

<!-- @coderabbitai ignore -->

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 02:41:28 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 37cfd734bc feat(perf): Preload SpaCy model at container startup (#233)
## Summary

- Preload SpaCy `en_core_web_sm` model at FastAPI container startup to
eliminate 60+ second delay during first agentic correction job
- Add `backend/services/spacy_preloader.py` singleton service for model
storage
- Modify `PhraseAnalyzer` and `SyllablesMatchHandler` to use preloaded
model when available
- Add timing logs to verify optimization is working

## Background

Job `2ccbdf6b` investigation revealed a 63-second gap in logs caused by
SpaCy model loading during `AnchorSequenceFinder` initialization. Cloud
Run's filesystem I/O is slower than local SSD, making lazy loading
expensive on first request.

## Changes

| File | Change |
|------|--------|
| `backend/services/spacy_preloader.py` | New singleton service for
preloading |
| `backend/main.py` | Call preload in FastAPI lifespan handler |
| `phrase_analyzer.py` | Check for preloaded model first |
| `syllables_match.py` | Check for preloaded model first |
| `anchor_sequence.py` | Add timing logs to initialization |
| `backend/tests/test_spacy_preloader.py` | Unit tests (7 tests) |

## Expected Logging (when working)

**Startup:**
```
Preloading SpaCy model 'en_core_web_sm'...
SpaCy model 'en_core_web_sm' preloaded in X.XXs
```

**During job:**
```
Initializing AnchorSequenceFinder...
Using preloaded SpaCy model: en_core_web_sm
Initialized AnchorSequenceFinder in 0.XXs
```

## Test plan

- [x] Unit tests pass (7/7 for spacy_preloader)
- [x] Full test suite passes (1018 passed, 2 pre-existing failures
unrelated)
- [ ] Deploy and run test job to verify logs show preloaded model usage
- [ ] Verify `Initialized AnchorSequenceFinder` time is < 1 second

<!-- @coderabbitai ignore -->

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 00:53:17 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 522ab64dec fix(agentic): Add thread-safe locking to LangChainBridge model initialization (#232)
## Summary

- Fix race condition where 5 parallel gap-processing threads all tried
to initialize the AI model simultaneously
- Caused ~6 minute delays and 0/8 gaps corrected during agentic lyrics
correction
- Discovered while investigating job `2ccbdf6b` (ABBA - Waterloo) which
took 17 minutes instead of ~5 minutes

## Changes

- `langchain_bridge.py`: Add `threading.Lock()` with double-checked
locking pattern to ensure only one thread initializes the model while
others wait
- `test_langchain_bridge_thread_safety.py`: New test file (305 lines)
with 4 tests covering concurrent initialization, sequential reuse,
failure handling, and timing
- `LESSONS-LEARNED.md`: Document the thread-safe lazy initialization
pattern

## Root Cause

The lazy initialization check `if not self._chat_model:` was not
thread-safe. When 5 parallel threads called
`generate_correction_proposals()` simultaneously, ALL threads saw
`_chat_model == None` and entered the initialization block, causing:
- 5 concurrent Vertex AI model initializations instead of 1
- Resource contention leading to ~6 minute delays
- All 5 threads timing out (30s limit)
- Circuit breaker triggered → 0/8 gaps corrected

## Testing

- [x] Unit tests pass (68 backend + 25 agentic tests)
- [x] New thread safety tests verify fix works
- [x] Tests used TDD approach - failed before fix, pass after

## Review

- [x] CodeRabbit CLI review completed locally
- [x] Review feedback addressed (removed unused variable)

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 00:12:33 -05:00
Andrew Beveridge ☄️andClaude f661b6a27b feat(branding): Implement consistent brand theming across all frontends (#231)
## Summary

- Implement unified brand theming with brand pink (#ff7acc) as primary
color across all Nomad Karaoke frontends
- Create comprehensive brand style guide for human and LLM reference
- Add proper light/dark mode support throughout
- Update homepage content for transparency and accuracy
- Create product communication guide for consistent messaging

## Changes

### Homepage Content Updates
- Changed "in minutes" to "in under 30 minutes" for accuracy
- Removed AI terminology per user preference (use "our system" instead)
- Made review step clearly required, not optional polish
- Fixed YouTube claim: videos go to Nomad Karaoke channel, not user's
- Added demo video embed
- Added placeholder sections for screenshots (job dashboard, lyrics
review, etc.)

### Brand Style Guide (`docs/BRAND-STYLE-GUIDE.md`)
- Comprehensive guide covering colors, typography, logo usage, UI
components, email design
- Updated to v1.1: radial background gradients, logo usage (SVG
preferred), light/dark theme requirements
- Fixed success color consistency (#22c55e)

### Product Communication Guide (`docs/PRODUCT-COMMUNICATION-GUIDE.md`)
- Core principles: transparency over hype, avoid AI term, acknowledge
user effort
- Terminology preferences table
- FAQ guidance for consistent answers
- Referenced in CLAUDE.md for future agent sessions

### Main Frontend (`frontend/`)
- Updated CSS variables in `globals.css` for brand colors and light/dark
mode
- Added ThemeProvider to layout for proper theme switching
- Updated landing page, auth dialog, job cards with brand pink (#ff7acc)
- Created Playwright screenshot tests for visual verification

### Lyrics Transcriber Frontend
- Updated MUI theme with brand pink primary color
- Fixed theme-aware colors in: EditModal, EditWordList, WordDivider,
Word, CorrectedWordWithActions, TranscriptionView, ReferenceView,
AIFeedbackModal
- Fixed TapButton onMouseLeave race condition
- Added clarifying tooltip to Undo Auto Corrections button

### Email Templates
- Refactored to use reusable helper methods (`_build_email_html`,
`_get_email_header`, `_get_email_footer`)
- Updated all 7 email templates with brand pink headers, buttons, and
accents
- Fixed success color to match brand guide (#22c55e)
- Updated preview script to use actual email service

### Other
- Bumped versions: backend 0.96.0, frontend 0.81.0
- Added archive entry for brand consistency work

## Screenshots

### Homepage

| Hero Section | Full Page |
|--------------|-----------|
| ![Homepage
Hero](https://raw.githubusercontent.com/nomadkaraoke/karaoke-gen/feat/sess-20260107-0008-brand-consistency/frontend/public/screenshots/homepage-hero.png)
| ![Homepage
Full](https://raw.githubusercontent.com/nomadkaraoke/karaoke-gen/feat/sess-20260107-0008-brand-consistency/frontend/public/screenshots/homepage-full.png)
|

### Lyrics Review UI

![Lyrics
Review](https://raw.githubusercontent.com/nomadkaraoke/karaoke-gen/feat/sess-20260107-0008-brand-consistency/frontend/public/screenshots/lyrics-review.png)

### Instrumental Review UI

![Instrumental
Review](https://raw.githubusercontent.com/nomadkaraoke/karaoke-gen/feat/sess-20260107-0008-brand-consistency/frontend/public/screenshots/instrumental-review.png)

### Email Templates

| Job Completion | Action Reminder | Beta Welcome |
|----------------|-----------------|--------------|
| ![Job
Completion](https://raw.githubusercontent.com/nomadkaraoke/karaoke-gen/feat/sess-20260107-0008-brand-consistency/frontend/public/screenshots/email-job_completion.png)
| ![Action
Reminder](https://raw.githubusercontent.com/nomadkaraoke/karaoke-gen/feat/sess-20260107-0008-brand-consistency/frontend/public/screenshots/email-action_reminder.png)
| ![Beta
Welcome](https://raw.githubusercontent.com/nomadkaraoke/karaoke-gen/feat/sess-20260107-0008-brand-consistency/frontend/public/screenshots/email-beta_welcome.png)
|

## Test plan

- [x] Playwright screenshots captured
- [x] Visual verification of brand consistency
- [x] Frontend build passes
- [x] CodeRabbit review feedback addressed (3 cycles)

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-07 23:10:35 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 3f02122cad fix(unicode): Comprehensive Unicode sanitization for artist/title fields (#230)
## Summary

Fixes job failures caused by non-ASCII "smart" characters (curly quotes,
em dashes, etc.) in artist/title fields. Job d49efab1 failed because the
title "Mama Says (You Can't Back Down)" contained a curly apostrophe
(U+2019) instead of a straight apostrophe, causing HTTP header encoding
failures in the Modal API.

### Root Cause
HTTP headers must be latin-1 encodable. Many Unicode characters that
look identical to ASCII equivalents (curly quotes, em dashes, ellipsis,
non-breaking spaces) are NOT latin-1 compatible. Users commonly
copy/paste text from Word, macOS, or websites that use "smart"
typography.

### Solution
Two-tier sanitization approach:

1. **Input-time normalization** (`normalize_text()`): Standardize
visually-similar Unicode characters to ASCII equivalents when data
enters the system
2. **Output-time sanitization** (`sanitize_filename()`): Additional
safety for filenames and HTTP headers

## Changes

### New Functions (`karaoke_gen/utils/__init__.py`)
- `normalize_text()` - Converts curly quotes → straight quotes, em
dashes → hyphens, various whitespace → regular space, ellipsis → three
dots
- Character mapping dicts: `APOSTROPHE_REPLACEMENTS`,
`DOUBLE_QUOTE_REPLACEMENTS`, `DASH_REPLACEMENTS`,
`WHITESPACE_REPLACEMENTS`
- Updated `sanitize_filename()` to use `normalize_text()` internally

### Automatic Normalization (Pydantic Validators)
- `JobCreate` model validator normalizes `artist` and `title` fields
- `AudioSearchRequest` model validator normalizes all text fields

### Sanitization Applied (11 files)
- `backend/workers/audio_worker.py` - Modal API artist_title
- `backend/services/email_service.py` - Email subjects
- `backend/api/routes/admin.py` - Admin email subjects
- `backend/api/routes/jobs.py` - Content-Disposition headers
- `backend/services/gdrive_service.py` - Google Drive filenames
- `backend/workers/video_worker.py` - Output filenames (3 locations)
- `backend/workers/video_worker_orchestrator.py` - Distribution
filenames (2 locations)
- `backend/workers/screens_worker.py` - Artist/title in screens (2
locations)
- `backend/workers/render_video_worker.py` - Output prefix
- `karaoke_gen/karaoke_finalise/karaoke_finalise.py` - Gmail draft
subject

### Tests
- 52 comprehensive tests in `backend/tests/test_unicode_sanitization.py`
- Tests for all character categories, model validators, real-world
examples, edge cases
- Full test suite passes (68 tests)

### Documentation
- Added lesson learned entry in `docs/LESSONS-LEARNED.md`

## Test plan

- [x] All 52 Unicode sanitization tests pass
- [x] Full backend test suite passes (68 tests)
- [x] CodeRabbit review clean
- [ ] Manual test: Create job with curly quote in title via web UI
- [ ] Manual test: Verify job completes successfully through all stages

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 01:05:57 -05:00
Andrew Beveridge ☄️andClaude 16e8c25d22 fix(email): Fix YouTube URL missing in job completion email (#229)
## Summary
- Fix race condition where YouTube URL was missing from job completion
emails
- Remove '🎤 Nomad Karaoke' header from email templates
- Add custom HTML email signature to job completion and action reminder
emails

## Problem
When a job completed, the automatic completion email was sent with
"[YouTube URL not available]" instead of the actual YouTube link. This
was because:
1. The job transitioned to COMPLETE status
2. During the transition, `_trigger_state_notifications()` was called
3. The completion email was sent using the current job data
4. **BUT** the youtube_url was being stored in state_data AFTER the
transition

## Solution
Move the `update_job()` call that stores `youtube_url`, `dropbox_link`,
and other metadata BEFORE calling `transition_to_state(COMPLETE)`. This
ensures the data is in the database when the completion email is
triggered.

Fixed in both code paths:
- Orchestrated path (`generate_video_orchestrated`)
- Legacy path (`generate_video_legacy`)

## Test plan
- [x] Backend tests pass (1071 passed)
- [ ] Deploy and verify next job completion email contains YouTube URL
- [ ] Verify email signature renders correctly

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-07 04:42:44 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 8e602a8173 feat(frontend): Add encoding worker status to footer (#228)
## Summary

Display encoding worker health and version in the footer alongside
frontend and backend versions, providing visibility into whether the
high-performance encoder is healthy and up-to-date.

## Changes

- **Backend**: Add `GET /health/encoding-worker` lightweight endpoint
for frontend consumption
- **Encoding Worker**: Enhance `/health` endpoint to include
`wheel_version` (karaoke-gen package version)
- **Frontend**: Update `VersionFooter` to fetch and display encoder
status with visual indicator
- **Version**: Bump to 0.95.1

## Footer Display

When healthy:
```
Frontend: v0.95.1 | Backend: v0.95.1 | ● Encoder: v0.95.1
```

With active jobs:
```
Frontend: v0.95.1 | Backend: v0.95.1 | ● Encoder: v0.95.1 (2 active)
```

When offline:
```
Frontend: v0.95.1 | Backend: v0.95.1 | ○ Encoder offline
```

## Testing

- [x] Frontend builds successfully
- [x] Backend tests pass (1071 passed)
- [x] TypeScript compiles without errors in changed files

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 23:18:25 -05:00
Andrew Beveridge ☄️andClaude 2783c2cfcf feat(audio-search): Add display artist/title override for search (#227)
## Summary
- Add optional "Display As" fields to audio search, allowing users to
search for audio by one artist/title but display a different
artist/title on title screens and filenames
- Useful for Broadway casts, covers, remixes where tracker metadata
differs from desired display

## Changes
- Add `display_artist`/`display_title` fields to `AudioSearchRequest`
model
- Update job creation to use display values for output (title screens,
filenames, YouTube, distribution)
- Add "Display As" optional fields to frontend search UI
- Update API documentation

## Example Use Case
- **Search For**: Artist = "Jeremy Kushnier", Title = "I Can't Stand
Still"
- **Display As**: Artist = "Footloose (Broadway Cast)"
- **Result**: Audio found using "Jeremy Kushnier", but title screen
shows "Footloose (Broadway Cast) - I Can't Stand Still"

## Test plan
- [ ] Search with display override → verify title screen shows display
values
- [ ] Search without display override → verify search values used
(backwards compatible)
- [ ] Verify filenames use display values

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-06 22:36:27 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 f6e91b0014 fix(infrastructure): Increase systemd startup timeout for wheel install (#226)
## Summary

Fix encoding worker startup timeout issue discovered after #225
deployment.

## Problem

The ExecStartPre script that installs the karaoke-gen wheel can take 2+
minutes on first boot. The default systemd timeout (90s) caused multiple
retries before eventually succeeding.

## Solution

Add `TimeoutStartSec=300` to the systemd service definition to allow up
to 5 minutes for the startup script to complete.

## Test Plan

- [ ] Merge triggers Packer image rebuild
- [ ] New image has longer timeout configured
- [ ] Next VM recreation starts without timeout retries

🤖 Generated with [Claude Code](https://claude.com/claude-code)

@coderabbitai ignore

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 15:18:57 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 4b58e9c040 feat(infrastructure): Add Packer custom image for encoding worker (#225)
## Summary

- Reduce encoding worker VM startup time from **~10 minutes to ~30
seconds** by pre-installing dependencies in a custom GCE image
- Add Packer template and provisioning script for building the custom
image
- Simplify startup script from ~700 lines to ~30 lines

## Changes

### New Files
- `infrastructure/packer/encoding-worker.pkr.hcl` - Packer template
- `infrastructure/packer/scripts/provision.sh` - Provisioning script
- `infrastructure/packer/README.md` - Documentation

### Modified Files
- `infrastructure/compute/encoding_worker_vm.py` - Use custom image
family `encoding-worker`
- `infrastructure/compute/startup_scripts/encoding_worker.sh` -
Simplified to just start service

## What's Pre-installed in the Image

| Component | Location |
|-----------|----------|
| Python 3.13.1 (PGO+LTO) | `/opt/python313` |
| FFmpeg 7.x | `/usr/local/bin/ffmpeg` |
| Noto fonts (incl. CJK) | System fonts |
| Virtual environment | `/opt/encoding-worker/venv` |
| Systemd service | `encoding-worker.service` |

## Startup Time Improvement

| Phase | Before | After |
|-------|--------|-------|
| Python 3.13 build | ~7 min | 0 |
| FFmpeg install | ~1 min | 0 |
| System packages | ~1 min | 0 |
| API key + wheel | ~30 sec | ~30 sec |
| **Total** | **~10 min** | **~30 sec** |

## Test Plan

- [x] Packer image builds successfully (`encoding-worker-1767723849`)
- [x] `pulumi preview` shows expected changes (VM replacement only)
- [ ] After merge: Verify VM starts in ~30 seconds
- [ ] After merge: Verify encoding jobs complete successfully

## Pulumi Preview

```
+- gcp:compute:Instance encoding-worker replace [diff: ~bootDisk,metadataStartupScript]
Resources: +-1 to replace, 131 unchanged
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

@coderabbitai ignore

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 14:21:56 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 ad37306704 refactor(infrastructure): Modularize Pulumi IaC from 2600-line monolith (#223)
## Summary

- Refactor monolithic `infrastructure/__main__.py` from **2,602 lines to
339 lines** (87% reduction)
- Split resources into organized, purpose-specific modules following
SOLID principles
- Extract VM startup scripts to separate `.sh` files for easier
maintenance
- Extract embedded Python encoding worker app to proper package

## Changes

### New Module Structure
```
infrastructure/
├── __main__.py              # Entry point (339 lines, was 2,602)
├── config.py                # Centralized configuration
├── modules/                 # Core GCP resources
│   ├── database.py          # Firestore + indexes
│   ├── storage.py           # GCS bucket + lifecycle
│   ├── artifact_registry.py # Docker repository
│   ├── secrets.py           # Secret Manager secrets
│   ├── cloud_tasks.py       # Task queues
│   ├── cloud_run.py         # Domain mapping, Cloud Run Jobs
│   ├── monitoring.py        # Alert policies
│   └── iam/                 # Service accounts by purpose
├── compute/                 # VM-based resources
│   ├── flacfetch_vm.py
│   ├── encoding_worker_vm.py
│   ├── github_runners.py
│   └── startup_scripts/     # External .sh files
└── docs/
    └── PHASE5-PACKER-IMAGE-PLAN.md  # Future optimization plan
```

### Extracted Code
- **Startup scripts**: Moved from embedded strings to separate `.sh`
files
- **GCE encoding service**: Extracted ~600-line Python app to
`backend/services/gce_encoding/`

### Documentation
- Updated `infrastructure/README.md` with new structure
- Added `docs/PHASE5-PACKER-IMAGE-PLAN.md` for future Packer image
optimization

## Verification

```bash
pulumi preview --diff
# Result: 120 resources unchanged
# Only 10 GitHub runner replacements (expected - startup script improvements from main)
```

## Test Plan

- [x] `pulumi preview` shows no unintended infrastructure changes
- [x] All module imports resolve correctly
- [x] Startup scripts produce identical output when read from files

## Notes

- Phase 5 (Packer custom images) documented but not implemented -
reduces encoding worker startup from ~10min to ~30sec
- `deploy.sh` for gdrive_validator kept - handles source upload to GCS
which Pulumi doesn't do automatically

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 18:01:07 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 0170e88652 refactor: Unify preview encoding with LocalPreviewEncodingService (#222)
## Summary
- Create `LocalPreviewEncodingService` as single source of truth for
preview video encoding
- Update GCE worker to use the service from the installed wheel (like
`LocalEncodingService`)
- Update `VideoGenerator` to delegate preview encoding to the new
service
- Remove ~140 lines of duplicated FFmpeg logic

## Changes
- **New:** `backend/services/local_preview_encoding_service.py` -
Standalone preview encoding service with NVENC support, FFmpeg filter
path escaping, and font directory support
- **New:** `backend/tests/test_local_preview_encoding_service.py` - 37
comprehensive tests
- **Modified:** `infrastructure/__main__.py` - GCE worker now uses
`LocalPreviewEncodingService` from wheel
- **Modified:**
`lyrics_transcriber_temp/lyrics_transcriber/output/video.py` - Delegates
to `LocalPreviewEncodingService`, removed duplicate
`_build_preview_ffmpeg_command`
- **Modified:** `docs/README.md`, `docs/LESSONS-LEARNED.md` -
Documentation updates

## Why This Matters
Previously, preview encoding had THREE implementations:
1. `VideoGenerator._build_preview_ffmpeg_command` (local CLI)
2. `run_preview_encoding` in GCE worker (hardcoded settings)
3. ~~Inline FFmpeg in old code~~ (already removed)

This created risk of inconsistent preview videos between environments.
Now there's ONE source of truth.

## Testing
- [x] 37 new tests for `LocalPreviewEncodingService` pass
- [x] 28 existing `LocalEncodingService` tests pass
- [x] 22 `VideoGenerator` tests pass  
- [x] Full backend test suite: 1042 passed

## Review
- [x] CodeRabbit CLI attempted (rate limited, tests verified instead)
- [x] Documentation reviewed and updated

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 12:50:17 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 4ad8018773 infra: Grant Editor role to GitHub Actions service account (#224)
## Summary
- Adds `roles/editor` to the GitHub Actions deployer service account
- Prevents permission errors when adding new GCP resources to
infrastructure
- Fixes the failed workflow that couldn't create `idle-reminder-queue`
due to missing `cloudtasks.queues.create` permission

## Test plan
- [x] Manually bootstrapped the permission via `gcloud` CLI
- [ ] Verify CI passes after merge (the Editor role is already applied)

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 12:06:42 -05:00
Andrew Beveridge ☄️andClaude 560c3e4904 feat: Add email notification system with admin controls (#220)
## Summary

- Add automated email notification system for job completion and action
reminders
- Admin UI buttons to copy completion message or send email directly
- Comprehensive test suite (81 new tests)
- **Email subjects include release ID and song info** for easy
identification

## Changes

### Backend
- **Email Service** (`email_service.py`): Extended with SendGrid CC
support, `send_job_completion()` and `send_action_reminder()` methods
- **Job Notification Service** (`job_notification_service.py`): New
service orchestrating completion emails and action reminder scheduling
- **Template Service** (`template_service.py`): GCS-backed HTML email
templates with fallback defaults
- **Worker Service** (`worker_service.py`): Cloud Tasks integration for
delayed idle reminders (5-min delay)
- **Job Manager** (`job_manager.py`): State transition triggers for
automated notifications
- **Admin Routes** (`admin.py`): New endpoints `GET
/api/admin/jobs/{id}/completion-message` and `POST
/api/admin/jobs/{id}/send-completion-email`
- **Internal Routes** (`internal.py`): Idle reminder check endpoint for
Cloud Tasks

### Email Subject Format
Email subjects now include brand code and song info for easy
identification:
- With brand code: `NOMAD-1178: Seether - Tonight (Your karaoke video is
ready!)`
- Without brand code: `Seether - Tonight (Your karaoke video is ready!)`
- Without song info: `Your karaoke video is ready!`

### Frontend
- **OutputLinks Component**: Admin-only "Copy Message" and "Send Email"
buttons with dialog UI
- **E2E Helpers** (`email-testing.ts`): `waitForCompletionEmail()` and
`isCompletionEmail()` methods for testmail.app integration
- **Happy Path Test**: Step 9.5 verifies completion email arrives after
job completion

### Documentation
- Updated API.md with new endpoints
- Updated ARCHITECTURE.md with notification system architecture
- Updated README.md Recent Changes

## Testing

- [x] 81 new unit tests covering:
- `test_email_service.py` (27 tests) - providers, CC support, job
completion/reminder methods
- `test_job_notification_service.py` (21 tests) - URL building, message
generation
- `test_job_manager_notifications.py` (15 tests) - state transition
triggers
  - `test_admin_email_endpoints.py` (12 tests) - admin API endpoints
- `test_template_service.py` (6 tests) - GCS template loading, fallbacks
- [x] E2E test extended to verify completion email delivery
- [x] All backend tests pass (`make test`)
- [x] TypeScript compiles

## Feature Flags

- `ENABLE_AUTO_EMAILS` (default: `false`) - Controls automatic email
sending on job state changes

## Review

- [x] Local CodeRabbit review completed (3 cycles)
- [x] Review feedback addressed:
  - Fixed potential None state_data in internal.py
  - Masked user emails in logs (PII protection)
  - Fixed React setTimeout memory leak
  - Changed placeholder URL default to empty
  - Added CC email deduplication

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-06 11:53:11 -05:00
Andrew Beveridge ☄️andClaude 07cde54b3f feat: Upgrade GCE encoding worker to Python 3.13 (#221)
## Summary

Upgrades the GCE encoding worker VM from Debian's default Python 3.11 to
Python 3.13 built from source, aligning it with CI and Cloud Run Python
version.

## Changes

- Build Python 3.13.1 from source with `--enable-optimizations
--with-lto` for best performance
- Install to `/opt/python313` on the Debian 12 VM
- Create virtual environment using Python 3.13 instead of system Python
3.11
- Add build dependencies (build-essential, libssl-dev, zlib1g-dev, etc.)
- Remove `python3-pip` and `python3-venv` apt packages (no longer
needed)
- Update docs/README.md with recent changes entry

## Testing

- [x] Pulumi preview shows only encoding-worker VM replacement
(expected)
- [x] Backend tests pass (`make test` - 1006 passed)
- [x] Frontend tests pass (`npm run test:all` - 65 passed)
- [x] Python and bash syntax validation pass

## Review

- [x] Local CodeRabbit review completed
- [x] No actionable issues found

## Notes

- The VM will be **replaced** (not updated in-place) when this deploys
- Python build from source takes ~5-10 minutes during VM startup
- Brief encoding downtime expected during deployment

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-06 10:43:47 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 bd26d742a2 fix: Include backend package in wheel for GCE worker LocalEncodingService (#219)
## Summary

- Include `backend` package in pyproject.toml packages list for wheel
distribution
- This enables the GCE encoding worker to import `LocalEncodingService`

## Problem

The GCE encoding worker failed with:
```
ERROR:__main__:LocalEncodingService not available: No module named 'backend'
```

The wheel was being installed correctly, but `backend/` wasn't included
in the package.

## Changes

- Added `{ include = "backend" }` to packages list in pyproject.toml
- Bumped version to 0.93.0

## Test Plan

- [x] Verify pyproject.toml changes are correct
- [ ] CI builds wheel with backend included
- [ ] GCE worker downloads new wheel at job start
- [ ] E2E happy path test passes

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 09:58:39 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 f308a7231c fix: Upgrade github-runner storage role to objectAdmin (#218)
## Summary

Wheel upload needs to overwrite existing files which requires
`storage.objects.delete` permission. Changed from `objectCreator` to
`objectAdmin` role.

## Previous Error
```
AccessDeniedException: 403 github-runner@nomadkaraoke.iam.gserviceaccount.com does not have storage.objects.delete access
```

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 09:09:04 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 53ade5b9dc fix: Pre-install wheel at VM startup and increase install timeout (#217)
## Summary

Fixes GCE encoding worker failing because wheel installation timed out
after 2 minutes.

**Root cause**: The karaoke-gen wheel (4.4MB) has many dependencies
(pytorch, langchain, etc.) that take several minutes to resolve and
install.

## Changes

1. **Pre-install wheel during VM startup** - Added to bash startup
script so wheel is installed before the service starts
2. **Increased runtime install timeout** - From 2 to 5 minutes for hot
code updates at job start

## Why This Works

- VM startup: Full install with all dependencies (takes ~3-4 minutes,
but that's OK during boot)
- Job start: Quick check for newer wheel, reinstall only if needed
(usually fast since deps cached)

## Test Plan

- [x] Syntax check passes
- [ ] CI passes
- [ ] After merge + Pulumi deploy: Restart GCE worker, trigger E2E happy
path test

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 09:03:20 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 c3b8cf880d fix: CI failures - escape triple quotes and add storage permission (#216)
## Summary

Fixes two CI failures from PR #214 merge:

1. **SyntaxError in Pulumi deployment**: The embedded Python code in the
GCE startup script used `"""` docstrings which prematurely terminated
the outer triple-quoted string. Changed to `'''` single-quoted
docstrings.

2. **Storage permission denied**: `github-runner` service account lacked
`storage.objects.create` permission to upload wheels to GCS. Added
`roles/storage.objectCreator` IAM binding.

## Changes

- Changed 3 docstrings in embedded Python code from `"""` to `'''`
- Added `github_runner_storage_iam` IAM binding with
`roles/storage.objectCreator`

## Test Plan

- [x] `python3 -m py_compile infrastructure/__main__.py` passes
- [x] `make test` passes locally
- [ ] CI should pass with these fixes

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 01:39:30 -05:00
Andrew Beveridge ☄️ 4ebf811c3f Update pyproject.toml (#215) 2026-01-06 06:26:32 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 b345135be1 feat: Unify GCE encoding with LocalEncodingService (single code path) (#214)
## Summary
- Unifies GCE encoding worker with LocalEncodingService (same code as
local CLI)
- Removes all fallback/simplified encoding logic
- Produces proper filenames: `Artist - Title (Final Karaoke Lossless
4k).mp4`
- Includes concatenated title + karaoke + end screens
- CI uploads wheel to GCS for hot code updates without restart

## Problem
Cloud/remote karaoke generation was using simplified, duplicated
encoding logic that:
- Used generic filenames like `output_4k_lossless.mp4`
- Did NOT concatenate title + karaoke + end screens
- Was missing MKV format for YouTube uploads

## Solution
GCE worker now:
1. Downloads latest karaoke-gen wheel from GCS at job start
2. Imports and uses `LocalEncodingService` (same as local CLI)
3. Produces identical output to local karaoke-gen

## Test plan
- [ ] Run a cloud generation job after merging
- [ ] Verify output files have proper names
- [ ] Verify videos include title/end screens
- [ ] Verify Dropbox/GDrive uploads work correctly

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 01:25:45 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 1114bd5554 fix: Use FFmpeg filter escaping instead of shell escaping for paths (#213)
## Summary
- Fixes video generation for songs with apostrophes in titles (e.g.,
"I'm With You")
- PR #212 introduced shell-style escaping (`'\''`) which doesn't work
with subprocess command lists
- Changed to FFmpeg's native filter escaping rules

## Test plan
- [x] All 22 video.py unit tests pass
- [x] All 68 backend tests pass  
- [x] End-to-end FFmpeg test with apostrophe in path succeeds
- [ ] Manual test with "Avril Lavigne - I'm With You"

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 23:45:27 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 60af90de31 fix: Escape FFmpeg filter paths using single-quote wrapping (#212)
## Summary
- Fix FFmpeg failing on paths with apostrophes (e.g., "I'm With You")
- Use single-quote wrapping with `'\''` pattern for embedded quotes
- Update both `video.py` and `video_background_processor.py`

## Test plan
- [x] Unit tests updated and passing (1006 tests)
- [x] E2E tests passing (68 tests)
- [x] Manually tested with "Avril Lavigne - I'm With You" - video
generates successfully

## Technical details
FFmpeg filter syntax interprets apostrophes as special characters.
Simple backslash escaping (`\'`) doesn't work because FFmpeg's filter
parser consumes the backslash. The solution is to wrap paths in single
quotes and escape embedded quotes using the `'\''` pattern (end quote,
literal quote, start quote).

Example: `I'm With You` becomes `'I'\''m With You'`

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 03:50:24 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 2110d0f84b perf: Optimize agentic correction with parallel gap processing (~5-6x speedup) (#211)
## Summary

- Fix anti-pattern of creating new LLM model instance per gap (caused
repeated 2s warm-up overhead)
- Create agentic corrector once and reuse for all gaps
- Process gaps in parallel using ThreadPoolExecutor (default 5 workers)
- Remove unnecessary warm-up code from langchain_bridge.py
- Add `AGENTIC_MAX_PARALLEL_GAPS` env var to configure parallelism

**Performance improvement**: ~5-6x speedup (20 gaps: ~55s vs ~5+ minutes
previously)

## Test plan

- [x] Ran local test with 20 gaps - completed in ~55 seconds
- [x] Verified parallel processing logs show concurrent gap processing
- [x] Verified model initialization only happens once at start
- [x] CodeRabbit review passed locally

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 22:43:13 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 4b0dd6169f ci: Propagate base image to all runners after rebuild (#210)
## Summary
- Adds new `propagate-base-image` job that runs after base image rebuild
- Spawns 20 parallel jobs to pull the new base image to all self-hosted
runners
- Eliminates ~8 minute pull delay on subsequent app builds

## Problem
When the base image is rebuilt (due to `poetry.lock` or
`Dockerfile.base` changes), other runners still have the old image
cached. The first app build on each runner must pull the new ~5GB image,
adding ~8 minutes to deployment time.

Example:
https://github.com/nomadkaraoke/karaoke-gen/actions/runs/20735879195/job/59532914508

## Solution
After base image is pushed, trigger 20 parallel jobs that pull the new
image to all runners immediately. Future app builds will already have
the latest base image cached.

## Implementation
- Added `base_rebuilt` output to `deploy-backend` job
- New `propagate-base-image` job with matrix strategy (20 parallel jobs)
- `fail-fast: false` so one offline runner doesn't block others
- Uses same Workload Identity auth as other deployment jobs

## Test plan
- [ ] Verify workflow syntax is valid (YAML parses correctly)
- [ ] Merge and observe next base image rebuild triggers propagation
- [ ] Confirm subsequent app builds don't need to pull base image

<!-- @coderabbitai ignore -->

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 21:54:55 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 8e3e3e4713 fix: Check base image hash from registry instead of stale local cache (#209)
## Summary
- Fix GitHub Actions base image cache check to query registry directly
instead of trusting potentially stale local Docker cache
- Prevent unnecessary base image rebuilds caused by runners with
outdated cached images
- Ensure app image is always built on the correct base image

## Changes
- Fetch image hash label via lightweight registry API (~10KB config
blob) instead of pulling full image
- Add step to verify local cache matches registry before building app
image
- Compare config digests consistently for reliable image matching

## Problem
GitHub Actions runners can retain Docker images between runs. When the
runner's local cache has an old base image, the previous logic would:
1. Check local cache first → find old image with wrong hash
2. Conclude base image needs rebuild → unnecessary ~10 min rebuild
3. Or worse: build app image on stale base if hashes happened to match
older state

## Testing
- [x] Verified registry API returns correct hash label
- [x] CodeRabbit CLI review completed locally

## Review
- [x] Local CodeRabbit review completed
- [x] Review feedback addressed (fixed digest comparison types)

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 21:27:02 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 fef9a45739 docs: Add Pulumi CI skip-preview lesson learned (#208)
## Summary
- Add Pulumi CI skip-preview lesson to `LESSONS-LEARNED.md`
- Update `CLAUDE.md` to emphasize infrastructure changes require PRs
- Add detailed Infrastructure section to `DEVELOPMENT.md` documenting:
  - What's managed by Pulumi
  - The PR-based workflow
  - Why not to run `pulumi up` locally

## Changes
| File | Change |
|------|--------|
| `docs/LESSONS-LEARNED.md` | Added "Pulumi CI Deployments Need
Skip-Preview" section |
| `CLAUDE.md` | Strengthened Infrastructure section with PR workflow |
| `docs/DEVELOPMENT.md` | Added comprehensive Infrastructure Changes
section |

## Test plan
- [x] Documentation only - no code changes

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 21:09:45 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 69b87c232a fix: Add IAM permissions for Pulumi to refresh state in CI (#207)
## Summary
- Add `roles/resourcemanager.projectIamAdmin` for IAM binding state
refresh
- Add `roles/secretmanager.admin` for secret state refresh
- Use `--skip-preview` instead of `--refresh` to avoid needing read
permissions on all GCP resource types

The service account would need many additional roles (Cloud Tasks,
Firestore, Monitoring, etc.) for Pulumi to refresh all resources. Using
`--skip-preview` is the standard approach for CI/CD where Pulumi state
is authoritative.

## Test plan
- [ ] CI passes on this PR
- [ ] After merge, Pulumi deployment succeeds

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 18:57:10 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 0530e5f226 fix: Use correct Pulumi stack name in CI infrastructure deploy (#206)
## Summary
- Fixed Pulumi stack reference from `nomadkaraoke/karaoke-gen/prod` to
`nomadkaraoke/karaoke-gen-infrastructure/prod` (matches project name in
`Pulumi.yaml`)
- Added GCP Workload Identity authentication for Pulumi deployment step

## Test plan
- [ ] CI passes on this PR
- [ ] After merge, infrastructure deployment job succeeds

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 15:34:00 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 d8710bc3d4 ci: Add infrastructure deployment job for GCE encoding worker (#205)
## Summary
Adds a Pulumi deployment job that runs on main branch pushes to
auto-deploy infrastructure changes.

## Problem
Infrastructure changes (like GCE encoding worker code) weren't being
deployed automatically. This caused issues where the backend was
deployed with new code but the GCE worker still had old code.

## Solution
New `deploy-infrastructure` job that:
- Runs Pulumi to deploy infrastructure changes
- Restarts the encoding worker service to pick up new code

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 15:07:41 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 c83ac4376c fix: Handle cached preview responses from GCE worker (#204)
## Summary
- Fixes issue where subsequent preview requests hang indefinitely
- GCE worker now returns "cached" status for already-encoded previews
instead of 409 Conflict
- Backend handles cached response by returning immediately without
polling

## Problem
When users clicked preview multiple times without changing lyrics:
1. First preview worked correctly
2. Subsequent previews hung forever because GCE returned 409 Conflict
3. Backend raised an error instead of handling the cached case

## Solution
- GCE `/encode-preview` endpoint now returns `{"status": "cached", ...}`
for completed jobs
- Backend `encode_preview_video()` checks submit response and returns
immediately for cached
- Also handles "in_progress" status when another request is already
encoding

## Test plan
- [x] First preview generates correctly
- [ ] Subsequent previews (unchanged lyrics) return cached video
instantly
- [ ] Changed lyrics generate new preview video

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 19:47:35 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 e70ce2b558 chore: Enable USE_GCE_PREVIEW_ENCODING in production CI (#203)
## Summary
Enable the GCE preview encoding feature in production CI deployments.

## Changes
- Add `USE_GCE_PREVIEW_ENCODING=true` to Cloud Run environment variables
in CI deploy

## Note
To enable immediately (before next deploy), run:
```bash
gcloud run services update karaoke-backend \
  --region us-central1 \
  --project nomadkaraoke \
  --update-env-vars "USE_GCE_PREVIEW_ENCODING=true"
```

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 18:17:40 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 506db844c2 fix: Double-escape backslashes in GCE embedded Python code (#202)
## Summary
Fix syntax error in GCE encoding worker startup script caused by
incorrect backslash escaping.

## Problem
The `escape_ffmpeg_filter_path` function in the GCE worker startup
script had a syntax error:
```
SyntaxError: unexpected character after line continuation character
```

The issue was that backslashes in the embedded Python code weren't
properly escaped for the triple-quoted string context in Pulumi.

## Solution
Double the backslashes in the Pulumi source:
- 4 backslashes in source → 2 in string value → 2 on VM (valid Python)

## Testing
- GCE encoding worker now starts successfully
- Health endpoint responds correctly
- Already deployed via `pulumi up` and verified working

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 02:03:33 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 be04d3efaf feat: Offload preview video encoding to GCE for faster generation (#200)
## Summary
- Move preview video encoding from Cloud Run to GCE encoding worker
- Reduce preview generation time from 60+ seconds to ~15-20 seconds
- Improve UX during lyrics review by dramatically speeding up preview
generation

## Changes
- **GCE Worker** (`infrastructure/__main__.py`): Add `/encode-preview`
endpoint with `EncodePreviewRequest` model, FFmpeg command for 480x270
preview encoding, and FFmpeg filter path escaping
- **EncodingService** (`backend/services/encoding_service.py`): Add
`is_preview_enabled` property, `submit_preview_encoding_job()`, and
`encode_preview_video()` methods
- **Config** (`backend/config.py`): Add `USE_GCE_PREVIEW_ENCODING`
setting (default: false)
- **Review Route** (`backend/api/routes/review.py`): Modify
`generate_preview_video()` to use GCE with fallback to local encoding
- **LyricsTranscriber** (`operations.py`, `generator.py`): Add
`ass_only` parameter to generate ASS subtitles without video encoding
- **Documentation**: Update ARCHITECTURE.md and LESSONS-LEARNED.md

## Technical Approach
The optimization splits preview generation into two phases:
1. **ASS Generation (Cloud Run)** - Fast text processing (~2s)
2. **Video Encoding (GCE)** - Offloaded to C4-standard-8 VM (~15-20s)

If GCE encoding fails or is disabled, falls back to local Cloud Run
encoding.

## Testing
- [x] Unit tests pass (1004 tests)
- [x] Emulator tests pass (68 tests)
- [x] CodeRabbit CLI review completed locally

## Deployment
1. Deploy infrastructure changes (`pulumi up` or restart GCE worker)
2. Deploy backend to Cloud Run
3. Enable via `USE_GCE_PREVIEW_ENCODING=true` in Cloud Run env

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 01:04:35 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 f2639754a9 fix: Download GCE-encoded files locally for YouTube upload (#201)
## Summary
- Fix YouTube upload failing when using GCE encoding backend
- GCE encoding returns GCS blob paths, but YouTube upload expected local
files
- Added `_download_gce_encoded_files()` to download encoded videos from
GCS before distribution

## Root Cause
Job `62a95e01` completed but didn't upload to YouTube. Investigation
showed:
1. GCE encoding stores files in GCS and returns blob paths like
`jobs/{id}/finals/output_4k_lossless.mp4`
2. The orchestrator stored these paths directly without downloading
3. YouTube upload checks `os.path.isfile()` which returns `False` for
GCS paths
4. Result: "No video file available for YouTube upload" warning

## Changes
| File | Change |
|------|--------|
| `backend/workers/video_worker_orchestrator.py` | Added
`_download_gce_encoded_files()` method called after GCE encoding |
| `backend/tests/test_video_worker_orchestrator.py` | Added 2 tests for
GCE download behavior |
| `docs/LESSONS-LEARNED.md` | Documented the issue and fix |

## Test plan
- [x] All 30 orchestrator tests pass
- [x] New test `test_run_encoding_gce_downloads_files` verifies GCE
paths are downloaded
- [x] New test `test_run_encoding_local_does_not_download` verifies
local encoding skips download
- [ ] Manual test: Create job via web UI and verify YouTube upload works

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 05:51:52 +00:00
Andrew Beveridge ☄️ 94182277fa Update pyproject.toml (#199) 2026-01-05 04:14:37 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 e16da0fa5f fix: Allow explicit override of YouTube upload server default (#198)
## Summary
- Changed `enable_youtube_upload` from `bool=False` to
`Optional[bool]=None` in request models
- Allows explicit `false` to override server default when
`DEFAULT_ENABLE_YOUTUBE_UPLOAD=true`
- E2E tests now intercept requests to disable YouTube upload for test
jobs

## Changes
- **Request models**: Changed to `Optional[bool] = None` to support
three states (None/True/False)
- **API routes**: All job creation endpoints now compute effective value
using `if is not None else` pattern
- **E2E test**: Added Playwright request interception to set
`enable_youtube_upload=false` for test jobs
- **Tests**: Updated to expect `None` instead of `False` for request
model defaults
- **Docs**: Added lesson learned about Python `or` operator gotcha with
boolean defaults

## Testing
- [x] Backend tests pass (`make test`)
- [x] No TypeScript errors in frontend changes

## Review
- [x] CodeRabbit CLI review completed locally
- [x] No issues found

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 23:04:06 -05:00
Andrew Beveridge ☄️andClaude b8e15b2af5 fix: Ensure user data is fetched after client-side navigation to /app (#197)
## Summary

- Fix E2E happy path test failure at Step 3 (Verify Credits)
- After beta enrollment, client-side navigation to `/app` didn't fetch
user data
- Credits weren't displayed because zustand auth store had `user: null`

## Changes

- **frontend/app/app/page.tsx**: Added `useEffect` that calls
`fetchUser()` when there's a token but no user data loaded. This ensures
user data is fetched regardless of how the user navigated to the page
(page reload vs client-side navigation).

- **frontend/e2e/regression/authentication.spec.ts**: Added new test
`'beta enrollment shows credits after redirect to app'` that
specifically verifies credits are displayed after the beta enrollment
flow.

- **docs/LESSONS-LEARNED.md**: Documented the root cause (zustand
module-level initialization only runs once, not on client-side
navigation) for future reference.

## Root Cause

The auth module's initialization code (`fetchUser()` call) only runs
once when the module is first imported. After beta enrollment,
`router.push('/app')` performs client-side navigation which doesn't
reload modules. The user data was never fetched, leaving `user: null` in
the store.

## Testing

- [x] New regression test passes locally (9/9 auth tests pass)
- [x] Build succeeds
- [x] CodeRabbit CLI review completed locally

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-04 21:33:01 -05:00
Andrew Beveridge ☄️ 2ee3cabc92 Added admin API script (#196) 2026-01-05 01:33:17 +00:00
231 changed files with 36221 additions and 9813 deletions

No files matched your search

@@ -0,0 +1,218 @@
# Plan: Job Email Notifications & Admin Message Tools
**Created:** 2026-01-06
**Branch:** feat/sess-20260106-0116-add-email-button
**Status:** Draft
## Overview
Add comprehensive email notification system for karaoke-gen that:
1. **Auto-sends emails** when jobs complete or need user input (after 5-minute idle threshold)
2. **Uses cloud-stored templates** for the job completion message
3. **Provides admin tools** to copy/send completion messages manually (for Fiverr, etc.)
## Requirements
### Functional Requirements
- [ ] Auto-send job completion email to user (CC: gen@nomadkaraoke.com)
- [ ] Auto-send "action needed" reminder emails when user is idle for 5+ minutes at blocking states (lyrics review, instrumental selection)
- [ ] Store email templates in GCS (editable without code deploy)
- [ ] Admin button: "Copy Message" - copies templated completion message to clipboard
- [ ] Admin button: "Send Email" - prompts for email address, then sends completion message
### Non-Functional Requirements
- [ ] Email sending must not block job completion (async/background)
- [ ] Idle detection via Cloud Tasks scheduled task (not polling)
- [ ] Template changes should take effect without restart (no long-term caching)
## Technical Approach
### 1. Template Storage (GCS)
Store templates in `gs://karaoke-gen-storage/templates/`:
- `job-completion.txt` - Plain text template for job completion
- `job-completion.html` - HTML version (optional, for rich emails)
- `action-needed-lyrics.txt` - Reminder for lyrics review
- `action-needed-instrumental.txt` - Reminder for instrumental selection
**Template Variables:**
- `{name}` - User's display name or "there" if unknown
- `{youtube_url}` - YouTube video URL
- `{dropbox_url}` - Dropbox folder URL
- `{artist}` - Artist name
- `{title}` - Song title
- `{job_id}` - Job ID
- `{review_url}` - Lyrics review URL (for action-needed emails)
- `{instrumental_url}` - Instrumental selection URL
- `{feedback_url}` - Feedback form URL (for completion emails)
### 2. Email Service Extensions
Extend `backend/services/email_service.py`:
- Add `send_job_completion()` method with CC support
- Add `send_action_reminder()` method
- Add `render_template()` helper to fetch & render GCS templates
### 3. Auto-Email Triggers
**Job Completion:**
- Trigger point: When job enters `COMPLETE` status in `job_manager.transition_to_state()`
- Implementation: Call email service async in background task
**Action Reminders (5-min idle detection):**
- When job enters `AWAITING_REVIEW` or `AWAITING_INSTRUMENTAL_SELECTION`:
- Record `blocking_state_entered_at` timestamp in `state_data`
- Schedule a Cloud Tasks task for 5 minutes later
- The scheduled task checks if job is still in blocking state and user hasn't started review
- If idle, send reminder email and set `reminder_sent` flag to prevent duplicates
- **Only one reminder per job** - if `reminder_sent` is true, skip sending
### 4. Admin UI (Frontend)
Add admin-only buttons to completed jobs in `OutputLinks.tsx`:
- "Copy Message" button - fetches rendered template, copies to clipboard
- "Send Email" button - opens dialog for email input, calls API to send
### 5. Backend API Endpoints
New endpoints in `backend/api/routes/admin.py`:
- `GET /api/admin/jobs/{job_id}/completion-message` - Returns rendered completion message
- `POST /api/admin/jobs/{job_id}/send-completion-email` - Sends email to specified address
New internal endpoint for scheduled reminder:
- `POST /api/internal/jobs/{job_id}/check-idle-reminder` - Called by Cloud Tasks
## Implementation Steps
### Phase 1: Template Infrastructure
1. [ ] Upload initial template to GCS from user's existing template file
2. [ ] Create `template_service.py` to fetch/render templates from GCS
3. [ ] Add unit tests for template rendering
### Phase 2: Email Extensions
4. [ ] Extend `email_service.py` with CC support in `send_email()`
5. [ ] Add `send_job_completion()` method
6. [ ] Add `send_action_reminder()` method
7. [ ] Add unit tests for new email methods
### Phase 3: Auto-Email on Completion
8. [ ] Modify `job_manager.py` to trigger completion email when entering COMPLETE
9. [ ] Add integration test for completion email trigger
### Phase 4: Idle Reminder System
10. [ ] Add `blocking_state_entered_at` field tracking to job state transitions
11. [ ] Create Cloud Tasks queue for idle reminders (`idle-reminder-queue`)
12. [ ] Schedule 5-minute delayed task when entering blocking states
13. [ ] Implement `/api/internal/jobs/{job_id}/check-idle-reminder` endpoint
14. [ ] Add tests for idle reminder flow
### Phase 5: Admin API
15. [ ] Add `GET /api/admin/jobs/{job_id}/completion-message` endpoint
16. [ ] Add `POST /api/admin/jobs/{job_id}/send-completion-email` endpoint
17. [ ] Add tests for admin endpoints
### Phase 6: Admin UI
18. [ ] Add `isAdmin` check to `OutputLinks.tsx` component
19. [ ] Add "Copy Message" button (admin-only) with loading state
20. [ ] Add "Send Email" button (admin-only) with email input dialog
21. [ ] Add E2E test for admin buttons
### Phase 7: Documentation
22. [ ] Document template management in docs/DEVELOPMENT.md
23. [ ] Update docs/API.md with new endpoints
## Files to Create/Modify
| File | Action | Description |
|------|--------|-------------|
| `backend/services/template_service.py` | Create | GCS template fetching and rendering |
| `backend/services/email_service.py` | Modify | Add CC support, completion/reminder methods |
| `backend/services/job_manager.py` | Modify | Trigger emails on state transitions |
| `backend/services/worker_service.py` | Modify | Add idle reminder task scheduling |
| `backend/api/routes/admin.py` | Modify | Add completion message/email endpoints |
| `backend/api/routes/internal.py` | Modify | Add idle reminder check endpoint |
| `backend/models/job.py` | Modify | Add `blocking_state_entered_at`, `reminder_sent_at` to state_data docs |
| `infrastructure/__main__.py` | Modify | Add idle-reminder-queue Cloud Tasks queue |
| `frontend/components/job/OutputLinks.tsx` | Modify | Add admin-only Copy/Send buttons |
| `frontend/lib/api.ts` | Modify | Add API methods for admin endpoints |
| `tests/unit/test_template_service.py` | Create | Template rendering tests |
| `tests/unit/test_email_service_completion.py` | Create | Completion email tests |
## Testing Strategy
**Unit Tests:**
- Template rendering with all variable combinations
- Email service CC support
- Completion/reminder email generation
- Idle detection logic
**Integration Tests:**
- Full flow: job completion → email sent
- Idle reminder: job enters blocking state → 5 min → reminder sent
- Admin endpoint authentication
**Manual Testing:**
- Verify emails render correctly in Gmail/Outlook
- Test admin buttons on completed job
- Test clipboard copy works across browsers
## Open Questions
- [x] Where to store templates? → **GCS bucket: `gs://karaoke-gen-storage/templates/`**
- [x] Should we rate-limit reminder emails? → **Max 1 reminder per job** (send once after 5 min idle, then stop)
- [x] Should completion emails also include a feedback request link? → **Yes, for all users**
## Rollback Plan
1. **Email sending failures** - All email sends are async/fire-and-forget; job completion unaffected
2. **Template fetch failures** - Fall back to hardcoded template in code
3. **Idle reminder failures** - Jobs continue normally; users can still access review UI manually
4. **Feature flag** - Add `ENABLE_AUTO_EMAILS=true/false` env var for quick disable
## Template Content
Initial template (from user's file, with feedback link added):
```
Hi {name},
Thanks for your order!
Here's the link for the karaoke video published to YouTube:
{youtube_url}
Here's the dropbox folder with all the finished files and source files, including:
- "(Final Karaoke Lossless).mkv": combined karaoke video in 4k H264 with lossless FLAC audio
- "(Final Karaoke).mp4": combined karaoke video with title/end screen in 4k H264/AAC
- "(Final Karaoke 720p).mp4": combined karaoke video in 720p H264/AAC (smaller file for older systems)
- "(With Vocals).mp4": sing along video in 4k H264/AAC with original vocals
- "(Karaoke).mov": karaoke video output from MidiCo (no title/end screen)
- "(Title).mov"/"(End).mov": title card and end screen videos
- "(Final Karaoke CDG).zip": CDG+MP3 format for older/commercial karaoke systems
- "(Final Karaoke TXT).zip": TXT+MP3 format for Power Karaoke
- stems/*.flac: various separated instrumental and vocal audio stems in lossless format
- lyrics/*.txt song lyrics from various sources in plain text format
{dropbox_url}
Let me know if anything isn't perfect and I'll happily tweak / fix, or if you need it in any other format I can probably convert it for you!
If you have a moment, I'd really appreciate your feedback (takes 2 minutes):
{feedback_url}
Thanks again and have a great day!
-Andrew
```
**Note:** The `{feedback_url}` placeholder will be populated with the feedback form URL. If no feedback URL is configured, this section can be omitted from the rendered output.
## Updating Templates
To update email templates after deployment:
```bash
# Upload new template
gsutil cp my-template.txt gs://karaoke-gen-storage/templates/job-completion.txt
# Templates are fetched fresh on each use (no caching in code)
# Changes take effect immediately
```
+317
View File
@@ -0,0 +1,317 @@
# Plan: White Label B2B Karaoke Portals
**Created:** 2026-01-08
**Branch:** feat/sess-20260108-0358-white-label-portal
**Status:** Draft
## Overview
Add multi-tenant white label portal capability to karaoke-gen, enabling B2B customers like Vocal Star to access a customized version of the generator app on their own subdomain with their branding, theme, and restricted feature set.
**First customer: Vocal Star**
- Subdomain: `vocalstar.nomadkaraoke.com`
- Workflow: Upload original + instrumental audio (no audio search)
- Theme: Custom Vocal Star karaoke theme (already exists as JSON)
- Distribution: Download only (no Dropbox/GDrive/YouTube)
- Auth: Magic link for `@vocal-star.com` / `@vocalstarmusic.com` domains, plus fixed API token option
## Requirements
### Functional Requirements
- [ ] Tenant identification via subdomain (e.g., `vocalstar.nomadkaraoke.com`)
- [ ] Per-tenant frontend branding (logo, colors, metadata)
- [ ] Per-tenant feature flags (hide audio search, hide distribution options)
- [ ] Per-tenant default theme (always use Vocal Star theme)
- [ ] Per-tenant allowed email domains for magic link auth
- [ ] Per-tenant fixed API token support (for automation)
- [ ] Jobs scoped to tenant (Vocal Star users only see Vocal Star jobs)
- [ ] Users scoped to tenant
### Non-Functional Requirements
- [ ] Minimal code duplication - single codebase serves all tenants
- [ ] Easy to add new tenants via config, not code changes
- [ ] Graceful fallback - default to Nomad Karaoke if no tenant match
- [ ] No performance impact on existing single-tenant flow
## Technical Approach
### Architecture Decision: Config-Driven Multi-Tenancy
Rather than building a full tenant management system (overkill for ~1 customer/year), we'll use a **config-driven approach**:
1. **Tenant configs stored in GCS** at `tenants/{tenant_id}/config.json`
2. **Frontend detects subdomain** and fetches tenant config from API
3. **Backend middleware** extracts tenant from request and applies config
4. **Firestore collections** get `tenant_id` field for isolation
This approach:
- Requires no database migrations for new tenants
- Configs can be managed via simple JSON files
- Easy to test locally with query param override (`?tenant=vocalstar`)
- Can evolve to admin UI later if needed
### Tenant Config Schema
```json
{
"id": "vocalstar",
"name": "Vocal Star",
"subdomain": "vocalstar.nomadkaraoke.com",
"branding": {
"logo_url": "gs://karaoke-gen-storage/tenants/vocalstar/logo.png",
"primary_color": "#ffff00",
"secondary_color": "#006CF9",
"favicon_url": "gs://karaoke-gen-storage/tenants/vocalstar/favicon.ico",
"site_title": "Vocal Star Karaoke Generator"
},
"features": {
"audio_search": false,
"file_upload": true,
"youtube_upload": false,
"dropbox_upload": false,
"gdrive_upload": false,
"theme_selection": false
},
"defaults": {
"theme_id": "vocalstar",
"distribution_mode": "download_only"
},
"auth": {
"allowed_email_domains": ["vocal-star.com", "vocalstarmusic.com"],
"fixed_tokens": ["<will be in Secret Manager>"]
}
}
```
### Key Implementation Details
#### 1. Subdomain Detection (Frontend)
- Next.js middleware checks `request.headers.host`
- If subdomain matches tenant, store tenant ID in cookie/context
- Fetch `/api/tenant/config` to get branding/features
- Apply CSS variables dynamically for colors
- Swap logo component based on config
#### 2. Tenant Middleware (Backend)
- FastAPI middleware extracts tenant from:
- `X-Tenant-ID` header (set by frontend)
- Or subdomain from `Host` header
- Or query param `?tenant=` (dev only)
- Attaches `tenant_id` to request state
- Tenant config loaded from GCS (cached 5 min like themes)
#### 3. Auth Enhancement
- `AuthService.validate_token_full()` already returns `user_email`
- Add tenant validation: check if user's email domain is in tenant's allowed list
- Support fixed tokens per tenant (stored in `auth_tokens` with `tenant_id`)
- Magic links include tenant context for proper redirect
#### 4. Data Isolation
- Add `tenant_id: Optional[str]` to User and Job models
- Default to `None` for existing Nomad Karaoke users (backwards compatible)
- Query filters: `where('tenant_id', '==', tenant_id or None)`
- No schema migration needed - new field, optional
#### 5. Theme Integration
- Upload Vocal Star theme to `themes/vocalstar/` in GCS
- Upload assets (backgrounds, fonts) to `themes/vocalstar/assets/`
- Tenant config specifies `default_theme_id: "vocalstar"`
- When `theme_selection: false`, UI hides theme picker and uses default
## Implementation Steps
### Phase 1: Backend Tenant Foundation (Day 1-2)
1. [ ] **Create tenant model and config loader**
- Create `backend/models/tenant.py` with `TenantConfig`, `TenantBranding`, `TenantFeatures`
- Create `backend/services/tenant_service.py` with GCS config loading and caching
- Add `GET /api/tenant/config` endpoint (unauthenticated, returns public config)
- Add `GET /api/tenant/config/{tenant_id}` for explicit tenant fetch
2. [ ] **Add tenant middleware**
- Create `backend/api/middleware/tenant.py`
- Extract tenant from subdomain or header
- Attach to `request.state.tenant_id` and `request.state.tenant_config`
- Add `get_tenant()` dependency for routes
3. [ ] **Update data models**
- Add `tenant_id: Optional[str] = None` to `User` model
- Add `tenant_id: Optional[str] = None` to `Job` model
- Update Firestore indexes in Pulumi for tenant-scoped queries
4. [ ] **Update auth for tenant awareness**
- Modify `AuthService` to validate email domain against tenant config
- Add tenant context to magic link generation
- Support per-tenant fixed tokens in `auth_tokens` collection
### Phase 2: Upload Vocal Star Theme (Day 2)
5. [ ] **Prepare Vocal Star theme for GCS**
- Convert local paths in `karaoke-prep-styles-vocalstar.json` to GCS paths
- Upload background images to `themes/vocalstar/assets/`
- Upload Oswald font to `themes/vocalstar/assets/`
- Upload `style_params.json` to `themes/vocalstar/`
- Create preview image and upload to `themes/vocalstar/preview.png`
- Add Vocal Star to `themes/_metadata.json`
6. [ ] **Create Vocal Star tenant config**
- Create `tenants/vocalstar/config.json` in GCS
- Upload Vocal Star logo to `tenants/vocalstar/logo.png`
- Upload favicon to `tenants/vocalstar/favicon.ico`
### Phase 3: Frontend Tenant Support (Day 3-4)
7. [ ] **Add tenant context provider**
- Create `frontend/lib/tenant.ts` with tenant config types
- Create `frontend/hooks/useTenant.ts` for tenant state
- Create `TenantProvider` component that fetches config on mount
- Detect subdomain in `_app.tsx` or middleware
8. [ ] **Dynamic branding**
- Create `TenantLogo` component that renders tenant or default logo
- Update `globals.css` to support CSS variable overrides
- Add `applyTenantColors()` function to set CSS variables from config
- Update page metadata (title, favicon) from tenant config
9. [ ] **Feature-flagged UI**
- Wrap audio search in `{tenant.features.audio_search && ...}`
- Wrap theme selector in `{tenant.features.theme_selection && ...}`
- Wrap distribution options based on tenant features
- Show simplified form for Vocal Star (just file upload fields)
10. [ ] **Update magic link flow**
- Pass tenant context when requesting magic link
- Magic link redirect goes to tenant subdomain
- Verify endpoint extracts tenant from link context
### Phase 4: Backend Feature Enforcement (Day 4-5)
11. [ ] **Enforce tenant features in API**
- Audio search endpoint: return 403 if `audio_search: false`
- File upload: check `file_upload: true`
- Distribution settings: ignore/error if disabled features requested
- Theme selection: use default if `theme_selection: false`
12. [ ] **Job creation with tenant context**
- `JobCreate` handler sets `tenant_id` from request state
- Distribution config respects tenant defaults
- Theme defaults to tenant's default theme
13. [ ] **Query scoping**
- Update `JobManager.get_jobs()` to filter by tenant_id
- Update `UserService` queries to include tenant_id
- Admin endpoints bypass tenant filter (see all tenants)
### Phase 5: Infrastructure & Deployment (Day 5-6)
14. [ ] **DNS setup**
- Add `vocalstar.nomadkaraoke.com` CNAME to Cloudflare
- Configure Cloudflare Pages to accept wildcard subdomain
- Add SSL certificate for subdomain
15. [ ] **Backend CORS update**
- Add `vocalstar.nomadkaraoke.com` to allowed origins
- Or use pattern matching for `*.nomadkaraoke.com`
16. [ ] **Pulumi updates**
- Add Firestore indexes for tenant_id queries
- Add any new secrets (tenant fixed tokens)
### Phase 6: Testing & Documentation (Day 6-7)
17. [ ] **Testing**
- Unit tests for TenantService config loading
- Unit tests for tenant middleware
- E2E test for Vocal Star flow (upload audio, generate, download)
- Test magic link works for @vocal-star.com email
18. [ ] **Documentation**
- Update CLAUDE.md with tenant info
- Create `docs/WHITE-LABEL.md` with tenant setup guide
- Document how to add new tenants
## Files to Create/Modify
| File | Action | Description |
|------|--------|-------------|
| `backend/models/tenant.py` | Create | Tenant config models |
| `backend/services/tenant_service.py` | Create | Tenant config loading/caching |
| `backend/api/routes/tenant.py` | Create | Tenant config API endpoint |
| `backend/api/middleware/tenant.py` | Create | Tenant extraction middleware |
| `backend/api/dependencies.py` | Modify | Add `get_tenant()` dependency |
| `backend/models/user.py` | Modify | Add `tenant_id` field |
| `backend/models/job.py` | Modify | Add `tenant_id` field |
| `backend/services/auth_service.py` | Modify | Tenant email domain validation |
| `backend/services/user_service.py` | Modify | Tenant-aware user queries |
| `backend/services/job_manager.py` | Modify | Tenant-aware job queries |
| `backend/api/routes/file_upload.py` | Modify | Tenant feature enforcement |
| `backend/api/routes/audio_search.py` | Modify | Tenant feature enforcement |
| `backend/main.py` | Modify | Add tenant middleware |
| `frontend/lib/tenant.ts` | Create | Tenant types and API |
| `frontend/hooks/useTenant.ts` | Create | Tenant context hook |
| `frontend/components/TenantProvider.tsx` | Create | Tenant context provider |
| `frontend/components/TenantLogo.tsx` | Create | Dynamic logo component |
| `frontend/app/layout.tsx` | Modify | Add TenantProvider |
| `frontend/app/globals.css` | Modify | CSS variable support for tenant colors |
| `frontend/app/app/page.tsx` | Modify | Feature-flag UI sections |
| `infrastructure/__main__.py` | Modify | Add tenant Firestore indexes |
| `docs/WHITE-LABEL.md` | Create | Tenant setup documentation |
## Testing Strategy
### Unit Tests
- `TenantService`: Config loading, caching, validation
- `TenantMiddleware`: Subdomain extraction, header extraction
- `AuthService`: Tenant email domain validation
- `JobManager`: Tenant-scoped queries
### Integration Tests (Emulator)
- Create job with tenant context
- Query jobs filtered by tenant
- Auth with tenant-allowed email domain
- Auth rejection for wrong email domain
### E2E Tests
- Vocal Star happy path: Upload audio files, review lyrics, download video
- Auth flow: Magic link for @vocal-star.com, verify redirect to subdomain
- Feature hiding: Verify audio search not shown on Vocal Star portal
## Open Questions
- [x] **Q: Should tenant configs be in GCS or Firestore?**
A: GCS - simpler, matches theme pattern, easy to edit JSON directly
- [x] **Q: How to handle localhost development for specific tenant?**
A: Query param override `?tenant=vocalstar` for local testing
- [x] **Q: Should admin portal be tenant-aware?**
A: No - admins see all tenants. Admin routes bypass tenant filter.
- [x] **Q: Do we need tenant-specific email sender identity?**
A: Yes - use `{tenant}@nomadkaraoke.com` (e.g., `vocalstar@nomadkaraoke.com`). Catchall is configured.
- [x] **Q: Vocal Star logo and favicon - do you have these assets?**
A: Logo provided as `vocalstar-logo.jpg` - yellow star, white text on black. Will convert to PNG with transparency for web use.
## Rollback Plan
Since this is additive (new optional `tenant_id` field), rollback is straightforward:
1. Revert frontend to not check tenant
2. Revert backend middleware to not extract tenant
3. Existing data unaffected (tenant_id = null = Nomad Karaoke)
4. No schema migration to reverse
For DNS: Simply remove CNAME record if needed.
## Success Criteria
- [ ] Vocal Star user can access `vocalstar.nomadkaraoke.com`
- [ ] Portal shows Vocal Star branding (logo, colors)
- [ ] Audio search is hidden; only file upload available
- [ ] Theme is locked to Vocal Star style
- [ ] Distribution options hidden; download only
- [ ] Magic link works for @vocal-star.com emails
- [ ] Fixed API token works for automation
- [ ] Generated videos use Vocal Star theme
- [ ] Nomad Karaoke portal unchanged at gen.nomadkaraoke.com
@@ -0,0 +1,115 @@
# Build Encoding Worker GCE Image
#
# Rebuilds the custom Packer image when provisioning changes.
# The image includes Python 3.13, FFmpeg, fonts, and dependencies
# pre-installed to reduce VM startup time from ~10 min to ~30 sec.
#
# Triggers:
# - Push to main with changes to infrastructure/packer/**
# - Manual dispatch (workflow_dispatch)
#
# After image build, the encoding-worker VM will use the new image
# on next recreation (e.g., via pulumi up or manual restart).
name: Build Encoding Worker Image
on:
push:
branches: [main]
paths:
- 'infrastructure/packer/**'
workflow_dispatch:
inputs:
python_version:
description: 'Python version to install'
required: false
default: '3.13.1'
concurrency:
group: build-encoding-worker-image
cancel-in-progress: false # Don't cancel in-progress builds
jobs:
build-image:
name: Build Packer Image
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write # Required for Workload Identity Federation
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Authenticate to Google Cloud (Workload Identity)
uses: google-github-actions/auth@v2
with:
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v2
with:
version: "latest"
- name: Install Packer
run: |
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt-get update && sudo apt-get install -y packer
- name: Initialize Packer
working-directory: infrastructure/packer
run: packer init encoding-worker.pkr.hcl
- name: Validate Packer Template
working-directory: infrastructure/packer
run: packer validate encoding-worker.pkr.hcl
- name: Build Image
working-directory: infrastructure/packer
env:
PYTHON_VERSION: ${{ github.event.inputs.python_version || '3.13.1' }}
run: |
echo "Building encoding worker image with Python ${PYTHON_VERSION}..."
packer build \
-var "python_version=${PYTHON_VERSION}" \
encoding-worker.pkr.hcl
- name: Get Latest Image Info
id: image-info
run: |
IMAGE_NAME=$(gcloud compute images list \
--project=nomadkaraoke \
--filter="family:encoding-worker" \
--sort-by="~creationTimestamp" \
--limit=1 \
--format="value(name)")
echo "image_name=${IMAGE_NAME}" >> $GITHUB_OUTPUT
echo "Built image: ${IMAGE_NAME}"
- name: Summary
run: |
echo "## Encoding Worker Image Built" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Image:** \`${{ steps.image-info.outputs.image_name }}\`" >> $GITHUB_STEP_SUMMARY
echo "**Family:** \`encoding-worker\`" >> $GITHUB_STEP_SUMMARY
echo "**Project:** \`nomadkaraoke\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "The encoding-worker VM will use this image on next recreation." >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "To force VM recreation now:" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`bash" >> $GITHUB_STEP_SUMMARY
echo "gcloud compute instances delete encoding-worker --zone=us-central1-a --project=nomadkaraoke" >> $GITHUB_STEP_SUMMARY
echo "# Then run pulumi up or wait for next infrastructure deploy" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
notify-on-failure:
name: Notify on Failure
runs-on: ubuntu-latest
needs: build-image
if: failure()
steps:
- name: Log Failure
run: |
echo "::error::Encoding worker image build failed. Check the build-image job logs."
+223 -22
View File
@@ -67,6 +67,29 @@ jobs:
# FRONTEND TESTS (PRs only - main skips tests)
# ============================================
frontend-lint:
name: "Frontend - Lint (Rules of Hooks)"
runs-on: ubuntu-latest
needs: changes
if: github.ref != 'refs/heads/main' && needs.changes.outputs.frontend == 'true'
steps:
- uses: actions/checkout@v4
- name: Setup Node.js with caching
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: frontend/package-lock.json
- name: Install dependencies
working-directory: frontend
run: npm ci --legacy-peer-deps
- name: Run ESLint
working-directory: frontend
run: npm run lint
frontend-unit-tests:
name: "Frontend - Unit Tests"
runs-on: ubuntu-latest
@@ -655,12 +678,14 @@ jobs:
runs-on: ubuntu-latest
# Note: frontend-e2e-full is optional (requires 'e2e' label) so not included here
# frontend-e2e-smoke IS required - fast smoke tests catch critical regressions
needs: [frontend-unit-tests, frontend-e2e-smoke, frontend-build, package-unit-tests, package-integration-tests, backend-unit-tests, backend-emulator-tests, backend-lint]
# frontend-lint IS required - catches React hooks violations that cause runtime crashes
needs: [frontend-lint, frontend-unit-tests, frontend-e2e-smoke, frontend-build, package-unit-tests, package-integration-tests, backend-unit-tests, backend-emulator-tests, backend-lint]
if: github.ref != 'refs/heads/main' && always()
steps:
- name: Check results
run: |
results=(
"${{ needs.frontend-lint.result }}"
"${{ needs.frontend-unit-tests.result }}"
"${{ needs.frontend-e2e-smoke.result }}"
"${{ needs.frontend-build.result }}"
@@ -903,11 +928,14 @@ jobs:
deploy-backend:
name: "Deploy - Backend (Cloud Run)"
needs: [package-build]
runs-on: [self-hosted, linux, gcp]
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
permissions:
contents: read
id-token: write
outputs:
base_rebuilt: ${{ steps.check-base.outputs.needs_rebuild }}
env:
REGISTRY: us-central1-docker.pkg.dev
PROJECT_ID: nomadkaraoke
@@ -965,6 +993,32 @@ jobs:
with:
version: "latest"
# ============================================
# Upload wheel to GCS for GCE encoding worker
# The GCE worker installs this wheel at job start for hot code updates
# ============================================
- name: Download wheel artifact
uses: actions/download-artifact@v4
with:
name: package-dist
path: dist/
- name: Upload wheel to GCS for GCE worker
run: |
echo "📦 Uploading wheel to GCS for GCE encoding worker..."
WHEEL_FILE=$(ls dist/*.whl | head -1)
if [ -z "$WHEEL_FILE" ]; then
echo "::error::No wheel file found in dist/"
exit 1
fi
echo "Found wheel: $WHEEL_FILE"
# Upload to GCS (GCE worker downloads latest at job start)
gsutil cp "$WHEEL_FILE" gs://karaoke-gen-storage-${{ env.PROJECT_ID }}/wheels/
echo "✅ Wheel uploaded - GCE worker will pick up changes on next job"
- name: Configure Docker for Artifact Registry
run: gcloud auth configure-docker ${{ env.REGISTRY }} --quiet
@@ -990,32 +1044,32 @@ jobs:
run: |
BASE_IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest"
# Try to get the existing base image's hash label
# First check if image exists locally (fast, no network call)
# Only pull from registry if not cached locally
# Fetch hash label directly from registry (10KB config blob, not full image)
# This avoids issues with stale local Docker cache on GitHub runners
EXISTING_HASH=""
if docker image inspect $BASE_IMAGE >/dev/null 2>&1; then
echo "✅ Base image found in local cache (skipping slow registry pull)"
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
# Fallback to old label name for backwards compatibility
if [ -z "$EXISTING_HASH" ]; then
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "poetry.lock.hash"}}' 2>/dev/null || echo "")
fi
echo "Existing base image hash: $EXISTING_HASH"
elif docker pull $BASE_IMAGE 2>/dev/null; then
echo "📥 Base image pulled from registry"
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
# Fallback to old label name for backwards compatibility
if [ -z "$EXISTING_HASH" ]; then
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "poetry.lock.hash"}}' 2>/dev/null || echo "")
fi
echo "Existing base image hash: $EXISTING_HASH"
# Get the manifest (small JSON, ~1KB) - includes config digest
MANIFEST=$(docker manifest inspect $BASE_IMAGE 2>/dev/null || echo "")
CONFIG_DIGEST=$(echo "$MANIFEST" | jq -r '.config.digest // ""' 2>/dev/null || echo "")
if [ -n "$CONFIG_DIGEST" ] && [ "$CONFIG_DIGEST" != "null" ] && [ "$CONFIG_DIGEST" != "" ]; then
echo "📋 Found manifest, config digest: $CONFIG_DIGEST"
# Fetch just the config blob (~10KB) to get labels - much faster than pulling full image
TOKEN=$(gcloud auth print-access-token)
CONFIG_JSON=$(curl -sL -H "Authorization: Bearer $TOKEN" \
"https://${{ env.REGISTRY }}/v2/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}/blobs/$CONFIG_DIGEST" 2>/dev/null)
EXISTING_HASH=$(echo "$CONFIG_JSON" | jq -r '.config.Labels["base.content.hash"] // .config.Labels["poetry.lock.hash"] // ""' 2>/dev/null || echo "")
echo "📋 Registry base image hash: $EXISTING_HASH"
else
echo "⚠️ No existing base image found - will build"
echo "⚠️ No existing base image found in registry - will build"
fi
if [ "${{ steps.base-hash.outputs.hash }}" != "$EXISTING_HASH" ]; then
echo "🔄 Base image needs rebuild (poetry.lock or Dockerfile.base changed)"
echo " Current: ${{ steps.base-hash.outputs.hash }}"
echo " Registry: $EXISTING_HASH"
echo "needs_rebuild=true" >> $GITHUB_OUTPUT
else
echo "✅ Base image is up to date"
@@ -1043,6 +1097,31 @@ jobs:
# Provenance disabled for smaller image and faster push
provenance: false
- name: Ensure local base image is current (if not rebuilt)
if: steps.check-base.outputs.needs_rebuild == 'false'
run: |
BASE_IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest"
# Get the config digest from registry manifest (identifies the image content)
# This is the same value as Docker's Image ID for the same image
REGISTRY_CONFIG_DIGEST=$(docker manifest inspect $BASE_IMAGE 2>/dev/null | jq -r '.config.digest' || echo "")
# Get local image's ID (which is the config digest)
LOCAL_IMAGE_ID=$(docker image inspect $BASE_IMAGE --format='{{.Id}}' 2>/dev/null || echo "")
echo "Registry config digest: $REGISTRY_CONFIG_DIGEST"
echo "Local image ID: $LOCAL_IMAGE_ID"
if [ -z "$LOCAL_IMAGE_ID" ]; then
echo "📥 Base image not in local cache - pulling from registry"
docker pull $BASE_IMAGE
elif [ "$LOCAL_IMAGE_ID" != "$REGISTRY_CONFIG_DIGEST" ]; then
echo "🔄 Local cache has different image - pulling latest"
docker pull $BASE_IMAGE
else
echo "✅ Local cache matches registry - skipping pull"
fi
- name: Build and push app image
uses: docker/build-push-action@v6
with:
@@ -1085,7 +1164,7 @@ jobs:
--concurrency 1 \
--max-instances 50 \
--min-instances 4 \
--set-env-vars "GOOGLE_CLOUD_PROJECT=${{ env.PROJECT_ID }},GCS_BUCKET_NAME=karaoke-gen-storage-${{ env.PROJECT_ID }},FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD,USE_GCE_ENCODING=true" \
--set-env-vars "GOOGLE_CLOUD_PROJECT=${{ env.PROJECT_ID }},GCS_BUCKET_NAME=karaoke-gen-storage-${{ env.PROJECT_ID }},FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD,USE_GCE_ENCODING=true,USE_GCE_PREVIEW_ENCODING=true" \
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest,RED_API_KEY=red-api-key:latest,RED_API_URL=red-api-url:latest,OPS_API_KEY=ops-api-key:latest,OPS_API_URL=ops-api-url:latest,FLACFETCH_API_URL=flacfetch-api-url:latest,FLACFETCH_API_KEY=flacfetch-api-key:latest,SENDGRID_API_KEY=sendgrid-api-key:latest,STRIPE_SECRET_KEY=stripe-secret-key:latest,STRIPE_WEBHOOK_SECRET=stripe-webhook-secret:latest,LANGFUSE_PUBLIC_KEY=langfuse-public-key:latest,LANGFUSE_SECRET_KEY=langfuse-secret-key:latest,LANGFUSE_HOST=langfuse-host:latest,ENCODING_WORKER_URL=encoding-worker-url:latest,ENCODING_WORKER_API_KEY=encoding-worker-api-key:latest"
- name: Verify deployment and version
@@ -1139,3 +1218,125 @@ jobs:
echo ""
echo "📊 Built on self-hosted GCP runner (200GB SSD, no disk space issues)"
echo " Runner: gcp-runner with 4 vCPU, 16GB RAM"
# ============================================
# PROPAGATE BASE IMAGE TO ALL RUNNERS
# When base image is rebuilt, pull it to all 20 runners
# so subsequent app builds don't wait 8 min for the pull
#
# NOTE: This is best-effort distribution. GitHub Actions doesn't
# support targeting specific runners by name, so we spawn 20 jobs
# and rely on the scheduler to distribute them across idle runners.
# In practice, with all runners idle after deploy-backend, distribution
# is good. Runners that miss this pull will get the image on their
# next app build (with the 8 min delay).
# ============================================
propagate-base-image:
name: "Propagate Base Image (${{ matrix.runner_index }})"
needs: deploy-backend
if: github.ref == 'refs/heads/main' && github.event_name == 'push' && needs.deploy-backend.outputs.base_rebuilt == 'true'
runs-on: [self-hosted, linux, gcp]
permissions:
contents: read
id-token: write
strategy:
matrix:
# 20 parallel jobs - GitHub distributes across available runners
runner_index: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]
max-parallel: 20
fail-fast: false # Don't cancel other runners if one fails
env:
REGISTRY: us-central1-docker.pkg.dev
PROJECT_ID: nomadkaraoke
REPOSITORY: karaoke-repo
BASE_IMAGE_NAME: karaoke-backend-base
steps:
- name: Authenticate to Google Cloud (Workload Identity)
uses: google-github-actions/auth@v2
with:
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v2
with:
version: "latest"
- name: Configure Docker for Artifact Registry
run: gcloud auth configure-docker ${{ env.REGISTRY }} --quiet
- name: Pull latest base image
run: |
BASE_IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest"
RUNNER_NAME=$(hostname)
echo "📥 Pulling latest base image on runner $RUNNER_NAME..."
docker pull $BASE_IMAGE
echo "✅ Base image cached on $RUNNER_NAME"
deploy-infrastructure:
name: "Deploy - Infrastructure (Pulumi)"
runs-on: [self-hosted, linux, gcp]
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
permissions:
contents: read
id-token: write
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Authenticate to Google Cloud (Workload Identity)
uses: google-github-actions/auth@v2
with:
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v2
with:
version: "latest"
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: '3.13'
- name: Install Pulumi
run: |
curl -fsSL https://get.pulumi.com | sh
echo "$HOME/.pulumi/bin" >> $GITHUB_PATH
- name: Install Python dependencies
working-directory: infrastructure
run: |
pip install pulumi pulumi-gcp pulumi-docker
- name: Configure Pulumi
env:
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
run: |
pulumi login
- name: Deploy Infrastructure
working-directory: infrastructure
env:
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
run: |
echo "🚀 Deploying infrastructure changes..."
# Skip preview to avoid needing broad read permissions on all GCP resources
# Pulumi state is authoritative since infrastructure is only managed via Pulumi
pulumi up --yes --skip-preview --stack nomadkaraoke/karaoke-gen-infrastructure/prod
- name: Restart GCE Encoding Worker
run: |
echo "🔄 Restarting encoding worker to pick up new code..."
gcloud compute ssh encoding-worker --zone=us-central1-a --project=nomadkaraoke \
--command="sudo systemctl restart encoding-worker" || true
echo "✅ Encoding worker restart triggered"
- name: Summary
if: success()
run: |
echo "🎉 Infrastructure deployment successful!"
echo "Commit: ${{ github.sha }}"
echo "Branch: ${{ github.ref_name }}"
+6 -2
View File
@@ -18,6 +18,8 @@
| Dev setup & testing | `docs/DEVELOPMENT.md` |
| API reference | `docs/API.md` |
| Past learnings | `docs/LESSONS-LEARNED.md` |
| **Product communication** | `docs/PRODUCT-COMMUNICATION-GUIDE.md` |
| Brand style guide | `docs/BRAND-STYLE-GUIDE.md` |
## Tech Stack
@@ -48,8 +50,10 @@ cd frontend && npm run test:all 2>&1 | tail -n 200
- Skip for docs-only changes
### Infrastructure
- All GCP changes via Pulumi in `infrastructure/`
- `gcloud` CLI for reading/debugging only
- **All GCP changes via Pulumi PRs** - changes in `infrastructure/` deploy automatically on merge to main
- Never modify GCP resources directly via console or `gcloud` CLI
- `gcloud` CLI for reading/debugging only (e.g., checking logs, SSH to VMs)
- Local `pulumi preview` works but `pulumi up` requires broader permissions than CI
- Stop and notify user on auth issues
## API Authentication (for Agents)
+419 -92
View File
@@ -20,6 +20,8 @@ from backend.services.user_service import get_user_service, UserService, USERS_C
from backend.services.job_manager import JobManager
from backend.services.flacfetch_client import get_flacfetch_client, FlacfetchServiceError
from backend.models.job import JobStatus
from backend.utils.test_data import is_test_email
from karaoke_gen.utils import sanitize_filename
logger = logging.getLogger(__name__)
@@ -60,12 +62,17 @@ class AdminStatsOverview(BaseModel):
@router.get("/stats/overview", response_model=AdminStatsOverview)
async def get_admin_stats_overview(
exclude_test: bool = True,
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
user_service: UserService = Depends(get_user_service),
):
"""
Get overview statistics for admin dashboard.
Args:
exclude_test: If True (default), exclude test data (users with @inbox.testmail.app emails
and jobs created by test users) from all counts.
Includes:
- User counts (total, active in 7d, active in 30d)
- Job counts (total, by status, recent)
@@ -80,48 +87,10 @@ async def get_admin_stats_overview(
seven_days_ago = now - timedelta(days=7)
thirty_days_ago = now - timedelta(days=30)
# Helper function to get count using aggregation
def get_count(query) -> int:
try:
agg_query = aggregation.AggregationQuery(query)
agg_query.count(alias="count")
results = agg_query.get()
return results[0][0].value if results else 0
except Exception as e:
logger.warning(f"Aggregation query failed: {e}")
return 0
# User statistics
users_collection = db.collection(USERS_COLLECTION)
total_users = get_count(users_collection)
active_users_7d = get_count(
users_collection.where(filter=FieldFilter("last_login_at", ">=", seven_days_ago))
)
active_users_30d = get_count(
users_collection.where(filter=FieldFilter("last_login_at", ">=", thirty_days_ago))
)
total_beta_testers = get_count(
users_collection.where(filter=FieldFilter("is_beta_tester", "==", True))
)
# Job statistics
jobs_collection = db.collection("jobs")
total_jobs = get_count(jobs_collection)
jobs_last_7d = get_count(
jobs_collection.where(filter=FieldFilter("created_at", ">=", seven_days_ago))
)
jobs_last_30d = get_count(
jobs_collection.where(filter=FieldFilter("created_at", ">=", thirty_days_ago))
)
# Jobs by status - map multiple statuses to simplified categories
# Jobs by status category mapping
processing_statuses = [
"downloading", "downloading_audio", "searching_audio", "awaiting_audio_selection",
"separating_stage1", "separating_stage2", "transcribing", "correcting",
@@ -130,63 +99,166 @@ async def get_admin_stats_overview(
"uploading", "notifying"
]
jobs_by_status = JobsByStatusResponse(
pending=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "pending"))
),
processing=sum(
get_count(jobs_collection.where(filter=FieldFilter("status", "==", status)))
for status in processing_statuses
),
awaiting_review=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "awaiting_review"))
) + get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "in_review"))
),
awaiting_instrumental=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "awaiting_instrumental_selection"))
),
complete=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "complete"))
) + get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "prep_complete"))
),
failed=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "failed"))
),
cancelled=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "cancelled"))
),
)
# Limits for streaming queries - these are safety limits to prevent memory issues
# If hit, stats may be incomplete so we log a warning
USERS_STREAM_LIMIT = 2000
JOBS_STREAM_LIMIT = 10000
# Credit statistics - sum credits added in last 30 days
# This is more expensive, so we'll just estimate from users
total_credits_issued_30d = 0
try:
# Get all users and sum recent credit transactions
users_docs = users_collection.limit(500).stream()
for user_doc in users_docs:
user_data = user_doc.to_dict()
if exclude_test:
# When excluding test data, we must stream and filter in Python
# because Firestore doesn't support "not ends with" queries
# Stream all users and filter
all_users = []
users_fetched = 0
for doc in users_collection.limit(USERS_STREAM_LIMIT).stream():
users_fetched += 1
user_data = doc.to_dict()
email = user_data.get("email", "")
if not is_test_email(email):
all_users.append(user_data)
if users_fetched >= USERS_STREAM_LIMIT:
logger.warning(f"Users stream hit limit ({USERS_STREAM_LIMIT}), stats may be incomplete")
# Calculate user stats from filtered list
total_users = len(all_users)
active_users_7d = sum(
1 for u in all_users
if u.get("last_login_at") and _normalize_datetime(u["last_login_at"]) >= seven_days_ago
)
active_users_30d = sum(
1 for u in all_users
if u.get("last_login_at") and _normalize_datetime(u["last_login_at"]) >= thirty_days_ago
)
total_beta_testers = sum(1 for u in all_users if u.get("is_beta_tester"))
# Calculate credits from filtered users
total_credits_issued_30d = 0
for user_data in all_users:
transactions = user_data.get("credit_transactions", [])
for txn in transactions:
txn_date = txn.get("created_at")
if txn_date:
# Handle both datetime and string formats
if isinstance(txn_date, str):
try:
txn_date = datetime.fromisoformat(txn_date.replace("Z", "+00:00"))
except Exception:
continue
if isinstance(txn_date, datetime):
txn_date = txn_date.replace(tzinfo=None)
else:
continue
if txn_date >= thirty_days_ago:
txn_date = _normalize_datetime(txn.get("created_at"))
if txn_date and txn_date >= thirty_days_ago:
amount = txn.get("amount", 0)
if amount > 0:
total_credits_issued_30d += amount
# Stream all jobs and filter by user_email
all_jobs = []
jobs_fetched = 0
for doc in jobs_collection.limit(JOBS_STREAM_LIMIT).stream():
jobs_fetched += 1
job_data = doc.to_dict()
user_email = job_data.get("user_email", "")
if not is_test_email(user_email):
all_jobs.append(job_data)
if jobs_fetched >= JOBS_STREAM_LIMIT:
logger.warning(f"Jobs stream hit limit ({JOBS_STREAM_LIMIT}), stats may be incomplete")
# Calculate job stats from filtered list
total_jobs = len(all_jobs)
jobs_last_7d = sum(
1 for j in all_jobs
if j.get("created_at") and _normalize_datetime(j["created_at"]) >= seven_days_ago
)
jobs_last_30d = sum(
1 for j in all_jobs
if j.get("created_at") and _normalize_datetime(j["created_at"]) >= thirty_days_ago
)
# Jobs by status
jobs_by_status = JobsByStatusResponse(
pending=sum(1 for j in all_jobs if j.get("status") == "pending"),
processing=sum(1 for j in all_jobs if j.get("status") in processing_statuses),
awaiting_review=sum(1 for j in all_jobs if j.get("status") in ["awaiting_review", "in_review"]),
awaiting_instrumental=sum(1 for j in all_jobs if j.get("status") == "awaiting_instrumental_selection"),
complete=sum(1 for j in all_jobs if j.get("status") in ["complete", "prep_complete"]),
failed=sum(1 for j in all_jobs if j.get("status") == "failed"),
cancelled=sum(1 for j in all_jobs if j.get("status") == "cancelled"),
)
else:
# When including test data, use efficient aggregation queries
def get_count(query) -> int:
try:
agg_query = aggregation.AggregationQuery(query)
agg_query.count(alias="count")
results = agg_query.get()
return results[0][0].value if results else 0
except Exception as e:
logger.warning(f"Aggregation query failed: {e}")
return 0
# User statistics
total_users = get_count(users_collection)
active_users_7d = get_count(
users_collection.where(filter=FieldFilter("last_login_at", ">=", seven_days_ago))
)
active_users_30d = get_count(
users_collection.where(filter=FieldFilter("last_login_at", ">=", thirty_days_ago))
)
total_beta_testers = get_count(
users_collection.where(filter=FieldFilter("is_beta_tester", "==", True))
)
# Job statistics
total_jobs = get_count(jobs_collection)
jobs_last_7d = get_count(
jobs_collection.where(filter=FieldFilter("created_at", ">=", seven_days_ago))
)
jobs_last_30d = get_count(
jobs_collection.where(filter=FieldFilter("created_at", ">=", thirty_days_ago))
)
# Jobs by status
jobs_by_status = JobsByStatusResponse(
pending=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "pending"))
),
processing=sum(
get_count(jobs_collection.where(filter=FieldFilter("status", "==", status)))
for status in processing_statuses
),
awaiting_review=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "awaiting_review"))
) + get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "in_review"))
),
awaiting_instrumental=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "awaiting_instrumental_selection"))
),
complete=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "complete"))
) + get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "prep_complete"))
),
failed=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "failed"))
),
cancelled=get_count(
jobs_collection.where(filter=FieldFilter("status", "==", "cancelled"))
),
)
# Credit statistics - sum credits added in last 30 days
total_credits_issued_30d = 0
try:
users_fetched = 0
for user_doc in users_collection.limit(USERS_STREAM_LIMIT).stream():
users_fetched += 1
user_data = user_doc.to_dict()
transactions = user_data.get("credit_transactions", [])
for txn in transactions:
txn_date = _normalize_datetime(txn.get("created_at"))
if txn_date and txn_date >= thirty_days_ago:
amount = txn.get("amount", 0)
if amount > 0: # Only count additions, not deductions
if amount > 0:
total_credits_issued_30d += amount
except Exception as e:
logger.warning(f"Error calculating credits: {e}")
if users_fetched >= USERS_STREAM_LIMIT:
logger.warning(f"Credit calculation hit user limit ({USERS_STREAM_LIMIT}), total may be incomplete")
except Exception as e:
logger.warning(f"Error calculating credits: {e}")
return AdminStatsOverview(
total_users=total_users,
@@ -201,6 +273,21 @@ async def get_admin_stats_overview(
)
def _normalize_datetime(dt_value) -> Optional[datetime]:
"""Normalize datetime values from Firestore (can be datetime or ISO string)."""
if dt_value is None:
return None
if isinstance(dt_value, datetime):
return dt_value.replace(tzinfo=None)
if isinstance(dt_value, str):
try:
parsed = datetime.fromisoformat(dt_value.replace("Z", "+00:00"))
return parsed.replace(tzinfo=None)
except Exception:
return None
return None
# =============================================================================
# Audio Search Management Models
# =============================================================================
@@ -272,6 +359,7 @@ class CacheStatsResponse(BaseModel):
async def list_audio_searches(
limit: int = 50,
status_filter: Optional[str] = None,
exclude_test: bool = True,
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
user_service: UserService = Depends(get_user_service),
):
@@ -286,6 +374,7 @@ async def list_audio_searches(
Args:
limit: Maximum number of jobs to return (default 50)
status_filter: Optional filter by job status (e.g., 'awaiting_audio_selection')
exclude_test: If True (default), exclude jobs from test users
"""
from google.cloud.firestore_v1 import FieldFilter
@@ -305,6 +394,11 @@ async def list_audio_searches(
for doc in query.stream():
data = doc.to_dict()
# Filter out test users if exclude_test is True
if exclude_test and is_test_email(data.get("user_email", "")):
continue
state_data = data.get("state_data", {})
audio_results = state_data.get("audio_search_results", [])
@@ -579,4 +673,237 @@ async def get_flacfetch_cache_stats(
status_code=502,
detail=f"Failed to get cache stats: {e}"
)
# Trigger deployment
# =============================================================================
# Job Completion Message Endpoints (for admin copy/send functionality)
# =============================================================================
class CompletionMessageResponse(BaseModel):
"""Response containing the rendered completion message."""
job_id: str
message: str
subject: str
youtube_url: Optional[str] = None
dropbox_url: Optional[str] = None
class SendCompletionEmailRequest(BaseModel):
"""Request to send a completion email."""
to_email: str
cc_admin: bool = True
class SendCompletionEmailResponse(BaseModel):
"""Response from sending a completion email."""
success: bool
job_id: str
to_email: str
message: str
@router.get("/jobs/{job_id}/completion-message", response_model=CompletionMessageResponse)
async def get_job_completion_message(
job_id: str,
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
):
"""
Get the rendered completion message for a job.
Returns the plain text message that would be sent to the user,
rendered using the job completion template with the job's details.
This is useful for:
- Copying the message to clipboard (e.g., for Fiverr)
- Previewing the email before sending
Requires admin authentication.
"""
from backend.services.job_notification_service import get_job_notification_service
job_manager = JobManager()
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
# Get youtube, dropbox URLs, and brand_code from state_data (may be None)
state_data = job.state_data or {}
youtube_url = state_data.get('youtube_url')
dropbox_url = state_data.get('dropbox_link')
brand_code = state_data.get('brand_code')
# Render the completion message
notification_service = get_job_notification_service()
message = notification_service.get_completion_message(
job_id=job.job_id,
user_name=None, # Use default "there"
artist=job.artist,
title=job.title,
youtube_url=youtube_url,
dropbox_url=dropbox_url,
)
# Build subject: "NOMAD-1178: Artist - Title (Your karaoke video is ready!)"
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
# that cause email header encoding issues (MIME headers use latin-1)
safe_artist = sanitize_filename(job.artist) if job.artist else None
safe_title = sanitize_filename(job.title) if job.title else None
if brand_code and safe_artist and safe_title:
subject = f"{brand_code}: {safe_artist} - {safe_title} (Your karaoke video is ready!)"
elif safe_artist and safe_title:
subject = f"{safe_artist} - {safe_title} (Your karaoke video is ready!)"
else:
subject = "Your karaoke video is ready!"
return CompletionMessageResponse(
job_id=job_id,
message=message,
subject=subject,
youtube_url=youtube_url,
dropbox_url=dropbox_url,
)
@router.post("/jobs/{job_id}/send-completion-email", response_model=SendCompletionEmailResponse)
async def send_job_completion_email(
job_id: str,
request: SendCompletionEmailRequest,
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
):
"""
Send a completion email for a job to a specified email address.
This allows admins to manually send (or re-send) completion emails,
useful for:
- Sending to customers who didn't have an email on file
- Re-sending if the original email was lost
- Sending to alternate email addresses
Requires admin authentication.
"""
from backend.services.job_notification_service import get_job_notification_service
from backend.services.email_service import get_email_service
job_manager = JobManager()
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
# Get youtube, dropbox URLs, and brand_code from state_data (may be None)
state_data = job.state_data or {}
youtube_url = state_data.get('youtube_url')
dropbox_url = state_data.get('dropbox_link')
brand_code = state_data.get('brand_code')
# Render the completion message
notification_service = get_job_notification_service()
message = notification_service.get_completion_message(
job_id=job.job_id,
user_name=None, # Use default "there"
artist=job.artist,
title=job.title,
youtube_url=youtube_url,
dropbox_url=dropbox_url,
)
# Send the email
email_service = get_email_service()
success = email_service.send_job_completion(
to_email=request.to_email,
message_content=message,
artist=job.artist,
title=job.title,
brand_code=brand_code,
cc_admin=request.cc_admin,
)
if success:
logger.info(f"Admin sent completion email for job {job_id} to {request.to_email}")
return SendCompletionEmailResponse(
success=True,
job_id=job_id,
to_email=request.to_email,
message=f"Completion email sent to {request.to_email}",
)
else:
logger.error(f"Failed to send completion email for job {job_id}")
raise HTTPException(
status_code=500,
detail="Failed to send email. Check email service configuration."
)
# =============================================================================
# User Impersonation
# =============================================================================
class ImpersonateUserResponse(BaseModel):
"""Response from impersonate user endpoint."""
session_token: str
user_email: str
message: str
@router.post("/users/{email}/impersonate", response_model=ImpersonateUserResponse)
async def impersonate_user(
email: str,
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
user_service: UserService = Depends(get_user_service),
):
"""
Create a session token to impersonate a user (admin only).
This allows admins to view the application exactly as a specific user would see it.
The admin's original session remains valid and can be restored client-side.
Security:
- Only admins can impersonate
- Creates a real session (auditable in Firestore)
- Impersonation is logged for security audit
Args:
email: Email of the user to impersonate
Returns:
session_token: A valid session token for the target user
user_email: The impersonated user's email
message: Success message
"""
admin_email = auth_data[0]
target_email = email.lower()
# Cannot impersonate yourself
if target_email == admin_email.lower():
raise HTTPException(
status_code=400,
detail="Cannot impersonate yourself"
)
# Verify target user exists
target_user = user_service.get_user(target_email)
if not target_user:
raise HTTPException(
status_code=404,
detail=f"User {target_email} not found"
)
# Create a real session for the target user
session = user_service.create_session(
user_email=target_email,
ip_address=None, # Not tracking IP for impersonation
user_agent=f"Impersonation by {admin_email}",
)
# Log impersonation for audit trail
logger.info(
f"IMPERSONATION: Admin {admin_email} started impersonating user {target_email}. "
f"Session token prefix: {session.token[:12]}..."
)
return ImpersonateUserResponse(
session_token=session.token,
user_email=target_email,
message=f"Now impersonating {target_email}",
)
+53 -13
View File
@@ -18,9 +18,10 @@ import tempfile
from typing import Optional, List, Dict, Any, Tuple
from fastapi import APIRouter, HTTPException, BackgroundTasks, Request, Depends
from pydantic import BaseModel, Field
from pydantic import BaseModel, Field, validator
from backend.models.job import JobCreate, JobStatus
from karaoke_gen.utils import normalize_text
from backend.services.job_manager import JobManager
from backend.services.storage_service import StorageService
from backend.services.worker_service import get_worker_service
@@ -32,10 +33,12 @@ from backend.services.audio_search_service import (
NoResultsError,
DownloadError,
)
from backend.services.theme_service import get_theme_service
from backend.config import get_settings
from backend.version import VERSION
from backend.api.dependencies import require_auth
from backend.services.auth_service import UserType, AuthResult
from backend.middleware.tenant import get_tenant_config_from_request
from pathlib import Path
logger = logging.getLogger(__name__)
@@ -95,7 +98,7 @@ class AudioSearchRequest(BaseModel):
# Finalisation options
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
enable_youtube_upload: Optional[bool] = Field(None, description="Upload to YouTube. None = use server default")
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
@@ -107,6 +110,10 @@ class AudioSearchRequest(BaseModel):
lyrics_artist: Optional[str] = Field(None, description="Override artist name for lyrics search")
lyrics_title: Optional[str] = Field(None, description="Override title for lyrics search")
subtitle_offset_ms: int = Field(0, description="Subtitle timing offset in milliseconds")
# Display overrides (optional - if empty, search values are used for display)
display_artist: Optional[str] = Field(None, description="Artist name for title screens/filenames. If empty, uses search artist.")
display_title: Optional[str] = Field(None, description="Title for title screens/filenames. If empty, uses search title.")
# Audio separation model configuration
clean_instrumental_model: Optional[str] = Field(None, description="Model for clean instrumental separation")
@@ -120,6 +127,13 @@ class AudioSearchRequest(BaseModel):
# Non-interactive mode
non_interactive: bool = Field(False, description="Skip interactive steps (lyrics review, instrumental selection)")
@validator('artist', 'title', 'lyrics_artist', 'lyrics_title', 'display_artist', 'display_title')
def normalize_text_fields(cls, v):
"""Normalize text fields to standardize Unicode characters."""
if v is not None and isinstance(v, str):
return normalize_text(v)
return v
class AudioSearchResultResponse(BaseModel):
"""A single audio search result.
@@ -466,17 +480,25 @@ async def search_audio(
):
"""
Search for audio by artist and title, creating a new job.
This endpoint:
1. Creates a job in PENDING state
2. Searches for audio using flacfetch
3. Either returns search results for user selection, or
4. If auto_download=True, automatically selects best and starts processing
Use cases:
- Interactive mode (default): Returns results, user calls /select endpoint
- Auto mode (auto_download=True): Automatically selects and downloads best
"""
# Check tenant feature flag
tenant_config = get_tenant_config_from_request(request)
if tenant_config and not tenant_config.features.audio_search:
raise HTTPException(
status_code=403,
detail="Audio search is not available for this portal"
)
try:
# Apply default distribution settings
settings = get_settings()
@@ -485,7 +507,8 @@ async def search_audio(
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
# Apply defaults for YouTube/Dropbox distribution (for web service)
effective_enable_youtube_upload = body.enable_youtube_upload or settings.default_enable_youtube_upload
# Use explicit value if provided, otherwise fall back to server default
effective_enable_youtube_upload = body.enable_youtube_upload if body.enable_youtube_upload is not None else settings.default_enable_youtube_upload
effective_brand_prefix = body.brand_prefix or settings.default_brand_prefix
effective_youtube_description = body.youtube_description or settings.default_youtube_description
@@ -521,19 +544,34 @@ async def search_audio(
# Extract request metadata
request_metadata = extract_request_metadata(request, created_from="audio_search")
# Apply default theme if none specified
# This ensures all karaoke videos use the Nomad theme by default
effective_theme_id = body.theme_id
if effective_theme_id is None:
theme_service = get_theme_service()
effective_theme_id = theme_service.get_default_theme_id()
if effective_theme_id:
logger.info(f"Applying default theme: {effective_theme_id}")
# Resolve CDG/TXT defaults based on theme
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
body.theme_id, body.enable_cdg, body.enable_txt
effective_theme_id, body.enable_cdg, body.enable_txt
)
# Use authenticated user's email
effective_user_email = auth_result.user_email
# Determine display values - use display_* if provided, otherwise fall back to search values
# Display values are used for title screens, filenames, YouTube, etc.
# Search values (body.artist, body.title) are used for audio search
effective_display_artist = body.display_artist.strip() if body.display_artist else body.artist
effective_display_title = body.display_title.strip() if body.display_title else body.title
# Create job
job_create = JobCreate(
artist=body.artist,
title=body.title,
theme_id=body.theme_id,
artist=effective_display_artist, # Display value for title screens, filenames
title=effective_display_title, # Display value for title screens, filenames
theme_id=effective_theme_id,
color_overrides=body.color_overrides or {},
enable_cdg=resolved_cdg,
enable_txt=resolved_txt,
@@ -556,6 +594,8 @@ async def search_audio(
auto_download=body.auto_download,
request_metadata=request_metadata,
non_interactive=body.non_interactive,
# Tenant scoping
tenant_id=tenant_config.id if tenant_config else None,
)
job = job_manager.create_job(job_create)
job_id = job.job_id
@@ -572,11 +612,11 @@ async def search_audio(
# If theme is set and no custom style files are being uploaded, prepare theme style now
# This copies the theme's style_params.json to the job folder so LyricsTranscriber
# can access the style configuration for preview videos
if body.theme_id and not body.style_files:
if effective_theme_id and not body.style_files:
from backend.api.routes.file_upload import _prepare_theme_for_job
try:
style_params_path, theme_style_assets, youtube_desc = _prepare_theme_for_job(
job_id, body.theme_id, body.color_overrides
job_id, effective_theme_id, body.color_overrides
)
theme_update = {
'style_params_gcs_path': style_params_path,
@@ -585,9 +625,9 @@ async def search_audio(
if youtube_desc and not effective_youtube_description:
theme_update['youtube_description_template'] = youtube_desc
job_manager.update_job(job_id, theme_update)
logger.info(f"Applied theme '{body.theme_id}' to job {job_id}")
logger.info(f"Applied theme '{effective_theme_id}' to job {job_id}")
except Exception as e:
logger.warning(f"Failed to prepare theme '{body.theme_id}' for job {job_id}: {e}")
logger.warning(f"Failed to prepare theme '{effective_theme_id}' for job {job_id}: {e}")
# Continue without theme - job can still be processed with defaults
# Handle style file uploads if provided
+135 -35
View File
@@ -34,6 +34,7 @@ from backend.version import VERSION
from backend.services.metrics import metrics
from backend.api.dependencies import require_auth
from backend.services.auth_service import UserType, AuthResult
from backend.middleware.tenant import get_tenant_config_from_request
logger = logging.getLogger(__name__)
router = APIRouter(tags=["jobs"])
@@ -69,7 +70,7 @@ class CreateJobFromUrlRequest(BaseModel):
# Finalisation options
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
enable_youtube_upload: Optional[bool] = Field(None, description="Upload to YouTube. None = use server default")
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
webhook_url: Optional[str] = Field(None, description="Generic webhook URL")
@@ -123,7 +124,7 @@ class CreateJobWithUploadUrlsRequest(BaseModel):
# Finalisation options
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
enable_youtube_upload: Optional[bool] = Field(None, description="Upload to YouTube. None = use server default")
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
webhook_url: Optional[str] = Field(None, description="Generic webhook URL")
@@ -406,7 +407,7 @@ async def upload_and_create_job(
enable_txt: Optional[bool] = Form(None, description="Generate TXT+MP3 package. Default: True if theme_id set, False otherwise"),
# Finalisation options
brand_prefix: Optional[str] = Form(None, description="Brand code prefix (e.g., NOMAD)"),
enable_youtube_upload: bool = Form(False, description="Upload to YouTube"),
enable_youtube_upload: Optional[bool] = Form(None, description="Upload to YouTube. None = use server default"),
youtube_description: Optional[str] = Form(None, description="YouTube video description text"),
discord_webhook_url: Optional[str] = Form(None, description="Discord webhook URL for notifications"),
webhook_url: Optional[str] = Form(None, description="Generic webhook URL"),
@@ -446,6 +447,14 @@ async def upload_and_create_job(
The style_params JSON can reference the uploaded images/fonts by their original
filenames, and the backend will update the paths to GCS locations.
"""
# Check tenant feature flag
tenant_config = get_tenant_config_from_request(request)
if tenant_config and not tenant_config.features.file_upload:
raise HTTPException(
status_code=403,
detail="File upload is not available for this portal"
)
try:
# Validate main audio file type
file_ext = Path(file.filename).suffix.lower()
@@ -510,12 +519,19 @@ async def upload_and_create_job(
if dist.brand_prefix and not brand_prefix:
logger.info(f"Using default brand_prefix: {dist.brand_prefix}")
# Apply YouTube upload default from settings
# Use explicit value if provided, otherwise fall back to server default
settings = get_settings()
effective_enable_youtube_upload = enable_youtube_upload if enable_youtube_upload is not None else settings.default_enable_youtube_upload
if effective_enable_youtube_upload and enable_youtube_upload is None:
logger.info("Using default enable_youtube_upload: True (from env)")
# Validate credentials for requested distribution services (including defaults)
# This prevents accepting jobs that will fail later due to missing credentials
invalid_services = []
credential_manager = get_credential_manager()
if enable_youtube_upload:
if effective_enable_youtube_upload:
result = credential_manager.check_youtube_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"youtube ({result.message})")
@@ -555,9 +571,18 @@ async def upload_and_create_job(
detail=f"Invalid color_overrides JSON: {e}"
)
# Apply default theme if none specified
# This ensures all karaoke videos use the Nomad theme by default
effective_theme_id = theme_id
if effective_theme_id is None:
theme_service = get_theme_service()
effective_theme_id = theme_service.get_default_theme_id()
if effective_theme_id:
logger.info(f"Applying default theme: {effective_theme_id}")
# Resolve CDG/TXT defaults based on theme
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
theme_id, enable_cdg, enable_txt
effective_theme_id, enable_cdg, enable_txt
)
# Check if any custom style files are being uploaded (overrides theme)
@@ -589,12 +614,12 @@ async def upload_and_create_job(
artist=artist,
title=title,
filename=file.filename,
theme_id=theme_id,
theme_id=effective_theme_id,
color_overrides=parsed_color_overrides,
enable_cdg=resolved_cdg,
enable_txt=resolved_txt,
brand_prefix=dist.brand_prefix,
enable_youtube_upload=enable_youtube_upload,
enable_youtube_upload=effective_enable_youtube_upload,
youtube_description=youtube_description,
discord_webhook_url=dist.discord_webhook_url,
webhook_url=webhook_url,
@@ -616,10 +641,12 @@ async def upload_and_create_job(
request_metadata=request_metadata,
# Non-interactive mode
non_interactive=non_interactive,
# Tenant scoping
tenant_id=tenant_config.id if tenant_config else None,
)
job = job_manager.create_job(job_create)
job_id = job.job_id
# Record job creation metric
metrics.record_job_created(job_id, source="upload")
@@ -631,16 +658,16 @@ async def upload_and_create_job(
theme_style_params_path = None
theme_style_assets = {}
theme_youtube_desc = None
if theme_id and not has_custom_style_files:
if effective_theme_id and not has_custom_style_files:
try:
theme_style_params_path, theme_style_assets, theme_youtube_desc = _prepare_theme_for_job(
job_id, theme_id, parsed_color_overrides or None
job_id, effective_theme_id, parsed_color_overrides or None
)
logger.info(f"Applied theme '{theme_id}' to job {job_id}")
logger.info(f"Applied theme '{effective_theme_id}' to job {job_id}")
except HTTPException:
raise # Re-raise validation errors (e.g., theme not found)
except Exception as e:
logger.warning(f"Failed to prepare theme '{theme_id}' for job {job_id}: {e}")
logger.warning(f"Failed to prepare theme '{effective_theme_id}' for job {job_id}: {e}")
# Continue without theme - job can still be processed with defaults
# Upload main audio file to GCS
@@ -815,11 +842,12 @@ async def upload_and_create_job(
"using_default": gdrive_folder_id is None,
}
if enable_youtube_upload:
if effective_enable_youtube_upload:
youtube_result = credential_manager.check_youtube_credentials()
distribution_services["youtube"] = {
"enabled": True,
"credentials_valid": youtube_result.status == CredentialStatus.VALID,
"using_default": enable_youtube_upload is None,
}
if dist.discord_webhook_url:
@@ -908,7 +936,7 @@ class CreateFinaliseOnlyJobRequest(BaseModel):
# Finalisation options
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
keep_brand_code: Optional[str] = Field(None, description="Preserve existing brand code from folder name")
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
enable_youtube_upload: Optional[bool] = Field(None, description="Upload to YouTube. None = use server default")
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
@@ -1012,6 +1040,14 @@ async def create_job_with_upload_urls(
- Works with any HTTP client (no HTTP/2 required)
- Resumable uploads possible with GCS
"""
# Check tenant feature flag
tenant_config = get_tenant_config_from_request(request)
if tenant_config and not tenant_config.features.file_upload:
raise HTTPException(
status_code=403,
detail="File upload is not available for this portal"
)
try:
# Validate files list
if not body.files:
@@ -1048,11 +1084,16 @@ async def create_job_with_upload_urls(
brand_prefix=body.brand_prefix,
)
# Apply YouTube upload default from settings
# Use explicit value if provided, otherwise fall back to server default
settings = get_settings()
effective_enable_youtube_upload = body.enable_youtube_upload if body.enable_youtube_upload is not None else settings.default_enable_youtube_upload
# Validate credentials for requested distribution services
invalid_services = []
credential_manager = get_credential_manager()
if body.enable_youtube_upload:
if effective_enable_youtube_upload:
result = credential_manager.check_youtube_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"youtube ({result.message})")
@@ -1084,9 +1125,18 @@ async def create_job_with_upload_urls(
# Get original audio filename
audio_file = audio_files[0]
# Apply default theme if none specified
# This ensures all karaoke videos use the Nomad theme by default
effective_theme_id = body.theme_id
if effective_theme_id is None:
theme_service = get_theme_service()
effective_theme_id = theme_service.get_default_theme_id()
if effective_theme_id:
logger.info(f"Applying default theme: {effective_theme_id}")
# Resolve CDG/TXT defaults based on theme
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
body.theme_id, body.enable_cdg, body.enable_txt
effective_theme_id, body.enable_cdg, body.enable_txt
)
# Check if style_params is being uploaded (overrides theme)
@@ -1100,12 +1150,12 @@ async def create_job_with_upload_urls(
artist=body.artist,
title=body.title,
filename=audio_file.filename,
theme_id=body.theme_id,
theme_id=effective_theme_id,
color_overrides=body.color_overrides or {},
enable_cdg=resolved_cdg,
enable_txt=resolved_txt,
brand_prefix=dist.brand_prefix,
enable_youtube_upload=body.enable_youtube_upload,
enable_youtube_upload=effective_enable_youtube_upload,
youtube_description=body.youtube_description,
youtube_description_template=body.youtube_description, # video_worker reads this field
discord_webhook_url=dist.discord_webhook_url,
@@ -1122,6 +1172,8 @@ async def create_job_with_upload_urls(
other_stems_models=body.other_stems_models,
request_metadata=request_metadata,
non_interactive=body.non_interactive,
# Tenant scoping
tenant_id=tenant_config.id if tenant_config else None,
)
job = job_manager.create_job(job_create)
job_id = job.job_id
@@ -1132,9 +1184,9 @@ async def create_job_with_upload_urls(
logger.info(f"Created job {job_id} for {body.artist} - {body.title} (signed URL upload flow)")
# If theme is set and no style_params uploaded, prepare theme style now
if body.theme_id and not has_style_params_upload:
if effective_theme_id and not has_style_params_upload:
style_params_path, style_assets, youtube_desc = _prepare_theme_for_job(
job_id, body.theme_id, body.color_overrides
job_id, effective_theme_id, body.color_overrides
)
# Update job with theme style data
update_data = {
@@ -1144,7 +1196,7 @@ async def create_job_with_upload_urls(
if youtube_desc and not body.youtube_description:
update_data['youtube_description_template'] = youtube_desc
job_manager.update_job(job_id, update_data)
logger.info(f"Applied theme '{body.theme_id}' to job {job_id}")
logger.info(f"Applied theme '{effective_theme_id}' to job {job_id}")
# Generate signed upload URLs for each file
upload_urls = []
@@ -1450,6 +1502,14 @@ async def create_job_from_url(
Note: YouTube rate limiting may cause occasional download failures.
The backend will retry automatically.
"""
# Check tenant feature flag
tenant_config = get_tenant_config_from_request(request)
if tenant_config and not tenant_config.features.youtube_url:
raise HTTPException(
status_code=403,
detail="URL-based job creation is not available for this portal"
)
try:
# Validate URL
if not _validate_url(body.url):
@@ -1466,11 +1526,16 @@ async def create_job_from_url(
brand_prefix=body.brand_prefix,
)
# Apply YouTube upload default from settings
# Use explicit value if provided, otherwise fall back to server default
settings = get_settings()
effective_enable_youtube_upload = body.enable_youtube_upload if body.enable_youtube_upload is not None else settings.default_enable_youtube_upload
# Validate credentials for requested distribution services
invalid_services = []
credential_manager = get_credential_manager()
if body.enable_youtube_upload:
if effective_enable_youtube_upload:
result = credential_manager.check_youtube_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"youtube ({result.message})")
@@ -1503,9 +1568,18 @@ async def create_job_from_url(
artist = body.artist
title = body.title
# Apply default theme if none specified
# This ensures all karaoke videos use the Nomad theme by default
effective_theme_id = body.theme_id
if effective_theme_id is None:
theme_service = get_theme_service()
effective_theme_id = theme_service.get_default_theme_id()
if effective_theme_id:
logger.info(f"Applying default theme: {effective_theme_id}")
# Resolve CDG/TXT defaults based on theme
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
body.theme_id, body.enable_cdg, body.enable_txt
effective_theme_id, body.enable_cdg, body.enable_txt
)
# Prefer authenticated user's email over request body
@@ -1517,12 +1591,12 @@ async def create_job_from_url(
artist=artist,
title=title,
filename=None, # No file uploaded
theme_id=body.theme_id,
theme_id=effective_theme_id,
color_overrides=body.color_overrides or {},
enable_cdg=resolved_cdg,
enable_txt=resolved_txt,
brand_prefix=dist.brand_prefix,
enable_youtube_upload=body.enable_youtube_upload,
enable_youtube_upload=effective_enable_youtube_upload,
youtube_description=body.youtube_description,
discord_webhook_url=dist.discord_webhook_url,
webhook_url=body.webhook_url,
@@ -1538,6 +1612,8 @@ async def create_job_from_url(
other_stems_models=body.other_stems_models,
request_metadata=request_metadata,
non_interactive=body.non_interactive,
# Tenant scoping
tenant_id=tenant_config.id if tenant_config else None,
)
job = job_manager.create_job(job_create)
job_id = job.job_id
@@ -1546,9 +1622,9 @@ async def create_job_from_url(
metrics.record_job_created(job_id, source="url")
# If theme is set, prepare theme style now
if body.theme_id:
if effective_theme_id:
style_params_path, style_assets, youtube_desc = _prepare_theme_for_job(
job_id, body.theme_id, body.color_overrides
job_id, effective_theme_id, body.color_overrides
)
# Update job with theme style data
update_data = {
@@ -1558,7 +1634,7 @@ async def create_job_from_url(
if youtube_desc and not body.youtube_description:
update_data['youtube_description_template'] = youtube_desc
job_manager.update_job(job_id, update_data)
logger.info(f"Applied theme '{body.theme_id}' to job {job_id}")
logger.info(f"Applied theme '{effective_theme_id}' to job {job_id}")
logger.info(f"Created URL-based job {job_id} for URL: {body.url}")
if artist:
@@ -1658,6 +1734,14 @@ async def create_finalise_only_job(
The endpoint returns signed URLs for uploading all the prep files.
"""
# Check tenant feature flag - finalise-only requires file upload capability
tenant_config = get_tenant_config_from_request(request)
if tenant_config and not tenant_config.features.file_upload:
raise HTTPException(
status_code=403,
detail="File upload is not available for this portal"
)
try:
# Validate files list
if not body.files:
@@ -1712,11 +1796,16 @@ async def create_finalise_only_job(
brand_prefix=body.brand_prefix,
)
# Apply YouTube upload default from settings
# Use explicit value if provided, otherwise fall back to server default
settings = get_settings()
effective_enable_youtube_upload = body.enable_youtube_upload if body.enable_youtube_upload is not None else settings.default_enable_youtube_upload
# Validate distribution credentials if services are requested
invalid_services = []
credential_manager = get_credential_manager()
if body.enable_youtube_upload:
if effective_enable_youtube_upload:
result = credential_manager.check_youtube_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"youtube ({result.message})")
@@ -1745,9 +1834,18 @@ async def create_finalise_only_job(
# Extract request metadata
request_metadata = extract_request_metadata(request, created_from="finalise_only_upload")
# Apply default theme if none specified
# This ensures all karaoke videos use the Nomad theme by default
effective_theme_id = body.theme_id
if effective_theme_id is None:
theme_service = get_theme_service()
effective_theme_id = theme_service.get_default_theme_id()
if effective_theme_id:
logger.info(f"Applying default theme: {effective_theme_id}")
# Resolve CDG/TXT defaults based on theme
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
body.theme_id, body.enable_cdg, body.enable_txt
effective_theme_id, body.enable_cdg, body.enable_txt
)
# Check if style_params is being uploaded (overrides theme)
@@ -1761,12 +1859,12 @@ async def create_finalise_only_job(
artist=body.artist,
title=body.title,
filename="finalise_only", # No single audio file - using prep outputs
theme_id=body.theme_id,
theme_id=effective_theme_id,
color_overrides=body.color_overrides or {},
enable_cdg=resolved_cdg,
enable_txt=resolved_txt,
brand_prefix=dist.brand_prefix,
enable_youtube_upload=body.enable_youtube_upload,
enable_youtube_upload=effective_enable_youtube_upload,
youtube_description=body.youtube_description,
discord_webhook_url=dist.discord_webhook_url,
dropbox_path=dist.dropbox_path,
@@ -1775,6 +1873,8 @@ async def create_finalise_only_job(
finalise_only=True,
keep_brand_code=body.keep_brand_code,
request_metadata=request_metadata,
# Tenant scoping
tenant_id=tenant_config.id if tenant_config else None,
)
job = job_manager.create_job(job_create)
job_id = job.job_id
@@ -1785,9 +1885,9 @@ async def create_finalise_only_job(
logger.info(f"Created finalise-only job {job_id} for {body.artist} - {body.title}")
# If theme is set and no style_params uploaded, prepare theme style now
if body.theme_id and not has_style_params_upload:
if effective_theme_id and not has_style_params_upload:
style_params_path, style_assets, youtube_desc = _prepare_theme_for_job(
job_id, body.theme_id, body.color_overrides
job_id, effective_theme_id, body.color_overrides
)
# Update job with theme style data
update_data = {
@@ -1797,7 +1897,7 @@ async def create_finalise_only_job(
if youtube_desc and not body.youtube_description:
update_data['youtube_description_template'] = youtube_desc
job_manager.update_job(job_id, update_data)
logger.info(f"Applied theme '{body.theme_id}' to job {job_id}")
logger.info(f"Applied theme '{effective_theme_id}' to job {job_id}")
# Generate signed upload URLs for each file
upload_urls = []
+99
View File
@@ -13,6 +13,9 @@ from backend.services.flacfetch_client import get_flacfetch_client
from backend.services.email_service import get_email_service
from backend.services.stripe_service import get_stripe_service
from backend.services.encoding_service import get_encoding_service
from backend.services.spacy_preloader import get_preloaded_model, is_model_preloaded
from backend.services.nltk_preloader import get_preloaded_cmudict, is_cmudict_preloaded
from backend.services.langfuse_preloader import get_preloaded_langfuse_handler, is_langfuse_preloaded, is_langfuse_configured
router = APIRouter()
logger = logging.getLogger(__name__)
@@ -227,6 +230,7 @@ async def check_encoding_worker_status() -> Dict[str, Any]:
"active_jobs": health.get("active_jobs", 0),
"queue_length": health.get("queue_length", 0),
"ffmpeg_version": health.get("ffmpeg_version"),
"wheel_version": health.get("wheel_version"),
}
except Exception as e:
return {
@@ -237,6 +241,39 @@ async def check_encoding_worker_status() -> Dict[str, Any]:
}
@router.get("/health/encoding-worker")
async def encoding_worker_health() -> Dict[str, Any]:
"""
Lightweight endpoint to check encoding worker status.
Returns minimal info for frontend footer display.
No authentication required.
"""
status = await check_encoding_worker_status()
# Return simplified response for frontend
if not status.get("configured"):
return {
"available": False,
"status": "not_configured",
}
if not status.get("available"):
return {
"available": False,
"status": "offline",
"error": status.get("error"),
}
return {
"available": True,
"status": "ok",
"version": status.get("wheel_version"),
"active_jobs": status.get("active_jobs", 0),
"queue_length": status.get("queue_length", 0),
}
@router.get("/health/detailed")
async def detailed_health_check() -> Dict[str, Any]:
"""
@@ -300,6 +337,68 @@ async def detailed_health_check() -> Dict[str, Any]:
}
@router.get("/health/preload-status")
async def preload_status() -> Dict[str, Any]:
"""
Check status of preloaded resources for performance optimization.
Use this endpoint to verify that NLTK, SpaCy, and Langfuse resources
were successfully preloaded at container startup. If any show as
not preloaded, check Cloud Run startup logs for errors.
Expected state after successful deployment:
- spacy.preloaded: true
- nltk.preloaded: true
- langfuse.preloaded: true (if configured) or configured: false
"""
# SpaCy status
spacy_model = get_preloaded_model("en_core_web_sm")
spacy_status = {
"preloaded": is_model_preloaded("en_core_web_sm"),
"model": "en_core_web_sm",
}
if spacy_model:
spacy_status["vocab_size"] = len(spacy_model.vocab)
# NLTK status
cmudict = get_preloaded_cmudict()
nltk_status = {
"preloaded": is_cmudict_preloaded(),
"resource": "cmudict",
}
if cmudict:
nltk_status["entries"] = len(cmudict)
# Langfuse status
langfuse_handler = get_preloaded_langfuse_handler()
langfuse_status = {
"configured": is_langfuse_configured(),
"preloaded": is_langfuse_preloaded(),
}
if langfuse_handler:
langfuse_status["handler_type"] = type(langfuse_handler).__name__
# Overall status
all_preloaded = (
spacy_status["preloaded"]
and nltk_status["preloaded"]
and (langfuse_status["preloaded"] or not langfuse_status["configured"])
)
return {
"status": "ok" if all_preloaded else "degraded",
"message": "All resources preloaded" if all_preloaded else "Some resources not preloaded - check startup logs",
"spacy": spacy_status,
"nltk": nltk_status,
"langfuse": langfuse_status,
"performance_impact": {
"spacy_preload": "Saves ~60s on first lyrics correction",
"nltk_preload": "Saves ~100-150s on SyllablesMatchHandler init",
"langfuse_preload": "Saves ~200s on AgenticCorrector init",
}
}
@router.get("/readiness")
async def readiness_check() -> Dict[str, str]:
"""Readiness check endpoint for Cloud Run."""
+106 -1
View File
@@ -316,13 +316,118 @@ async def trigger_render_video_worker(
)
@router.post("/jobs/{job_id}/check-idle-reminder")
async def check_idle_reminder(
job_id: str,
http_request: Request,
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
):
"""
Check if a job needs an idle reminder email.
This endpoint is called by a Cloud Tasks scheduled task 5 minutes after
a job enters a blocking state (AWAITING_REVIEW or AWAITING_INSTRUMENTAL_SELECTION).
If the job is still in the blocking state and no reminder has been sent yet,
sends a reminder email to the user.
Idempotency: Only one reminder per job (tracked via reminder_sent flag).
"""
from backend.models.job import JobStatus
from backend.services.job_notification_service import get_job_notification_service
# Extract trace context from incoming request
trace_context = extract_trace_context(dict(http_request.headers))
logger.info(f"[job:{job_id}] IDLE_REMINDER_CHECK starting")
add_span_attribute("job_id", job_id)
add_span_attribute("operation", "idle_reminder_check")
job_manager = JobManager()
job = job_manager.get_job(job_id)
if not job:
logger.warning(f"[job:{job_id}] Job not found for idle reminder check")
add_span_event("job_not_found")
return {"status": "not_found", "job_id": job_id, "message": "Job not found"}
# Check if job is still in a blocking state
blocking_states = [JobStatus.AWAITING_REVIEW, JobStatus.AWAITING_INSTRUMENTAL_SELECTION]
if job.status not in [s.value for s in blocking_states]:
logger.info(f"[job:{job_id}] Job no longer in blocking state ({job.status}), skipping reminder")
add_span_event("not_blocking", {"current_status": job.status})
return {
"status": "skipped",
"job_id": job_id,
"message": f"Job not in blocking state (current: {job.status})"
}
# Normalize state_data to prevent None errors
state_data = job.state_data or {}
# Check if reminder was already sent (idempotency)
if state_data.get('reminder_sent'):
logger.info(f"[job:{job_id}] Reminder already sent, skipping")
add_span_event("already_sent")
return {"status": "already_sent", "job_id": job_id, "message": "Reminder already sent"}
# Check if user has an email
if not job.user_email:
logger.warning(f"[job:{job_id}] No user email, cannot send reminder")
add_span_event("no_email")
return {"status": "no_email", "job_id": job_id, "message": "No user email configured"}
# Determine action type
action_type = state_data.get('blocking_action_type')
if not action_type:
action_type = "lyrics" if job.status == JobStatus.AWAITING_REVIEW.value else "instrumental"
# Send the reminder email
try:
notification_service = get_job_notification_service()
success = await notification_service.send_action_reminder_email(
job_id=job.job_id,
user_email=job.user_email,
action_type=action_type,
user_name=None, # Could fetch from user service if needed
artist=job.artist,
title=job.title,
audio_hash=job.audio_hash,
review_token=job.review_token,
instrumental_token=job.instrumental_token,
)
if success:
# Mark reminder as sent (prevents duplicate sends)
job_manager.firestore.update_job(job_id, {
'state_data': {**state_data, 'reminder_sent': True}
})
logger.info(f"[job:{job_id}] Sent {action_type} reminder email to {job.user_email}")
add_span_event("reminder_sent", {"action_type": action_type})
return {
"status": "sent",
"job_id": job_id,
"message": f"Sent {action_type} reminder to {job.user_email}"
}
else:
logger.error(f"[job:{job_id}] Failed to send reminder email")
add_span_event("send_failed")
return {"status": "failed", "job_id": job_id, "message": "Failed to send reminder"}
except Exception as e:
logger.exception(f"[job:{job_id}] Error sending reminder: {e}")
add_span_event("error", {"error": str(e)})
return {"status": "error", "job_id": job_id, "message": str(e)}
@router.get("/health")
async def internal_health(
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
):
"""
Internal health check endpoint.
Used to verify the internal API is responsive.
Requires admin authentication.
"""
+39 -17
View File
@@ -11,7 +11,7 @@ import asyncio
import logging
import httpx
from typing import List, Optional, Dict, Any, Tuple
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends, Request
from backend.models.job import Job, JobCreate, JobResponse, JobStatus
from backend.models.requests import (
@@ -25,10 +25,13 @@ from backend.models.requests import (
from backend.services.job_manager import JobManager
from backend.services.worker_service import get_worker_service
from backend.services.storage_service import StorageService
from backend.services.theme_service import get_theme_service
from backend.config import get_settings
from backend.api.dependencies import require_admin, require_auth, require_instrumental_auth
from backend.services.auth_service import UserType, AuthResult
from backend.services.metrics import metrics
from backend.middleware.tenant import get_tenant_from_request
from backend.utils.test_data import is_test_email
logger = logging.getLogger(__name__)
@@ -87,14 +90,30 @@ async def create_job(
user_email = request.user_email
logger.info(f"Admin {auth_result.user_email} creating job on behalf of {user_email}")
# Apply YouTube upload default from settings
# Use explicit value if provided, otherwise fall back to server default
settings = get_settings()
effective_enable_youtube_upload = request.enable_youtube_upload if request.enable_youtube_upload is not None else settings.default_enable_youtube_upload
# Apply default theme - all jobs require a theme
theme_service = get_theme_service()
effective_theme_id = theme_service.get_default_theme_id()
if not effective_theme_id:
raise HTTPException(
status_code=422,
detail="No default theme configured. Please contact support or specify a theme_id."
)
logger.info(f"Applying default theme: {effective_theme_id}")
# Create job with all preferences
job_create = JobCreate(
url=str(request.url),
artist=request.artist,
title=request.title,
theme_id=effective_theme_id, # Required - all jobs must have a theme
enable_cdg=request.enable_cdg,
enable_txt=request.enable_txt,
enable_youtube_upload=request.enable_youtube_upload,
enable_youtube_upload=effective_enable_youtube_upload,
youtube_description=request.youtube_description,
webhook_url=request.webhook_url,
user_email=user_email
@@ -172,11 +191,13 @@ def _check_job_ownership(job: Job, auth_result: AuthResult) -> bool:
@router.get("", response_model=List[Job])
async def list_jobs(
request: Request,
status: Optional[JobStatus] = None,
environment: Optional[str] = None,
client_id: Optional[str] = None,
created_after: Optional[str] = None,
created_before: Optional[str] = None,
exclude_test: bool = True,
limit: int = 100,
auth_result: AuthResult = Depends(require_auth)
) -> List[Job]:
@@ -184,6 +205,7 @@ async def list_jobs(
List jobs with optional filters.
Regular users only see their own jobs. Admins see all jobs.
Users on tenant portals only see jobs from their tenant.
Args:
status: Filter by job status (pending, complete, failed, etc.)
@@ -191,6 +213,7 @@ async def list_jobs(
client_id: Filter by request_metadata.client_id (customer identifier)
created_after: Filter jobs created after this ISO datetime (e.g., 2024-01-01T00:00:00Z)
created_before: Filter jobs created before this ISO datetime
exclude_test: If True (default), exclude jobs from test users (admin only)
limit: Maximum number of jobs to return (default 100)
Returns:
@@ -227,6 +250,10 @@ async def list_jobs(
logger.warning("Non-admin auth without user_email, returning empty job list")
return []
# Get tenant_id from request for portal scoping
# Tenant users only see jobs from their tenant
tenant_id = get_tenant_from_request(request)
jobs = job_manager.list_jobs(
status=status,
environment=environment,
@@ -234,9 +261,14 @@ async def list_jobs(
created_after=created_after_dt,
created_before=created_before_dt,
user_email=user_email_filter,
tenant_id=tenant_id,
limit=limit
)
# Filter out test user jobs if exclude_test is True (admin only)
if exclude_test and auth_result.is_admin:
jobs = [j for j in jobs if not is_test_email(j.user_email or "")]
logger.debug(f"Listed {len(jobs)} jobs for user={auth_result.user_email}, admin={auth_result.is_admin}")
return jobs
except HTTPException:
@@ -1159,21 +1191,11 @@ async def download_file(
content_type = content_types.get(ext, 'application/octet-stream')
# Build proper filename: "Artist - Title (Final Karaoke Lossy 4k).mp4"
# Sanitize artist/title to remove filesystem-unsafe characters
def sanitize_filename_part(s: str) -> str:
if not s:
return ""
# Remove/replace characters unsafe for filesystems: / \ : * ? " < > |
unsafe_chars = r'[/\\:*?"<>|]'
import re
sanitized = re.sub(unsafe_chars, '', s)
# Collapse multiple spaces and trim
sanitized = ' '.join(sanitized.split())
# Limit length to prevent overly long filenames
return sanitized[:100].strip()
artist_clean = sanitize_filename_part(job.artist) if job.artist else None
title_clean = sanitize_filename_part(job.title) if job.title else None
# Use sanitize_filename to handle Unicode characters (curly quotes, em dashes, etc.)
# that cause HTTP header encoding issues (Content-Disposition uses latin-1)
from karaoke_gen.utils import sanitize_filename
artist_clean = sanitize_filename(job.artist) if job.artist else None
title_clean = sanitize_filename(job.title) if job.title else None
base_name = f"{artist_clean} - {title_clean}" if artist_clean and title_clean else None
if base_name and file_key in DOWNLOAD_FILENAME_SUFFIXES:
+122 -40
View File
@@ -27,8 +27,10 @@ from backend.services.job_manager import JobManager
from backend.services.storage_service import StorageService
from backend.services.job_logging import job_log_context, JobLogger
from backend.services.tracing import create_span, add_span_attribute, add_span_event
from backend.services.encoding_service import get_encoding_service
from backend.api.dependencies import require_auth, require_review_auth
from backend.services.auth_service import UserType
from backend.config import get_settings
# LyricsTranscriber imports for preview generation
from lyrics_transcriber.types import CorrectionResult
@@ -385,28 +387,35 @@ async def generate_preview_video(
):
"""
Generate a preview video with the current corrections.
Uses the LyricsTranscriber's CorrectionOperations to generate a 360p preview
video with the user's current corrections applied.
When USE_GCE_PREVIEW_ENCODING is enabled, video encoding is offloaded to
the high-performance GCE worker for faster generation (15-20s vs 60+s).
"""
job_manager = JobManager()
storage = StorageService()
settings = get_settings()
encoding_service = get_encoding_service()
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Job must be in review state to generate preview
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
raise HTTPException(
status_code=400,
detail=f"Job not in review state (current status: {job.status})"
)
logger.info(f"Job {job_id}: Generating preview video")
# Check if GCE preview encoding is enabled
use_gce_preview = encoding_service.is_preview_enabled
logger.info(f"Job {job_id}: Generating preview video (GCE preview: {use_gce_preview})")
# Use tracing and job_log_context for full observability
with create_span("generate-preview-video", {"job_id": job_id}) as span:
with create_span("generate-preview-video", {"job_id": job_id, "use_gce": use_gce_preview}) as span:
with job_log_context(job_id, worker="preview"):
try:
# Create temp directory for this preview operation
@@ -416,19 +425,19 @@ async def generate_preview_video(
corrections_gcs = f"jobs/{job_id}/lyrics/corrections.json"
corrections_path = os.path.join(temp_dir, "corrections.json")
storage.download_file(corrections_gcs, corrections_path)
with open(corrections_path, 'r', encoding='utf-8') as f:
original_data = json.load(f)
# 2. Download input audio
audio_path = os.path.join(temp_dir, "audio.flac")
storage.download_file(job.input_media_gcs_path, audio_path)
download_span.set_attribute("audio_gcs_path", job.input_media_gcs_path)
# 3. Load original as CorrectionResult
correction_result = CorrectionResult.from_dict(original_data)
add_span_event("corrections_loaded")
# 4. Get or create styles file for preview using unified style loader
with create_span("load-styles") as styles_span:
styles_path, _ = load_styles_from_gcs(
@@ -439,54 +448,127 @@ async def generate_preview_video(
logger=logger,
)
styles_span.set_attribute("styles_path", styles_path)
# 5. Set up output config for preview
output_dir = os.path.join(temp_dir, "output")
cache_dir = os.path.join(temp_dir, "cache")
os.makedirs(output_dir, exist_ok=True)
os.makedirs(cache_dir, exist_ok=True)
output_config = OutputConfig(
output_styles_json=styles_path,
output_dir=output_dir,
cache_dir=cache_dir,
video_resolution="360p",
)
# 6. Generate preview video using CorrectionOperations (heavy operation)
with create_span("render-preview-video") as render_span:
render_span.set_attribute("resolution", "360p")
result = CorrectionOperations.generate_preview_video(
correction_result=correction_result,
updated_data=updated_data,
output_config=output_config,
audio_filepath=audio_path,
artist=job.artist,
title=job.title,
logger=logger
)
add_span_event("render_complete")
# 7. Upload preview video to GCS for serving
preview_hash = result["preview_hash"]
video_path = result["video_path"]
with create_span("upload-preview-video") as upload_span:
preview_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.mp4"
storage.upload_file(video_path, preview_gcs_path)
upload_span.set_attribute("gcs_path", preview_gcs_path)
# 6. Generate preview (ASS-only if using GCE, or full video if local)
preview_gcs_path = None
if use_gce_preview:
# GCE path: Generate ASS only, then offload encoding to GCE
try:
with create_span("generate-ass-subtitles") as ass_span:
result = CorrectionOperations.generate_preview_video(
correction_result=correction_result,
updated_data=updated_data,
output_config=output_config,
audio_filepath=audio_path,
artist=job.artist,
title=job.title,
logger=logger,
ass_only=True, # Only generate ASS, skip video encoding
)
preview_hash = result["preview_hash"]
ass_path = result["ass_path"]
ass_span.set_attribute("ass_path", ass_path)
add_span_event("ass_generated")
# Upload ASS to GCS
with create_span("upload-ass-to-gcs") as upload_ass_span:
ass_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.ass"
storage.upload_file(ass_path, ass_gcs_path)
upload_ass_span.set_attribute("ass_gcs_path", ass_gcs_path)
# Call GCE encoding service
with create_span("gce-preview-encoding") as gce_span:
bucket_name = settings.gcs_bucket_name
preview_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.mp4"
# Get background image and font from style assets if available
style_assets = job.style_assets or {}
background_image_gcs_path = None
for key in ["karaoke_background", "style_karaoke_background"]:
if key in style_assets:
background_image_gcs_path = f"gs://{bucket_name}/{style_assets[key]}"
gce_span.set_attribute("background_image", background_image_gcs_path)
break
font_gcs_path = None
for key in ["font", "style_font"]:
if key in style_assets:
font_gcs_path = f"gs://{bucket_name}/{style_assets[key]}"
gce_span.set_attribute("font", font_gcs_path)
break
gce_result = await encoding_service.encode_preview_video(
job_id=f"preview_{job_id}_{preview_hash}",
ass_gcs_path=f"gs://{bucket_name}/{ass_gcs_path}",
audio_gcs_path=f"gs://{bucket_name}/{job.input_media_gcs_path}",
output_gcs_path=f"gs://{bucket_name}/{preview_gcs_path}",
background_color="black",
background_image_gcs_path=background_image_gcs_path,
font_gcs_path=font_gcs_path,
)
gce_span.set_attribute("gce_status", gce_result.get("status"))
add_span_event("gce_encoding_complete")
logger.info(f"Job {job_id}: Preview generated via GCE: {preview_hash}")
except Exception as gce_error:
# Fall back to local encoding if GCE fails
logger.warning(
f"Job {job_id}: GCE preview encoding failed, falling back to local: {gce_error}"
)
span.set_attribute("gce_fallback", True)
use_gce_preview = False # Fall through to local encoding below
if not use_gce_preview:
# Local path: Generate full preview video locally
with create_span("render-preview-video-local") as render_span:
render_span.set_attribute("resolution", "360p")
result = CorrectionOperations.generate_preview_video(
correction_result=correction_result,
updated_data=updated_data,
output_config=output_config,
audio_filepath=audio_path,
artist=job.artist,
title=job.title,
logger=logger,
ass_only=False, # Generate full video locally
)
preview_hash = result["preview_hash"]
video_path = result["video_path"]
add_span_event("render_complete")
# Upload preview video to GCS
with create_span("upload-preview-video") as upload_span:
preview_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.mp4"
storage.upload_file(video_path, preview_gcs_path)
upload_span.set_attribute("gcs_path", preview_gcs_path)
# Store the GCS path for serving
if job_id not in _preview_videos:
_preview_videos[job_id] = {}
_preview_videos[job_id][preview_hash] = preview_gcs_path
logger.info(f"Job {job_id}: Preview video generated: {preview_hash}")
span.set_attribute("preview_hash", preview_hash)
span.set_attribute("success", True)
return {"status": "success", "preview_hash": preview_hash}
except Exception as e:
logger.error(f"Job {job_id}: Failed to generate preview video: {e}", exc_info=True)
span.set_attribute("error", str(e))
+120
View File
@@ -0,0 +1,120 @@
"""
Tenant API routes for white-label portal configuration.
These endpoints are public (no auth required) since the frontend needs
to fetch tenant branding before the user logs in.
"""
import logging
from typing import Optional
from fastapi import APIRouter, Query, Request
from backend.models.tenant import TenantConfigResponse, TenantPublicConfig
from backend.services.tenant_service import get_tenant_service
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/tenant", tags=["tenant"])
@router.get("/config", response_model=TenantConfigResponse)
async def get_tenant_config(
request: Request,
tenant: Optional[str] = Query(
None,
description="Tenant ID override (for development). If not provided, detected from subdomain.",
),
):
"""
Get tenant configuration for the current portal.
This endpoint detects the tenant from:
1. Query parameter `tenant` (for development/testing)
2. X-Tenant-ID header (set by frontend)
3. Host header subdomain
Returns the public tenant configuration (branding, features, defaults)
or indicates this is the default Nomad Karaoke portal.
"""
tenant_service = get_tenant_service()
# Priority 1: Query parameter (for dev/testing)
tenant_id = tenant
# Priority 2: X-Tenant-ID header
if not tenant_id:
tenant_id = request.headers.get("X-Tenant-ID")
# Priority 3: Detect from Host header
if not tenant_id:
host = request.headers.get("Host", "")
# Check if this is a tenant subdomain
# Pattern: {tenant}.nomadkaraoke.com or {tenant}.gen.nomadkaraoke.com
if host and "nomadkaraoke.com" in host.lower():
parts = host.lower().split(".")
# Skip known non-tenant subdomains
if parts[0] not in ["gen", "api", "www", "buy", "admin"]:
potential_tenant = parts[0]
if tenant_service.tenant_exists(potential_tenant):
tenant_id = potential_tenant
# If no tenant detected, return default config
if not tenant_id:
logger.debug("No tenant detected, returning default config")
return TenantConfigResponse(tenant=None, is_default=True)
# Load tenant config
public_config = tenant_service.get_public_config(tenant_id)
if not public_config:
logger.warning(f"Tenant not found: {tenant_id}")
return TenantConfigResponse(tenant=None, is_default=True)
if not public_config.is_active:
logger.warning(f"Tenant is inactive: {tenant_id}")
return TenantConfigResponse(tenant=None, is_default=True)
logger.info(f"Returning config for tenant: {tenant_id}")
return TenantConfigResponse(tenant=public_config, is_default=False)
@router.get("/config/{tenant_id}", response_model=TenantConfigResponse)
async def get_tenant_config_by_id(tenant_id: str):
"""
Get tenant configuration by explicit tenant ID.
This is useful for admin tools or debugging.
"""
tenant_service = get_tenant_service()
public_config = tenant_service.get_public_config(tenant_id)
if not public_config:
return TenantConfigResponse(tenant=None, is_default=True)
# Check if tenant is active
if not public_config.is_active:
logger.warning(f"Tenant is inactive: {tenant_id}")
return TenantConfigResponse(tenant=None, is_default=True)
return TenantConfigResponse(tenant=public_config, is_default=False)
@router.get("/asset/{tenant_id}/{asset_name}")
async def get_tenant_asset(tenant_id: str, asset_name: str):
"""
Get a signed URL for a tenant asset (logo, favicon, etc.).
This redirects to the signed GCS URL for the asset.
"""
from fastapi.responses import RedirectResponse
tenant_service = get_tenant_service()
url = tenant_service.get_asset_url(tenant_id, asset_name)
if not url:
from fastapi import HTTPException
raise HTTPException(status_code=404, detail="Asset not found")
return RedirectResponse(url=url)
+598 -49
View File
@@ -8,8 +8,28 @@ Handles:
- Stripe checkout and webhooks
- Admin user management
"""
import hashlib
import logging
from typing import Optional, Tuple
def _mask_email(email: str) -> str:
"""Mask email for logging to avoid PII exposure.
Example: test@example.com -> te***@ex***.com
"""
if not email or "@" not in email:
return "***"
local, domain = email.split("@", 1)
masked_local = local[:2] + "***" if len(local) > 2 else "***"
domain_parts = domain.split(".")
if len(domain_parts) >= 2:
masked_domain = domain_parts[0][:2] + "***." + domain_parts[-1]
else:
masked_domain = "***"
return f"{masked_local}@{masked_domain}"
from fastapi import APIRouter, HTTPException, Depends, Request, Header
from pydantic import BaseModel, EmailStr
@@ -32,8 +52,11 @@ from backend.models.user import (
from backend.services.user_service import get_user_service, UserService, USERS_COLLECTION
from backend.services.email_service import get_email_service, EmailService
from backend.services.stripe_service import get_stripe_service, StripeService, CREDIT_PACKAGES
from backend.services.theme_service import get_theme_service
from backend.api.dependencies import require_admin
from backend.api.routes.file_upload import _prepare_theme_for_job
from backend.services.auth_service import UserType
from backend.utils.test_data import is_test_email
logger = logging.getLogger(__name__)
@@ -57,6 +80,16 @@ class CreateCheckoutResponse(BaseModel):
message: str
class MadeForYouCheckoutRequest(BaseModel):
"""Request to create a made-for-you karaoke video order."""
email: EmailStr
artist: str
title: str
source_type: str = "search" # search, youtube, or upload
youtube_url: Optional[str] = None
notes: Optional[str] = None
class CreditPackage(BaseModel):
"""Credit package information."""
id: str
@@ -99,7 +132,11 @@ async def send_magic_link(
The user will receive an email with a link that logs them in.
Links expire after 15 minutes and can only be used once.
For white-label tenants, validates email domain against tenant config.
"""
from backend.middleware.tenant import get_tenant_from_request, get_tenant_config_from_request
# Check if email service is configured
if not email_service.is_configured():
logger.error("Email service not configured - cannot send magic links")
@@ -110,19 +147,39 @@ async def send_magic_link(
email = request.email.lower()
# Get tenant context from middleware
tenant_id = get_tenant_from_request(http_request)
tenant_config = get_tenant_config_from_request(http_request)
# Validate email domain for tenant if configured
if tenant_config and tenant_config.auth.allowed_email_domains:
if not tenant_config.is_email_allowed(email):
logger.warning(f"Email domain not allowed for tenant {tenant_id}: {_mask_email(email)}")
# Return success anyway to prevent email enumeration
return SendMagicLinkResponse(
status="success",
message="If this email is registered, you will receive a sign-in link shortly."
)
# Get client info for security logging
ip_address = http_request.client.host if http_request.client else None
user_agent = http_request.headers.get("user-agent")
# Create magic link token
# Create magic link token with tenant context
magic_link = user_service.create_magic_link(
email,
ip_address=ip_address,
user_agent=user_agent
user_agent=user_agent,
tenant_id=tenant_id
)
# Send email
sent = email_service.send_magic_link(email, magic_link.token)
# Get tenant-specific sender email
sender_email = None
if tenant_config:
sender_email = tenant_config.get_sender_email()
# Send email (with tenant-specific sender if configured)
sent = email_service.send_magic_link(email, magic_link.token, sender_email=sender_email)
if not sent:
logger.error(f"Failed to send magic link email to {email}")
@@ -146,6 +203,7 @@ async def verify_magic_link(
Verify a magic link token and create a session.
Returns a session token that should be stored and used for subsequent requests.
The session will be associated with the tenant from the magic link.
"""
# Get client info
ip_address = http_request.client.host if http_request.client else None
@@ -153,11 +211,14 @@ async def verify_magic_link(
# Check if this is a first login BEFORE verification (which sets last_login_at)
# We need to get the user's state before verify_magic_link updates it
# Also extract tenant_id from the magic link for session creation
from backend.services.user_service import MAGIC_LINKS_COLLECTION
magic_link_doc = user_service.db.collection(MAGIC_LINKS_COLLECTION).document(token).get()
is_first_login = False
magic_link_tenant_id = None
if magic_link_doc.exists:
magic_link_data = magic_link_doc.to_dict()
magic_link_tenant_id = magic_link_data.get('tenant_id')
pre_verify_user = user_service.get_user(magic_link_data.get('email', ''))
if pre_verify_user:
is_first_login = pre_verify_user.total_jobs_created == 0 and not pre_verify_user.last_login_at
@@ -168,18 +229,19 @@ async def verify_magic_link(
if not success or not user:
raise HTTPException(status_code=401, detail=message)
# Create session
# Create session with tenant context from the magic link
session = user_service.create_session(
user.email,
ip_address=ip_address,
user_agent=user_agent
user_agent=user_agent,
tenant_id=magic_link_tenant_id
)
# Send welcome email to first-time users
if is_first_login:
email_service.send_welcome_email(user.email, user.credits)
# Return user info
# Return user info with tenant_id
user_public = UserPublic(
email=user.email,
role=user.role,
@@ -187,6 +249,7 @@ async def verify_magic_link(
display_name=user.display_name,
total_jobs_created=user.total_jobs_created,
total_jobs_completed=user.total_jobs_completed,
tenant_id=user.tenant_id,
)
return VerifyMagicLinkResponse(
@@ -327,10 +390,459 @@ async def create_checkout(
)
@router.post("/made-for-you/checkout", response_model=CreateCheckoutResponse)
async def create_made_for_you_checkout(
request: MadeForYouCheckoutRequest,
stripe_service: StripeService = Depends(get_stripe_service),
):
"""
Create a Stripe checkout session for a made-for-you karaoke video order.
This is the full-service option where Nomad Karaoke handles everything:
- Finding or processing the audio
- Reviewing and correcting lyrics
- Selecting the best instrumental
- Generating the final video
$15 with 24-hour delivery guarantee.
No authentication required - customer email is provided in the request.
"""
if not stripe_service.is_configured():
raise HTTPException(status_code=503, detail="Payment processing is not available")
success, checkout_url, message = stripe_service.create_made_for_you_checkout_session(
customer_email=request.email,
artist=request.artist,
title=request.title,
source_type=request.source_type,
youtube_url=request.youtube_url,
notes=request.notes,
)
if not success or not checkout_url:
raise HTTPException(status_code=400, detail=message)
return CreateCheckoutResponse(
status="success",
checkout_url=checkout_url,
message=message,
)
# =============================================================================
# Stripe Webhooks
# =============================================================================
# Admin email for made-for-you order notifications
ADMIN_EMAIL = "andrew@nomadkaraoke.com"
async def _handle_made_for_you_order(
session_id: str,
metadata: dict,
user_service: UserService,
email_service: EmailService,
) -> None:
"""
Handle a completed made-for-you order by creating a job and notifying Andrew.
For orders with a YouTube URL, the job is created and workers are triggered immediately.
For orders without a URL (search mode), the audio search flow is used to find and
download the best matching audio source automatically.
Args:
session_id: Stripe checkout session ID
metadata: Order metadata from Stripe session
user_service: User service for marking session processed
email_service: Email service for notifications
"""
from backend.models.job import JobCreate, JobStatus
from backend.services.job_manager import JobManager
from backend.services.worker_service import get_worker_service
from backend.services.audio_search_service import (
get_audio_search_service,
NoResultsError,
AudioSearchError,
)
from backend.services.storage_service import StorageService
import asyncio
import tempfile
import os
customer_email = metadata.get("customer_email", "")
artist = metadata.get("artist", "Unknown Artist")
title = metadata.get("title", "Unknown Title")
source_type = metadata.get("source_type", "search")
youtube_url = metadata.get("youtube_url")
notes = metadata.get("notes", "")
logger.info(
f"Processing made-for-you order: {artist} - {title} for {customer_email} "
f"(session: {session_id}, source_type: {source_type})"
)
try:
job_manager = JobManager()
worker_service = get_worker_service()
storage_service = StorageService()
# Apply default theme (Nomad) - same as audio_search endpoint
theme_service = get_theme_service()
effective_theme_id = theme_service.get_default_theme_id()
if effective_theme_id:
logger.info(f"Applying default theme '{effective_theme_id}' for made-for-you order")
# Create job for the customer
# Note: made-for-you jobs should NOT be non_interactive - Andrew needs to review
job_create = JobCreate(
url=youtube_url if youtube_url else None,
artist=artist,
title=title,
user_email=customer_email, # Customer owns the job
theme_id=effective_theme_id, # Apply default theme
non_interactive=False, # Andrew will review lyrics/instrumental
# Set audio search fields for search-based orders
audio_search_artist=artist if not youtube_url else None,
audio_search_title=title if not youtube_url else None,
auto_download=True, # Auto-select best audio source
)
job = job_manager.create_job(job_create)
job_id = job.job_id
logger.info(f"Created made-for-you job {job_id} for {customer_email}")
# Prepare theme style assets for the job (same as audio_search endpoint)
if effective_theme_id:
try:
style_params_path, theme_style_assets, youtube_desc = _prepare_theme_for_job(
job_id, effective_theme_id, None # No color overrides for made-for-you
)
theme_update = {
'style_params_gcs_path': style_params_path,
'style_assets': theme_style_assets,
}
if youtube_desc:
theme_update['youtube_description_template'] = youtube_desc
job_manager.update_job(job_id, theme_update)
logger.info(f"Applied theme '{effective_theme_id}' to made-for-you job {job_id}")
except Exception as e:
logger.warning(f"Failed to prepare theme for made-for-you job {job_id}: {e}")
# Mark session as processed for idempotency
# Note: Using internal method since this isn't a credit transaction
user_service._mark_stripe_session_processed(
stripe_session_id=session_id,
email=customer_email,
amount=0 # No credits, just tracking the session
)
# Handle based on whether we have a YouTube URL or need to search
if youtube_url:
# URL provided - trigger workers directly
logger.info(f"Job {job_id}: YouTube URL provided, triggering workers")
await asyncio.gather(
worker_service.trigger_audio_worker(job_id),
worker_service.trigger_lyrics_worker(job_id)
)
else:
# No URL - use audio search flow with auto_download
logger.info(f"Job {job_id}: No URL, using audio search for '{artist} - {title}'")
# Update job with audio search fields
job_manager.update_job(job_id, {
'audio_search_artist': artist,
'audio_search_title': title,
'auto_download': True,
})
# Transition to searching state
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.SEARCHING_AUDIO,
progress=5,
message=f"Searching for audio: {artist} - {title}"
)
# Perform audio search
audio_search_service = get_audio_search_service()
try:
search_results = audio_search_service.search(artist, title)
except NoResultsError as e:
# No results found - transition to AWAITING_AUDIO_SELECTION so Andrew can handle manually
logger.warning(f"Job {job_id}: No audio sources found for '{artist} - {title}'")
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_AUDIO_SELECTION,
progress=10,
message=f"No automatic audio sources found. Manual intervention required."
)
# Don't fail the job - Andrew can manually provide audio
search_results = None
except AudioSearchError as e:
logger.error(f"Job {job_id}: Audio search failed: {e}")
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_AUDIO_SELECTION,
progress=10,
message=f"Audio search error. Manual intervention required."
)
search_results = None
if search_results:
# Store search results in state_data
results_dicts = [r.to_dict() for r in search_results]
state_data_update = {
'audio_search_results': results_dicts,
'audio_search_count': len(results_dicts),
}
if audio_search_service.last_remote_search_id:
state_data_update['remote_search_id'] = audio_search_service.last_remote_search_id
job_manager.update_job(job_id, {'state_data': state_data_update})
# Auto-select best result and download
best_index = audio_search_service.select_best(search_results)
selected = results_dicts[best_index]
logger.info(f"Job {job_id}: Auto-selected result {best_index}: {selected.get('provider')} - {selected.get('title')}")
# Transition to downloading state
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.DOWNLOADING_AUDIO,
progress=10,
message=f"Downloading from {selected.get('provider')}: {selected.get('artist')} - {selected.get('title')}",
state_data_updates={
'selected_audio_index': best_index,
'selected_audio_provider': selected.get('provider'),
}
)
# Download audio
try:
is_torrent_source = selected.get('provider') in ['RED', 'OPS']
is_remote_enabled = audio_search_service.is_remote_enabled()
source_id = selected.get('source_id')
source_name = selected.get('provider')
target_file = selected.get('target_file')
download_url = selected.get('url')
remote_search_id = state_data_update.get('remote_search_id')
if is_torrent_source and is_remote_enabled:
# Remote torrent download - upload directly to GCS
gcs_destination = f"uploads/{job_id}/audio/"
if source_id and source_name:
result = audio_search_service.download_by_id(
source_name=source_name,
source_id=source_id,
output_dir="",
target_file=target_file,
download_url=download_url,
gcs_path=gcs_destination,
)
else:
result = audio_search_service.download(
result_index=best_index,
output_dir="",
gcs_path=gcs_destination,
remote_search_id=remote_search_id,
)
# Extract GCS path
if result.filepath.startswith("gs://"):
parts = result.filepath.replace("gs://", "").split("/", 1)
audio_gcs_path = parts[1] if len(parts) == 2 else result.filepath
filename = os.path.basename(result.filepath)
else:
filename = os.path.basename(result.filepath)
audio_gcs_path = f"uploads/{job_id}/audio/{filename}"
else:
# Local download (YouTube or local torrent)
temp_dir = tempfile.mkdtemp(prefix=f"audio_download_{job_id}_")
import shutil
try:
if source_id and source_name and is_remote_enabled:
result = audio_search_service.download_by_id(
source_name=source_name,
source_id=source_id,
output_dir=temp_dir,
target_file=target_file,
download_url=download_url,
)
elif source_name == 'YouTube' and download_url:
# YouTube download
from backend.workers.audio_worker import download_from_url
local_path = await download_from_url(
download_url,
temp_dir,
selected.get('artist'),
selected.get('title')
)
if not local_path or not os.path.exists(local_path):
raise Exception(f"Failed to download from YouTube: {download_url}")
class DownloadResult:
def __init__(self, filepath):
self.filepath = filepath
result = DownloadResult(local_path)
else:
result = audio_search_service.download(
result_index=best_index,
output_dir=temp_dir,
remote_search_id=remote_search_id,
)
# Upload to GCS
filename = os.path.basename(result.filepath)
audio_gcs_path = f"uploads/{job_id}/audio/{filename}"
with open(result.filepath, 'rb') as f:
storage_service.upload_fileobj(f, audio_gcs_path, content_type='audio/flac')
finally:
# Always cleanup temp directory
shutil.rmtree(temp_dir, ignore_errors=True)
# Update job with GCS path
job_manager.update_job(job_id, {
'input_media_gcs_path': audio_gcs_path,
'filename': filename,
})
# Transition to DOWNLOADING and trigger workers
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.DOWNLOADING,
progress=15,
message="Audio downloaded, starting processing"
)
# Trigger workers
await asyncio.gather(
worker_service.trigger_audio_worker(job_id),
worker_service.trigger_lyrics_worker(job_id)
)
logger.info(f"Job {job_id}: Audio downloaded and workers triggered")
except Exception as download_error:
logger.error(f"Job {job_id}: Audio download failed: {download_error}")
# Don't fail job - transition to awaiting selection so Andrew can handle
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_AUDIO_SELECTION,
progress=10,
message=f"Auto-download failed: {download_error}. Manual intervention required."
)
# Send confirmation email to customer
email_service.send_email(
to_email=customer_email,
subject=f"Your Karaoke Video Order: {artist} - {title}",
html_content=f"""
<h2>Thank you for your order!</h2>
<p>We've received your request for a karaoke video:</p>
<ul>
<li><strong>Artist:</strong> {artist}</li>
<li><strong>Title:</strong> {title}</li>
{f'<li><strong>Notes:</strong> {notes}</li>' if notes else ''}
</ul>
<p>Our team will review and create your video within <strong>24 hours</strong>.</p>
<p>You'll receive another email with download links when your video is ready.</p>
<p>If you have any questions, reply to this email or contact us at support@nomadkaraoke.com</p>
<p>Thanks for using Nomad Karaoke!</p>
""",
text_content=f"""
Thank you for your order!
We've received your request for a karaoke video:
- Artist: {artist}
- Title: {title}
{f'- Notes: {notes}' if notes else ''}
Our team will review and create your video within 24 hours.
You'll receive another email with download links when your video is ready.
If you have any questions, reply to this email or contact us at support@nomadkaraoke.com
Thanks for using Nomad Karaoke!
""".strip(),
)
# Send notification email to Andrew
email_service.send_email(
to_email=ADMIN_EMAIL,
subject=f"[Made For You Order] {artist} - {title}",
html_content=f"""
<h2>New Made-For-You Order</h2>
<p>A customer has ordered a karaoke video:</p>
<ul>
<li><strong>Customer:</strong> {customer_email}</li>
<li><strong>Artist:</strong> {artist}</li>
<li><strong>Title:</strong> {title}</li>
<li><strong>Source:</strong> {source_type}</li>
{f'<li><strong>YouTube URL:</strong> <a href="{youtube_url}">{youtube_url}</a></li>' if youtube_url else ''}
{f'<li><strong>Notes:</strong> {notes}</li>' if notes else ''}
</ul>
<p><strong>Job ID:</strong> {job_id}</p>
<p>View job in admin: <a href="https://gen.nomadkaraoke.com/admin/jobs/{job_id}">Admin Link</a></p>
<p>View job as customer: <a href="https://gen.nomadkaraoke.com/jobs/{job_id}">Customer Link</a></p>
<p><strong>Deadline:</strong> 24 hours from now</p>
""",
text_content=f"""
New Made-For-You Order
Customer: {customer_email}
Artist: {artist}
Title: {title}
Source: {source_type}
{f'YouTube URL: {youtube_url}' if youtube_url else ''}
{f'Notes: {notes}' if notes else ''}
Job ID: {job_id}
Admin: https://gen.nomadkaraoke.com/admin/jobs/{job_id}
Customer: https://gen.nomadkaraoke.com/jobs/{job_id}
Deadline: 24 hours from now
""".strip(),
)
logger.info(f"Sent made-for-you order notifications for job {job_id}")
except Exception as e:
logger.error(f"Error processing made-for-you order: {e}", exc_info=True)
# Still try to notify Andrew of the failure
try:
email_service.send_email(
to_email=ADMIN_EMAIL,
subject=f"[FAILED] Made For You Order: {artist} - {title}",
html_content=f"""
<h2>Made-For-You Order Failed</h2>
<p>An error occurred processing this order:</p>
<ul>
<li><strong>Customer:</strong> {customer_email}</li>
<li><strong>Artist:</strong> {artist}</li>
<li><strong>Title:</strong> {title}</li>
<li><strong>Error:</strong> {str(e)}</li>
</ul>
<p>Please manually create this job and notify the customer.</p>
""",
text_content=f"""
Made-For-You Order Failed
Customer: {customer_email}
Artist: {artist}
Title: {title}
Error: {str(e)}
Please manually create this job and notify the customer.
""".strip(),
)
except Exception as email_error:
logger.error(f"Failed to send error notification: {email_error}")
@router.post("/webhooks/stripe")
async def stripe_webhook(
request: Request,
@@ -365,30 +877,41 @@ async def stripe_webhook(
if event_type == "checkout.session.completed":
session = event["data"]["object"]
session_id = session.get("id")
metadata = session.get("metadata", {})
# Idempotency check: Skip if this session was already processed
if session_id and user_service.is_stripe_session_processed(session_id):
logger.info(f"Skipping already processed session: {session_id}")
return {"status": "received", "type": event_type, "note": "already_processed"}
# Process the completed checkout
success, user_email, credits, _ = stripe_service.handle_checkout_completed(session)
if success and user_email and credits > 0:
# Add credits to user account
ok, new_balance, credit_msg = user_service.add_credits(
email=user_email,
amount=credits,
reason="stripe_purchase",
stripe_session_id=session_id,
# Check if this is a made-for-you order
if metadata.get("order_type") == "made_for_you":
# Handle made-for-you order - create a job
await _handle_made_for_you_order(
session_id=session_id,
metadata=metadata,
user_service=user_service,
email_service=email_service,
)
else:
# Handle regular credit purchase
success, user_email, credits, _ = stripe_service.handle_checkout_completed(session)
if ok:
# Send confirmation email
email_service.send_credits_added(user_email, credits, new_balance)
logger.info(f"Added {credits} credits to {user_email}, new balance: {new_balance}")
else:
logger.error(f"Failed to add credits: {credit_msg}")
if success and user_email and credits > 0:
# Add credits to user account
ok, new_balance, credit_msg = user_service.add_credits(
email=user_email,
amount=credits,
reason="stripe_purchase",
stripe_session_id=session_id,
)
if ok:
# Send confirmation email
email_service.send_credits_added(user_email, credits, new_balance)
logger.info(f"Added {credits} credits to {user_email}, new balance: {new_balance}")
else:
logger.error(f"Failed to add credits: {credit_msg}")
elif event_type == "checkout.session.expired":
logger.info(f"Checkout session expired: {event['data']['object'].get('id')}")
@@ -663,6 +1186,7 @@ async def list_users(
sort_by: str = "created_at",
sort_order: str = "desc",
include_inactive: bool = False,
exclude_test: bool = True,
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
user_service: UserService = Depends(get_user_service),
):
@@ -676,6 +1200,7 @@ async def list_users(
sort_by: Field to sort by (created_at, last_login_at, credits, email)
sort_order: Sort direction (asc, desc)
include_inactive: Include disabled users
exclude_test: If True (default), exclude test users (e.g., @inbox.testmail.app)
"""
from google.cloud import firestore
from google.cloud.firestore_v1 import FieldFilter
@@ -704,9 +1229,14 @@ async def list_users(
else:
query = query.order_by("created_at", direction=direction)
# Get total count (without pagination) for has_more calculation
# Get all docs and filter in Python
# Note: This is expensive for large datasets, consider caching
all_docs = list(query.stream())
# Filter out test users if exclude_test is True
if exclude_test:
all_docs = [d for d in all_docs if not is_test_email(d.to_dict().get('email', ''))]
total_count = len(all_docs)
# Apply pagination manually (Firestore doesn't support offset well)
@@ -942,44 +1472,64 @@ async def list_beta_feedback(
@router.get("/admin/beta/stats")
async def get_beta_stats(
exclude_test: bool = True,
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
user_service: UserService = Depends(get_user_service),
):
"""
Get beta tester program statistics (admin only).
Args:
exclude_test: If True (default), exclude test users from beta stats
"""
from google.cloud.firestore_v1 import FieldFilter
from google.cloud.firestore_v1 import aggregation
# Count beta testers by status using efficient aggregation queries
users_collection = user_service.db.collection(USERS_COLLECTION)
# Helper function to get count using aggregation
def get_count(query) -> int:
agg_query = aggregation.AggregationQuery(query)
agg_query.count(alias="count")
results = agg_query.get()
return results[0][0].value if results else 0
if exclude_test:
# Stream and filter in Python since Firestore doesn't support "not ends with"
all_beta_users = []
for doc in users_collection.where(filter=FieldFilter("is_beta_tester", "==", True)).stream():
data = doc.to_dict()
if not is_test_email(data.get("email", "")):
all_beta_users.append(data)
total_beta_testers = get_count(
users_collection.where(filter=FieldFilter("is_beta_tester", "==", True))
)
total_beta_testers = len(all_beta_users)
active_testers = sum(1 for u in all_beta_users if u.get("beta_tester_status") == "active")
pending_feedback = sum(1 for u in all_beta_users if u.get("beta_tester_status") == "pending_feedback")
completed_feedback = sum(1 for u in all_beta_users if u.get("beta_tester_status") == "completed")
active_testers = get_count(
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "active"))
)
# Filter feedback by non-test users
all_feedback = []
for doc in user_service.db.collection("beta_feedback").stream():
data = doc.to_dict()
if not is_test_email(data.get("user_email", "")):
all_feedback.append(data)
feedback_docs = all_feedback
else:
# Use efficient aggregation queries when including test data
def get_count(query) -> int:
agg_query = aggregation.AggregationQuery(query)
agg_query.count(alias="count")
results = agg_query.get()
return results[0][0].value if results else 0
pending_feedback = get_count(
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "pending_feedback"))
)
completed_feedback = get_count(
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "completed"))
)
# Get average ratings from feedback
feedback_docs = list(user_service.db.collection("beta_feedback").stream())
total_beta_testers = get_count(
users_collection.where(filter=FieldFilter("is_beta_tester", "==", True))
)
active_testers = get_count(
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "active"))
)
pending_feedback = get_count(
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "pending_feedback"))
)
completed_feedback = get_count(
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "completed"))
)
feedback_docs = [doc.to_dict() for doc in user_service.db.collection("beta_feedback").stream()]
# Calculate average ratings from feedback
avg_overall = 0
avg_ease = 0
avg_accuracy = 0
@@ -987,8 +1537,7 @@ async def get_beta_stats(
if feedback_docs:
total = len(feedback_docs)
for doc in feedback_docs:
data = doc.to_dict()
for data in feedback_docs:
avg_overall += data.get("overall_rating", 0)
avg_ease += data.get("ease_of_use_rating", 0)
avg_accuracy += data.get("lyrics_accuracy_rating", 0)
+6
View File
@@ -68,6 +68,12 @@ class Settings(BaseSettings):
use_gce_encoding: bool = os.getenv("USE_GCE_ENCODING", "false").lower() in ("true", "1", "yes")
encoding_worker_url: Optional[str] = os.getenv("ENCODING_WORKER_URL") # e.g., http://136.119.50.148:8080
encoding_worker_api_key: Optional[str] = os.getenv("ENCODING_WORKER_API_KEY")
# GCE Preview Encoding (for faster preview video generation)
# When enabled, preview video encoding during lyrics review is offloaded to the GCE worker.
# This reduces preview generation time from 60+ seconds to ~15-20 seconds.
# Requires use_gce_encoding to be enabled and the GCE worker to support /encode-preview endpoint.
use_gce_preview_encoding: bool = os.getenv("USE_GCE_PREVIEW_ENCODING", "false").lower() in ("true", "1", "yes")
# Storage paths
temp_dir: str = os.getenv("TEMP_DIR", "/tmp/karaoke-gen")
+31 -2
View File
@@ -7,10 +7,14 @@ from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from backend.config import settings
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes, users, admin
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes, users, admin, tenant
from backend.services.tracing import setup_tracing, instrument_app, get_current_trace_id
from backend.services.structured_logging import setup_structured_logging
from backend.services.spacy_preloader import preload_spacy_model
from backend.services.nltk_preloader import preload_all_nltk_resources
from backend.services.langfuse_preloader import preload_langfuse_handler
from backend.middleware.audit_logging import AuditLoggingMiddleware
from backend.middleware.tenant import TenantMiddleware
from backend.version import VERSION
@@ -67,7 +71,28 @@ async def lifespan(app: FastAPI):
logger.info(f"Environment: {settings.environment}")
logger.info(f"GCS Bucket: {settings.gcs_bucket_name}")
logger.info(f"Tracing enabled: {tracing_enabled}")
# Preload NLP models and resources to avoid cold start delays
# See docs/archive/2026-01-08-performance-investigation.md for background
# 1. SpaCy model (60+ second delay without preload)
try:
preload_spacy_model("en_core_web_sm")
except Exception as e:
logger.warning(f"SpaCy preload failed (will load lazily): {e}")
# 2. NLTK cmudict (50-100+ second delay without preload)
try:
preload_all_nltk_resources()
except Exception as e:
logger.warning(f"NLTK preload failed (will load lazily): {e}")
# 3. Langfuse callback handler (200+ second delay without preload)
try:
preload_langfuse_handler()
except Exception as e:
logger.warning(f"Langfuse preload failed (will initialize lazily): {e}")
# Validate OAuth credentials (non-blocking)
try:
await validate_credentials_on_startup()
@@ -104,6 +129,9 @@ app.add_middleware(
# Add audit logging middleware (captures all requests with request_id for correlation)
app.add_middleware(AuditLoggingMiddleware)
# Add tenant detection middleware (extracts tenant from subdomain/headers)
app.add_middleware(TenantMiddleware)
# Include routers
app.include_router(health.router, prefix="/api")
app.include_router(jobs.router, prefix="/api")
@@ -115,6 +143,7 @@ app.include_router(audio_search.router, prefix="/api") # Audio search (artist+t
app.include_router(themes.router, prefix="/api") # Theme selection for styles
app.include_router(users.router, prefix="/api") # User auth, credits, and Stripe webhooks
app.include_router(admin.router, prefix="/api") # Admin dashboard and management
app.include_router(tenant.router) # Tenant/white-label configuration (no /api prefix, router has it)
@app.get("/")
+7 -1
View File
@@ -1,5 +1,11 @@
"""Middleware package for FastAPI application."""
from backend.middleware.audit_logging import AuditLoggingMiddleware
from backend.middleware.tenant import TenantMiddleware, get_tenant_from_request, get_tenant_config_from_request
__all__ = ["AuditLoggingMiddleware"]
__all__ = [
"AuditLoggingMiddleware",
"TenantMiddleware",
"get_tenant_from_request",
"get_tenant_config_from_request",
]
+192
View File
@@ -0,0 +1,192 @@
"""
Tenant detection middleware for white-label B2B portals.
This middleware extracts the tenant ID from incoming requests and attaches
the tenant configuration to the request state for use by downstream handlers.
Tenant detection priority:
1. X-Tenant-ID header (explicitly set by frontend)
2. `tenant` query parameter (for development/testing)
3. Host header subdomain detection (production flow)
The middleware is non-blocking - if no tenant is detected, the request
proceeds as a default Nomad Karaoke request (tenant_id = None).
"""
import logging
import os
from typing import Optional
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import Response
from backend.services.tenant_service import get_tenant_service
from backend.models.tenant import TenantConfig
logger = logging.getLogger(__name__)
# Only allow query param tenant override in non-production environments
IS_PRODUCTION = os.environ.get("ENV", "").lower() == "production" or \
os.environ.get("ENVIRONMENT", "").lower() == "production"
# Known non-tenant subdomains that should be treated as default Nomad Karaoke
NON_TENANT_SUBDOMAINS = {"gen", "api", "www", "buy", "admin", "app", "beta"}
class TenantMiddleware(BaseHTTPMiddleware):
"""
Middleware that detects tenant from request and attaches config to state.
After this middleware runs, routes can access:
- request.state.tenant_id: str or None
- request.state.tenant_config: TenantConfig or None
Usage in route handlers:
```python
@router.get("/something")
async def handler(request: Request):
tenant_id = getattr(request.state, "tenant_id", None)
tenant_config = getattr(request.state, "tenant_config", None)
```
"""
async def dispatch(self, request: Request, call_next) -> Response:
# Extract tenant ID from request
tenant_id = self._extract_tenant_id(request)
# Load tenant config if tenant detected
tenant_config: Optional[TenantConfig] = None
if tenant_id:
tenant_service = get_tenant_service()
tenant_config = tenant_service.get_tenant_config(tenant_id)
if tenant_config and not tenant_config.is_active:
# Tenant exists but is inactive - treat as default
logger.warning(f"Inactive tenant requested: {tenant_id}")
tenant_id = None
tenant_config = None
# Attach to request state
request.state.tenant_id = tenant_id
request.state.tenant_config = tenant_config
# Log tenant detection for debugging
if tenant_id:
logger.debug(f"Request tenant: {tenant_id} (path: {request.url.path})")
# Process request
response = await call_next(request)
# Add tenant ID to response headers for debugging
if tenant_id:
response.headers["X-Tenant-ID"] = tenant_id
return response
def _extract_tenant_id(self, request: Request) -> Optional[str]:
"""
Extract tenant ID from request using priority-based detection.
Returns:
Tenant ID if detected, None for default Nomad Karaoke
"""
# Priority 1: X-Tenant-ID header (explicitly set by frontend)
header_tenant = request.headers.get("X-Tenant-ID")
if header_tenant:
return header_tenant.lower().strip()
# Priority 2: Query parameter (for development/testing only)
# Disabled in production to prevent tenant spoofing
if not IS_PRODUCTION:
query_tenant = request.query_params.get("tenant")
if query_tenant:
return query_tenant.lower().strip()
# Priority 3: Host header subdomain detection
host = request.headers.get("Host", "")
return self._extract_tenant_from_host(host)
def _extract_tenant_from_host(self, host: str) -> Optional[str]:
"""
Extract tenant ID from Host header subdomain.
Patterns handled:
- vocalstar.nomadkaraoke.com -> vocalstar
- vocalstar.gen.nomadkaraoke.com -> vocalstar
- localhost:3000 -> None (local dev)
- gen.nomadkaraoke.com -> None (main app)
- api.nomadkaraoke.com -> None (API)
Returns:
Tenant ID if subdomain matches tenant, None otherwise
"""
if not host:
return None
# Normalize
host_lower = host.lower()
# Remove port if present
if ":" in host_lower:
host_lower = host_lower.split(":")[0]
# Check if this is a nomadkaraoke.com domain
if "nomadkaraoke.com" not in host_lower:
return None
# Split into parts
parts = host_lower.split(".")
# Need at least 3 parts: subdomain.nomadkaraoke.com
if len(parts) < 3:
return None
# First part is the potential tenant ID
potential_tenant = parts[0]
# Skip known non-tenant subdomains
if potential_tenant in NON_TENANT_SUBDOMAINS:
return None
# Verify tenant exists in GCS
tenant_service = get_tenant_service()
if tenant_service.tenant_exists(potential_tenant):
return potential_tenant
return None
def get_tenant_from_request(request: Request) -> Optional[str]:
"""
Helper function to get tenant ID from request state.
Usage in route handlers:
```python
from backend.middleware.tenant import get_tenant_from_request
@router.get("/something")
async def handler(request: Request):
tenant_id = get_tenant_from_request(request)
```
"""
return getattr(request.state, "tenant_id", None)
def get_tenant_config_from_request(request: Request) -> Optional[TenantConfig]:
"""
Helper function to get tenant config from request state.
Usage in route handlers:
```python
from backend.middleware.tenant import get_tenant_config_from_request
@router.get("/something")
async def handler(request: Request):
config = get_tenant_config_from_request(request)
if config and not config.features.audio_search:
raise HTTPException(403, "Audio search not available")
```
"""
return getattr(request.state, "tenant_config", None)
+28 -4
View File
@@ -9,6 +9,8 @@ from datetime import datetime
from enum import Enum
from pydantic import BaseModel, Field, validator
from karaoke_gen.utils import normalize_text
class JobStatus(str, Enum):
"""
@@ -205,6 +207,9 @@ class Job(BaseModel):
webhook_url: Optional[str] = None # Webhook for notifications
user_email: Optional[str] = None # Email for notifications
non_interactive: bool = False # Skip interactive steps (lyrics review, instrumental selection)
# Multi-tenant support (None = default Nomad Karaoke)
tenant_id: Optional[str] = None # Tenant ID for white-label portal scoping
# Theme configuration (pre-made themes from GCS)
theme_id: Optional[str] = None # Theme identifier (e.g., "nomad", "default")
@@ -484,14 +489,33 @@ class JobCreate(BaseModel):
- server_version: Current server version
- custom_headers: All X-* headers
"""
@validator('url', 'artist', 'title')
def validate_inputs(cls, v):
"""Validate string inputs are not empty."""
# Tenant scoping for white-label portals
tenant_id: Optional[str] = None # Tenant ID for job scoping
@validator('url')
def validate_url(cls, v):
"""Validate URL is not empty."""
if v is not None and isinstance(v, str) and not v.strip():
raise ValueError("Field cannot be empty string")
return v.strip() if isinstance(v, str) else v
@validator('artist', 'title')
def normalize_artist_title(cls, v):
"""Normalize artist/title text to standardize Unicode characters.
This ensures consistent data storage by converting:
- Curly quotes -> straight quotes
- Various dashes -> hyphen
- Unusual whitespace -> regular space
"""
if v is not None and isinstance(v, str):
if not v.strip():
raise ValueError("Field cannot be empty string")
# normalize_text handles stripping and Unicode normalization
return normalize_text(v)
return v
class JobResponse(BaseModel):
"""Job response model."""
+2 -2
View File
@@ -14,7 +14,7 @@ class URLSubmissionRequest(BaseModel):
# Optional preferences
enable_cdg: bool = False # Requires style config
enable_txt: bool = False # Requires style config
enable_youtube_upload: bool = False
enable_youtube_upload: Optional[bool] = None # None = use server default
youtube_description: Optional[str] = None
webhook_url: Optional[str] = None
user_email: Optional[str] = None
@@ -28,7 +28,7 @@ class UploadSubmissionRequest(BaseModel):
# Optional preferences
enable_cdg: bool = False # Requires style config
enable_txt: bool = False # Requires style config
enable_youtube_upload: bool = False
enable_youtube_upload: Optional[bool] = None # None = use server default
youtube_description: Optional[str] = None
webhook_url: Optional[str] = None
user_email: Optional[str] = None
+208
View File
@@ -0,0 +1,208 @@
"""
Tenant data models for white-label B2B portals.
Tenants are B2B customers who get their own branded portal with custom
configuration. Each tenant has:
- Custom branding (logo, colors, site title)
- Feature flags (enable/disable audio search, distribution, etc.)
- Default settings (theme, distribution mode)
- Auth configuration (allowed email domains, fixed tokens)
Tenant configs are stored in GCS at tenants/{tenant_id}/config.json
"""
from datetime import datetime
from typing import List, Optional
from pydantic import BaseModel, Field
class TenantBranding(BaseModel):
"""Branding configuration for a tenant's portal."""
logo_url: Optional[str] = Field(
None, description="URL to tenant's logo image (PNG with transparency preferred)"
)
logo_height: int = Field(40, description="Logo height in pixels for header display")
primary_color: str = Field(
"#ff5bb8", description="Primary brand color (hex format)"
)
secondary_color: str = Field(
"#8b5cf6", description="Secondary brand color (hex format)"
)
accent_color: Optional[str] = Field(
None, description="Accent color for highlights (hex format)"
)
background_color: Optional[str] = Field(
None, description="Custom background color (defaults to dark theme)"
)
favicon_url: Optional[str] = Field(None, description="URL to custom favicon")
site_title: str = Field(
"Karaoke Generator", description="Browser tab title"
)
tagline: Optional[str] = Field(
None, description="Optional tagline shown below logo"
)
class TenantFeatures(BaseModel):
"""Feature flags controlling what's available in the tenant's portal."""
# Audio input methods
audio_search: bool = Field(
True, description="Enable audio search (flacfetch integration)"
)
file_upload: bool = Field(True, description="Enable direct file upload")
youtube_url: bool = Field(True, description="Enable YouTube URL input")
# Distribution options
youtube_upload: bool = Field(True, description="Enable YouTube upload option")
dropbox_upload: bool = Field(True, description="Enable Dropbox upload option")
gdrive_upload: bool = Field(True, description="Enable Google Drive upload option")
# Customization options
theme_selection: bool = Field(
True, description="Allow user to select theme (false = use default)"
)
color_overrides: bool = Field(
True, description="Allow user to customize colors"
)
# Output formats
enable_cdg: bool = Field(True, description="Generate CDG format output")
enable_4k: bool = Field(True, description="Generate 4K video output")
# Advanced features
admin_access: bool = Field(
False, description="Allow access to admin dashboard"
)
class TenantDefaults(BaseModel):
"""Default settings applied to all jobs for this tenant."""
theme_id: Optional[str] = Field(
None, description="Default theme ID (required if theme_selection=false)"
)
locked_theme: Optional[str] = Field(
None,
description="If set, users cannot change theme - always uses this theme ID"
)
distribution_mode: str = Field(
"all",
description="Distribution mode: 'all', 'download_only', or 'cloud_only'"
)
brand_prefix: Optional[str] = Field(
None, description="Prefix for output filenames (e.g., 'VSTAR')"
)
youtube_description_template: Optional[str] = Field(
None, description="Default YouTube description template"
)
class TenantAuth(BaseModel):
"""Authentication configuration for a tenant."""
allowed_email_domains: List[str] = Field(
default_factory=list,
description="Email domains allowed for magic link auth (e.g., ['vocal-star.com'])"
)
require_email_domain: bool = Field(
True,
description="If true, only allowed domains can sign up. If false, domains get auto-approved."
)
fixed_token_ids: List[str] = Field(
default_factory=list,
description="IDs of fixed API tokens for this tenant (tokens stored in auth_tokens)"
)
sender_email: Optional[str] = Field(
None,
description="Email sender address for this tenant (e.g., 'vocalstar@nomadkaraoke.com')"
)
class TenantConfig(BaseModel):
"""Complete configuration for a white-label tenant."""
id: str = Field(..., description="Unique tenant identifier (e.g., 'vocalstar')")
name: str = Field(..., description="Display name (e.g., 'Vocal Star')")
subdomain: str = Field(
..., description="Full subdomain (e.g., 'vocalstar.nomadkaraoke.com')"
)
is_active: bool = Field(True, description="Whether tenant portal is active")
branding: TenantBranding = Field(default_factory=TenantBranding)
features: TenantFeatures = Field(default_factory=TenantFeatures)
defaults: TenantDefaults = Field(default_factory=TenantDefaults)
auth: TenantAuth = Field(default_factory=TenantAuth)
created_at: Optional[datetime] = None
updated_at: Optional[datetime] = None
def get_sender_email(self) -> str:
"""Get the email sender address for this tenant."""
if self.auth.sender_email:
return self.auth.sender_email
# Default pattern: {tenant_id}@nomadkaraoke.com
return f"{self.id}@nomadkaraoke.com"
def is_email_allowed(self, email: str) -> bool:
"""Check if an email address is allowed for this tenant."""
if not self.auth.allowed_email_domains:
# No domain restrictions
return True
email_lower = email.lower()
for domain in self.auth.allowed_email_domains:
if email_lower.endswith(f"@{domain.lower()}"):
return True
return not self.auth.require_email_domain
class TenantPublicConfig(BaseModel):
"""
Public tenant configuration returned to frontend.
This excludes sensitive auth details like token IDs.
"""
id: str
name: str
subdomain: str
is_active: bool
branding: TenantBranding
features: TenantFeatures
defaults: TenantDefaults
# Auth info limited to what frontend needs
allowed_email_domains: List[str] = Field(default_factory=list)
@classmethod
def from_config(cls, config: TenantConfig) -> "TenantPublicConfig":
"""Create public config from full config."""
return cls(
id=config.id,
name=config.name,
subdomain=config.subdomain,
is_active=config.is_active,
branding=config.branding,
features=config.features,
defaults=TenantDefaults(
theme_id=config.defaults.theme_id,
locked_theme=config.defaults.locked_theme,
distribution_mode=config.defaults.distribution_mode,
# Don't expose brand_prefix or youtube_description_template
),
allowed_email_domains=config.auth.allowed_email_domains,
)
class TenantConfigResponse(BaseModel):
"""Response from GET /api/tenant/config endpoint."""
tenant: Optional[TenantPublicConfig] = Field(
None, description="Tenant config if found, null for default Nomad Karaoke"
)
is_default: bool = Field(
True, description="True if using default Nomad Karaoke config"
)
+18
View File
@@ -45,11 +45,17 @@ class User(BaseModel):
Users are identified by email address. Authentication is via magic links
(no passwords). Credits are consumed when creating karaoke jobs.
For multi-tenant white-label portals, users are scoped to a tenant.
Users with tenant_id=None are default Nomad Karaoke users.
"""
email: str # Primary identifier
role: UserRole = UserRole.USER
credits: int = 0
# Multi-tenant support (None = default Nomad Karaoke)
tenant_id: Optional[str] = None
# Stripe integration
stripe_customer_id: Optional[str] = None
@@ -87,6 +93,9 @@ class MagicLinkToken(BaseModel):
Tokens are short-lived (15 minutes) and single-use.
Stored in Firestore with automatic TTL.
For multi-tenant portals, the tenant_id determines which portal
the user will be redirected to after verification.
"""
token: str # Secure random token
email: str
@@ -97,6 +106,9 @@ class MagicLinkToken(BaseModel):
ip_address: Optional[str] = None
user_agent: Optional[str] = None
# Multi-tenant support (None = default Nomad Karaoke)
tenant_id: Optional[str] = None
class Session(BaseModel):
"""
@@ -104,6 +116,8 @@ class Session(BaseModel):
Sessions are created after successful magic link verification.
Sessions expire after 7 days of inactivity or 30 days absolute.
For multi-tenant portals, sessions are scoped to a tenant.
"""
token: str # Secure random session token
user_email: str
@@ -114,6 +128,9 @@ class Session(BaseModel):
user_agent: Optional[str] = None
is_active: bool = True
# Multi-tenant support (None = default Nomad Karaoke)
tenant_id: Optional[str] = None
# Pydantic models for API requests/responses
@@ -149,6 +166,7 @@ class UserPublic(BaseModel):
display_name: Optional[str] = None
total_jobs_created: int = 0
total_jobs_completed: int = 0
tenant_id: Optional[str] = None
class AddCreditsRequest(BaseModel):
File diff suppressed because it is too large. Load diff
+11 -8
View File
@@ -351,8 +351,8 @@ class GCEEncodingBackend(EncodingBackend):
result = {}
raw_output_files = result.get("output_files", {})
# Convert output_files from list of paths to dict if needed
# GCE worker returns list like: ["path/output_4k_lossless.mp4", "path/output_720p.mp4"]
# Convert output_files from list of paths to dict
# GCE worker returns list like: ["path/Artist - Title (Final Karaoke Lossless 4k).mp4", ...]
# We need dict like: {"mp4_4k_lossless": "path/...", "mp4_720p": "path/..."}
if isinstance(raw_output_files, list):
output_files = {}
@@ -360,15 +360,18 @@ class GCEEncodingBackend(EncodingBackend):
if not isinstance(path, str):
continue
filename = path.split("/")[-1] if "/" in path else path
filename_lower = filename.lower()
# Map filename patterns to output format keys
if "4k_lossless" in filename:
output_files["mp4_4k_lossless"] = path
elif "4k_lossy" in filename:
# Files are named like "Artist - Title (Final Karaoke Lossless 4k).mp4"
if "lossless 4k" in filename_lower:
if filename.endswith(".mkv"):
output_files["mkv_4k"] = path
else:
output_files["mp4_4k_lossless"] = path
elif "lossy 4k" in filename_lower:
output_files["mp4_4k_lossy"] = path
elif "720p" in filename:
elif "720p" in filename_lower:
output_files["mp4_720p"] = path
elif filename.endswith(".mkv"):
output_files["mkv_4k"] = path
self.logger.info(f"Converted output_files list to dict: {output_files}")
else:
output_files = raw_output_files if isinstance(raw_output_files, dict) else {}
+243 -20
View File
@@ -25,6 +25,11 @@ from backend.config import get_settings
logger = logging.getLogger(__name__)
# Retry configuration for handling transient failures (e.g., worker restarts)
MAX_RETRIES = 3
INITIAL_BACKOFF_SECONDS = 2.0
MAX_BACKOFF_SECONDS = 10.0
class EncodingService:
"""Service for dispatching encoding jobs to GCE worker."""
@@ -63,6 +68,86 @@ class EncodingService:
"""Check if GCE encoding is enabled and configured."""
return self.settings.use_gce_encoding and self.is_configured
@property
def is_preview_enabled(self) -> bool:
"""Check if GCE preview encoding is enabled and configured."""
return self.settings.use_gce_preview_encoding and self.is_configured
async def _request_with_retry(
self,
method: str,
url: str,
headers: Dict[str, str],
json_payload: Optional[Dict[str, Any]] = None,
timeout: float = 30.0,
job_id: str = "unknown",
) -> Dict[str, Any]:
"""
Make an HTTP request with retry logic for transient failures.
This handles connection errors that occur when the GCE worker is
restarting (e.g., during deployments) by retrying with exponential backoff.
Args:
method: HTTP method (GET, POST)
url: Request URL
headers: Request headers
json_payload: JSON body for POST requests
timeout: Request timeout in seconds
job_id: Job ID for logging
Returns:
Dict with keys:
- status (int): HTTP status code
- json (Any): Parsed JSON response body (if status 200, else None)
- text (str): Raw response text (if status != 200, else None)
Raises:
aiohttp.ClientConnectorError: If all retries fail due to connection errors
aiohttp.ServerDisconnectedError: If all retries fail due to server disconnect
asyncio.TimeoutError: If all retries fail due to timeout
"""
last_exception = None
backoff = INITIAL_BACKOFF_SECONDS
for attempt in range(MAX_RETRIES + 1):
try:
async with aiohttp.ClientSession() as session:
if method.upper() == "POST":
async with session.post(
url, json=json_payload, headers=headers, timeout=timeout
) as resp:
# Return a copy of the response data since we exit the context
return {
"status": resp.status,
"json": await resp.json() if resp.status == 200 else None,
"text": await resp.text() if resp.status != 200 else None,
}
else: # GET
async with session.get(
url, headers=headers, timeout=timeout
) as resp:
return {
"status": resp.status,
"json": await resp.json() if resp.status == 200 else None,
"text": await resp.text() if resp.status != 200 else None,
}
except (aiohttp.ClientConnectorError, aiohttp.ServerDisconnectedError, asyncio.TimeoutError) as e:
last_exception = e
if attempt < MAX_RETRIES:
logger.warning(
f"[job:{job_id}] GCE worker connection failed (attempt {attempt + 1}/{MAX_RETRIES + 1}): {e}. "
f"Retrying in {backoff:.1f}s..."
)
await asyncio.sleep(backoff)
backoff = min(backoff * 2, MAX_BACKOFF_SECONDS)
else:
logger.error(
f"[job:{job_id}] GCE worker connection failed after {MAX_RETRIES + 1} attempts: {e}"
)
raise last_exception
async def submit_encoding_job(
self,
job_id: str,
@@ -101,17 +186,23 @@ class EncodingService:
logger.info(f"[job:{job_id}] Submitting encoding job to GCE worker: {url}")
async with aiohttp.ClientSession() as session:
async with session.post(url, json=payload, headers=headers, timeout=30) as resp:
if resp.status == 401:
raise RuntimeError("Invalid API key for encoding worker")
if resp.status == 409:
raise RuntimeError(f"Encoding job {job_id} already exists")
if resp.status != 200:
text = await resp.text()
raise RuntimeError(f"Failed to submit encoding job: {resp.status} - {text}")
resp = await self._request_with_retry(
method="POST",
url=url,
headers=headers,
json_payload=payload,
timeout=30.0,
job_id=job_id,
)
return await resp.json()
if resp["status"] == 401:
raise RuntimeError("Invalid API key for encoding worker")
if resp["status"] == 409:
raise RuntimeError(f"Encoding job {job_id} already exists")
if resp["status"] != 200:
raise RuntimeError(f"Failed to submit encoding job: {resp['status']} - {resp['text']}")
return resp["json"]
async def get_job_status(self, job_id: str) -> Dict[str, Any]:
"""
@@ -131,17 +222,22 @@ class EncodingService:
url = f"{self._url}/status/{job_id}"
headers = {"X-API-Key": self._api_key}
async with aiohttp.ClientSession() as session:
async with session.get(url, headers=headers, timeout=30) as resp:
if resp.status == 401:
raise RuntimeError("Invalid API key for encoding worker")
if resp.status == 404:
raise RuntimeError(f"Encoding job {job_id} not found")
if resp.status != 200:
text = await resp.text()
raise RuntimeError(f"Failed to get job status: {resp.status} - {text}")
resp = await self._request_with_retry(
method="GET",
url=url,
headers=headers,
timeout=30.0,
job_id=job_id,
)
return await resp.json()
if resp["status"] == 401:
raise RuntimeError("Invalid API key for encoding worker")
if resp["status"] == 404:
raise RuntimeError(f"Encoding job {job_id} not found")
if resp["status"] != 200:
raise RuntimeError(f"Failed to get job status: {resp['status']} - {resp['text']}")
return resp["json"]
async def wait_for_completion(
self,
@@ -242,6 +338,133 @@ class EncodingService:
job_id, progress_callback=progress_callback
)
async def submit_preview_encoding_job(
self,
job_id: str,
ass_gcs_path: str,
audio_gcs_path: str,
output_gcs_path: str,
background_color: str = "black",
background_image_gcs_path: Optional[str] = None,
font_gcs_path: Optional[str] = None,
) -> Dict[str, Any]:
"""
Submit a preview video encoding job to the GCE worker.
Args:
job_id: Unique job identifier
ass_gcs_path: GCS path to ASS subtitles file (gs://bucket/path/file.ass)
audio_gcs_path: GCS path to audio file
output_gcs_path: GCS path for output video
background_color: Background color (default: black)
background_image_gcs_path: Optional GCS path to background image
font_gcs_path: Optional GCS path to custom font file
Returns:
Response from the encoding worker
Raises:
Exception: If submission fails
"""
self._load_credentials()
if not self.is_configured:
raise RuntimeError("Encoding service not configured")
url = f"{self._url}/encode-preview"
headers = {"X-API-Key": self._api_key, "Content-Type": "application/json"}
payload = {
"job_id": job_id,
"ass_gcs_path": ass_gcs_path,
"audio_gcs_path": audio_gcs_path,
"output_gcs_path": output_gcs_path,
"background_color": background_color,
}
if background_image_gcs_path:
payload["background_image_gcs_path"] = background_image_gcs_path
if font_gcs_path:
payload["font_gcs_path"] = font_gcs_path
logger.info(f"[job:{job_id}] Submitting preview encoding job to GCE worker: {url}")
resp = await self._request_with_retry(
method="POST",
url=url,
headers=headers,
json_payload=payload,
timeout=30.0,
job_id=job_id,
)
if resp["status"] == 401:
raise RuntimeError("Invalid API key for encoding worker")
if resp["status"] == 409:
raise RuntimeError(f"Preview encoding job {job_id} already exists")
if resp["status"] != 200:
raise RuntimeError(f"Failed to submit preview encoding job: {resp['status']} - {resp['text']}")
return resp["json"]
async def encode_preview_video(
self,
job_id: str,
ass_gcs_path: str,
audio_gcs_path: str,
output_gcs_path: str,
background_color: str = "black",
background_image_gcs_path: Optional[str] = None,
font_gcs_path: Optional[str] = None,
timeout: float = 90.0,
poll_interval: float = 2.0,
) -> Dict[str, Any]:
"""
Submit preview encoding job and wait for completion.
This is a convenience method that combines submit + wait with shorter
timeouts suitable for preview videos.
Args:
job_id: Unique job identifier
ass_gcs_path: GCS path to ASS subtitles file
audio_gcs_path: GCS path to audio file
output_gcs_path: GCS path for output video
background_color: Background color (default: black)
background_image_gcs_path: Optional GCS path to background image
font_gcs_path: Optional GCS path to custom font file
timeout: Maximum time to wait (default 90s for preview)
poll_interval: Seconds between status checks (default 2s)
Returns:
Final job status with output files
"""
# Submit the job
submit_result = await self.submit_preview_encoding_job(
job_id=job_id,
ass_gcs_path=ass_gcs_path,
audio_gcs_path=audio_gcs_path,
output_gcs_path=output_gcs_path,
background_color=background_color,
background_image_gcs_path=background_image_gcs_path,
font_gcs_path=font_gcs_path,
)
# If cached, return immediately - video already exists in GCS
submit_status = submit_result.get("status")
if submit_status == "cached":
logger.info(f"[job:{job_id}] Preview already cached, returning immediately")
return {"status": "complete", "output_path": submit_result.get("output_path")}
# If in_progress, another request is encoding it - just wait for that
if submit_status == "in_progress":
logger.info(f"[job:{job_id}] Preview encoding already in progress, waiting")
# Wait for completion with shorter timeout
return await self.wait_for_completion(
job_id=job_id,
poll_interval=poll_interval,
timeout=timeout,
)
async def health_check(self) -> Dict[str, Any]:
"""
Check the health of the encoding worker.
+6
View File
@@ -116,6 +116,7 @@ class FirestoreService:
created_after: Optional[datetime] = None,
created_before: Optional[datetime] = None,
user_email: Optional[str] = None,
tenant_id: Optional[str] = None,
limit: int = 100
) -> List[Job]:
"""
@@ -128,6 +129,7 @@ class FirestoreService:
created_after: Filter jobs created after this datetime
created_before: Filter jobs created before this datetime
user_email: Filter by user_email (owner of the job)
tenant_id: Filter by tenant_id (white-label portal scoping)
limit: Maximum number of jobs to return
Returns:
@@ -150,6 +152,10 @@ class FirestoreService:
if user_email:
query = query.where(filter=FieldFilter('user_email', '==', user_email.lower()))
# Filter by tenant_id (white-label portal scoping)
if tenant_id:
query = query.where(filter=FieldFilter('tenant_id', '==', tenant_id))
# Date range filters
if created_after:
query = query.where(filter=FieldFilter('created_at', '>=', created_after))
+72
View File
@@ -0,0 +1,72 @@
# GCE Encoding Worker
HTTP API service that runs on the GCE encoding worker VM for video encoding jobs.
## Overview
This service provides HTTP endpoints for submitting and monitoring video encoding jobs. It runs as a systemd service on the `encoding-worker` GCE VM instance.
## Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/encode` | POST | Submit a full encoding job |
| `/encode-preview` | POST | Submit a preview encoding job (480x270, fast) |
| `/status/{job_id}` | GET | Get job status and progress |
| `/health` | GET | Health check |
## Authentication
All endpoints (except `/health`) require an API key via the `X-API-Key` header. The key is stored in Secret Manager (`encoding-worker-api-key`).
## Encoding Process
### Full Encoding (`/encode`)
1. Downloads input files from GCS (title screen, karaoke video, end screen, instrumental audio)
2. Uses `LocalEncodingService` from the karaoke-gen wheel to produce:
- Lossless 4K MP4
- Lossy 4K MP4
- Lossless MKV
- 720p MP4
3. Uploads results to GCS
4. Reports progress via status endpoint
### Preview Encoding (`/encode-preview`)
1. Downloads ASS subtitle and audio files from GCS
2. Runs FFmpeg to create a quick 480x270 preview video
3. Uploads result to GCS
4. Used for real-time preview in the web UI
## Dependencies
- **FFmpeg**: Static build from John Van Sickle
- **Python 3.13**: Built from source (required for karaoke-gen)
- **karaoke-gen wheel**: Downloaded from GCS at job start (enables hot updates)
## Deployment
This service is deployed via the VM's startup script. The script:
1. Installs system dependencies (FFmpeg, fonts, Python build deps)
2. Builds Python 3.13 from source
3. Creates venv and installs dependencies
4. Writes `main.py` to `/opt/encoding-worker/`
5. Creates systemd service
6. Starts the service on port 8080
## Local Development
This code is extracted from the infrastructure for maintainability. To test locally:
```bash
cd backend/services/gce_encoding
pip install -r requirements.txt
pip install ../../../ # Install karaoke-gen wheel
ENCODING_API_KEY=test python main.py
```
## Architecture Notes
- **Hot code updates**: The karaoke-gen wheel is re-downloaded at the start of each job, allowing code updates without VM restart
- **In-memory job tracking**: Job state is stored in memory. Restart clears queue.
- **Parallel encoding**: ThreadPoolExecutor with 4 workers
- **GCS integration**: Direct download/upload via google-cloud-storage client
+22
View File
@@ -0,0 +1,22 @@
"""
GCE Encoding Worker Service.
This package contains the FastAPI application that runs on the GCE encoding worker VM.
It provides HTTP endpoints for video encoding jobs.
The service:
- Downloads input files from GCS
- Runs FFmpeg encoding via LocalEncodingService
- Uploads results back to GCS
- Provides job status tracking
Endpoints:
- POST /encode - Submit a full encoding job
- POST /encode-preview - Submit a preview encoding job (fast, low-res)
- GET /status/{job_id} - Get job status
- GET /health - Health check
"""
from .main import app
__all__ = ["app"]
+589
View File
@@ -0,0 +1,589 @@
import asyncio
import glob as glob_module
import json
import logging
import os
import subprocess
import sys
import tempfile
import uuid
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from typing import Optional
from fastapi import FastAPI, BackgroundTasks, HTTPException, Header, Depends
from google.cloud import storage
from pydantic import BaseModel
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
app = FastAPI(title="Encoding Worker", version="1.0.0")
# API key authentication
API_KEY = os.environ.get("ENCODING_API_KEY", "")
async def verify_api_key(x_api_key: str = Header(None)):
# Verify API key for authentication
if not API_KEY:
logger.warning("No API key configured - authentication disabled")
return True
if x_api_key != API_KEY:
raise HTTPException(status_code=401, detail="Invalid API key")
return True
# Job tracking
jobs: dict[str, dict] = {}
executor = ThreadPoolExecutor(max_workers=4) # 4 parallel encoding jobs
# GCS client
storage_client = storage.Client()
class EncodeRequest(BaseModel):
job_id: str
input_gcs_path: str # gs://bucket/path/to/inputs/
output_gcs_path: str # gs://bucket/path/to/outputs/
encoding_config: dict # Video formats to generate
class EncodePreviewRequest(BaseModel):
job_id: str
ass_gcs_path: str # gs://bucket/path/to/subtitles.ass
audio_gcs_path: str # gs://bucket/path/to/audio.flac
output_gcs_path: str # gs://bucket/path/to/output.mp4
background_color: str = "black"
background_image_gcs_path: Optional[str] = None
font_gcs_path: Optional[str] = None # gs://bucket/path/to/custom-font.ttf
class JobStatus(BaseModel):
job_id: str
status: str # pending, running, complete, failed
progress: int # 0-100
error: Optional[str] = None
output_files: Optional[list[str]] = None
def download_from_gcs(gcs_uri: str, local_path: Path):
# Download a file or folder from GCS
# Parse gs://bucket/path
parts = gcs_uri.replace("gs://", "").split("/", 1)
bucket_name = parts[0]
prefix = parts[1] if len(parts) > 1 else ""
bucket = storage_client.bucket(bucket_name)
blobs = list(bucket.list_blobs(prefix=prefix))
for blob in blobs:
# Get relative path from prefix
rel_path = blob.name[len(prefix):].lstrip("/")
if not rel_path:
continue
dest = local_path / rel_path
dest.parent.mkdir(parents=True, exist_ok=True)
blob.download_to_filename(str(dest))
logger.info(f"Downloaded: {blob.name} -> {dest}")
def upload_to_gcs(local_path: Path, gcs_uri: str):
# Upload a file or folder to GCS
parts = gcs_uri.replace("gs://", "").split("/", 1)
bucket_name = parts[0]
prefix = parts[1].rstrip("/") if len(parts) > 1 else ""
bucket = storage_client.bucket(bucket_name)
if local_path.is_file():
blob_name = f"{prefix}/{local_path.name}" if prefix else local_path.name
blob = bucket.blob(blob_name)
blob.upload_from_filename(str(local_path))
logger.info(f"Uploaded: {local_path} -> gs://{bucket_name}/{blob_name}")
else:
for file in local_path.rglob("*"):
if file.is_file():
rel_path = file.relative_to(local_path)
blob_name = f"{prefix}/{rel_path}" if prefix else str(rel_path)
blob = bucket.blob(blob_name)
blob.upload_from_filename(str(file))
logger.info(f"Uploaded: {file} -> gs://{bucket_name}/{blob_name}")
def download_single_file_from_gcs(gcs_uri: str, local_path: Path):
# Download a single file from GCS
parts = gcs_uri.replace("gs://", "").split("/", 1)
bucket_name = parts[0]
blob_name = parts[1] if len(parts) > 1 else ""
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(blob_name)
local_path.parent.mkdir(parents=True, exist_ok=True)
blob.download_to_filename(str(local_path))
logger.info(f"Downloaded: {gcs_uri} -> {local_path}")
def upload_single_file_to_gcs(local_path: Path, gcs_uri: str):
# Upload a single file to a specific GCS path
parts = gcs_uri.replace("gs://", "").split("/", 1)
bucket_name = parts[0]
blob_name = parts[1] if len(parts) > 1 else ""
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(blob_name)
blob.upload_from_filename(str(local_path))
logger.info(f"Uploaded: {local_path} -> {gcs_uri}")
def run_preview_encoding(job_id: str, work_dir: Path, request: "EncodePreviewRequest"):
# Run FFmpeg encoding for preview video (480x270, fast settings)
jobs[job_id]["status"] = "running"
jobs[job_id]["progress"] = 10
try:
# Download input files
ass_path = work_dir / "subtitles.ass"
audio_path = work_dir / "audio.flac"
download_single_file_from_gcs(request.ass_gcs_path, ass_path)
jobs[job_id]["progress"] = 20
download_single_file_from_gcs(request.audio_gcs_path, audio_path)
jobs[job_id]["progress"] = 30
# Download background image if provided
bg_image_path = None
if request.background_image_gcs_path:
bg_image_path = work_dir / "background.png"
download_single_file_from_gcs(request.background_image_gcs_path, bg_image_path)
# Download custom font if provided and register with fontconfig
if request.font_gcs_path:
# Use standard fontconfig location that's already in the search path
fonts_dir = Path("/usr/local/share/fonts/custom")
fonts_dir.mkdir(parents=True, exist_ok=True)
font_filename = request.font_gcs_path.split("/")[-1]
font_path = fonts_dir / font_filename
download_single_file_from_gcs(request.font_gcs_path, font_path)
logger.info(f"Downloaded custom font: {font_path}")
# Update fontconfig cache so libass can find the font
subprocess.run(["fc-cache", "-fv"], capture_output=True)
logger.info(f"Updated fontconfig cache with custom font: {font_filename}")
# Build FFmpeg command
output_path = work_dir / "preview.mp4"
# Escape special characters in path for FFmpeg filter syntax
# FFmpeg filter parsing requires escaping: \ : , [ ] ;
def escape_ffmpeg_filter_path(path: str) -> str:
# Note: Extra escaping needed since this is inside a triple-quoted string in Pulumi
return path.replace("\\", "\\\\").replace(":", "\\:").replace(",", "\\,").replace("[", "\\[").replace("]", "\\]").replace(";", "\\;")
escaped_ass_path = escape_ffmpeg_filter_path(str(ass_path))
# Base command with frame rate
cmd = ["ffmpeg", "-y", "-r", "24"]
# Video input: background image or solid color
if bg_image_path and bg_image_path.exists():
cmd.extend(["-loop", "1", "-i", str(bg_image_path)])
# Scale and pad background to 480x270
vf = f"scale=480:270:force_original_aspect_ratio=decrease,pad=480:270:(ow-iw)/2:(oh-ih)/2,ass={escaped_ass_path}"
else:
# Solid color background
color = request.background_color or "black"
cmd.extend(["-f", "lavfi", "-i", f"color=c={color}:s=480x270:r=24"])
vf = f"ass={escaped_ass_path}"
# Audio input
cmd.extend(["-i", str(audio_path)])
# Video filter and encoding settings
cmd.extend([
"-vf", vf,
"-c:a", "aac", "-b:a", "96k",
"-c:v", "libx264",
"-preset", "superfast",
"-crf", "28",
"-pix_fmt", "yuv420p",
"-movflags", "+faststart",
"-threads", "8",
"-shortest",
str(output_path)
])
jobs[job_id]["progress"] = 40
logger.info(f"Running preview encoding: {' '.join(cmd)}")
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
logger.error(f"FFmpeg failed: {result.stderr}")
raise RuntimeError(f"FFmpeg preview encoding failed: {result.stderr[-500:]}")
jobs[job_id]["progress"] = 80
logger.info(f"Preview encoded: {output_path}")
# Upload output to GCS
upload_single_file_to_gcs(output_path, request.output_gcs_path)
jobs[job_id]["progress"] = 95
jobs[job_id]["output_files"] = [request.output_gcs_path]
return output_path
except Exception as e:
logger.error(f"Preview encoding failed: {e}")
jobs[job_id]["status"] = "failed"
jobs[job_id]["error"] = str(e)
raise
def ensure_latest_wheel():
'''Download and install latest karaoke-gen wheel from GCS.
Called at the start of each job to enable hot code updates without restart.
In-progress jobs continue with their version, new jobs get latest code.
'''
try:
logger.info("Checking for latest karaoke-gen wheel in GCS...")
# Download latest wheel
result = subprocess.run(
["gsutil", "cp", "gs://karaoke-gen-storage-nomadkaraoke/wheels/karaoke_gen-*.whl", "/tmp/"],
capture_output=True, text=True, timeout=60
)
# Find the downloaded wheel (get the latest by version sorting)
wheels = glob_module.glob("/tmp/karaoke_gen-*.whl")
if not wheels:
logger.warning("No wheel found in GCS, using fallback encoding logic")
return False
# Sort to get latest version
wheel_path = sorted(wheels)[-1]
logger.info(f"Installing wheel: {wheel_path}")
# Install (or upgrade) the wheel
# Use 5-minute timeout - first install at job start may need to resolve dependencies
# Subsequent installs are faster since dependencies are cached
install_result = subprocess.run(
[sys.executable, "-m", "pip", "install", "--upgrade", "--quiet", wheel_path],
capture_output=True, text=True, timeout=300
)
if install_result.returncode != 0:
logger.warning(f"Wheel installation failed: {install_result.stderr}")
return False
logger.info(f"Successfully installed wheel: {wheel_path}")
return True
except subprocess.TimeoutExpired:
logger.warning("Wheel download/install timed out, using fallback")
return False
except Exception as e:
logger.warning(f"Failed to ensure latest wheel: {e}")
return False
def find_file(work_dir: Path, *patterns):
'''Find a file matching any of the given glob patterns.'''
for pattern in patterns:
matches = list(work_dir.glob(f"**/{pattern}"))
if matches:
return matches[0]
return None
def run_encoding(job_id: str, work_dir: Path, config: dict):
'''Run encoding using LocalEncodingService (single source of truth).
Uses LocalEncodingService from the installed karaoke-gen wheel to ensure
output files match local CLI exactly:
- Proper names like "Artist - Title (Final Karaoke Lossless 4k).mp4"
- Concatenated title + karaoke + end screens
- All formats: lossless 4K MP4, lossy 4K MP4, lossless MKV, 720p MP4
Requires the karaoke-gen wheel to be installed (done by ensure_latest_wheel).
'''
jobs[job_id]["status"] = "running"
jobs[job_id]["progress"] = 10
try:
# Import LocalEncodingService from installed wheel (required, no fallback)
from backend.services.local_encoding_service import LocalEncodingService, EncodingConfig
logger.info("Using LocalEncodingService from installed wheel")
# Get artist/title from config for proper naming
artist = config.get("artist", "Unknown Artist")
title = config.get("title", "Unknown Title")
base_name = f"{artist} - {title}"
logger.info(f"Encoding for: {base_name}")
# Find input files in work_dir
# Title/end screens are in screens/ subdirectory
title_video = find_file(work_dir, "screens/title.mov", "*Title*.mov", "*title*.mov")
end_video = find_file(work_dir, "screens/end.mov", "*End*.mov", "*end*.mov")
# Karaoke video - search for With Vocals or main karaoke video
karaoke_video = find_file(
work_dir,
"*With Vocals*.mov", "*With Vocals*.mkv",
"*Vocals*.mov", "*Vocals*.mkv",
"*.mkv", "*.mov"
)
# Exclude title/end/output videos
if karaoke_video:
name_lower = karaoke_video.name.lower()
if "title" in name_lower or "end" in name_lower or "outputs" in str(karaoke_video):
# Search more specifically for karaoke video
karaoke_video = find_file(work_dir, "*Karaoke*.mkv", "*Karaoke*.mov", "*vocals*.mkv")
# Instrumental audio
instrumental = find_file(
work_dir,
"*instrumental_clean*.flac", "*Instrumental Clean*.flac",
"*instrumental*.flac", "*Instrumental*.flac",
"*instrumental*.wav"
)
logger.info(f"Found files:")
logger.info(f" Title video: {title_video}")
logger.info(f" Karaoke video: {karaoke_video}")
logger.info(f" End video: {end_video}")
logger.info(f" Instrumental: {instrumental}")
# Validate required files
if not title_video:
raise ValueError(f"No title video found in {work_dir}. Check screens/ subdirectory.")
if not karaoke_video:
raise ValueError(f"No karaoke video found in {work_dir}")
if not instrumental:
raise ValueError(f"No instrumental audio found in {work_dir}")
output_dir = work_dir / "outputs"
output_dir.mkdir(exist_ok=True)
jobs[job_id]["progress"] = 20
# Build encoding config with proper file names
encoding_config = EncodingConfig(
title_video=str(title_video),
karaoke_video=str(karaoke_video),
instrumental_audio=str(instrumental),
end_video=str(end_video) if end_video else None,
output_karaoke_mp4=str(output_dir / f"{base_name} (Karaoke).mp4"),
output_with_vocals_mp4=str(output_dir / f"{base_name} (With Vocals).mp4"),
output_lossless_4k_mp4=str(output_dir / f"{base_name} (Final Karaoke Lossless 4k).mp4"),
output_lossy_4k_mp4=str(output_dir / f"{base_name} (Final Karaoke Lossy 4k).mp4"),
output_lossless_mkv=str(output_dir / f"{base_name} (Final Karaoke Lossless 4k).mkv"),
output_720p_mp4=str(output_dir / f"{base_name} (Final Karaoke Lossy 720p).mp4"),
)
# Create service and run encoding
service = LocalEncodingService(logger=logger)
jobs[job_id]["progress"] = 30
logger.info("Starting LocalEncodingService.encode_all_formats()")
result = service.encode_all_formats(encoding_config)
if not result.success:
raise RuntimeError(f"Encoding failed: {result.error}")
jobs[job_id]["progress"] = 90
# Collect output files
output_files = [str(f) for f in output_dir.glob("*") if f.is_file()]
jobs[job_id]["output_files"] = output_files
logger.info(f"Encoding complete. Output files: {output_files}")
return output_dir
except ImportError as e:
# No fallback - wheel must be installed
error_msg = (
f"LocalEncodingService not available: {e}. "
"The karaoke-gen wheel must be installed. "
"Check that ensure_latest_wheel() succeeded and wheel exists in GCS."
)
logger.error(error_msg)
jobs[job_id]["status"] = "failed"
jobs[job_id]["error"] = error_msg
raise RuntimeError(error_msg) from e
except Exception as e:
logger.error(f"Encoding failed: {e}")
jobs[job_id]["status"] = "failed"
jobs[job_id]["error"] = str(e)
raise
async def process_job(job_id: str, request: EncodeRequest):
# Process an encoding job asynchronously
try:
# Download and install latest wheel at job start (allows hot updates without restart)
# This means in-progress jobs continue with their version, new jobs get latest code
ensure_latest_wheel()
with tempfile.TemporaryDirectory() as temp_dir:
work_dir = Path(temp_dir) / "work"
work_dir.mkdir()
# Download input files
jobs[job_id]["progress"] = 5
logger.info(f"Downloading from {request.input_gcs_path}")
download_from_gcs(request.input_gcs_path, work_dir)
# Run encoding in thread pool (CPU-bound)
loop = asyncio.get_event_loop()
output_dir = await loop.run_in_executor(
executor,
run_encoding,
job_id,
work_dir,
request.encoding_config
)
# Upload outputs
jobs[job_id]["progress"] = 95
logger.info(f"Uploading to {request.output_gcs_path}")
upload_to_gcs(output_dir, request.output_gcs_path)
# Convert local paths to blob paths (backend expects blob paths, not full gs:// URIs)
# output_gcs_path is like "gs://bucket/jobs/id/encoded/"
# We need paths like "jobs/id/encoded/Artist - Title (Final Karaoke Lossless 4k).mp4"
gcs_path = request.output_gcs_path.replace("gs://", "")
parts = gcs_path.split("/", 1)
prefix = parts[1].rstrip("/") if len(parts) > 1 else ""
local_output_files = jobs[job_id].get("output_files", [])
blob_paths = []
for local_path in local_output_files:
filename = Path(local_path).name
blob_path = f"{prefix}/{filename}" if prefix else filename
blob_paths.append(blob_path)
jobs[job_id]["output_files"] = blob_paths
logger.info(f"Output files (blob paths): {blob_paths}")
jobs[job_id]["status"] = "complete"
jobs[job_id]["progress"] = 100
logger.info(f"Job {job_id} complete")
except Exception as e:
logger.error(f"Job {job_id} failed: {e}")
jobs[job_id]["status"] = "failed"
jobs[job_id]["error"] = str(e)
async def process_preview_job(job_id: str, request: EncodePreviewRequest):
# Process a preview encoding job asynchronously
try:
with tempfile.TemporaryDirectory() as temp_dir:
work_dir = Path(temp_dir) / "work"
work_dir.mkdir()
# Run preview encoding in thread pool (CPU-bound)
# Note: run_preview_encoding handles download/upload internally
loop = asyncio.get_event_loop()
await loop.run_in_executor(
executor,
run_preview_encoding,
job_id,
work_dir,
request
)
jobs[job_id]["status"] = "complete"
jobs[job_id]["progress"] = 100
logger.info(f"Preview job {job_id} complete")
except Exception as e:
logger.error(f"Preview job {job_id} failed: {e}")
jobs[job_id]["status"] = "failed"
jobs[job_id]["error"] = str(e)
@app.post("/encode-preview")
async def submit_preview_encode_job(request: EncodePreviewRequest, background_tasks: BackgroundTasks, _auth: bool = Depends(verify_api_key)):
# Submit a preview encoding job
job_id = request.job_id
# If job already exists, return cached result or current status
if job_id in jobs:
existing_job = jobs[job_id]
if existing_job["status"] == "complete":
# Return cached result - preview already encoded
return {"status": "cached", "job_id": job_id, "output_path": existing_job.get("output_path")}
elif existing_job["status"] == "failed":
# Previous attempt failed, allow retry by replacing the job
pass
else:
# Job is still in progress
return {"status": "in_progress", "job_id": job_id}
jobs[job_id] = {
"job_id": job_id,
"status": "pending",
"progress": 0,
"error": None,
"output_files": None,
}
background_tasks.add_task(process_preview_job, job_id, request)
return {"status": "accepted", "job_id": job_id}
@app.post("/encode")
async def submit_encode_job(request: EncodeRequest, background_tasks: BackgroundTasks, _auth: bool = Depends(verify_api_key)):
# Submit an encoding job
job_id = request.job_id
if job_id in jobs:
raise HTTPException(status_code=409, detail=f"Job {job_id} already exists")
jobs[job_id] = {
"job_id": job_id,
"status": "pending",
"progress": 0,
"error": None,
"output_files": None,
}
background_tasks.add_task(process_job, job_id, request)
return {"status": "accepted", "job_id": job_id}
@app.get("/status/{job_id}")
async def get_job_status(job_id: str, _auth: bool = Depends(verify_api_key)) -> JobStatus:
# Get the status of an encoding job
if job_id not in jobs:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
return JobStatus(**jobs[job_id])
@app.get("/health")
async def health_check():
# Health check endpoint
active_jobs = sum(1 for j in jobs.values() if j["status"] == "running")
# Get karaoke-gen wheel version if installed
wheel_version = None
try:
from importlib.metadata import version as get_version
wheel_version = get_version("karaoke-gen")
except Exception:
pass
return {
"status": "ok",
"active_jobs": active_jobs,
"queue_length": sum(1 for j in jobs.values() if j["status"] == "pending"),
"ffmpeg_version": subprocess.run(["ffmpeg", "-version"], capture_output=True, text=True).stdout.split("\n")[0],
"wheel_version": wheel_version,
}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8080)
@@ -0,0 +1,16 @@
# GCE Encoding Worker dependencies
# These are installed in the VM's Python virtual environment
# Web framework
fastapi>=0.109.0
uvicorn>=0.27.0
# GCS client
google-cloud-storage>=2.14.0
# Async utilities (optional, for future improvements)
aiofiles>=23.2.0
aiohttp>=3.9.0
# Note: The karaoke-gen wheel is installed separately from GCS.
# It provides LocalEncodingService which does the actual encoding.
+5 -1
View File
@@ -15,6 +15,7 @@ import os
from typing import Any, Dict, Optional
from backend.config import get_settings
from karaoke_gen.utils import sanitize_filename
logger = logging.getLogger(__name__)
@@ -253,7 +254,10 @@ class GoogleDriveService:
Returns:
Dictionary mapping category to uploaded file ID
"""
filename_base = f"{brand_code} - {base_name}"
# Sanitize base_name to handle Unicode characters (curly quotes, em dashes, etc.)
# that could cause issues with Google Drive API queries and file naming
safe_base_name = sanitize_filename(base_name) if base_name else base_name
filename_base = f"{brand_code} - {safe_base_name}"
uploaded_files = {}
logger.info(
+156 -2
View File
@@ -34,10 +34,21 @@ class JobManager:
def create_job(self, job_create: JobCreate) -> Job:
"""
Create a new job with initial state PENDING.
Jobs start in PENDING state and transition to DOWNLOADING
when a worker picks them up.
Raises:
ValueError: If theme_id is not provided (all jobs require a theme)
"""
# Enforce theme requirement - all jobs must have a theme
# This prevents unstyled videos from ever being generated
if not job_create.theme_id:
raise ValueError(
"theme_id is required for all jobs. "
"Use get_theme_service().get_default_theme_id() to get the default theme."
)
job_id = str(uuid.uuid4())[:8]
now = datetime.utcnow()
@@ -74,6 +85,8 @@ class JobManager:
keep_brand_code=job_create.keep_brand_code,
# Request metadata (for tracking and filtering)
request_metadata=job_create.request_metadata,
# Tenant scoping
tenant_id=job_create.tenant_id,
)
self.firestore.create_job(job)
@@ -114,6 +127,7 @@ class JobManager:
created_after: Optional[datetime] = None,
created_before: Optional[datetime] = None,
user_email: Optional[str] = None,
tenant_id: Optional[str] = None,
limit: int = 100
) -> List[Job]:
"""
@@ -126,6 +140,7 @@ class JobManager:
created_after: Filter jobs created after this datetime
created_before: Filter jobs created before this datetime
user_email: Filter by user_email (job owner)
tenant_id: Filter by tenant_id (white-label portal scoping)
limit: Maximum number of jobs to return
Returns:
@@ -138,6 +153,7 @@ class JobManager:
created_after=created_after,
created_before=created_before,
user_email=user_email,
tenant_id=tenant_id,
limit=limit
)
@@ -359,9 +375,147 @@ class JobManager:
'instrumental_token': updates['instrumental_token'],
'instrumental_token_expires_at': updates['instrumental_token_expires_at']
})
logger.info(f"Job {job_id} transitioned to {new_status}")
# Trigger notifications asynchronously (fire-and-forget)
self._trigger_state_notifications(job_id, new_status)
return True
def _trigger_state_notifications(self, job_id: str, new_status: JobStatus) -> None:
"""
Trigger email notifications based on state transitions.
This is fire-and-forget - notification failures don't affect job processing.
Args:
job_id: Job ID
new_status: New job status
"""
import asyncio
try:
# Get the job to access user info
job = self.get_job(job_id)
if not job or not job.user_email:
logger.debug(f"No user email for job {job_id}, skipping notifications")
return
# Job completion notification
if new_status == JobStatus.COMPLETE:
self._schedule_completion_email(job)
# Idle reminder scheduling for blocking states
elif new_status in [JobStatus.AWAITING_REVIEW, JobStatus.AWAITING_INSTRUMENTAL_SELECTION]:
self._schedule_idle_reminder(job, new_status)
except Exception as e:
# Never let notification failures affect job processing
logger.error(f"Error triggering notifications for job {job_id}: {e}")
def _schedule_completion_email(self, job: Job) -> None:
"""
Schedule sending a job completion email.
Uses asyncio to fire-and-forget the email sending.
"""
import asyncio
import threading
try:
from backend.services.job_notification_service import get_job_notification_service
notification_service = get_job_notification_service()
# Get youtube, dropbox URLs, and brand_code from state_data (may be None)
state_data = job.state_data or {}
youtube_url = state_data.get('youtube_url')
dropbox_url = state_data.get('dropbox_link')
brand_code = state_data.get('brand_code')
# Create async task (fire-and-forget)
async def send_email():
await notification_service.send_job_completion_email(
job_id=job.job_id,
user_email=job.user_email,
user_name=None, # Could fetch from user service if needed
artist=job.artist,
title=job.title,
youtube_url=youtube_url,
dropbox_url=dropbox_url,
brand_code=brand_code,
)
# Try to get existing event loop, create new one if none exists
try:
loop = asyncio.get_running_loop()
# If we're in an async context, create a task
loop.create_task(send_email())
except RuntimeError:
# No event loop - we're likely in a sync context
# Use daemon thread to avoid blocking job completion
def run_in_thread():
asyncio.run(send_email())
thread = threading.Thread(target=run_in_thread, daemon=True)
thread.start()
logger.info(f"Scheduled completion email for job {job.job_id}")
except Exception as e:
logger.error(f"Failed to schedule completion email for job {job.job_id}: {e}")
def _schedule_idle_reminder(self, job: Job, new_status: JobStatus) -> None:
"""
Schedule an idle reminder for a blocking state.
Records the timestamp when the blocking state was entered and
schedules a Cloud Tasks task for 5 minutes later.
"""
import asyncio
import threading
try:
# Record when we entered blocking state (for idle detection)
blocking_entered_at = datetime.utcnow().isoformat()
action_type = "lyrics" if new_status == JobStatus.AWAITING_REVIEW else "instrumental"
# Update state_data with blocking state info (handle None state_data)
existing_state_data = job.state_data or {}
state_data_update = {
'blocking_state_entered_at': blocking_entered_at,
'blocking_action_type': action_type,
'reminder_sent': False, # Will be set to True after reminder is sent
}
self.firestore.update_job(job.job_id, {
'state_data': {**existing_state_data, **state_data_update}
})
# Schedule the idle reminder check via worker service (5 min delay)
from backend.services.worker_service import get_worker_service
async def schedule_reminder():
worker_service = get_worker_service()
await worker_service.schedule_idle_reminder(job.job_id)
# Try to get existing event loop, create new one if none exists
try:
loop = asyncio.get_running_loop()
loop.create_task(schedule_reminder())
except RuntimeError:
# No event loop - we're in a sync context
# Use daemon thread to avoid blocking job processing
def run_in_thread():
asyncio.run(schedule_reminder())
thread = threading.Thread(target=run_in_thread, daemon=True)
thread.start()
logger.info(f"Scheduled idle reminder for job {job.job_id} ({action_type})")
except Exception as e:
logger.error(f"Failed to schedule idle reminder for job {job.job_id}: {e}")
def update_state_data(self, job_id: str, key: str, value: Any) -> None:
"""
@@ -0,0 +1,271 @@
"""
Job notification service for sending emails on job state changes.
Handles:
- Job completion emails (when job enters COMPLETE status)
- Action reminder emails (when user is idle at blocking states)
This service orchestrates between the template service and email service,
building the complete notification flow for jobs.
"""
import logging
import os
from typing import Optional, Dict, Any
from backend.services.email_service import get_email_service
from backend.services.template_service import get_template_service
logger = logging.getLogger(__name__)
# Environment variable to enable/disable auto emails
ENABLE_AUTO_EMAILS = os.getenv("ENABLE_AUTO_EMAILS", "true").lower() == "true"
# Feedback form URL (configured per environment, empty by default to avoid placeholder in emails)
FEEDBACK_FORM_URL = os.getenv("FEEDBACK_FORM_URL", "")
def _mask_email(email: str) -> str:
"""Mask email for logging to protect PII. Shows first char + domain."""
if not email or "@" not in email:
return "***"
local, domain = email.split("@", 1)
if len(local) <= 1:
return f"*@{domain}"
return f"{local[0]}***@{domain}"
class JobNotificationService:
"""
Service for sending job-related email notifications.
Coordinates between template service (for message rendering) and
email service (for sending).
"""
def __init__(self):
"""Initialize notification service."""
self.email_service = get_email_service()
self.template_service = get_template_service()
self.frontend_url = os.getenv("FRONTEND_URL", "https://gen.nomadkaraoke.com")
self.backend_url = os.getenv("BACKEND_URL", "https://api.nomadkaraoke.com")
def _build_review_url(self, job_id: str, audio_hash: Optional[str] = None, review_token: Optional[str] = None) -> str:
"""Build the lyrics review URL for a job."""
import urllib.parse
review_ui_url = os.getenv("REVIEW_UI_URL", f"{self.frontend_url}/lyrics/")
base_api_url = f"{self.backend_url}/api/review/{job_id}"
encoded_api_url = urllib.parse.quote(base_api_url, safe='')
url = f"{review_ui_url}?baseApiUrl={encoded_api_url}"
if audio_hash:
url += f"&audioHash={urllib.parse.quote(audio_hash, safe='')}"
if review_token:
url += f"&reviewToken={urllib.parse.quote(review_token, safe='')}"
return url
def _build_instrumental_url(self, job_id: str, instrumental_token: Optional[str] = None) -> str:
"""Build the instrumental selection URL for a job."""
base_api_url = f"{self.backend_url}/api/jobs/{job_id}"
import urllib.parse
encoded_api_url = urllib.parse.quote(base_api_url, safe='')
url = f"{self.frontend_url}/instrumental/?baseApiUrl={encoded_api_url}"
if instrumental_token:
url += f"&instrumentalToken={urllib.parse.quote(instrumental_token, safe='')}"
return url
async def send_job_completion_email(
self,
job_id: str,
user_email: str,
user_name: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
youtube_url: Optional[str] = None,
dropbox_url: Optional[str] = None,
brand_code: Optional[str] = None,
) -> bool:
"""
Send job completion email to user.
Args:
job_id: Job ID
user_email: User's email address
user_name: User's display name (optional)
artist: Artist name
title: Song title
youtube_url: YouTube video URL
dropbox_url: Dropbox folder URL
brand_code: Release ID (e.g., "NOMAD-1178")
Returns:
True if email was sent successfully
"""
if not ENABLE_AUTO_EMAILS:
logger.info(f"Auto emails disabled, skipping completion email for job {job_id}")
return False
if not user_email:
logger.warning(f"No user email for job {job_id}, skipping completion email")
return False
try:
# Render the completion message using template service
message_content = self.template_service.render_job_completion(
name=user_name,
youtube_url=youtube_url,
dropbox_url=dropbox_url,
artist=artist,
title=title,
job_id=job_id,
feedback_url=FEEDBACK_FORM_URL,
)
# Send the email with CC to admin
success = self.email_service.send_job_completion(
to_email=user_email,
message_content=message_content,
artist=artist,
title=title,
brand_code=brand_code,
cc_admin=True,
)
if success:
logger.info(f"Sent completion email for job {job_id} to {_mask_email(user_email)}")
else:
logger.error(f"Failed to send completion email for job {job_id}")
return success
except Exception as e:
logger.exception(f"Error sending completion email for job {job_id}: {e}")
return False
async def send_action_reminder_email(
self,
job_id: str,
user_email: str,
action_type: str,
user_name: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
audio_hash: Optional[str] = None,
review_token: Optional[str] = None,
instrumental_token: Optional[str] = None,
) -> bool:
"""
Send action-needed reminder email to user.
Args:
job_id: Job ID
user_email: User's email address
action_type: Type of action needed ("lyrics" or "instrumental")
user_name: User's display name
artist: Artist name
title: Song title
audio_hash: Audio hash for review URL
review_token: Review token for URL
instrumental_token: Instrumental token for URL
Returns:
True if email was sent successfully
"""
if not ENABLE_AUTO_EMAILS:
logger.info(f"Auto emails disabled, skipping reminder for job {job_id}")
return False
if not user_email:
logger.warning(f"No user email for job {job_id}, skipping reminder")
return False
try:
# Render the appropriate template
if action_type == "lyrics":
review_url = self._build_review_url(job_id, audio_hash, review_token)
message_content = self.template_service.render_action_needed_lyrics(
name=user_name,
artist=artist,
title=title,
review_url=review_url,
)
elif action_type == "instrumental":
instrumental_url = self._build_instrumental_url(job_id, instrumental_token)
message_content = self.template_service.render_action_needed_instrumental(
name=user_name,
artist=artist,
title=title,
instrumental_url=instrumental_url,
)
else:
logger.error(f"Unknown action type: {action_type}")
return False
# Send the reminder email (no CC)
success = self.email_service.send_action_reminder(
to_email=user_email,
message_content=message_content,
action_type=action_type,
artist=artist,
title=title,
)
if success:
logger.info(f"Sent {action_type} reminder for job {job_id} to {_mask_email(user_email)}")
else:
logger.error(f"Failed to send {action_type} reminder for job {job_id}")
return success
except Exception as e:
logger.exception(f"Error sending {action_type} reminder for job {job_id}: {e}")
return False
def get_completion_message(
self,
job_id: str,
user_name: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
youtube_url: Optional[str] = None,
dropbox_url: Optional[str] = None,
) -> str:
"""
Get the rendered completion message for a job (for admin copy functionality).
Args:
job_id: Job ID
user_name: User's display name
artist: Artist name
title: Song title
youtube_url: YouTube video URL
dropbox_url: Dropbox folder URL
Returns:
Rendered message content as plain text
"""
return self.template_service.render_job_completion(
name=user_name,
youtube_url=youtube_url,
dropbox_url=dropbox_url,
artist=artist,
title=title,
job_id=job_id,
feedback_url=FEEDBACK_FORM_URL,
)
# Global instance
_job_notification_service: Optional[JobNotificationService] = None
def get_job_notification_service() -> JobNotificationService:
"""Get the global job notification service instance."""
global _job_notification_service
if _job_notification_service is None:
_job_notification_service = JobNotificationService()
return _job_notification_service
+98
View File
@@ -0,0 +1,98 @@
"""Langfuse callback handler preloader for container startup.
Initializes the Langfuse callback handler at container startup to avoid
slow initialization during request processing. The CallbackHandler()
constructor makes blocking network calls to the Langfuse API, which can
take 3+ minutes on Cloud Run cold starts.
See docs/archive/2026-01-08-performance-investigation.md for background.
"""
import logging
import os
import time
from typing import Optional, Any
logger = logging.getLogger(__name__)
# Singleton storage for preloaded handler
_preloaded_handler: Optional[Any] = None
_initialization_attempted: bool = False
def preload_langfuse_handler() -> Optional[Any]:
"""Preload Langfuse callback handler at startup.
Only initializes if LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY
environment variables are set.
Returns:
The preloaded CallbackHandler, or None if not configured
"""
global _preloaded_handler, _initialization_attempted
if _initialization_attempted:
logger.debug("Langfuse initialization already attempted")
return _preloaded_handler
_initialization_attempted = True
# Check if Langfuse is configured
public_key = os.getenv("LANGFUSE_PUBLIC_KEY")
secret_key = os.getenv("LANGFUSE_SECRET_KEY")
if not (public_key and secret_key):
logger.info("Langfuse not configured (missing keys), skipping preload")
return None
logger.info("Preloading Langfuse callback handler...")
start_time = time.time()
try:
from langfuse.langchain import CallbackHandler
# Initialize the handler - this is the slow part that makes network calls
_preloaded_handler = CallbackHandler()
elapsed = time.time() - start_time
host = os.getenv("LANGFUSE_HOST", "cloud.langfuse.com")
logger.info(f"Langfuse handler preloaded in {elapsed:.2f}s (host: {host})")
return _preloaded_handler
except ImportError:
logger.warning("langfuse package not installed, skipping preload")
return None
except Exception as e:
elapsed = time.time() - start_time
logger.error(f"Failed to preload Langfuse handler after {elapsed:.2f}s: {e}")
# Don't raise - Langfuse is optional
return None
def get_preloaded_langfuse_handler() -> Optional[Any]:
"""Get the preloaded Langfuse callback handler if available.
Returns:
The preloaded CallbackHandler, or None if not preloaded/configured
"""
return _preloaded_handler
def is_langfuse_preloaded() -> bool:
"""Check if Langfuse handler has been preloaded."""
return _preloaded_handler is not None
def is_langfuse_configured() -> bool:
"""Check if Langfuse environment variables are configured."""
return bool(
os.getenv("LANGFUSE_PUBLIC_KEY") and os.getenv("LANGFUSE_SECRET_KEY")
)
def clear_preloaded_handler() -> None:
"""Clear preloaded handler. Useful for testing."""
global _preloaded_handler, _initialization_attempted
_preloaded_handler = None
_initialization_attempted = False
@@ -0,0 +1,407 @@
"""
Local Preview Encoding Service.
Provides local FFmpeg-based preview video encoding functionality, extracted from
VideoGenerator for use by the GCE worker. This ensures the same encoding logic
is used across local CLI, Cloud Run, and GCE worker environments.
This service handles:
- Generating preview videos with ASS subtitle overlay
- Hardware acceleration detection (NVENC) with fallback to software encoding
- Background image/color support
- Custom font support
- Optimized settings for fast preview generation
"""
import logging
import os
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import Optional, List
logger = logging.getLogger(__name__)
@dataclass
class PreviewEncodingConfig:
"""Configuration for preview video encoding."""
ass_path: str # Path to ASS subtitles file
audio_path: str # Path to audio file
output_path: str # Path for output video file
# Optional background settings
background_image_path: Optional[str] = None # Path to background image
background_color: str = "black" # Fallback background color
# Optional font settings
font_path: Optional[str] = None # Path to custom font file
@dataclass
class PreviewEncodingResult:
"""Result of preview encoding operation."""
success: bool
output_path: Optional[str] = None
error: Optional[str] = None
class LocalPreviewEncodingService:
"""
Service for local FFmpeg-based preview video encoding.
This is the single source of truth for preview encoding logic, used by:
- Local CLI (via VideoGenerator which delegates here)
- Cloud Run (when GCE is unavailable)
- GCE worker (via installed wheel)
Supports hardware acceleration (NVENC) with automatic fallback
to software encoding (libx264) when unavailable.
"""
# Preview video settings - these are the canonical values
PREVIEW_WIDTH = 480
PREVIEW_HEIGHT = 270
PREVIEW_FPS = 24
PREVIEW_AUDIO_BITRATE = "96k"
def __init__(
self,
logger: Optional[logging.Logger] = None,
):
"""
Initialize the local preview encoding service.
Args:
logger: Optional logger instance
"""
self.logger = logger or logging.getLogger(__name__)
# Hardware acceleration settings (detected on first use)
self._nvenc_available: Optional[bool] = None
self._video_encoder: Optional[str] = None
self._hwaccel_flags: Optional[List[str]] = None
def _detect_nvenc_support(self) -> bool:
"""
Detect if NVENC hardware encoding is available.
Returns:
True if NVENC is available, False otherwise
"""
try:
self.logger.info("Detecting NVENC hardware acceleration...")
# Test h264_nvenc encoder directly
test_cmd = [
"ffmpeg", "-hide_banner", "-loglevel", "error",
"-f", "lavfi", "-i", "testsrc=duration=1:size=320x240:rate=1",
"-c:v", "h264_nvenc", "-f", "null", "-"
]
result = subprocess.run(
test_cmd,
capture_output=True,
text=True,
timeout=30
)
if result.returncode == 0:
self.logger.info("NVENC hardware acceleration available")
return True
# Try alternative test with different source
alt_test_cmd = [
"ffmpeg", "-hide_banner", "-loglevel", "error",
"-f", "lavfi", "-i", "color=red:size=320x240:duration=0.1",
"-c:v", "h264_nvenc", "-preset", "fast", "-f", "null", "-"
]
alt_result = subprocess.run(
alt_test_cmd,
capture_output=True,
text=True,
timeout=30
)
if alt_result.returncode == 0:
self.logger.info("NVENC hardware acceleration available")
return True
self.logger.info("NVENC not available, using software encoding")
return False
except subprocess.TimeoutExpired:
self.logger.debug("NVENC detection timed out")
return False
except Exception as e:
self.logger.debug(f"NVENC detection failed: {e}")
return False
def _configure_hardware_acceleration(self) -> None:
"""Configure hardware acceleration settings based on detected capabilities."""
if self._nvenc_available is None:
self._nvenc_available = self._detect_nvenc_support()
if self._nvenc_available:
self._video_encoder = "h264_nvenc"
self._hwaccel_flags = ["-hwaccel", "cuda", "-hwaccel_output_format", "cuda"]
self.logger.info("Using NVENC hardware acceleration for preview encoding")
else:
self._video_encoder = "libx264"
self._hwaccel_flags = []
self.logger.info("Using software encoding (libx264) for preview")
@property
def nvenc_available(self) -> bool:
"""Check if NVENC hardware acceleration is available."""
if self._nvenc_available is None:
self._configure_hardware_acceleration()
return self._nvenc_available
@property
def video_encoder(self) -> str:
"""Get the video encoder to use."""
if self._video_encoder is None:
self._configure_hardware_acceleration()
return self._video_encoder
@property
def hwaccel_flags(self) -> List[str]:
"""Get hardware acceleration flags."""
if self._hwaccel_flags is None:
self._configure_hardware_acceleration()
return self._hwaccel_flags
def _escape_ffmpeg_filter_path(self, path: str) -> str:
"""
Escape a path for FFmpeg filter expressions (for subprocess without shell).
When using subprocess with a command list (no shell), FFmpeg receives the
filter string directly. FFmpeg's filter parser requires escaping:
- Backslashes: double them (\\ -> \\\\)
- Single quotes/apostrophes: escape with three backslashes (' -> \\\\')
- Spaces: escape with backslash ( -> \\ )
- Special characters: :,[];
Example: "I'm With You" becomes "I\\\\'m\\ With\\ You"
"""
# First escape existing backslashes
escaped = path.replace("\\", "\\\\")
# Escape single quotes
escaped = escaped.replace("'", "\\\\\\'")
# Escape spaces
escaped = escaped.replace(" ", "\\ ")
# Escape FFmpeg filter special characters
escaped = escaped.replace(":", "\\:")
escaped = escaped.replace(",", "\\,")
escaped = escaped.replace("[", "\\[")
escaped = escaped.replace("]", "\\]")
escaped = escaped.replace(";", "\\;")
return escaped
def _build_ass_filter(self, ass_path: str, font_path: Optional[str] = None) -> str:
"""
Build ASS filter with optional font directory support.
Args:
ass_path: Path to ASS subtitles file
font_path: Optional path to custom font file
Returns:
FFmpeg ASS filter string
"""
escaped_ass_path = self._escape_ffmpeg_filter_path(ass_path)
ass_filter = f"ass={escaped_ass_path}"
if font_path and os.path.isfile(font_path):
font_dir = os.path.dirname(font_path)
escaped_font_dir = self._escape_ffmpeg_filter_path(font_dir)
ass_filter += f":fontsdir={escaped_font_dir}"
self.logger.debug(f"Using font directory: {font_dir}")
return ass_filter
def _build_preview_ffmpeg_command(self, config: PreviewEncodingConfig) -> List[str]:
"""
Build FFmpeg command for preview video generation.
This is the canonical preview encoding command, ensuring consistency
across all environments (local CLI, Cloud Run, GCE worker).
Args:
config: Preview encoding configuration
Returns:
FFmpeg command as a list of arguments
"""
width, height = self.PREVIEW_WIDTH, self.PREVIEW_HEIGHT
cmd = [
"ffmpeg",
"-hide_banner",
"-loglevel", "error",
"-r", str(self.PREVIEW_FPS),
]
# Add hardware acceleration flags if available
cmd.extend(self.hwaccel_flags)
# Input source (background image or solid color)
if config.background_image_path and os.path.isfile(config.background_image_path):
self.logger.debug(f"Using background image: {config.background_image_path}")
cmd.extend([
"-loop", "1",
"-i", config.background_image_path,
])
# Build video filter with scaling and ASS subtitles
video_filter = (
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2,"
f"{self._build_ass_filter(config.ass_path, config.font_path)}"
)
else:
self.logger.debug(f"Using solid {config.background_color} background")
cmd.extend([
"-f", "lavfi",
"-i", f"color=c={config.background_color}:s={width}x{height}:r={self.PREVIEW_FPS}",
])
# Just ASS subtitles, no scaling needed
video_filter = self._build_ass_filter(config.ass_path, config.font_path)
cmd.extend([
"-i", config.audio_path,
"-vf", video_filter,
"-c:a", "aac",
"-b:a", self.PREVIEW_AUDIO_BITRATE,
"-c:v", self.video_encoder,
])
# Add encoder-specific settings optimized for speed
if self.nvenc_available:
cmd.extend([
"-preset", "p1", # Fastest NVENC preset
"-tune", "ll", # Low latency
"-rc", "cbr", # Constant bitrate for speed
"-b:v", "800k", # Lower bitrate for speed
"-profile:v", "baseline", # Most compatible profile
"-level", "3.1", # Lower level for speed
])
self.logger.debug("Using NVENC with maximum speed settings")
else:
cmd.extend([
"-profile:v", "baseline",
"-level", "3.0",
"-preset", "superfast",
"-tune", "fastdecode",
"-b:v", "600k",
"-maxrate", "800k",
"-bufsize", "1200k",
"-crf", "28",
])
self.logger.debug("Using software encoding with maximum speed settings")
cmd.extend([
"-pix_fmt", "yuv420p", # Required for browser compatibility
"-movflags", "+faststart+frag_keyframe+empty_moov+dash",
"-g", "48", # Keyframe every 48 frames (2 seconds at 24fps)
"-keyint_min", "48",
"-sc_threshold", "0", # Disable scene change detection for speed
"-threads", "0", # Use all available CPU threads
"-shortest",
"-y",
config.output_path,
])
return cmd
def encode_preview(self, config: PreviewEncodingConfig) -> PreviewEncodingResult:
"""
Encode a preview video.
Args:
config: Preview encoding configuration
Returns:
PreviewEncodingResult with success status and output path
"""
self.logger.info(f"Encoding preview video: {config.output_path}")
# Validate input files
if not os.path.isfile(config.ass_path):
return PreviewEncodingResult(
success=False,
error=f"ASS subtitles file not found: {config.ass_path}"
)
if not os.path.isfile(config.audio_path):
return PreviewEncodingResult(
success=False,
error=f"Audio file not found: {config.audio_path}"
)
# Ensure output directory exists
output_dir = os.path.dirname(config.output_path)
if output_dir:
os.makedirs(output_dir, exist_ok=True)
try:
# Build and execute FFmpeg command
cmd = self._build_preview_ffmpeg_command(config)
self.logger.debug(f"FFmpeg command: {' '.join(cmd)}")
result = subprocess.run(
cmd,
capture_output=True,
text=True,
timeout=300 # 5 minute timeout for preview encoding
)
if result.returncode != 0:
error_msg = result.stderr[-500:] if result.stderr else "Unknown error"
self.logger.error(f"FFmpeg failed: {error_msg}")
return PreviewEncodingResult(
success=False,
error=f"FFmpeg preview encoding failed: {error_msg}"
)
self.logger.info(f"Preview encoded successfully: {config.output_path}")
return PreviewEncodingResult(
success=True,
output_path=config.output_path
)
except subprocess.TimeoutExpired:
self.logger.error("Preview encoding timed out")
return PreviewEncodingResult(
success=False,
error="Preview encoding timed out after 5 minutes"
)
except Exception as e:
self.logger.error(f"Preview encoding failed: {e}")
return PreviewEncodingResult(
success=False,
error=str(e)
)
# Singleton instance and factory function
_local_preview_encoding_service: Optional[LocalPreviewEncodingService] = None
def get_local_preview_encoding_service(**kwargs) -> LocalPreviewEncodingService:
"""
Get a local preview encoding service instance.
Args:
**kwargs: Arguments passed to LocalPreviewEncodingService
Returns:
LocalPreviewEncodingService instance
"""
global _local_preview_encoding_service
if _local_preview_encoding_service is None:
_local_preview_encoding_service = LocalPreviewEncodingService(**kwargs)
return _local_preview_encoding_service
+122
View File
@@ -0,0 +1,122 @@
"""NLTK resource preloader for container startup.
Loads NLTK data at container startup to avoid slow downloads during request processing.
Cloud Run's ephemeral filesystem means NLTK data must be re-downloaded on each cold start,
which can take 30-100+ seconds for cmudict.
See docs/archive/2026-01-08-performance-investigation.md for background.
"""
import logging
import time
from typing import Optional, Dict, Any
logger = logging.getLogger(__name__)
# Singleton storage for preloaded resources
_preloaded_resources: Dict[str, Any] = {}
def preload_nltk_cmudict() -> None:
"""Preload NLTK's CMU Pronouncing Dictionary at startup.
The cmudict is used by SyllablesMatchHandler for syllable counting.
Without preloading, each SyllablesMatchHandler init downloads ~30MB,
which took 50-100+ seconds in Cloud Run.
"""
global _preloaded_resources
if "cmudict" in _preloaded_resources:
logger.info("NLTK cmudict already preloaded")
return
logger.info("Preloading NLTK cmudict...")
start_time = time.time()
try:
import nltk
# Ensure the data is downloaded
try:
from nltk.corpus import cmudict
# Try to access it - will raise LookupError if not downloaded
_ = cmudict.dict()
except LookupError:
logger.info("Downloading NLTK cmudict data...")
nltk.download("cmudict", quiet=True)
from nltk.corpus import cmudict
# Load into memory
cmu_dict = cmudict.dict()
_preloaded_resources["cmudict"] = cmu_dict
elapsed = time.time() - start_time
logger.info(f"NLTK cmudict preloaded in {elapsed:.2f}s ({len(cmu_dict)} entries)")
except Exception as e:
logger.error(f"Failed to preload NLTK cmudict: {e}")
raise
def get_preloaded_cmudict() -> Optional[Dict]:
"""Get the preloaded CMU dictionary if available.
Returns:
The preloaded cmudict dictionary, or None if not preloaded
"""
return _preloaded_resources.get("cmudict")
def is_cmudict_preloaded() -> bool:
"""Check if cmudict has been preloaded."""
return "cmudict" in _preloaded_resources
def preload_nltk_punkt() -> None:
"""Preload NLTK's punkt tokenizer (optional, used for sentence tokenization)."""
global _preloaded_resources
if "punkt" in _preloaded_resources:
logger.info("NLTK punkt already preloaded")
return
logger.info("Preloading NLTK punkt tokenizer...")
start_time = time.time()
try:
import nltk
try:
from nltk.tokenize import word_tokenize
# Test it works
_ = word_tokenize("test")
except LookupError:
logger.info("Downloading NLTK punkt data...")
nltk.download("punkt", quiet=True)
nltk.download("punkt_tab", quiet=True)
_preloaded_resources["punkt"] = True
elapsed = time.time() - start_time
logger.info(f"NLTK punkt preloaded in {elapsed:.2f}s")
except Exception as e:
logger.warning(f"Failed to preload NLTK punkt (non-critical): {e}")
def preload_all_nltk_resources() -> None:
"""Preload all NLTK resources used by the application."""
preload_nltk_cmudict()
# punkt is optional and less critical
try:
preload_nltk_punkt()
except Exception:
pass # Non-critical
def clear_preloaded_resources() -> None:
"""Clear all preloaded resources. Useful for testing."""
global _preloaded_resources
_preloaded_resources.clear()
+65
View File
@@ -0,0 +1,65 @@
"""SpaCy model preloader for container startup.
Loads SpaCy models at container startup to avoid slow loading during request processing.
Cloud Run filesystem I/O can cause 60+ second delays when loading SpaCy models lazily.
"""
import logging
import time
from typing import Optional
logger = logging.getLogger(__name__)
# Singleton storage for preloaded models
_preloaded_models: dict = {}
def preload_spacy_model(model_name: str = "en_core_web_sm") -> None:
"""Preload a SpaCy model at startup.
Args:
model_name: The SpaCy model to load (default: en_core_web_sm)
"""
global _preloaded_models
if model_name in _preloaded_models:
logger.info(f"SpaCy model '{model_name}' already preloaded")
return
logger.info(f"Preloading SpaCy model '{model_name}'...")
start_time = time.time()
try:
import spacy
nlp = spacy.load(model_name)
_preloaded_models[model_name] = nlp
elapsed = time.time() - start_time
logger.info(f"SpaCy model '{model_name}' preloaded in {elapsed:.2f}s")
except Exception as e:
logger.error(f"Failed to preload SpaCy model '{model_name}': {e}")
raise
def get_preloaded_model(model_name: str = "en_core_web_sm") -> Optional[object]:
"""Get a preloaded SpaCy model if available.
Args:
model_name: The SpaCy model name
Returns:
The preloaded SpaCy Language object, or None if not preloaded
"""
return _preloaded_models.get(model_name)
def is_model_preloaded(model_name: str = "en_core_web_sm") -> bool:
"""Check if a SpaCy model has been preloaded."""
return model_name in _preloaded_models
def clear_preloaded_models() -> None:
"""Clear all preloaded models. Useful for testing."""
global _preloaded_models
_preloaded_models.clear()
+100 -1
View File
@@ -46,6 +46,14 @@ CREDIT_PACKAGES = {
},
}
# Made-for-you service package (not a credit package - creates a job directly)
MADE_FOR_YOU_PACKAGE = {
"price_cents": 1500, # $15.00
"name": "Made For You Karaoke Video",
"description": "Professional 4K karaoke video with perfectly synced lyrics, delivered to your email within 24 hours",
"images": ["https://gen.nomadkaraoke.com/nomad-logo.png"],
}
class StripeService:
"""Service for Stripe payment processing."""
@@ -108,13 +116,14 @@ class StripeService:
# Create checkout session
session = stripe.checkout.Session.create(
payment_method_types=['card'],
# Omit payment_method_types to auto-enable Apple Pay, Google Pay, Link, etc.
line_items=[{
'price_data': {
'currency': 'usd',
'product_data': {
'name': package['name'],
'description': package['description'],
'images': ['https://gen.nomadkaraoke.com/nomad-logo.png'],
},
'unit_amount': package['price_cents'],
},
@@ -143,6 +152,96 @@ class StripeService:
logger.error(f"Error creating checkout session: {e}")
return False, None, "Failed to create checkout session"
def create_made_for_you_checkout_session(
self,
customer_email: str,
artist: str,
title: str,
source_type: str = "search",
youtube_url: Optional[str] = None,
notes: Optional[str] = None,
success_url: Optional[str] = None,
cancel_url: Optional[str] = None,
) -> Tuple[bool, Optional[str], str]:
"""
Create a Stripe Checkout session for a made-for-you order.
This is for the full-service karaoke video creation where Nomad Karaoke
handles all the work (lyrics review, instrumental selection, etc.).
Args:
customer_email: Customer's email for delivery
artist: Song artist
title: Song title
source_type: Audio source type (search, youtube, upload)
youtube_url: YouTube URL if source_type is youtube
notes: Any special requests from customer
success_url: URL to redirect to on success (optional)
cancel_url: URL to redirect to on cancel (optional)
Returns:
(success, checkout_url, message)
"""
if not self.is_configured():
return False, None, "Payment processing is not configured"
try:
# Default URLs - redirect to homepage success page
if not success_url:
success_url = "https://nomadkaraoke.com/order/success/?session_id={CHECKOUT_SESSION_ID}"
if not cancel_url:
cancel_url = "https://nomadkaraoke.com/#do-it-for-me"
# Build metadata for job creation after payment
metadata = {
'order_type': 'made_for_you',
'customer_email': customer_email,
'artist': artist,
'title': title,
'source_type': source_type,
}
if youtube_url:
metadata['youtube_url'] = youtube_url
if notes:
# Truncate notes to fit Stripe's 500 char limit per metadata value
metadata['notes'] = notes[:500] if len(notes) > 500 else notes
# Create checkout session
session = stripe.checkout.Session.create(
# Omit payment_method_types to auto-enable Apple Pay, Google Pay, Link, etc.
line_items=[{
'price_data': {
'currency': 'usd',
'product_data': {
'name': f"Karaoke Video: {artist} - {title}",
'description': MADE_FOR_YOU_PACKAGE['description'],
'images': MADE_FOR_YOU_PACKAGE['images'],
},
'unit_amount': MADE_FOR_YOU_PACKAGE['price_cents'],
},
'quantity': 1,
}],
mode='payment',
success_url=success_url,
cancel_url=cancel_url,
customer_email=customer_email,
metadata=metadata,
allow_promotion_codes=True,
)
logger.info(
f"Created made-for-you checkout session {session.id} for {customer_email}, "
f"song: {artist} - {title}"
)
return True, session.url, "Checkout session created"
except stripe.error.StripeError as e:
logger.error(f"Stripe error creating made-for-you checkout session: {e}")
return False, None, f"Payment error: {str(e)}"
except Exception as e:
logger.error(f"Error creating made-for-you checkout session: {e}")
return False, None, "Failed to create checkout session"
def verify_webhook_signature(self, payload: bytes, signature: str) -> Tuple[bool, Optional[Dict], str]:
"""
Verify a Stripe webhook signature.
+330
View File
@@ -0,0 +1,330 @@
"""
Template service for managing email templates stored in GCS.
Templates are stored in GCS bucket and rendered with job-specific variables.
This allows updating email content without code deployment.
Template locations:
- gs://{bucket}/templates/job-completion.txt - Job completion email (plain text)
- gs://{bucket}/templates/action-needed-lyrics.txt - Lyrics review reminder
- gs://{bucket}/templates/action-needed-instrumental.txt - Instrumental selection reminder
"""
import logging
import re
from typing import Optional, Dict, Any
from backend.config import get_settings
logger = logging.getLogger(__name__)
# Default templates (fallback if GCS fetch fails)
DEFAULT_JOB_COMPLETION_TEMPLATE = """Hi {name},
Thanks for your order!
Here's the link for the karaoke video published to YouTube:
{youtube_url}
Here's the dropbox folder with all the finished files and source files, including:
- "(Final Karaoke Lossless).mkv": combined karaoke video in 4k H264 with lossless FLAC audio
- "(Final Karaoke).mp4": combined karaoke video with title/end screen in 4k H264/AAC
- "(Final Karaoke 720p).mp4": combined karaoke video in 720p H264/AAC (smaller file for older systems)
- "(With Vocals).mp4": sing along video in 4k H264/AAC with original vocals
- "(Karaoke).mov": karaoke video output from MidiCo (no title/end screen)
- "(Title).mov"/"(End).mov": title card and end screen videos
- "(Final Karaoke CDG).zip": CDG+MP3 format for older/commercial karaoke systems
- "(Final Karaoke TXT).zip": TXT+MP3 format for Power Karaoke
- stems/*.flac: various separated instrumental and vocal audio stems in lossless format
- lyrics/*.txt song lyrics from various sources in plain text format
{dropbox_url}
Let me know if anything isn't perfect and I'll happily tweak / fix, or if you need it in any other format I can probably convert it for you!
If you have a moment, I'd really appreciate your feedback (takes 2 minutes):
{feedback_url}
Thanks again and have a great day!
-Andrew
"""
DEFAULT_ACTION_NEEDED_LYRICS_TEMPLATE = """Hi {name},
Your karaoke video for "{artist} - {title}" is ready for lyrics review!
Our system has transcribed and synchronized the lyrics, but they may need some corrections. Please review and make any corrections necessary so we can finish generating your video.
Review your lyrics here:
{review_url}
This usually takes just a few minutes.
Thanks!
-Andrew
"""
DEFAULT_ACTION_NEEDED_INSTRUMENTAL_TEMPLATE = """Hi {name},
Your karaoke video for "{artist} - {title}" is almost done!
We've separated the audio into different versions. Please select which instrumental track you'd like to use for the final video.
Select your instrumental here:
{instrumental_url}
Thanks!
-Andrew
"""
class TemplateService:
"""
Service for fetching and rendering email templates from GCS.
Templates support the following variables:
- {name} - User's display name or "there" if unknown
- {youtube_url} - YouTube video URL
- {dropbox_url} - Dropbox folder URL
- {artist} - Artist name
- {title} - Song title
- {job_id} - Job ID
- {review_url} - Lyrics review URL
- {instrumental_url} - Instrumental selection URL
- {feedback_url} - Feedback form URL
"""
TEMPLATE_PREFIX = "templates/"
def __init__(self):
"""Initialize template service."""
self.settings = get_settings()
self._storage_client = None
self._bucket = None
@property
def storage_client(self):
"""Lazy-initialize storage client."""
if self._storage_client is None:
from google.cloud import storage
self._storage_client = storage.Client(project=self.settings.google_cloud_project)
return self._storage_client
@property
def bucket(self):
"""Get the GCS bucket for templates."""
if self._bucket is None:
self._bucket = self.storage_client.bucket(self.settings.gcs_bucket_name)
return self._bucket
def _fetch_template_from_gcs(self, template_name: str) -> Optional[str]:
"""
Fetch a template from GCS.
Args:
template_name: Name of template file (e.g., "job-completion.txt")
Returns:
Template content as string, or None if not found
"""
blob_path = f"{self.TEMPLATE_PREFIX}{template_name}"
try:
blob = self.bucket.blob(blob_path)
if blob.exists():
content = blob.download_as_text()
logger.debug(f"Fetched template from GCS: {blob_path}")
return content
else:
logger.warning(f"Template not found in GCS: {blob_path}")
return None
except Exception as e:
logger.error(f"Failed to fetch template {blob_path}: {e}")
return None
def get_job_completion_template(self) -> str:
"""Get the job completion email template."""
template = self._fetch_template_from_gcs("job-completion.txt")
if template is None:
logger.info("Using default job completion template")
return DEFAULT_JOB_COMPLETION_TEMPLATE
return template
def get_action_needed_lyrics_template(self) -> str:
"""Get the lyrics review reminder template."""
template = self._fetch_template_from_gcs("action-needed-lyrics.txt")
if template is None:
logger.info("Using default lyrics reminder template")
return DEFAULT_ACTION_NEEDED_LYRICS_TEMPLATE
return template
def get_action_needed_instrumental_template(self) -> str:
"""Get the instrumental selection reminder template."""
template = self._fetch_template_from_gcs("action-needed-instrumental.txt")
if template is None:
logger.info("Using default instrumental reminder template")
return DEFAULT_ACTION_NEEDED_INSTRUMENTAL_TEMPLATE
return template
def render_template(self, template: str, variables: Dict[str, Any]) -> str:
"""
Render a template with the given variables.
Missing variables are replaced with empty strings.
Handles conditional sections like feedback URL.
Args:
template: Template string with {variable} placeholders
variables: Dictionary of variable values
Returns:
Rendered template string
"""
result = template
# Handle feedback URL section - remove if not provided
if not variables.get("feedback_url"):
# Remove feedback section (lines containing feedback_url placeholder and surrounding text)
result = re.sub(
r'\n*If you have a moment.*?\{feedback_url\}\n*',
'\n',
result,
flags=re.DOTALL
)
variables["feedback_url"] = ""
# Replace all variables
for key, value in variables.items():
placeholder = "{" + key + "}"
result = result.replace(placeholder, str(value) if value else "")
# Clean up any remaining unreplaced placeholders
result = re.sub(r'\{[a-z_]+\}', '', result)
return result.strip()
def render_job_completion(
self,
name: Optional[str] = None,
youtube_url: Optional[str] = None,
dropbox_url: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
job_id: Optional[str] = None,
feedback_url: Optional[str] = None,
) -> str:
"""
Render the job completion email template.
Args:
name: User's display name (defaults to "there")
youtube_url: YouTube video URL
dropbox_url: Dropbox folder URL
artist: Artist name
title: Song title
job_id: Job ID
feedback_url: Feedback form URL (optional)
Returns:
Rendered email content
"""
template = self.get_job_completion_template()
variables = {
"name": name or "there",
"youtube_url": youtube_url or "[YouTube URL not available]",
"dropbox_url": dropbox_url or "[Dropbox URL not available]",
"artist": artist or "Unknown Artist",
"title": title or "Unknown Title",
"job_id": job_id or "",
"feedback_url": feedback_url,
}
return self.render_template(template, variables)
def render_action_needed_lyrics(
self,
name: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
review_url: str = "",
) -> str:
"""
Render the lyrics review reminder template.
Args:
name: User's display name
artist: Artist name
title: Song title
review_url: Lyrics review URL
Returns:
Rendered email content
"""
template = self.get_action_needed_lyrics_template()
variables = {
"name": name or "there",
"artist": artist or "Unknown Artist",
"title": title or "Unknown Title",
"review_url": review_url,
}
return self.render_template(template, variables)
def render_action_needed_instrumental(
self,
name: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
instrumental_url: str = "",
) -> str:
"""
Render the instrumental selection reminder template.
Args:
name: User's display name
artist: Artist name
title: Song title
instrumental_url: Instrumental selection URL
Returns:
Rendered email content
"""
template = self.get_action_needed_instrumental_template()
variables = {
"name": name or "there",
"artist": artist or "Unknown Artist",
"title": title or "Unknown Title",
"instrumental_url": instrumental_url,
}
return self.render_template(template, variables)
def upload_template(self, template_name: str, content: str) -> bool:
"""
Upload a template to GCS.
Args:
template_name: Name of template file (e.g., "job-completion.txt")
content: Template content
Returns:
True if upload successful
"""
blob_path = f"{self.TEMPLATE_PREFIX}{template_name}"
try:
blob = self.bucket.blob(blob_path)
blob.upload_from_string(content, content_type="text/plain")
logger.info(f"Uploaded template to GCS: {blob_path}")
return True
except Exception as e:
logger.error(f"Failed to upload template {blob_path}: {e}")
return False
# Global instance
_template_service: Optional[TemplateService] = None
def get_template_service() -> TemplateService:
"""Get the global template service instance."""
global _template_service
if _template_service is None:
_template_service = TemplateService()
return _template_service
+285
View File
@@ -0,0 +1,285 @@
"""
Tenant service for managing white-label B2B portal configurations.
Tenant configs are stored in GCS at tenants/{tenant_id}/config.json.
The service provides:
- Tenant config loading with caching
- Subdomain to tenant ID resolution
- Tenant validation
Similar to ThemeService, configs are cached for 5 minutes to reduce GCS reads.
"""
import logging
import threading
from datetime import datetime
from typing import Dict, Optional
from backend.models.tenant import TenantConfig, TenantPublicConfig
from backend.services.storage_service import StorageService
logger = logging.getLogger(__name__)
# GCS paths for tenant configs
TENANTS_PREFIX = "tenants"
# Default sender email (shared with EmailService for consistency)
DEFAULT_SENDER_EMAIL = "gen@nomadkaraoke.com"
class TenantService:
"""Service for managing tenant configurations from GCS."""
def __init__(self, storage: Optional[StorageService] = None):
"""
Initialize the tenant service.
Args:
storage: StorageService instance (creates new one if not provided)
"""
self.storage = storage or StorageService()
self._config_cache: Dict[str, TenantConfig] = {}
self._cache_times: Dict[str, datetime] = {}
self._subdomain_map: Dict[str, str] = {} # subdomain -> tenant_id
self._subdomain_map_time: Optional[datetime] = None
self.CACHE_TTL_SECONDS = 300 # 5 minute cache
def _get_config_path(self, tenant_id: str) -> str:
"""Get the GCS path for a tenant's config file."""
return f"{TENANTS_PREFIX}/{tenant_id}/config.json"
def _is_cache_valid(self, tenant_id: str) -> bool:
"""Check if the config cache for a tenant is still valid."""
if tenant_id not in self._config_cache or tenant_id not in self._cache_times:
return False
age = datetime.now() - self._cache_times[tenant_id]
return age.total_seconds() < self.CACHE_TTL_SECONDS
def _is_subdomain_map_valid(self) -> bool:
"""Check if the subdomain map cache is still valid."""
if not self._subdomain_map or self._subdomain_map_time is None:
return False
age = datetime.now() - self._subdomain_map_time
return age.total_seconds() < self.CACHE_TTL_SECONDS
def get_tenant_config(
self, tenant_id: str, force_refresh: bool = False
) -> Optional[TenantConfig]:
"""
Load tenant configuration from GCS with caching.
Args:
tenant_id: The tenant identifier
force_refresh: Force reload from GCS even if cache is valid
Returns:
TenantConfig if found, None otherwise
"""
if not force_refresh and self._is_cache_valid(tenant_id):
return self._config_cache[tenant_id]
try:
config_path = self._get_config_path(tenant_id)
if not self.storage.file_exists(config_path):
logger.debug(f"Tenant config not found: {tenant_id}")
return None
data = self.storage.download_json(config_path)
config = TenantConfig(**data)
# Update cache
self._config_cache[tenant_id] = config
self._cache_times[tenant_id] = datetime.now()
# Update subdomain map and its timestamp
self._subdomain_map[config.subdomain.lower()] = tenant_id
self._subdomain_map_time = datetime.now()
logger.info(f"Loaded tenant config: {tenant_id}")
return config
except Exception as e:
logger.error(f"Failed to load tenant config {tenant_id}: {e}")
return None
def get_tenant_by_subdomain(self, subdomain: str) -> Optional[TenantConfig]:
"""
Get tenant config by subdomain.
Args:
subdomain: Full subdomain (e.g., 'vocalstar.nomadkaraoke.com')
Returns:
TenantConfig if found, None otherwise
"""
subdomain_lower = subdomain.lower()
# Check cache first
if subdomain_lower in self._subdomain_map:
tenant_id = self._subdomain_map[subdomain_lower]
return self.get_tenant_config(tenant_id)
# Try to find tenant by listing all configs
# This is a fallback for when we don't have the subdomain cached
tenant_id = self._resolve_subdomain_to_tenant_id(subdomain_lower)
if tenant_id:
return self.get_tenant_config(tenant_id)
return None
def _resolve_subdomain_to_tenant_id(self, subdomain: str) -> Optional[str]:
"""
Resolve a subdomain to tenant ID by checking GCS.
This extracts the tenant ID from the subdomain pattern:
- 'vocalstar.nomadkaraoke.com' -> 'vocalstar'
- 'vocalstar.gen.nomadkaraoke.com' -> 'vocalstar'
"""
# Extract potential tenant ID from subdomain
# Pattern: {tenant}.nomadkaraoke.com or {tenant}.gen.nomadkaraoke.com
parts = subdomain.split(".")
if len(parts) >= 3:
# First part is likely the tenant ID
potential_tenant_id = parts[0]
# Check if config exists
config_path = self._get_config_path(potential_tenant_id)
if self.storage.file_exists(config_path):
return potential_tenant_id
return None
def get_public_config(self, tenant_id: str) -> Optional[TenantPublicConfig]:
"""
Get the public (frontend-safe) tenant configuration.
Args:
tenant_id: The tenant identifier
Returns:
TenantPublicConfig if found, None otherwise
"""
config = self.get_tenant_config(tenant_id)
if config:
return TenantPublicConfig.from_config(config)
return None
def get_public_config_by_subdomain(
self, subdomain: str
) -> Optional[TenantPublicConfig]:
"""
Get the public tenant configuration by subdomain.
Args:
subdomain: Full subdomain
Returns:
TenantPublicConfig if found, None otherwise
"""
config = self.get_tenant_by_subdomain(subdomain)
if config:
return TenantPublicConfig.from_config(config)
return None
def tenant_exists(self, tenant_id: str) -> bool:
"""
Check if a tenant exists.
Args:
tenant_id: The tenant identifier
Returns:
True if tenant config exists, False otherwise
"""
config_path = self._get_config_path(tenant_id)
return self.storage.file_exists(config_path)
def is_email_allowed_for_tenant(self, tenant_id: str, email: str) -> bool:
"""
Check if an email is allowed for a specific tenant.
Args:
tenant_id: The tenant identifier
email: Email address to check
Returns:
True if email is allowed, False otherwise
"""
config = self.get_tenant_config(tenant_id)
if not config:
return False
return config.is_email_allowed(email)
def get_tenant_sender_email(self, tenant_id: str) -> str:
"""
Get the email sender address for a tenant.
Args:
tenant_id: The tenant identifier
Returns:
Sender email address
"""
config = self.get_tenant_config(tenant_id)
if config:
return config.get_sender_email()
# Default fallback (consistent with EmailService)
return DEFAULT_SENDER_EMAIL
def invalidate_cache(self, tenant_id: Optional[str] = None) -> None:
"""
Invalidate tenant config cache.
Args:
tenant_id: Specific tenant to invalidate, or None for all
"""
if tenant_id:
self._config_cache.pop(tenant_id, None)
self._cache_times.pop(tenant_id, None)
# Also remove any subdomain map entries pointing to this tenant
subdomains_to_remove = [
subdomain
for subdomain, tid in self._subdomain_map.items()
if tid == tenant_id
]
for subdomain in subdomains_to_remove:
self._subdomain_map.pop(subdomain, None)
logger.info(f"Tenant config cache invalidated: {tenant_id}")
else:
self._config_cache.clear()
self._cache_times.clear()
self._subdomain_map.clear()
self._subdomain_map_time = None
logger.info("All tenant config caches invalidated")
def get_asset_url(self, tenant_id: str, asset_name: str) -> Optional[str]:
"""
Get a signed URL for a tenant asset (logo, favicon, etc.).
Args:
tenant_id: The tenant identifier
asset_name: Asset filename (e.g., 'logo.png')
Returns:
Signed URL if asset exists, None otherwise
"""
asset_path = f"{TENANTS_PREFIX}/{tenant_id}/{asset_name}"
if self.storage.file_exists(asset_path):
return self.storage.generate_signed_url(asset_path, expiration_minutes=60)
return None
# Singleton instance with thread-safe initialization
_tenant_service: Optional[TenantService] = None
_tenant_service_lock = threading.Lock()
def get_tenant_service() -> TenantService:
"""Get or create the singleton TenantService instance (thread-safe)."""
global _tenant_service
if _tenant_service is None:
with _tenant_service_lock:
# Double-check after acquiring lock
if _tenant_service is None:
_tenant_service = TenantService()
return _tenant_service
+37 -7
View File
@@ -73,16 +73,26 @@ class UserService:
logger.exception(f"Error getting user {email}")
return None
def get_or_create_user(self, email: str) -> User:
def get_or_create_user(self, email: str, tenant_id: Optional[str] = None) -> User:
"""
Get existing user or create a new one.
New users receive a welcome credit to try the service.
Args:
email: User's email address
tenant_id: Tenant ID for white-label portals (None = default Nomad Karaoke)
Note: If user exists but has a different tenant_id, the existing user is returned.
Users are uniquely identified by email, not email+tenant.
"""
email = email.lower()
user = self.get_user(email)
if user:
# If user exists but tenant_id differs, we still return the user
# This allows users to access multiple tenants with one account
# (though features may be restricted per-tenant)
return user
# Create new user with welcome credit
@@ -97,9 +107,10 @@ class UserService:
email=email,
credits=welcome_credit,
credit_transactions=[welcome_transaction],
tenant_id=tenant_id, # Associate with tenant on creation
)
self._save_user(user)
logger.info(f"Created new user: {email} with {welcome_credit} welcome credit(s)")
logger.info(f"Created new user: {email} with {welcome_credit} welcome credit(s) (tenant: {tenant_id or 'default'})")
return user
def _save_user(self, user: User) -> None:
@@ -153,6 +164,7 @@ class UserService:
display_name=user.display_name,
total_jobs_created=user.total_jobs_created,
total_jobs_completed=user.total_jobs_completed,
tenant_id=user.tenant_id,
)
# =========================================================================
@@ -163,18 +175,25 @@ class UserService:
self,
email: str,
ip_address: Optional[str] = None,
user_agent: Optional[str] = None
user_agent: Optional[str] = None,
tenant_id: Optional[str] = None
) -> MagicLinkToken:
"""
Create a magic link token for email authentication.
Returns the token object. The actual sending of the email
should be handled by the caller (or an email service).
Args:
email: User's email address
ip_address: Client IP for auditing
user_agent: Client user agent for auditing
tenant_id: Tenant ID for white-label portals (None = default Nomad Karaoke)
"""
email = email.lower()
# Ensure user exists
self.get_or_create_user(email)
# Ensure user exists (with tenant association)
self.get_or_create_user(email, tenant_id=tenant_id)
# Generate secure token
token = secrets.token_urlsafe(32)
@@ -185,6 +204,7 @@ class UserService:
expires_at=datetime.utcnow() + timedelta(minutes=MAGIC_LINK_EXPIRY_MINUTES),
ip_address=ip_address,
user_agent=user_agent,
tenant_id=tenant_id,
)
# Save to Firestore
@@ -262,9 +282,18 @@ class UserService:
self,
user_email: str,
ip_address: Optional[str] = None,
user_agent: Optional[str] = None
user_agent: Optional[str] = None,
tenant_id: Optional[str] = None
) -> Session:
"""Create a new session for an authenticated user."""
"""
Create a new session for an authenticated user.
Args:
user_email: User's email address
ip_address: Client IP for auditing
user_agent: Client user agent for auditing
tenant_id: Tenant ID for white-label portals (None = default Nomad Karaoke)
"""
token = secrets.token_urlsafe(32)
token_prefix = token[:12]
@@ -274,6 +303,7 @@ class UserService:
expires_at=datetime.utcnow() + timedelta(days=SESSION_ABSOLUTE_EXPIRY_DAYS),
ip_address=ip_address,
user_agent=user_agent,
tenant_id=tenant_id,
)
# Serialize and write to Firestore
+98
View File
@@ -37,6 +37,7 @@ WORKER_QUEUES = {
"screens": "screens-worker-queue",
"render-video": "render-worker-queue",
"video": "video-worker-queue",
"idle-reminder": "idle-reminder-queue", # For delayed idle reminder checks
}
# Dispatch deadlines for each worker type (in seconds)
@@ -49,8 +50,12 @@ WORKER_DISPATCH_DEADLINES = {
"screens": 600, # 10 min - Screen generation is fast
"render-video": 1800, # 30 min - Video encoding can be slow
"video": 1800, # 30 min - Video encoding can be slow
"idle-reminder": 60, # 1 min - Quick check and potential email send
}
# Default delay for idle reminders (seconds)
IDLE_REMINDER_DELAY_SECONDS = 5 * 60 # 5 minutes
class WorkerService:
"""
@@ -430,6 +435,99 @@ class WorkerService:
"""Trigger render video worker (post-review)."""
return await self.trigger_worker("render-video", job_id)
async def schedule_idle_reminder(
self,
job_id: str,
delay_seconds: int = IDLE_REMINDER_DELAY_SECONDS
) -> bool:
"""
Schedule an idle reminder check for a job.
The reminder task will be delivered after the specified delay.
When the task executes, it checks if the job is still in a blocking
state and sends a reminder email if the user hasn't taken action.
Args:
job_id: Job ID to check
delay_seconds: Delay before executing the check (default: 5 minutes)
Returns:
True if task was scheduled successfully, False otherwise
"""
if not self._use_cloud_tasks:
# In development mode, log and skip (no delayed execution support)
logger.info(
f"[job:{job_id}] Idle reminder not scheduled (Cloud Tasks disabled). "
f"Would have fired in {delay_seconds}s."
)
return True
try:
from google.cloud import tasks_v2
from google.protobuf import timestamp_pb2
import time
queue_name = WORKER_QUEUES.get("idle-reminder")
if not queue_name:
logger.error("Idle reminder queue not configured")
return False
project = self.settings.google_cloud_project
if not project:
logger.error("GOOGLE_CLOUD_PROJECT not set")
return False
location = self.settings.gcp_region
# Build queue path
parent = self.tasks_client.queue_path(project, location, queue_name)
# Build headers
headers = {
"Content-Type": "application/json",
}
if self._admin_token:
headers["X-Admin-Token"] = self._admin_token
# Inject trace context
headers = inject_trace_context(headers)
# Calculate schedule time
schedule_time = timestamp_pb2.Timestamp()
schedule_time.FromSeconds(int(time.time()) + delay_seconds)
dispatch_deadline_seconds = WORKER_DISPATCH_DEADLINES.get("idle-reminder", 60)
# Build task payload
task = {
"http_request": {
"http_method": tasks_v2.HttpMethod.POST,
"url": f"{self._base_url}/api/internal/jobs/{job_id}/check-idle-reminder",
"headers": headers,
"body": json.dumps({"job_id": job_id}).encode(),
"oidc_token": {
"service_account_email": f"karaoke-backend@{project}.iam.gserviceaccount.com",
},
},
"schedule_time": schedule_time,
"dispatch_deadline": duration_pb2.Duration(seconds=dispatch_deadline_seconds),
}
# Create task
response = self.tasks_client.create_task(parent=parent, task=task)
logger.info(
f"[job:{job_id}] Scheduled idle reminder check in {delay_seconds}s: {response.name}"
)
return True
except Exception as e:
logger.error(
f"[job:{job_id}] Failed to schedule idle reminder: {e}",
exc_info=True
)
return False
# Global worker service instance
_worker_service: Optional[WorkerService] = None
+22 -1
View File
@@ -39,6 +39,24 @@ os.environ["ADMIN_TOKENS"] = "test-admin-token"
# Only import app if emulators are running
if emulators_running():
from unittest.mock import Mock
# Mock theme service BEFORE importing app to ensure all jobs get theme_id="nomad"
_mock_theme_service = Mock()
_mock_theme_service.get_default_theme_id.return_value = "nomad"
_mock_theme_service.get_theme.return_value = None
# Apply patch at module load time (before app imports the service)
_theme_patches = [
patch("backend.services.theme_service.get_theme_service", return_value=_mock_theme_service),
patch("backend.api.routes.audio_search.get_theme_service", return_value=_mock_theme_service),
patch("backend.api.routes.file_upload.get_theme_service", return_value=_mock_theme_service),
patch("backend.api.routes.users.get_theme_service", return_value=_mock_theme_service),
patch("backend.api.routes.jobs.get_theme_service", return_value=_mock_theme_service),
]
for p in _theme_patches:
p.start()
from fastapi.testclient import TestClient
from backend.main import app
else:
@@ -76,7 +94,10 @@ def mock_worker_service():
@pytest.fixture(scope="session")
def client(mock_worker_service):
"""Create FastAPI test client with mocked workers."""
"""Create FastAPI test client with mocked workers.
Theme service is mocked at module load time (see above).
"""
with patch("backend.api.routes.file_upload.worker_service", mock_worker_service):
return TestClient(app)
+411
View File
@@ -0,0 +1,411 @@
"""
Unit tests for admin email endpoints.
Tests the completion message and send email API endpoints.
"""
import pytest
from unittest.mock import Mock, patch, MagicMock, AsyncMock
from fastapi.testclient import TestClient
from fastapi import FastAPI
from backend.api.routes.admin import router
from backend.api.dependencies import require_admin
from backend.models.job import Job, JobStatus
# Create a test app with the admin router
# The router already has prefix="/admin", so we add /api prefix
app = FastAPI()
app.include_router(router, prefix="/api")
def get_mock_admin():
"""Override for require_admin dependency."""
return ("admin@example.com", "admin", 1)
# Override the require_admin dependency
app.dependency_overrides[require_admin] = get_mock_admin
@pytest.fixture
def client():
"""Create a test client."""
return TestClient(app)
@pytest.fixture
def mock_job():
"""Create a mock job."""
job = Mock(spec=Job)
job.job_id = "test-job-123"
job.user_email = "user@example.com"
job.artist = "Test Artist"
job.title = "Test Song"
job.status = JobStatus.COMPLETE
job.audio_hash = "hash123"
job.review_token = "review123"
job.instrumental_token = "inst123"
job.state_data = {
"youtube_url": "https://youtube.com/watch?v=test123",
"dropbox_link": "https://dropbox.com/folder/test",
"brand_code": "NOMAD-1234",
}
return job
class TestGetCompletionMessage:
"""Tests for GET /api/admin/jobs/{job_id}/completion-message endpoint."""
def test_returns_completion_message(self, client, mock_job):
"""Test successful completion message retrieval."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
# Setup mocks
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Your video is ready!"
mock_get_ns.return_value = mock_ns
response = client.get(
"/api/admin/jobs/test-job-123/completion-message",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["job_id"] == "test-job-123"
assert data["message"] == "Your video is ready!"
# Subject format: "NOMAD-1234: Test Artist - Test Song (Your karaoke video is ready!)"
assert data["subject"] == "NOMAD-1234: Test Artist - Test Song (Your karaoke video is ready!)"
assert data["youtube_url"] == "https://youtube.com/watch?v=test123"
assert data["dropbox_url"] == "https://dropbox.com/folder/test"
def test_returns_404_when_job_not_found(self, client):
"""Test 404 when job doesn't exist."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = None
mock_jm_class.return_value = mock_jm
response = client.get(
"/api/admin/jobs/nonexistent-job/completion-message",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 404
def test_handles_none_state_data(self, client, mock_job):
"""Test handling of job with None state_data."""
mock_job.state_data = None
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Your video is ready!"
mock_get_ns.return_value = mock_ns
response = client.get(
"/api/admin/jobs/test-job-123/completion-message",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["youtube_url"] is None
assert data["dropbox_url"] is None
def test_default_subject_without_song_info(self, client, mock_job):
"""Test default subject when no artist/title."""
mock_job.artist = None
mock_job.title = None
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Message"
mock_get_ns.return_value = mock_ns
response = client.get(
"/api/admin/jobs/test-job-123/completion-message",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["subject"] == "Your karaoke video is ready!"
class TestSendCompletionEmail:
"""Tests for POST /api/admin/jobs/{job_id}/send-completion-email endpoint."""
def test_sends_email_successfully(self, client, mock_job):
"""Test successful email sending."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
patch('backend.services.email_service.get_email_service') as mock_get_es:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Your video is ready!"
mock_get_ns.return_value = mock_ns
mock_es = Mock()
mock_es.send_job_completion.return_value = True
mock_get_es.return_value = mock_es
response = client.post(
"/api/admin/jobs/test-job-123/send-completion-email",
json={"to_email": "customer@example.com", "cc_admin": True},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["success"] is True
assert data["job_id"] == "test-job-123"
assert data["to_email"] == "customer@example.com"
# Verify email was sent with correct params
mock_es.send_job_completion.assert_called_once_with(
to_email="customer@example.com",
message_content="Your video is ready!",
artist="Test Artist",
title="Test Song",
brand_code="NOMAD-1234",
cc_admin=True,
)
def test_sends_email_without_cc(self, client, mock_job):
"""Test email sending without CC."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
patch('backend.services.email_service.get_email_service') as mock_get_es:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Message"
mock_get_ns.return_value = mock_ns
mock_es = Mock()
mock_es.send_job_completion.return_value = True
mock_get_es.return_value = mock_es
response = client.post(
"/api/admin/jobs/test-job-123/send-completion-email",
json={"to_email": "customer@example.com", "cc_admin": False},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
# Verify CC was not included
call_kwargs = mock_es.send_job_completion.call_args.kwargs
assert call_kwargs["cc_admin"] is False
def test_returns_404_when_job_not_found(self, client):
"""Test 404 when job doesn't exist."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = None
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/nonexistent-job/send-completion-email",
json={"to_email": "customer@example.com"},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 404
def test_returns_500_when_email_fails(self, client, mock_job):
"""Test 500 when email sending fails."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
patch('backend.services.email_service.get_email_service') as mock_get_es:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Message"
mock_get_ns.return_value = mock_ns
mock_es = Mock()
mock_es.send_job_completion.return_value = False # Email failed
mock_get_es.return_value = mock_es
response = client.post(
"/api/admin/jobs/test-job-123/send-completion-email",
json={"to_email": "customer@example.com"},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 500
assert "Failed to send email" in response.json()["detail"]
def test_handles_none_state_data(self, client, mock_job):
"""Test handling of job with None state_data."""
mock_job.state_data = None
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
patch('backend.services.email_service.get_email_service') as mock_get_es:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Message"
mock_get_ns.return_value = mock_ns
mock_es = Mock()
mock_es.send_job_completion.return_value = True
mock_get_es.return_value = mock_es
response = client.post(
"/api/admin/jobs/test-job-123/send-completion-email",
json={"to_email": "customer@example.com"},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
def test_accepts_any_email_string(self, client, mock_job):
"""Test that any string is currently accepted as email (no validation).
Documents current behavior: FastAPI/Pydantic doesn't validate email
format by default. If validation is added later, this test should
be updated to expect 422 for invalid emails.
"""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
patch('backend.services.email_service.get_email_service') as mock_get_es:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Message"
mock_get_ns.return_value = mock_ns
mock_es = Mock()
mock_es.send_job_completion.return_value = True
mock_get_es.return_value = mock_es
response = client.post(
"/api/admin/jobs/test-job-123/send-completion-email",
json={"to_email": "invalid-email"}, # Not a valid email format
headers={"Authorization": "Bearer admin-token"}
)
# Currently accepts any string - endpoint succeeds
assert response.status_code == 200
# Verify the invalid email was passed through
mock_es.send_job_completion.assert_called_once()
call_kwargs = mock_es.send_job_completion.call_args.kwargs
assert call_kwargs["to_email"] == "invalid-email"
class TestIdleReminderEndpoint:
"""Tests for the internal idle reminder endpoint."""
def test_sends_reminder_when_still_idle(self, mock_job):
"""Test that reminder is sent when user is still idle."""
from backend.api.routes.internal import router as internal_router
from backend.api.dependencies import require_admin
# Create test app for internal router
# The router already has prefix="/internal", so we add /api prefix
test_app = FastAPI()
test_app.include_router(internal_router, prefix="/api")
test_app.dependency_overrides[require_admin] = get_mock_admin
mock_job.status = JobStatus.AWAITING_REVIEW.value # Use .value for string comparison
mock_job.state_data = {
"blocking_state_entered_at": "2024-01-01T00:00:00",
"blocking_action_type": "lyrics",
"reminder_sent": False,
}
with patch('backend.api.routes.internal.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.firestore = Mock() # Mock firestore for update_job call
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.send_action_reminder_email = AsyncMock(return_value=True)
mock_get_ns.return_value = mock_ns
test_client = TestClient(test_app)
response = test_client.post(
"/api/internal/jobs/test-job-123/check-idle-reminder",
headers={"Authorization": "Bearer admin-token"}
)
# Should succeed and return "sent" status
assert response.status_code == 200
data = response.json()
assert data["status"] == "sent"
assert data["job_id"] == "test-job-123"
def test_skips_reminder_when_already_sent(self, mock_job):
"""Test that reminder is skipped if already sent."""
from backend.api.routes.internal import router as internal_router
from backend.api.dependencies import require_admin
# Create test app for internal router
# The router already has prefix="/internal", so we add /api prefix
test_app = FastAPI()
test_app.include_router(internal_router, prefix="/api")
test_app.dependency_overrides[require_admin] = get_mock_admin
mock_job.status = JobStatus.AWAITING_REVIEW.value # Use .value for string comparison
mock_job.state_data = {
"blocking_state_entered_at": "2024-01-01T00:00:00",
"blocking_action_type": "lyrics",
"reminder_sent": True, # Already sent
}
with patch('backend.api.routes.internal.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
test_client = TestClient(test_app)
response = test_client.post(
"/api/internal/jobs/test-job-123/check-idle-reminder",
headers={"Authorization": "Bearer admin-token"}
)
# Should succeed with "already_sent" status
assert response.status_code == 200
data = response.json()
assert data["status"] == "already_sent"
assert data["job_id"] == "test-job-123"
+492
View File
@@ -0,0 +1,492 @@
"""
Unit tests for email service.
Tests the new job completion and action reminder email methods,
as well as CC support.
"""
import pytest
from unittest.mock import Mock, patch, MagicMock
from backend.services.email_service import (
EmailService,
EmailProvider,
ConsoleEmailProvider,
SendGridEmailProvider,
get_email_service,
)
class TestConsoleEmailProvider:
"""Tests for console email provider."""
def test_send_email_basic(self):
"""Test basic email logging."""
provider = ConsoleEmailProvider()
result = provider.send_email(
to_email="user@example.com",
subject="Test Subject",
html_content="<p>Test content</p>",
)
assert result is True
def test_send_email_with_text_content(self):
"""Test email with text content."""
provider = ConsoleEmailProvider()
result = provider.send_email(
to_email="user@example.com",
subject="Test Subject",
html_content="<p>Test</p>",
text_content="Test plain text",
)
assert result is True
def test_send_email_with_cc(self):
"""Test email with CC recipients."""
provider = ConsoleEmailProvider()
result = provider.send_email(
to_email="user@example.com",
subject="Test Subject",
html_content="<p>Test</p>",
cc_emails=["cc1@example.com", "cc2@example.com"],
)
assert result is True
class TestSendGridEmailProvider:
"""Tests for SendGrid email provider."""
def test_send_email_success(self):
"""Test successful email sending via SendGrid."""
provider = SendGridEmailProvider(
api_key="test-api-key",
from_email="from@example.com",
from_name="Test Sender",
)
mock_response = Mock()
mock_response.status_code = 202
# Patch at the sendgrid module level since it's imported inline
with patch('sendgrid.SendGridAPIClient') as mock_sg:
mock_client = Mock()
mock_client.send.return_value = mock_response
mock_sg.return_value = mock_client
result = provider.send_email(
to_email="user@example.com",
subject="Test Subject",
html_content="<p>Test</p>",
)
assert result is True
mock_client.send.assert_called_once()
def test_send_email_with_cc(self):
"""Test email sending with CC via SendGrid."""
provider = SendGridEmailProvider(
api_key="test-api-key",
from_email="from@example.com",
)
mock_response = Mock()
mock_response.status_code = 202
with patch('sendgrid.SendGridAPIClient') as mock_sg:
mock_client = Mock()
mock_client.send.return_value = mock_response
mock_sg.return_value = mock_client
result = provider.send_email(
to_email="user@example.com",
subject="Test Subject",
html_content="<p>Test</p>",
cc_emails=["cc@example.com"],
)
assert result is True
def test_send_email_failure_status(self):
"""Test email sending failure due to bad status."""
provider = SendGridEmailProvider(
api_key="test-api-key",
from_email="from@example.com",
)
mock_response = Mock()
mock_response.status_code = 400
with patch('sendgrid.SendGridAPIClient') as mock_sg:
mock_client = Mock()
mock_client.send.return_value = mock_response
mock_sg.return_value = mock_client
result = provider.send_email(
to_email="user@example.com",
subject="Test Subject",
html_content="<p>Test</p>",
)
assert result is False
def test_send_email_exception(self):
"""Test email sending exception handling."""
provider = SendGridEmailProvider(
api_key="test-api-key",
from_email="from@example.com",
)
with patch('sendgrid.SendGridAPIClient') as mock_sg:
mock_sg.return_value.send.side_effect = Exception("API error")
result = provider.send_email(
to_email="user@example.com",
subject="Test Subject",
html_content="<p>Test</p>",
)
assert result is False
class TestEmailServiceJobCompletion:
"""Tests for job completion email method."""
def test_send_job_completion_basic(self):
"""Test basic job completion email."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
result = service.send_job_completion(
to_email="user@example.com",
message_content="Your video is ready!",
)
assert result is True
service.provider.send_email.assert_called_once()
def test_send_job_completion_with_song_info(self):
"""Test job completion email includes song in subject."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
service.send_job_completion(
to_email="user@example.com",
message_content="Your video is ready!",
artist="Test Artist",
title="Test Song",
)
call_args = service.provider.send_email.call_args
subject = call_args.kwargs.get('subject') or call_args[0][1]
assert "Test Artist" in subject
assert "Test Song" in subject
def test_send_job_completion_with_brand_code(self):
"""Test job completion email includes brand code in subject."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
service.send_job_completion(
to_email="user@example.com",
message_content="Your video is ready!",
artist="Seether",
title="Tonight",
brand_code="NOMAD-1178",
)
call_args = service.provider.send_email.call_args
subject = call_args.kwargs.get('subject') or call_args[0][1]
# Subject format: "NOMAD-1178: Seether - Tonight (Your karaoke video is ready!)"
assert subject == "NOMAD-1178: Seether - Tonight (Your karaoke video is ready!)"
def test_send_job_completion_default_subject(self):
"""Test job completion email default subject without song info."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
service.send_job_completion(
to_email="user@example.com",
message_content="Your video is ready!",
)
call_args = service.provider.send_email.call_args
subject = call_args.kwargs.get('subject') or call_args[0][1]
assert "karaoke video is ready" in subject.lower()
def test_send_job_completion_with_cc(self):
"""Test job completion email with CC to admin."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
service.send_job_completion(
to_email="user@example.com",
message_content="Your video is ready!",
cc_admin=True,
)
call_kwargs = service.provider.send_email.call_args.kwargs
assert "gen@nomadkaraoke.com" in call_kwargs.get('cc_emails', [])
def test_send_job_completion_without_cc(self):
"""Test job completion email without CC."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
service.send_job_completion(
to_email="user@example.com",
message_content="Your video is ready!",
cc_admin=False,
)
call_kwargs = service.provider.send_email.call_args.kwargs
assert call_kwargs.get('cc_emails') is None
def test_send_job_completion_escapes_html(self):
"""Test that message content is HTML-escaped."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
service.send_job_completion(
to_email="user@example.com",
message_content="Test <script>alert('xss')</script>",
)
call_kwargs = service.provider.send_email.call_args.kwargs
html_content = call_kwargs.get('html_content')
assert "<script>" not in html_content
assert "&lt;script&gt;" in html_content
def test_send_job_completion_includes_plain_text(self):
"""Test that plain text content is included."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
message = "Your video is ready!"
service.send_job_completion(
to_email="user@example.com",
message_content=message,
)
call_kwargs = service.provider.send_email.call_args.kwargs
assert call_kwargs.get('text_content') == message
class TestEmailServiceActionReminder:
"""Tests for action reminder email method."""
def test_send_action_reminder_lyrics(self):
"""Test lyrics action reminder email."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
result = service.send_action_reminder(
to_email="user@example.com",
message_content="Please review your lyrics",
action_type="lyrics",
)
assert result is True
call_kwargs = service.provider.send_email.call_args.kwargs
subject = call_kwargs.get('subject')
assert "lyrics" in subject.lower()
def test_send_action_reminder_instrumental(self):
"""Test instrumental action reminder email."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
result = service.send_action_reminder(
to_email="user@example.com",
message_content="Please select instrumental",
action_type="instrumental",
)
assert result is True
call_kwargs = service.provider.send_email.call_args.kwargs
subject = call_kwargs.get('subject')
assert "instrumental" in subject.lower()
def test_send_action_reminder_with_song_info(self):
"""Test action reminder includes song in subject."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
service.send_action_reminder(
to_email="user@example.com",
message_content="Please review",
action_type="lyrics",
artist="Test Artist",
title="Test Song",
)
call_kwargs = service.provider.send_email.call_args.kwargs
subject = call_kwargs.get('subject')
assert "Test Artist" in subject
assert "Test Song" in subject
def test_send_action_reminder_unknown_type(self):
"""Test action reminder with unknown type still sends."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
result = service.send_action_reminder(
to_email="user@example.com",
message_content="Please take action",
action_type="unknown",
)
assert result is True
call_kwargs = service.provider.send_email.call_args.kwargs
subject = call_kwargs.get('subject')
assert "Action needed" in subject
def test_send_action_reminder_escapes_html(self):
"""Test that message content is HTML-escaped."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
service.send_action_reminder(
to_email="user@example.com",
message_content="Test <script>alert('xss')</script>",
action_type="lyrics",
)
call_kwargs = service.provider.send_email.call_args.kwargs
html_content = call_kwargs.get('html_content')
assert "<script>" not in html_content
assert "&lt;script&gt;" in html_content
def test_send_action_reminder_no_cc(self):
"""Test that action reminders don't have CC."""
service = EmailService()
service.provider = Mock()
service.provider.send_email.return_value = True
service.send_action_reminder(
to_email="user@example.com",
message_content="Please review",
action_type="lyrics",
)
call_kwargs = service.provider.send_email.call_args.kwargs
# Action reminders should not have CC
assert call_kwargs.get('cc_emails') is None
class TestEmailServiceConfiguration:
"""Tests for email service configuration."""
def test_uses_sendgrid_when_configured(self):
"""Test that SendGrid is used when API key is set."""
with patch.dict('os.environ', {'SENDGRID_API_KEY': 'test-key'}):
service = EmailService()
assert isinstance(service.provider, SendGridEmailProvider)
def test_uses_console_when_not_configured(self):
"""Test that console is used when no API key."""
with patch.dict('os.environ', {}, clear=True):
# Remove SENDGRID_API_KEY if it exists
import os
original = os.environ.pop('SENDGRID_API_KEY', None)
try:
service = EmailService()
assert isinstance(service.provider, ConsoleEmailProvider)
finally:
if original:
os.environ['SENDGRID_API_KEY'] = original
def test_is_configured_true_for_sendgrid(self):
"""Test is_configured returns True for SendGrid."""
service = EmailService()
service.provider = SendGridEmailProvider("key", "from@example.com")
assert service.is_configured() is True
def test_is_configured_false_for_console(self):
"""Test is_configured returns False for console."""
service = EmailService()
service.provider = ConsoleEmailProvider()
assert service.is_configured() is False
class TestGlobalInstance:
"""Tests for global instance management."""
def test_get_email_service_returns_same_instance(self):
"""Test that get_email_service returns singleton."""
# Reset global
import backend.services.email_service as es
es._email_service = None
service1 = get_email_service()
service2 = get_email_service()
assert service1 is service2
class TestCCFunctionality:
"""Integration tests for CC functionality across providers."""
def test_console_provider_logs_cc(self, caplog):
"""Test that console provider logs CC recipients."""
import logging
caplog.set_level(logging.INFO)
provider = ConsoleEmailProvider()
provider.send_email(
to_email="user@example.com",
subject="Test",
html_content="<p>Test</p>",
cc_emails=["cc1@example.com", "cc2@example.com"],
)
# Check that CC was logged
assert "cc1@example.com" in caplog.text or "CC:" in caplog.text
def test_sendgrid_provider_adds_cc_recipients(self):
"""Test that SendGrid provider adds CC recipients to message."""
provider = SendGridEmailProvider(
api_key="test-api-key",
from_email="from@example.com",
)
mock_response = Mock()
mock_response.status_code = 202
# Patch at sendgrid module level since imports are inline
with patch('sendgrid.SendGridAPIClient') as mock_sg:
mock_client = Mock()
mock_client.send.return_value = mock_response
mock_sg.return_value = mock_client
# We need to capture the Mail object
with patch('sendgrid.helpers.mail.Mail') as mock_mail_class:
mock_mail = MagicMock()
mock_mail_class.return_value = mock_mail
provider.send_email(
to_email="user@example.com",
subject="Test",
html_content="<p>Test</p>",
cc_emails=["cc@example.com"],
)
# Verify add_cc was called
mock_mail.add_cc.assert_called()
+2 -1
View File
@@ -1128,7 +1128,8 @@ class TestFinaliseOnlyModels:
assert request.enable_txt is None
assert request.brand_prefix is None
assert request.keep_brand_code is None
assert request.enable_youtube_upload is False
# YouTube upload default is None (server applies default_enable_youtube_upload)
assert request.enable_youtube_upload is None
assert request.dropbox_path is None
assert request.gdrive_folder_id is None
+223
View File
@@ -0,0 +1,223 @@
"""
Unit tests for user impersonation endpoint.
Tests the admin impersonation API that allows admins to view the app as other users.
"""
import pytest
from unittest.mock import Mock, patch
from fastapi.testclient import TestClient
from fastapi import FastAPI
from backend.api.routes.admin import router
from backend.api.dependencies import require_admin
from backend.services.user_service import get_user_service
from backend.models.user import User, Session
# Create a test app with the admin router
app = FastAPI()
app.include_router(router, prefix="/api")
def get_mock_admin():
"""Override for require_admin dependency - returns admin user."""
return ("admin@nomadkaraoke.com", "admin", 1)
def get_mock_regular_user():
"""Override for require_admin dependency - returns regular user (should fail)."""
# This simulates what happens when a non-admin tries to access
# In reality, require_admin raises 403, but we test the logic
return ("user@example.com", "user", 0)
@pytest.fixture
def client():
"""Create a test client with admin access."""
app.dependency_overrides[require_admin] = get_mock_admin
yield TestClient(app)
app.dependency_overrides.clear()
@pytest.fixture
def mock_user_service():
"""Create a mock user service."""
service = Mock()
return service
@pytest.fixture
def mock_target_user():
"""Create a mock target user to impersonate."""
user = Mock(spec=User)
user.email = "target@example.com"
user.credits = 5
user.role = "user"
user.is_active = True
return user
@pytest.fixture
def mock_session():
"""Create a mock session."""
session = Mock(spec=Session)
session.token = "test-session-token-12345678901234567890"
session.user_email = "target@example.com"
return session
class TestImpersonateUser:
"""Tests for POST /api/admin/users/{email}/impersonate endpoint."""
def test_admin_can_impersonate_user(self, client, mock_target_user, mock_session):
"""Admin gets session token for target user."""
with patch('backend.api.routes.admin.get_user_service') as mock_get_service:
mock_service = Mock()
mock_service.get_user.return_value = mock_target_user
mock_service.create_session.return_value = mock_session
mock_get_service.return_value = mock_service
# Override the dependency
app.dependency_overrides[get_user_service] = lambda: mock_service
response = client.post(
"/api/admin/users/target@example.com/impersonate",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["session_token"] == mock_session.token
assert data["user_email"] == "target@example.com"
assert "impersonating" in data["message"].lower()
# Verify user lookup was called
mock_service.get_user.assert_called_once_with("target@example.com")
# Verify session was created for target user
mock_service.create_session.assert_called_once()
call_args = mock_service.create_session.call_args
assert call_args.kwargs["user_email"] == "target@example.com"
assert "Impersonation by admin@nomadkaraoke.com" in call_args.kwargs["user_agent"]
def test_impersonate_nonexistent_user_returns_404(self, client):
"""Target user must exist."""
with patch('backend.api.routes.admin.get_user_service') as mock_get_service:
mock_service = Mock()
mock_service.get_user.return_value = None # User not found
mock_get_service.return_value = mock_service
app.dependency_overrides[get_user_service] = lambda: mock_service
response = client.post(
"/api/admin/users/nonexistent@example.com/impersonate",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 404
assert "not found" in response.json()["detail"].lower()
def test_cannot_impersonate_self(self, client, mock_target_user):
"""Admin cannot impersonate themselves."""
with patch('backend.api.routes.admin.get_user_service') as mock_get_service:
mock_service = Mock()
# Set up user to be found (same as admin)
mock_target_user.email = "admin@nomadkaraoke.com"
mock_service.get_user.return_value = mock_target_user
mock_get_service.return_value = mock_service
app.dependency_overrides[get_user_service] = lambda: mock_service
response = client.post(
"/api/admin/users/admin@nomadkaraoke.com/impersonate",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 400
assert "yourself" in response.json()["detail"].lower()
def test_impersonation_creates_valid_session(self, client, mock_target_user, mock_session):
"""Returned token is a valid session for target user."""
with patch('backend.api.routes.admin.get_user_service') as mock_get_service:
mock_service = Mock()
mock_service.get_user.return_value = mock_target_user
mock_service.create_session.return_value = mock_session
mock_get_service.return_value = mock_service
app.dependency_overrides[get_user_service] = lambda: mock_service
response = client.post(
"/api/admin/users/target@example.com/impersonate",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
# Verify create_session was called with correct user
mock_service.create_session.assert_called_once()
call_kwargs = mock_service.create_session.call_args.kwargs
assert call_kwargs["user_email"] == "target@example.com"
def test_impersonation_logs_audit_trail(self, client, mock_target_user, mock_session):
"""Impersonation is logged for security audit."""
with patch('backend.api.routes.admin.get_user_service') as mock_get_service, \
patch('backend.api.routes.admin.logger') as mock_logger:
mock_service = Mock()
mock_service.get_user.return_value = mock_target_user
mock_service.create_session.return_value = mock_session
mock_get_service.return_value = mock_service
app.dependency_overrides[get_user_service] = lambda: mock_service
response = client.post(
"/api/admin/users/target@example.com/impersonate",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
# Verify logging was called with impersonation info
mock_logger.info.assert_called()
log_message = mock_logger.info.call_args[0][0]
assert "IMPERSONATION" in log_message
assert "admin@nomadkaraoke.com" in log_message
assert "target@example.com" in log_message
def test_email_is_normalized_to_lowercase(self, client, mock_target_user, mock_session):
"""Email addresses are normalized to lowercase."""
with patch('backend.api.routes.admin.get_user_service') as mock_get_service:
mock_service = Mock()
mock_service.get_user.return_value = mock_target_user
mock_service.create_session.return_value = mock_session
mock_get_service.return_value = mock_service
app.dependency_overrides[get_user_service] = lambda: mock_service
response = client.post(
"/api/admin/users/TARGET@EXAMPLE.COM/impersonate",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
# Verify user lookup was with lowercase
mock_service.get_user.assert_called_once_with("target@example.com")
class TestImpersonateUserNonAdmin:
"""Tests to verify non-admins cannot impersonate."""
def test_non_admin_cannot_impersonate(self):
"""
Regular user gets 403 Forbidden.
Note: This is enforced by the require_admin dependency at the route level.
We test that the dependency is correctly applied.
"""
# Create a fresh client without admin override
test_app = FastAPI()
test_app.include_router(router, prefix="/api")
# Don't override require_admin - it should reject non-admin requests
# In a real test, we'd mock the auth service to return a non-admin user
# For now, we just verify the endpoint requires admin via dependency
# The require_admin dependency will reject requests without proper admin auth
# This is tested implicitly by the fact that all other tests use admin override
pass # Actual enforcement tested via integration tests
@@ -128,6 +128,8 @@ class TestUserEmailExtraction:
mock_request.headers = {}
mock_request.client = Mock(host="127.0.0.1")
mock_request.url = Mock(path="/api/jobs/create-from-url")
# Mock tenant state (no tenant = default Nomad Karaoke)
mock_request.state.tenant_config = None
mock_background_tasks = Mock()
body = CreateJobFromUrlRequest(
@@ -168,6 +170,8 @@ class TestUserEmailExtraction:
mock_request.headers = {}
mock_request.client = Mock(host="127.0.0.1")
mock_request.url = Mock(path="/api/audio-search/search")
# Mock tenant state (no tenant = default Nomad Karaoke)
mock_request.state.tenant_config = None
mock_background_tasks = Mock()
body = AudioSearchRequest(
+25 -8
View File
@@ -33,13 +33,28 @@ def job_manager(mock_firestore_service):
class TestJobCreation:
"""Test job creation logic."""
def test_create_job_requires_theme_id(self, job_manager, mock_firestore_service):
"""Test that jobs without theme_id are rejected."""
job_create = JobCreate(
artist="Test Artist",
title="Test Song"
# No theme_id - should fail
)
with pytest.raises(ValueError, match="theme_id is required"):
job_manager.create_job(job_create)
# Verify Firestore was NOT called
mock_firestore_service.create_job.assert_not_called()
def test_create_job_with_url(self, job_manager, mock_firestore_service):
"""Test creating a job with YouTube URL."""
job_create = JobCreate(
url="https://youtube.com/watch?v=test",
artist="Test Artist",
title="Test Song"
title="Test Song",
theme_id="nomad" # Required for all jobs
)
# The actual create_job method creates the job and returns it
@@ -59,7 +74,8 @@ class TestJobCreation:
"""Test creating a job without URL (for file upload)."""
job_create = JobCreate(
artist="Test Artist",
title="Test Song"
title="Test Song",
theme_id="nomad" # Required for all jobs
)
job = job_manager.create_job(job_create)
@@ -70,8 +86,8 @@ class TestJobCreation:
def test_create_job_generates_unique_id(self, job_manager, mock_firestore_service):
"""Test that each job gets a unique ID."""
job_create = JobCreate()
job_create = JobCreate(theme_id="nomad") # Required for all jobs
# Create multiple jobs
ids = []
for i in range(5):
@@ -89,8 +105,8 @@ class TestJobCreation:
def test_create_job_sets_initial_status(self, job_manager, mock_firestore_service):
"""Test that new jobs start with PENDING status."""
job_create = JobCreate()
job_create = JobCreate(theme_id="nomad") # Required for all jobs
mock_firestore_service.create_job.return_value = Job(
job_id="test123",
status=JobStatus.PENDING,
@@ -105,13 +121,14 @@ class TestJobCreation:
def test_create_job_with_distribution_settings(self, job_manager, mock_firestore_service):
"""Test that distribution settings are passed from JobCreate to Job.
This was a bug where brand_prefix, dropbox_path, gdrive_folder_id, and
discord_webhook_url were NOT being passed to the Job constructor.
"""
job_create = JobCreate(
artist="Test Artist",
title="Test Song",
theme_id="nomad", # Required for all jobs
brand_prefix="NOMAD",
discord_webhook_url="https://discord.com/webhook/test",
dropbox_path="/Karaoke/Tracks-Organized",
@@ -0,0 +1,329 @@
"""
Unit tests for job manager notification methods.
Tests the state transition triggers for email notifications.
"""
import pytest
from unittest.mock import Mock, patch, MagicMock, AsyncMock
from datetime import datetime
from backend.services.job_manager import JobManager
from backend.models.job import Job, JobStatus
class TestTriggerStateNotifications:
"""Tests for _trigger_state_notifications method."""
def test_triggers_completion_email_on_complete(self):
"""Test that completion email is triggered when job completes."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.state_data = {"youtube_url": "https://youtube.com/123"}
manager.get_job = Mock(return_value=mock_job)
with patch.object(manager, '_schedule_completion_email') as mock_schedule:
manager._trigger_state_notifications("job-123", JobStatus.COMPLETE)
mock_schedule.assert_called_once_with(mock_job)
def test_triggers_idle_reminder_on_awaiting_review(self):
"""Test that idle reminder is triggered when job enters awaiting review."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.state_data = {}
manager.get_job = Mock(return_value=mock_job)
with patch.object(manager, '_schedule_idle_reminder') as mock_schedule:
manager._trigger_state_notifications("job-123", JobStatus.AWAITING_REVIEW)
mock_schedule.assert_called_once_with(mock_job, JobStatus.AWAITING_REVIEW)
def test_triggers_idle_reminder_on_awaiting_instrumental(self):
"""Test that idle reminder is triggered when job enters awaiting instrumental."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.state_data = {}
manager.get_job = Mock(return_value=mock_job)
with patch.object(manager, '_schedule_idle_reminder') as mock_schedule:
manager._trigger_state_notifications(
"job-123",
JobStatus.AWAITING_INSTRUMENTAL_SELECTION
)
mock_schedule.assert_called_once_with(
mock_job,
JobStatus.AWAITING_INSTRUMENTAL_SELECTION
)
def test_skips_notification_when_no_job(self):
"""Test that notification is skipped when job not found."""
manager = JobManager()
manager.firestore = Mock()
manager.get_job = Mock(return_value=None)
with patch.object(manager, '_schedule_completion_email') as mock_schedule:
manager._trigger_state_notifications("job-123", JobStatus.COMPLETE)
mock_schedule.assert_not_called()
def test_skips_notification_when_no_user_email(self):
"""Test that notification is skipped when job has no user email."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = None
manager.get_job = Mock(return_value=mock_job)
with patch.object(manager, '_schedule_completion_email') as mock_schedule:
manager._trigger_state_notifications("job-123", JobStatus.COMPLETE)
mock_schedule.assert_not_called()
def test_does_not_trigger_on_other_states(self):
"""Test that no notification is triggered for non-notification states."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.state_data = {}
manager.get_job = Mock(return_value=mock_job)
with patch.object(manager, '_schedule_completion_email') as mock_complete, \
patch.object(manager, '_schedule_idle_reminder') as mock_idle:
# Test various non-notification states
for status in [JobStatus.PENDING, JobStatus.DOWNLOADING, JobStatus.TRANSCRIBING,
JobStatus.RENDERING_VIDEO, JobStatus.FAILED]:
manager._trigger_state_notifications("job-123", status)
mock_complete.assert_not_called()
mock_idle.assert_not_called()
def test_handles_exception_gracefully(self):
"""Test that exceptions don't propagate from notifications."""
manager = JobManager()
manager.firestore = Mock()
manager.get_job = Mock(side_effect=Exception("Database error"))
# Should not raise
manager._trigger_state_notifications("job-123", JobStatus.COMPLETE)
class TestScheduleCompletionEmail:
"""Tests for _schedule_completion_email method."""
def test_extracts_urls_from_state_data(self):
"""Test that YouTube and Dropbox URLs are extracted from state_data."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.artist = "Test Artist"
mock_job.title = "Test Song"
mock_job.state_data = {
"youtube_url": "https://youtube.com/watch?v=123",
"dropbox_link": "https://dropbox.com/folder/abc",
}
with patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_service:
mock_service = Mock()
mock_service.send_job_completion_email = AsyncMock(return_value=True)
mock_get_service.return_value = mock_service
# Run in sync context - this will start a daemon thread
manager._schedule_completion_email(mock_job)
# Give the thread a moment to execute
import time
time.sleep(0.1)
def test_handles_none_state_data(self):
"""Test that None state_data is handled gracefully."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.artist = None
mock_job.title = None
mock_job.state_data = None # None state_data
with patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_service:
mock_service = Mock()
mock_service.send_job_completion_email = AsyncMock(return_value=True)
mock_get_service.return_value = mock_service
# Should not raise
manager._schedule_completion_email(mock_job)
def test_handles_exception_gracefully(self):
"""Test that exceptions don't propagate."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.state_data = {}
with patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_service:
mock_get_service.side_effect = Exception("Service error")
# Should not raise
manager._schedule_completion_email(mock_job)
class TestScheduleIdleReminder:
"""Tests for _schedule_idle_reminder method."""
def test_updates_state_data_for_lyrics_review(self):
"""Test that state_data is updated with blocking state info for lyrics."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.state_data = {"existing_key": "value"}
with patch('backend.services.worker_service.get_worker_service') as mock_get_service:
mock_service = Mock()
mock_service.schedule_idle_reminder = AsyncMock(return_value=True)
mock_get_service.return_value = mock_service
manager._schedule_idle_reminder(mock_job, JobStatus.AWAITING_REVIEW)
# Verify state_data was updated
update_call = manager.firestore.update_job.call_args
assert update_call is not None
updated_data = update_call[0][1] # Second positional arg
state_data = updated_data.get('state_data', {})
assert state_data.get('blocking_action_type') == 'lyrics'
assert state_data.get('reminder_sent') is False
assert 'blocking_state_entered_at' in state_data
assert state_data.get('existing_key') == 'value' # Preserved
def test_updates_state_data_for_instrumental_selection(self):
"""Test that state_data is updated with blocking state info for instrumental."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.state_data = {}
with patch('backend.services.worker_service.get_worker_service') as mock_get_service:
mock_service = Mock()
mock_service.schedule_idle_reminder = AsyncMock(return_value=True)
mock_get_service.return_value = mock_service
manager._schedule_idle_reminder(
mock_job,
JobStatus.AWAITING_INSTRUMENTAL_SELECTION
)
# Verify state_data was updated
update_call = manager.firestore.update_job.call_args
updated_data = update_call[0][1]
state_data = updated_data.get('state_data', {})
assert state_data.get('blocking_action_type') == 'instrumental'
def test_handles_none_state_data(self):
"""Test that None state_data is handled gracefully."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.state_data = None
with patch('backend.services.worker_service.get_worker_service') as mock_get_service:
mock_service = Mock()
mock_service.schedule_idle_reminder = AsyncMock(return_value=True)
mock_get_service.return_value = mock_service
# Should not raise
manager._schedule_idle_reminder(mock_job, JobStatus.AWAITING_REVIEW)
# Verify update was still called
manager.firestore.update_job.assert_called_once()
def test_handles_exception_gracefully(self):
"""Test that exceptions don't propagate."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
mock_job.user_email = "user@example.com"
mock_job.state_data = {}
# Make firestore raise an exception
manager.firestore.update_job.side_effect = Exception("Database error")
# Should not raise
manager._schedule_idle_reminder(mock_job, JobStatus.AWAITING_REVIEW)
class TestTransitionTriggersNotifications:
"""Integration tests for transition_to_state triggering notifications."""
def test_transition_to_complete_triggers_notification(self):
"""Test that transitioning to COMPLETE triggers notification."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
# Use ENCODING as the starting state since ENCODING -> COMPLETE is valid
mock_job.status = JobStatus.ENCODING
mock_job.user_email = "user@example.com"
mock_job.state_data = {}
manager.get_job = Mock(return_value=mock_job)
with patch.object(manager, '_trigger_state_notifications') as mock_trigger:
manager.transition_to_state("job-123", JobStatus.COMPLETE)
mock_trigger.assert_called_once_with("job-123", JobStatus.COMPLETE)
def test_transition_to_awaiting_review_triggers_notification(self):
"""Test that transitioning to AWAITING_REVIEW triggers notification."""
manager = JobManager()
manager.firestore = Mock()
mock_job = Mock(spec=Job)
mock_job.job_id = "job-123"
# Use APPLYING_PADDING as the starting state since APPLYING_PADDING -> AWAITING_REVIEW is valid
mock_job.status = JobStatus.APPLYING_PADDING
mock_job.user_email = "user@example.com"
mock_job.state_data = {}
manager.get_job = Mock(return_value=mock_job)
with patch.object(manager, '_trigger_state_notifications') as mock_trigger:
manager.transition_to_state("job-123", JobStatus.AWAITING_REVIEW)
mock_trigger.assert_called_once_with("job-123", JobStatus.AWAITING_REVIEW)
@@ -0,0 +1,443 @@
"""
Unit tests for job notification service.
"""
import pytest
from unittest.mock import Mock, patch, AsyncMock
import urllib.parse
from backend.services.job_notification_service import (
JobNotificationService,
get_job_notification_service,
ENABLE_AUTO_EMAILS,
FEEDBACK_FORM_URL,
)
class TestURLBuilding:
"""Tests for URL building methods."""
def test_build_review_url_basic(self):
"""Test basic review URL building."""
service = JobNotificationService()
service.frontend_url = "https://gen.nomadkaraoke.com"
service.backend_url = "https://api.nomadkaraoke.com"
url = service._build_review_url("job-123")
assert "gen.nomadkaraoke.com/lyrics/" in url
assert "baseApiUrl=" in url
# The API URL should be URL-encoded
assert urllib.parse.quote("https://api.nomadkaraoke.com/api/review/job-123", safe='') in url
def test_build_review_url_with_audio_hash(self):
"""Test review URL with audio hash parameter."""
service = JobNotificationService()
service.frontend_url = "https://gen.nomadkaraoke.com"
service.backend_url = "https://api.nomadkaraoke.com"
url = service._build_review_url("job-123", audio_hash="abc123")
assert "audioHash=abc123" in url
def test_build_review_url_with_review_token(self):
"""Test review URL with review token parameter."""
service = JobNotificationService()
service.frontend_url = "https://gen.nomadkaraoke.com"
service.backend_url = "https://api.nomadkaraoke.com"
url = service._build_review_url("job-123", review_token="token456")
assert "reviewToken=token456" in url
def test_build_review_url_with_all_params(self):
"""Test review URL with all parameters."""
service = JobNotificationService()
service.frontend_url = "https://gen.nomadkaraoke.com"
service.backend_url = "https://api.nomadkaraoke.com"
url = service._build_review_url(
"job-123",
audio_hash="hash789",
review_token="token456"
)
assert "baseApiUrl=" in url
assert "audioHash=hash789" in url
assert "reviewToken=token456" in url
def test_build_review_url_encodes_special_chars(self):
"""Test that special characters in job ID are encoded."""
service = JobNotificationService()
service.frontend_url = "https://gen.nomadkaraoke.com"
service.backend_url = "https://api.nomadkaraoke.com"
url = service._build_review_url("job/with/slashes")
# The baseApiUrl parameter should have encoded slashes
assert "%2F" in url
def test_build_instrumental_url_basic(self):
"""Test basic instrumental URL building."""
service = JobNotificationService()
service.frontend_url = "https://gen.nomadkaraoke.com"
service.backend_url = "https://api.nomadkaraoke.com"
url = service._build_instrumental_url("job-123")
assert "gen.nomadkaraoke.com/instrumental/" in url
assert "baseApiUrl=" in url
def test_build_instrumental_url_with_token(self):
"""Test instrumental URL with token parameter."""
service = JobNotificationService()
service.frontend_url = "https://gen.nomadkaraoke.com"
service.backend_url = "https://api.nomadkaraoke.com"
url = service._build_instrumental_url("job-123", instrumental_token="inst-token")
assert "instrumentalToken=inst-token" in url
class TestCompletionEmail:
"""Tests for job completion email sending."""
@pytest.mark.asyncio
async def test_send_completion_email_success(self):
"""Test successful completion email sending."""
service = JobNotificationService()
service.email_service = Mock()
service.email_service.send_job_completion.return_value = True
service.template_service = Mock()
service.template_service.render_job_completion.return_value = "Test message"
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
result = await service.send_job_completion_email(
job_id="job-123",
user_email="user@example.com",
user_name="Test User",
artist="Test Artist",
title="Test Song",
youtube_url="https://youtube.com/watch?v=123",
dropbox_url="https://dropbox.com/folder/abc",
)
assert result is True
service.template_service.render_job_completion.assert_called_once()
service.email_service.send_job_completion.assert_called_once()
@pytest.mark.asyncio
async def test_send_completion_email_disabled(self):
"""Test that completion email is skipped when auto emails are disabled."""
service = JobNotificationService()
service.email_service = Mock()
service.template_service = Mock()
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', False):
result = await service.send_job_completion_email(
job_id="job-123",
user_email="user@example.com",
)
assert result is False
service.email_service.send_job_completion.assert_not_called()
@pytest.mark.asyncio
async def test_send_completion_email_no_email(self):
"""Test that completion email is skipped when no user email."""
service = JobNotificationService()
service.email_service = Mock()
service.template_service = Mock()
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
result = await service.send_job_completion_email(
job_id="job-123",
user_email=None,
)
assert result is False
service.email_service.send_job_completion.assert_not_called()
@pytest.mark.asyncio
async def test_send_completion_email_empty_email(self):
"""Test that completion email is skipped when user email is empty."""
service = JobNotificationService()
service.email_service = Mock()
service.template_service = Mock()
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
result = await service.send_job_completion_email(
job_id="job-123",
user_email="",
)
assert result is False
service.email_service.send_job_completion.assert_not_called()
@pytest.mark.asyncio
async def test_send_completion_email_send_failure(self):
"""Test handling of email send failure."""
service = JobNotificationService()
service.email_service = Mock()
service.email_service.send_job_completion.return_value = False
service.template_service = Mock()
service.template_service.render_job_completion.return_value = "Test message"
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
result = await service.send_job_completion_email(
job_id="job-123",
user_email="user@example.com",
)
assert result is False
@pytest.mark.asyncio
async def test_send_completion_email_exception(self):
"""Test handling of exceptions during email sending."""
service = JobNotificationService()
service.email_service = Mock()
service.email_service.send_job_completion.side_effect = Exception("Send error")
service.template_service = Mock()
service.template_service.render_job_completion.return_value = "Test message"
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
result = await service.send_job_completion_email(
job_id="job-123",
user_email="user@example.com",
)
assert result is False
@pytest.mark.asyncio
async def test_send_completion_email_passes_cc_admin(self):
"""Test that completion email is sent with CC to admin."""
service = JobNotificationService()
service.email_service = Mock()
service.email_service.send_job_completion.return_value = True
service.template_service = Mock()
service.template_service.render_job_completion.return_value = "Test message"
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
await service.send_job_completion_email(
job_id="job-123",
user_email="user@example.com",
)
# Verify cc_admin=True was passed
call_kwargs = service.email_service.send_job_completion.call_args.kwargs
assert call_kwargs.get('cc_admin') is True
class TestActionReminderEmail:
"""Tests for action reminder email sending."""
@pytest.mark.asyncio
async def test_send_lyrics_reminder_success(self):
"""Test successful lyrics reminder email sending."""
service = JobNotificationService()
service.email_service = Mock()
service.email_service.send_action_reminder.return_value = True
service.template_service = Mock()
service.template_service.render_action_needed_lyrics.return_value = "Review your lyrics"
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
result = await service.send_action_reminder_email(
job_id="job-123",
user_email="user@example.com",
action_type="lyrics",
artist="Test Artist",
title="Test Song",
)
assert result is True
service.template_service.render_action_needed_lyrics.assert_called_once()
service.email_service.send_action_reminder.assert_called_once()
@pytest.mark.asyncio
async def test_send_instrumental_reminder_success(self):
"""Test successful instrumental reminder email sending."""
service = JobNotificationService()
service.email_service = Mock()
service.email_service.send_action_reminder.return_value = True
service.template_service = Mock()
service.template_service.render_action_needed_instrumental.return_value = "Select instrumental"
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
result = await service.send_action_reminder_email(
job_id="job-123",
user_email="user@example.com",
action_type="instrumental",
)
assert result is True
service.template_service.render_action_needed_instrumental.assert_called_once()
service.email_service.send_action_reminder.assert_called_once()
@pytest.mark.asyncio
async def test_send_reminder_disabled(self):
"""Test that reminder email is skipped when auto emails are disabled."""
service = JobNotificationService()
service.email_service = Mock()
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', False):
result = await service.send_action_reminder_email(
job_id="job-123",
user_email="user@example.com",
action_type="lyrics",
)
assert result is False
service.email_service.send_action_reminder.assert_not_called()
@pytest.mark.asyncio
async def test_send_reminder_no_email(self):
"""Test that reminder email is skipped when no user email."""
service = JobNotificationService()
service.email_service = Mock()
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
result = await service.send_action_reminder_email(
job_id="job-123",
user_email=None,
action_type="lyrics",
)
assert result is False
@pytest.mark.asyncio
async def test_send_reminder_unknown_action_type(self):
"""Test handling of unknown action type."""
service = JobNotificationService()
service.email_service = Mock()
service.template_service = Mock()
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
result = await service.send_action_reminder_email(
job_id="job-123",
user_email="user@example.com",
action_type="unknown",
)
assert result is False
service.email_service.send_action_reminder.assert_not_called()
@pytest.mark.asyncio
async def test_send_lyrics_reminder_includes_review_url(self):
"""Test that lyrics reminder includes correct review URL."""
service = JobNotificationService()
service.frontend_url = "https://gen.nomadkaraoke.com"
service.backend_url = "https://api.nomadkaraoke.com"
service.email_service = Mock()
service.email_service.send_action_reminder.return_value = True
service.template_service = Mock()
service.template_service.render_action_needed_lyrics.return_value = "Review"
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
await service.send_action_reminder_email(
job_id="job-123",
user_email="user@example.com",
action_type="lyrics",
audio_hash="hash123",
review_token="token456",
)
# Verify the review URL was passed to template
call_kwargs = service.template_service.render_action_needed_lyrics.call_args.kwargs
review_url = call_kwargs.get('review_url')
assert "audioHash=hash123" in review_url
assert "reviewToken=token456" in review_url
@pytest.mark.asyncio
async def test_send_instrumental_reminder_includes_url(self):
"""Test that instrumental reminder includes correct URL."""
service = JobNotificationService()
service.frontend_url = "https://gen.nomadkaraoke.com"
service.backend_url = "https://api.nomadkaraoke.com"
service.email_service = Mock()
service.email_service.send_action_reminder.return_value = True
service.template_service = Mock()
service.template_service.render_action_needed_instrumental.return_value = "Select"
with patch('backend.services.job_notification_service.ENABLE_AUTO_EMAILS', True):
await service.send_action_reminder_email(
job_id="job-123",
user_email="user@example.com",
action_type="instrumental",
instrumental_token="inst-token",
)
# Verify the instrumental URL was passed to template
call_kwargs = service.template_service.render_action_needed_instrumental.call_args.kwargs
instrumental_url = call_kwargs.get('instrumental_url')
assert "instrumentalToken=inst-token" in instrumental_url
class TestGetCompletionMessage:
"""Tests for get_completion_message method."""
def test_get_completion_message_basic(self):
"""Test basic completion message retrieval."""
service = JobNotificationService()
service.template_service = Mock()
service.template_service.render_job_completion.return_value = "Your video is ready!"
result = service.get_completion_message(
job_id="job-123",
user_name="Test User",
artist="Test Artist",
title="Test Song",
youtube_url="https://youtube.com/watch?v=123",
dropbox_url="https://dropbox.com/folder/abc",
)
assert result == "Your video is ready!"
service.template_service.render_job_completion.assert_called_once()
def test_get_completion_message_includes_feedback_url(self):
"""Test that completion message includes feedback URL."""
service = JobNotificationService()
service.template_service = Mock()
service.template_service.render_job_completion.return_value = "Message"
service.get_completion_message(job_id="job-123")
# Verify feedback URL was passed
call_kwargs = service.template_service.render_job_completion.call_args.kwargs
assert 'feedback_url' in call_kwargs
def test_get_completion_message_with_defaults(self):
"""Test completion message with default values."""
service = JobNotificationService()
service.template_service = Mock()
service.template_service.render_job_completion.return_value = "Message"
service.get_completion_message(job_id="job-123")
call_kwargs = service.template_service.render_job_completion.call_args.kwargs
assert call_kwargs.get('job_id') == "job-123"
# Other params should be None/default
assert call_kwargs.get('name') is None
assert call_kwargs.get('artist') is None
class TestGlobalInstance:
"""Tests for global instance management."""
def test_get_job_notification_service_returns_same_instance(self):
"""Test that get_job_notification_service returns singleton."""
# Reset global
import backend.services.job_notification_service as jns
jns._job_notification_service = None
service1 = get_job_notification_service()
service2 = get_job_notification_service()
assert service1 is service2
def test_service_initializes_with_dependencies(self):
"""Test that service initializes with email and template services."""
service = JobNotificationService()
assert service.email_service is not None
assert service.template_service is not None
assert service.frontend_url is not None
assert service.backend_url is not None
+11 -1
View File
@@ -50,7 +50,16 @@ def mock_worker_service():
@pytest.fixture
def client(mock_job_manager, mock_worker_service):
def mock_theme_service():
"""Create a mock ThemeService that returns 'nomad' as default theme."""
service = MagicMock()
service.get_default_theme_id.return_value = "nomad"
service.get_theme.return_value = None
return service
@pytest.fixture
def client(mock_job_manager, mock_worker_service, mock_theme_service):
"""Create TestClient with mocked dependencies."""
mock_creds = MagicMock()
mock_creds.universe_domain = 'googleapis.com'
@@ -63,6 +72,7 @@ def client(mock_job_manager, mock_worker_service):
# Also patch JobManager class used in dependencies.py for auth checks
with patch('backend.api.routes.jobs.job_manager', mock_job_manager), \
patch('backend.api.routes.jobs.worker_service', mock_worker_service), \
patch('backend.api.routes.jobs.get_theme_service', return_value=mock_theme_service), \
patch('backend.services.job_manager.JobManager', mock_job_manager_factory), \
patch('backend.services.firestore_service.firestore'), \
patch('backend.services.storage_service.storage'), \
@@ -0,0 +1,567 @@
"""
Tests for LocalPreviewEncodingService.
Tests cover:
- Service initialization
- Hardware acceleration detection (NVENC)
- FFmpeg filter path escaping
- ASS filter building
- Preview FFmpeg command construction
- Preview encoding execution
- Error handling
"""
import os
import tempfile
import pytest
from unittest.mock import MagicMock, patch
from backend.services.local_preview_encoding_service import (
LocalPreviewEncodingService,
PreviewEncodingConfig,
PreviewEncodingResult,
get_local_preview_encoding_service,
)
class TestLocalPreviewEncodingServiceInit:
"""Test service initialization."""
def test_init_default_values(self):
"""Test default initialization."""
service = LocalPreviewEncodingService()
assert service._nvenc_available is None # Lazy detection
assert service._video_encoder is None # Lazy detection
assert service._hwaccel_flags is None # Lazy detection
def test_init_with_custom_logger(self):
"""Test initialization with custom logger."""
import logging
custom_logger = logging.getLogger("test")
service = LocalPreviewEncodingService(logger=custom_logger)
assert service.logger is custom_logger
class TestLocalPreviewEncodingServiceHWAccel:
"""Test hardware acceleration detection."""
@patch("subprocess.run")
def test_detect_nvenc_available(self, mock_run):
"""Test NVENC detection when available."""
mock_run.return_value = MagicMock(returncode=0)
service = LocalPreviewEncodingService()
result = service._detect_nvenc_support()
assert result is True
@patch("subprocess.run")
def test_detect_nvenc_not_available(self, mock_run):
"""Test fallback when NVENC not available."""
import subprocess
mock_run.side_effect = subprocess.CalledProcessError(1, "test")
service = LocalPreviewEncodingService()
result = service._detect_nvenc_support()
assert result is False
@patch("subprocess.run")
def test_detect_nvenc_timeout(self, mock_run):
"""Test NVENC detection timeout handling."""
import subprocess
mock_run.side_effect = subprocess.TimeoutExpired("test", 30)
service = LocalPreviewEncodingService()
result = service._detect_nvenc_support()
assert result is False
@patch.object(LocalPreviewEncodingService, "_detect_nvenc_support")
def test_configure_nvenc_available(self, mock_detect):
"""Test configuration when NVENC is available."""
mock_detect.return_value = True
service = LocalPreviewEncodingService()
service._configure_hardware_acceleration()
assert service._nvenc_available is True
assert service._video_encoder == "h264_nvenc"
assert "-hwaccel" in service._hwaccel_flags
@patch.object(LocalPreviewEncodingService, "_detect_nvenc_support")
def test_configure_no_nvenc(self, mock_detect):
"""Test configuration when NVENC not available."""
mock_detect.return_value = False
service = LocalPreviewEncodingService()
service._configure_hardware_acceleration()
assert service._nvenc_available is False
assert service._video_encoder == "libx264"
assert service._hwaccel_flags == []
class TestLocalPreviewEncodingServiceProperties:
"""Test property accessors that trigger lazy detection."""
@patch.object(LocalPreviewEncodingService, "_configure_hardware_acceleration")
def test_nvenc_available_triggers_detection(self, mock_config):
"""Test that nvenc_available property triggers detection."""
service = LocalPreviewEncodingService()
service._nvenc_available = None
# Access property
_ = service.nvenc_available
mock_config.assert_called_once()
@patch.object(LocalPreviewEncodingService, "_configure_hardware_acceleration")
def test_video_encoder_triggers_detection(self, mock_config):
"""Test that video_encoder property triggers detection."""
service = LocalPreviewEncodingService()
service._video_encoder = None
# Access property
_ = service.video_encoder
mock_config.assert_called_once()
@patch.object(LocalPreviewEncodingService, "_configure_hardware_acceleration")
def test_hwaccel_flags_triggers_detection(self, mock_config):
"""Test that hwaccel_flags property triggers detection."""
service = LocalPreviewEncodingService()
service._hwaccel_flags = None
# Access property
_ = service.hwaccel_flags
mock_config.assert_called_once()
class TestLocalPreviewEncodingServicePathEscaping:
"""Test FFmpeg filter path escaping."""
def test_escape_simple_path(self):
"""Test escaping a simple path."""
service = LocalPreviewEncodingService()
result = service._escape_ffmpeg_filter_path("/simple/path.ass")
assert result == "/simple/path.ass"
def test_escape_path_with_spaces(self):
"""Test escaping path with spaces."""
service = LocalPreviewEncodingService()
result = service._escape_ffmpeg_filter_path("/path/with spaces/file.ass")
assert "\\ " in result # Spaces should be escaped
def test_escape_path_with_apostrophe(self):
"""Test escaping path with apostrophe (common in song titles)."""
service = LocalPreviewEncodingService()
result = service._escape_ffmpeg_filter_path("/path/I'm With You/file.ass")
assert "\\\\\\'" in result # Apostrophe should be triple-backslash escaped
def test_escape_path_with_special_chars(self):
"""Test escaping path with FFmpeg special characters."""
service = LocalPreviewEncodingService()
result = service._escape_ffmpeg_filter_path("/path:with[special];chars,here.ass")
# Should escape :,[];
assert "\\:" in result
assert "\\[" in result
assert "\\]" in result
assert "\\;" in result
assert "\\," in result
def test_escape_path_with_backslashes(self):
"""Test escaping path with existing backslashes."""
service = LocalPreviewEncodingService()
result = service._escape_ffmpeg_filter_path("/path\\with\\backslashes.ass")
assert "\\\\" in result
class TestLocalPreviewEncodingServiceASSFilter:
"""Test ASS filter building."""
def test_build_ass_filter_simple(self):
"""Test building ASS filter without font."""
service = LocalPreviewEncodingService()
result = service._build_ass_filter("/path/to/file.ass")
assert result.startswith("ass=")
assert "fontsdir" not in result
def test_build_ass_filter_with_font(self):
"""Test building ASS filter with custom font."""
with tempfile.TemporaryDirectory() as tmpdir:
font_path = os.path.join(tmpdir, "custom.ttf")
with open(font_path, "w") as f:
f.write("fake font")
service = LocalPreviewEncodingService()
result = service._build_ass_filter("/path/to/file.ass", font_path)
assert "fontsdir=" in result
def test_build_ass_filter_nonexistent_font(self):
"""Test building ASS filter with nonexistent font file."""
service = LocalPreviewEncodingService()
result = service._build_ass_filter("/path/to/file.ass", "/nonexistent/font.ttf")
# Should not include fontsdir for nonexistent font
assert "fontsdir" not in result
class TestLocalPreviewEncodingServiceFFmpegCommand:
"""Test FFmpeg command building."""
def test_build_command_solid_background_libx264(self):
"""Test command building with solid background and libx264."""
service = LocalPreviewEncodingService()
service._nvenc_available = False
service._video_encoder = "libx264"
service._hwaccel_flags = []
config = PreviewEncodingConfig(
ass_path="/path/to/subs.ass",
audio_path="/path/to/audio.flac",
output_path="/path/to/output.mp4",
background_color="black"
)
cmd = service._build_preview_ffmpeg_command(config)
assert "ffmpeg" in cmd
assert "-r" in cmd and "24" in cmd # Frame rate
assert "-c:v" in cmd
assert "libx264" in cmd
assert "-preset" in cmd and "superfast" in cmd
assert "-crf" in cmd and "28" in cmd
assert "color=c=black" in " ".join(cmd)
def test_build_command_with_nvenc(self):
"""Test command building with NVENC hardware acceleration."""
service = LocalPreviewEncodingService()
service._nvenc_available = True
service._video_encoder = "h264_nvenc"
service._hwaccel_flags = ["-hwaccel", "cuda", "-hwaccel_output_format", "cuda"]
config = PreviewEncodingConfig(
ass_path="/path/to/subs.ass",
audio_path="/path/to/audio.flac",
output_path="/path/to/output.mp4"
)
cmd = service._build_preview_ffmpeg_command(config)
assert "h264_nvenc" in cmd
assert "-hwaccel" in cmd
assert "-preset" in cmd and "p1" in cmd # Fastest NVENC preset
def test_build_command_with_background_image(self):
"""Test command building with background image."""
with tempfile.TemporaryDirectory() as tmpdir:
bg_path = os.path.join(tmpdir, "background.png")
with open(bg_path, "wb") as f:
f.write(b"fake png")
service = LocalPreviewEncodingService()
service._nvenc_available = False
service._video_encoder = "libx264"
service._hwaccel_flags = []
config = PreviewEncodingConfig(
ass_path="/path/to/subs.ass",
audio_path="/path/to/audio.flac",
output_path="/path/to/output.mp4",
background_image_path=bg_path
)
cmd = service._build_preview_ffmpeg_command(config)
assert "-loop" in cmd
assert "1" in cmd
assert bg_path in cmd
# Should have scale filter for background image
vf_index = cmd.index("-vf")
vf_value = cmd[vf_index + 1]
assert "scale=" in vf_value
assert "pad=" in vf_value
def test_build_command_resolution(self):
"""Test that command uses correct preview resolution."""
service = LocalPreviewEncodingService()
service._nvenc_available = False
service._video_encoder = "libx264"
service._hwaccel_flags = []
config = PreviewEncodingConfig(
ass_path="/path/to/subs.ass",
audio_path="/path/to/audio.flac",
output_path="/path/to/output.mp4"
)
cmd = service._build_preview_ffmpeg_command(config)
cmd_str = " ".join(cmd)
# Resolution should be 480x270
assert "480" in cmd_str
assert "270" in cmd_str
def test_build_command_audio_settings(self):
"""Test that command uses correct audio settings."""
service = LocalPreviewEncodingService()
service._nvenc_available = False
service._video_encoder = "libx264"
service._hwaccel_flags = []
config = PreviewEncodingConfig(
ass_path="/path/to/subs.ass",
audio_path="/path/to/audio.flac",
output_path="/path/to/output.mp4"
)
cmd = service._build_preview_ffmpeg_command(config)
assert "-c:a" in cmd
assert "aac" in cmd
assert "-b:a" in cmd
assert "96k" in cmd
class TestLocalPreviewEncodingServiceEncode:
"""Test preview encoding execution."""
@patch("subprocess.run")
def test_encode_preview_success(self, mock_run):
"""Test successful preview encoding."""
mock_run.return_value = MagicMock(returncode=0, stderr="")
with tempfile.TemporaryDirectory() as tmpdir:
ass_path = os.path.join(tmpdir, "subs.ass")
audio_path = os.path.join(tmpdir, "audio.flac")
output_path = os.path.join(tmpdir, "output.mp4")
# Create input files
with open(ass_path, "w") as f:
f.write("[Script Info]\n")
with open(audio_path, "wb") as f:
f.write(b"fake audio")
service = LocalPreviewEncodingService()
service._nvenc_available = False
service._video_encoder = "libx264"
service._hwaccel_flags = []
config = PreviewEncodingConfig(
ass_path=ass_path,
audio_path=audio_path,
output_path=output_path
)
result = service.encode_preview(config)
assert result.success is True
assert result.output_path == output_path
assert result.error is None
mock_run.assert_called_once()
@patch("subprocess.run")
def test_encode_preview_ffmpeg_failure(self, mock_run):
"""Test preview encoding with FFmpeg failure."""
mock_run.return_value = MagicMock(returncode=1, stderr="FFmpeg error message")
with tempfile.TemporaryDirectory() as tmpdir:
ass_path = os.path.join(tmpdir, "subs.ass")
audio_path = os.path.join(tmpdir, "audio.flac")
output_path = os.path.join(tmpdir, "output.mp4")
with open(ass_path, "w") as f:
f.write("[Script Info]\n")
with open(audio_path, "wb") as f:
f.write(b"fake audio")
service = LocalPreviewEncodingService()
service._nvenc_available = False
service._video_encoder = "libx264"
service._hwaccel_flags = []
config = PreviewEncodingConfig(
ass_path=ass_path,
audio_path=audio_path,
output_path=output_path
)
result = service.encode_preview(config)
assert result.success is False
assert "FFmpeg preview encoding failed" in result.error
def test_encode_preview_missing_ass_file(self):
"""Test preview encoding with missing ASS file."""
with tempfile.TemporaryDirectory() as tmpdir:
audio_path = os.path.join(tmpdir, "audio.flac")
output_path = os.path.join(tmpdir, "output.mp4")
with open(audio_path, "wb") as f:
f.write(b"fake audio")
service = LocalPreviewEncodingService()
config = PreviewEncodingConfig(
ass_path="/nonexistent/subs.ass",
audio_path=audio_path,
output_path=output_path
)
result = service.encode_preview(config)
assert result.success is False
assert "not found" in result.error
def test_encode_preview_missing_audio_file(self):
"""Test preview encoding with missing audio file."""
with tempfile.TemporaryDirectory() as tmpdir:
ass_path = os.path.join(tmpdir, "subs.ass")
output_path = os.path.join(tmpdir, "output.mp4")
with open(ass_path, "w") as f:
f.write("[Script Info]\n")
service = LocalPreviewEncodingService()
config = PreviewEncodingConfig(
ass_path=ass_path,
audio_path="/nonexistent/audio.flac",
output_path=output_path
)
result = service.encode_preview(config)
assert result.success is False
assert "not found" in result.error
@patch("subprocess.run")
def test_encode_preview_timeout(self, mock_run):
"""Test preview encoding timeout."""
import subprocess
mock_run.side_effect = subprocess.TimeoutExpired("ffmpeg", 300)
with tempfile.TemporaryDirectory() as tmpdir:
ass_path = os.path.join(tmpdir, "subs.ass")
audio_path = os.path.join(tmpdir, "audio.flac")
output_path = os.path.join(tmpdir, "output.mp4")
with open(ass_path, "w") as f:
f.write("[Script Info]\n")
with open(audio_path, "wb") as f:
f.write(b"fake audio")
service = LocalPreviewEncodingService()
service._nvenc_available = False
service._video_encoder = "libx264"
service._hwaccel_flags = []
config = PreviewEncodingConfig(
ass_path=ass_path,
audio_path=audio_path,
output_path=output_path
)
result = service.encode_preview(config)
assert result.success is False
assert "timed out" in result.error
class TestPreviewEncodingConfig:
"""Test PreviewEncodingConfig dataclass."""
def test_config_required_fields(self):
"""Test that required fields must be provided."""
config = PreviewEncodingConfig(
ass_path="/path/to/subs.ass",
audio_path="/path/to/audio.flac",
output_path="/path/to/output.mp4"
)
assert config.ass_path == "/path/to/subs.ass"
assert config.background_color == "black" # Default
assert config.background_image_path is None # Optional
def test_config_all_fields(self):
"""Test config with all fields."""
config = PreviewEncodingConfig(
ass_path="/path/to/subs.ass",
audio_path="/path/to/audio.flac",
output_path="/path/to/output.mp4",
background_image_path="/path/to/bg.png",
background_color="red",
font_path="/path/to/font.ttf"
)
assert config.background_image_path == "/path/to/bg.png"
assert config.background_color == "red"
assert config.font_path == "/path/to/font.ttf"
class TestPreviewEncodingResult:
"""Test PreviewEncodingResult dataclass."""
def test_result_success(self):
"""Test successful result."""
result = PreviewEncodingResult(
success=True,
output_path="/path/to/output.mp4"
)
assert result.success is True
assert result.error is None
assert result.output_path == "/path/to/output.mp4"
def test_result_failure(self):
"""Test failure result."""
result = PreviewEncodingResult(
success=False,
error="Something went wrong"
)
assert result.success is False
assert result.error == "Something went wrong"
assert result.output_path is None
class TestGetLocalPreviewEncodingService:
"""Test factory function."""
def test_get_service_creates_instance(self):
"""Test that factory function creates a new instance."""
import backend.services.local_preview_encoding_service as module
module._local_preview_encoding_service = None
service = get_local_preview_encoding_service()
assert service is not None
assert isinstance(service, LocalPreviewEncodingService)
def test_get_service_returns_singleton(self):
"""Test that factory function returns the same instance."""
import backend.services.local_preview_encoding_service as module
module._local_preview_encoding_service = None
service1 = get_local_preview_encoding_service()
service2 = get_local_preview_encoding_service()
assert service1 is service2
class TestPreviewEncodingConstants:
"""Test that encoding constants match expected values."""
def test_preview_resolution(self):
"""Test preview resolution constants."""
service = LocalPreviewEncodingService()
assert service.PREVIEW_WIDTH == 480
assert service.PREVIEW_HEIGHT == 270
def test_preview_fps(self):
"""Test preview frame rate constant."""
service = LocalPreviewEncodingService()
assert service.PREVIEW_FPS == 24
def test_preview_audio_bitrate(self):
"""Test preview audio bitrate constant."""
service = LocalPreviewEncodingService()
assert service.PREVIEW_AUDIO_BITRATE == "96k"
+2 -1
View File
@@ -149,7 +149,8 @@ class TestUploadSubmissionRequest:
# CDG/TXT disabled by default (requires style config)
assert request.enable_cdg is False
assert request.enable_txt is False
assert request.enable_youtube_upload is False
# YouTube upload default is None (use server default)
assert request.enable_youtube_upload is None
class TestStartReviewRequest:
+119
View File
@@ -0,0 +1,119 @@
"""Tests for SpaCy preloader service."""
import pytest
from unittest.mock import patch, MagicMock
from backend.services.spacy_preloader import (
preload_spacy_model,
get_preloaded_model,
is_model_preloaded,
clear_preloaded_models,
)
class TestSpacyPreloader:
"""Tests for SpaCy preloading functionality."""
def setup_method(self):
"""Clear preloaded models before each test."""
clear_preloaded_models()
def teardown_method(self):
"""Clear preloaded models after each test."""
clear_preloaded_models()
def test_preload_spacy_model_loads_and_stores(self):
"""GIVEN no preloaded models
WHEN preload_spacy_model is called
THEN model should be loaded and stored in singleton."""
mock_nlp = MagicMock()
with patch("spacy.load", return_value=mock_nlp) as mock_load:
preload_spacy_model("en_core_web_sm")
mock_load.assert_called_once_with("en_core_web_sm")
assert is_model_preloaded("en_core_web_sm")
assert get_preloaded_model("en_core_web_sm") is mock_nlp
def test_preload_is_idempotent(self):
"""GIVEN a model already preloaded
WHEN preload_spacy_model is called again
THEN model should not be reloaded."""
mock_nlp = MagicMock()
with patch("spacy.load", return_value=mock_nlp) as mock_load:
preload_spacy_model("en_core_web_sm")
preload_spacy_model("en_core_web_sm") # Second call
# Should only load once
assert mock_load.call_count == 1
def test_get_preloaded_model_returns_none_if_not_loaded(self):
"""GIVEN no preloaded models
WHEN get_preloaded_model is called
THEN should return None."""
assert get_preloaded_model("en_core_web_sm") is None
assert not is_model_preloaded("en_core_web_sm")
def test_preload_different_models(self):
"""GIVEN no preloaded models
WHEN preload_spacy_model is called with different model names
THEN each model should be loaded and stored separately."""
mock_nlp_sm = MagicMock(name="en_core_web_sm")
mock_nlp_md = MagicMock(name="en_core_web_md")
def mock_load(model_name):
if model_name == "en_core_web_sm":
return mock_nlp_sm
elif model_name == "en_core_web_md":
return mock_nlp_md
raise ValueError(f"Unknown model: {model_name}")
with patch("spacy.load", side_effect=mock_load):
preload_spacy_model("en_core_web_sm")
preload_spacy_model("en_core_web_md")
assert is_model_preloaded("en_core_web_sm")
assert is_model_preloaded("en_core_web_md")
assert get_preloaded_model("en_core_web_sm") is mock_nlp_sm
assert get_preloaded_model("en_core_web_md") is mock_nlp_md
def test_clear_preloaded_models(self):
"""GIVEN preloaded models
WHEN clear_preloaded_models is called
THEN all models should be removed."""
mock_nlp = MagicMock()
with patch("spacy.load", return_value=mock_nlp):
preload_spacy_model("en_core_web_sm")
assert is_model_preloaded("en_core_web_sm")
clear_preloaded_models()
assert not is_model_preloaded("en_core_web_sm")
assert get_preloaded_model("en_core_web_sm") is None
def test_preload_failure_raises_exception(self):
"""GIVEN spacy.load raises an exception
WHEN preload_spacy_model is called
THEN exception should be propagated."""
with patch(
"spacy.load",
side_effect=OSError("Model not found"),
):
with pytest.raises(OSError, match="Model not found"):
preload_spacy_model("nonexistent_model")
# Model should not be marked as preloaded
assert not is_model_preloaded("nonexistent_model")
def test_preload_uses_default_model_name(self):
"""GIVEN no model name specified
WHEN preload_spacy_model is called without arguments
THEN should use en_core_web_sm as default."""
mock_nlp = MagicMock()
with patch("spacy.load", return_value=mock_nlp) as mock_load:
preload_spacy_model()
mock_load.assert_called_once_with("en_core_web_sm")
assert is_model_preloaded("en_core_web_sm")
+295
View File
@@ -0,0 +1,295 @@
"""
Unit tests for template service.
"""
import pytest
from unittest.mock import Mock, patch, MagicMock
from backend.services.template_service import (
TemplateService,
get_template_service,
DEFAULT_JOB_COMPLETION_TEMPLATE,
DEFAULT_ACTION_NEEDED_LYRICS_TEMPLATE,
DEFAULT_ACTION_NEEDED_INSTRUMENTAL_TEMPLATE,
)
class TestTemplateRendering:
"""Tests for template rendering functionality."""
def test_render_template_basic(self):
"""Test basic template variable replacement."""
service = TemplateService()
template = "Hello {name}, your job {job_id} is ready!"
variables = {"name": "Alice", "job_id": "123"}
result = service.render_template(template, variables)
assert result == "Hello Alice, your job 123 is ready!"
def test_render_template_missing_variables(self):
"""Test that missing variables are replaced with empty strings."""
service = TemplateService()
template = "Hello {name}, your {thing} is {status}!"
variables = {"name": "Bob"}
result = service.render_template(template, variables)
assert result == "Hello Bob, your is !"
def test_render_template_none_values(self):
"""Test that None values are handled correctly."""
service = TemplateService()
template = "Hello {name}!"
variables = {"name": None}
result = service.render_template(template, variables)
assert result == "Hello !"
def test_render_template_removes_feedback_section_when_no_url(self):
"""Test that feedback section is removed when feedback_url is not provided."""
service = TemplateService()
template = """Thanks for your order!
If you have a moment, I'd really appreciate your feedback (takes 2 minutes):
{feedback_url}
Have a great day!"""
variables = {"feedback_url": None}
result = service.render_template(template, variables)
assert "feedback" not in result.lower()
assert "Thanks for your order!" in result
assert "Have a great day!" in result
def test_render_template_keeps_feedback_section_when_url_provided(self):
"""Test that feedback section is kept when feedback_url is provided."""
service = TemplateService()
template = """Thanks!
If you have a moment, I'd really appreciate your feedback (takes 2 minutes):
{feedback_url}
Bye!"""
variables = {"feedback_url": "https://example.com/feedback"}
result = service.render_template(template, variables)
assert "https://example.com/feedback" in result
assert "feedback" in result.lower()
class TestJobCompletionTemplate:
"""Tests for job completion template rendering."""
def test_render_job_completion_all_fields(self):
"""Test rendering job completion with all fields."""
service = TemplateService()
# Mock _fetch_template_from_gcs to return None (use default)
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
result = service.render_job_completion(
name="Alice",
youtube_url="https://youtube.com/watch?v=123",
dropbox_url="https://dropbox.com/folder/abc",
artist="Test Artist",
title="Test Song",
job_id="job-123",
feedback_url="https://example.com/feedback",
)
assert "Alice" in result
assert "https://youtube.com/watch?v=123" in result
assert "https://dropbox.com/folder/abc" in result
assert "https://example.com/feedback" in result
def test_render_job_completion_defaults(self):
"""Test rendering job completion with default values."""
service = TemplateService()
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
result = service.render_job_completion()
assert "there" in result # Default name
assert "[YouTube URL not available]" in result
assert "[Dropbox URL not available]" in result
def test_render_job_completion_no_feedback_url(self):
"""Test rendering job completion without feedback URL removes section."""
service = TemplateService()
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
result = service.render_job_completion(
name="Bob",
youtube_url="https://youtube.com/123",
dropbox_url="https://dropbox.com/abc",
)
assert "Bob" in result
# Feedback section should be removed
assert "really appreciate your feedback" not in result
class TestActionNeededTemplates:
"""Tests for action-needed template rendering."""
def test_render_action_needed_lyrics(self):
"""Test rendering lyrics review reminder."""
service = TemplateService()
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
result = service.render_action_needed_lyrics(
name="Charlie",
artist="Test Artist",
title="Test Song",
review_url="https://example.com/review",
)
assert "Charlie" in result
assert "Test Artist" in result
assert "Test Song" in result
assert "https://example.com/review" in result
def test_render_action_needed_instrumental(self):
"""Test rendering instrumental selection reminder."""
service = TemplateService()
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
result = service.render_action_needed_instrumental(
name="Diana",
artist="Test Artist",
title="Test Song",
instrumental_url="https://example.com/instrumental",
)
assert "Diana" in result
assert "Test Artist" in result
assert "Test Song" in result
assert "https://example.com/instrumental" in result
class TestGCSFetching:
"""Tests for GCS template fetching."""
def test_fetch_template_from_gcs_success(self):
"""Test successful template fetch from GCS."""
service = TemplateService()
# Mock the bucket and blob
mock_blob = Mock()
mock_blob.exists.return_value = True
mock_blob.download_as_text.return_value = "Custom template {name}"
mock_bucket = Mock()
mock_bucket.blob.return_value = mock_blob
with patch.object(service, "_bucket", mock_bucket):
result = service._fetch_template_from_gcs("test.txt")
assert result == "Custom template {name}"
mock_bucket.blob.assert_called_once_with("templates/test.txt")
def test_fetch_template_from_gcs_not_found(self):
"""Test template fetch when file doesn't exist in GCS."""
service = TemplateService()
mock_blob = Mock()
mock_blob.exists.return_value = False
mock_bucket = Mock()
mock_bucket.blob.return_value = mock_blob
with patch.object(service, "_bucket", mock_bucket):
result = service._fetch_template_from_gcs("nonexistent.txt")
assert result is None
def test_fetch_template_from_gcs_error(self):
"""Test template fetch handles errors gracefully."""
service = TemplateService()
mock_bucket = Mock()
mock_bucket.blob.side_effect = Exception("GCS error")
with patch.object(service, "_bucket", mock_bucket):
result = service._fetch_template_from_gcs("test.txt")
assert result is None
def test_get_job_completion_template_fallback(self):
"""Test that default template is used when GCS fetch fails."""
service = TemplateService()
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
result = service.get_job_completion_template()
assert result == DEFAULT_JOB_COMPLETION_TEMPLATE
def test_get_action_needed_lyrics_template_fallback(self):
"""Test that default lyrics template is used when GCS fetch fails."""
service = TemplateService()
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
result = service.get_action_needed_lyrics_template()
assert result == DEFAULT_ACTION_NEEDED_LYRICS_TEMPLATE
def test_get_action_needed_instrumental_template_fallback(self):
"""Test that default instrumental template is used when GCS fetch fails."""
service = TemplateService()
with patch.object(service, "_fetch_template_from_gcs", return_value=None):
result = service.get_action_needed_instrumental_template()
assert result == DEFAULT_ACTION_NEEDED_INSTRUMENTAL_TEMPLATE
class TestTemplateUpload:
"""Tests for template upload functionality."""
def test_upload_template_success(self):
"""Test successful template upload."""
service = TemplateService()
mock_blob = Mock()
mock_bucket = Mock()
mock_bucket.blob.return_value = mock_blob
with patch.object(service, "_bucket", mock_bucket):
result = service.upload_template("test.txt", "Test content")
assert result is True
mock_bucket.blob.assert_called_once_with("templates/test.txt")
mock_blob.upload_from_string.assert_called_once_with(
"Test content", content_type="text/plain"
)
def test_upload_template_error(self):
"""Test template upload handles errors."""
service = TemplateService()
mock_blob = Mock()
mock_blob.upload_from_string.side_effect = Exception("Upload failed")
mock_bucket = Mock()
mock_bucket.blob.return_value = mock_blob
with patch.object(service, "_bucket", mock_bucket):
result = service.upload_template("test.txt", "Test content")
assert result is False
class TestGlobalInstance:
"""Tests for global instance management."""
def test_get_template_service_returns_same_instance(self):
"""Test that get_template_service returns singleton."""
# Reset global
import backend.services.template_service as ts
ts._template_service = None
service1 = get_template_service()
service2 = get_template_service()
assert service1 is service2
+350
View File
@@ -0,0 +1,350 @@
"""
Unit tests for tenant API routes.
Tests the /api/tenant/config endpoints.
Note: We need to patch get_tenant_service in both the routes module AND the
middleware module since the middleware runs first and also calls get_tenant_service.
"""
import pytest
from unittest.mock import MagicMock, patch
from fastapi.testclient import TestClient
from backend.models.tenant import (
TenantConfig,
TenantBranding,
TenantFeatures,
TenantDefaults,
TenantAuth,
TenantPublicConfig,
)
# Sample tenant config for mocking
SAMPLE_VOCALSTAR_CONFIG = TenantConfig(
id="vocalstar",
name="Vocal Star",
subdomain="vocalstar.nomadkaraoke.com",
is_active=True,
branding=TenantBranding(
logo_url="https://example.com/logo.png",
primary_color="#ffff00",
secondary_color="#006CF9",
site_title="Vocal Star Karaoke Generator",
),
features=TenantFeatures(
audio_search=False,
youtube_url=False,
youtube_upload=False,
dropbox_upload=False,
gdrive_upload=False,
theme_selection=False,
),
defaults=TenantDefaults(
theme_id="vocalstar",
locked_theme="vocalstar",
distribution_mode="download_only",
),
auth=TenantAuth(
allowed_email_domains=["vocal-star.com", "vocalstarmusic.com"],
require_email_domain=True,
sender_email="vocalstar@nomadkaraoke.com",
),
)
INACTIVE_CONFIG = TenantConfig(
id="inactive",
name="Inactive Tenant",
subdomain="inactive.nomadkaraoke.com",
is_active=False,
)
class TestTenantConfigEndpoint:
"""Tests for GET /api/tenant/config endpoint."""
@pytest.fixture
def mock_tenant_service(self):
"""Mock the tenant service in both routes and middleware."""
service = MagicMock()
# Patch in both places: routes AND middleware
with patch("backend.api.routes.tenant.get_tenant_service", return_value=service), \
patch("backend.middleware.tenant.get_tenant_service", return_value=service):
yield service
@pytest.fixture
def client(self, mock_tenant_service):
"""Create test client with mocked tenant service."""
from backend.main import app
return TestClient(app)
def test_get_config_default(self, client, mock_tenant_service):
"""Test returns default config when no tenant detected."""
mock_tenant_service.tenant_exists.return_value = False
response = client.get("/api/tenant/config")
assert response.status_code == 200
data = response.json()
assert data["tenant"] is None
assert data["is_default"] is True
def test_get_config_with_query_param(self, client, mock_tenant_service):
"""Test tenant detection via query parameter."""
public_config = TenantPublicConfig.from_config(SAMPLE_VOCALSTAR_CONFIG)
mock_tenant_service.get_public_config.return_value = public_config
response = client.get("/api/tenant/config?tenant=vocalstar")
assert response.status_code == 200
data = response.json()
assert data["tenant"]["id"] == "vocalstar"
assert data["tenant"]["name"] == "Vocal Star"
assert data["is_default"] is False
def test_get_config_with_header(self, client, mock_tenant_service):
"""Test tenant detection via X-Tenant-ID header."""
public_config = TenantPublicConfig.from_config(SAMPLE_VOCALSTAR_CONFIG)
mock_tenant_service.get_public_config.return_value = public_config
response = client.get(
"/api/tenant/config",
headers={"X-Tenant-ID": "vocalstar"},
)
assert response.status_code == 200
data = response.json()
assert data["tenant"]["id"] == "vocalstar"
assert data["is_default"] is False
def test_get_config_query_param_takes_priority(self, client, mock_tenant_service):
"""Test query param takes priority over header."""
public_config = TenantPublicConfig.from_config(SAMPLE_VOCALSTAR_CONFIG)
mock_tenant_service.get_public_config.return_value = public_config
response = client.get(
"/api/tenant/config?tenant=vocalstar",
headers={"X-Tenant-ID": "other"},
)
assert response.status_code == 200
# Should use vocalstar from query param
mock_tenant_service.get_public_config.assert_called_with("vocalstar")
def test_get_config_tenant_not_found(self, client, mock_tenant_service):
"""Test returns default when tenant not found."""
mock_tenant_service.get_public_config.return_value = None
response = client.get("/api/tenant/config?tenant=nonexistent")
assert response.status_code == 200
data = response.json()
assert data["tenant"] is None
assert data["is_default"] is True
def test_get_config_inactive_tenant(self, client, mock_tenant_service):
"""Test returns default when tenant is inactive."""
inactive_public = TenantPublicConfig.from_config(INACTIVE_CONFIG)
mock_tenant_service.get_public_config.return_value = inactive_public
response = client.get("/api/tenant/config?tenant=inactive")
assert response.status_code == 200
data = response.json()
assert data["tenant"] is None
assert data["is_default"] is True
def test_get_config_includes_branding(self, client, mock_tenant_service):
"""Test response includes full branding config."""
public_config = TenantPublicConfig.from_config(SAMPLE_VOCALSTAR_CONFIG)
mock_tenant_service.get_public_config.return_value = public_config
response = client.get("/api/tenant/config?tenant=vocalstar")
assert response.status_code == 200
data = response.json()
branding = data["tenant"]["branding"]
assert branding["primary_color"] == "#ffff00"
assert branding["secondary_color"] == "#006CF9"
assert branding["site_title"] == "Vocal Star Karaoke Generator"
def test_get_config_includes_features(self, client, mock_tenant_service):
"""Test response includes feature flags."""
public_config = TenantPublicConfig.from_config(SAMPLE_VOCALSTAR_CONFIG)
mock_tenant_service.get_public_config.return_value = public_config
response = client.get("/api/tenant/config?tenant=vocalstar")
assert response.status_code == 200
data = response.json()
features = data["tenant"]["features"]
assert features["audio_search"] is False
assert features["file_upload"] is True
assert features["youtube_upload"] is False
def test_get_config_includes_defaults(self, client, mock_tenant_service):
"""Test response includes default settings."""
public_config = TenantPublicConfig.from_config(SAMPLE_VOCALSTAR_CONFIG)
mock_tenant_service.get_public_config.return_value = public_config
response = client.get("/api/tenant/config?tenant=vocalstar")
assert response.status_code == 200
data = response.json()
defaults = data["tenant"]["defaults"]
assert defaults["theme_id"] == "vocalstar"
assert defaults["locked_theme"] == "vocalstar"
assert defaults["distribution_mode"] == "download_only"
def test_get_config_includes_allowed_domains(self, client, mock_tenant_service):
"""Test response includes allowed email domains."""
public_config = TenantPublicConfig.from_config(SAMPLE_VOCALSTAR_CONFIG)
mock_tenant_service.get_public_config.return_value = public_config
response = client.get("/api/tenant/config?tenant=vocalstar")
assert response.status_code == 200
data = response.json()
assert "vocal-star.com" in data["tenant"]["allowed_email_domains"]
assert "vocalstarmusic.com" in data["tenant"]["allowed_email_domains"]
class TestTenantConfigByIdEndpoint:
"""Tests for GET /api/tenant/config/{tenant_id} endpoint."""
@pytest.fixture
def mock_tenant_service(self):
"""Mock the tenant service in both routes and middleware."""
service = MagicMock()
with patch("backend.api.routes.tenant.get_tenant_service", return_value=service), \
patch("backend.middleware.tenant.get_tenant_service", return_value=service):
yield service
@pytest.fixture
def client(self, mock_tenant_service):
"""Create test client with mocked tenant service."""
from backend.main import app
return TestClient(app)
def test_get_config_by_id_success(self, client, mock_tenant_service):
"""Test getting config by explicit ID."""
public_config = TenantPublicConfig.from_config(SAMPLE_VOCALSTAR_CONFIG)
mock_tenant_service.get_public_config.return_value = public_config
response = client.get("/api/tenant/config/vocalstar")
assert response.status_code == 200
data = response.json()
assert data["tenant"]["id"] == "vocalstar"
assert data["is_default"] is False
def test_get_config_by_id_not_found(self, client, mock_tenant_service):
"""Test returns default when tenant ID not found."""
mock_tenant_service.get_public_config.return_value = None
response = client.get("/api/tenant/config/nonexistent")
assert response.status_code == 200
data = response.json()
assert data["tenant"] is None
assert data["is_default"] is True
def test_get_config_by_id_inactive(self, client, mock_tenant_service):
"""Test returns default when tenant is inactive."""
inactive_public = TenantPublicConfig.from_config(INACTIVE_CONFIG)
mock_tenant_service.get_public_config.return_value = inactive_public
response = client.get("/api/tenant/config/inactive")
assert response.status_code == 200
data = response.json()
assert data["tenant"] is None
assert data["is_default"] is True
class TestTenantAssetEndpoint:
"""Tests for GET /api/tenant/asset/{tenant_id}/{asset_name} endpoint."""
@pytest.fixture
def mock_tenant_service(self):
"""Mock the tenant service in both routes and middleware."""
service = MagicMock()
with patch("backend.api.routes.tenant.get_tenant_service", return_value=service), \
patch("backend.middleware.tenant.get_tenant_service", return_value=service):
yield service
@pytest.fixture
def client(self, mock_tenant_service):
"""Create test client with mocked tenant service."""
from backend.main import app
return TestClient(app)
def test_get_asset_redirects(self, client, mock_tenant_service):
"""Test asset endpoint redirects to signed URL."""
mock_tenant_service.get_asset_url.return_value = "https://storage.googleapis.com/signed-url"
response = client.get(
"/api/tenant/asset/vocalstar/logo.png",
follow_redirects=False,
)
# Should redirect (307 Temporary Redirect is FastAPI default)
assert response.status_code == 307
assert "storage.googleapis.com" in response.headers["location"]
def test_get_asset_not_found(self, client, mock_tenant_service):
"""Test returns 404 when asset not found."""
mock_tenant_service.get_asset_url.return_value = None
response = client.get("/api/tenant/asset/vocalstar/missing.png")
assert response.status_code == 404
assert "not found" in response.json()["detail"].lower()
class TestTenantConfigResponseSchema:
"""Tests for response schema validation."""
@pytest.fixture
def mock_tenant_service(self):
"""Mock the tenant service in both routes and middleware."""
service = MagicMock()
with patch("backend.api.routes.tenant.get_tenant_service", return_value=service), \
patch("backend.middleware.tenant.get_tenant_service", return_value=service):
yield service
@pytest.fixture
def client(self, mock_tenant_service):
"""Create test client with mocked tenant service."""
from backend.main import app
return TestClient(app)
def test_response_has_required_fields(self, client, mock_tenant_service):
"""Test response always has required fields."""
mock_tenant_service.tenant_exists.return_value = False
response = client.get("/api/tenant/config")
assert response.status_code == 200
data = response.json()
assert "tenant" in data
assert "is_default" in data
def test_tenant_config_excludes_sensitive_fields(self, client, mock_tenant_service):
"""Test public config doesn't include sensitive auth data."""
public_config = TenantPublicConfig.from_config(SAMPLE_VOCALSTAR_CONFIG)
mock_tenant_service.get_public_config.return_value = public_config
response = client.get("/api/tenant/config?tenant=vocalstar")
assert response.status_code == 200
data = response.json()
tenant = data["tenant"]
# Should NOT have full auth object
assert "auth" not in tenant
# Should NOT have sensitive defaults
assert tenant["defaults"].get("brand_prefix") is None
# Should have allowed_email_domains at top level
assert "allowed_email_domains" in tenant
+345
View File
@@ -0,0 +1,345 @@
"""
Unit tests for TenantMiddleware.
Tests tenant detection from headers, query params, and subdomains,
as well as request state attachment.
"""
import pytest
from unittest.mock import MagicMock, patch, AsyncMock
import os
from starlette.requests import Request
from starlette.datastructures import Headers
from starlette.responses import Response
from backend.middleware.tenant import (
TenantMiddleware,
get_tenant_from_request,
get_tenant_config_from_request,
NON_TENANT_SUBDOMAINS,
)
from backend.models.tenant import TenantConfig, TenantFeatures
# Sample tenant config for mocking
SAMPLE_CONFIG = TenantConfig(
id="vocalstar",
name="Vocal Star",
subdomain="vocalstar.nomadkaraoke.com",
is_active=True,
features=TenantFeatures(audio_search=False),
)
INACTIVE_CONFIG = TenantConfig(
id="inactive",
name="Inactive",
subdomain="inactive.nomadkaraoke.com",
is_active=False,
)
class MockRequest:
"""Mock Request object for testing."""
def __init__(self, headers=None, query_params=None):
self.headers = headers or {}
self.query_params = query_params or {}
self.url = MagicMock()
self.url.path = "/api/test"
self.state = MagicMock()
class TestTenantMiddleware:
"""Tests for TenantMiddleware class."""
@pytest.fixture
def middleware(self):
"""Create middleware instance."""
app = MagicMock()
return TenantMiddleware(app)
@pytest.fixture
def mock_tenant_service(self):
"""Create mock tenant service."""
with patch("backend.middleware.tenant.get_tenant_service") as mock:
service = MagicMock()
mock.return_value = service
yield service
# Tests for _extract_tenant_from_host()
def test_extract_tenant_from_host_standard_subdomain(self, middleware, mock_tenant_service):
"""Test extracting tenant from vocalstar.nomadkaraoke.com."""
mock_tenant_service.tenant_exists.return_value = True
tenant_id = middleware._extract_tenant_from_host("vocalstar.nomadkaraoke.com")
assert tenant_id == "vocalstar"
def test_extract_tenant_from_host_gen_subdomain(self, middleware, mock_tenant_service):
"""Test extracting tenant from vocalstar.gen.nomadkaraoke.com."""
mock_tenant_service.tenant_exists.return_value = True
tenant_id = middleware._extract_tenant_from_host("vocalstar.gen.nomadkaraoke.com")
assert tenant_id == "vocalstar"
def test_extract_tenant_from_host_with_port(self, middleware, mock_tenant_service):
"""Test extracting tenant when host includes port."""
mock_tenant_service.tenant_exists.return_value = True
tenant_id = middleware._extract_tenant_from_host("vocalstar.nomadkaraoke.com:443")
assert tenant_id == "vocalstar"
def test_extract_tenant_from_host_case_insensitive(self, middleware, mock_tenant_service):
"""Test host parsing is case insensitive."""
mock_tenant_service.tenant_exists.return_value = True
tenant_id = middleware._extract_tenant_from_host("VOCALSTAR.NomadKaraoke.COM")
assert tenant_id == "vocalstar"
def test_extract_tenant_from_host_empty(self, middleware):
"""Test returns None for empty host."""
tenant_id = middleware._extract_tenant_from_host("")
assert tenant_id is None
def test_extract_tenant_from_host_localhost(self, middleware):
"""Test returns None for localhost."""
tenant_id = middleware._extract_tenant_from_host("localhost:3000")
assert tenant_id is None
def test_extract_tenant_from_host_non_nomad_domain(self, middleware):
"""Test returns None for non-nomadkaraoke.com domains."""
tenant_id = middleware._extract_tenant_from_host("example.com")
assert tenant_id is None
def test_extract_tenant_from_host_base_domain(self, middleware):
"""Test returns None for nomadkaraoke.com without subdomain."""
tenant_id = middleware._extract_tenant_from_host("nomadkaraoke.com")
assert tenant_id is None
@pytest.mark.parametrize("subdomain", NON_TENANT_SUBDOMAINS)
def test_extract_tenant_from_host_non_tenant_subdomains(self, middleware, subdomain):
"""Test returns None for known non-tenant subdomains."""
host = f"{subdomain}.nomadkaraoke.com"
tenant_id = middleware._extract_tenant_from_host(host)
assert tenant_id is None
def test_extract_tenant_from_host_tenant_not_exists(self, middleware, mock_tenant_service):
"""Test returns None when tenant doesn't exist in GCS."""
mock_tenant_service.tenant_exists.return_value = False
tenant_id = middleware._extract_tenant_from_host("nonexistent.nomadkaraoke.com")
assert tenant_id is None
# Tests for _extract_tenant_id()
def test_extract_tenant_id_from_header(self, middleware):
"""Test X-Tenant-ID header takes priority."""
request = MockRequest(
headers={"X-Tenant-ID": "vocalstar", "Host": "other.nomadkaraoke.com"},
query_params={"tenant": "different"},
)
tenant_id = middleware._extract_tenant_id(request)
assert tenant_id == "vocalstar"
def test_extract_tenant_id_header_normalized(self, middleware):
"""Test header value is normalized (lowercased, stripped)."""
request = MockRequest(headers={"X-Tenant-ID": " VOCALSTAR "})
tenant_id = middleware._extract_tenant_id(request)
assert tenant_id == "vocalstar"
@patch.dict(os.environ, {"ENV": ""})
def test_extract_tenant_id_from_query_param_dev(self, middleware):
"""Test query param works in non-production."""
request = MockRequest(query_params={"tenant": "vocalstar"})
tenant_id = middleware._extract_tenant_id(request)
assert tenant_id == "vocalstar"
@patch.dict(os.environ, {"ENV": "production"})
def test_extract_tenant_id_query_param_disabled_in_prod(self, middleware, mock_tenant_service):
"""Test query param is ignored in production."""
# Need to reimport to pick up new env value
import importlib
import backend.middleware.tenant as tenant_module
# Save original value
original_is_prod = tenant_module.IS_PRODUCTION
# Temporarily set to True for this test
tenant_module.IS_PRODUCTION = True
try:
mock_tenant_service.tenant_exists.return_value = False
request = MockRequest(
query_params={"tenant": "vocalstar"},
headers={"Host": "gen.nomadkaraoke.com"},
)
tenant_id = middleware._extract_tenant_id(request)
# Should not use query param, and gen subdomain is non-tenant
assert tenant_id is None
finally:
# Restore
tenant_module.IS_PRODUCTION = original_is_prod
def test_extract_tenant_id_from_host(self, middleware, mock_tenant_service):
"""Test falls back to host header subdomain detection."""
mock_tenant_service.tenant_exists.return_value = True
request = MockRequest(headers={"Host": "vocalstar.nomadkaraoke.com"})
tenant_id = middleware._extract_tenant_id(request)
assert tenant_id == "vocalstar"
def test_extract_tenant_id_no_tenant(self, middleware, mock_tenant_service):
"""Test returns None when no tenant detected."""
mock_tenant_service.tenant_exists.return_value = False
request = MockRequest(headers={"Host": "gen.nomadkaraoke.com"})
tenant_id = middleware._extract_tenant_id(request)
assert tenant_id is None
# Tests for dispatch()
@pytest.mark.asyncio
async def test_dispatch_attaches_tenant_to_state(self, middleware, mock_tenant_service):
"""Test middleware attaches tenant info to request.state."""
mock_tenant_service.get_tenant_config.return_value = SAMPLE_CONFIG
mock_tenant_service.tenant_exists.return_value = True
request = MockRequest(headers={"X-Tenant-ID": "vocalstar"})
response = Response(content="OK")
call_next = AsyncMock(return_value=response)
result = await middleware.dispatch(request, call_next)
assert request.state.tenant_id == "vocalstar"
assert request.state.tenant_config == SAMPLE_CONFIG
@pytest.mark.asyncio
async def test_dispatch_adds_header_to_response(self, middleware, mock_tenant_service):
"""Test middleware adds X-Tenant-ID header to response."""
mock_tenant_service.get_tenant_config.return_value = SAMPLE_CONFIG
mock_tenant_service.tenant_exists.return_value = True
request = MockRequest(headers={"X-Tenant-ID": "vocalstar"})
response = Response(content="OK")
call_next = AsyncMock(return_value=response)
result = await middleware.dispatch(request, call_next)
assert result.headers.get("X-Tenant-ID") == "vocalstar"
@pytest.mark.asyncio
async def test_dispatch_no_tenant(self, middleware, mock_tenant_service):
"""Test middleware handles no tenant gracefully."""
mock_tenant_service.tenant_exists.return_value = False
request = MockRequest(headers={"Host": "gen.nomadkaraoke.com"})
response = Response(content="OK")
call_next = AsyncMock(return_value=response)
result = await middleware.dispatch(request, call_next)
assert request.state.tenant_id is None
assert request.state.tenant_config is None
assert "X-Tenant-ID" not in result.headers
@pytest.mark.asyncio
async def test_dispatch_inactive_tenant(self, middleware, mock_tenant_service):
"""Test middleware treats inactive tenant as no tenant."""
mock_tenant_service.get_tenant_config.return_value = INACTIVE_CONFIG
request = MockRequest(headers={"X-Tenant-ID": "inactive"})
response = Response(content="OK")
call_next = AsyncMock(return_value=response)
result = await middleware.dispatch(request, call_next)
# Inactive tenant should be treated as default
assert request.state.tenant_id is None
assert request.state.tenant_config is None
@pytest.mark.asyncio
async def test_dispatch_calls_next(self, middleware, mock_tenant_service):
"""Test middleware calls the next handler."""
mock_tenant_service.tenant_exists.return_value = False
request = MockRequest(headers={"Host": "gen.nomadkaraoke.com"})
response = Response(content="OK")
call_next = AsyncMock(return_value=response)
await middleware.dispatch(request, call_next)
call_next.assert_called_once_with(request)
class TestHelperFunctions:
"""Tests for helper functions."""
def test_get_tenant_from_request(self):
"""Test get_tenant_from_request helper."""
request = MagicMock()
request.state.tenant_id = "vocalstar"
assert get_tenant_from_request(request) == "vocalstar"
def test_get_tenant_from_request_none(self):
"""Test get_tenant_from_request when no tenant."""
request = MagicMock()
del request.state.tenant_id # Simulate missing attribute
assert get_tenant_from_request(request) is None
def test_get_tenant_config_from_request(self):
"""Test get_tenant_config_from_request helper."""
request = MagicMock()
request.state.tenant_config = SAMPLE_CONFIG
config = get_tenant_config_from_request(request)
assert config == SAMPLE_CONFIG
assert config.features.audio_search is False
def test_get_tenant_config_from_request_none(self):
"""Test get_tenant_config_from_request when no config."""
request = MagicMock()
del request.state.tenant_config
assert get_tenant_config_from_request(request) is None
class TestNonTenantSubdomains:
"""Tests for NON_TENANT_SUBDOMAINS constant."""
def test_gen_is_non_tenant(self):
"""Test 'gen' is in non-tenant subdomains."""
assert "gen" in NON_TENANT_SUBDOMAINS
def test_api_is_non_tenant(self):
"""Test 'api' is in non-tenant subdomains."""
assert "api" in NON_TENANT_SUBDOMAINS
def test_www_is_non_tenant(self):
"""Test 'www' is in non-tenant subdomains."""
assert "www" in NON_TENANT_SUBDOMAINS
def test_admin_is_non_tenant(self):
"""Test 'admin' is in non-tenant subdomains."""
assert "admin" in NON_TENANT_SUBDOMAINS
+406
View File
@@ -0,0 +1,406 @@
"""
Unit tests for tenant data models.
Tests the TenantConfig, TenantBranding, TenantFeatures, TenantDefaults,
TenantAuth, and TenantPublicConfig models.
"""
import pytest
from datetime import datetime
from backend.models.tenant import (
TenantConfig,
TenantBranding,
TenantFeatures,
TenantDefaults,
TenantAuth,
TenantPublicConfig,
TenantConfigResponse,
)
class TestTenantBranding:
"""Tests for TenantBranding model."""
def test_default_values(self):
"""Test that TenantBranding has sensible defaults."""
branding = TenantBranding()
assert branding.logo_url is None
assert branding.logo_height == 40
assert branding.primary_color == "#ff5bb8"
assert branding.secondary_color == "#8b5cf6"
assert branding.accent_color is None
assert branding.background_color is None
assert branding.favicon_url is None
assert branding.site_title == "Karaoke Generator"
assert branding.tagline is None
def test_custom_values(self):
"""Test TenantBranding with custom values."""
branding = TenantBranding(
logo_url="https://example.com/logo.png",
logo_height=60,
primary_color="#ffff00",
secondary_color="#0000ff",
accent_color="#ff0000",
background_color="#000000",
favicon_url="https://example.com/favicon.ico",
site_title="Custom Karaoke",
tagline="Make music magic",
)
assert branding.logo_url == "https://example.com/logo.png"
assert branding.logo_height == 60
assert branding.primary_color == "#ffff00"
assert branding.site_title == "Custom Karaoke"
class TestTenantFeatures:
"""Tests for TenantFeatures model."""
def test_default_features_enabled(self):
"""Test that most features are enabled by default."""
features = TenantFeatures()
# Input methods
assert features.audio_search is True
assert features.file_upload is True
assert features.youtube_url is True
# Distribution
assert features.youtube_upload is True
assert features.dropbox_upload is True
assert features.gdrive_upload is True
# Customization
assert features.theme_selection is True
assert features.color_overrides is True
# Output formats
assert features.enable_cdg is True
assert features.enable_4k is True
# Admin is disabled by default
assert features.admin_access is False
def test_restricted_features(self):
"""Test a restricted feature set like Vocal Star would have."""
features = TenantFeatures(
audio_search=False,
youtube_url=False,
youtube_upload=False,
dropbox_upload=False,
gdrive_upload=False,
theme_selection=False,
color_overrides=False,
)
assert features.audio_search is False
assert features.file_upload is True # Still allowed
assert features.youtube_url is False
assert features.youtube_upload is False
class TestTenantDefaults:
"""Tests for TenantDefaults model."""
def test_default_values(self):
"""Test TenantDefaults has sensible defaults."""
defaults = TenantDefaults()
assert defaults.theme_id is None
assert defaults.locked_theme is None
assert defaults.distribution_mode == "all"
assert defaults.brand_prefix is None
assert defaults.youtube_description_template is None
def test_locked_theme(self):
"""Test locked_theme setting."""
defaults = TenantDefaults(
theme_id="vocalstar",
locked_theme="vocalstar",
distribution_mode="download_only",
brand_prefix="VSTAR",
)
assert defaults.locked_theme == "vocalstar"
assert defaults.distribution_mode == "download_only"
assert defaults.brand_prefix == "VSTAR"
class TestTenantAuth:
"""Tests for TenantAuth model."""
def test_default_values(self):
"""Test TenantAuth defaults."""
auth = TenantAuth()
assert auth.allowed_email_domains == []
assert auth.require_email_domain is True
assert auth.fixed_token_ids == []
assert auth.sender_email is None
def test_restricted_domains(self):
"""Test restricted email domains configuration."""
auth = TenantAuth(
allowed_email_domains=["vocal-star.com", "vocalstarmusic.com"],
require_email_domain=True,
sender_email="vocalstar@nomadkaraoke.com",
)
assert "vocal-star.com" in auth.allowed_email_domains
assert auth.require_email_domain is True
assert auth.sender_email == "vocalstar@nomadkaraoke.com"
class TestTenantConfig:
"""Tests for TenantConfig model."""
@pytest.fixture
def basic_config(self):
"""Create a basic tenant config for testing."""
return TenantConfig(
id="vocalstar",
name="Vocal Star",
subdomain="vocalstar.nomadkaraoke.com",
)
@pytest.fixture
def full_config(self):
"""Create a fully configured tenant."""
return TenantConfig(
id="vocalstar",
name="Vocal Star",
subdomain="vocalstar.nomadkaraoke.com",
is_active=True,
branding=TenantBranding(
logo_url="https://example.com/logo.png",
primary_color="#ffff00",
secondary_color="#006CF9",
site_title="Vocal Star Karaoke Generator",
),
features=TenantFeatures(
audio_search=False,
youtube_url=False,
youtube_upload=False,
dropbox_upload=False,
gdrive_upload=False,
theme_selection=False,
),
defaults=TenantDefaults(
theme_id="vocalstar",
locked_theme="vocalstar",
distribution_mode="download_only",
),
auth=TenantAuth(
allowed_email_domains=["vocal-star.com", "vocalstarmusic.com"],
require_email_domain=True,
sender_email="vocalstar@nomadkaraoke.com",
),
)
def test_required_fields(self):
"""Test that id, name, and subdomain are required."""
config = TenantConfig(
id="test",
name="Test Tenant",
subdomain="test.nomadkaraoke.com",
)
assert config.id == "test"
assert config.name == "Test Tenant"
assert config.subdomain == "test.nomadkaraoke.com"
def test_default_is_active(self, basic_config):
"""Test that tenants are active by default."""
assert basic_config.is_active is True
def test_nested_models_have_defaults(self, basic_config):
"""Test that nested models are created with defaults."""
assert basic_config.branding is not None
assert basic_config.features is not None
assert basic_config.defaults is not None
assert basic_config.auth is not None
# Tests for get_sender_email()
def test_get_sender_email_custom(self, full_config):
"""Test get_sender_email returns custom sender when configured."""
assert full_config.get_sender_email() == "vocalstar@nomadkaraoke.com"
def test_get_sender_email_default_pattern(self, basic_config):
"""Test get_sender_email returns default pattern when not configured."""
# basic_config has no custom sender_email
assert basic_config.get_sender_email() == "vocalstar@nomadkaraoke.com"
def test_get_sender_email_different_tenant(self):
"""Test sender email follows tenant ID pattern."""
config = TenantConfig(
id="customtenant",
name="Custom",
subdomain="custom.nomadkaraoke.com",
)
assert config.get_sender_email() == "customtenant@nomadkaraoke.com"
# Tests for is_email_allowed()
def test_is_email_allowed_matching_domain(self, full_config):
"""Test email is allowed when domain matches."""
assert full_config.is_email_allowed("user@vocal-star.com") is True
assert full_config.is_email_allowed("user@vocalstarmusic.com") is True
def test_is_email_allowed_non_matching_domain(self, full_config):
"""Test email is rejected when domain doesn't match and require_email_domain=True."""
assert full_config.is_email_allowed("user@gmail.com") is False
assert full_config.is_email_allowed("user@other.com") is False
def test_is_email_allowed_case_insensitive(self, full_config):
"""Test email domain matching is case insensitive."""
assert full_config.is_email_allowed("User@VOCAL-STAR.COM") is True
assert full_config.is_email_allowed("User@VocalStarMusic.com") is True
def test_is_email_allowed_no_domain_restrictions(self, basic_config):
"""Test any email allowed when no domain restrictions."""
# basic_config has empty allowed_email_domains
assert basic_config.is_email_allowed("anyone@gmail.com") is True
assert basic_config.is_email_allowed("user@anything.com") is True
def test_is_email_allowed_require_domain_false(self):
"""Test non-matching emails allowed when require_email_domain=False."""
config = TenantConfig(
id="flexible",
name="Flexible Tenant",
subdomain="flexible.nomadkaraoke.com",
auth=TenantAuth(
allowed_email_domains=["preferred.com"],
require_email_domain=False, # Don't require matching
),
)
# Matching domain still works
assert config.is_email_allowed("user@preferred.com") is True
# Non-matching also allowed since require_email_domain=False
assert config.is_email_allowed("user@other.com") is True
def test_is_email_allowed_partial_domain_no_match(self, full_config):
"""Test partial domain matches don't work (must be exact suffix)."""
# "star.com" should not match "vocal-star.com"
assert full_config.is_email_allowed("user@star.com") is False
# Subdomain of allowed domain should work
assert full_config.is_email_allowed("user@sub.vocal-star.com") is False
class TestTenantPublicConfig:
"""Tests for TenantPublicConfig model."""
@pytest.fixture
def full_config(self):
"""Create a fully configured tenant for conversion testing."""
return TenantConfig(
id="vocalstar",
name="Vocal Star",
subdomain="vocalstar.nomadkaraoke.com",
is_active=True,
branding=TenantBranding(
logo_url="https://example.com/logo.png",
primary_color="#ffff00",
),
features=TenantFeatures(
audio_search=False,
admin_access=True, # This should be included in public config
),
defaults=TenantDefaults(
theme_id="vocalstar",
locked_theme="vocalstar",
distribution_mode="download_only",
brand_prefix="VSTAR", # This should NOT be in public config
youtube_description_template="Custom template", # This should NOT be in public config
),
auth=TenantAuth(
allowed_email_domains=["vocal-star.com"],
require_email_domain=True,
fixed_token_ids=["secret-token-123"], # This should NOT be in public config
sender_email="vocalstar@nomadkaraoke.com", # This should NOT be in public config
),
)
def test_from_config_basic_fields(self, full_config):
"""Test that basic fields are copied correctly."""
public = TenantPublicConfig.from_config(full_config)
assert public.id == "vocalstar"
assert public.name == "Vocal Star"
assert public.subdomain == "vocalstar.nomadkaraoke.com"
assert public.is_active is True
def test_from_config_branding_included(self, full_config):
"""Test that branding is fully included."""
public = TenantPublicConfig.from_config(full_config)
assert public.branding.logo_url == "https://example.com/logo.png"
assert public.branding.primary_color == "#ffff00"
def test_from_config_features_included(self, full_config):
"""Test that features are fully included."""
public = TenantPublicConfig.from_config(full_config)
assert public.features.audio_search is False
assert public.features.admin_access is True
def test_from_config_allowed_email_domains_included(self, full_config):
"""Test that allowed_email_domains is included for frontend validation."""
public = TenantPublicConfig.from_config(full_config)
assert "vocal-star.com" in public.allowed_email_domains
def test_from_config_sensitive_auth_excluded(self, full_config):
"""Test that sensitive auth fields are not in public config."""
public = TenantPublicConfig.from_config(full_config)
# TenantPublicConfig only has allowed_email_domains from auth
# It should NOT have fixed_token_ids or sender_email
assert not hasattr(public, 'auth')
# allowed_email_domains is a top-level field
assert hasattr(public, 'allowed_email_domains')
def test_from_config_defaults_partially_included(self, full_config):
"""Test that only safe defaults are included."""
public = TenantPublicConfig.from_config(full_config)
# These should be included
assert public.defaults.theme_id == "vocalstar"
assert public.defaults.locked_theme == "vocalstar"
assert public.defaults.distribution_mode == "download_only"
# brand_prefix and youtube_description_template should NOT be included
# The from_config method creates a new TenantDefaults without these
assert public.defaults.brand_prefix is None
assert public.defaults.youtube_description_template is None
class TestTenantConfigResponse:
"""Tests for TenantConfigResponse model."""
def test_default_response(self):
"""Test default response indicates no tenant."""
response = TenantConfigResponse()
assert response.tenant is None
assert response.is_default is True
def test_tenant_response(self):
"""Test response with tenant config."""
public_config = TenantPublicConfig(
id="vocalstar",
name="Vocal Star",
subdomain="vocalstar.nomadkaraoke.com",
is_active=True,
branding=TenantBranding(),
features=TenantFeatures(),
defaults=TenantDefaults(),
)
response = TenantConfigResponse(tenant=public_config, is_default=False)
assert response.tenant is not None
assert response.tenant.id == "vocalstar"
assert response.is_default is False
+418
View File
@@ -0,0 +1,418 @@
"""
Unit tests for TenantService.
Tests config loading from GCS, caching behavior, subdomain resolution,
and email domain validation.
"""
import pytest
from datetime import datetime, timedelta
from unittest.mock import MagicMock, patch
from backend.services.tenant_service import TenantService, get_tenant_service
from backend.models.tenant import TenantConfig, TenantPublicConfig
# Sample tenant config data that would come from GCS
SAMPLE_VOCALSTAR_CONFIG = {
"id": "vocalstar",
"name": "Vocal Star",
"subdomain": "vocalstar.nomadkaraoke.com",
"is_active": True,
"branding": {
"logo_url": "https://example.com/logo.png",
"logo_height": 50,
"primary_color": "#ffff00",
"secondary_color": "#006CF9",
"site_title": "Vocal Star Karaoke Generator",
},
"features": {
"audio_search": False,
"file_upload": True,
"youtube_url": False,
"youtube_upload": False,
"dropbox_upload": False,
"gdrive_upload": False,
"theme_selection": False,
},
"defaults": {
"theme_id": "vocalstar",
"locked_theme": "vocalstar",
"distribution_mode": "download_only",
},
"auth": {
"allowed_email_domains": ["vocal-star.com", "vocalstarmusic.com"],
"require_email_domain": True,
"sender_email": "vocalstar@nomadkaraoke.com",
},
}
SAMPLE_INACTIVE_CONFIG = {
"id": "inactive",
"name": "Inactive Tenant",
"subdomain": "inactive.nomadkaraoke.com",
"is_active": False,
"branding": {},
"features": {},
"defaults": {},
"auth": {},
}
class TestTenantService:
"""Tests for TenantService class."""
@pytest.fixture
def mock_storage(self):
"""Create a mock StorageService."""
storage = MagicMock()
storage.file_exists.return_value = False
storage.download_json.return_value = {}
storage.generate_signed_url.return_value = "https://signed-url.example.com"
return storage
@pytest.fixture
def tenant_service(self, mock_storage):
"""Create a TenantService with mock storage."""
return TenantService(storage=mock_storage)
# Tests for get_tenant_config()
def test_get_tenant_config_success(self, tenant_service, mock_storage):
"""Test successful config loading from GCS."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
config = tenant_service.get_tenant_config("vocalstar")
assert config is not None
assert config.id == "vocalstar"
assert config.name == "Vocal Star"
assert config.features.audio_search is False
mock_storage.download_json.assert_called_once_with("tenants/vocalstar/config.json")
def test_get_tenant_config_not_found(self, tenant_service, mock_storage):
"""Test returns None when tenant config doesn't exist."""
mock_storage.file_exists.return_value = False
config = tenant_service.get_tenant_config("nonexistent")
assert config is None
mock_storage.download_json.assert_not_called()
def test_get_tenant_config_gcs_error(self, tenant_service, mock_storage):
"""Test handles GCS errors gracefully."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.side_effect = Exception("GCS error")
config = tenant_service.get_tenant_config("vocalstar")
assert config is None
# Tests for caching behavior
def test_config_is_cached(self, tenant_service, mock_storage):
"""Test that configs are cached after first load."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
# First call loads from GCS
config1 = tenant_service.get_tenant_config("vocalstar")
# Second call should use cache
config2 = tenant_service.get_tenant_config("vocalstar")
assert config1 is config2
# download_json should only be called once
assert mock_storage.download_json.call_count == 1
def test_force_refresh_bypasses_cache(self, tenant_service, mock_storage):
"""Test that force_refresh reloads from GCS."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
# First call
tenant_service.get_tenant_config("vocalstar")
# Force refresh
tenant_service.get_tenant_config("vocalstar", force_refresh=True)
# download_json should be called twice
assert mock_storage.download_json.call_count == 2
def test_cache_expires_after_ttl(self, tenant_service, mock_storage):
"""Test that cache expires after TTL."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
# Load config
tenant_service.get_tenant_config("vocalstar")
# Simulate cache expiration by manipulating cache time
tenant_service._cache_times["vocalstar"] = datetime.now() - timedelta(seconds=400)
# Next call should reload
tenant_service.get_tenant_config("vocalstar")
assert mock_storage.download_json.call_count == 2
# Tests for get_tenant_by_subdomain()
def test_get_tenant_by_subdomain_cached(self, tenant_service, mock_storage):
"""Test subdomain lookup uses cache."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
# First load by ID to populate cache
tenant_service.get_tenant_config("vocalstar")
# Now lookup by subdomain should use cached mapping
config = tenant_service.get_tenant_by_subdomain("vocalstar.nomadkaraoke.com")
assert config is not None
assert config.id == "vocalstar"
def test_get_tenant_by_subdomain_fallback_resolution(self, tenant_service, mock_storage):
"""Test subdomain resolution when not in cache."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
# Lookup by subdomain without prior cache
config = tenant_service.get_tenant_by_subdomain("vocalstar.nomadkaraoke.com")
assert config is not None
assert config.id == "vocalstar"
def test_get_tenant_by_subdomain_not_found(self, tenant_service, mock_storage):
"""Test returns None for unknown subdomain."""
mock_storage.file_exists.return_value = False
config = tenant_service.get_tenant_by_subdomain("unknown.nomadkaraoke.com")
assert config is None
def test_get_tenant_by_subdomain_case_insensitive(self, tenant_service, mock_storage):
"""Test subdomain lookup is case insensitive."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
config = tenant_service.get_tenant_by_subdomain("VOCALSTAR.NomadKaraoke.COM")
assert config is not None
assert config.id == "vocalstar"
# Tests for _resolve_subdomain_to_tenant_id()
def test_resolve_subdomain_3_parts(self, tenant_service, mock_storage):
"""Test resolving tenant.nomadkaraoke.com pattern."""
mock_storage.file_exists.return_value = True
tenant_id = tenant_service._resolve_subdomain_to_tenant_id("vocalstar.nomadkaraoke.com")
assert tenant_id == "vocalstar"
def test_resolve_subdomain_4_parts(self, tenant_service, mock_storage):
"""Test resolving tenant.gen.nomadkaraoke.com pattern."""
mock_storage.file_exists.return_value = True
tenant_id = tenant_service._resolve_subdomain_to_tenant_id("vocalstar.gen.nomadkaraoke.com")
assert tenant_id == "vocalstar"
def test_resolve_subdomain_too_short(self, tenant_service, mock_storage):
"""Test returns None for subdomains with fewer than 3 parts."""
tenant_id = tenant_service._resolve_subdomain_to_tenant_id("nomadkaraoke.com")
assert tenant_id is None
def test_resolve_subdomain_tenant_not_exists(self, tenant_service, mock_storage):
"""Test returns None when tenant config doesn't exist in GCS."""
mock_storage.file_exists.return_value = False
tenant_id = tenant_service._resolve_subdomain_to_tenant_id("nonexistent.nomadkaraoke.com")
assert tenant_id is None
# Tests for get_public_config()
def test_get_public_config_success(self, tenant_service, mock_storage):
"""Test getting public config."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
public_config = tenant_service.get_public_config("vocalstar")
assert public_config is not None
assert isinstance(public_config, TenantPublicConfig)
assert public_config.id == "vocalstar"
def test_get_public_config_not_found(self, tenant_service, mock_storage):
"""Test returns None when tenant not found."""
mock_storage.file_exists.return_value = False
public_config = tenant_service.get_public_config("nonexistent")
assert public_config is None
def test_get_public_config_by_subdomain(self, tenant_service, mock_storage):
"""Test getting public config by subdomain."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
public_config = tenant_service.get_public_config_by_subdomain("vocalstar.nomadkaraoke.com")
assert public_config is not None
assert public_config.id == "vocalstar"
# Tests for tenant_exists()
def test_tenant_exists_true(self, tenant_service, mock_storage):
"""Test returns True when config exists."""
mock_storage.file_exists.return_value = True
assert tenant_service.tenant_exists("vocalstar") is True
mock_storage.file_exists.assert_called_with("tenants/vocalstar/config.json")
def test_tenant_exists_false(self, tenant_service, mock_storage):
"""Test returns False when config doesn't exist."""
mock_storage.file_exists.return_value = False
assert tenant_service.tenant_exists("nonexistent") is False
# Tests for is_email_allowed_for_tenant()
def test_is_email_allowed_for_tenant_valid(self, tenant_service, mock_storage):
"""Test email allowed for valid domain."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
assert tenant_service.is_email_allowed_for_tenant("vocalstar", "user@vocal-star.com") is True
def test_is_email_allowed_for_tenant_invalid(self, tenant_service, mock_storage):
"""Test email not allowed for invalid domain."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
assert tenant_service.is_email_allowed_for_tenant("vocalstar", "user@gmail.com") is False
def test_is_email_allowed_for_tenant_not_found(self, tenant_service, mock_storage):
"""Test returns False when tenant not found."""
mock_storage.file_exists.return_value = False
assert tenant_service.is_email_allowed_for_tenant("nonexistent", "user@any.com") is False
# Tests for get_tenant_sender_email()
def test_get_tenant_sender_email_custom(self, tenant_service, mock_storage):
"""Test returns custom sender email when configured."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
sender = tenant_service.get_tenant_sender_email("vocalstar")
assert sender == "vocalstar@nomadkaraoke.com"
def test_get_tenant_sender_email_default(self, tenant_service, mock_storage):
"""Test returns default sender when tenant not found."""
mock_storage.file_exists.return_value = False
sender = tenant_service.get_tenant_sender_email("nonexistent")
# Should match DEFAULT_SENDER_EMAIL constant (consistent with EmailService)
assert sender == "gen@nomadkaraoke.com"
# Tests for invalidate_cache()
def test_invalidate_cache_specific_tenant(self, tenant_service, mock_storage):
"""Test invalidating cache for specific tenant."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
# Load into cache
tenant_service.get_tenant_config("vocalstar")
assert "vocalstar" in tenant_service._config_cache
# Invalidate
tenant_service.invalidate_cache("vocalstar")
assert "vocalstar" not in tenant_service._config_cache
assert "vocalstar" not in tenant_service._cache_times
def test_invalidate_cache_specific_tenant_clears_subdomain_map(self, tenant_service, mock_storage):
"""Test invalidating specific tenant also clears its subdomain map entry."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
# Load into cache (this also populates subdomain map)
tenant_service.get_tenant_config("vocalstar")
assert "vocalstar.nomadkaraoke.com" in tenant_service._subdomain_map
assert tenant_service._subdomain_map["vocalstar.nomadkaraoke.com"] == "vocalstar"
# Invalidate specific tenant
tenant_service.invalidate_cache("vocalstar")
# Subdomain map entry should also be removed
assert "vocalstar.nomadkaraoke.com" not in tenant_service._subdomain_map
def test_invalidate_cache_all(self, tenant_service, mock_storage):
"""Test invalidating all caches."""
mock_storage.file_exists.return_value = True
mock_storage.download_json.return_value = SAMPLE_VOCALSTAR_CONFIG
# Load into cache
tenant_service.get_tenant_config("vocalstar")
# Invalidate all
tenant_service.invalidate_cache()
assert len(tenant_service._config_cache) == 0
assert len(tenant_service._cache_times) == 0
assert len(tenant_service._subdomain_map) == 0
# Tests for get_asset_url()
def test_get_asset_url_success(self, tenant_service, mock_storage):
"""Test getting signed URL for existing asset."""
mock_storage.file_exists.return_value = True
url = tenant_service.get_asset_url("vocalstar", "logo.png")
assert url == "https://signed-url.example.com"
mock_storage.file_exists.assert_called_with("tenants/vocalstar/logo.png")
mock_storage.generate_signed_url.assert_called_with("tenants/vocalstar/logo.png", expiration_minutes=60)
def test_get_asset_url_not_found(self, tenant_service, mock_storage):
"""Test returns None when asset doesn't exist."""
mock_storage.file_exists.return_value = False
url = tenant_service.get_asset_url("vocalstar", "missing.png")
assert url is None
mock_storage.generate_signed_url.assert_not_called()
class TestGetTenantServiceSingleton:
"""Tests for the get_tenant_service() singleton function."""
def test_returns_same_instance(self):
"""Test that get_tenant_service returns singleton."""
# Reset the singleton
import backend.services.tenant_service as module
module._tenant_service = None
with patch.object(TenantService, '__init__', return_value=None):
service1 = get_tenant_service()
service2 = get_tenant_service()
assert service1 is service2
def test_thread_safe_initialization(self):
"""Test that singleton initialization is thread-safe."""
import threading
import backend.services.tenant_service as module
# Reset the singleton
module._tenant_service = None
instances = []
def get_instance():
with patch.object(TenantService, '__init__', return_value=None):
instances.append(get_tenant_service())
threads = [threading.Thread(target=get_instance) for _ in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
# All instances should be the same
assert all(i is instances[0] for i in instances)
+522
View File
@@ -0,0 +1,522 @@
"""
Comprehensive tests for Unicode and special character handling.
This test file verifies that artist/title with Unicode characters (curly quotes,
em dashes, non-ASCII characters) are properly sanitized throughout the codebase
to prevent:
1. HTTP header encoding failures (Content-Disposition, email subjects)
2. API query failures (Google Drive)
3. Filename encoding issues (Modal API, filesystem)
The root cause was job d49efab1 which had title "Mama Says (You Can't Back Down)"
with a curly apostrophe (U+2019) that caused the audio separation to fail because
the Modal API couldn't handle the non-latin-1 character in HTTP headers.
"""
import pytest
from karaoke_gen.utils import (
sanitize_filename,
normalize_text,
UNICODE_REPLACEMENTS,
TEXT_NORMALIZATIONS,
APOSTROPHE_REPLACEMENTS,
DOUBLE_QUOTE_REPLACEMENTS,
DASH_REPLACEMENTS,
WHITESPACE_REPLACEMENTS,
)
class TestSanitizeFilename:
"""Test the sanitize_filename function handles all edge cases."""
def test_curly_single_quotes(self):
"""Test that curly single quotes are converted to straight quotes."""
# The exact character that caused job d49efab1 to fail
assert sanitize_filename("Can't") == "Can't"
assert sanitize_filename("It's") == "It's"
# Left single quote
assert sanitize_filename("'Hello'") == "'Hello'"
def test_curly_double_quotes(self):
"""Test that curly double quotes are converted to underscores.
Note: Double quotes are filesystem-unsafe so they become underscores."""
# Curly quotes first become straight, then straight quotes become underscore
assert sanitize_filename("\u201cHello\u201d") == "_Hello_"
def test_em_dash(self):
"""Test that em dashes are converted to regular hyphens."""
assert sanitize_filename("Artist — Title") == "Artist - Title"
assert sanitize_filename("Song—Name") == "Song-Name"
def test_en_dash(self):
"""Test that en dashes are converted to regular hyphens."""
assert sanitize_filename("1990–2000") == "1990-2000"
def test_ellipsis(self):
"""Test that horizontal ellipsis is converted to three dots."""
# Note: The ellipsis becomes "..." but trailing dots are stripped
assert sanitize_filename("Wait\u2026") == "Wait"
# But in the middle of a string they're preserved
assert sanitize_filename("Wait\u2026Here") == "Wait...Here"
def test_non_breaking_space(self):
"""Test that non-breaking spaces are converted to regular spaces."""
# U+00A0 is non-breaking space
assert sanitize_filename("Hello\u00A0World") == "Hello World"
def test_filesystem_unsafe_characters(self):
"""Test that filesystem-unsafe characters are replaced."""
assert sanitize_filename("file/name") == "file_name"
assert sanitize_filename("file\\name") == "file_name"
assert sanitize_filename("file:name") == "file_name"
assert sanitize_filename("file*name") == "file_name"
assert sanitize_filename("file?name") == "file_name"
assert sanitize_filename('file"name') == "file_name"
assert sanitize_filename("file<name") == "file_name"
assert sanitize_filename("file>name") == "file_name"
assert sanitize_filename("file|name") == "file_name"
def test_trailing_periods_and_spaces(self):
"""Test that trailing periods and spaces are removed."""
assert sanitize_filename("filename.") == "filename"
assert sanitize_filename("filename...") == "filename"
assert sanitize_filename("filename ") == "filename"
assert sanitize_filename("filename . ") == "filename"
def test_leading_periods_and_spaces(self):
"""Test that leading periods and spaces are removed."""
assert sanitize_filename(".filename") == "filename"
assert sanitize_filename("...filename") == "filename"
assert sanitize_filename(" filename") == "filename"
assert sanitize_filename(" . filename") == "filename"
def test_multiple_underscores_collapsed(self):
"""Test that multiple consecutive underscores are collapsed to one."""
assert sanitize_filename("file___name") == "file_name"
# Multiple unsafe chars in a row
assert sanitize_filename("file?*:name") == "file_name"
def test_multiple_spaces_collapsed(self):
"""Test that multiple consecutive spaces are collapsed to one."""
assert sanitize_filename("file name") == "file name"
def test_none_input(self):
"""Test that None input returns None."""
assert sanitize_filename(None) is None
def test_empty_string(self):
"""Test that empty string returns empty string."""
assert sanitize_filename("") == ""
def test_real_world_examples(self):
"""Test real-world examples that have caused issues."""
# The exact title from job d49efab1
assert sanitize_filename("Mama Says (You Can't Back Down)") == "Mama Says (You Can't Back Down)"
# Broadway cast with smart quotes
assert sanitize_filename("Footloose (Broadway Cast) \u2014 \u201cFinal Song\u201d") == "Footloose (Broadway Cast) - _Final Song_"
# Japanese artist with em dash
assert sanitize_filename("宇多田ヒカル — First Love") == "宇多田ヒカル - First Love"
# Korean title (should pass through unchanged)
assert sanitize_filename("아이유 - 좋은 날") == "아이유 - 좋은 날"
# Mixed content
assert sanitize_filename("L'Arc~en~Ciel - Driver's High") == "L'Arc~en~Ciel - Driver's High"
def test_combination_of_issues(self):
"""Test strings with multiple problematic characters."""
result = sanitize_filename("It\u2019s \u201cMy\u201d Song \u2014 Volume\u20261")
# Curly ' -> ', curly " -> _ (via filesystem check), em dash -> -, ellipsis -> ...
assert result == "It's _My_ Song - Volume...1"
def test_unicode_replacements_dict_complete(self):
"""Verify all expected Unicode characters are in the replacements dict."""
# Curly quotes
assert "\u2018" in UNICODE_REPLACEMENTS # LEFT SINGLE QUOTATION MARK
assert "\u2019" in UNICODE_REPLACEMENTS # RIGHT SINGLE QUOTATION MARK
assert "\u201A" in UNICODE_REPLACEMENTS # SINGLE LOW-9 QUOTATION MARK
assert "\u201B" in UNICODE_REPLACEMENTS # SINGLE HIGH-REVERSED-9 QUOTATION MARK
assert "\u201C" in UNICODE_REPLACEMENTS # LEFT DOUBLE QUOTATION MARK
assert "\u201D" in UNICODE_REPLACEMENTS # RIGHT DOUBLE QUOTATION MARK
assert "\u201E" in UNICODE_REPLACEMENTS # DOUBLE LOW-9 QUOTATION MARK
assert "\u201F" in UNICODE_REPLACEMENTS # DOUBLE HIGH-REVERSED-9 QUOTATION MARK
# Dashes
assert "\u2013" in UNICODE_REPLACEMENTS # EN DASH
assert "\u2014" in UNICODE_REPLACEMENTS # EM DASH
# Other
assert "\u2026" in UNICODE_REPLACEMENTS # HORIZONTAL ELLIPSIS
assert "\u00A0" in UNICODE_REPLACEMENTS # NON-BREAKING SPACE
class TestSanitizationInContext:
"""Test that sanitization is applied correctly in various contexts.
These tests verify that the fixes we applied work in the actual
code paths that use artist/title data.
"""
def test_artist_title_format_for_audio_separation(self):
"""Verify artist-title formatting is sanitized for Modal API."""
# Simulate what audio_worker.py does
artist = "Footloose (Broadway Cast)"
title = "Mama Says (You Can\u2019t Back Down)" # Curly apostrophe U+2019
safe_artist = sanitize_filename(artist) if artist else "Unknown"
safe_title = sanitize_filename(title) if title else "Unknown"
artist_title = f"{safe_artist} - {safe_title}"
# The curly apostrophe should be converted to straight
assert "'" in artist_title # Straight apostrophe
assert "\u2019" not in artist_title # No curly apostrophe (U+2019)
def test_email_subject_safe_for_latin1(self):
"""Verify email subjects can be encoded to latin-1 after sanitization."""
artist = "Café del Mar" # Has accented e
title = "It\u2019s a \u201cBeautiful\u201d Day \u2014 Remix"
safe_artist = sanitize_filename(artist) if artist else None
safe_title = sanitize_filename(title) if title else None
subject = f"{safe_artist} - {safe_title}"
# Accented characters should pass through (they're valid latin-1)
assert "é" in subject
# But curly quotes and em dash should be converted
assert "'" in subject # Straight apostrophe
assert "-" in subject # Regular hyphen
# No curly apostrophe (U+2019)
assert "\u2019" not in subject
def test_content_disposition_safe(self):
"""Verify filenames are safe for HTTP Content-Disposition headers."""
artist = "Artist\u2019s \u201cName\u201d"
title = "Song\u2014Title"
safe_artist = sanitize_filename(artist) if artist else None
safe_title = sanitize_filename(title) if title else None
filename = f"{safe_artist} - {safe_title} (Final Karaoke).mp4"
# Should be able to encode to latin-1 for HTTP headers
try:
filename.encode('latin-1')
can_encode = True
except UnicodeEncodeError:
can_encode = False
# Standard ASCII chars and latin-1 safe accented chars should work
# Note: The test above with 'é' passes latin-1; smart quotes do not
assert can_encode
def test_google_drive_query_safe(self):
"""Verify filenames don't break Google Drive API queries."""
base_name = "Artist\u2019s \u201cSong\u201d"
safe_base_name = sanitize_filename(base_name) if base_name else base_name
filename = f"NOMAD-1234 - {safe_base_name}.mp4"
# Google Drive queries use single quotes - our sanitized string
# should have straight single quotes, not curly
# The query escaping handles straight quotes: '
escaped = filename.replace("'", "\\'")
query = f"name='{escaped}'"
# Should be a valid query string (no curly quotes to break syntax)
assert "\u2019" not in query # No curly single quote (U+2019)
assert "\u2018" not in query # No curly single quote (U+2018)
def test_dropbox_path_safe(self):
"""Verify Dropbox paths don't have problematic characters."""
artist = "Don\u2019t Stop"
title = "Believin\u2019"
safe_artist = sanitize_filename(artist) if artist else "Unknown"
safe_title = sanitize_filename(title) if title else "Unknown"
folder_name = f"NOMAD-1234 - {safe_artist} - {safe_title}"
remote_path = f"/Karaoke/{folder_name}"
# Path should not have curly quotes (U+2018 and U+2019)
assert "\u2018" not in remote_path
assert "\u2019" not in remote_path
# But should have the converted straight apostrophe
assert "'" in remote_path
class TestInternationalCharacters:
"""Test that international/non-ASCII characters are handled properly.
The goal is NOT to strip all non-ASCII, but to convert problematic
Unicode to safe equivalents while preserving legitimate international text.
"""
def test_japanese_preserved(self):
"""Japanese characters should pass through unchanged."""
assert sanitize_filename("宇多田ヒカル") == "宇多田ヒカル"
assert sanitize_filename("君が代") == "君が代"
def test_korean_preserved(self):
"""Korean characters should pass through unchanged."""
assert sanitize_filename("방탄소년단") == "방탄소년단"
assert sanitize_filename("좋은 날") == "좋은 날"
def test_chinese_preserved(self):
"""Chinese characters should pass through unchanged."""
assert sanitize_filename("周杰倫") == "周杰倫"
assert sanitize_filename("青花瓷") == "青花瓷"
def test_cyrillic_preserved(self):
"""Cyrillic characters should pass through unchanged."""
assert sanitize_filename("Тату") == "Тату"
assert sanitize_filename("Не верь, не бойся") == "Не верь, не бойся"
def test_arabic_preserved(self):
"""Arabic characters should pass through unchanged."""
assert sanitize_filename("فيروز") == "فيروز"
def test_accented_latin_preserved(self):
"""Accented Latin characters should pass through unchanged."""
assert sanitize_filename("Café") == "Café"
assert sanitize_filename("Señorita") == "Señorita"
assert sanitize_filename("Naïve") == "Naïve"
assert sanitize_filename("Björk") == "Björk"
def test_mixed_script_preserved(self):
"""Mixed script text should work correctly."""
# Japanese artist, English title
assert sanitize_filename("宇多田ヒカル - First Love") == "宇多田ヒカル - First Love"
# K-pop with English
assert sanitize_filename("BTS 방탄소년단 - Dynamite") == "BTS 방탄소년단 - Dynamite"
class TestEdgeCasesAndRegression:
"""Edge cases and regression tests for specific bugs."""
def test_job_d49efab1_exact_title(self):
"""Exact reproduction of job d49efab1 failure case.
The job had:
- artist: "Footloose (Broadway Cast)" (from display_artist override)
- title: "Mama Says (You Can't Back Down)" (with curly apostrophe U+2019)
This caused Stage 2 audio separation to fail because the filename
with the curly apostrophe couldn't be encoded in HTTP headers.
"""
artist = "Footloose (Broadway Cast)"
title = "Mama Says (You Can\u2019t Back Down)" # Explicit U+2019
safe_artist = sanitize_filename(artist)
safe_title = sanitize_filename(title)
artist_title = f"{safe_artist} - {safe_title}"
# The result should use straight apostrophe
assert "Can't" in artist_title
assert "\u2019" not in artist_title # No curly apostrophe
# Should be HTTP-header safe
try:
artist_title.encode('latin-1')
header_safe = True
except UnicodeEncodeError:
header_safe = False
assert header_safe
def test_double_sanitization_idempotent(self):
"""Sanitizing twice should give the same result as once."""
original = "It\u2019s \u201cMy\u201d Song \u2014 Test\u2026"
once = sanitize_filename(original)
twice = sanitize_filename(once)
assert once == twice
def test_only_problematic_chars_string(self):
"""Test string made entirely of problematic characters."""
result = sanitize_filename("\u2018\u2019\u201c\u201d\u2014\u2026")
# Should become: ''""--... then quotes -> underscores, collapses
# Actually: ' ' " " - ... -> underscores for filesystem chars
assert result # Should not be empty
def test_very_long_filename(self):
"""Test that very long filenames are handled."""
long_name = "A" * 1000
result = sanitize_filename(long_name)
assert result == long_name # No truncation in sanitize_filename itself
def test_special_musical_characters(self):
"""Test musical symbols and special characters."""
# These should pass through as they're not in our replacement list
assert "♪" in sanitize_filename("♪ Intro ♪")
assert "♫" in sanitize_filename("♫ Music ♫")
assert "♯" in sanitize_filename("C♯ Minor")
assert "♭" in sanitize_filename("B♭ Major")
class TestNormalizeText:
"""Test the normalize_text function for data consistency normalization."""
def test_curly_single_quotes_normalized(self):
"""Test that curly single quotes are converted to straight apostrophe."""
assert normalize_text("Can\u2019t") == "Can't"
assert normalize_text("It\u2018s") == "It's"
assert normalize_text("\u201Aquote\u201B") == "'quote'"
def test_curly_double_quotes_normalized(self):
"""Test that curly double quotes are converted to straight double quotes."""
assert normalize_text("\u201CHello\u201D") == '"Hello"'
assert normalize_text("\u201Equote\u201F") == '"quote"'
def test_backticks_normalized(self):
"""Test that backticks and similar marks are converted to apostrophe."""
assert normalize_text("code`here") == "code'here"
assert normalize_text("acute\u00B4accent") == "acute'accent"
assert normalize_text("prime\u2032mark") == "prime'mark"
def test_dashes_normalized(self):
"""Test that various dashes are converted to hyphen-minus."""
# EN DASH
assert normalize_text("1990\u20132000") == "1990-2000"
# EM DASH
assert normalize_text("word\u2014word") == "word-word"
# MINUS SIGN
assert normalize_text("a\u2212b") == "a-b"
# FIGURE DASH
assert normalize_text("phone\u2012number") == "phone-number"
def test_whitespace_normalized(self):
"""Test that various whitespace characters are normalized."""
# NON-BREAKING SPACE
assert normalize_text("hello\u00A0world") == "hello world"
# EM SPACE
assert normalize_text("hello\u2003world") == "hello world"
# IDEOGRAPHIC SPACE (CJK full-width)
assert normalize_text("hello\u3000world") == "hello world"
# ZERO WIDTH SPACE (removed entirely)
assert normalize_text("hello\u200Bworld") == "helloworld"
def test_ellipsis_normalized(self):
"""Test that ellipsis character is converted to three dots."""
assert normalize_text("Wait\u2026") == "Wait..."
assert normalize_text("Loading\u2026please wait") == "Loading...please wait"
def test_multiple_spaces_collapsed(self):
"""Test that multiple spaces are collapsed to one."""
assert normalize_text("hello world") == "hello world"
# After normalizing multiple whitespace chars
assert normalize_text("hello\u00A0\u00A0\u00A0world") == "hello world"
def test_leading_trailing_whitespace_stripped(self):
"""Test that leading/trailing whitespace is stripped."""
assert normalize_text(" hello ") == "hello"
assert normalize_text("\u00A0hello\u00A0") == "hello"
def test_none_input(self):
"""Test that None input returns None."""
assert normalize_text(None) is None
def test_non_string_input(self):
"""Test that non-string input is returned unchanged."""
assert normalize_text(123) == 123
assert normalize_text(["list"]) == ["list"]
def test_international_characters_preserved(self):
"""Test that international characters are NOT normalized away."""
assert normalize_text("日本語") == "日本語"
assert normalize_text("한국어") == "한국어"
assert normalize_text("Café") == "Café"
assert normalize_text("Björk") == "Björk"
def test_real_world_examples(self):
"""Test real-world examples with mixed content."""
# Job d49efab1's title
assert normalize_text("Mama Says (You Can\u2019t Back Down)") == "Mama Says (You Can't Back Down)"
# Broadway cast with smart quotes and em dash
assert normalize_text("Footloose \u2014 \u201CFinal Song\u201D") == 'Footloose - "Final Song"'
# Japanese with em dash
assert normalize_text("宇多田ヒカル \u2014 First Love") == "宇多田ヒカル - First Love"
def test_idempotent(self):
"""Test that normalizing twice gives the same result."""
original = "It\u2019s \u201CMy\u201D Song \u2014 Test\u2026"
once = normalize_text(original)
twice = normalize_text(once)
assert once == twice
def test_text_normalizations_dict_complete(self):
"""Verify TEXT_NORMALIZATIONS includes all expected categories."""
# Check apostrophe-like characters
for char in APOSTROPHE_REPLACEMENTS:
assert char in TEXT_NORMALIZATIONS
# Check double quote-like characters
for char in DOUBLE_QUOTE_REPLACEMENTS:
assert char in TEXT_NORMALIZATIONS
# Check dash-like characters
for char in DASH_REPLACEMENTS:
assert char in TEXT_NORMALIZATIONS
# Check whitespace characters
for char in WHITESPACE_REPLACEMENTS:
assert char in TEXT_NORMALIZATIONS
class TestModelValidatorNormalization:
"""Test that model validators correctly normalize input text."""
def test_job_create_normalizes_artist_title(self):
"""Test that JobCreate model normalizes artist and title fields."""
from backend.models.job import JobCreate
job = JobCreate(
artist="Don\u2019t Stop",
title="Believin\u2019 \u2014 Live",
)
assert job.artist == "Don't Stop"
assert job.title == "Believin' - Live"
def test_job_create_preserves_international_chars(self):
"""Test that JobCreate preserves international characters."""
from backend.models.job import JobCreate
job = JobCreate(
artist="宇多田ヒカル",
title="First Love",
)
assert job.artist == "宇多田ヒカル"
assert job.title == "First Love"
def test_job_create_handles_none(self):
"""Test that JobCreate handles None values correctly."""
from backend.models.job import JobCreate
job = JobCreate(
artist=None,
title="Some Title",
)
assert job.artist is None
assert job.title == "Some Title"
def test_audio_search_normalizes_fields(self):
"""Test that AudioSearchRequest normalizes text fields."""
from backend.api.routes.audio_search import AudioSearchRequest
request = AudioSearchRequest(
artist="Artist\u2019s Name",
title="Song \u2014 Remix",
display_artist="Display\u2019s Artist",
display_title="Display\u2019s Title",
)
assert request.artist == "Artist's Name"
assert request.title == "Song - Remix"
assert request.display_artist == "Display's Artist"
assert request.display_title == "Display's Title"
def test_audio_search_preserves_international_chars(self):
"""Test that AudioSearchRequest preserves international characters."""
from backend.api.routes.audio_search import AudioSearchRequest
request = AudioSearchRequest(
artist="방탄소년단",
title="Dynamite",
)
assert request.artist == "방탄소년단"
assert request.title == "Dynamite"
@@ -425,6 +425,111 @@ class TestVideoWorkerOrchestratorEncoding:
assert "Encoding failed" in str(exc_info.value)
@pytest.mark.asyncio
async def test_run_encoding_gce_downloads_files(self):
"""Test that GCE encoding downloads files from GCS to local directory.
This test verifies the fix for YouTube upload failure when using GCE encoding.
GCE encoding returns GCS blob paths, which need to be downloaded locally
before YouTube upload can access them.
"""
with tempfile.TemporaryDirectory() as temp_dir:
config = OrchestratorConfig(
job_id="test-job",
artist="Test Artist",
title="Test Title",
title_video_path=os.path.join(temp_dir, "title.mov"),
karaoke_video_path=os.path.join(temp_dir, "karaoke.mov"),
instrumental_audio_path=os.path.join(temp_dir, "audio.flac"),
output_dir=temp_dir,
)
# Mock storage service
mock_storage = MagicMock()
mock_storage.download_file = MagicMock()
orchestrator = VideoWorkerOrchestrator(
config,
storage=mock_storage,
)
with patch.object(orchestrator, "_get_encoding_backend") as mock_get:
from backend.services.encoding_interface import EncodingOutput
# GCE backend returns GCS blob paths (not local paths)
mock_backend = MagicMock()
mock_backend.name = "gce" # Important: must be "gce" to trigger download
mock_backend.encode = AsyncMock(return_value=EncodingOutput(
success=True,
lossless_4k_mp4_path="jobs/test-job/finals/output_4k_lossless.mp4",
lossy_4k_mp4_path="jobs/test-job/finals/output_4k_lossy.mp4",
lossy_720p_mp4_path="jobs/test-job/finals/output_720p.mp4",
lossless_mkv_path="jobs/test-job/finals/output_4k.mkv",
encoding_time_seconds=60.0,
encoding_backend="gce",
))
mock_get.return_value = mock_backend
await orchestrator._run_encoding()
# Verify download_file was called for each output file
assert mock_storage.download_file.call_count == 4
# Verify the result paths were updated to local paths
assert orchestrator.result.final_video.startswith(temp_dir)
assert orchestrator.result.final_video_mkv.startswith(temp_dir)
assert orchestrator.result.final_video_lossy.startswith(temp_dir)
assert orchestrator.result.final_video_720p.startswith(temp_dir)
@pytest.mark.asyncio
async def test_run_encoding_local_does_not_download(self):
"""Test that local encoding does NOT trigger GCS download.
Local encoding produces files directly in the output directory,
so no download is needed.
"""
with tempfile.TemporaryDirectory() as temp_dir:
config = OrchestratorConfig(
job_id="test-job",
artist="Test Artist",
title="Test Title",
title_video_path=os.path.join(temp_dir, "title.mov"),
karaoke_video_path=os.path.join(temp_dir, "karaoke.mov"),
instrumental_audio_path=os.path.join(temp_dir, "audio.flac"),
output_dir=temp_dir,
)
# Mock storage service
mock_storage = MagicMock()
mock_storage.download_file = MagicMock()
orchestrator = VideoWorkerOrchestrator(
config,
storage=mock_storage,
)
with patch.object(orchestrator, "_get_encoding_backend") as mock_get:
from backend.services.encoding_interface import EncodingOutput
# Local backend returns local paths directly
mock_backend = MagicMock()
mock_backend.name = "local" # Local backend
mock_backend.encode = AsyncMock(return_value=EncodingOutput(
success=True,
lossless_4k_mp4_path=os.path.join(temp_dir, "lossless.mp4"),
lossy_4k_mp4_path=os.path.join(temp_dir, "lossy.mp4"),
lossy_720p_mp4_path=os.path.join(temp_dir, "720p.mp4"),
lossless_mkv_path=os.path.join(temp_dir, "lossless.mkv"),
encoding_time_seconds=120.0,
encoding_backend="local",
))
mock_get.return_value = mock_backend
await orchestrator._run_encoding()
# Verify download_file was NOT called for local encoding
mock_storage.download_file.assert_not_called()
class TestVideoWorkerOrchestratorOrganization:
"""Test organization stage."""
+27
View File
@@ -0,0 +1,27 @@
"""
Utilities for identifying test data.
Test data is generated by automated E2E tests and should be filterable
in admin dashboards to show only real user data by default.
"""
# Email domains used by automated testing frameworks
TEST_EMAIL_DOMAINS = [
"inbox.testmail.app", # Used by E2E happy path tests
]
def is_test_email(email: str) -> bool:
"""
Check if an email address belongs to automated test data.
Args:
email: Email address to check
Returns:
True if the email matches a test email domain pattern
"""
if not email:
return False
email_lower = email.lower()
return any(email_lower.endswith(f"@{domain}") for domain in TEST_EMAIL_DOMAINS)
+6 -1
View File
@@ -344,7 +344,12 @@ async def process_audio_separation(job_id: str) -> bool:
job_log.info(f"Stored effective model names: clean={effective_model_names['clean_instrumental_model']}")
# Format artist-title for file naming (matches CLI behavior)
artist_title = f"{job.artist} - {job.title}"
# Sanitize to handle Unicode characters (curly quotes, em dashes, etc.)
# that cause HTTP header encoding issues with the remote API
from karaoke_gen.utils import sanitize_filename
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
artist_title = f"{safe_artist} - {safe_title}"
# Run audio separation (calls Modal API internally)
# This returns a dict with paths to all separated stems
+22 -17
View File
@@ -639,40 +639,45 @@ async def upload_lyrics_results(
logger.error(f"Job {job_id}: {error_msg}")
raise Exception(error_msg)
# Upload reference lyrics if available
# Upload ALL reference lyrics files (not just first) so they're available for distribution
# Note: Source names use .title() so "lrclib" -> "Lrclib", "genius" -> "Genius"
# Use sanitized artist/title to match LyricsProcessor's file naming
reference_files = [
(f"{safe_artist} - {safe_title} (Lyrics Genius).txt", "Genius"),
(f"{safe_artist} - {safe_title} (Lyrics Spotify).txt", "Spotify"),
(f"{safe_artist} - {safe_title} (Lyrics Musixmatch).txt", "Musixmatch"),
(f"{safe_artist} - {safe_title} (Lyrics Lrclib).txt", "LRCLib"), # Note: source is 'lrclib', .title() = 'Lrclib'
(f"{safe_artist} - {safe_title} (Lyrics Genius).txt", "genius"),
(f"{safe_artist} - {safe_title} (Lyrics Spotify).txt", "spotify"),
(f"{safe_artist} - {safe_title} (Lyrics Musixmatch).txt", "musixmatch"),
(f"{safe_artist} - {safe_title} (Lyrics Lrclib).txt", "lrclib"),
]
found_reference = False
for ref_filename, source_name in reference_files:
found_references = []
for ref_filename, source_key in reference_files:
ref_path = os.path.join(lyrics_dir, ref_filename)
if os.path.exists(ref_path):
# Upload with original filename to preserve proper naming
gcs_path = f"jobs/{job_id}/lyrics/{ref_filename}"
url = storage.upload_file(ref_path, gcs_path)
# Track in job.files so video_worker can download for distribution
job_manager.update_file_url(job_id, 'lyrics', f'reference_{source_key}', url)
if job_log:
job_log.info(f"Found reference lyrics from {source_name}")
job_log.info(f"Found reference lyrics from {source_key}")
logger.info(f"Job {job_id}: Uploaded reference lyrics: {ref_filename}")
found_reference = True
break # Only upload first available reference
if not found_reference:
found_references.append(source_key)
if not found_references:
if job_log:
job_log.warning("No reference lyrics found from any source (Genius, Spotify, Musixmatch, LRCLib)")
logger.warning(f"Job {job_id}: No reference lyrics found from any source")
else:
logger.info(f"Job {job_id}: Found {len(found_references)} reference lyrics sources: {found_references}")
# Upload uncorrected transcription if available (use sanitized names)
uncorrected_file = os.path.join(lyrics_dir, f"{safe_artist} - {safe_title} (Lyrics Uncorrected).txt")
# Upload uncorrected transcription if available (preserve original filename for distribution)
uncorrected_filename = f"{safe_artist} - {safe_title} (Lyrics Uncorrected).txt"
uncorrected_file = os.path.join(lyrics_dir, uncorrected_filename)
if os.path.exists(uncorrected_file):
gcs_path = f"jobs/{job_id}/lyrics/uncorrected.txt"
gcs_path = f"jobs/{job_id}/lyrics/{uncorrected_filename}"
url = storage.upload_file(uncorrected_file, gcs_path)
job_manager.update_file_url(job_id, 'lyrics', 'uncorrected', url)
logger.info(f"Job {job_id}: Uploaded uncorrected transcription")
logger.info(f"Job {job_id}: Uploaded uncorrected transcription: {uncorrected_filename}")
logger.info(f"Job {job_id}: All lyrics results uploaded successfully")
+5 -1
View File
@@ -43,6 +43,7 @@ from lyrics_transcriber.core.config import OutputConfig
# Import from the unified style loader
from karaoke_gen.style_loader import load_styles_from_gcs
from karaoke_gen.utils import sanitize_filename
logger = logging.getLogger(__name__)
@@ -242,7 +243,10 @@ async def process_render_video(job_id: str) -> bool:
output_generator = OutputGenerator(config, logger)
# 8. Generate outputs (video, LRC, ASS, etc.)
output_prefix = f"{job.artist or 'Unknown'} - {job.title or 'Unknown'}"
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
output_prefix = f"{safe_artist} - {safe_title}"
job_log.info(f"Generating outputs with prefix '{output_prefix}'")
logger.info(f"Job {job_id}: Generating outputs with prefix '{output_prefix}'")
+27 -10
View File
@@ -39,6 +39,7 @@ from backend.services.tracing import job_span, add_span_event, add_span_attribut
# Import from karaoke_gen package
from karaoke_gen.video_generator import VideoGenerator
from karaoke_gen.utils import sanitize_filename
logger = logging.getLogger(__name__)
@@ -217,26 +218,36 @@ async def generate_screens(job_id: str) -> bool:
def _validate_prerequisites(job) -> bool:
"""
Validate that both audio and lyrics processing are complete.
Single Responsibility: Validation logic separated from main flow.
Args:
job: Job object
Returns:
True if prerequisites met, False otherwise
"""
# SAFETY NET: Enforce theme requirement at processing time
# This catches any jobs that somehow bypassed JobManager.create_job() validation
if not job.theme_id:
logger.error(
f"Job {job.job_id}: CRITICAL - No theme_id configured. "
"All jobs must have a theme to generate styled videos. "
"This job should have been rejected at creation time."
)
return False
audio_complete = job.state_data.get('audio_complete', False)
lyrics_complete = job.state_data.get('lyrics_complete', False)
if not audio_complete:
logger.error(f"Job {job.job_id}: Audio processing not complete")
return False
if not lyrics_complete:
logger.error(f"Job {job.job_id}: Lyrics processing not complete")
return False
if not job.artist or not job.title:
logger.error(f"Job {job.job_id}: Missing artist or title")
return False
@@ -294,9 +305,12 @@ async def _generate_title_screen(
"""
try:
logger.info(f"Job {job_id}: Generating title screen")
# Set up output paths
artist_title = f"{job.artist} - {job.title}"
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
artist_title = f"{safe_artist} - {safe_title}"
output_image_filepath_noext = os.path.join(temp_dir, f"{artist_title} (Title)")
output_video_filepath = os.path.join(temp_dir, f"{artist_title} (Title).mov")
@@ -372,9 +386,12 @@ async def _generate_end_screen(
"""
try:
logger.info(f"Job {job_id}: Generating end screen")
# Set up output paths
artist_title = f"{job.artist} - {job.title}"
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
artist_title = f"{safe_artist} - {safe_title}"
output_image_filepath_noext = os.path.join(temp_dir, f"{artist_title} (End)")
output_video_filepath = os.path.join(temp_dir, f"{artist_title} (End).mov")
+103 -49
View File
@@ -44,6 +44,7 @@ from backend.services.tracing import job_span, add_span_event, add_span_attribut
# Import from karaoke_gen package - reuse existing implementation
from karaoke_gen.karaoke_finalise.karaoke_finalise import KaraokeFinalise
from karaoke_gen.utils import sanitize_filename
logger = logging.getLogger(__name__)
@@ -160,14 +161,17 @@ async def _encode_via_gce(
storage.download_file(gcs_path, local_path)
# Map to result keys expected by _upload_results
if "4K Lossless" in filename or "4k_lossless" in filename.lower():
local_files["final_video"] = local_path
elif "4K Lossy" in filename or "4k_lossy" in filename.lower():
# Files are named like "Artist - Title (Final Karaoke Lossless 4k).mp4"
filename_lower = filename.lower()
if "lossless 4k" in filename_lower:
if filename.endswith(".mkv"):
local_files["final_video_mkv"] = local_path
else:
local_files["final_video"] = local_path
elif "lossy 4k" in filename_lower:
local_files["final_video_lossy"] = local_path
elif "720p" in filename:
elif "720p" in filename_lower:
local_files["final_video_720p"] = local_path
elif filename.endswith(".mkv"):
local_files["final_video_mkv"] = local_path
job_log.info(f"Downloaded {len(local_files)} encoded files")
@@ -258,7 +262,10 @@ async def generate_video_orchestrated(job_id: str) -> bool:
)
# Download and set up files
base_name = f"{job.artist} - {job.title}"
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
base_name = f"{safe_artist} - {safe_title}"
with job_span("download-files", job_id):
job_log.info("Downloading files from GCS...")
await _setup_working_directory(job_id, job, storage, temp_dir, base_name, job_log)
@@ -332,16 +339,9 @@ async def generate_video_orchestrated(job_id: str) -> bool:
'final_karaoke_txt_zip': result.final_karaoke_txt_zip,
})
# Mark job as complete
# Store result metadata in job BEFORE transitioning to COMPLETE
# This ensures youtube_url is available when completion email is sent
logger.info(f"[job:{job_id}] Video generation complete")
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.COMPLETE,
progress=100,
message="Karaoke generation complete!"
)
# Store result metadata in job
job_manager.update_job(job_id, {
'state_data': {
**job.state_data,
@@ -352,6 +352,14 @@ async def generate_video_orchestrated(job_id: str) -> bool:
}
})
# Mark job as complete (triggers completion email with youtube_url now available)
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.COMPLETE,
progress=100,
message="Karaoke generation complete!"
)
duration = time.time() - start_time
root_span.set_attribute("duration_seconds", duration)
root_span.set_attribute("brand_code", result.brand_code or '')
@@ -489,7 +497,10 @@ async def generate_video_legacy(job_id: str) -> bool:
)
# Download and set up files in the format KaraokeFinalise expects
base_name = f"{job.artist} - {job.title}"
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
base_name = f"{safe_artist} - {safe_title}"
with job_span("download-files", job_id):
job_log.info("Downloading files from GCS (this may take a few minutes for large files)...")
await _setup_working_directory(job_id, job, storage, temp_dir, base_name, job_log)
@@ -662,19 +673,12 @@ async def generate_video_legacy(job_id: str) -> bool:
with job_span("upload-results", job_id):
await _upload_results(job_id, job_manager, storage, temp_dir, result)
# Mark job as complete
logger.info(f"[job:{job_id}] Video generation complete")
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.COMPLETE,
progress=100,
message="Karaoke generation complete!"
)
# Store result metadata in job
# Store result metadata in job BEFORE transitioning to COMPLETE
# This ensures youtube_url is available when completion email is sent
# NOTE: Must include dropbox_link and gdrive_files from result, since
# _handle_native_distribution already saved them but job.state_data
# is stale (fetched before distribution ran)
logger.info(f"[job:{job_id}] Video generation complete")
job_manager.update_job(job_id, {
'state_data': {
**job.state_data,
@@ -684,7 +688,15 @@ async def generate_video_legacy(job_id: str) -> bool:
'gdrive_files': result.get('gdrive_files'),
}
})
# Mark job as complete (triggers completion email with youtube_url now available)
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.COMPLETE,
progress=100,
message="Karaoke generation complete!"
)
duration = time.time() - start_time
root_span.set_attribute("duration_seconds", duration)
root_span.set_attribute("brand_code", result.get('brand_code', ''))
@@ -750,8 +762,11 @@ async def _handle_native_distribution(
storage: StorageService instance for downloading stems/lyrics
"""
brand_code = result.get('brand_code')
base_name = f"{job.artist} - {job.title}"
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
base_name = f"{safe_artist} - {safe_title}"
# Check if we should preserve existing brand code (Batch 6: --keep-brand-code)
keep_brand_code = getattr(job, 'keep_brand_code', None)
if keep_brand_code:
@@ -816,9 +831,12 @@ async def _handle_native_distribution(
# Don't fail the job - distribution is optional
# Upload to Google Drive using native API
# Skip if orchestrator already uploaded (gdrive_files already populated)
# This prevents duplicate uploads when using the orchestrator path
gdrive_folder_id = getattr(job, 'gdrive_folder_id', None)
if gdrive_folder_id:
existing_gdrive_files = result.get('gdrive_files')
if gdrive_folder_id and not existing_gdrive_files:
try:
from backend.services.gdrive_service import get_gdrive_service
@@ -854,7 +872,9 @@ async def _handle_native_distribution(
except Exception as e:
job_log.error(f"Native Google Drive upload failed: {e}", exc_info=True)
# Don't fail the job - distribution is optional
elif existing_gdrive_files:
job_log.info(f"Skipping Google Drive upload - orchestrator already uploaded {len(existing_gdrive_files)} files")
# Update job state_data with brand code and links
if brand_code or result.get('dropbox_link') or result.get('gdrive_files'):
try:
@@ -1108,29 +1128,63 @@ async def _prepare_distribution_directory(
log_progress(f"Downloaded {stem_key} to stems/")
# --- Download lyrics files to lyrics/ subfolder ---
# Use proper filenames matching local CLI output structure
# Common lyrics file types
lyrics_file_keys = ['lrc', 'ass', 'srt', 'txt', 'json', 'original_json', 'corrected_json']
for lyrics_key in lyrics_file_keys:
# Map job.files.lyrics keys to proper local filenames
# Format: (job_files_key, local_filename_pattern)
lyrics_file_mappings = [
# Karaoke output files
('lrc', f"{base_name} (Karaoke).lrc"),
('ass', f"{base_name} (Karaoke).ass"),
# Corrections and transcription files
('corrections', f"{base_name} (Lyrics Corrections).json"),
('corrected_txt', f"{base_name} (Lyrics Corrected).txt"),
('uncorrected', f"{base_name} (Lyrics Uncorrected).txt"),
# Reference lyrics from various sources
('reference_genius', f"{base_name} (Lyrics Genius).txt"),
('reference_spotify', f"{base_name} (Lyrics Spotify).txt"),
('reference_musixmatch', f"{base_name} (Lyrics Musixmatch).txt"),
('reference_lrclib', f"{base_name} (Lyrics Lrclib).txt"),
]
for lyrics_key, local_filename in lyrics_file_mappings:
if lyrics.get(lyrics_key):
# Determine proper extension
ext_map = {
'lrc': '.lrc',
'ass': '.ass',
'srt': '.srt',
'txt': '.txt',
'json': '.json',
'original_json': '_original.json',
'corrected_json': '_corrected.json',
}
ext = ext_map.get(lyrics_key, f'.{lyrics_key}')
dest_path = os.path.join(lyrics_dir, f"{base_name} (Karaoke){ext}")
dest_path = os.path.join(lyrics_dir, local_filename)
try:
storage.download_file(lyrics[lyrics_key], dest_path)
log_progress(f"Downloaded {lyrics_key} to lyrics/")
log_progress(f"Downloaded {lyrics_key} to lyrics/{local_filename}")
except Exception as e:
log_progress(f"Could not download {lyrics_key}: {e}")
# --- Download (With Vocals).mkv to lyrics/ subfolder ---
# Local CLI places the karaoke video with vocals in the lyrics folder
videos = job.file_urls.get('videos', {})
if videos.get('with_vocals'):
with_vocals_dest = os.path.join(lyrics_dir, f"{base_name} (With Vocals).mkv")
try:
storage.download_file(videos['with_vocals'], with_vocals_dest)
log_progress(f"Downloaded with_vocals to lyrics/{base_name} (With Vocals).mkv")
except Exception as e:
log_progress(f"Could not download with_vocals to lyrics/: {e}")
# --- Download preview ASS files to lyrics/previews/ subfolder ---
# Preview files are stored in jobs/{job_id}/previews/ in GCS
previews_dir = os.path.join(lyrics_dir, "previews")
os.makedirs(previews_dir, exist_ok=True)
# List all preview files from GCS and download them
try:
preview_prefix = f"jobs/{job_id}/previews/"
preview_files = storage.list_files(preview_prefix)
for blob_name in preview_files:
if blob_name.endswith('.ass'):
filename = os.path.basename(blob_name)
dest_path = os.path.join(previews_dir, filename)
storage.download_file(blob_name, dest_path)
log_progress(f"Downloaded preview {filename} to lyrics/previews/")
except Exception as e:
log_progress(f"Could not download preview files: {e}")
log_progress("Distribution directory prepared with stems/ and lyrics/ subfolders")
+65 -3
View File
@@ -27,6 +27,7 @@ from backend.models.job import JobStatus
from backend.services.job_manager import JobManager
from backend.services.storage_service import StorageService
from backend.services.tracing import job_span, add_span_event
from karaoke_gen.utils import sanitize_filename
logger = logging.getLogger(__name__)
@@ -367,15 +368,70 @@ class VideoWorkerOrchestrator:
if not output.success:
raise Exception(f"Encoding failed: {output.error_message}")
# Store results
# Store results - for GCE backend, these are GCS blob paths that need to be downloaded
self.result.final_video = output.lossless_4k_mp4_path
self.result.final_video_mkv = output.lossless_mkv_path
self.result.final_video_lossy = output.lossy_4k_mp4_path
self.result.final_video_720p = output.lossy_720p_mp4_path
self.result.encoding_time_seconds = output.encoding_time_seconds
# For GCE encoding, download the encoded files from GCS to local directory
# This is required for YouTube upload and other local file operations
if encoding_backend.name == "gce" and self.storage:
await self._download_gce_encoded_files(output)
self.job_log.info(f"Encoding complete ({encoding_backend.name}) in {output.encoding_time_seconds:.1f}s")
async def _download_gce_encoded_files(self, output):
"""
Download GCE-encoded files from GCS to the local output directory.
GCE encoding stores files in GCS and returns blob paths like:
'jobs/{job_id}/finals/Artist - Title (Final Karaoke Lossless 4k).mp4'
This method downloads those files locally so that subsequent stages
(YouTube upload, etc.) can access them as local files.
Args:
output: EncodingOutput from the GCE backend with GCS blob paths
"""
self.job_log.info("Downloading GCE-encoded files from GCS")
# Map of result attributes to download
file_mappings = [
('lossless_4k_mp4_path', 'final_video'),
('lossless_mkv_path', 'final_video_mkv'),
('lossy_4k_mp4_path', 'final_video_lossy'),
('lossy_720p_mp4_path', 'final_video_720p'),
]
downloaded_count = 0
for output_attr, result_attr in file_mappings:
gcs_path = getattr(output, output_attr, None)
if not gcs_path:
continue
# Extract filename from GCS path
filename = os.path.basename(gcs_path)
local_path = os.path.join(self.config.output_dir, filename)
try:
self.job_log.info(f"Downloading {filename} from GCS")
self.storage.download_file(gcs_path, local_path)
# Update the result to point to local file
setattr(self.result, result_attr, local_path)
downloaded_count += 1
self.job_log.info(f"Downloaded {filename} to {local_path}")
except Exception as e:
self.job_log.error(f"Failed to download {filename}: {e}")
# Clear the result attribute so downstream doesn't try to use invalid GCS path
setattr(self.result, result_attr, None)
# Don't fail - some formats might not be generated
self.job_log.info(f"Downloaded {downloaded_count} encoded files from GCS")
async def _run_organization(self):
"""Run the organization stage (brand code generation)."""
self.job_log.info("Starting organization stage")
@@ -483,7 +539,10 @@ class VideoWorkerOrchestrator:
self.job_log.warning("Dropbox not configured, skipping upload")
return
base_name = f"{self.config.artist} - {self.config.title}"
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
safe_artist = sanitize_filename(self.config.artist) if self.config.artist else "Unknown"
safe_title = sanitize_filename(self.config.title) if self.config.title else "Unknown"
base_name = f"{safe_artist} - {safe_title}"
folder_name = f"{self.result.brand_code or 'TRACK-0000'} - {base_name}"
remote_folder = f"{self.config.dropbox_path}/{folder_name}"
@@ -582,7 +641,10 @@ def create_orchestrator_config_from_job(
Returns:
OrchestratorConfig for the orchestrator
"""
base_name = f"{job.artist} - {job.title}"
# Sanitize artist/title to handle Unicode characters (curly quotes, em dashes, etc.)
safe_artist = sanitize_filename(job.artist) if job.artist else "Unknown"
safe_title = sanitize_filename(job.title) if job.title else "Unknown"
base_name = f"{safe_artist} - {safe_title}"
# Determine instrumental file path
instrumental_selection = job.state_data.get('instrumental_selection', 'clean')
+276 -4
View File
@@ -20,6 +20,30 @@ GET /health
No auth required. Returns service status.
#### Encoding Worker Health
```http
GET /health/encoding-worker
```
No auth required. Returns GCE encoding worker status for frontend display.
Response:
```json
{
"available": true,
"status": "ok",
"version": "0.95.1",
"active_jobs": 0,
"queue_length": 0
}
```
Status values:
- `ok` - Worker is healthy and available
- `offline` - Worker is unavailable (includes `error` field)
- `not_configured` - Encoding worker not configured on backend
### Jobs
#### Create Job with Upload
@@ -53,8 +77,14 @@ Response includes:
GET /api/jobs
GET /api/jobs?status=complete
GET /api/jobs?limit=10&offset=0
GET /api/jobs?exclude_test=false
```
Query parameters:
- `exclude_test` (bool, default: true) - Admin only: filter out jobs from test users
- `status` - Filter by job status
- `limit` / `offset` - Pagination
#### Delete Job
```http
@@ -114,11 +144,52 @@ Content-Type: application/json
### Audio Search
#### Search for Audio
```http
GET /api/audio-search?q=song+name
POST /api/audio-search/search
Content-Type: application/json
{
"artist": "Artist Name",
"title": "Song Title",
"auto_download": false,
"theme_id": "nomad",
"display_artist": "Display Artist (optional)",
"display_title": "Display Title (optional)"
}
```
Search for songs (flacfetch integration).
Searches for audio sources (flacfetch integration). Creates a job and returns search results.
**Key fields:**
- `artist`, `title` - Used to search for audio on trackers
- `display_artist`, `display_title` - Optional overrides for how artist/title appear on title screens and filenames. If omitted, search values are used.
- `auto_download` - If true, automatically selects best result and starts processing
- `theme_id` - Video theme to use
**Use case:** Search for "Jeremy Kushnier" but display "Footloose (Broadway Cast)" on the karaoke video.
#### Get Search Results
```http
GET /api/audio-search/{job_id}/results
```
Returns cached search results for a job awaiting audio selection.
#### Select Audio Source
```http
POST /api/audio-search/{job_id}/select
Content-Type: application/json
{
"selection_index": 0
}
```
Selects an audio source from the search results and starts processing.
### Themes
@@ -128,6 +199,70 @@ GET /api/themes
No auth required. Returns available video themes.
### Tenant Config
```http
GET /api/tenant/config
GET /api/tenant/config?tenant=vocalstar
```
No auth required. Returns tenant configuration for white-label portals.
**Detection priority**:
1. `X-Tenant-ID` header (explicitly set by frontend)
2. `tenant` query parameter (development/testing only, disabled in production)
3. Host header subdomain detection (e.g., `vocalstar.nomadkaraoke.com`)
Response:
```json
{
"tenant": {
"id": "vocalstar",
"name": "Vocal Star",
"subdomain": "vocalstar.nomadkaraoke.com",
"is_active": true,
"branding": {
"logo_url": "https://storage.googleapis.com/...",
"logo_height": 60,
"primary_color": "#ffff00",
"secondary_color": "#006CF9",
"accent_color": "#ffffff",
"background_color": "#000000",
"favicon_url": null,
"site_title": "Vocal Star Karaoke Generator",
"tagline": "Whoever You Are, Be a Vocal Star!"
},
"features": {
"audio_search": false,
"file_upload": true,
"youtube_url": false,
"youtube_upload": false,
"dropbox_upload": false,
"gdrive_upload": false,
"theme_selection": false,
"color_overrides": false,
"enable_cdg": true,
"enable_4k": true,
"admin_access": false
},
"defaults": {
"theme_id": "vocalstar",
"locked_theme": "vocalstar",
"distribution_mode": "download_only"
}
},
"is_default": false
}
```
When no tenant is detected, returns `tenant: null` with `is_default: true` and the default Nomad Karaoke configuration is used.
```http
GET /api/tenant/config/{tenant_id}
```
Get specific tenant config by ID. Returns 404 if tenant not found or inactive.
### Internal (Admin Only)
These endpoints are used by workers and require admin tokens.
@@ -217,7 +352,7 @@ GET /api/users/credits/packages
No auth required. Returns available packages and prices.
### Create Checkout Session
### Create Checkout Session (Credits)
```http
POST /api/users/credits/checkout
@@ -226,7 +361,27 @@ Content-Type: application/json
{"package_id": "5_credits", "email": "user@example.com"}
```
Returns Stripe checkout URL.
Returns Stripe checkout URL. Supports Apple Pay, Google Pay, Link, and other payment methods based on customer device.
### Create Checkout Session (Made For You)
```http
POST /api/users/made-for-you/checkout
Content-Type: application/json
{
"email": "customer@example.com",
"artist": "Queen",
"title": "Bohemian Rhapsody",
"source_type": "search",
"youtube_url": null,
"notes": "Optional special requests"
}
```
Creates a $15 checkout for the full-service "Made For You" karaoke video. After payment, a job is automatically created and the customer receives email updates. Supports all enabled payment methods.
**source_type**: `search` (find audio automatically), `youtube` (use provided URL)
### Stripe Webhook
@@ -276,10 +431,24 @@ Bonus credit for detailed feedback (50+ chars).
All admin endpoints require an admin-role session token.
### Test Data Filtering
Most admin list/stats endpoints support an `exclude_test` query parameter (default: `true`) to filter out E2E test data:
- Test users: email addresses ending in `@inbox.testmail.app`
- Test jobs: jobs created by test users
```http
GET /api/admin/stats/overview?exclude_test=true # Default - hide test data
GET /api/admin/stats/overview?exclude_test=false # Show all data including tests
```
The frontend admin dashboard includes a "Show test data" toggle that controls this filter globally.
### Admin Dashboard Stats
```http
GET /api/admin/stats/overview
GET /api/admin/stats/overview?exclude_test=false
Authorization: Bearer ADMIN_TOKEN
```
@@ -310,9 +479,16 @@ Returns platform statistics:
GET /api/users/admin/users
GET /api/users/admin/users?search=user@example.com
GET /api/users/admin/users?limit=20&offset=0&sort_by=created_at&sort_order=desc
GET /api/users/admin/users?exclude_test=false
Authorization: Bearer ADMIN_TOKEN
```
Query parameters:
- `exclude_test` (bool, default: true) - Filter out test users
- `limit` / `offset` - Pagination
- `search` - Email prefix search
- `sort_by` / `sort_order` - Sorting
Returns paginated user list with total count.
### User Detail (Admin)
@@ -342,6 +518,30 @@ Content-Type: application/json
}
```
### Impersonate User (Admin)
Allows admins to view the application as any user by creating a session token for that user.
```http
POST /api/admin/users/{email}/impersonate
Authorization: Bearer ADMIN_TOKEN
```
Response:
```json
{
"session_token": "impersonation-session-token",
"user_email": "user@example.com",
"message": "Now impersonating user@example.com"
}
```
Notes:
- Creates a real session for the target user (auditable in Firestore)
- Admin cannot impersonate themselves
- Returns 404 if user doesn't exist
- The frontend stores the original admin token in memory (not localStorage) for security
### Set User Role (Admin)
```http
@@ -352,6 +552,8 @@ Content-Type: application/json
{"role": "admin"}
```
Note: This endpoint exists but is not exposed in the admin UI. Use database operations for role changes.
### Enable/Disable User (Admin)
```http
@@ -364,9 +566,13 @@ Authorization: Bearer ADMIN_TOKEN
```http
GET /api/users/admin/beta/stats
GET /api/users/admin/beta/stats?exclude_test=false
Authorization: Bearer ADMIN_TOKEN
```
Query parameters:
- `exclude_test` (bool, default: true) - Filter out test users from beta statistics
### Beta Feedback List (Admin)
```http
@@ -390,9 +596,15 @@ Admins can delete any job. Regular users can only delete their own jobs.
```http
GET /api/admin/audio-searches
GET /api/admin/audio-searches?limit=50&status_filter=awaiting_audio_selection
GET /api/admin/audio-searches?exclude_test=false
Authorization: Bearer ADMIN_TOKEN
```
Query parameters:
- `exclude_test` (bool, default: true) - Filter out searches from test users
- `limit` - Max results to return
- `status_filter` - Filter by job status
Returns jobs with cached audio search results. Useful for:
- Monitoring search activity
- Identifying stale cached results (YouTube-only when lossless should be available)
@@ -487,6 +699,66 @@ Response:
}
```
### Email Notifications (Admin)
#### Get Completion Message
```http
GET /api/admin/jobs/{job_id}/completion-message
Authorization: Bearer ADMIN_TOKEN
```
Returns the rendered completion message for a job (for copying to clipboard).
Response:
```json
{
"job_id": "abc123",
"message": "Hi there! Your karaoke video is ready...",
"subject": "Your Karaoke Video is Ready! 🎤",
"youtube_url": "https://youtube.com/watch?v=...",
"dropbox_url": "https://dropbox.com/..."
}
```
#### Send Completion Email
```http
POST /api/admin/jobs/{job_id}/send-completion-email
Authorization: Bearer ADMIN_TOKEN
Content-Type: application/json
{
"to_email": "customer@example.com",
"cc_admin": true
}
```
Sends the job completion email to the specified address. When `cc_admin` is true (default), CCs gen@nomadkaraoke.com.
Response:
```json
{
"success": true,
"job_id": "abc123",
"to_email": "customer@example.com",
"message": "Completion email sent successfully"
}
```
### Internal Email Endpoints
These endpoints are used by Cloud Tasks for automated notifications.
#### Check Idle Reminder
```http
POST /api/internal/jobs/{job_id}/check-idle-reminder
Authorization: Bearer ADMIN_TOKEN
```
Called by Cloud Tasks 5 minutes after a job enters a blocking state (awaiting_review or awaiting_instrumental_selection). Sends a reminder email if the user is still idle and no reminder has been sent yet.
## Rate Limits
No rate limits currently implemented.
+134 -4
View File
@@ -30,8 +30,10 @@
│ (jobs) │ │ (files) │ │ (API keys) │ │ Worker (*) │
└──────────┘ └──────────┘ └──────────────┘ └──────────────┘
(*) GCE Encoding Worker: C4-standard-8 VM with Intel Granite Rapids for
high-performance FFmpeg encoding. Optional - falls back to Cloud Run.
(*) GCE Encoding Worker: c4d-highcpu-32 VM with AMD EPYC 9B45 (Turin) for
high-performance FFmpeg encoding (4.92x faster than previous c4-standard-8).
Used for both final video encoding and preview video generation.
Uses LocalEncodingService via GCS wheel deployment.
```
## Processing Pipeline
@@ -57,7 +59,7 @@
▼
4. HUMAN REVIEW (AWAITING_REVIEW)
• React UI for lyrics correction
• Preview video generation
• Preview video generation (optionally via GCE for speed)
• Submit corrections
▼
5. VIDEO RENDERING
@@ -154,6 +156,8 @@ LyricsTranscriber LyricsTranscriber
| Vertex AI | Agentic AI correction (Gemini 3 Flash) | Default on |
| Genius | Reference lyrics | Yes |
| YouTube API | Video upload | Optional |
| SendGrid | Email notifications | Optional |
| Cloud Tasks | Delayed task scheduling (idle reminders) | Optional |
## Firestore Collections
@@ -190,7 +194,8 @@ The Video Worker uses an orchestrator pattern to ensure all features work regard
│ └─ YouTube upload │
│ │
│ Stage 5: NOTIFICATIONS │
│ └─ Discord webhooks │
│ ├─ Discord webhooks │
│ └─ Email notifications (completion, reminders) │
└─────────────────────────────────────────────────────────────┘
```
@@ -208,6 +213,129 @@ The Video Worker uses an orchestrator pattern to ensure all features work regard
| `local_encoding_service.py` | FFmpeg encoding (Cloud Run) |
| `encoding_interface.py` | Abstract interface + GCE backend |
| `video_worker_orchestrator.py` | Stage coordination |
| `email_service.py` | SendGrid email delivery with CC support |
| `template_service.py` | GCS-backed email templates |
| `job_notification_service.py` | Email orchestration (completion, reminders) |
## Multitenancy
The platform supports white-label B2B portals where business customers get their own branded karaoke generation experience.
### Architecture
```text
┌─────────────────────────────────────────────────────────────────┐
│ Tenant Detection Flow │
│ │
│ Request arrives at vocalstar.nomadkaraoke.com │
│ │ │
│ ▼ │
│ Frontend: detectTenantFromUrl() │
│ - Extracts subdomain from hostname │
│ - Calls GET /api/tenant/config?tenant=vocalstar │
│ │ │
│ ▼ │
│ Backend: TenantMiddleware │
│ 1. X-Tenant-ID header (from frontend) │
│ 2. Query param (dev only, disabled in production) │
│ 3. Host header subdomain detection │
│ │ │
│ ▼ │
│ TenantService.get_tenant_config() │
│ - Loads from GCS: tenants/{tenant_id}/config.json │
│ - Caches in memory (5 min TTL) │
│ │ │
│ ▼ │
│ Request proceeds with tenant context │
│ - request.state.tenant_id │
│ - request.state.tenant_config │
└─────────────────────────────────────────────────────────────────┘
```
### GCS Storage Layout
```text
tenants/{tenant_id}/
├── config.json # TenantConfig with branding, features, defaults
└── logo.jpg # Tenant logo (optional)
themes/{theme_id}/
├── style_params.json # Full theme configuration
└── assets/
├── intro_background.png
├── karaoke_background.jpg
├── end_background.png
├── font.ttf
└── cdg_*.gif
```
### Tenant Config Schema
```python
TenantConfig:
id: str # e.g., "vocalstar"
name: str # Display name: "Vocal Star"
subdomain: str # e.g., "vocalstar.nomadkaraoke.com"
is_active: bool # Enable/disable tenant
branding:
logo_url: str | None # GCS path to logo
logo_height: int # Pixels
primary_color: str # Hex color
secondary_color: str
accent_color: str | None
background_color: str | None
favicon_url: str | None
site_title: str
tagline: str | None
features:
audio_search: bool # Enable audio search (vs file upload only)
file_upload: bool
youtube_url: bool
youtube_upload: bool # YouTube distribution
dropbox_upload: bool
gdrive_upload: bool
theme_selection: bool # Allow user to pick theme
color_overrides: bool
enable_cdg: bool
enable_4k: bool
admin_access: bool
defaults:
theme_id: str | None # Default theme if not locked
locked_theme: str | None # Force this theme (user cannot change)
distribution_mode: str # "all", "download_only", etc.
auth:
allowed_email_domains: list[str] # e.g., ["vocal-star.com"]
require_email_domain: bool
sender_email: str | None # Override email sender
```
### Feature Enforcement
**Frontend**: The `useTenant()` hook provides `features` object. UI components conditionally render based on enabled features:
```typescript
const { features } = useTenant()
if (!features.audio_search) {
// Hide audio search tab
}
```
**Backend**: Routes check tenant config from request state:
```python
tenant_config = get_tenant_config_from_request(request)
if tenant_config and not tenant_config.features.audio_search:
raise HTTPException(403, "Audio search not enabled for this tenant")
```
### First Tenant: Vocal Star
- Subdomain: `vocalstar.nomadkaraoke.com`
- Features: File upload only (no audio search, no YouTube URL)
- Distribution: Download only (no YouTube/Dropbox/GDrive)
- Theme: Locked to "vocalstar" theme (yellow/blue)
- Auth: Restricted to `@vocal-star.com` and `@vocalstarmusic.com` emails
- Setup: `python scripts/setup-vocalstar-tenant.py`
## Tech Stack
@@ -216,5 +344,7 @@ The Video Worker uses an orchestrator pattern to ensure all features work regard
- **Database**: Firestore
- **Storage**: Google Cloud Storage
- **Secrets**: Google Secret Manager
- **Task Queues**: Cloud Tasks (idle reminders)
- **Email**: SendGrid
- **IaC**: Pulumi
- **CI/CD**: GitHub Actions
+709
View File
@@ -0,0 +1,709 @@
# Nomad Karaoke Brand Style Guide
> **Version**: 1.2
> **Last Updated**: 2026-01-08
> **Purpose**: Unified brand identity for all Nomad Karaoke products and communications
This style guide is designed for both human designers and LLM agents implementing the Nomad Karaoke brand across web applications, emails, marketing materials, and other touchpoints.
---
## Table of Contents
1. [Brand Overview](#brand-overview)
2. [Color Palette](#color-palette)
3. [Typography](#typography)
4. [Logo Usage](#logo-usage)
5. [Iconography](#iconography)
6. [UI Components](#ui-components)
7. [Layout & Spacing](#layout--spacing)
8. [Email Design](#email-design)
9. [Tone & Voice](#tone--voice)
10. [Light/Dark Theme Requirements](#lightdark-theme-requirements)
11. [Implementation Reference](#implementation-reference)
---
## Brand Overview
### Brand Personality
Nomad Karaoke is **approachable**, **professional**, and **modern**. We make sophisticated technology feel accessible and fun. Our brand bridges the gap between serious audio engineering and the joy of karaoke.
### Core Values
- **Approachable**: Technology that anyone can use
- **Professional**: Studio-quality output
- **Playful**: Karaoke should be fun
- **Trustworthy**: Reliable service and results
### Brand Keywords
`modern` `vibrant` `accessible` `professional` `musical` `AI-powered` `creative`
---
## Color Palette
### Primary Colors
| Name | Hex | RGB | Usage |
|------|-----|-----|-------|
| **Hot Pink** | `#ff5bb8` | rgb(255, 91, 184) | Primary brand color, buttons with white text (better contrast) |
| **Hot Pink Light** | `#ff7acc` | rgb(255, 122, 204) | Logo, hover states, accents where contrast is less critical |
| **Gold** | `#ffdf6b` | rgb(255, 223, 107) | Secondary accent, highlights, success states, premium feel |
> **Accessibility Note**: We use the darker pink (`#ff5bb8`) as the default for UI elements with white text because it provides better contrast ratio (WCAG compliance). The lighter pink (`#ff7acc`) is used for hover states and decorative accents where white text contrast is less critical.
### Supporting Colors
| Name | Hex | RGB | Usage |
|------|-----|-----|-------|
| **Deep Purple** | `#8b5cf6` | rgb(139, 92, 246) | Gradients, special features, beta badges |
| **Electric Blue** | `#3b82f6` | rgb(59, 130, 246) | Links, interactive elements, info states |
| **Soft Pink** | `#ec4899` | rgb(236, 72, 153) | Gradient end points, hover states |
### Semantic Colors
| Name | Hex | Usage |
|------|-----|-------|
| **Success** | `#22c55e` | Completed states, positive feedback |
| **Warning** | `#f59e0b` | Attention needed, caution states |
| **Error** | `#ef4444` | Errors, destructive actions |
| **Info** | `#3b82f6` | Informational messages |
### Dark Theme (Default)
| Name | Hex | Usage |
|------|-----|-------|
| **Background** | `#0f0f0f` | Page background |
| **Card Background** | `#1a1a1a` | Card surfaces |
| **Card Border** | `#2a2a2a` | Subtle borders |
| **Text Primary** | `#e5e5e5` | Main content text |
| **Text Muted** | `#888888` | Secondary text, labels |
| **Text Subtle** | `#666666` | Tertiary text, hints |
### Light Theme
| Name | Hex | Usage |
|------|-----|-------|
| **Background** | `#f8fafc` | Page background |
| **Card Background** | `#ffffff` | Card surfaces |
| **Card Border** | `#e2e8f0` | Subtle borders |
| **Text Primary** | `#1e293b` | Main content text |
| **Text Muted** | `#64748b` | Secondary text, labels |
### Gradients
#### Primary Gradient (Brand Gradient)
```css
background: linear-gradient(135deg, #3b82f6 0%, #8b5cf6 50%, #ff7acc 100%);
```
Use for: Hero text, feature highlights, premium elements. The gradient flows from blue through purple to the brand pink.
#### Radial Background Gradient (Landing Pages)
```css
/* Layered radial gradients using brand colors */
body::after {
content: '';
position: fixed;
inset: 0;
z-index: -1;
background:
radial-gradient(ellipse at 20% 0%, rgba(139, 92, 246, 0.12) 0%, transparent 50%), /* Purple glow top-left */
radial-gradient(ellipse at 80% 100%, rgba(255, 122, 204, 0.10) 0%, transparent 50%), /* Pink glow bottom-right */
radial-gradient(ellipse at 50% 50%, rgba(255, 223, 107, 0.03) 0%, transparent 70%); /* Gold subtle center */
pointer-events: none;
}
/* Optional: Add subtle pulsing animation for ambient glow effect */
@keyframes ambient-glow {
0%, 100% { opacity: 1; }
50% { opacity: 0.7; }
}
body::after {
animation: ambient-glow 8s ease-in-out infinite;
}
```
Use for: Landing page backgrounds, hero sections. Creates depth with brand colors without being distracting.
**Light theme variant**: Reduce opacity values (0.08, 0.06, 0.02) for subtler effect on light backgrounds.
#### Beta/Special Gradient
```css
background: linear-gradient(135deg, #8b5cf6, #ec4899);
```
Use for: Beta badges, special promotions, exclusive features.
---
## Typography
### Font Stack
#### Primary (Sans-serif)
```css
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
```
Use system fonts for optimal performance and native feel.
#### Monospace (Code/Technical)
```css
font-family: "SF Mono", Monaco, "Cascadia Code", "Roboto Mono", monospace;
```
Use for: Code snippets, technical information, job IDs.
### Type Scale
| Name | Size | Weight | Line Height | Usage |
|------|------|--------|-------------|-------|
| **Display** | 48-60px | 800 (extrabold) | 1.1 | Hero headlines |
| **H1** | 36-48px | 700 (bold) | 1.2 | Page titles |
| **H2** | 28-32px | 700 (bold) | 1.3 | Section headers |
| **H3** | 20-24px | 600 (semibold) | 1.4 | Card titles |
| **Body** | 16px | 400 (regular) | 1.6 | Main content |
| **Body Small** | 14px | 400 (regular) | 1.5 | Secondary content |
| **Caption** | 12px | 400 (regular) | 1.4 | Labels, hints |
### Text Styling
- **Gradient Text**: Apply brand gradient to hero headlines
- **Antialiasing**: Always use `-webkit-font-smoothing: antialiased`
- **Letter Spacing**: Default, slightly increased for ALL CAPS text
---
## Logo Usage
**Important**: Always use the Nomad Karaoke logo rather than generic emojis (like 🎤) in headers, navigation, and brand touchpoints.
### Primary Logo
The Nomad Karaoke logo consists of:
1. **Wordmark**: "NOMAD" stylized letters (pink #ff7acc)
2. **Tagline**: "KARAOKE" below (pink #ff7acc)
3. **Subline**: "WHERE EVERY SONG'S YOURS, SINGING!" (gold #ffdf6b)
### Logo Format Priority
1. **SVG (Preferred)**: Use `nomad-karaoke-logo.svg` for all web applications
- Vector format scales perfectly at any size
- Smaller file size
- Supports both light and dark themes
- Located in: `frontend/public/` directory
2. **PNG**: Use for contexts that don't support SVG
- `nomad-logo.png` - Standard PNG fallback
3. **GIF (Email Only)**: Use the hosted GIF **only** for email templates
- URL: `https://beveradb.github.io/public-images/Nomad-Karaoke-Logo-small-indexed-websafe-rectangle.gif`
- This hosted URL exists because emails cannot embed local assets
- Do NOT use this URL in web applications - use the local SVG instead
### Logo Files
- `nomad-karaoke-logo.svg` - Full vector logo (**primary choice**)
- `nomad-logo.png` - PNG fallback
- `favicon.ico`, `favicon-16x16.png`, `favicon-32x32.png` - Favicons
- `apple-touch-icon.png` - iOS home screen icon
### Logo Placement
- **Navigation**: Left-aligned, 32-40px height, use SVG
- **Footer**: Smaller, 24-32px height, use SVG
- **Email Header**: Centered, 130-180px width, use hosted GIF URL
### Logo Clear Space
Maintain clear space equal to the "N" letter height around all sides of the logo.
### Logo Don'ts
- Don't use emojis (🎤, 🎵) instead of the logo
- Don't change the logo colors
- Don't stretch or distort the logo
- Don't add effects (shadows, gradients, etc.)
- Don't place on busy backgrounds without contrast
- Don't use the logo smaller than 24px height
- Don't use the hosted GIF URL in web applications (use SVG)
---
## Iconography
### Icon Library
Use [Lucide Icons](https://lucide.dev/) for UI icons. They are:
- Open source
- Consistent stroke width
- Good visual balance
### Common Icons
| Icon | Name | Usage |
|------|------|-------|
| `MicVocal` | Microphone | Brand representation in nav (when logo not used) |
| `Music` | Music note | Audio/song related |
| `Music2` | Double notes | Empty states for songs |
| `Sparkles` | Stars | AI/magic features |
| `Video` | Video | Video output |
| `Youtube` | YouTube logo | YouTube integration |
| `Gift` | Gift box | Free credits, promotions |
| `Check` | Checkmark | Success, confirmation |
| `Loader2` | Spinner | Loading states |
### Icon Sizing
- **Small**: 16px (inline with text)
- **Medium**: 20-24px (buttons, lists)
- **Large**: 32-48px (feature highlights)
- **XL**: 64px+ (empty states, hero)
### Icon Colors
- Match text color in context
- Use brand pink (#ff7acc) for emphasis
- Use muted color for decorative icons
---
## UI Components
### Buttons
#### Primary Button (CTA)
```css
/* Default state uses darker pink for better white text contrast */
background-color: #ff5bb8;
color: #ffffff;
padding: 14px 28px;
border-radius: 12px;
font-weight: 600;
box-shadow: 0 0 20px rgba(255, 122, 204, 0.4);
transition: all 0.2s ease;
/* Hover - lightens to original brand pink */
background-color: #ff7acc;
box-shadow: 0 0 30px rgba(255, 122, 204, 0.6);
```
#### Secondary Button
```css
background-color: #252525;
color: #e5e5e5;
padding: 12px 24px;
border-radius: 8px;
border: 1px solid #2a2a2a;
/* Hover */
background-color: #333333;
```
#### Ghost Button
```css
background-color: transparent;
color: #888888;
padding: 8px 16px;
/* Hover */
color: #e5e5e5;
background-color: rgba(255, 255, 255, 0.05);
```
### Cards
```css
background-color: var(--card); /* #1a1a1a dark / #ffffff light */
border: 1px solid var(--card-border); /* #2a2a2a dark / #e2e8f0 light */
border-radius: 12px;
padding: 24px;
/* Hover effect (optional) */
transition: transform 0.2s ease, box-shadow 0.2s ease;
transform: translateY(-4px);
box-shadow: 0 20px 40px rgba(0, 0, 0, 0.3);
```
### Form Inputs
```css
background-color: #0f0f0f; /* Darker than card */
border: 1px solid #2a2a2a;
border-radius: 8px;
padding: 12px 16px;
color: #e5e5e5;
/* Focus */
border-color: #ff7acc;
box-shadow: 0 0 0 2px rgba(255, 122, 204, 0.2);
outline: none;
/* Placeholder */
color: #666666;
```
### Badges
#### Beta Badge
```css
background: linear-gradient(135deg, #8b5cf6, #ec4899);
color: #ffffff;
padding: 4px 12px;
border-radius: 20px;
font-size: 12px;
font-weight: 600;
```
#### Status Badges
Use semantic colors with 20% opacity backgrounds:
- Success: `background: rgba(34, 197, 94, 0.2); color: #22c55e;`
- Warning: `background: rgba(245, 158, 11, 0.2); color: #f59e0b;`
- Error: `background: rgba(239, 68, 68, 0.2); color: #ef4444;`
---
## Layout & Spacing
### Spacing Scale
Use a consistent 4px base unit:
| Name | Value | Usage |
|------|-------|-------|
| `xs` | 4px | Tight spacing |
| `sm` | 8px | Small gaps |
| `md` | 16px | Default gaps |
| `lg` | 24px | Section padding |
| `xl` | 32px | Large sections |
| `2xl` | 48px | Major sections |
| `3xl` | 64px | Page sections |
### Container Widths
- **Max content width**: 1200px (6xl)
- **Text content**: 672px (2xl) for readability
- **Form width**: 448px (md) for focused input
### Grid
Use CSS Grid or Flexbox with responsive breakpoints:
- Mobile: 1 column
- Tablet (md: 768px): 2 columns
- Desktop (lg: 1024px): 2-4 columns
### Border Radius
- **Small** (inputs, small buttons): 8px
- **Medium** (cards, buttons): 12px
- **Large** (feature cards, modals): 16px
- **Pill** (badges, tags): 20px or 9999px
---
## Email Design
### Email Structure
```html
<!DOCTYPE html>
<html>
<head>
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
line-height: 1.6;
color: #333333;
max-width: 600px;
margin: 0 auto;
padding: 20px;
background-color: #ffffff;
}
</style>
</head>
<body>
<!-- Header with logo -->
<div style="text-align: center; padding: 20px 0;">
<img src="logo-url" alt="Nomad Karaoke" width="150" />
</div>
<!-- Content -->
<div>...</div>
<!-- Footer -->
<div style="margin-top: 30px; padding-top: 20px; border-top: 1px solid #ff7acc; font-size: 12px; color: #666;">
...
</div>
</body>
</html>
```
### Email Colors
- **Background**: `#ffffff` (white for email client compatibility)
- **Text**: `#333333`
- **Primary Button**: `#ff7acc` with white text
- **Links**: `#3b82f6`
- **Dividers**: `#ff7acc` (brand pink) or `#eeeeee`
- **Footer text**: `#666666`
### Email Button
```html
<a href="URL" style="display: inline-block; background-color: #ff7acc; color: #ffffff; padding: 14px 28px; text-decoration: none; border-radius: 8px; font-weight: 600;">
Button Text
</a>
```
### Email Header
Always use the Nomad Karaoke logo image in email headers, not an emoji.
```html
<div style="text-align: center; padding: 20px 0;">
<a href="https://nomadkaraoke.com">
<img src="https://beveradb.github.io/public-images/Nomad-Karaoke-Logo-small-indexed-websafe-rectangle.gif"
alt="Nomad Karaoke"
style="max-width: 180px; height: auto; border-radius: 10px;" />
</a>
</div>
```
### Email Signature
Include consistent signature with:
- Profile image
- Name and title
- Social media icons
- Contact information
- Pink (#ff7acc) accent lines
---
## Tone & Voice
### Writing Style
- **Friendly but professional**: "Hi there!" not "Dear Customer"
- **Clear and concise**: Get to the point
- **Encouraging**: Celebrate user accomplishments
- **Helpful**: Guide users through actions
### Example Phrases
| Instead of... | Use... |
|---------------|--------|
| "Your order has been processed" | "Your karaoke video is ready!" |
| "Error occurred" | "Something went wrong - we're on it!" |
| "Please wait" | "Just a moment..." |
| "Submit" | "Create Karaoke" or "Get Started" |
### Emoji Usage
- Use sparingly for warmth
- Common: 🎤 (brand), 🎵 (music), ✨ (magic), 🎉 (celebration)
- Avoid in error messages
- Email subject lines: One emoji max
---
## Light/Dark Theme Requirements
### Overview
All Nomad Karaoke web application screens must support both light and dark themes. Dark theme is the default, but users should have easy access to toggle between themes.
### Implementation Requirements
1. **Theme Toggle**: Every screen/page must include a visible theme toggle
- Location: Typically in the header or settings area
- Use `next-themes` library for Next.js applications
- Toggle should be easily discoverable
2. **Default Theme**: Dark mode
- Respects system preference on first visit
- User preference is persisted
3. **Color Consistency**: All brand colors must work in both themes
- Brand pink (#ff7acc) remains consistent
- Background, card, and text colors adapt per theme tables above
- Ensure sufficient contrast in both modes
### next-themes Setup (Next.js)
```tsx
// app/layout.tsx or _app.tsx
import { ThemeProvider } from 'next-themes'
export default function RootLayout({ children }) {
return (
<html suppressHydrationWarning>
<body>
<ThemeProvider attribute="class" defaultTheme="dark" enableSystem>
{children}
</ThemeProvider>
</body>
</html>
)
}
// Theme toggle component
import { useTheme } from 'next-themes'
import { Sun, Moon } from 'lucide-react'
export function ThemeToggle() {
const { theme, setTheme } = useTheme()
return (
<button
onClick={() => setTheme(theme === 'dark' ? 'light' : 'dark')}
className="p-2 rounded-lg hover:bg-white/10"
aria-label="Toggle theme"
>
{theme === 'dark' ? <Sun size={20} /> : <Moon size={20} />}
</button>
)
}
```
### CSS Variables for Theming
```css
/* Dark theme (default) */
:root {
--background: #0f0f0f;
--card: #1a1a1a;
--card-border: #2a2a2a;
--text: #e5e5e5;
--text-muted: #888888;
}
/* Light theme */
.light {
--background: #f8fafc;
--card: #ffffff;
--card-border: #e2e8f0;
--text: #1e293b;
--text-muted: #64748b;
}
/* Brand colors stay consistent across themes */
:root {
--brand-pink: #ff5bb8; /* Darker - better white text contrast */
--brand-pink-hover: #ff7acc; /* Lighter - for hover states */
--brand-gold: #ffdf6b;
--brand-purple: #8b5cf6;
}
```
### Testing Checklist
- [ ] Theme toggle is visible on all pages
- [ ] Toggle persists preference across sessions
- [ ] All text is readable in both themes
- [ ] Brand colors render correctly in both themes
- [ ] Interactive elements have proper hover/focus states in both themes
- [ ] Radial background gradient adjusts opacity for light theme
---
## Implementation Reference
### CSS Variables (Recommended)
```css
:root {
/* Brand Colors */
/* Note: darker pink used as default for better white text contrast */
--brand-pink: #ff5bb8; /* Default - better contrast with white text */
--brand-pink-hover: #ff7acc; /* Hover - lighter, original brand pink */
--brand-gold: #ffdf6b;
--brand-purple: #8b5cf6;
--brand-blue: #3b82f6;
/* Semantic Colors */
--success: #22c55e;
--warning: #f59e0b;
--danger: #ef4444;
/* Dark Theme */
--bg: #0f0f0f;
--card: #1a1a1a;
--card-border: #2a2a2a;
--text: #e5e5e5;
--text-muted: #888888;
/* Spacing */
--space-xs: 4px;
--space-sm: 8px;
--space-md: 16px;
--space-lg: 24px;
--space-xl: 32px;
/* Border Radius */
--radius-sm: 8px;
--radius-md: 12px;
--radius-lg: 16px;
--radius-pill: 9999px;
}
```
### Tailwind Configuration
```javascript
// tailwind.config.js
module.exports = {
theme: {
extend: {
colors: {
brand: {
// Darker pink as default for better white text contrast
pink: '#ff5bb8',
'pink-hover': '#ff7acc', // Lighter for hover states
gold: '#ffdf6b',
purple: '#8b5cf6',
blue: '#3b82f6',
},
},
},
},
}
```
### React Component Example
```tsx
// Primary Button Component
export function PrimaryButton({ children, ...props }) {
return (
<button
className="bg-brand-pink hover:bg-brand-pink-hover text-white font-semibold px-7 py-3.5 rounded-xl transition-all shadow-[0_0_20px_rgba(255,122,204,0.4)] hover:shadow-[0_0_30px_rgba(255,122,204,0.6)]"
{...props}
>
{children}
</button>
)
}
```
---
## Checklist for Brand Compliance
### Landing Pages
- [ ] Uses brand pink (#ff5bb8) for primary CTAs with white text
- [ ] Logo displayed correctly
- [ ] Dark theme as default
- [ ] Gradient text on hero headlines
- [ ] Consistent typography scale
- [ ] Proper spacing (multiples of 4px)
### Application UI
- [ ] Logo in header (32-40px height)
- [ ] Brand pink for primary actions
- [ ] Consistent card styling
- [ ] Proper focus states with brand pink
- [ ] Dark/light theme toggle working
### Emails
- [ ] Header with logo image (not emoji)
- [ ] Brand pink buttons
- [ ] Pink divider lines
- [ ] Consistent footer with signature
- [ ] White background for compatibility
### Marketing Materials
- [ ] Logo with proper clear space
- [ ] Brand colors only
- [ ] Consistent typography
- [ ] No logo modifications
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.2 | 2026-01-08 | **Accessibility improvement**: Swapped brand pink colors for better white text contrast. Default `--brand-pink` is now `#ff5bb8` (darker) for buttons/CTAs, hover state `--brand-pink-hover` is now `#ff7acc` (lighter). The original lighter pink had insufficient contrast ratio with white text. |
| 1.1 | 2026-01-07 | Updated radial background gradient (now uses layered radial gradients with optional animation); Updated logo usage (SVG preferred, hosted GIF URL for emails only); Updated email header (use logo image, not emoji); Added Light/Dark Theme Requirements section; Fixed success color consistency (#22c55e) |
| 1.0 | 2026-01-07 | Initial brand guide created |
---
*This guide is the source of truth for Nomad Karaoke brand identity. For questions or clarifications, contact the design team.*
+29 -5
View File
@@ -110,17 +110,41 @@ Push to `main` triggers GitHub Actions:
3. Deploy to Cloud Run
4. Deploy frontend to Cloudflare Pages
### Manual Backend Deploy
### Infrastructure Changes (Pulumi)
**All infrastructure changes must go through PRs.** The CI pipeline automatically deploys Pulumi changes when PRs merge to main.
```bash
# Preview infrastructure changes
# Preview changes locally (requires GCP auth)
cd infrastructure
pulumi preview
pulumi preview --stack nomadkaraoke/karaoke-gen-infrastructure/prod
# Apply changes
pulumi up
# DO NOT run `pulumi up` locally - let CI handle deployment
# CI uses --skip-preview due to limited service account permissions
```
**What's managed by Pulumi:**
- Firestore database and indexes
- GCS buckets
- Cloud Run service configuration
- Cloud Tasks queues
- Secret Manager secrets
- Service accounts and IAM bindings
- Workload Identity Federation
- GCE instances (GitHub runners, encoding worker)
**Workflow:**
1. Make changes in the appropriate module under `infrastructure/`
- `modules/` - Core GCP resources (database, storage, secrets, IAM, etc.)
- `compute/` - VM definitions and startup scripts
- `config.py` - Shared configuration constants
- `__main__.py` - Entry point that wires modules together
2. Run `pulumi preview` locally to verify
3. Create PR and merge
4. CI deploys automatically
See `docs/LESSONS-LEARNED.md` for Pulumi CI gotchas.
### Manual Frontend Deploy
```bash
File diff suppressed because it is too large. Load diff
+106
View File
@@ -0,0 +1,106 @@
# Product Communication Guide
Guidelines for how to communicate about Nomad Karaoke to users, in marketing materials, UI copy, and support interactions.
## Core Principles
1. **Transparency over hype** - Set realistic expectations so users aren't surprised
2. **Avoid AI terminology** - "AI" has become a loaded term; say "our system" or "the tool" instead
3. **Acknowledge user effort** - The human review step is required, not optional
4. **Highlight real differentiators** - Real instrumentals (not covers), precise sync, multiple formats
---
## Key Facts to Communicate Accurately
### Timing
- **Accurate:** "Create a karaoke video in under 30 minutes"
- **Avoid:** "in minutes" (implies much faster than reality)
- Typical flow: ~20-30 minutes including human review
### Song Availability
- **Accurate:** "Any song" - this IS true
- Users can: search by artist/title, paste YouTube URL, or upload their own audio
- If users can get the audio from somewhere, we can make karaoke from it
### Lyrics Sync Quality
- **Accurate:** "Precise lyrics sync" - the timing/sync IS very accurate
- The sync (when words appear) is reliable and handled automatically
- The challenge is lyrics ACCURACY (correct words), not timing
### Human Review Requirement
- **Be clear:** Users ALWAYS need to review the transcribed lyrics
- For songs with clear vocals (country, ballads): few or no corrections needed
- For complex songs: expect 5-10 minutes of corrections
- Frame as "Review & Correct" not "Preview and fine-tune" (the latter sounds optional)
### Audio Separation
- **Accurate:** "Real instrumentals from the original song"
- **Avoid:** "Studio-quality separation" (oversells it)
- Key differentiator: NOT a cover band or MIDI recreation - it's the actual song
- Separation quality is very good but not perfect for all songs
### Backing Vocals Option
- Users can choose between:
- Clean instrumental (all vocals removed)
- Instrumental with backing vocals preserved
- This matters for songs with backing vocals; irrelevant for songs without
### Output Formats
- 4K MP4 karaoke video (main output)
- "With Vocals" version for sing-along practice
- CDG+MP3 ZIP for older/commercial karaoke systems
- All videos published to Nomad Karaoke YouTube channel (not user's channel)
### YouTube Integration
- **Accurate:** Videos are published to the Nomad Karaoke YouTube channel
- **Wrong:** "Upload to your YouTube channel" - we don't do this
- This is for easy sharing, not user channel integration
---
## Beta Tester Program
- Beta testers get up to 5 free karaoke video generations
- Mandatory feedback required after each video
- Must acknowledge they'll review/correct lyrics transcription
- Helps improve the system through real-world testing
---
## Terminology Preferences
| Avoid | Use Instead |
|-------|-------------|
| AI, AI-powered, AI-generated | "our system", "the tool", "automated" |
| Perfect lyrics | "precise sync", "accurate timing" |
| Studio-quality | "real instrumentals", "actual song" |
| In minutes | "in under 30 minutes" |
| Fine-tune (sounds optional) | "review and correct" (sounds required) |
| Upload to YouTube | "published to our YouTube channel" |
---
## FAQ Guidance
When answering user questions:
**Q: Is it fully automatic?**
A: The transcription and sync are automatic, but you'll need to review and correct any misheard words before the final video is generated. For clear songs, this might take just a minute. For complex songs, expect 5-10 minutes.
**Q: What songs work?**
A: Any song you can get audio for. Search our database, paste a YouTube link, or upload your own audio file.
**Q: What do I get?**
A: Multiple files - 4K karaoke video, a sing-along version with vocals, and CDG+MP3 for older systems. Videos are also published to our YouTube channel.
**Q: How long does it take?**
A: About 20-30 minutes total, including your review time.
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2026-01-07 | Initial guide created based on founder feedback |
+39 -1
View File
@@ -28,14 +28,51 @@
| Beta tester program | Working |
| Admin dashboard | Working |
| CI/CD self-hosted runner | Working (GCP) |
| E2E happy path test | Working (38 min full pipeline) |
| E2E happy path test | Working (~20-25 min full pipeline) |
| **White-label B2B portals** | Working (Vocal Star first tenant) |
## Known Issues
- CDG format generation requires additional style configuration
## Pending Work
(No pending work items)
## Recent Changes
- **White-Label B2B Portal Infrastructure** (2026-01-08): Added multi-tenant support for white-label karaoke portals. Vocal Star is the first tenant at `vocalstar.nomadkaraoke.com`. Features: subdomain-based tenant detection (frontend and backend), GCS-backed tenant config storage, per-tenant feature flags (audio search, file upload, YouTube, theme selection, etc.), dynamic branding (logo, colors, site title via CSS variables), locked themes per tenant, email domain restrictions for auth, and tenant-scoped jobs/users in Firestore. Backend: `TenantMiddleware` extracts tenant from X-Tenant-ID header, query param (dev only), or Host subdomain; `TenantService` loads config from GCS with caching. Frontend: Zustand store for tenant state, `TenantProvider` for initialization, `TenantLogo` component. Setup script: `scripts/setup-vocalstar-tenant.py`. See [ARCHITECTURE.md](ARCHITECTURE.md#multitenancy) and [API.md](API.md#tenant-config).
- **Encoding Worker Infrastructure Upgrade** (2026-01-08): Upgraded GCE encoding worker from c4-standard-8 (Intel) to c4d-highcpu-32 (AMD EPYC 9B45 Turin). Benchmarking showed **4.92x faster encoding** (135s vs 666s total). Key improvements: With Vocals 4K rendering 6x faster (54s vs 324s), 720p downscale 4.6x faster (11s vs 50s). Reduces total encoding time from ~11 min to ~2.3 min per job. See [archive/2026-01-08-performance-investigation.md](archive/2026-01-08-performance-investigation.md) for full benchmark methodology and results.
- **Theme Enforcement & Encoding Resilience** (2026-01-08): Added defense-in-depth theme enforcement - jobs without theme_id are now rejected at creation (JobManager) with a safety net at processing time (screens_worker). Fixed made-for-you webhook to apply default theme. Added retry logic with exponential backoff (3 retries, 2s→4s→8s) to GCE encoding service for transient connection failures during worker restarts. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#defense-in-depth-enforce-critical-requirements-at-multiple-layers).
- **Comprehensive Performance Optimization** (2026-01-08): Reduced lyrics processing from **16+ minutes to ~5 minutes** through multiple optimizations. Key changes: (1) NLTK cmudict preloading at startup saves 100-150s, (2) Langfuse callback handler preloading saves 200s, (3) Model warmup in `AgenticCorrector.from_model()` prevents parallel initialization race, (4) Parallel anchor sequence search (4 workers) reduces n-gram processing from 38s to ~10s. Added `/health/preload-status` endpoint for deployment verification. PR #236. See [archive/2026-01-08-performance-investigation.md](archive/2026-01-08-performance-investigation.md) and [LESSONS-LEARNED.md](LESSONS-LEARNED.md#preloading-heavy-resources-at-container-startup).
- **UI Simplification & Brand Accessibility** (2026-01-08): Simplified the "Create Karaoke Video" form by hiding advanced options (theme settings now admin-only, "Display As" behind toggle). Improved audio search dialog UX by hiding technical tracker names and showing user-friendly availability badges. Fixed brand pink color contrast for accessibility - swapped default/hover values so white text on pink buttons meets WCAG contrast ratio. See [BRAND-STYLE-GUIDE.md](BRAND-STYLE-GUIDE.md) v1.2.
- **SpaCy Preloading** (2026-01-08): Implemented SpaCy model preloading at container startup to eliminate 60+ second delay during agentic correction. The `en_core_web_sm` model is now loaded during FastAPI lifespan startup, and `PhraseAnalyzer`/`SyllablesMatchHandler` reuse the preloaded model. Added timing logs to verify performance. See [archive/2026-01-08-spacy-preload-plan.md](archive/2026-01-08-spacy-preload-plan.md).
- **Thread-Safe LangChain Model Initialization** (2026-01-08): Fixed race condition in `LangChainBridge` where parallel threads could all try to initialize the AI model simultaneously, causing 6+ minute delays. Added double-checked locking with `threading.Lock()`. PR #232. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#thread-safe-lazy-initialization-in-shared-components).
- **Brand Consistency & Style Guide** (2026-01-07): Implemented unified brand theming across all Nomad Karaoke frontends. Primary color changed from blue to brand pink (#ff7acc). Added comprehensive [BRAND-STYLE-GUIDE.md](BRAND-STYLE-GUIDE.md) for human and LLM reference. Includes: CSS variables for light/dark mode, MUI theme updates for lyrics transcriber, email template branding, and Playwright visual tests. See [BRAND-STYLE-GUIDE.md](BRAND-STYLE-GUIDE.md).
- **Audio Search Display Override** (2026-01-06): Added optional "Display As" fields to audio search, allowing users to search for audio by one artist/title (e.g., "Jeremy Kushnier") but display a different artist/title on title screens and filenames (e.g., "Footloose (Broadway Cast)"). Useful for Broadway casts, covers, remixes where tracker metadata differs from desired display. See [API.md](API.md#audio-search).
- **Infrastructure Modularization** (2026-01-06): Refactored `infrastructure/__main__.py` from 2,602 lines to 339 lines (87% reduction). Split into organized modules: `modules/` for core GCP resources, `compute/` for VMs and startup scripts, `config.py` for shared constants. Extracted embedded encoding worker Python app to `backend/services/gce_encoding/`. See `infrastructure/README.md` for new structure and `infrastructure/docs/PHASE5-PACKER-IMAGE-PLAN.md` for planned Packer image optimization.
- **Email Notification System** (2026-01-06): Added automated email notifications for job completion and user action reminders. Features: GCS-backed HTML email templates with fallback defaults, SendGrid with CC support, auto-completion emails on job finish, idle reminder emails via Cloud Tasks (5-min delay for blocking states), admin UI buttons to copy message or send email manually. Endpoints: `GET /api/admin/jobs/{id}/completion-message`, `POST /api/admin/jobs/{id}/send-completion-email`. Feature flag `ENABLE_AUTO_EMAILS` (default: false). See [API.md](API.md#email-notifications-admin) and [ARCHITECTURE.md](ARCHITECTURE.md#video-worker-orchestrator).
- **GCE Preview Encoding Unified with LocalPreviewEncodingService** (2026-01-06): GCE preview encoding now uses the same `LocalPreviewEncodingService` from the installed wheel, eliminating 100 lines of duplicated FFmpeg logic. Preview videos are now identical across local CLI, Cloud Run, and GCE worker environments. Benefits include: consistent encoding settings (480x270, 24fps, crf 28), proper FFmpeg filter path escaping for special characters, hardware acceleration (NVENC) when available, and custom font support. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#unify-preview-encoding-with-localpreviewencodingservice).
- **GCE Encoding Worker Python 3.13** (2026-01-06): Upgraded GCE encoding worker from Debian's Python 3.11 to Python 3.13 built from source. Uses dedicated virtual environment at `/opt/encoding-worker/venv`, removing the need for `--break-system-packages`. Aligns the encoding worker with CI/Cloud Run Python version.
- **GCE Encoding Unified with LocalEncodingService** (2026-01-06): GCE encoding worker now uses the same `LocalEncodingService` as the local CLI, eliminating duplicated encoding logic. Output files now have proper names (`Artist - Title (Final Karaoke Lossless 4k).mp4` instead of `output_4k_lossless.mp4`) and include title/end screen concatenation. Wheel deployed to GCS; worker installs at job start for hot updates without VM restart. Removed all fallback logic - single code path for consistent output across CLI, Cloud Run, and GCE. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#unify-encoding-logic-with-gcs-wheel-deployment).
- **Cloud Distribution Matches Local CLI** (2026-01-06): Fixed cloud distribution (Dropbox uploads) to match local CLI output structure. The `lyrics/` folder now includes: proper filenames (`(Lyrics Corrections).json` instead of `(Karaoke).json`), all reference lyrics files (`(Lyrics Genius).txt`, `(Lyrics Spotify).txt`, etc.), `(With Vocals).mkv`, and `previews/` subfolder with preview ASS files. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#cloud-distribution-must-match-local-cli-output-structure).
- **Agentic Correction Performance** (2026-01-05): Optimized agentic AI correction from ~5 minutes to ~55 seconds for 20 gaps (~5-6x speedup). Fixed anti-pattern of creating new model instance per gap (caused repeated 2s warm-up overhead). Now creates model once and processes gaps in parallel using ThreadPoolExecutor (default 5 workers). Configure via `AGENTIC_MAX_PARALLEL_GAPS` env var. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#reuse-llm-model-instances-across-operations).
- **Worker Logs Subcollection** (2026-01-04): Moved `worker_logs` from embedded array in job documents to Firestore subcollection (`jobs/{job_id}/logs`). Fixes job failures when logs exceed 1MB (job 501258e1 had 1.26MB of logs). New logs stored with 30-day TTL via Firestore TTL policy. Feature flag `USE_LOG_SUBCOLLECTION=true` (default). See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#firestore-document-1mb-limit-with-embedded-arrays).
- **GCE Encoding Response Fixes** (2026-01-04): Fixed multiple response format mismatches with GCE encoding worker. The worker returns `output_files` as a list of paths, not a dict with format keys - added conversion logic. Also added defensive type checking for status responses that could be lists. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#external-service-response-format-mismatches).
@@ -81,6 +118,7 @@
| [ARCHITECTURE.md](ARCHITECTURE.md) | System design, data flow, integrations |
| [DEVELOPMENT.md](DEVELOPMENT.md) | Local setup, testing, deployment |
| [API.md](API.md) | Backend endpoint reference |
| [BRAND-STYLE-GUIDE.md](BRAND-STYLE-GUIDE.md) | Brand colors, typography, UI patterns |
| [LESSONS-LEARNED.md](LESSONS-LEARNED.md) | Key insights for future agents |
| [STRIPE-SETUP.md](STRIPE-SETUP.md) | Payment integration setup |
| [MOBILE-UX-BEST-PRACTICES.md](MOBILE-UX-BEST-PRACTICES.md) | Mobile/responsive design guidelines |
@@ -0,0 +1,65 @@
# Brand Consistency Implementation - 2026-01-07
## Summary
Implemented unified brand theming across all Nomad Karaoke frontends and created a comprehensive brand style guide for human and LLM reference.
## Key Changes
### Frontend (Next.js)
- Added `ThemeToggle` component using `next-themes`
- Made all components theme-aware (light/dark mode support)
- Changed primary color from blue to brand pink (#ff7acc)
- Updated CSS variables for consistent theming
- Added brand screenshots for visual verification
### Lyrics Transcriber Frontend (MUI)
- Updated MUI theme with brand colors
- Made timeline, modals, and controls theme-aware
- Fixed mobile responsiveness issues
- Added mobile TAP button for manual sync
- Improved WordDivider for mobile/desktop parity
### Email Templates
- Refactored to use reusable helper methods (`_build_email_html`, `_get_email_header`, `_get_email_footer`)
- Fixed success color to match brand guide (#22c55e)
- Updated preview script to use actual email service instead of duplicated content
- Changed email header from emoji to logo image
### Brand Style Guide (docs/BRAND-STYLE-GUIDE.md)
- Created comprehensive v1.1 guide covering:
- Color palette (primary, supporting, semantic colors)
- Typography (font stack, type scale)
- Logo usage (SVG preferred, hosted GIF for emails)
- UI components (buttons, cards, badges)
- Email design standards
- Light/dark theme requirements with `next-themes` examples
- Implementation reference (CSS variables, Tailwind config)
## Decisions Made
1. **Dark theme as default** - More common in modern applications, easier on the eyes for karaoke use
2. **SVG logo preferred** - Scalable, smaller file size; GIF only for emails where SVG not supported
3. **No emojis in headers** - Always use the Nomad Karaoke logo for brand consistency
4. **Hosted logo URL for emails only** - Local SVG for webapp, hosted GIF only when necessary
## Files Changed
### Core Brand Files
- `docs/BRAND-STYLE-GUIDE.md` - New comprehensive guide
- `frontend/components/ThemeToggle.tsx` - New theme toggle
- `frontend/app/globals.css` - Updated CSS variables
### Email System
- `backend/services/email_service.py` - Refactored with reusable helpers
- `scripts/preview-emails.py` - Now uses real email service
### Lyrics Transcriber (submodule)
- Multiple component files updated for theme awareness
- `src/theme.ts` - Updated MUI theme with brand colors
## Future Considerations
- Consider adding ambient glow animation to landing page background
- Monitor theme toggle usage to ensure discoverability
- Keep brand guide updated as new components are added
@@ -0,0 +1,467 @@
# Performance Investigation: Karaoke Generation Pipeline
**Date**: 2026-01-08
**Status**: Phase 2 Complete - Encoding Infrastructure Upgraded
**Goal**: Reduce total karaoke generation time from ~30 minutes to ~10 minutes
## Executive Summary
We achieved a **~50% reduction in lyrics processing time** (from ~13 min to ~6.5 min) through preloading optimizations. However, the overall pipeline still takes ~25-30 minutes. Further optimization is needed to reach the 10-minute target.
**Phase 2 Focus**: Encoding infrastructure optimization. Benchmarking confirms the current GCE encoding worker (c4-standard-8) is **1.75x slower** than an M3 Max MacBook Pro, validating the need for infrastructure upgrades.
### Key Achievements (PR #236)
| Optimization | Before | After | Savings |
|-------------|--------|-------|---------|
| NLTK cmudict preloading | 100-150s lazy load | ~2s at startup | **~150s** |
| Langfuse handler preloading | 201s lazy load | ~2s at startup | **~200s** |
| Model warmup (eager init) | 300s+ parallel timeout | 0s (pre-warmed) | **~300s** |
| Parallel anchor search | 38s sequential | ~10s parallel | **~28s** |
| **Total Lyrics Stage** | ~13 min | **~6.5 min** | **~50%** |
### Remaining Bottlenecks
| Stage | Current Duration | Target | Gap |
|-------|-----------------|--------|-----|
| Audio separation (Modal) | ~3 min | 2 min | 1 min |
| Lyrics processing | ~6.5 min | 3 min | 3.5 min |
| Video rendering | ~6 min | 3 min | 3 min |
| Final encoding (GCE) | ~9 min | 2 min | 7 min |
| **Total (excl. human review)** | **~25 min** | **10 min** | **15 min** |
---
## Empirical Data: E2E Test Comparison
### Before Optimization (Job 6b5a354e - 09:17 UTC)
```
Test song: piri - dog
Total E2E test time: 31.4 minutes
Stage Breakdown:
├─ Steps 1-5 (Setup/Audio selection): 25 seconds
├─ Step 6 (Lyrics processing): 12 min 53 sec ← PRIMARY BOTTLENECK
├─ Step 7 (Lyrics review UI): 2 min 12 sec
├─ Step 8 (Video render + instrumental): 6 min 3 sec
└─ Step 9 (Final encoding): 9 min 28 sec
```
### After Optimization (Job b099c91b - 10:02 UTC)
```
Test song: piri - dog
Total E2E test time: ~19 min (encoding failed, unrelated bug)
Stage Breakdown:
├─ Steps 1-5 (Setup/Audio selection): 32 seconds
├─ Step 6 (Lyrics processing): 6 min 28 sec ← 50% FASTER!
├─ Step 7 (Lyrics review UI): 2 min 20 sec
├─ Step 8 (Video render + instrumental): 6 min 10 sec
└─ Step 9 (Final encoding): FAILED (MOV→MP4 bug, unrelated)
```
### Improvement Summary
| Stage | Before | After | Change |
|-------|--------|-------|--------|
| Lyrics Processing | 12m 53s | **6m 28s** | **-50%** |
| Video Rendering | 6m 3s | 6m 10s | ~same |
| Encoding | 9m 28s | N/A (failed) | expected same |
---
## Complete Pipeline Timing Breakdown
Based on analysis of multiple jobs, here's the detailed breakdown of each stage:
### Stage 1: Audio Worker (~3 minutes)
| Sub-stage | Duration | Notes |
|-----------|----------|-------|
| Job creation → Worker start | 13s | Cloud Tasks dispatch |
| Modal audio separation | 72s | GPU-accelerated, fast |
| Stems download from Modal | 47s | 8 files from Modal storage |
| Stage 2 local separation | 23s | CPU-bound on Cloud Run |
| **Total Audio Worker** | **~3 min** | Acceptable |
**Optimization opportunities**:
- [ ] Faster Modal model (research needed)
- [ ] Parallel stem downloads
- [ ] Skip Stage 2 separation for some use cases
### Stage 2: Lyrics Worker (~6.5 minutes after optimization)
| Sub-stage | Before | After | Notes |
|-----------|--------|-------|-------|
| AudioShake API call | 78s | 78s | External API, hard to optimize |
| NLTK cmudict load | 106s | **~0s** | Preloaded at startup |
| SpaCy model load | 63s | **~0s** | Preloaded at startup |
| Langfuse init | 201s | **~0s** | Preloaded at startup |
| Anchor sequence search | 38s | **~10s** | Parallelized |
| Agentic correction | 300s+ | **~60s** | Model pre-warmed |
| Output generation | ~30s | ~30s | - |
| **Total Lyrics Worker** | **~16 min** | **~6.5 min** | **-60%** |
**Optimization opportunities**:
- [ ] Local Whisper transcription (eliminates AudioShake 78s)
- [ ] GCE-based lyrics worker (eliminates all cold start overhead)
- [ ] Faster agentic model (smaller model, fewer gaps)
- [ ] Skip agentic correction for simple songs
### Stage 3: Video Rendering (~6 minutes)
| Sub-stage | Duration | Notes |
|-----------|----------|-------|
| ASS subtitle generation | ~5s | Fast, CPU-only |
| FFmpeg video render | ~5-6 min | CPU-bound, single-threaded |
| Upload to GCS | ~10s | - |
**Optimization opportunities**:
- [ ] GPU-accelerated FFmpeg (NVENC on GCE)
- [ ] Lower resolution preview during render stage
- [ ] Parallel encoding of segments
### Stage 4: Final Encoding (~9 minutes)
| Sub-stage | Duration | Notes |
|-----------|----------|-------|
| GCE job dispatch | ~5s | - |
| File downloads from GCS | ~30s | Videos + stems |
| Title/karaoke/end concatenation | ~2 min | FFmpeg concat |
| Multi-format encoding | ~6 min | 4K lossless + 720p |
| Upload results to GCS | ~30s | - |
**Optimization opportunities**:
- [ ] GPU encoding (NVENC) - would reduce 6 min → ~1-2 min
- [ ] Skip 4K lossless for most users
- [ ] Parallel format encoding
- [ ] Pre-rendered title/end screens
---
## Optimization Ideas: Not Yet Implemented
### High Impact, Medium Effort
#### 1. GPU-Accelerated Encoding on GCE
**Current**: GCE worker uses CPU encoding (libx264)
**Proposed**: Add NVIDIA GPU to GCE VM, use NVENC
**Expected savings**: 6 min → 1-2 min (~4-5 min saved)
**Effort**: 1-2 days (Pulumi changes, FFmpeg flags)
```python
# Current (CPU)
ffmpeg -i input.mkv -c:v libx264 -preset medium output.mp4
# With GPU (NVENC)
ffmpeg -hwaccel cuda -i input.mkv -c:v h264_nvenc -preset p4 output.mp4
```
#### 2. Local Whisper Transcription
**Current**: AudioShake API call takes 78 seconds
**Proposed**: Run Whisper locally on Modal GPU or GCE
**Expected savings**: 78s → ~30s (API overhead eliminated)
**Effort**: 1 week (integration, testing, fallback logic)
#### 3. GCE Lyrics Worker
**Current**: Lyrics processing on Cloud Run (cold starts, ephemeral filesystem)
**Proposed**: Dedicated GCE VM with persistent cache
**Expected savings**: Additional 1-2 min (no remaining init overhead)
**Effort**: 2-3 days (infrastructure + integration)
### Medium Impact, Low Effort
#### 4. Skip 4K Lossless for Web Users
**Current**: Always encode 4K lossless + 720p
**Proposed**: Only encode 720p by default, 4K on request
**Expected savings**: ~3 min per job
**Effort**: 1 day (API flag, conditional encoding)
#### 5. Pre-rendered Title/End Screens
**Current**: Title screens rendered per-job
**Proposed**: Pre-render common templates, composite at encode time
**Expected savings**: ~1 min
**Effort**: 2 days
#### 6. Parallel Multi-Format Encoding
**Current**: Formats encoded sequentially (4K then 720p)
**Proposed**: Encode both formats in parallel
**Expected savings**: ~2-3 min (depends on CPU cores)
**Effort**: 1 day
### Lower Impact, Research Needed
#### 7. Faster Audio Separation Model
**Current**: Modal uses bs_roformer model
**Proposed**: Research faster models with acceptable quality
**Expected savings**: Unknown
**Effort**: Research project
#### 8. Smart Agentic Correction Skipping
**Current**: Always run agentic correction
**Proposed**: Skip for songs with high-confidence transcription
**Expected savings**: ~60s for some songs
**Effort**: 2 days (confidence scoring, threshold tuning)
#### 9. Cached Lyrics from Previous Jobs
**Current**: Each job fetches/transcribes fresh
**Proposed**: Cache lyrics by audio hash, reuse if available
**Expected savings**: Up to 5+ min for repeat songs
**Effort**: 1 day
---
## Implementation Status
### Completed (PR #236)
| Fix | File | Status |
|-----|------|--------|
| NLTK cmudict preloading | `backend/services/nltk_preloader.py` | ✅ Deployed |
| Langfuse handler preloading | `backend/services/langfuse_preloader.py` | ✅ Deployed |
| SpaCy model preloading | `backend/services/spacy_preloader.py` | ✅ Deployed |
| Model warmup method | `langchain_bridge.py` | ✅ Deployed |
| Parallel anchor search | `anchor_sequence.py` | ✅ Deployed |
| Preload status endpoint | `/health/preload-status` | ✅ Deployed |
### Verification
Production endpoint confirms all preloaders working:
```bash
curl https://api.nomadkaraoke.com/api/health/preload-status
```
```json
{
"status": "ok",
"message": "All resources preloaded",
"spacy": {"preloaded": true, "vocab_size": 764},
"nltk": {"preloaded": true, "entries": 123455},
"langfuse": {"preloaded": true, "handler_type": "LangchainCallbackHandler"}
}
```
### Not Started
| Optimization | Priority | Effort | Expected Impact |
|-------------|----------|--------|-----------------|
| GPU encoding (NVENC) | High | 2 days | -4 min |
| Skip 4K lossless | High | 1 day | -3 min |
| Local Whisper | Medium | 1 week | -1 min |
| GCE lyrics worker | Medium | 3 days | -2 min |
| Parallel format encoding | Medium | 1 day | -2 min |
---
## Phase 2: Encoding Infrastructure Benchmarking
### Motivation
The GCE encoding worker (final encoding stage) currently takes ~9-11 minutes. User hypothesis: the current c4-standard-8 instance is underperforming compared to modern hardware (e.g., M3 MacBook Pro).
**Important constraint**: GPU encoding (NVENC) is NOT viable because:
1. We deliberately use `libx264` for quality
2. `libass` subtitle rendering is CPU-only and cannot use GPU
3. The heaviest operations (4K ASS overlay) require CPU
### Benchmark Methodology
Created `scripts/benchmark_encoding.py` and `scripts/benchmark_encoding_gce.sh` to measure actual encoding operations using the same test data (job `fddad04d` - "piri - dog"):
**Test files**:
- `karaoke.ass` - ASS subtitle file
- `vocals.flac` / `instrumental_clean.flac` - Audio stems
- `title.mov` / `end.mov` - Title/end screen videos
- `background.png` - Nomad theme 4K background
- `font.ttf` - AvenirNext-Bold font
**Benchmark stages** (matching actual production code paths):
1. Preview Video (480x270 with ASS overlay) - `LocalPreviewEncodingService`
2. With Vocals Video (4K with ASS overlay) - `VideoGenerator.generate_video`
3. Remux with Instrumental - `LocalEncodingService.remux_with_instrumental`
4. Convert Title/End MOV to MP4 - `LocalEncodingService.convert_mov_to_mp4`
5. Lossless 4K Concat - `LocalEncodingService.encode_lossless_mp4`
6. Lossy 4K (AAC audio) - `LocalEncodingService.encode_lossy_mp4`
7. MKV (FLAC audio) - `LocalEncodingService.encode_lossless_mkv`
8. 720p Downscale - `LocalEncodingService.encode_720p`
### Benchmark Results: Current Infrastructure
#### System Specifications
| Spec | M3 Max MacBook Pro | GCE c4-standard-8 |
|------|-------------------|-------------------|
| CPU | Apple M3 Max | Intel Xeon Platinum 8581C @ 2.30GHz |
| Cores | 14 (10P + 4E) | 8 vCPUs (4 physical + HT) |
| Memory | 36 GB | 29 GB |
| FFmpeg | 8.0.1 (Homebrew) | 7.0.2-static (John Van Sickle) |
| Architecture | ARM64 (Apple Silicon) | x86_64 |
#### Detailed Results
| Stage | M3 Max | GCE c4-standard-8 | Ratio |
|-------|--------|-------------------|-------|
| Preview (480x270 ASS) | 50.12s | 110.36s | **2.2x slower** |
| **With Vocals (4K ASS)** | 200.71s | 324.16s | **1.6x slower** |
| Remux with Instrumental | 0.21s | 0.24s | ~same |
| Convert Title MOV→MP4 | 1.87s | 3.58s | 1.9x slower |
| Convert End MOV→MP4 | 1.72s | 3.54s | 2.1x slower |
| **Lossless 4K Concat** | 90.95s | 164.86s | **1.8x slower** |
| Lossy 4K (AAC) | 7.03s | 8.85s | 1.3x slower |
| MKV (FLAC) | 0.54s | 0.40s | ~same |
| **720p Downscale** | 27.74s | 50.20s | **1.8x slower** |
| **TOTAL** | **380.91s (6.3 min)** | **666.19s (11.1 min)** | **1.75x slower** |
#### Key Findings
1. **Overall**: GCE c4-standard-8 is **1.75x slower** than M3 Max MacBook Pro
2. **Heaviest operations** (CPU-bound libx264 + libass):
- With Vocals (4K ASS): 324s GCE vs 201s Mac (61% of GCE total)
- Lossless 4K Concat: 165s GCE vs 91s Mac (25% of GCE total)
- These two operations account for **73%** of total GCE encoding time
3. **Root causes**:
- Apple M3 Max has significantly better single-threaded performance
- More CPU cores (14 vs 8)
- Better memory bandwidth and cache architecture
- libass subtitle rendering is particularly demanding
### Optimization Opportunities
Since GPU is NOT an option (libass limitation), focus on:
1. **More powerful CPU instance types**:
- More cores for parallel FFmpeg operations
- Higher clock speeds for single-threaded libass rendering
- Better memory bandwidth
2. **FFmpeg optimizations**:
- Explicit thread count tuning
- Preset adjustments where quality loss is acceptable
- Parallel output format encoding
3. **Alternative machine families**:
- c3/c3d (Intel Sapphire Rapids, higher clocks)
- c4a (AMD EPYC Genoa, high core count)
- n2/n2d (balanced performance/cost)
- t2a (Arm-based Ampere Altra, potentially better perf/watt)
### Next Steps: Infrastructure Testing
After researching latest GCP compute options (C4, C4A, C4D, C3, C3D series), selected 5 candidates focusing on:
- High single-thread performance (critical for libass subtitle rendering)
- Good multi-thread scaling (for libx264 encoding)
- Latest generation CPUs with highest clock speeds
#### Selected Candidates
| # | Instance Type | vCPUs | CPU | Clock Speed | Memory | Hypothesis |
|---|--------------|-------|-----|-------------|--------|------------|
| **Baseline** | c4-standard-8 | 8 | Intel Emerald Rapids | 2.3/3.9 GHz | 30 GB | Current production |
| 1 | **c4d-highcpu-16** | 16 | AMD EPYC Turin (5th gen) | 4.1 GHz turbo | 30 GB | **Newest, fastest CPU** |
| 2 | **c4-highcpu-16** | 16 | Intel Granite Rapids (6th gen) | 3.9/4.2 GHz | 32 GB | 2x cores, same vendor |
| 3 | **c4-highcpu-32** | 32 | Intel Granite Rapids (6th gen) | 3.9/4.2 GHz | 64 GB | 4x cores, max parallelism |
| 4 | **c3d-highcpu-30** | 30 | AMD EPYC Genoa (4th gen) | 3.3 GHz | 59 GB | AMD high core count |
| 5 | **c4a-highcpu-16** | 16 | Google Axion (ARM Neoverse V2) | - | 32 GB | ARM architecture test |
**Rationale**:
- **C4D**: Latest AMD Turin with 4.1GHz max boost - likely best single-thread perf
- **C4 16/32**: Intel Granite Rapids scaling test - same arch as current, more cores
- **C3D**: AMD Genoa known for excellent multi-thread - test FFmpeg scaling
- **C4A**: ARM Axion to test if Apple Silicon-like performance translates to GCP ARM
**Benchmark scripts**:
- `scripts/benchmark_encoding_gce.sh` - Shell script for GCE VMs
- `scripts/benchmark_candidates.sh` - Orchestrates multi-VM testing
#### Candidate Test Results
| Instance Type | CPU | Total Time | vs Baseline | With Vocals (4K) | Lossless Concat | 720p | Status |
|--------------|-----|------------|-------------|------------------|-----------------|------|--------|
| c4-standard-8 (baseline) | Intel Xeon 8581C | 666.19s | 1.00x | 324.16s | 164.86s | 50.20s | ✅ Complete |
| **c4d-highcpu-32** | **AMD EPYC 9B45 (Turin)** | **135.27s** | **4.92x** | **53.89s** | **33.97s** | **10.84s** | ✅ **WINNER** |
| c4d-highcpu-16 | AMD EPYC 9B45 (Turin) | 220.00s | 3.03x | 93.73s | 57.99s | 19.94s | ✅ Complete |
| c4a-highcpu-16 | Google Axion (ARM) | 248.07s | 2.69x | 114.13s | 59.54s | 19.26s | ✅ Complete |
| c4-highcpu-16 | Intel Xeon 8581C | 308.89s | 2.16x | 135.38s | 81.10s | 24.93s | ✅ Complete |
| M3 Max MacBook (reference) | Apple M3 Max | 380.91s | 1.75x | 200.71s | 90.95s | 27.74s | ✅ Complete |
| c4-highcpu-32 | Intel Xeon 8581C | - | - | - | - | - | ❌ Quota limit |
| c3d-highcpu-30 | AMD EPYC Genoa | - | - | - | - | - | ❌ Quota limit |
#### Key Findings
1. **c4d-highcpu-32 (AMD EPYC 9B45 Turin) is the clear winner**:
- **4.92x faster** than baseline (135s vs 666s)
- **6.01x faster** on the heaviest operation (With Vocals 4K ASS: 54s vs 324s)
- Even faster than M3 Max MacBook Pro by 2.8x!
- Doubling cores 16→32 gave 1.63x additional speedup (220s → 135s)
2. **Scaling analysis**:
- c4d-highcpu-16: 220s (3.03x faster)
- c4d-highcpu-32: 135s (4.92x faster)
- Near-linear scaling for this workload - FFmpeg efficiently uses all 32 cores
3. **AMD dominates Intel for this workload**:
- c4d-highcpu-16 (AMD): 220s
- c4-highcpu-16 (Intel): 309s
- AMD is 1.4x faster at same core count
4. **ARM (Axion) competitive but not optimal**:
- c4a-highcpu-16: 248s (2.69x faster)
- Good performance but AMD x86 still wins
#### Recommendation
**Deploy c4d-highcpu-32 as the new encoding worker**:
- Reduces encoding time from ~11 min to ~2.3 min (4.92x faster)
- Overall job time reduction: ~9 min saved per job
- Cost is justified by GCP free credits and dramatically improved UX
---
## Target Timeline to 10-Minute Goal
| Phase | Optimization | Cumulative Time |
|-------|-------------|-----------------|
| Baseline | Current state | ~25 min |
| Done | NLTK/Langfuse/SpaCy preload | ~19 min |
| Next | GPU encoding (NVENC) | ~15 min |
| Next | Skip 4K lossless | ~12 min |
| Next | Local Whisper | ~11 min |
| Stretch | GCE lyrics worker | ~10 min |
---
## Debugging Tips
### Check Preload Status
```bash
curl https://api.nomadkaraoke.com/api/health/preload-status | jq
```
### Check GCE Encoding Worker Logs
```bash
gcloud compute ssh encoding-worker --zone=us-central1-a --project=nomadkaraoke \
--command="journalctl -u encoding-worker --since '30 min ago' | tail -100"
```
### Monitor E2E Test Progress
```bash
gh run view <run_id> --log 2>&1 | grep -E 'STEP.*COMPLETE'
```
### Check FFmpeg Hardware Acceleration
```bash
curl https://api.nomadkaraoke.com/api/health/detailed | jq '.ffmpeg'
```
---
## References
- PR #232: Thread-safe locking for LangChainBridge
- PR #233: SpaCy model preloading
- PR #236: Comprehensive performance optimization (this work)
- [LESSONS-LEARNED.md - Preloading Heavy Resources](../LESSONS-LEARNED.md#preloading-heavy-resources-at-container-startup)
- [LESSONS-LEARNED.md - Thread-Safe Lazy Initialization](../LESSONS-LEARNED.md#thread-safe-lazy-initialization-in-shared-components)
@@ -0,0 +1,270 @@
# Implementation Plan: SpaCy Preloading at Container Startup
**Date**: 2026-01-08
**Status**: Ready for implementation
**Related Issue**: Job `2ccbdf6b` slow execution investigation
**Related PR**: #232 (thread-safety fix - already merged)
## Background
Job `2ccbdf6b` (ABBA - Waterloo) took 17 minutes instead of ~5 minutes expected. Investigation revealed two issues:
1. **FIXED (PR #232)**: Race condition in `LangChainBridge` where 5 parallel threads all tried to initialize the AI model simultaneously, causing 6+ minute delays.
2. **PENDING (this plan)**: SpaCy model loading takes ~63 seconds even on warm Cloud Run instances. This happens when `PhraseAnalyzer` is first instantiated during request processing.
## Problem Details
From job `2ccbdf6b` worker logs:
- `04:10:21` - "Initializing PhraseAnalyzer with language model: en_core_web_sm"
- `04:11:24` - "Initialized AnchorSequenceFinder" (63 seconds later)
Key observations:
- **SpaCy model**: `en_core_web_sm` (~14MB) is already pre-downloaded in `backend/Dockerfile.base` line 64
- **Cloud Run config**: `min-instances: 4` configured - instances are warm
- **Root cause**: SpaCy loads lazily when first needed during request processing, not at container startup
- **Cloud Run I/O**: Filesystem access on Cloud Run can be slow, causing extended load times
## Solution: Preload SpaCy at Container Startup
Load SpaCy during FastAPI lifespan handler (before accepting requests) and make it available as a singleton for reuse.
## Files to Create/Modify
### 1. NEW: `backend/services/spacy_preloader.py`
Create a new module to handle SpaCy preloading:
```python
"""SpaCy model preloader for container startup.
Loads SpaCy models at container startup to avoid slow loading during request processing.
Cloud Run filesystem I/O can cause 60+ second delays when loading SpaCy models lazily.
"""
import logging
import time
from typing import Optional
logger = logging.getLogger(__name__)
# Singleton storage for preloaded models
_preloaded_models: dict = {}
def preload_spacy_model(model_name: str = "en_core_web_sm") -> None:
"""Preload a SpaCy model at startup.
Args:
model_name: The SpaCy model to load (default: en_core_web_sm)
"""
global _preloaded_models
if model_name in _preloaded_models:
logger.info(f"SpaCy model '{model_name}' already preloaded")
return
logger.info(f"Preloading SpaCy model '{model_name}'...")
start_time = time.time()
try:
import spacy
nlp = spacy.load(model_name)
_preloaded_models[model_name] = nlp
elapsed = time.time() - start_time
logger.info(f"SpaCy model '{model_name}' preloaded in {elapsed:.2f}s")
except Exception as e:
logger.error(f"Failed to preload SpaCy model '{model_name}': {e}")
raise
def get_preloaded_model(model_name: str = "en_core_web_sm") -> Optional[object]:
"""Get a preloaded SpaCy model if available.
Args:
model_name: The SpaCy model name
Returns:
The preloaded SpaCy Language object, or None if not preloaded
"""
return _preloaded_models.get(model_name)
def is_model_preloaded(model_name: str = "en_core_web_sm") -> bool:
"""Check if a SpaCy model has been preloaded."""
return model_name in _preloaded_models
```
### 2. MODIFY: `backend/main.py`
Add SpaCy preloading to the lifespan handler. Current code is at lines 62-80.
**Add import at top of file:**
```python
from backend.services.spacy_preloader import preload_spacy_model
```
**Modify lifespan function (around line 62):**
```python
@asynccontextmanager
async def lifespan(app: FastAPI):
"""Lifespan event handler for startup and shutdown."""
# Startup
logger.info("Starting karaoke generation backend")
logger.info(f"Environment: {settings.environment}")
logger.info(f"GCS Bucket: {settings.gcs_bucket_name}")
logger.info(f"Tracing enabled: {tracing_enabled}")
# Preload SpaCy model to avoid 60+ second delay on first request
try:
preload_spacy_model("en_core_web_sm")
except Exception as e:
logger.warning(f"SpaCy preload failed (will load lazily): {e}")
# Validate OAuth credentials (non-blocking)
try:
await validate_credentials_on_startup()
except Exception as e:
logger.error(f"Credential validation failed: {e}")
yield
# Shutdown
logger.info("Shutting down karaoke generation backend")
```
### 3. MODIFY: `lyrics_transcriber_temp/lyrics_transcriber/correction/phrase_analyzer.py`
The `PhraseAnalyzer` class loads SpaCy in its `__init__` method. Modify to use preloaded model if available.
**Add imports at top:**
```python
# Try to import preloader (may not exist in standalone library usage)
try:
from backend.services.spacy_preloader import get_preloaded_model
_HAS_PRELOADER = True
except ImportError:
_HAS_PRELOADER = False
```
**Modify `__init__` method:**
```python
def __init__(self, logger: logging.Logger, language_code: str = "en_core_web_sm"):
self.logger = logger
# Try to use preloaded model first (avoids 60+ second load on Cloud Run)
if _HAS_PRELOADER:
preloaded = get_preloaded_model(language_code)
if preloaded is not None:
self.logger.info(f"Using preloaded SpaCy model: {language_code}")
self.nlp = preloaded
return
# Fall back to loading model directly
self.logger.info(f"Initializing PhraseAnalyzer with language model: {language_code}")
try:
self.nlp = spacy.load(language_code)
except OSError:
# ... keep existing download fallback code ...
```
### 4. MODIFY: `lyrics_transcriber_temp/lyrics_transcriber/correction/handlers/syllables_match.py`
This file also loads SpaCy. Apply the same pattern.
**Add imports at top:**
```python
try:
from backend.services.spacy_preloader import get_preloaded_model
_HAS_PRELOADER = True
except ImportError:
_HAS_PRELOADER = False
```
**Modify the SpaCy loading section in `__init__`:**
```python
# Try to use preloaded model first
if _HAS_PRELOADER:
preloaded = get_preloaded_model("en_core_web_sm")
if preloaded is not None:
self.logger.info("Using preloaded SpaCy model for syllable analysis")
self.nlp = preloaded
# Still need to add syllables component if not present
if "syllables" not in self.nlp.pipe_names:
self.nlp.add_pipe("syllables", after="tagger")
return
# ... existing loading code as fallback ...
```
## Testing Approach
### Unit Tests for Preloader
See `backend/tests/test_spacy_preloader.py` for the actual implementation. Key tests:
- `test_preload_spacy_model_loads_and_stores` - Verifies model is loaded and stored
- `test_preload_is_idempotent` - Verifies repeated calls don't reload
- `test_get_preloaded_model_returns_none_if_not_loaded` - Verifies None returned when not preloaded
- `test_preload_different_models` - Verifies multiple models can be preloaded
- `test_clear_preloaded_models` - Verifies clearing works
- `test_preload_failure_raises_exception` - Verifies errors propagate
- `test_preload_uses_default_model_name` - Verifies default is en_core_web_sm
All tests use `clear_preloaded_models()` in setup/teardown for isolation.
### Integration Verification
After deployment, check Cloud Run logs:
1. **At startup**: Look for "SpaCy model 'en_core_web_sm' preloaded in X.XXs"
2. **During job processing**: Look for "Using preloaded SpaCy model: en_core_web_sm"
3. **Timing**: Verify no 60+ second gaps in agentic correction phase
## Expected Outcomes
1. **Startup time**: SpaCy load time (~60s) moves from request processing to container startup
2. **Request latency**: First agentic correction request no longer has 60+ second SpaCy delay
3. **Visibility**: Clear logs showing preload time at startup
4. **Total job time**: Agentic correction phase should complete in ~5 minutes instead of ~17 minutes
## Implementation Steps
1. Create new worktree:
```bash
cd /Users/andrew/Projects/karaoke-gen-multiagent/karaoke-gen
git worktree add -b feat/spacy-preload ../karaoke-gen-spacy-preload origin/main
cd ../karaoke-gen-spacy-preload
```
2. Create `backend/services/spacy_preloader.py`
3. Modify `backend/main.py` to add preloading in lifespan
4. Modify `lyrics_transcriber_temp/lyrics_transcriber/correction/phrase_analyzer.py`
5. Modify `lyrics_transcriber_temp/lyrics_transcriber/correction/handlers/syllables_match.py`
6. Create unit tests for the preloader
7. Run tests:
```bash
make test 2>&1 | tail -n 500
```
8. Bump version in `pyproject.toml`
9. Follow standard PR workflow: `/docs-review` -> `/coderabbit` -> `/pr`
## Risk Considerations
- **Startup time increase**: Container startup will take ~60s longer, but this happens once per container lifecycle, not per request
- **Memory usage**: SpaCy model (~50MB in memory) will persist for container lifetime - acceptable tradeoff
- **Import error handling**: Graceful fallback if preloader module unavailable (library standalone use case)
## Related Files Reference
- `backend/Dockerfile.base:64` - SpaCy model download at build time
- `infrastructure/cloudrun.ts` - Cloud Run configuration (min-instances: 4)
- `docs/LESSONS-LEARNED.md` - Contains thread-safety lesson from related fix
@@ -0,0 +1,271 @@
# White-Label B2B Portal Implementation
**Date**: 2026-01-08
**Status**: Complete
**First Tenant**: Vocal Star
## Overview
Implemented multi-tenant white-label portal infrastructure allowing B2B customers to have their own branded karaoke generation experience at custom subdomains.
## First Customer: Vocal Star
**Requirements**:
- Subdomain: `vocalstar.nomadkaraoke.com`
- File upload only (no audio search, no YouTube URL input)
- Download only distribution (no YouTube/Dropbox/GDrive upload)
- Theme locked to "vocalstar" (yellow/blue color scheme)
- Auth restricted to `@vocal-star.com` and `@vocalstarmusic.com` email domains
- Custom branding: logo, colors, site title, tagline
## Architecture
### Tenant Detection Flow
```
Request → Frontend detectTenantFromUrl() → Backend TenantMiddleware → TenantService → GCS Config
```
**Frontend** (`frontend/lib/tenant.ts`):
- Zustand store for tenant state
- `detectTenantFromUrl()` extracts subdomain from hostname
- Strict patterns: `{tenant}.nomadkaraoke.com` or `{tenant}.gen.nomadkaraoke.com`
- Query param `?tenant=X` for local development only
**Backend** (`backend/middleware/tenant.py`):
- `TenantMiddleware` extracts tenant from:
1. `X-Tenant-ID` header (from frontend)
2. Query param (dev only - disabled in production via `IS_PRODUCTION` check)
3. Host header subdomain
- Attaches `tenant_id` and `tenant_config` to `request.state`
**Config Storage** (`backend/services/tenant_service.py`):
- Loads from GCS: `tenants/{tenant_id}/config.json`
- In-memory cache with 5-minute TTL
- Signed URL generation for logos
### GCS Structure
```
tenants/{tenant_id}/
├── config.json # TenantConfig
└── logo.jpg # Tenant logo
themes/{theme_id}/
├── style_params.json # Theme configuration
└── assets/
├── intro_background.png
├── karaoke_background.jpg
├── end_background.png
├── Oswald-SemiBold.ttf
├── cdg_instrumental_background.gif
└── cdg_title_background.gif
```
### Config Schema
```python
TenantConfig:
id: str
name: str
subdomain: str
is_active: bool
branding: TenantBranding
logo_url: str | None
logo_height: int
primary_color: str
secondary_color: str
accent_color: str | None
background_color: str | None
favicon_url: str | None
site_title: str
tagline: str | None
features: TenantFeatures
audio_search: bool
file_upload: bool
youtube_url: bool
youtube_upload: bool
dropbox_upload: bool
gdrive_upload: bool
theme_selection: bool
color_overrides: bool
enable_cdg: bool
enable_4k: bool
admin_access: bool
defaults: TenantDefaults
theme_id: str | None
locked_theme: str | None
distribution_mode: str
auth: TenantAuth
allowed_email_domains: list[str]
require_email_domain: bool
sender_email: str | None
```
## Implementation Details
### Backend Changes
1. **Tenant Models** (`backend/models/tenant.py`):
- Pydantic models for config schema
- `locked_theme` field for forcing specific theme
2. **Tenant Service** (`backend/services/tenant_service.py`):
- GCS config loading with caching
- Logo signed URL generation
- `tenant_exists()` check for subdomain validation
3. **Tenant API** (`backend/api/routes/tenant.py`):
- `GET /api/tenant/config` - auto-detect or explicit tenant
- `GET /api/tenant/config/{tenant_id}` - specific tenant lookup
4. **Middleware** (`backend/middleware/tenant.py`):
- Priority-based tenant detection
- Production security: query param disabled
- `get_tenant_from_request()` helper
5. **Auth Updates** (`backend/services/auth_service.py`):
- Email domain validation for tenant-restricted auth
- Sender email override from tenant config
6. **Feature Enforcement**:
- Audio search endpoint checks `features.audio_search`
- Returns 403 if feature disabled for tenant
7. **Job/User Scoping**:
- `tenant_id` field added to `JobCreate`
- Jobs created with tenant context from request
### Frontend Changes
1. **Tenant Store** (`frontend/lib/tenant.ts`):
- Zustand store with tenant state
- `useTenant()` hook for components
- Default Nomad Karaoke branding fallback
- CSS variable application via `applyTenantBranding()`
2. **TenantProvider** (`frontend/components/tenant-provider.tsx`):
- Wraps app in layout
- Fetches config on mount
- Prevents hydration issues (no module-level init)
3. **TenantLogo** (`frontend/components/tenant-logo.tsx`):
- Dynamic logo based on tenant
- Falls back to Nomad Karaoke logo
- Supports GCS signed URLs for tenant logos
4. **Feature-Flagged UI** (`frontend/components/job/JobSubmission.tsx`):
- Tabs conditionally rendered based on `features`
- Active tab sync when features change
- `useMemo` for available tabs list
5. **Locked Themes**:
- `defaults.locked_theme` hides theme selector
- Uses locked theme for all jobs
6. **Output Links**:
- Filtered based on `features.youtube_upload`, `features.dropbox_upload`, etc.
### Security Measures
1. **Query param disabled in production**:
```python
IS_PRODUCTION = os.environ.get("ENV", "").lower() == "production"
if not IS_PRODUCTION:
query_tenant = request.query_params.get("tenant")
```
2. **Strict subdomain patterns**:
```typescript
const isValidPattern =
parts.length === 3 ||
(parts.length === 4 && parts[1] === "gen")
```
3. **PII protection in logs**:
```python
def _mask_email(email: str) -> str:
# "andrew@vocal-star.com" → "an***@vo***.com"
```
4. **is_active check**: Inactive tenants treated as default
## Setup Script
`scripts/setup-vocalstar-tenant.py`:
- Uploads theme assets to GCS
- Creates theme config (`style_params.json`)
- Uploads tenant logo
- Creates tenant config (`config.json`)
- Updates theme registry
**Usage**:
```bash
# Set resources path (or pass as argument)
export VOCALSTAR_RESOURCES=/path/to/VocalStar/Resources
python scripts/setup-vocalstar-tenant.py
# Or with argument
python scripts/setup-vocalstar-tenant.py /path/to/VocalStar/Resources
```
## Files Changed
### Backend
- `backend/models/tenant.py` - New
- `backend/services/tenant_service.py` - New
- `backend/api/routes/tenant.py` - New
- `backend/middleware/tenant.py` - New
- `backend/main.py` - Added middleware and routes
- `backend/models/job.py` - Added tenant_id field
- `backend/services/auth_service.py` - Email domain validation
- `backend/api/routes/users.py` - PII masking in logs
- `backend/api/routes/audio_search.py` - Feature check
### Frontend
- `frontend/lib/tenant.ts` - New
- `frontend/components/tenant-provider.tsx` - New
- `frontend/components/tenant-logo.tsx` - New
- `frontend/components/app-layout.tsx` - TenantProvider wrapper
- `frontend/components/job/JobSubmission.tsx` - Feature flags
- `frontend/components/job/OutputLinks.tsx` - Distribution filtering
### Scripts
- `scripts/setup-vocalstar-tenant.py` - New
## Testing
- Frontend build verified (TypeScript compilation)
- CodeRabbit review: 3 cycles, all issues fixed
- Manual testing with `?tenant=vocalstar` query param
### Unit Tests Added
**Backend tests (110 tests)**:
- `test_tenant_models.py` - 28 tests for Pydantic models (TenantConfig, branding, features, defaults, auth, public config, `is_email_allowed()`, `get_sender_email()`, `from_config()`)
- `test_tenant_service.py` - 30 tests for TenantService (config loading, caching, subdomain resolution)
- `test_tenant_middleware.py` - 35 tests for TenantMiddleware (header/query param/subdomain detection, NON_TENANT_SUBDOMAINS filtering)
- `test_tenant_api.py` - 17 tests for `/api/tenant/config` endpoints
**Frontend tests (22 tests)**:
- `tenant.test.ts` - Tests for Zustand store, `getTenantHeaders()`, `isFeatureEnabled()`
**Key testing patterns**:
- API route tests must patch `get_tenant_service` in BOTH routes AND middleware modules since middleware runs first
- Zustand tests use `useTenant.getState().setTenant()` instead of direct `setState()` to properly handle computed properties
- `test_job_creation_regression.py` fixed to mock `request.state.tenant_config = None`
## Next Steps
1. Configure DNS: CNAME for `vocalstar.nomadkaraoke.com`
2. Update Cloudflare Pages for subdomain
3. Run setup script to upload Vocal Star assets
4. Test at `https://vocalstar.nomadkaraoke.com`
## Lessons Learned
1. **Config-driven multitenancy** scales better than code branches
2. **Query params for dev, headers for prod** prevents spoofing
3. **Feature flags belong in config**, not environment variables
4. **Zustand + Next.js**: Don't auto-init at module level, use useEffect
5. **Tab state must react to feature changes** - sync active tab with available tabs
+3 -3
View File
@@ -26,7 +26,7 @@ describe('getJobStep', () => {
expect(result.total).toBe(10);
expect(result.label).toBe('Setting up');
expect(result.isBlocking).toBe(false);
expect(result.color).toBe('text-slate-400');
expect(result.color).toBe('text-muted-foreground');
});
});
@@ -328,7 +328,7 @@ describe('getJobStep', () => {
const result = getJobStep(job);
expect(result.step).toBe(0);
expect(result.label).toBe('Cancelled');
expect(result.color).toBe('text-slate-500');
expect(result.color).toBe('text-muted-foreground');
});
});
@@ -339,7 +339,7 @@ describe('getJobStep', () => {
expect(result.step).toBe(0);
expect(result.label).toBe('some unknown status');
expect(result.isBlocking).toBe(false);
expect(result.color).toBe('text-slate-400');
expect(result.color).toBe('text-muted-foreground');
});
it('handles null status gracefully', () => {
+4 -2
View File
@@ -2,6 +2,7 @@
import { useEffect, useState } from "react"
import { adminApi, AdminBetaStats, AdminBetaFeedback } from "@/lib/api"
import { useAdminSettings } from "@/lib/admin-settings"
import { StatsCard, StatsGrid } from "@/components/admin/stats-card"
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
import { Badge } from "@/components/ui/badge"
@@ -27,6 +28,7 @@ import { useToast } from "@/hooks/use-toast"
export default function AdminBetaPage() {
const { toast } = useToast()
const { showTestData } = useAdminSettings()
const [stats, setStats] = useState<AdminBetaStats | null>(null)
const [feedback, setFeedback] = useState<AdminBetaFeedback[]>([])
const [loading, setLoading] = useState(true)
@@ -35,7 +37,7 @@ export default function AdminBetaPage() {
try {
setLoading(true)
const [statsData, feedbackData] = await Promise.all([
adminApi.getBetaStats(),
adminApi.getBetaStats({ exclude_test: !showTestData }),
adminApi.getBetaFeedback(50),
])
setStats(statsData)
@@ -54,7 +56,7 @@ export default function AdminBetaPage() {
useEffect(() => {
loadData()
}, [])
}, [showTestData])
const renderStars = (rating?: number) => {
if (!rating) return "—"
+1 -1
View File
@@ -125,7 +125,7 @@ export default function AdminJobDetailPage() {
case "INFO":
return "text-blue-500"
case "DEBUG":
return "text-gray-500"
return "text-muted-foreground"
default:
return "text-foreground"
}
+4 -1
View File
@@ -3,6 +3,7 @@
import { useEffect, useState, useCallback } from "react"
import { useRouter } from "next/navigation"
import { adminApi, Job } from "@/lib/api"
import { useAdminSettings } from "@/lib/admin-settings"
import { Button } from "@/components/ui/button"
import { Input } from "@/components/ui/input"
import { Badge } from "@/components/ui/badge"
@@ -62,6 +63,7 @@ const statusOptions = [
export default function AdminJobsPage() {
const router = useRouter()
const { toast } = useToast()
const { showTestData } = useAdminSettings()
const [jobs, setJobs] = useState<Job[]>([])
const [loading, setLoading] = useState(true)
const [statusFilter, setStatusFilter] = useState("all")
@@ -80,6 +82,7 @@ export default function AdminJobsPage() {
status: statusFilter !== "all" ? statusFilter : undefined,
user_email: userEmailFilter || undefined,
limit: 100,
exclude_test: !showTestData,
})
setJobs(data)
} catch (err: any) {
@@ -92,7 +95,7 @@ export default function AdminJobsPage() {
} finally {
setLoading(false)
}
}, [statusFilter, userEmailFilter, toast])
}, [statusFilter, userEmailFilter, showTestData, toast])
useEffect(() => {
loadJobs()
+16
View File
@@ -4,10 +4,13 @@ import React, { useEffect, useState } from "react"
import { useRouter } from "next/navigation"
import { getAccessToken } from "@/lib/api"
import { useAuth } from "@/lib/auth"
import { useAdminSettings } from "@/lib/admin-settings"
import { Loader2 } from "lucide-react"
import { SidebarProvider, SidebarInset, SidebarTrigger } from "@/components/ui/sidebar"
import { AdminSidebar } from "@/components/admin/admin-sidebar"
import { Separator } from "@/components/ui/separator"
import { Switch } from "@/components/ui/switch"
import { Label } from "@/components/ui/label"
import {
Breadcrumb,
BreadcrumbItem,
@@ -87,6 +90,9 @@ export default function AdminLayout({
const router = useRouter()
const { user } = useAuth()
const [isChecking, setIsChecking] = useState(true)
// IMPORTANT: All hooks must be called before any conditional returns
// to satisfy React's Rules of Hooks
const { showTestData, setShowTestData } = useAdminSettings()
useEffect(() => {
const token = getAccessToken()
@@ -132,6 +138,16 @@ export default function AdminLayout({
<SidebarTrigger className="-ml-1" />
<Separator orientation="vertical" className="mr-2 h-4" />
<AdminBreadcrumb />
<div className="ml-auto flex items-center gap-2">
<Switch
id="show-test-data"
checked={showTestData}
onCheckedChange={setShowTestData}
/>
<Label htmlFor="show-test-data" className="text-sm text-muted-foreground cursor-pointer">
Show test data
</Label>
</div>
</header>
<main className="flex-1 p-4 md:p-6">{children}</main>
</SidebarInset>
+5 -3
View File
@@ -2,6 +2,7 @@
import { useEffect, useState } from "react"
import { adminApi, AdminStatsOverview } from "@/lib/api"
import { useAdminSettings } from "@/lib/admin-settings"
import { StatsCard, StatsGrid } from "@/components/admin/stats-card"
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
import { Badge } from "@/components/ui/badge"
@@ -23,12 +24,13 @@ export default function AdminDashboardPage() {
const [stats, setStats] = useState<AdminStatsOverview | null>(null)
const [loading, setLoading] = useState(true)
const [error, setError] = useState<string | null>(null)
const { showTestData } = useAdminSettings()
const loadStats = async () => {
try {
setLoading(true)
setError(null)
const data = await adminApi.getStats()
const data = await adminApi.getStats({ exclude_test: !showTestData })
setStats(data)
} catch (err: unknown) {
const message = err instanceof Error ? err.message : "Failed to load statistics"
@@ -41,7 +43,7 @@ export default function AdminDashboardPage() {
useEffect(() => {
loadStats()
}, [])
}, [showTestData])
if (error) {
return (
@@ -164,7 +166,7 @@ export default function AdminDashboardPage() {
</div>
<div className="flex items-center justify-between">
<div className="flex items-center gap-2">
<XCircle className="w-4 h-4 text-gray-500" />
<XCircle className="w-4 h-4 text-muted-foreground" />
<span className="text-sm">Cancelled</span>
</div>
<Badge variant="outline">{stats?.jobs_by_status?.cancelled ?? 0}</Badge>
+5 -2
View File
@@ -2,6 +2,7 @@
import { useEffect, useState, useCallback } from "react"
import { adminApi, AudioSearchJobSummary, CacheStatsResponse } from "@/lib/api"
import { useAdminSettings } from "@/lib/admin-settings"
import { Button } from "@/components/ui/button"
import { Badge } from "@/components/ui/badge"
import {
@@ -58,6 +59,7 @@ const statusOptions = [
export default function AdminSearchesPage() {
const { toast } = useToast()
const { showTestData } = useAdminSettings()
const [searches, setSearches] = useState<AudioSearchJobSummary[]>([])
const [loading, setLoading] = useState(true)
const [statusFilter, setStatusFilter] = useState("all")
@@ -81,6 +83,7 @@ export default function AdminSearchesPage() {
const data = await adminApi.listAudioSearches({
limit: 100,
status_filter: statusFilter !== "all" ? statusFilter : undefined,
exclude_test: !showTestData,
})
setSearches(data.jobs)
} catch (err: any) {
@@ -93,7 +96,7 @@ export default function AdminSearchesPage() {
} finally {
setLoading(false)
}
}, [statusFilter, toast])
}, [statusFilter, showTestData, toast])
const loadCacheStats = useCallback(async () => {
try {
@@ -190,7 +193,7 @@ export default function AdminSearchesPage() {
case "ops":
return "bg-blue-500/20 text-blue-400 border-blue-500/30"
default:
return "bg-slate-500/20 text-slate-400 border-slate-500/30"
return "bg-muted text-muted-foreground border-border"
}
}
-42
View File
@@ -29,8 +29,6 @@ import {
RefreshCw,
Loader2,
Plus,
Shield,
ShieldOff,
UserX,
UserCheck,
CreditCard,
@@ -114,29 +112,6 @@ export default function AdminUserDetailPage() {
}
}
const handleToggleRole = async () => {
if (!user) return
const newRole = user.role === "admin" ? "user" : "admin"
try {
setActionLoading(true)
await adminApi.setUserRole(email, newRole)
toast({
title: "Role Updated",
description: `User is now ${newRole}`,
})
loadUser()
} catch (err: any) {
toast({
title: "Error",
description: err.message || "Failed to update role",
variant: "destructive",
})
} finally {
setActionLoading(false)
}
}
const handleToggleStatus = async () => {
if (!user) return
@@ -237,23 +212,6 @@ export default function AdminUserDetailPage() {
<Plus className="w-4 h-4 mr-2" />
Add Credits
</Button>
<Button
variant="outline"
onClick={handleToggleRole}
disabled={actionLoading}
>
{user.role === "admin" ? (
<>
<ShieldOff className="w-4 h-4 mr-2" />
Remove Admin
</>
) : (
<>
<Shield className="w-4 h-4 mr-2" />
Make Admin
</>
)}
</Button>
<Button
variant={user.is_active ? "destructive" : "outline"}
onClick={handleToggleStatus}
+31 -18
View File
@@ -3,6 +3,7 @@
import { useEffect, useState, useCallback } from "react"
import { useRouter } from "next/navigation"
import { adminApi, AdminUser, AdminUserListResponse } from "@/lib/api"
import { useAdminSettings } from "@/lib/admin-settings"
import { Button } from "@/components/ui/button"
import { Input } from "@/components/ui/input"
import { Badge } from "@/components/ui/badge"
@@ -39,16 +40,18 @@ import {
Plus,
Loader2,
CreditCard,
Shield,
ShieldOff,
LogIn,
UserX,
UserCheck,
} from "lucide-react"
import { useToast } from "@/hooks/use-toast"
import { useAuth } from "@/lib/auth"
export default function AdminUsersPage() {
const router = useRouter()
const { toast } = useToast()
const { showTestData } = useAdminSettings()
const { startImpersonation, user: currentUser } = useAuth()
const [users, setUsers] = useState<AdminUser[]>([])
const [loading, setLoading] = useState(true)
const [total, setTotal] = useState(0)
@@ -76,6 +79,7 @@ export default function AdminUsersPage() {
search: search || undefined,
sort_by: sortBy,
sort_order: sortOrder,
exclude_test: !showTestData,
})
setUsers(data.users)
setTotal(data.total)
@@ -90,7 +94,7 @@ export default function AdminUsersPage() {
} finally {
setLoading(false)
}
}, [limit, offset, search, sortBy, sortOrder, toast])
}, [limit, offset, search, sortBy, sortOrder, showTestData, toast])
useEffect(() => {
loadUsers()
@@ -143,19 +147,31 @@ export default function AdminUsersPage() {
}
}
const handleToggleRole = async (user: AdminUser) => {
const newRole = user.role === "admin" ? "user" : "admin"
try {
await adminApi.setUserRole(user.email, newRole)
const handleImpersonate = async (user: AdminUser) => {
// Don't allow impersonating yourself
if (user.email === currentUser?.email) {
toast({
title: "Role Updated",
description: `${user.email} is now ${newRole}`,
title: "Cannot Impersonate",
description: "You cannot impersonate yourself",
variant: "destructive",
})
loadUsers()
return
}
try {
const success = await startImpersonation(user.email)
if (success) {
toast({
title: "Impersonating User",
description: `Now viewing as ${user.email}`,
})
// Redirect to main app to see the user's view
router.push("/app")
}
} catch (err: any) {
toast({
title: "Error",
description: err.message || "Failed to update role",
description: err.message || "Failed to impersonate user",
variant: "destructive",
})
}
@@ -280,14 +296,11 @@ export default function AdminUsersPage() {
<Button
variant="ghost"
size="icon"
onClick={() => handleToggleRole(user)}
title={user.role === "admin" ? "Remove admin" : "Make admin"}
onClick={() => handleImpersonate(user)}
title="Impersonate user"
disabled={user.email === currentUser?.email}
>
{user.role === "admin" ? (
<ShieldOff className="w-4 h-4" />
) : (
<Shield className="w-4 h-4" />
)}
<LogIn className="w-4 h-4" />
</Button>
</div>
</TableCell>
+28 -7
View File
@@ -3,6 +3,8 @@
import { useState, useEffect, useCallback } from "react"
import { useRouter } from "next/navigation"
import { api, Job, getAccessToken } from "@/lib/api"
import { useAuth } from "@/lib/auth"
import { useAdminSettings } from "@/lib/admin-settings"
import { useJobNotifications, useVisibilityRefresh } from "@/hooks/use-notifications"
import { Button } from "@/components/ui/button"
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
@@ -28,6 +30,11 @@ export default function AppPage() {
const [isInitialLoad, setIsInitialLoad] = useState(true)
const [isAuthenticated, setIsAuthenticated] = useState<boolean | null>(null)
const { isDarkMode, toggleTheme, mounted } = useTheme()
const { user, fetchUser } = useAuth()
const { showTestData } = useAdminSettings()
// Check if user is admin (for exclude_test parameter)
const isAdmin = user?.role === "admin" || user?.email?.endsWith("@nomadkaraoke.com")
// Memoize loadJobs for use with visibility refresh
const loadJobs = useCallback(async () => {
@@ -37,7 +44,11 @@ export default function AppPage() {
return
}
try {
const data = await api.listJobs({ limit: 20 })
// Only admins can filter test jobs, and by default we hide them
const data = await api.listJobs({
limit: 20,
exclude_test: isAdmin ? !showTestData : undefined
})
setJobs(data)
} catch (err: any) {
// Don't log auth errors - user just needs to authenticate
@@ -48,7 +59,7 @@ export default function AppPage() {
setIsLoadingJobs(false)
setIsInitialLoad(false)
}
}, [])
}, [isAdmin, showTestData])
// Enable notifications for job status changes (sound + title animation)
useJobNotifications(jobs)
@@ -67,6 +78,16 @@ export default function AppPage() {
setIsAuthenticated(true)
}, [router])
// Ensure user data is loaded after authentication
// This handles the case where user navigates via client-side navigation
// (e.g., after beta enrollment) and the module-level fetchUser() didn't run
useEffect(() => {
const token = getAccessToken()
if (token && !user) {
fetchUser()
}
}, [user, fetchUser])
// Load jobs on mount (only if authenticated)
useEffect(() => {
if (isAuthenticated !== true) {
@@ -82,7 +103,7 @@ export default function AppPage() {
// Show nothing while checking auth (prevents flash)
if (isAuthenticated === null) {
return (
<div className="min-h-screen flex items-center justify-center" style={{ background: 'var(--bg)' }}>
<div className="min-h-screen flex items-center justify-center animated-gradient">
<Loader2 className="w-8 h-8 animate-spin" style={{ color: 'var(--text-muted)' }} />
</div>
)
@@ -91,19 +112,19 @@ export default function AppPage() {
// Show warming up loader during initial job load (cold start scenario)
if (isInitialLoad && isLoadingJobs) {
return (
<div className="min-h-screen flex items-center justify-center" style={{ background: 'var(--bg)' }}>
<div className="min-h-screen flex items-center justify-center animated-gradient">
<WarmingUpLoader spinnerClassName="w-10 h-10" />
</div>
)
}
return (
<div className="min-h-screen" style={{ background: 'var(--bg)' }}>
<div className="min-h-screen animated-gradient">
{/* AutoProcessor - handles non-interactive mode for jobs with that flag */}
<AutoProcessor jobs={jobs} onJobsChanged={loadJobs} />
{/* Header */}
<header className="border-b backdrop-blur-sm sticky top-0 z-10" style={{ borderColor: 'var(--card-border)', backgroundColor: 'var(--card)' }}>
<header className="fixed top-0 left-0 right-0 z-50 bg-dark-900/80 backdrop-blur-md border-b border-dark-700">
<div className="px-3 sm:px-4 py-2 sm:py-3 flex items-center justify-between gap-2">
<div className="flex items-center gap-2 sm:gap-3 min-w-0">
<img src="/nomad-karaoke-logo.svg" alt="Nomad Karaoke" className="h-8 sm:h-10 shrink-0" />
@@ -152,7 +173,7 @@ export default function AppPage() {
</div>
</header>
<main className="px-4 py-8 space-y-6">
<main className="px-4 pt-24 pb-8 space-y-6">
<div className="grid lg:grid-cols-2 gap-8">
{/* Submit Job Card */}
<Card className="backdrop-blur" style={{ borderColor: 'var(--card-border)', backgroundColor: 'var(--card)' }}>
+18 -18
View File
@@ -51,14 +51,14 @@ function VerifyMagicLinkContent() {
}, [searchParams, verifyMagicLink, router])
return (
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
<div className="max-w-md w-full bg-card border border-border rounded-xl p-8 text-center">
{state === "loading" && (
<>
<Loader2 className="w-12 h-12 text-blue-400 animate-spin mx-auto mb-4" />
<h1 className="text-xl font-semibold text-white mb-2">
<Loader2 className="w-12 h-12 text-primary animate-spin mx-auto mb-4" />
<h1 className="text-xl font-semibold text-foreground mb-2">
Verifying your sign-in link...
</h1>
<p className="text-slate-400">
<p className="text-muted-foreground">
Please wait while we sign you in.
</p>
</>
@@ -66,14 +66,14 @@ function VerifyMagicLinkContent() {
{state === "success" && (
<>
<CheckCircle className="w-12 h-12 text-green-400 mx-auto mb-4" />
<h1 className="text-xl font-semibold text-white mb-2">
<CheckCircle className="w-12 h-12 text-success mx-auto mb-4" />
<h1 className="text-xl font-semibold text-foreground mb-2">
Successfully signed in!
</h1>
<p className="text-slate-400 mb-4">
<p className="text-muted-foreground mb-4">
Welcome back{user?.email ? `, ${user.email}` : ""}!
</p>
<p className="text-sm text-slate-500">
<p className="text-sm text-muted-foreground">
Redirecting you to the app...
</p>
</>
@@ -81,21 +81,21 @@ function VerifyMagicLinkContent() {
{state === "error" && (
<>
<XCircle className="w-12 h-12 text-red-400 mx-auto mb-4" />
<h1 className="text-xl font-semibold text-white mb-2">
<XCircle className="w-12 h-12 text-destructive mx-auto mb-4" />
<h1 className="text-xl font-semibold text-foreground mb-2">
Sign-in failed
</h1>
<p className="text-slate-400 mb-6">
<p className="text-muted-foreground mb-6">
{errorMessage || "The link may have expired or already been used."}
</p>
<div className="space-y-2">
<Button
onClick={() => router.push("/app")}
className="w-full bg-blue-600 hover:bg-blue-500"
className="w-full bg-primary hover:bg-primary/90"
>
Go to App
</Button>
<p className="text-xs text-slate-500">
<p className="text-xs text-muted-foreground">
You can request a new sign-in link from the app.
</p>
</div>
@@ -107,12 +107,12 @@ function VerifyMagicLinkContent() {
function VerifyLoadingFallback() {
return (
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
<Loader2 className="w-12 h-12 text-blue-400 animate-spin mx-auto mb-4" />
<h1 className="text-xl font-semibold text-white mb-2">
<div className="max-w-md w-full bg-card border border-border rounded-xl p-8 text-center">
<Loader2 className="w-12 h-12 text-primary animate-spin mx-auto mb-4" />
<h1 className="text-xl font-semibold text-foreground mb-2">
Loading...
</h1>
<p className="text-slate-400">
<p className="text-muted-foreground">
Please wait.
</p>
</div>
@@ -121,7 +121,7 @@ function VerifyLoadingFallback() {
export default function VerifyMagicLinkPage() {
return (
<div className="min-h-screen bg-slate-950 flex items-center justify-center p-4">
<div className="min-h-screen bg-background flex items-center justify-center p-4">
<Suspense fallback={<VerifyLoadingFallback />}>
<VerifyMagicLinkContent />
</Suspense>
+109 -41
View File
@@ -4,22 +4,50 @@
@custom-variant dark (&:is(.dark *));
:root {
/* Dark theme (default) - Matching instrumental review UI color scheme */
/* ===========================================
Nomad Karaoke Brand Colors
See docs/BRAND-STYLE-GUIDE.md for details
=========================================== */
/* Brand Primary Colors */
/* Note: darker pink (#ff5bb8) used as default for better white text contrast */
--brand-pink: #ff5bb8;
--brand-pink-hover: #ff7acc;
--brand-pink-glow: rgba(255, 122, 204, 0.4);
--brand-gold: #ffdf6b;
--brand-purple: #8b5cf6;
--brand-blue: #3b82f6;
--brand-soft-pink: #ec4899;
/* ===========================================
Tenant Branding Colors (set dynamically by JS)
These are overridden by TenantProvider for white-label portals
=========================================== */
--tenant-primary: var(--brand-pink);
--tenant-secondary: var(--brand-purple);
--tenant-accent: var(--brand-gold);
--tenant-background: #0f0f0f;
/* Dark theme (default) */
--bg: #0f0f0f;
--card: #1a1a1a;
--card-border: #2a2a2a;
--waveform-bg: #0d1117;
--text: #e5e5e5;
--text-muted: #888;
--primary: #3b82f6;
--primary-hover: #2563eb;
--text-muted: #888888;
--text-subtle: #666666;
/* Primary uses tenant color (defaults to brand pink) */
--primary: var(--tenant-primary);
--primary-hover: color-mix(in srgb, var(--tenant-primary) 80%, white 20%);
--secondary: #252525;
--secondary-hover: #333;
--secondary-hover: #333333;
--success: #22c55e;
--warning: #f59e0b;
--danger: #ef4444;
--pink: #ec4899;
--blue: #60a5fa;
--pink: var(--brand-pink);
--blue: var(--brand-blue);
/* Tailwind compatibility mappings */
--background: var(--bg);
@@ -38,26 +66,38 @@
--border: var(--card-border);
--input: var(--secondary);
--ring: var(--primary);
--radius: 0.5rem;
--radius: 0.75rem;
}
:root.light {
/* Light theme */
.light {
/* Light theme - applied via next-themes class on html element */
--bg: #f8fafc;
--card: #ffffff;
--card-border: #e2e8f0;
--waveform-bg: #f1f5f9;
--text: #1e293b;
--text-muted: #64748b;
--primary: #3b82f6;
--primary-hover: #2563eb;
--text-subtle: #94a3b8;
--primary: var(--brand-pink);
--primary-hover: var(--brand-pink-hover);
--secondary: #f1f5f9;
--secondary-hover: #e2e8f0;
--success: #22c55e;
--warning: #f59e0b;
--danger: #ef4444;
--pink: #ec4899;
--blue: #60a5fa;
--pink: var(--brand-pink);
--blue: var(--brand-blue);
/* Tailwind compatibility - light mode adjustments */
--background: var(--bg);
--foreground: var(--text);
--card-foreground: var(--text);
--popover: var(--card);
--popover-foreground: var(--text);
--muted: var(--secondary);
--muted-foreground: var(--text-muted);
--border: var(--card-border);
--input: var(--secondary);
}
@theme inline {
@@ -95,21 +135,45 @@
body {
@apply bg-background text-foreground;
}
/* Radial gradient background overlay (like decide.nomadkaraoke.com) */
body::after {
content: '';
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background:
radial-gradient(ellipse at 20% 0%, rgba(139, 92, 246, 0.12) 0%, transparent 50%),
radial-gradient(ellipse at 80% 100%, rgba(255, 122, 204, 0.1) 0%, transparent 50%),
radial-gradient(ellipse at 50% 50%, rgba(255, 223, 107, 0.03) 0%, transparent 70%);
pointer-events: none;
z-index: -1;
}
/* Light mode uses subtle pink tints */
.light body::after {
background:
radial-gradient(ellipse at 20% 0%, rgba(139, 92, 246, 0.08) 0%, transparent 50%),
radial-gradient(ellipse at 80% 100%, rgba(255, 122, 204, 0.06) 0%, transparent 50%),
radial-gradient(ellipse at 50% 50%, rgba(255, 223, 107, 0.02) 0%, transparent 70%);
}
}
/* Landing page utilities */
@layer utilities {
/* Glow effect for primary buttons */
/* Glow effect for primary buttons - uses brand pink */
.btn-glow {
box-shadow: 0 0 20px rgba(59, 130, 246, 0.5);
box-shadow: 0 0 20px var(--brand-pink-glow);
}
.btn-glow:hover {
box-shadow: 0 0 30px rgba(59, 130, 246, 0.7);
box-shadow: 0 0 30px rgba(255, 122, 204, 0.6);
}
/* Gradient text */
/* Gradient text - flows to brand pink */
.gradient-text {
background: linear-gradient(135deg, #3b82f6 0%, #8b5cf6 50%, #ec4899 100%);
background: linear-gradient(135deg, #3b82f6 0%, #8b5cf6 50%, #ff7acc 100%);
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
background-clip: text;
@@ -124,31 +188,35 @@
box-shadow: 0 20px 40px rgba(0, 0, 0, 0.3);
}
/* Animated background gradient */
@keyframes gradient-shift {
0% { background-position: 0% 50%; }
50% { background-position: 100% 50%; }
100% { background-position: 0% 50%; }
}
/* Animated background gradient - subtle brand integration */
/* animated-gradient class - now just ensures min-height, gradient provided by body::after */
.animated-gradient {
background: linear-gradient(-45deg, #0f172a, #1e293b, #1e3a8a, #312e81);
background-size: 400% 400%;
animation: gradient-shift 15s ease infinite;
/* Background is now provided by body::after radial gradient */
background: transparent;
}
/* Dark theme color utilities for landing page */
.text-dark-300 { color: #d1d5db; }
.text-dark-400 { color: #9ca3af; }
.text-dark-500 { color: #6b7280; }
.bg-dark-700 { background-color: #374151; }
.bg-dark-800 { background-color: #1f2937; }
.bg-dark-900 { background-color: #111827; }
.border-dark-600 { border-color: #4b5563; }
.border-dark-700 { border-color: #374151; }
.text-primary-400 { color: #60a5fa; }
.text-primary-500 { color: #3b82f6; }
.bg-primary-500 { background-color: #3b82f6; }
.hover\:bg-primary-600:hover { background-color: #2563eb; }
/* Theme-aware color utilities - use CSS variables for theme switching */
.text-dark-300 { color: var(--text); }
.text-dark-400 { color: var(--text-muted); }
.text-dark-500 { color: var(--text-subtle); }
.bg-dark-700 { background-color: var(--secondary); }
.bg-dark-800 { background-color: var(--card); }
.bg-dark-900 { background-color: var(--bg); }
.border-dark-600 { border-color: var(--card-border); }
.border-dark-700 { border-color: var(--card-border); }
/* Primary color utilities - now brand pink */
.text-primary-400 { color: #ff8fd4; } /* Lighter pink */
.text-primary-500 { color: var(--brand-pink); }
.bg-primary-500 { background-color: var(--brand-pink); }
.hover\:bg-primary-600:hover { background-color: var(--brand-pink-hover); }
/* Brand accent utilities */
.text-brand-pink { color: var(--brand-pink); }
.text-brand-gold { color: var(--brand-gold); }
.bg-brand-pink { background-color: var(--brand-pink); }
.bg-brand-gold { background-color: var(--brand-gold); }
.border-brand-pink { border-color: var(--brand-pink); }
/* Fade in animation for cold start messages */
@keyframes fade-in {
+15 -2
View File
@@ -1,5 +1,8 @@
import type React from "react"
import type { Metadata } from "next"
import { ThemeProvider } from "@/components/theme-provider"
import { TenantProvider } from "@/components/tenant-provider"
import { ImpersonationBannerWrapper } from "@/components/impersonation-banner-wrapper"
import "./globals.css"
export const metadata: Metadata = {
@@ -21,9 +24,19 @@ export default function RootLayout({
children: React.ReactNode
}>) {
return (
<html lang="en" className="dark">
<html lang="en" suppressHydrationWarning>
<body className="font-sans antialiased" style={{ background: 'var(--bg)', color: 'var(--text)' }}>
{children}
<ThemeProvider
attribute="class"
defaultTheme="dark"
enableSystem
disableTransitionOnChange
>
<TenantProvider>
<ImpersonationBannerWrapper />
{children}
</TenantProvider>
</ThemeProvider>
</body>
</html>
)
+153 -69
View File
@@ -2,8 +2,8 @@
import { useState, useEffect, useRef } from 'react';
import { useRouter } from 'next/navigation';
import Image from 'next/image';
import {
MicVocal,
Music,
Sparkles,
Video,
@@ -18,28 +18,29 @@ import {
import { api, setAccessToken, getAccessToken, CreditPackage, BetaEnrollResponse } from '@/lib/api';
import { AuthDialog } from '@/components/auth/AuthDialog';
import { WarmingUpLoader } from '@/components/WarmingUpLoader';
import { ThemeToggle } from '@/components/ThemeToggle';
// FAQ data
const faqs = [
{
question: 'How does it work?',
answer:
'Enter the artist and song title, and our AI finds the audio, removes the vocals, transcribes and syncs the lyrics, then generates a professional karaoke video. You can review and edit the lyrics before final export.',
'Enter an artist and song title (or paste a YouTube link, or upload your own audio file). Our system finds the audio, separates the vocals from the instrumental, transcribes the lyrics, and syncs them to the music. You then review and correct any transcription errors before we generate your final karaoke video.',
},
{
question: 'What songs can I use?',
answer:
'Any song! Our system searches multiple audio sources to find high-quality versions. If a song exists online, we can likely create a karaoke video for it.',
'Any song! Search by artist and title, paste a YouTube URL, or upload your own audio file. If you can get the audio, we can make it into karaoke.',
},
{
question: 'What format is the output?',
answer:
'You get a 4K MP4 video file with perfectly synced lyrics. We also offer direct YouTube upload and cloud storage integration.',
'You get multiple files: a 4K MP4 karaoke video, a "With Vocals" version for sing-along practice, and a CDG+MP3 ZIP for older karaoke systems. All videos are also published to our YouTube channel for easy sharing.',
},
{
question: 'What if something goes wrong?',
question: 'Do I need to do anything, or is it fully automatic?',
answer:
"If the AI makes mistakes, you can edit the lyrics and timing before generating the final video. If there's a technical issue, contact us and we'll make it right.",
"The lyrics transcription is very accurate, but you'll need to review it and correct any misheard words before the final video is generated. For songs with clear vocals, there may be few or no corrections needed. For complex songs, expect to spend 5-10 minutes fixing errors. The word timing/sync is handled automatically and is very precise.",
},
{
question: 'Do credits expire?',
@@ -69,8 +70,6 @@ export default function LandingPage() {
const [betaSuccess, setBetaSuccess] = useState(false);
const [betaWillRedirect, setBetaWillRedirect] = useState(false);
// FAQ expansion state
const [expandedFaq, setExpandedFaq] = useState<number | null>(null);
// Auth dialog state
const [showAuthDialog, setShowAuthDialog] = useState(false);
@@ -212,16 +211,18 @@ export default function LandingPage() {
{/* Navigation */}
<nav className="fixed top-0 left-0 right-0 z-50 bg-dark-900/80 backdrop-blur-md border-b border-dark-700">
<div className="max-w-6xl mx-auto px-4 py-4 flex items-center justify-between">
<div className="flex items-center gap-2">
<MicVocal className="w-8 h-8 text-primary-500" />
<span className="text-xl font-bold">Nomad Karaoke</span>
<div className="flex items-center gap-3">
<img src="/nomad-karaoke-logo.svg" alt="Nomad Karaoke" className="h-10" />
</div>
<div className="flex items-center gap-3">
<ThemeToggle />
<button
onClick={() => setShowAuthDialog(true)}
className="px-4 py-2 text-sm font-semibold rounded-lg bg-[var(--brand-pink)] text-white hover:bg-[var(--brand-pink-hover)] hover:scale-105 hover:shadow-[0_0_20px_rgba(255,122,204,0.5)] active:scale-95 transition-all duration-200"
>
Sign In
</button>
</div>
<button
onClick={() => setShowAuthDialog(true)}
className="text-sm text-dark-300 hover:text-white transition-colors"
>
Already have credits? Sign in
</button>
</div>
</nav>
@@ -229,33 +230,51 @@ export default function LandingPage() {
<section className="pt-32 pb-16 px-4">
<div className="max-w-4xl mx-auto text-center">
<h1 className="text-4xl sm:text-5xl md:text-6xl font-extrabold mb-6 leading-tight">
Turn Any Song Into a{' '}
<span className="gradient-text">Karaoke Video</span>
Create a <span className="gradient-text">Karaoke Video</span> for Any Song
</h1>
<p className="text-xl text-dark-300 mb-8 max-w-2xl mx-auto">
Professional karaoke videos in minutes. AI-powered vocal removal,
perfect lyrics sync, and stunning video output.
Create professional karaoke videos in under 30 minutes. Real instrumentals
from the original song, precise lyrics sync, and 4K video output.
</p>
<a
href="#pricing"
href="#beta"
className="inline-flex items-center gap-2 bg-primary-500 hover:bg-primary-600 text-white font-semibold px-8 py-4 rounded-xl transition-all btn-glow"
>
Get Started
Try It For Free
<ChevronDown className="w-5 h-5" />
</a>
</div>
</section>
{/* Demo Video Section */}
<section className="py-8 px-4">
<div className="max-w-4xl mx-auto">
<div className="bg-dark-800 border border-dark-700 rounded-2xl p-2 overflow-hidden">
<div className="aspect-video">
<iframe
className="w-full h-full rounded-xl"
src="https://www.youtube.com/embed/uvKtLOUiqz8?si=K06hw2B-QJ81Loeo"
title="Nomad Karaoke Demo"
frameBorder="0"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
referrerPolicy="strict-origin-when-cross-origin"
allowFullScreen
/>
</div>
</div>
</div>
</section>
{/* How It Works */}
<section className="py-16 px-4">
<div className="max-w-6xl mx-auto">
<h2 className="text-3xl font-bold text-center mb-12">How It Works</h2>
<div className="grid md:grid-cols-4 gap-8">
{[
{ icon: Music, title: 'Search', desc: 'Enter artist & song title' },
{ icon: Sparkles, title: 'AI Magic', desc: 'We remove vocals & sync lyrics' },
{ icon: Video, title: 'Review', desc: 'Preview and fine-tune' },
{ icon: Youtube, title: 'Export', desc: 'Download or upload to YouTube' },
{ icon: Music, title: 'Choose a Song', desc: 'Search, paste a YouTube link, or upload audio' },
{ icon: Sparkles, title: 'We Do the Heavy Lifting', desc: 'Vocals removed, lyrics transcribed & synced' },
{ icon: Video, title: 'Review & Correct', desc: 'Fix any transcription errors (usually 5-10 min)' },
{ icon: Youtube, title: 'Get Your Video', desc: 'Download files or watch on our YouTube' },
].map((step, i) => (
<div key={i} className="text-center">
<div className="w-16 h-16 bg-primary-500/20 rounded-2xl flex items-center justify-center mx-auto mb-4">
@@ -275,12 +294,12 @@ export default function LandingPage() {
<h2 className="text-3xl font-bold text-center mb-12">What You Get</h2>
<div className="grid md:grid-cols-3 gap-6">
{[
{ title: 'Studio-Quality Separation', desc: 'State-of-the-art AI removes vocals cleanly, keeping the instrumental crisp.' },
{ title: 'Perfect Lyrics Sync', desc: 'Word-by-word timing that highlights exactly when to sing.' },
{ title: 'Multiple Instrumental Options', desc: 'Choose from different vocal removal levels to get the sound you want.' },
{ title: '4K Video Output', desc: 'Stunning video quality that looks great on any screen.' },
{ title: 'YouTube Integration', desc: 'Upload directly to your YouTube channel with one click.' },
{ title: 'Edit Before Export', desc: 'Fix any lyrics issues before generating your final video.' },
{ title: 'Real Instrumentals', desc: "Uses the actual instrumental from your song—not a cover band or MIDI recreation." },
{ title: 'Precise Lyrics Sync', desc: 'Word-by-word timing that highlights exactly when to sing. You review the lyrics for accuracy.' },
{ title: 'Keep or Remove Backing Vocals', desc: 'Choose a clean instrumental or one that preserves backing vocals for a fuller sound.' },
{ title: '4K Video + Multiple Formats', desc: 'Get a 4K karaoke video, a sing-along version with vocals, and CDG+MP3 for older systems.' },
{ title: 'Published to YouTube', desc: 'Every video is automatically published to our YouTube channel for easy sharing.' },
{ title: 'Full Control Before Export', desc: 'Review and correct any transcription errors before the final video is generated.' },
].map((feature, i) => (
<div key={i} className="bg-dark-800 border border-dark-700 rounded-xl p-6 card-hover">
<h3 className="font-semibold text-lg mb-2">{feature.title}</h3>
@@ -291,25 +310,104 @@ export default function LandingPage() {
</div>
</section>
{/* Screenshots Section */}
<section className="py-16 px-4">
<div className="max-w-6xl mx-auto">
<h2 className="text-3xl font-bold text-center mb-4">See It In Action</h2>
<p className="text-dark-400 text-center mb-12 max-w-xl mx-auto">
Here&apos;s what the process looks like at each step
</p>
<div className="grid md:grid-cols-2 gap-8">
{/* Job Dashboard Screenshot */}
<div className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
<div className="aspect-video bg-dark-900 relative">
<Image
src="/screenshots/job-dashboard.avif"
alt="Job Dashboard showing karaoke video projects with status indicators"
fill
className="object-cover"
sizes="(max-width: 768px) 100vw, 50vw"
/>
</div>
<div className="p-4">
<h3 className="font-semibold mb-1">Job Dashboard</h3>
<p className="text-dark-400 text-sm">Track progress of all your karaoke video projects</p>
</div>
</div>
{/* Lyrics Review Screenshot */}
<div className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
<div className="aspect-video bg-dark-900 relative">
<Image
src="/screenshots/lyrics-review.avif"
alt="Lyrics Review UI with word-by-word editing and timing controls"
fill
className="object-cover"
sizes="(max-width: 768px) 100vw, 50vw"
/>
</div>
<div className="p-4">
<h3 className="font-semibold mb-1">Lyrics Review</h3>
<p className="text-dark-400 text-sm">Correct any transcription errors before generating your video</p>
</div>
</div>
{/* Instrumental Selection Screenshot */}
<div className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
<div className="aspect-video bg-dark-900 relative">
<Image
src="/screenshots/instrumental-review.avif"
alt="Instrumental Selection UI with audio player and backing vocals options"
fill
className="object-cover"
sizes="(max-width: 768px) 100vw, 50vw"
/>
</div>
<div className="p-4">
<h3 className="font-semibold mb-1">Instrumental Selection</h3>
<p className="text-dark-400 text-sm">Pick the sound that works best for your karaoke</p>
</div>
</div>
{/* Example Output Screenshot */}
<div className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
<div className="aspect-video bg-dark-900 relative">
<Image
src="/screenshots/example-output.avif"
alt="Example karaoke video output with synchronized lyrics"
fill
className="object-cover"
sizes="(max-width: 768px) 100vw, 50vw"
/>
</div>
<div className="p-4">
<h3 className="font-semibold mb-1">Final Video</h3>
<p className="text-dark-400 text-sm">4K karaoke video ready for your next singing session</p>
</div>
</div>
</div>
</div>
</section>
{/* Beta Tester Program */}
<section className="py-12 px-4">
<section id="beta" className="py-12 px-4">
<div className="max-w-2xl mx-auto">
<div className="bg-gradient-to-r from-purple-900/40 to-pink-900/40 border border-purple-500/30 rounded-2xl p-8 text-center">
<div className="inline-flex items-center gap-2 bg-purple-500/20 text-purple-300 px-4 py-2 rounded-full text-sm font-medium mb-4">
<div className="bg-gradient-to-r from-pink-500/10 to-purple-500/10 border border-primary/30 rounded-2xl p-8 text-center">
<div className="inline-flex items-center gap-2 bg-primary-500/20 text-primary-400 px-4 py-2 rounded-full text-sm font-medium mb-4">
<Gift className="w-4 h-4" />
Beta Tester Program
</div>
<h3 className="text-2xl font-bold mb-3">Try It Free!</h3>
<p className="text-dark-300 mb-6">
Help us improve by testing the tool and sharing your feedback. Get a{' '}
<span className="text-green-400 font-semibold">free credit</span> to
create your first karaoke video!
Help us improve by testing the tool and sharing your feedback. Beta testers get{' '}
<span className="text-green-400 font-semibold">up to 5 free karaoke videos</span>{' '}
in exchange for providing feedback on each one.
</p>
{!showBetaForm && !betaSuccess && (
<button
onClick={() => setShowBetaForm(true)}
className="inline-flex items-center gap-2 bg-purple-500 hover:bg-purple-600 text-white font-semibold px-6 py-3 rounded-xl transition-all"
className="inline-flex items-center gap-2 bg-primary-500 hover:bg-primary-600 text-white font-semibold px-6 py-3 rounded-xl transition-all btn-glow"
>
Join Beta Program
<MessageSquare className="w-5 h-5" />
@@ -328,7 +426,7 @@ export default function LandingPage() {
value={betaEmail}
onChange={(e) => setBetaEmail(e.target.value)}
placeholder="you@example.com"
className="w-full px-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-purple-500 text-white placeholder-dark-500"
className="w-full px-4 py-3 bg-secondary border border-border rounded-lg focus:outline-none focus:ring-2 focus:ring-primary text-foreground placeholder:text-muted-foreground"
required
/>
</div>
@@ -343,7 +441,7 @@ export default function LandingPage() {
onChange={(e) => setBetaPromise(e.target.value)}
placeholder="Tell us about a song you'd love to sing..."
rows={3}
className="w-full px-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-purple-500 text-white placeholder-dark-500 resize-none"
className="w-full px-4 py-3 bg-secondary border border-border rounded-lg focus:outline-none focus:ring-2 focus:ring-primary text-foreground placeholder:text-muted-foreground resize-none"
/>
</div>
@@ -353,10 +451,10 @@ export default function LandingPage() {
id="beta-accept"
checked={betaAccept}
onChange={(e) => setBetaAccept(e.target.checked)}
className="mt-1 w-4 h-4 rounded border-dark-600 bg-dark-900 text-purple-500 focus:ring-purple-500"
className="mt-1 w-4 h-4 rounded border-border bg-secondary text-primary focus:ring-primary"
/>
<label htmlFor="beta-accept" className="text-sm text-dark-300">
I understand that beta testers help improve lyrics accuracy by reviewing and correcting AI-generated lyrics
<label htmlFor="beta-accept" className="text-sm text-muted-foreground">
I understand that I&apos;ll need to review and correct any lyrics transcription errors, and provide feedback after each video
</label>
</div>
@@ -367,7 +465,7 @@ export default function LandingPage() {
<button
type="submit"
disabled={betaLoading}
className="w-full bg-purple-500 hover:bg-purple-600 disabled:bg-purple-500/50 text-white font-semibold py-3 rounded-xl transition-all flex items-center justify-center gap-2"
className="w-full bg-primary-500 hover:bg-primary-600 disabled:bg-primary-500/50 text-white font-semibold py-3 rounded-xl transition-all btn-glow flex items-center justify-center gap-2"
>
{betaLoading ? 'Enrolling...' : 'Get My Free Credit'}
{!betaLoading && <Gift className="w-5 h-5" />}
@@ -461,14 +559,14 @@ export default function LandingPage() {
Email address
</label>
<div className="relative">
<Mail className="absolute left-3 top-1/2 -translate-y-1/2 w-5 h-5 text-dark-500" />
<Mail className="absolute left-3 top-1/2 -translate-y-1/2 w-5 h-5 text-muted-foreground" />
<input
type="email"
id="email"
value={email}
onChange={(e) => setEmail(e.target.value)}
placeholder="you@example.com"
className="w-full pl-10 pr-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-primary-500 focus:border-transparent text-white placeholder-dark-500"
className="w-full pl-10 pr-4 py-3 bg-secondary border border-border rounded-lg focus:outline-none focus:ring-2 focus:ring-primary focus:border-transparent text-foreground placeholder:text-muted-foreground"
required
/>
</div>
@@ -503,22 +601,9 @@ export default function LandingPage() {
<h2 className="text-3xl font-bold text-center mb-12">Questions?</h2>
<div className="space-y-4">
{faqs.map((faq, i) => (
<div key={i} className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
<button
onClick={() => setExpandedFaq(expandedFaq === i ? null : i)}
aria-expanded={expandedFaq === i}
className="w-full px-6 py-4 text-left flex items-center justify-between hover:bg-dark-700/50 transition-colors"
>
<span className="font-medium">{faq.question}</span>
<ChevronDown
className={`w-5 h-5 text-dark-400 transition-transform ${
expandedFaq === i ? 'rotate-180' : ''
}`}
/>
</button>
{expandedFaq === i && (
<div className="px-6 pb-4 text-dark-300 text-sm">{faq.answer}</div>
)}
<div key={i} className="bg-dark-800 border border-dark-700 rounded-xl p-6">
<h3 className="font-medium mb-2">{faq.question}</h3>
<p className="text-dark-400 text-sm">{faq.answer}</p>
</div>
))}
</div>
@@ -526,20 +611,19 @@ export default function LandingPage() {
</section>
{/* Footer */}
<footer className="py-12 px-4 border-t border-dark-700">
<footer className="py-12 px-4 border-t border-primary-500/30">
<div className="max-w-6xl mx-auto flex flex-col md:flex-row items-center justify-between gap-4">
<div className="flex items-center gap-2">
<MicVocal className="w-6 h-6 text-primary-500" />
<span className="font-semibold">Nomad Karaoke</span>
<img src="/nomad-karaoke-logo.svg" alt="Nomad Karaoke" className="h-8" />
</div>
<div className="text-sm text-dark-500">
© {new Date().getFullYear()} Nomad Karaoke. All rights reserved.
</div>
<div className="flex gap-6 text-sm text-dark-400">
<a href="mailto:support@nomadkaraoke.com" className="hover:text-white transition-colors">
<a href="mailto:support@nomadkaraoke.com" className="hover:text-primary-400 transition-colors">
Support
</a>
<a href="https://nomadkaraoke.com" className="hover:text-white transition-colors">
<a href="https://nomadkaraoke.com" className="hover:text-primary-400 transition-colors">
About
</a>
</div>
+14 -14
View File
@@ -25,28 +25,28 @@ function PaymentSuccessContent() {
const sessionId = searchParams.get("session_id")
return (
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
<CheckCircle className="w-16 h-16 text-green-400 mx-auto mb-4" />
<div className="max-w-md w-full bg-card border border-border rounded-xl p-8 text-center">
<CheckCircle className="w-16 h-16 text-success mx-auto mb-4" />
<h1 className="text-2xl font-bold text-white mb-2">
<h1 className="text-2xl font-bold text-foreground mb-2">
Payment Successful!
</h1>
<p className="text-slate-400 mb-6">
<p className="text-muted-foreground mb-6">
Thank you for your purchase. Your credits have been added to your account.
</p>
{isLoading ? (
<div className="flex items-center justify-center gap-2 text-slate-400 mb-6">
<div className="flex items-center justify-center gap-2 text-muted-foreground mb-6">
<Loader2 className="w-4 h-4 animate-spin" />
Loading your balance...
</div>
) : user ? (
<div className="bg-slate-800 rounded-lg p-4 mb-6">
<div className="flex items-center justify-center gap-2 text-amber-400">
<div className="bg-secondary rounded-lg p-4 mb-6">
<div className="flex items-center justify-center gap-2 text-warning">
<Coins className="w-5 h-5" />
<span className="text-2xl font-bold">{user.credits}</span>
<span className="text-slate-400">credits available</span>
<span className="text-muted-foreground">credits available</span>
</div>
</div>
) : null}
@@ -54,12 +54,12 @@ function PaymentSuccessContent() {
<div className="space-y-3">
<Button
onClick={() => router.push("/app")}
className="w-full bg-blue-600 hover:bg-blue-500"
className="w-full bg-primary hover:bg-primary/90"
>
Start Creating Karaoke
</Button>
<p className="text-xs text-slate-500">
<p className="text-xs text-muted-foreground">
A confirmation email has been sent to your email address.
</p>
</div>
@@ -69,12 +69,12 @@ function PaymentSuccessContent() {
export default function PaymentSuccessPage() {
return (
<div className="min-h-screen bg-slate-950 flex items-center justify-center p-4">
<div className="min-h-screen bg-background flex items-center justify-center p-4">
<Suspense
fallback={
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
<Loader2 className="w-16 h-16 text-blue-400 mx-auto mb-4 animate-spin" />
<p className="text-slate-400">Loading...</p>
<div className="max-w-md w-full bg-card border border-border rounded-xl p-8 text-center">
<Loader2 className="w-16 h-16 text-primary mx-auto mb-4 animate-spin" />
<p className="text-muted-foreground">Loading...</p>
</div>
}
>
Binary file not shown.

After

Width:  |  Height:  |  Size: 920 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 270 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 220 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 166 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 733 KiB

Loaded 100 of 231 files, more files were not shown because too many files have changed in this diff. Show more